🤗 Original model | 🤗 Collection | 🖥️ Github | 📄 Technical Report This repository contains a quantized GGUF build of skt/A.X-K2. A.X K2 is a Mixture-of-Experts model with 688B parameters, 33B of them active per token, released in block-scaled FP8. The file here is converted from that checkpoint and quantized to the format in Model files. The original model card covers the architecture, training data, benchmarks, intended use and limitations. This card only adds what is specific to the GGUF build. Tensors left at full precision include output.weight, tokenembd.weight, the Gated Norm projections (normgatea/b.weight), the sparse-attention indexer projection (indexer.proj.weight), and the MoE router (ffngateinp.weight`). Official llama.cpp does not support A.X K2 yet, so build from the A.X-K2 fork. It is upstream b10236 plus A.X K2 support, and nothing else: For anything the examples do not cover, see the llama.cpp documentation — it all applies here, as long as you build the fork above. —temp and —top-p control the sampling: lower temperature is more deterministic, and top-p caps the cumulative probability of the token pool. -st runs a single turn: llama-cli answers the prompt and…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: skt
Теги: gguf, llama.cpp, vllm, quantized, skt, a.x, conversational, en
Лайков: 3 | Загрузок: 242
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
