AtomicChat/Ling-3.1-flash-1019-GGUF - Каталог нейросетей
Генерация текста

AtomicChat/Ling-3.1-flash-1019-GGUF

Добавлено:
AtomicChat/Ling-3.1-flash-1019-GGUF

GGUF builds of inclusionAI’s Ling 3.1 flash 1019, with the multi-token prediction head inside. The calibration corpora behind our builds are public. Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline. Runs in llama.cpp build b11571 or newer. The multi-token prediction head is inside the file: add —spec-type draft-mtp to use it. In our check 57 to 69% of its drafted tokens were accepted. AD- marks an Atomic Dynamic layout: the routed experts (97% of the weights) take the low-bit type, while attention, the shared expert, the router, the dense layers and the multi-token prediction block stay at q8_0 or higher. The file name is the type of the experts’ gate and up projections; their down projections, and all experts of the last five blocks, get a step more. This file has not been measured against the BF16 model. The same layout on the previous checkpoint, with its KL divergence and top-1 agreement, and larger files are in AtomicChat/Ling-3.1-flash-GGUF. The file has to fit, plus a little for the context. Context is cheap on this model: 63 of its 72 layers use linear attention with a fixed-size state, and only 9 keep a compressed…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: AtomicChat
Теги: gguf, llama.cpp, ling, inclusionai, moe, imatrix, quantized, conversational
Лайков: 3  |  Загрузок: 337

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.