DeepSeek-V4.1-Flash-1017 in mixed-precision GGUF for text inference. Q3 is the recommended variant; Q2 is a smaller alternative. Q2 reduces download size by 14.32% with the same main-text and Engram components. These eight-prompt checks per variant do not establish equivalent quality or lower runtime memory. Both variants matched the original tokenizer’s input IDs; code and tool-call outputs were inspected, not executed. Model type: chat. Recommended variant: DeepSeek-V4.1-Flash-1017-Artemis-Q3K — 385.583 GB total**, 7 required files. — For the recommended Q3 variant, download all seven shards into the directory used below: — Use llama.cpp release v0.9.0 or commit 149f5641; both gave the same results on our eight checks with these files. All shards of the selected variant must be in the same directory. The command below is the tested CPU route; GPU serving has not been qualified. — Start the server from your llama.cpp checkout: — For the Q2 alternative, download its six shards and use its tested CPU command: Send Q2 requests to http://127.0.0.1:18742; the Q3 command uses port 18741. — Use the official DeepSeek encoder, not a generic chat template. Encode messages with…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: turintech
Теги: gguf, endpoints_compatible
Лайков: 3 | Загрузок: 185
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
