Quantized, Abliterated, and fitted by LexiPanel. Its Fit planner chose the tensor mix to fill one 24 GB GPU at about 240k tokens of context. It built the importance matrix from code-heavy text and kept the MTP head at Q80, so —spec-type draft-mtp` works without a separate draft model. The uploader runs it every day for agentic coding. On an RX 7900 XTX it decodes at 36–41 tokens/s with 110k–170k tokens already in context. These are 98 requests from agentic coding sessions on 2026-10-01, timed by the server itself: — MTP draft acceptance: the median is 85% (the middle half of requests falls between 75% and 94%). That works out to about 2.7 tokens per decode step at draft depth 2. — Prefill: — 387 t/s for a cold 108k-token prompt; — 175–183 t/s for about 4.5k new tokens added at 147k–156k depth. — VRAM: 24.2 of 24.6 GB in use at 262144 tokens of context, with a q4_1 KV cache and the vision projector on the CPU. Setup: AMD Radeon RX 7900 XTX (24 GB), llama.cpp b11235 on the Vulkan backend, all layers on the GPU, one slot. — Context: -c 262144 is what fits next to the weights on a 24 GB card with a q41 KV cache. The model’s native window is 262,144 tokens. On a smaller card, lower -c…
Модальности:
Генерация текста
Области применения:
Генерация кода Диалог / чат
Задача: Генерация текста
Автор: Wa1k3r
Теги: gguf, llama.cpp, imatrix, IQ4_XS, qwen3_8, mtp, speculative-decoding, long-context
Лайков: 3 | Загрузок: 358
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
