Two quantizations of the DFlash 2 draft model for Qwen/Qwen3.8-27B, both calibrated with an importance matrix captured from the drafter’s own activations during live speculative decoding. This is not a standalone language model. It runs inside a speculative-decoding server and drafts tokens for a target model to verify. On its own it produces nothing useful. The upstream repositories publish BF16, Q80 and Q4KM, none calibrated. Calibrating a DFlash 2 drafter is not straightforward: llama-imatrix cannot run one at all, because the architecture has no output layer and no token embedding and borrows both from its target, so it cannot form a context alone (dflash requires ctxother to be set). The collector has to be attached to the draft context inside a running speculative session instead. Two filters in llama.cpp’s collector then discard most of what matters. It skips any batch with fewer than 16 rows — and DFlash 2’s draft step evaluates only nmax + 1 tokens, so every draft-decode matmul is rejected — and it only collects tensors named blk., which excludes DFlash 2’s own top-level fc.weight and selectorhidden.weight. Attaching the stock collector captures 10 of 49 tensors. Relaxing…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: andrew-paul
Теги: llama.cpp, gguf, dflash2, speculative-decoding, draft-model, quantized, imatrix, conversational
Лайков: 4 | Загрузок: 1,643
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
