same grug as ProCreations/grug-v1.1-qwen-3.8-27b, plus a draft head that guess ahead, tuned on grug own output. Qwen3.8 ship native MTP (multi-token prediction) head. it predict token t+2 so engine can guess ahead and verify many token in one pass. speculative decoding only go fast when draft head agree with real model. grug retune that head on grug distribution. disagree rate go from 1-in-10 to 1-in-21. old grug v1 mtp sit at 89.3%. honest part: native head was already 90.0% before any training. grug not rescue broken head — grug sharpen already-good head. and gold top-1 go up too (85.5 -> 88.2), so head get genuinely better at real token, not just better at copying verifier mistake. also honest: 95.37% is top-1 agreement with the verifier on grug spans — the quantity that decide whether a speculative draft get accepted. it is NOT a measured wall-clock speedup. real tok/s depend on your engine, batch size and draft length. — backbone frozen. only the 425M-param mtp. module move. — teacher = grug-v1.1-qwen-3.8-27b own greedy next-token choice (self-distillation). this target acceptance directly: acceptance IS agreement with verifier. — data = ProCreations/grug-27b-v2-corrective —…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: ProCreations
Теги: qwen3_5, image-text-to-text, grug, mtp, speculative-decoding, token-efficient, agentic, conversational
Лайков: 4 | Загрузок: 924
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
