> Update 20/05/2026 — MTP draft head available: A compatible self-speculative decoding companion is now published: Qwen3.6-27B-Claude-Opus-Reasoning-MTP-Q4KM-gguf — load it via —model-draft for +40% decode throughput at no quality cost (llama.cpp ≥ b9245, —spec-type draft-mtp). RA is a domain-specific quantization methodology that allocates precision based on tensor importance during reasoning tasks, not generic text. 📄 Full methodology: blog.daxzeit.eu/reasoning-aware-quantization.html Standard quantization (bartowski, mradermacher) calibrates imatrix on generic text — wikitext or similar. This tells you which tensors are important for general text continuation. For a reasoning-distilled model, this averages out the signal: tensors critical for reasoning get the same precision as tensors that are barely active during reasoning. RA calibrates on the model’s own training data — 14K Opus reasoning traces, the same data used to distill the model. By comparing per-tensor importance between the reasoning imatrix and the wikitext imatrix, we identify which tensors are specifically activated by the reasoning domain. The delta is the signal. Precision follows activation, not convention.…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: DAXZEIT
Теги: gguf, quantized, reasoning, qwen3, imatrix, qwen3.6, en, endpoints_compatible
Лайков: 3 | Загрузок: 534
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
