trevon/Qwen3.6-27B-mtp - Каталог нейросетей
Генерация текста

trevon/Qwen3.6-27B-mtp

Добавлено:
trevon/Qwen3.6-27B-mtp

MLX conversion of Qwen/Qwen3.6-27B, affine 8-bit (group_size 64), with the native Multi-Token-Prediction (MTP) head embedded in the main weight shards for native speculative decoding. This release replaces the previous mxfp8 packaging, which produced a ~29% slowdown when —mtp was enabled. Two compounding causes: 1. mxfp8 backbone: the speculative verify pass processes 2 tokens through the full backbone every step. mxfp8’s per-call dequant overhead does not amortize across that path on Apple Silicon — re-quantizing the same artifact to affine-8 reduced the regression from 29% to 10%. 2. Lossy round-trip in the original packaging: even after re-quantization, the MTP head’s calibration was damaged enough that MTP remained a net regression. A fresh affine-8 conversion straight from Qwen/Qwen3.6-27B (this artifact) restores +48% with —mtp. This release is therefore a fresh affine-8 conversion from the upstream Qwen base, not a re-packaging of the previous mxfp8 artifact.

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: trevon
Теги: mlx, qwen3_5, mtp, qwen, speculative-decoding, conversational, 8-bit
Лайков: 4  |  Загрузок: 111

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.