35B MoE Qwen3.6-A3B trunk (3B active) + embedded NextN-MTP head, quantized for single-GPU inference. — Trunk: IQ4XS (imatrix-calibrated) — MTP head: Q80 (NextN, kvonlynextn=true) — File size: ~18.3 GB — Min VRAM: ~21 GB at 32K ctx with KV Q8/Q8 The MTP head is embedded inside the GGUF — no separate drafter file. Recent llama.cpp builds activate it via —spec-type draft-mtp. INVALID LANGUAGE PAIR SPECIFIED. EXAMPLE: LANGPAIR=EN|IT USING 2 LETTER ISO OR RFC3066 LIKE ZH-CN. ALMOST ALL LANGUAGES SUPPORTED BUT SOME MAY HAVE NO CONTENT Single-stream decode on a 24 GB consumer GPU (RTX 3090 Ti), CUDA build, full-GPU offload, FA on, N=512 generated tokens, 5-warm-run mean. n_max=2 wins; deeper chains regress (MoE accept rate falls faster with depth). — Trunk weights: IQ4XS calibrated with an imatrix derived from a mixed-domain calibration set. — MTP head: kept at Q80 (NextN). Sensitive to precision. — KV cache: Q80/Q80 recommended. — Build supporting Qwen3.6 NextN MTP…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: localweights
Теги: gguf, qwen3_5_moe_text, qwen, qwen3.6, 35b, a3b, moe, mtp
Лайков: 4 | Загрузок: 764
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
