4-bit weight-only quantization (W4A16) of DavidAU’s 40B «Chimera» — the 6-Core Fable Fusion Heretic Uncensored model. Quantized with Intel AutoRound recipe best, group_size 64, calibrated on high-quality reasoning traces. Base model: DavidAU/Qwen3.6-40B-Fable-Fusion-6-Core-Deckard-Eleanor-Heretic-Uncensored Unquantized layers (kept BF16): lmhead, embedtokens, model.visual (vision tower), all linearattn` projections (Gated DeltaNet), and MTP head via WOQ[RTN]. This model uses a 3:1 hybrid attention pattern — 72 linear-attention layers (Gated DeltaNet with Mamba SSM) and 24 full-attention layers. During quantization, —bs 1 is mandatory: the partial rotary embedding (partialrotaryfactor=0.25, headdim=256`) causes batch-dimension mismatches at higher batch sizes on full-attention layers. Quality verdict: Reasoning task solved correctly (both the sheep logic puzzle and the two-train meeting problem). Code generation produced full production-ready Python with bitarray optimization, type hints, docstrings, error handling, and CLI benchmarking. Creative output sustained 8K+ tokens coherently. Reference — base 27B Qwopus INT4 on same harness: 42–59 tok/s. The 40B quant is slower (more…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: goldhub
Теги: auto-round, qwen3_5, 40b, quantization, w4a16, gptq, mtp, heretic
Лайков: 4 | Загрузок: 1,855
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
