This quant uses a custom mixed-precision recipe designed specifically for the Qwen3.6-27B dense architecture to maximize quality within a ~5.2 BPW budget. Attention layers use non-linear mapping (IQ4NL), SSM state arrays are pinned to F32, and the FFN bottleneck (ffndown) is boosted to IQ5KS` along with the first and last 8 «sandwich» layers. Measured against wiki.test.raw with nctx=512` over 580 chunks: The custom ruleset used for the mixed-precision tensor overrides: Because generating an imatrix directly from a BF16 model requires massive system RAM, we use a Q8_0 intermediate step to generate the imatrix without running out of memory. 3. Generate the Imatrix Note: GGMLCUDANOPINNED=1 is used to prevent system RAM exhaustion on 24GB VRAM setups.* 4. Quantize BF16 to Hybrid IQ4KS We use the original BF16 model here with the Q80-generated imatrix for maximum fidelity. IQ4KS is passed as the base type.*
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: RDson
Теги: gguf, qwen, qwen3.6, iq4_ks, llama.cpp, ik_llama.cpp, endpoints_compatible, imatrix
Лайков: 3 | Загрузок: 39
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
