> Runtime note: This quant was built and tested with ikawrakow’s ikllama.cpp fork. It has not been tested on the mainline ggml/llama.cpp. For best results and full feature support (mixed-bit quant loading), use ikllama.cpp. Base model: OmniMerge-v4 on Qwen3.6-27B Size: 12.76 GiB (13.7 GB, 13,054 MiB on disk) — 4.06 bpw VRAM: Fits 16 GB GPUs comfortably with 32k+ context License: Apache-2.0 A custom importance-matrix-guided mixed-bit quantization of the OmniMerge-v4 merge. Achieves lower perplexity than the official uniform IQ4XS release while being 1.3 GiB smaller** — a clear win for targeted bit allocation at this size tier. — Both quants lose ~0.15–0.20 PPL from the F16 baseline. — The mixed-bit recipe recovers ~25% of that loss compared to uniform IQ4XS, while using 9% less disk space** Percentile breakdown of disagreement (the 8% where top-p differs): The distribution is strongly right-skewed: the two quants behave nearly identically on the vast majority of tokens, with meaningful divergence only in a thin tail. Initial results were corrupted by a bug in lmeval’s model=»gguf» type which calls .strip() on completions, removing the 4-space indentation from the first line of…
Модальности:
Генерация текста Компьютерное зрение Мультимодальность
Области применения:
Генерация кода Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: k0valik
Теги: gguf, ik-llama-cpp, imatrix, quantized, mixed-bit, qwen3_6, qwen3_5, omnimerge
Лайков: 3 | Загрузок: 252
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
