Custom IQ2_M quantization of google/gemma-4-31b-it with an importance matrix calibrated on CLI / shell-assistant traces. — Size: 10.17 GB (6× smaller than f16) — BPW: 2.84 — Layers: 60 (unmodified) — Calibration: imatrix from 4 chunks of CLI/bash-focused text — Tool: llama.cpp (llama-quantize —imatrix …) This IQ2M matches f16 Char-F1 and beats f16 on BLEU-4 (22.39 vs 21.02) at 6× smaller. Also beats Unsloth UD-IQ2M on every metric at 0.6 GB smaller. Run with: llama-cli -m gemma4-31b-IQ2M.gguf -ngl 99 —ctx-size 4096 —temp 0.1` > Note on BFCL scores: All BFCL scores reported here use our internal simplified evaluation > (single-function-call subset with custom prompt/scoring), NOT the official Berkeley Function Calling > Leaderboard methodology. Our scores are not directly comparable to the official leaderboard. > We are working on running the official BFCL evaluation for comparable numbers. This model is the best available for code generation in the 10-12 GB tier — HumanEval+ 88.41% beats every Qwen3.6 variant including full precision. Full comparison (8 quantizations, layer-importance study, ablation charts) at otter.utopiaia.com and github.com/KikoCisBot/gemma4-31b-study. At…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: KikoCis
Теги: gguf, quantized, imatrix, IQ2_M, 2-bit, llama-cpp, gemma4, cli
Лайков: 3 | Загрузок: 76
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
