This is an unofficial build, made and published independently of Google. It is a llama.cpp GGUF of Gemma 4 E4B-it in which every 4-bit weight is the value Google’s quantization-aware training (QAT) produced. Every weight matrix is stored at 4 bits, including both embedding tables. It is 4.22 GB, against 5.15 GB for Google’s gemma-4-E4B-it-qat-q40-gguf, and follows the unquantized model much more closely. Google’s model card is kept unchanged in ORIGINALREADME.md. The E2B counterpart, built by the same script, is xbill9/gemma-4-E2B-it-qat-q40-exact-gguf`. Measured with llama.cpp build f95b0d9 (318) on an i7-10750H laptop with a GTX 1650 Ti Max-Q (4 GB, no tensor cores). Divergence and perplexity: llama-perplexity on wikitext-2 test, 16 chunks of 512 tokens, on the CPU, against a bf16 GGUF of the same QAT weights with identical metadata. Speed: llama-bench, 5 repetitions, four passes alternating which file runs first, each pass starting at or below 50 °C; the table gives the mean of the four passes, and the generation ratio was 1.058–1.063 in every pass. Logs are in evidence/. It fits a 4 GB card. With -ngl 99 -c 8192 the whole process holds 2848 MiB, because perlayertokenembd`…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: xbill9
Теги: gguf, gemma4, llama.cpp, q4_0, qat, cpu, cuda, text-only
Лайков: 3 | Загрузок: 711
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
