An antirez-style asymmetric low-bit GGUF quant of diffusiongemma-26B-A4B-it, a Gemma-4 MoE diffusion language model (26B total parameters, ~4B active, 128 experts, 8 active per token, 30 layers). > This is a DIFFUSION model, not a standard autoregressive LM. It generates by > iterative parallel canvas denoising (diffusion.canvaslength = 256, attention.causal = false), > not left-to-right next-token sampling. Standard llama-cli AR generation and perplexity > (PPL) are not the right validation harness** for this model class — coherence is judged > by generation, not PPL (see validation below). > v2 supersedes the original release. The original was built from Q4KM, imatrix-free, > with Q2K experts, and serving was not validated. This v2 rebuild fixes all three: > built from Q80 (near-lossless source), imatrix-optimized experts, and serving-validated > on a CUDA diffusion-gemma visual-server. Same filename — diffusiongemma-26B-A4B-asym-2bitexp.gguf — > so existing wiring resolves unchanged. 10.98 GB (was 12.02 GB). The whole point of an asymmetric quant is to spend bits where they matter. The routed experts are the bulk of the weights but each is touched by only a fraction of tokens,…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: hyperspaceai
Теги: gguf, diffusion, gemma, moe, 2-bit, imatrix, asymmetric-quant, endpoints_compatible
Лайков: 4 | Загрузок: 136
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
