> A 34%-expert-pruned GLM-5.2 that holds parity with the full unpruned model on a > well-powered real-world eval — recovered by training only the router gates (0.016% of params), > and quantized to NVFP4 for 8×B200-class serving. This is the flagship cut of the GLM-5.2 REAP series — the largest and highest-fidelity, a strict superset of the earlier (now-retired) 481B / 469B cuts. > 📄 Full technical report: REPORT.md — the complete study (methods, the n=50→n=2000 > correction, the significance stats, the GGUF DSA-indexer surgery, negative results, and the free > sampler fix). GLM-5.2 is a GlmMoeDsaForCausalLM MoE — 78 layers (3 dense + 75 MoE) + 1 MTP layer, 256 routed experts per layer (top-8) + 1 shared expert, DeepSeek-style MLA attention with a DSA sparse «indexer,» hidden size 6144. This model keeps 168 of the 256 routed experts per layer (≈504B params, down from ~744–763B), consistently across every MoE layer and the MTP layer (nroutedexperts: 168), so it loads and serves cleanly in vLLM. Pruning experts damages routing. Instead of expensive full fine-tuning, we freeze the entire network — experts, attention, embeddings — and train only the 75 router gate matrices (~0.016% of…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: 0xSero
Теги: glm_moe_dsa, moe, reap, pruning, expert-pruning, router-kd, nvfp4, glm
Лайков: 4 | Загрузок: 96
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
