DeepSeek-V4-Flash-0731 with its routed experts reduced from 256 to 132 per layer by router-weighted expert activation pruning (REAP), taking the checkpoint from 156 GiB to 79 GiB. All 43 layers are kept; only the expert population inside each MoE block changes. No fine-tuning, no distillation, and no gradient step of any kind — the experts are pruned from calibration statistics in a single pass. Metrics: GSM8K exactmatch,strict-match, MATH-500 mathverify,none, HumanEval+/MBPP+ pass@1plus. All greedy (n=1), 4096-token context, enablethinking=false, served with SGLang. — The multi-token-prediction modules are gone. The base checkpoint carries mtp.0, mtp.1 and mtp.2; this one carries none of them. MTP-based speculative decoding is therefore unavailable. Engines that look for those weights will fall back to ordinary decoding. Nothing else references them, so standard generation is unaffected. Everything else about the prompt and generation defaults is the base model’s. Reasoning effort: Set reasoningeffort to «low» (default), «high», or «max» (applies only in thinking mode). 1. Token Budget: Thinking consumes tokens before the actual answer starts. Ensure your maxtokens is high enough…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: puwaer
Теги: deepseek_v4, moe, expert-pruning, reap, compressed, conversational, endpoints_compatible, 8-bit
Лайков: 3 | Загрузок: 2,360
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
