REAP-pruned DeepSeek V4 Flash at K128 (128 routed experts per MoE layer). Prunes 50% of routed experts via Cerebras REAP (Router-weighted Expert Activation Pruning), preserving all attention, embeddings, shared experts, router, and MTP components. > Note: This variant preserves the donor’s mixed quantization (Q4K in layers 37-42, > IQ2XXS/Q2K elsewhere). Requires a ds4 runtime that handles per-layer quant type dispatch. > If you encounter «expected IQ2XXS expert tensors» errors, use the > uniform variant. 8,000 prompts · 5.0M tokens · 1.3B routed expert observations Calibration used the REAP activationenergysum2 score metric with 4,096 token context per prompt. Top-to-bottom expert score gap in layer 3: 2,200x (strong pruning signal). Requires eouya2/ds4-for-reaped (ds4 engine with compact GGUF support): 1. Donor GGUF: Downloaded antirez Layers37-42Q4KExperts variant (90.9 GiB) — Q4K experts in deep layers, IQ2XXS/Q2K elsewhere 2. Calibration: 8,000 prompts collected and run through ds4’s imatrix collector on a DGX Spark (NVIDIA GB10) at 4,096 token context 3. REAP scoring: Imatrix activation data converted to per-expert REAP scores using activationenergysum2 4. Pruning: 50%…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: sleepyeldrazi
Теги: gguf, deepseek, deepseek-v4, deepseek-v4-flash, mixture-of-experts, reap, expert-pruning, ds4
Лайков: 3 | Загрузок: 80
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
