𓌳 REAP𓌳 the Experts: Why Pruning Prevails for One-Shot MoE Compression Introducing MiniMax-M2-REAP-162B-A10B, a memory-efficient compressed variant of MiniMax-M2 that maintains near-identical performance while being 30% lighter. This model was created using REAP (Router-weighted Expert Activation Pruning), a novel expert pruning method that selectively removes redundant experts while preserving the router’s independent control over remaining experts. Key features include: — Near-Lossless Performance: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 230B model — 30% Memory Reduction: Compressed from 230B to 162B parameters, significantly lowering deployment costs and memory requirements — Preserved Capabilities: Retains all core functionalities including code generation, math & reasoning and tool calling. — Drop-in Compatibility: Works with vanilla vLLM — no source modifications or custom patches required — Optimized for Real-World Use: Particularly effective for resource-constrained environments, local deployments, and academic research Benchmark MiniMax-M2 MiniMax-M2-REAP-172B-A10B MiniMax-M2-REAP-162B-A10B…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: cyankiwi
Теги: minimax_m2, minimax, MOE, pruning, compression, conversational, custom_code, en
Лайков: 4 | Загрузок: 30
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
