inclusionAI/GroveMoE-Base

🤗 Models&nbsp&nbsp | &nbsp&nbsp 📑 Paper &nbsp&nbsp | &nbsp&nbsp 🔗 Github&nbsp&nbsp We introduce GroveMoE, a new sparse architecture using adjugate experts for dynamic computation allocation, featuring the following key highlights: — Architecture: Novel adjugate experts grouped with ordinary experts; shared computation is executed once, then reused, cutting FLOPs. — Sparse Activation: 33 B params total, only 3.14–3.28 B active per token. — Traning: Mid-training + SFT, up-cycled from Qwen3-30B-A3B-Base; preserves prior knowledge while adding new capabilities.

Модальности:
Генерация текста

Области применения:
Диалог / чат

Задача: Генерация текста
Автор: inclusionAI
Теги: qwen3_moe, conversational, custom_code, endpoints_compatible
Лайков: 3  |  Загрузок: 81

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.