This is the final flagship model from the Open-MOPD pipeline. Starting from BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT, it is trained with multi-teacher online policy distillation using three domain-specific RL teachers for math, code, and instruction following. This release corresponds to training step 200. Each prompt is hard-routed to its domain teacher. The dense reward is the teacher-student log-probability gap over the student’s top-k distribution (k=16, nucleus p=0.99), weighted by the student probabilities and applied directly as the token-level advantage. Open-MOPD uses three mechanisms to prevent cross-domain training imbalance: — Token-share balancing keeps the weighted gradient-token share near one third per domain. Without it, math and code consume about 99% of gradient tokens. — Gap-aware allocation (alpha=1) shifts the training budget toward domains with more remaining teacher-student improvement. — Reward refresh recomputes the student-dependent part of the dense reward at every inner update without adding extra prefills. Training uses a global batch size of 1,024, mini-batch size 256, constant learning rate 1.5e-6, clipping at 0.2/0.28, no KL penalty, and a…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: BytedTsinghua-SIA
Теги: smollm3, open-mopd, online-policy-distillation, conversational, en, endpoints_compatible
Лайков: 3 | Загрузок: 174
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
