DeepCoder-1.5B-Preview 🚀 Democratizing Reinforcement Learning for LLMs (RLLM) 🌟 DeepCoder-1.5B-Preview is a code reasoning LLM fine-tuned from DeepSeek-R1-Distilled-Qwen-1.5B using distributed reinforcement learning (RL) to scale up to long context lengths. Our training dataset consists of approximately 24K unique problem-tests pairs compiled from: — Taco-Verified — PrimeIntellect SYNTHETIC-1 — LiveCodeBench v5 (5/1/23-7/31/24) Our training recipe relies on an improved version of GRPO (GRPO+) and iterative context lengthening, introduced in DeepScaleR. We enhance the original GRPO algorithm with insights from DAPO to enable more stable training: — Offline Difficulty Filtering: DAPO employs online dynamic sampling, discarding both entirely correct and entirely incorrect samples on the fly. While this helps maintain a more stable effective batch size, it introduces significant runtime overhead due to rejection sampling. Instead, we perform offline difficulty filtering on a subset of coding problems to ensure the training dataset remains within a suitable difficulty range. — No Entropy Loss: We observed that including an entropy loss term often led to instability, with entropy…
Модальности:
Генерация текста
Области применения:
Диалог / чат Генерация кода
Задача: Генерация текста
Автор: Mungert
Теги: gguf, en, endpoints_compatible, conversational
Лайков: 4 | Загрузок: 243
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
