A fine-tuned DREAM 7B masked diffusion language model trained with the ReverseThought objective. Given a question and its answer, the model learns to produce the step-by-step reasoning chain that bridges the question to the answer. This trains the model to generate coherent chain-of-thought reasoning via DREAM’s masked diffusion process. — Input: Question + Answer — Output: Detailed reasoning trace connecting them — Base model: Dream-org/Dream-v0-Instruct-7B — Training data: 75,000 examples from KIMI-K2.5-1000000x (General-Distillation subset) — Objective: DREAM masked diffusion with CART time reweighting — Hardware: 8x NVIDIA H100 80GB — Epochs: 3 — Batch size: 128 — Learning rate: 2e-6 (cosine schedule) — Max sequence length: 2048 tokens — Precision: bf16 mixed precision (FSDP) This is a masked diffusion la It uses bidirectional attention and generates text by iteratively denoising masked tokens. See the DREAM paper for details.
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: WilhelmH
Теги: Dream, diffusion, reasoning, reversethought, dream, conversational, custom_code
Лайков: 3 | Загрузок: 18
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
