d3LLM-Dream-Coder is an ultra-fast diffusion language model introduced in the paper d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation. It is built on Dream-org/Dream-Coder-v0-Instruct-7B. d3LLM (pseuDo-Distilled Diffusion Large Language Model) is a framework designed to strike a balance between accuracy and parallelism in diffusion LLMs. It achieves up to 10× speedup over vanilla diffusion models like LLaDA/Dream and 5× speedup over autoregressive (AR) models. The model utilizes two primary innovations: — Pseudo-Trajectory Distillation: A training method that teaches the model which tokens can be decoded confidently at early steps. — Entropy-Based Multi-Block Decoding: An inference strategy using a KV-cache refresh mechanism to maintain accuracy while maximizing parallelism. — Paper: d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation — Repository: https://github.com/hao-ai-lab/d3LLM — Blog: https://hao-ai-lab.github.io/blogs/text-diffusion/ — Demo: https://d3llm-team.github.io/ For detailed usage instructions, evaluation scripts, and training code, please refer to the official GitHub repository. Since the model uses a custom architecture,…
Модальности:
Генерация текста
Области применения:
Диалог / чат Генерация кода
Задача: Генерация текста
Автор: d3LLM
Теги: Dream, feature-extraction, diffusion, fast-inference, d3llm, conversational, custom_code
Лайков: 4 | Загрузок: 37
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
