AngelSlim/Qwen3-8b-dflare

This is the official DFlare draft model checkpoint for Qwen/Qwen3-8B`**, released alongside the paper: > DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding DFlare is a block-diffusion speculative decoding framework that accelerates large language model inference by predicting an entire block of tokens in one shot for the target model to verify in parallel. It removes the narrow conditioning bottleneck of the prior state-of-the-art DFlash through a lightweight layer-wise fusion mechanism: each draft layer attends to its own learnable combination of a broad set of target layers at negligible overhead, simultaneously injecting richer target knowledge and giving every draft layer a distinct input. Combined with training-data scaling, this enhanced per-layer expressiveness allows the draft model to scale to deeper architectures with consistent gains, achieving 5.46× end-to-end speedup on Qwen3-8B without compromising output quality. 📖 Documentation & code: https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/dflare.html The draft predicts a block of blocksize` tokens in parallel, conditioned on (i) target hidden states extracted from the…

Модальности:
Генерация текста

Задача: Генерация текста
Автор: AngelSlim
Теги: qwen3, speculative-decoding, block-diffusion, dflare, draft-model, text-generation-inference, endpoints_compatible
Лайков: 3  |  Загрузок: 127

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.