This repository contains a DLite draft checkpoint presented in DLite: History-free Speculative Decoding for Agentic AI. DLite is a history-free speculative decoding framework for agentic AI. It reuses contextual hidden states from the target model to predict draft tokens without maintaining a historical draft KV cache. A lightweight sequential head restores dependencies within each speculative block, while a two-stage training strategy aligns the draft model with the target model. DLite draft checkpoints are paired with specific target models. See the Hugging Face collection for available checkpoints. The draft checkpoint must use DLiteDraftModel, modelrole: «student», and sequentialhead: «rnn». See https://github.com/SJTU-DAI-Sys/DLite for the inference release, including the Python API, CLI, and JSONL throughput benchmark.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: SJTU-DAI-Sys
Теги: qwen3, feature-extraction, custom_code, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 138
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
