Eagle3-Qwen3-32B-zh is a retrained model based on the open-source Qwen3-32B. It is designed for the Eagle-3 speculative decoding algorithm and features bilingual (Chinese/English) acceleration capabilities, aiming to speed up the inference phase of Large Language Models (LLMs). Trained on a mixed Chinese-English dataset, this model demonstrates an improved acceptance rate for Chinese text, making it highly suitable for mixed-language inference tasks. To reduce training costs, we built a low-cost Eagle-3 training pipeline based on the Eagle and SpecForge frameworks, specifically optimized for consumer-grade GPUs and All-in-one AI server production environments. The end-to-end training process can run on NVIDIA RTX 4090 workstations with optimized efficiency, completing the training in approximately one week. Notably, our draft model, trained on only about 100k samples, outperforms currently available open-source weights of the same class (showing slightly better performance in English and significant improvements in Chinese acceleration). In real-world tests on a 4x NVIDIA RTX 4090 workstation, we achieved a decoding acceleration of up to nearly 2x on GSM8K with a concurrency of…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Zjcxy-SmartAI
Теги: llama
Лайков: 3 | Загрузок: 102
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
