This repository hosts the instruction‑tuned Qwen2.5-7B-Instruct model. Compared to earlier versions, Qwen2.5 offers stronger performance in areas such as coding, mathematics, long‑form generation, structured data handling, and JSON‑based outputs. It also responds more reliably to a wide range of system prompts, useful for scenarios like role‑playing agents or chatbot configuration. This specific repository provides the 7.61B‑parameter instruction‑tuned model quantized and compiled for Ara240 DNPU acceleration. This is a quantized and compiled version of Qwen/Qwen2.5-7B-Instruct optimized for Ara240 DNPU. — Base Model: Qwen/Qwen2.5-7B-Instruct — Original Model Authors: Qwen Team, Alibaba Cloud — Original License: Apache-2.0 — Modified by: NXP — SpecD — Uses a small draft model to generate speculated tokens, which the main model then verifies. — Unpack — A smaller model with 4-bit layers for prompt processing; these layers are unpacked to 8-bit precision at runtime. — TTFT: — Time to first token (TTFT). Reported as a range: the lower bound corresponds to prompts up to 128 tokens, and the upper bound reflects prompts at maximum context length. — Avg. Token Rate: Averge token rate…
Модальности:
Генерация текста
Области применения:
Следование инструкциям
Задача: Генерация текста
Автор: nxp
Теги: ara_qwen, ara240, qwen2.5, quantized, en
Лайков: 4 | Загрузок: 183
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
