Qwen-2.5-7B-ConsistentChat is a 7B instruction-tuned chat model focused on multi-turn consistency. It is fine-tuned from the Qwen/Qwen2.5-7B base model on the ConsistentChat dataset, which is built with a skeleton-guided pipeline that explicitly models human conversational intent to reduce topic drift and improve goal completion in long dialogues. The dataset contains ~15K multi-turn conversations and ~224K utterances. Compared with generic SFT data, ConsistentChat emphasizes cross-turn consistency: it first models one of nine conversation intent trajectories, then generates a query “skeleton,” and finally fills responses, leading to substantially better consistency and task success on Light, TopDial, and MT-Eval benchmarks. This repo contains the instruction-tuned 7B ConsistentChat model, with the following base specs inherited from Qwen2.5-7B: Type: Causal Language Model Training Stage: Pretraining + Supervised Fine-Tuning (this repo) Architecture: Transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias Parameters: 7.61B (6.53B non-embedding) Layers: 28 Attention Heads (GQA): 28 for Q, 4 for KV * Context Length: up to 131,072 tokens We recommend the latest…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: jiawei-ucas
Теги: qwen2, chat, conversational, en, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 14
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
