Zurich 14B GammaCorpus v2-10k is a fine-tune of Alibaba’s Qwen 2.5 14B Instruct model. Zurich is designed to outperform other models that have a similar size while also showcasing GammaCorpus v2-10k. — Base Model: Qwen/Qwen2.5-14B-Instruct — Type: Causal Language Models — Architecture: Transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias — Number of Parameters: 14.7B — Number of Paramaters (Non-Embedding): 13.1B — Number of Layers: 48 — Number of Attention Heads (GQA): 40 for Q and 8 for KV Zurich-14B-GCv2-10k underwent fine-tuning with 1 A100 GPU for ~10 minutes and trained with the Unsloth framework. Zurich-14B-GCv2-10k was trained for 60 Epochs. We strongly recommend you use the latest version of the transformers package. You may install it via pip as follows: Here is a code snippet with applychattemplate to show you how to load the tokenizer and model and how to generate contents; This model, and all Zurich models, are trained with GammaCorpus. GammaCorpus is a dataset on HuggingFace that is filled with structured and filtered multi-turn conversations. GammaCorpus has 4 version with different sizes in each. These are the following versions and sizes: Here is a link…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: rubenroy
Теги: qwen2, text-generation-inference, unsloth, trl, gammacorpus, zurich, chat, conversational
Лайков: 3 | Загрузок: 24
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
