Zurich 1.5B GammaCorpus v2-5m is a fine-tune of Alibaba’s Qwen 2.5 1.5B Instruct model. Zurich is designed to outperform other models that have a similar size while also showcasing GammaCorpus v2-5m. — Base Model: Qwen/Qwen2.5-1.5B-Instruct — Type: Causal Language Models — Architecture: Transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias — Number of Parameters: 1.54B — Number of Paramaters (Non-Embedding): 1.31B — Number of Layers: 28 — Number of Attention Heads (GQA): 12 for Q and 2 for KV Zurich-1.5B-GCv2-5m underwent fine-tuning with 1 A100 GPU for ~45 minutes and trained with the Unsloth framework. Zurich-1.5B-GCv2-5m was trained for 60 Epochs. We strongly recommend you use the latest version of the transformers package. You may install it via pip as follows: Here is a code snippet with applychattemplate to show you how to load the tokenizer and model and how to generate contents; This model, and all Zurich models, are trained with GammaCorpus. GammaCorpus is a dataset on HuggingFace that is filled with structured and filtered multi-turn conversations. GammaCorpus has 4 version with different sizes in each. These are the following versions and sizes: Here is a link…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: rubenroy
Теги: qwen2, text-generation-inference, unsloth, trl, gammacorpus, zurich, chat, conversational
Лайков: 3 | Загрузок: 32
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
