Zurich 7B GammaCorpus v2-50k is a fine-tune of Alibaba’s Qwen 2.5 7B Instruct model. Zurich is designed to outperform other models that have a similar size while also showcasing GammaCorpus v2-50k. — Base Model: Qwen/Qwen2.5-7B-Instruct — Type: Causal Language Models — Architecture: Transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias — Number of Parameters: 7.61B — Number of Paramaters (Non-Embedding): 6.53B — Number of Layers: 28 — Number of Attention Heads (GQA): 28 for Q and 4 for KV Zurich-7B-GCv2-50k underwent fine-tuning with 1 T4 GPU for ~45 minutes and trained with the Unsloth framework. Zurich-7B-GCv2-50k was trained for 60 Epochs. We strongly recommend you use the latest version of the transformers package. You may install it via pip as follows: Here is a code snippet with applychattemplate to show you how to load the tokenizer and model and how to generate contents; This model, and all Zurich models, are trained with GammaCorpus. GammaCorpus is a dataset on HuggingFace that is filled with structured and filtered multi-turn conversations. GammaCorpus has 4 version with different sizes in each. These are the following versions and sizes: Here is a link to the…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: rubenroy
Теги: qwen2, text-generation-inference, unsloth, trl, gammacorpus, zurich, chat, conversational
Лайков: 4 | Загрузок: 11
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
