Gemma 2: 2B QazPerry is a fine-tuned version of the Gemma 2B model, specifically optimized for the Kazakh language. This model is part of the QazPerry initiative, which aims to develop Small Large Language Models (SLLMs) to enhance Kazakh NLP capabilities. — Base Model: Gemma 2B — Training Dataset: saillab/alpacakazakhtaco — Training Method: Fine-tuned with LoRA (Rank=64) and optimized using AdamW — License: MIT The model was fine-tuned on a dataset containing 30,000 Kazakh instruction-response pairs, ensuring accurate instruction-following performance. The training process included: — Token limit of 3,000 per response — LoRA adaptation for efficient fine-tuning — AdamW optimizer with learning rate of 5e-5 — Batch size: 2 — Training Epochs: 1 Inference Example: https://www.kaggle.com/code/armanzhalgasbayev/gemma2-2b-qazperry-first-inference The fine-tuned model demonstrates improved Kazakh language understanding, but further refinements are planned to enhance coherence and factual correctness. — Optimize inference speed for deployment in real-world applications This model is released under the MIT License. Feel free to use, modify, and distribute it under the terms of the license.…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: silvermete0r
Теги: keras, gemma, text-generation-inference, conversational, kk
Лайков: 3 | Загрузок: 43
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
