This is a GPT-2 (Generative Pre-trained Transformer 2) model, fine-tuned specifically for the Karakalpak language. The model is designed to generate coherent and contextually relevant text in Karakalpak, and capable of basic conversational responses based on the training data. The model was fine-tuned on a custom dataset combining: — Approximately 300,000 lines of parallel text data — Approximately 150 question-answer pairs of conversational dialogues in Karakalpak. These dialogues focus on simple greetings and basic exchanges. Data Preparation: — The training data was consolidated into a single text file. — Special tokens (, ) were used to delineate conversational turns in the dialogue data, allowing the model to learn conversational structure. — Karakalpak specific characters (Á, á, Ǵ, ǵ, Ń, ń, Ó, ó, Ú, ú, Í, ı) were added as special tokens to the tokenizer to ensure proper handling of the language’s unique alphabet. — Base Model: gpt2 (from Hugging Face Transformers) — Framework: PyTorch with Hugging Face Transformers Trainer API — GPU: NVIDIA RTX 6000 Ada (48 GB VRAM) — Batch Size: 8 (or higher, depending on VRAM) — Number of Epochs: 5 — Optimizer: AdamW — Learning Rate: 5e-5…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: nickoo004
Теги: gpt2, kaa
Лайков: 4 | Загрузок: 10
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
