This is an experiment for developing a language generation model for the Nepali language. Causal Language Model which can predict the next possible tokens given a context in Nepali language. A large corpus of 9.3 GB size has been collected from different sources on the internet. The sources include — Nepali Books found online. — Nepali News Article from Nepali news portals. — Nepali text collected from different open source Nepali NLP datasets. Learning rate -> 2e-5 Weight Decay -> 0.01 Number of training epochs -> 5 bf16 -> True Base Model Architecture -> GPT-2
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Shushant
Теги: gpt2, ne, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 37
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
