An exclusively Italian speaking, instruction finetuned, Large Language model. 🇮🇹 The Loquace Italian LLM models are created as a proof-of-concept to evaluate on how language tuning can be achieved using QLoRa by instruct-tunings foundational LLMs using dataset of a specific language. The QLoRa (https://github.com/artidoro/qlora) method of fine-tuning significantly lower the resources requirements compared to any other methods available, this allow to easily execute the process on significanly larger dataset while still using consumers GPUs and still achieve high accuracy. Loquace-70m is the smallest model of the Loquace family. It was trained using QLoRa on a large dataset of 102k question/answer pairs exclusively in Italian. The related code can be found at: https://github.com/cosimoiaia/Loquace https://huggingface.co/cosimoiaia/Loquace-70m — Based on pythia-70m https://huggingface.co/cosimoiaia/Loquace-410m — Based on pythia-410m https://huggingface.co/cosimoiaia/Loquace-7B — Based on Falcon-7B. https://huggingface.co/cosimoiaia/Loquace-12B — Based on pythia-12B https://huggingface.co/cosimoiaia/Loquace-20B — Based on gpt-neox-20B Loquace-70m was trained on a…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: cosimoiaia
Теги: gpt_neox, alpaca, llama, llm, finetune, Italian, qlora, conversational
Лайков: 3 | Загрузок: 11
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
