An exclusively Italian speaking, instruction finetuned, Large Language model. 🇮🇹 The Loquace Italian LLM models are created as a proof-of-concept to evaluate on how language tuning can be achieved using QLoRa by instruct-tunings foundational LLMs using dataset of a specific language. The QLoRa (https://github.com/artidoro/qlora) method of fine-tuning significantly lower the resources requirements compared to any other methods available, this allow to easily execute the process on significanly larger dataset while still using consumers GPUs and still achieve high accuracy. Loquace-7B is the first 7B italian Large Language Model trained using QLoRa on a large dataset of 102k question/answer pairs exclusively in Italian and that uses Falcon-7B model as base, the most accurate model of it’s class. The related code can be found at: https://github.com/cosimoiaia/Loquace https://huggingface.co/cosimoiaia/Loquace-70m — Based on pythia-70m https://huggingface.co/cosimoiaia/Loquace-410m — Based on pythia-410m https://huggingface.co/cosimoiaia/Loquace-7B — Based on Falcon-7B https://huggingface.co/cosimoiaia/Loquace-12B — Based on pythia-12B…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: cosimoiaia
Теги: RefinedWebModel, alpaca, llama, llm, finetune, Italian, qlora, conversational
Лайков: 3 | Загрузок: 23
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
