Romulus is a series of continually pre-trained models enriched in French law and intended to serve as the basis for a fine-tuning process on labeled data. Please note that these models have not been aligned for the production of usable text as they stand, and will certainly need to be fine-tuned for the desired tasks in order to produce satisfactory results. The training corpus is made up of around 34,864,949 tokens (calculated with the meta-llama/Meta-Llama-3.1-8B-Instruct tokenizer). The following table outlines the key hyperparameters used for training Romulus. Romulus was trained using Unsloth on a Nvidia H100 Azure EST US instance provided by the Microsoft for Startups program from this script: If you use this code in your research, please use the following BibTeX entry. If you have any feedback, please reach out at louisbrulenaudet@icloud.com.
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: louisbrulenaudet
Теги: llama, law, droit, unsloth, trl, sft, conversational, fr
Лайков: 3 | Загрузок: 38
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
