Chocolatine-Admin-3B version specialized in French administrative language, supervised fine-tuning of jpacifico/Chocolatine-3B-Instruct-DPO-v1.2 based on microsoft/Phi-3.5-mini-instruct Developed in collaboration with Microsoft. The dataset based on the official lexicon published by the French DITP, gathers 2362 administrative terms constituting the basis of the simulation of prompt-answer pairs. The GPT-4o model deployed on Azure OpenAI was used to carry out the building of the dataset in several phases: — Extraction of the lexicon pages (previously converted into jpg format) — Reformulation of the definitions to make them more readable and natural to be used by an LLM in order to ensure high quality data. — Generation of questions from the terms and definitions — Generation of answers in three successive rounds taking into account the previous generations to ensure variety. For this 0.3b version, the Fine Tuning (SFT) was performed on 11 epochs with an A100 GPU instance on Azure Machine Learning. The Chocolatine model series is a quick demonstration that a base model can be easily fine-tuned to achieve compelling performance. It does not have any moderation mechanism. -…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: jpacifico
Теги: phi3, text-generation-inference, sft, chocolatine, conversational, custom_code, fr, endpoints_compatible
Лайков: 4 | Загрузок: 14
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
