This model follows the Humpback architecture, proposed in the paper Self-Alignment with Instruction Backtranslation by Li et al. It represents the resulting model after the first iteration of self-curation, which is trained on a small amount of gold data and a set of generated data curated by the «seed model». This model can be used for instruction-following. It may also be used to, again, score the instruction-response pairs generated by the «backward model» for a second iteration of self-curation. Humpback uses instruction backtranslation on a web corpus to generate input-output pairs (self-augmentation), creating a richer dataset for fine-tuning models without the need for additional manual annotation. The model then iteratively curates the created dataset, scoring the pairs by quality, and is then finetuned on the resulting subset of all pairs with the highest possible score (self-curation). Varying from the original paper, this model is a fine-tuned version of meta-llama/Llama-3.2-3B. It has been trained using TRL. The dataset used to train this model is a combination of data sampled from the oasst1 dataset and the synthetic dataset which was mentioned above. The latter has…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Alepach
Теги: llama, generated_from_trainer, trl, sft, conversational, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 35
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
