This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct on the ReDiX/DataForge dataset. It achieves the following results on the evaluation set: — Loss: 1.4100 This model is an example of finetuning a sLLM. Italian eval improved and the model learned as espected from the training data The following hyperparameters were used during training: — learningrate: 0.0001 — trainbatchsize: 4 — evalbatchsize: 4 — seed: 42 — gradientaccumulationsteps: 4 — totaltrainbatchsize: 16 — optimizer: Use adamwbnb8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments — lrschedulertype: cosine — lrschedulerwarmupsteps: 10 — num_epochs: 2 — Transformers 4.46.
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: ReDiX
Теги: qwen2, generated_from_trainer, axolotl, conversational, it, en, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 1,268
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
