Stambecco 7B Plus is an Italian instruction-following Parameter-Efficient Fine-Tuning (PEFT) adapter based on Meta’s LLaMA-7B architecture. It was trained on the Stambecco Plus Dataset, an Italian instruction dataset traslated from Alpaca-GPT4 (GPT-4-LLM), a dataset distilled directly from GPT-4. As a result, this model offers superior Italian reasoning, vocabulary, and response quality compared to standard GPT-3.5-based models. GitHub Project: mchl-labs/stambecco Dataset Used: mchl-labs/stambeccodataplusit Model Type: PEFT / LoRA Adapter Delta This repository contains the LoRA adapter weights (delta). To run inference, load these weights on top of the base LLaMA-13B model using Hugging Face’s transformers and peft libraries: > 💡 Try it in Colab: You can also run interactive chat directly in Google Colab using our Inference Notebook. The adapter was fine-tuned using the following hyperparameters: — learningrate: 0.0003 — trainbatchsize: 4 — evalbatchsize: 8 — gradientaccumulationsteps: 32 — totaltrainbatchsize: 128 — optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 — lrschedulertype: linear — lrschedulerwarmupsteps: 100 — numepochs: 10 — mixedprecisiontraining: Native…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: mchl-labs
Теги: peft, llama, lora, italian, instruction-tuning, gpt-4, stambecco, it
Лайков: 2 | Загрузок: 7
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
