This is a NB GPT-J-6B Norwegian Bokmål model fine-tuned on the Norwegian Alpaca dataset. For generation, we can either use pipeline() or the model’s .generate() method. Remember that the prompt needs a Norwegian template: The dataset is a translation to Norwegian Bokmål of alpacadatacleaned.json (a clean version of the Alpaca dataset made at Stanford) using OpenAI’s gpt-3.5-turbo model. We translated using a full-sample prompt instead of per strings, which resulted in more coherent tuples of (instruction, input, output) and costed around $60.0. This dataset cannot be used to create models that compete in any way with OpenAI. To fine-tune the NB GPT-J-6B model we used the code available on NB’s fork of mesh-transformer-jax, which provides code adapt an Alpaca dataset to finetune any GPT-J-6B model. We run finetuning for 3 epochs using sequence length of 2048 on a single TPUv3-8 for 3 hours on top of NB GPT-J-6B. — LLaMA — Stanford Alpaca — Norwegian Alpaca — ChatGPT — Hugging Face For training we have used a Google Cloud TPUv3-8 VM. For eval, you can use a T4.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: NbAiLab
Теги: gptj, alpaca, no, nb, endpoints_compatible
Лайков: 2 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.