NbAiLab/nb-gpt-j-6B-alpaca - Каталог нейросетей
Генерация текста

NbAiLab/nb-gpt-j-6B-alpaca

Добавлено:
NbAiLab/nb-gpt-j-6B-alpaca

This is a NB GPT-J-6B Norwegian Bokmål model fine-tuned on the Norwegian Alpaca dataset. For generation, we can either use pipeline() or the model’s .generate() method. Remember that the prompt needs a Norwegian template: The dataset is a translation to Norwegian Bokmål of alpacadatacleaned.json (a clean version of the Alpaca dataset made at Stanford) using OpenAI’s gpt-3.5-turbo model. We translated using a full-sample prompt instead of per strings, which resulted in more coherent tuples of (instruction, input, output) and costed around $60.0. This dataset cannot be used to create models that compete in any way with OpenAI. To fine-tune the NB GPT-J-6B model we used the code available on NB’s fork of mesh-transformer-jax, which provides code adapt an Alpaca dataset to finetune any GPT-J-6B model. We run finetuning for 3 epochs using sequence length of 2048 on a single TPUv3-8 for 3 hours on top of NB GPT-J-6B. — LLaMA — Stanford Alpaca — Norwegian Alpaca — ChatGPT — Hugging Face For training we have used a Google Cloud TPUv3-8 VM. For eval, you can use a T4.

Модальности:
Генерация текста


Задача: Генерация текста
Автор: NbAiLab
Теги: gptj, alpaca, no, nb, endpoints_compatible
Лайков: 2  |  Загрузок: 0

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.