Aira-2 is the second version of the Aira instruction-tuned series. Aira-2-774M is an instruction-tuned model based on GPT-2. The model was trained with a dataset composed of prompts and completions generated synthetically by prompting already-tuned models (ChatGPT, Llama, Open-Assistant, etc). — Size: 774,032,640 parameters — Dataset: Instruct-Aira Dataset — Language: English — Number of Epochs: 3 — Batch size: 8 — Optimizer: torch.optim.AdamW (warmupsteps = 1e2, learningrate = 5e-4, epsilon = 1e-8) — GPU: 1 NVIDIA A100-SXM4-40GB — Emissions: 0.77 KgCO2 (Singapore) — Total Energy Consumption: 1.58 kWh This repository has the source code used to train this model. Three special tokens are used to mark the user side of the interaction and the model’s response: What is a language model?A language model is a probability distribution over a vocabulary. — Hallucinations: This model can produce content that can be mistaken for truth but is, in fact, misleading or entirely false, i.e., hallucination. — Biases and Toxicity: This model inherits the social and historical stereotypes from the data used to train it. Given these biases, the model can produce toxic content, i.e., harmful,…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: nicholasKluge
Теги: gpt2, alignment, instruction tuned, text generation, conversation, assistant, en, co2_eq_emissions
Лайков: 3 | Загрузок: 175
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
