Aira-2 is the second version of the Aira instruction-tuned series. Aira-2-1B1 is an instruction-tuned model based on TinyLlama-1.1B. The model was trained with a dataset composed of prompts and completions generated synthetically by prompting already-tuned models (ChatGPT, Llama, Open-Assistant, etc). — Size: 1,261,545,472 parameters — Dataset: Instruct-Aira Dataset — Language: English — Number of Epochs: 3 — Batch size: 4 — Optimizer: torch.optim.AdamW (warmupsteps = 1e2, learningrate = 5e-4, epsilon = 1e-8) — GPU: 1 NVIDIA A100-SXM4-40GB — Emissions: 1.71 KgCO2 (Singapore) — Total Energy Consumption: 3.51 kWh This repository has the source code used to train this model. Three special tokens are used to mark the user side of the interaction and the model’s response: What is a language model?A language model is a probability distribution over a vocabulary. — Hallucinations: This model can produce content that can be mistaken for truth but is, in fact, misleading or entirely false, i.e., hallucination. — Biases and Toxicity: This model inherits the social and historical stereotypes from the data used to train it. Given these biases, the model can produce toxic content, i.e.,…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: nicholasKluge
Теги: llama, alignment, instruction tuned, text generation, conversation, assistant, en, co2_eq_emissions
Лайков: 2 | Загрузок: 163
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.