SambaLingo-Arabic-Chat-70B is a human aligned chat model trained in Arabic and English. It is trained using direct preference optimization on top the base model SambaLingo-Arabic-Base-70B. The base model adapts Llama-2-70b to Arabic by training on 28 billion tokens from the Arabic split of the Cultura-X dataset. Try This Model at SambaLingo-chat-space. — Developed by: SambaNova Systems — Model type: Language Model — Language(s): Arabic, English — Finetuned from model: Llama-2-70b — Paper: SambaLingo: Teaching Large Language Models New Languages — Blog Post: sambalingo-open-source-language-experts Please make sure to set use_fast=False when loading the tokenizer. Please make sure to set use_fast=False when loading the tokenizer. To prompt this model, please use the following chat template: The alignment phase follows the recipe for Zephyr-7B, and comprises two stages: supervised fine-tuning (SFT) and Direct Performance Optimization (DPO). The SFT phase was done on the ultrachat200k dataset mixed with the Google translated version of the ultrachat200k dataset. It was trained for one epoch with global batch size 512 and max sequence length 2048 tokens. We used a linear decay learning…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: sambanovasystems
Теги: llama, conversational, ar, en, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 46
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
