This is a 13 billion parameter fine-tuned bilingual large language model for both Arabic and English. It is based on transformer-based decoder-only (GPT-3) architecture and uses SwiGLU non-linearity. It implements ALiBi position embeddings, enabling the model to extrapolate to long sequence lengths, providing improved context handling and model precision. Jais-13b-chat is Jais-13b fine-tuned over a curated set of 4 million Arabic and 6 million English prompt-response pairs. We further fine-tune our model with safety-oriented instruction, as well as providing extra guardrails in the form of a safety prompt. Our pre-trained model, Jais-13b, is trained on 116 billion Arabic tokens and 279 billion English tokens. The combination of the largest curated Arabic and English instruction tuning dataset along with the addition of multi-turn conversations allows the model to converse in a variety of topics, with a particular focus on the Arab world. Below is sample code to use the model. Note that the model requires a custom model class, so users must enable trustremotecode=True while loading the model. In order to get the same performance as our testing, a specific prompt needs to be…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: asas-ai
Теги: jais, Arabic, English, LLM, Decoder, causal-lm, conversational, custom_code
Лайков: 2 | Загрузок: 11
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
