— Llama-Carvalho-PT-GL — Table of Contents — Model description — Intended uses and limitations — How to use — Training — Tools — Training data — Training hyperparameters — Framework — Evaluation — Additional information — Contact — License — Funding Llama-Carvalho-PT-GL is a 8B-parameter transformer-based causal language model for Galician, Portuguese, Spanish and English. It is the result of a continual pretraining of meta-llama/Llama-3.1-8B with a multilingual corpus consisting of 540M tokens of plain text and 72M tokens of instructions (formated as plain text) This model is part of the Carvalho familily, a family of LLMs specialized in Portuguese and Galician which can be found here. The Llama-Carvalho-PT-GL model is ready-to-use only for causal language modeling. It can perform text-generation tasks and be fine-tuned for specific scenarios. It was trained using HuggingFace Transformers and Pytorch, using the Causal Modeling Language script. We also use DeepSpeed to deal with the huge size of the model. The training corpus consists of texts in 4 languages, with an emphasis on Portuguese and Galician. The main aim of this is to ensure that the model learns to work with this…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Nos-PT
Теги: llama, Llama, gl, es, en, pt, model-index, text-generation-inference
Лайков: 3 | Загрузок: 45
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
