Occiglot-7B-FR-EN is a generative language model with 7B parameters for French and English and trained by the Occiglot Research Collective. It is based on Mistral-7B-v0.1 and trained on 113B tokens of additional multilingual and code data with a block size of 8,192 tokens per sample. Note that the model is a general-purpose base model and was not instruction-fine-tuned nor optimized for chat or other applications. We make an instruction tuned variant available as occiglot-7b-fr-en-instruct This is the first release of an ongoing open research project for multilingual language models. If you want to train a model for your own language or are working on evaluations, please contact us or join our Discord server. We are open for collaborations! — Continued-pretraining from: Mistral-7B-v0.1 — Model type: Causal decoder-only transformer language model — Languages: English, French, and code. — License: Apache 2.0 — Compute resources: HessianAI’s 42 — Contributors: Manuel Brack, Patrick Schramowski, Pedro Ortiz, Malte Ostendorff, Fabio Barth, Georg Rehm, Kristian Kersting — Research labs: Occiglot with support from SAINT and SLT — Contact: Discord You can use this model directly with a…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: occiglot
Теги: mistral, en, fr, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 105
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
