The Mistral-Chem-v1-417M Large Language Model (LLM) is a pretrained generative chemical molecule model with 417M parameters. It is derived from Mixtral-8x7B-v0.1 model, which was simplified for molecules: the number of layers and the hidden size were reduced. The model was pretrained using 10M molecule SMILES strings from the PubChem database. Like Mixtral-8x7B-v0.1, it is a transformer model, with the following architecture choices: — Grouped-Query Attention — Sliding-Window Attention — Byte-fallback BPE tokenizer — Mixture of Experts Ensure you are utilizing a stable version of Transformers, 4.34.0 or newer. Mistral-Chem-v1-417M is a pretrained base model for chemistry.
Модальности:
Генерация текста
Области применения:
Химия
Задача: Генерация текста
Автор: RaphaelMourad
Теги: mixtral, pretrained, mistral, chemistry, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 18
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
