RaphaelMourad/Mistral-DNA-v1-422M-hg38

The Mistral-DNA-v1-422M-hg38 Large Language Model (LLM) is a pretrained generative DNA sequence model with 422M parameters. It is derived from Mixtral-8x7B-v0.1 model, which was simplified for DNA: the number of layers and the hidden size were reduced. The model was pretrained using 10kb DNA sequences from the hg38 human genome assembly. Like Mixtral-8x7B-v0.1, it is a transformer model, with the following architecture choices: — Grouped-Query Attention — Sliding-Window Attention — Byte-fallback BPE tokenizer — Mixture of Experts Ensure you are utilizing a stable version of Transformers, 4.34.0 or newer. Mistral-DNA-v1-422M-hg38 is a pretrained base model for DNA.

Модальности:
Генерация текста

Задача: Генерация текста
Автор: RaphaelMourad
Теги: mixtral, pretrained, mistral, DNA, text-generation-inference, endpoints_compatible
Лайков: 3  |  Загрузок: 2,228

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.