budecosystem/genz-13b-infinite - Каталог нейросетей
Генерация текста

budecosystem/genz-13b-infinite

Добавлено:

The model is a finetuned version of Genz-13B-v2 with a context size of 16K. The model architecture is updated to have lamda attention from the LM-Infinite paper which gives the model capability of 120K+ sequence length without affecting the preplexity You can integrate the model in your code my loading convertllamamodel function. The model is trained of 4 A100 80GB for approximately 55hrs. We’d like to thank the open-source community and the researchers whose foundational work laid the path to this model. Special shoutout to the authors of LM-Infinite paper and the GitHub repo

Модальности:
Генерация текста


Задача: Генерация текста
Автор: budecosystem
Теги: llama, text-generation-inference, endpoints_compatible
Лайков: 2  |  Загрузок: 50

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.