MPT-7B-Instruct is a model for short-form instruction following. It is built by finetuning MPT-7B on a dataset derived from the Databricks Dolly-15k and the Anthropic Helpful and Harmless (HH-RLHF) datasets. License: CC-By-SA-3.0 Demo on Hugging Face Spaces This model was trained by MosaicML and follows a modified decoder-only transformer architecture. Blog post: Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs Codebase (mosaicml/llm-foundry repo) * Questions: Feel free to contact us via the MosaicML Community Slack! >A Quoll (pronounced “cool”) is one of Australia’s native carnivorous marsupial mammals, which are also known as macropods or wallabies in other parts around Asia and South America It includes options for many training efficiency features such as FlashAttention (Dao et al. 2022), ALiBi, QK LayerNorm, and more. To use the optimized triton implementation of FlashAttention, you can load the model with attnimpl=’triton’ and move the model to bfloat16`: Although the model was trained with a sequence length of 2048, ALiBi enables users to increase the maximum sequence length during finetuning and/or inference. For example: This model was…
Модальности:
Генерация текста
Области применения:
Следование инструкциям
Задача: Генерация текста
Автор: k0t1k
Теги: mpt, Composer, MosaicML, llm-foundry, custom_code, text-generation-inference
Лайков: 3 | Загрузок: 27
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
