This is a llama-2-70B model combined with two adapters via appropriate methods. Orca-style 40K dataset. This dataset is a filtered subset of OpenOrca-GPT4 and airoboros-gpt4-1.4.1. Samantha made by Eric Hartford and cleaned by us, about 6.5K samples. We train our adapters with jondurbin’s forked QLoRA repo We add multi-turn conversational data support from fastchat repo, with minor modifications. We use bash shell script similar to airoboros-70b-gpt4-1.4.1 to train our two adapters. We found that combining multiple adapters to a single llama-2-70B could achieve better performance than merging only 1 adapter to llama-2-70B. The details of combining multiple adapters will be unveiled in our upcoming paper. This model is bound by the license & usage restrictions of the original Llama-2 model. And comes with no warranty or gurantees of any kind. Llama 2 and fine-tuned variants are a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Llama 2 and any fine-tuned varient’s potential outputs cannot be predicted in advance, and the model may in some…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: ICBU-NPU
Теги: llama, en, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 68
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
