This is a lora + gradient merge between: — Chronos 13b v2 — Airoboros l2 13b gpt4 2.0 — LimaRP llama 2 Lora from July 28, 2023 at a weight of 0.25. You can check out the sister model chronolima airo grad l2 13B which also produces great responses. For this merge, Airoboros merged with LimaRP at a 0.25 weight was added in an inverted curve gradient at a 0.9 ratio and slowly trickled down to 0 at the 25th layer. I have provided an illustration to help visualize this merge. Blue is chronos and green is airolima. !airolima-chronos-illustration Unlike a basic ratio merge (ex. 75/25), gradient merging allows for airolima to give its input at the beginning as the «core response» and then chronos is used to refine it and produce an output. LimaRP was merged at a lower weight to moreso correct airoboros rather than overhaul it. Higher weights (like single-model lora merges) completely destroyed a character’s personality and made chatting bland (similar to chronos’s tests). Since this is a merge between Airoboros, Chronos, and LimaRP, the following instruction formats should work: LimaRP instruction format (this might not work due to its weight): Chronos has a bias to talk very expressively…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: kingbri
Теги: llama, llama-2, en, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 60
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
