GPTQ weights can be found here: https://huggingface.co/bhenrym14/airophin-v2-13b-PI-8k-GPTQ This is a finetune of Llama-2-13b, intended to extend the useful context window to 8192 tokens via position interpolation (PI). There are two training phases, but in this model I only perform the final finetune on the Airoboros m2.0 dataset. 1. I start with OpenAssistant/llama2-13b-orca-8k-3319. This model has been trained on a mix of orca-chat (dolphin derived), fanfics, and redpajama; the majority of the dataset is orca-chat, hence why I retain the airophin naming for this model. 2. The model was then finetuned on the merged Airoboros dataset (1.4.1 merged with 2.0) Jon Durbin’s Airoboros GPT4 m2.0, with same scaling approach, for 2 epochs. This model employs linear RoPE scaling, which is now has native support in Transformers (be sure to update it if you have issues). Use it as you would with any normal context length variant. Please comment with any questions. The GPTQ version can be found here. I may upload a GGML version soon, especially if anyone expresses interest. Ooba use: Be sure to increase the Truncate the prompt up to this length parameter to 8192 to utilize the full context…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: bhenrym14
Теги: llama, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 50
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
