Based on Meta-Llama-3-8b-Instruct, and is governed by Meta Llama 3 License agreement: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct We don’t know how good this model is exactly in benchmarks since we have not benched this yet, but we think real prompts and usage is more telling anyways. — Less Refusals — More Uncensored — Follows requests better — Can reply in requested formats better without adding unnecesary information We are happy for anyone to try it out and give some feedback. Training: — 2048 sequence length, while the base model is 8192 sequence length. From testing it still performs the same 8192 context just fine. — Trained on a modified and improved version of Cognitive Computations Eric Hartford’s Dolphin dataset. https://huggingface.co/datasets/cognitivecomputations/dolphin — Training duration is around 2 days on 2x RTX3090 on our own machine, using 4-bit loading and Qlora 64-rank 128-alpha resulting in ~2% trainable weights. The goal for this model is to have the model less-censored and great at general tasks like the previous dolphin based models by Eric Hartford. We started training this BEFORE they launched their own full weight trained…
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: OwenArli
Теги: llama, conversational, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 35
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
