A small-scale RP/storywriting specialist model, full-parameter finetune of Qwen2.5-1.5B on mixture of synthetic and natural data. It uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and «flavor» of the resulting model. Unlike EVA-D 1.5B v0.0, this model was created without using DistillKit, and unlike other versions of EVA, Spectrum wasn’t used either, since layer freezing is inefficient at small scale. Training data: Celeste 70B 0.1 data mixture minus Opus Instruct subset. See that model’s card for details. Kalomaze’s OpusInstruct25k dataset, filtered for refusals. A subset (1k rows) of ChatGPT-4o-WritingPrompts by Gryphe A subset (2k rows) of Sonnet3.5-Charcards-Roleplay by Gryphe Synthstruct and SynthRP datasets by Epiculous A subset from Dolphin-2.9.3, including filtered version of not_samantha and a small subset of systemchat. Training time and hardware: 9 hours on 4x3090Ti Model was created by Kearm, Auri and Cahvay. Special thanks: to Cahvay for his work on investigating and reprocessing the corrupted dataset, removing the single biggest source of data poisoning. to Gryphe, Lemmy, Kalomaze, Nopm, Epiculous and…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: EVA-UNIT-01
Теги: qwen2, generated_from_trainer, conversational, en, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 35
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
