Continued from the 4.3 lineage with Gemma staying securely in her role as judge rather than rewriting directly. 600+ steps RLVR GRPO in spicy roleplaying first-turns, with Gemma 4 31b as judge paying attention to writing craft; followed by DPO including self-rewrites of about half of the particularly bad trajectories from RLVR (until Gemma rated them well in comparison, and particular issues like a godmoded POV were fixed). Ran 5 seeds at low batch size to explore and merged via Karcher mean. A mix of approaches including utilizing and not-utilizing the tags were tried in RLVR. She’s trying some interesting things with the tokens, including compound-word emotions that split the difference between word and phrase, but the ratings aren’t notably better with the tags than without them — so I wouldn’t consider her a reasoning model yet, and it’s not in the chat template. This is a merge of pre-trained language models created using mergekit. The following models were included in the merge: /home/ann/Documents/Projects/trainingmodels/outputs/bakedv44craftseed1 /home/ann/Documents/Projects/trainingmodels/outputs/bakedv44craftseed2 …
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Lambent
Теги: mistral, mergekit, merge, not-for-all-audiences, conversational, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 6
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
