BigQwen2.5-Echo-47B-Instruct is a Qwen/Qwen2-32B-Instruct self-merge made with MergeKit. I’ve tried a more gradual approach with a distributed repetition pattern. Instead of replicating blocks of 8 or more layers, I’m replicating individual layers in these blocks: — First 8 layers: No replication — Next 8 layers: Replicate 2 layers (first one, middle one) — Next 8 layers: Replicate 4 layers (1st, 3rd, 5th, 7th) — Next 8 layers: Replicate 8 layers (all of them) — Next 8 layers: Replicate 4 layers (1st, 3rd, 5th, 7th) — Next 8 layers: Replicate 2 layers (first one, middle one) — First 8 layers: No replication I used this string to visual The additional layers are designed to increase the model’s capacity without breaking the information flow, which often creates «insane» self-merges. The following YAML configuration was used to produce this model:
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: mlabonne
Теги: qwen2, mergekit, merge, lazymergekit, conversational, zho, eng, fra
Лайков: 3 | Загрузок: 34
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
