A 76-layer, 31.9B-parameter model created by transplanting chunks of layers from Qwen3.6-27B into Qwen3.8-27B. Jumbo is the inverse of pruning: instead of removing layers to make a model smaller, layers are added from a sibling model to make it larger and potentially restore knowledge that was lost during continued training between versions 3.6 and 3.8. Qwen3.8-27B and Qwen3.6-27B share the same architecture and were confirmed via alignment census to share a weight basis (global median cosine similarity 0.917). Qwen3.8 is a continued-train of Qwen3.6 that traded some breadth of knowledge for improved code and agentic capabilities. Jumbo restores that breadth by transplanting the three most-changed organs from Qwen3.6 back into Qwen3.8, adding them as additional layers rather than replacing the originals. Layer chunk alignment census between Qwen3.8 (stock) and Qwen3.6 (donor), based on computing the cosine similarity of every weight tensor across all 64 layers grouped into 16 chunks (4 layers each). Selected chunks 3, 5, and 10: — Chunk 3 (cos 0.884): the single most-drifted organ, mid-stack MLP tissue where factual associations are concentrated — Chunk 5 (cos 0.888): the second…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: StargazerLabs
Теги: mlx, qwen3_5, model-surgery, stargazer, conversational, en
Лайков: 3 | Загрузок: 238
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
