A deep-dive research suite exploring layer-wise parameter surgery, Cascaded Spherical Linear Interpolation (SLERP), and DARE-TIES sign election applied to the hybrid linear/full-attention architecture of Qwen 3.5 4B. This repository hosts three quantized GGUF variants (Q4KM) engineered to test the limits of parameter retention, coding density, multi-step deduction, and refusal elimination in 4B small language models (SLMs). 1. Substrate Base: rodrigomt/Qwen3.5-4B-Uncensored-Aggressive (Abliterated base with censorship vectors neutralized). 2. Cognitive Distillation: clzoro/Qwen3.5-4B-Claude-distill (Dense reasoning traces and chain-of-thought structures). 3. Engineering Specialist: microsoft/FrogNano-4B-2609 (Compact systems programming, tooling, and algorithmic structure). Because Qwen 3.5 incorporates hybrid linear and full attention layers, a uniform merge degrades performance. Frobenius norm weight-variance mapping was used to apply a tiered parameter schedule: Layers 0–7 (Grammar & Syntactic Root): Anchored primarily to the Uncensored Base ($t approx 0.20 — 0.25$) to maintain foundational token embeddings and linguistic stability. Layers 8–22 (Cognitive Hotspot): Heavily…
Модальности:
Генерация текста
Области применения:
Генерация кода Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: Sayansantra
Теги: gguf, merge, mergekit, slerp, dare-ties, qwen3_5, uncensored, reasoning
Лайков: 3 | Загрузок: 271
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
