> Experimental research artifact. This is a 50% expert-pruned model, not an official Qwen release. The pruning calibration is small and coding/agent biased. Test the Q8 artifact on your workload before trusting a lower quant. This repository contains Q8 through one-bit-class GGUF variants of Qwen/Qwen3.6-35B-A3B. The routed MoE was reduced from 256 to 128 experts per layer while retaining the original top-k 8 active experts per token. Pruning reduces stored parameters and memory, but it does not reduce the eight active expert matmuls per token. GGUF has no conventional Q7K family, so the ladder follows the supported practical families: Q8, Q6, Q5, Q4, Q3, Q2/IQ2, then IQ1. IQ1M and IQ1S` are both included because they are distinct one-bit-class formats. Requantization warning: every file below Q80 was produced from the pruned Q80 artifact with —allow-requantize, not from F16/BF16. The quantizer itself warns that this can reduce quality more than direct quantization from 16/32-bit weights. A remapped 128-expert importance matrix was used for every lower quant. This checkpoint is not a direct REAP implementation. REAP motivated the experiment, but the Qwen selection used low…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: xero0000
Теги: gguf, qwen3.6, mixture-of-experts, expert-pruning, experimental, llama.cpp, endpoints_compatible, imatrix
Лайков: 3 | Загрузок: 446
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
