Partial GPTQ Int4 quant of Qwen/Qwen3.6-27B, produced with the verbatim recipe from Qwen’s own Qwen/Qwen3.5-27B-GPTQ-Int4 — only MLP/FFN layers are Int4, everything else stays BF16. BF16 (not quantized): lmhead, embedtokens, .attn. (Gated DeltaNet + Gated Attention), .mtp., .sharedexpert., .visual.*`. Use —quantization moewna16 (NOT —quantization gptq` — that kernel rejects the BF16 attention this recipe keeps). — 4× RTX 3090 (96 GB total), TP=4 — ≈ 7.7 GB/GPU weights, 100K ctx fits comfortably — Should also work on single A100 40GB+, single H100, or 2× RTX 4090 — Recipe from Qwen/Qwen3.5-27B-GPTQ-Int4 — GPTQModel by ModelCloud
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: raydelossantos
Теги: qwen3_5, qwen3.6, gptq, int4, quantized, agentic, hybrid, conversational
Лайков: 4 | Загрузок: 431
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.