Tencent Hy3-preview — 102 GB on disk (down from ~557 GB BF16 source) — mixed-bit JANGTQK quantization on routed experts + 8-bit affine elsewhere. ~30 % bigger than Hy3-preview-JANGTQ (2-bit on routed experts) in exchange for a measurable quality bump on downproj sensitivity, especially on long-output generation. — Source: tencent/Hy3-preview (Hy3 architecture, 295 B total / 21 B active, BF16 native, 256 K context, 80 transformer layers + 1 MTP, 192 routed experts top-8 + 1 shared) — Quantization: mixed-bit MXTQ on routed experts: — downproj: 4-bit (4096-out, residual-stream sensitive) — gateproj: 2-bit (gated by SwiGLU) — upproj: 2-bit (multiplied with gate) — attention / shared expert / dense layer-0 / embed / lmhead / MTP matmuls: 8-bit affine — RMSNorms / router gate / expertbia — Bundle size: 102 GB on-disk across 109 shards — Runs on:** M4 Max 128 GB / M5 Max 128 GB / Mac Studio 192 GB+ jangtqruntime.safetensors sidecar (~22 KB) for Swift runtimes — covers (in=1536, bits=4) + (in=4096, bits=2) codebooks +…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: JANGQ-AI
Теги: mlx, hy_v3, jang, jangtq, jangtq-k, mixed-precision, hy3, hunyuan
Лайков: 3 | Загрузок: 81
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
