Calibrated NVFP4 quantization of deepreinforce-ai/Ornith-1.0-35B (Qwen3.5-35B-A3B hybrid MoE) for vLLM. 24 GB, and it decodes at the same speed as FP8 while every quality axis lands at parity or better. This is the model we run as our own daily driver — the gate below is the quant measured against the exact checkpoint serving our production traffic. Quantized and gate-verified by protoLabs on 2× RTX PRO 6000 Blackwell (sm120), vLLM 0.22.1. Same suites, same judge, same harness, thinking-on, 8192-token budget both sides. Outliers re-trialed ×3 on both sides before verdicts — one apparent regression (0.88 → 0.00 on a single task) turned out to be baseline drift: the FP8 scores 0.00 on it today too. Quant evals need paired, same-day baselines or they lie. axis FP8 (prod) NVFP4 (this) delta ———————————- ———— ———— —— claw agentic (paired-35, judged) 0.672 0.677 +0.005 function-call (54, deterministic) 93.0% 94.4% +1.4 reasoning-v2 (24, solver, x3) 0.882 ±0.021 0.889 ±0.042 +0.007 code spec-delta (8, exec, x3) 0.616 0.580 ±0.062 −0.036…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: protoLabsAI
Теги: qwen3_5_moe, nvfp4, vllm, compressed-tensors, moe, agentic, qwen3.5, blackwell
Лайков: 3 | Загрузок: 55
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
