A second quantization of inclusionAI/Ling-3.0-flash, alongside Ling-3.0-flash-HybridQuant-NVFP4-W4A16. Identical bit placement and identical serving contract; it differs in how the weight scales were chosen (local-Hessian error-weighted search instead of a plain MSE sweep). Both are W4A16NVFP4`, ~72 GiB, 8 shards, and both run on a single 121 GB GB10. On a 69-scenario tool-calling benchmark, n=3 per arm, one serve session per arm, prefix cache off, temperature 0: Read these honestly. Run-to-run σ on this harness is ≈2.5 points, so at n=3 the gap between the two artifacts (~2 points either mode) is about 1σ — suggestive, not established. What is larger than the noise is the configuration: this artifact with thinking on scores ~5 points above the MSE sibling with thinking off, and thinking accounts for most of that. If you are choosing between them, either is defensible; if you are choosing a serving configuration, turn thinking on. One measured difference that is not in the headline: this artifact stably passes an ambiguity-resolution scenario that an earlier local-Hessian build failed, and it produced zero safety-critical flags across all runs in both thinking modes. Unchanged…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: JasonW2025
Теги: bailing_hybrid, nvfp4, w4a16, modelopt, vllm, moe, quantized, local-hessian
Лайков: 4 | Загрузок: 854
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
