Derived directly from Nanbeige/Nanbeige4.3-33B, revision c1093c501cc6460427c0ac634db7223d0450c0ee. The original 14 weight shards were verified against the upstream LFS SHA256 values. No third-party modified model is used as the source for this release. These checks establish provenance; they do not prove the absence of undesirable model behavior. On eight RTX PRO 2000 Blackwell GPUs with vLLM 0.32.0 and TP8, three English prose prompts produced pooled end-to-end output throughput of 64.78 tok/s with plain NVFP4 and 53.66 tok/s with optional DFlash2 (0.83x across these three requests). These are capped response prefixes with one concurrent request, temperature 0 and thinking off, one response per case. See the per-case table under English prose throughput. For English prose on this tested setup, use plain NVFP4 decoding as the default: DFlash2 was slower in all three cases. The English coding example is a separate workload: 63.82 to 120.20 end-to-end output tok/s (1.88x) in one 512-token example. DFlash2’s benefit depends on the prompt and concurrency. The six reasoning sanity checks use Japanese prompts and are reported separately; the English prose measurements evaluate speed,…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: sakamakismile
Теги: llama, nvfp4, autoround, heretic, conversational, text-generation-inference, endpoints_compatible, 8-bit
Лайков: 3 | Загрузок: 365
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.