> Mirrors: canonical repo is on GitHub at > fattchris/qwen-dflash2-spark > (identical content). One-stop, copy-paste reproducible stack: run Qwen3.8-27B NVFP4 with DFlash2 block speculative decoding on two NVIDIA DGX Spark (GB10) nodes over RoCE, hitting 124–135 tok/s single-stream (C1) — a +43% speedup over DSpark (MTP-style) speculative decoding at equal quality, verified end-to-end. Everything needed is in this repo: build the vLLM image, fix the checkpoints, launch the 2-node pair, and verify. No external state assumed beyond two DGX Sparks with Docker + SSH, internet to pull the base image and checkpoints. Result: OpenAI-compatible endpoint at http://NODE1:8004/v1, model name qwen3.8-27b-dflash2, 131–135 tok/s typical on the edit-heavy C1 benchmark. — K sweep on GB10: sweet spot is K=16 (2 draft blocks). K=24 drops acceptance (66%) and net throughput. K must be a multiple of blocksize 8. — ×5 repeat variance: 131.5–134.8 tok/s, accept 93.1% every run. — Quality: 6/6 greedy correct vs baseline (math/Canberra/syllogism/bugfix/code) — BF16 lmhead dequant introduces zero measurable drift. — Aggregate/concurrent throughput scales differently — C1 numbers are single-stream only,…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: cfontes
Теги: vllm, dflash2, speculative-decoding, nvfp4, qwen3, dgx-spark, gb10
Лайков: 3 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
