A reproducible deployment and benchmark package for the official deepseek-ai/DeepSeek-V4-Flash-0731 checkpoint on two NVIDIA GB10-class nodes using vLLM tensor parallelism, DSpark speculative decoding, NVFP4 DS-MLA KV cache, FlashInfer B12X MoE, RoCE, and ordinary CUDA Graph capture through batch shape 8. > Independent community recipe. This repository is not an official DeepSeek release and contains no model weights. Download the official checkpoint from DeepSeek-AI. DSpark accelerates target-verified generation; it does not change the target model’s knowledge or semantics. The current safer production profile runs two concurrent sequences with GPUMEMORYUTILIZATION=0.75, 4,096 batched tokens, Graph-8, and the UCX registration-cache safeguards documented in docs/STABILITY.md. In a simultaneous two-request proof, both requests stayed scheduled (running=2, waiting=0) and each generated 1,005 exact bounded tokens in 21.05 seconds: At the end of that proof the nodes retained approximately 20 GiB and 18 GiB MemAvailable, memory PSI avg10 was 0.00, and the service had zero restarts. This is the recommended default profile; it does not imply two simultaneous full-1M-token requests…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: bertholomus
Теги: vllm, deepseek, dspark, speculative-decoding, dgx-spark, gb10, tensor-parallel, long-context
Лайков: 4 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
