llamaperf

DeepSeek V4 Flash 284B (13B active)

on 2× NVIDIA DGX Spark · vLLM · 1,000,000 ctx

Tone: positiveNVIDIA hardware
Sep 7, 2026
Throughput
74.8 t/s gen · 1800.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4
KV cache
NVFP4

Summary

User reports DeepSeek-V4-Flash-0731 at 74.8 tok/s single-stream on real coding prompts on 2x DGX Spark. Setup is vLLM 0.25.2 with tensor parallel 2 over RoCE, DSpark speculative decoding (spec7) and NVFP4 MLA KV cache, with KV capacity of 2.74M tokens. The user puts the speculative speedup at 2.74x over a 27.3 tok/s bandwidth floor. Prefill runs at ~1.7-1.9k tok/s with a 42 s TTFT on a 64k prompt, a random-token benchmark gives 33.8 tok/s, and spec3 wins at 4 concurrent streams with 105.7 aggregate.