DeepSeek V4 Flash 284B (13B active)
on 2× NVIDIA DGX Spark · vLLM · 1,000,000 ctx
Summary
User reports DeepSeek-V4-Flash-0731 at 74.8 tok/s single-stream on real coding prompts on 2x DGX Spark.
Setup is vLLM 0.25.2 with tensor parallel 2 over RoCE, DSpark speculative decoding (spec7) and NVFP4 MLA KV cache, with KV capacity of 2.74M tokens.
The user puts the speculative speedup at 2.74x over a 27.3 tok/s bandwidth floor. Prefill runs at ~1.7-1.9k tok/s with a 42 s TTFT on a 64k prompt, a random-token benchmark gives 33.8 tok/s, and spec3 wins at 4 concurrent streams with 105.7 aggregate.