llamaperf

DeepSeek V4 Flash 285B (13B active) Vision-Exp

on 2× NVIDIA DGX Spark · vLLM · 1,048,576 ctx

NVIDIA hardware
Oct 8, 2026
Throughput
62-83 t/s gen
Quant
NVFP4 (NVFP4)
KV cache
nvfp4_ds_mla

Use cases

visioncodinglong-contextagentic

Summary

User reports a two-node DGX Spark recipe for DeepSeek V4 Flash Vision-Exp under vLLM with TP=2 and DSpark speculative decoding, giving ~62-83 decode tok/s on a single chat through 128K context. Setup is vLLM with NVFP4 weights and nvfp4_ds_mla KV cache, 17.04 GiB KV pool holding 2,331,430 tokens at 0.835 GPU utilization, 1M-token ceiling and 6 concurrent sequences. Six short chats reach ~160-190 tok/s aggregate (~30-37 per stream); a 2026-08-14 run measured 162 aggregate at c=6 and 75 tok/s at 128K single-stream, while a 2026-09-02 run measured 56 tok/s at 256 tokens and 139 aggregate at c=6. A three-Spark TP=3 lane gives ~200 tok/s aggregate at 16 streams with 4-13% faster decode per stream but 4-13% slower prefill to 64K and ~22% slower at 128K-256K. The 315/205 tok/s figures need the Stage-C plus Keys path.