llamaperf

DeepSeek V4 Flash 284B (13B active)

on NVIDIA DGX Spark · vLLM · 524,288 ctx

Tone: positiveNVIDIA hardware
Sep 7, 2026
Throughput
19.1 t/s gen · 1000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
2-bit (FP8)
KV cache
fp8
System RAM
128 GB

Summary

User reports DeepSeek-V4-Flash-0731 at 155.43 GiB FP8 across 48 shards on a DGX Spark (GB10, aarch64, sm_121, 121.7 GiB unified memory, 128 GiB swapfile), with prefill steady at 1000 tps and decode at about 19 tok/s. Setup is vLLM-Moet (vLLM v0.25.0 plus patch) with 2-bit MoE experts, a KV cache of 4.56M tokens at 512K context and util 0.90, and the MTP head from ycui7/DeepSeek-V4-Flash-MTP. With the MTP head decode reaches 26.6 tok/s, 48% over plain. Aggregate MTP versus plain is 25.2 versus 19.1 at conc1, 30.9 versus 26.4 at conc2, and 43.2 versus 45.6 at conc4. Per-request MTP versus plain is 25.2 versus 19.1 at conc1, 23.0 versus 17.7 at conc2, and 14.5 versus 13.9 at conc4. The 2-bit planes are 43 layers x 1.69 GiB, about 73 GiB. Boot takes about 10 min warm and 31-46 min cold.