llamaperf

DeepSeek V4 Flash 284B (13B active)

on NVIDIA DGX Spark · vLLM · 256,000 ctx

Tone: positiveNVIDIA hardware
Oct 8, 2026
Throughput
9.8 t/s gen · 500.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
2-bit expert planes
System RAM
128 GB

Use cases

codingagentictool-uselong-context

Summary

User reports DeepSeek-V4-Flash-0731 at ~9.8 tok/s decode on a single DGX Spark (GB10, 128GB unified memory). Setup is a vLLM-based stack (vLLM-Moet adapted for unified memory and sm_121) with 2-bit expert planes and an FP4 tier, serving 256K context with a KV pool of ~660K tokens. Decode uses the built-in DSpark speculative head at k=2; the recommended k=7 was slower at 5.1 tok/s. Prefill is ~500 tok/s chunked at 2048, with a 28K-token prompt giving ~53 s TTFT. First boot took ~35 min for one-time expert quantization, later boots ~10 min via persistent cache. Tool calling and per-request thinking toggle work out of the box.