llamaperf

DeepSeek V4 Flash 284B (13B active)

on NVIDIA L40S · Strata · 510,000 ctx

Tone: positiveNVIDIA hardware
Oct 26, 2026
Throughput
73.7 t/s gen · 1974.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ1_M (GGUF)
System RAM
1024 GB
VRAM reported
48 GB

Summary

User reports DeepSeek V4 Flash at 1974.7 t/s prefill and 73.7 t/s decode at 510K input on a single L40S 48GB, up from 1147.2 t/s prefill and 26.1 t/s decode with the official engine's auto settings. Setup is a modified Strata v0.2.6 build with IQ1_M weights, CPU experts kept on NUMA0 of dual EPYC 9754 with about 1TB system RAM, 16384-token prefill chunks, a 384MiB reconstructed-weight cache, and DSpark/MTP speculative decoding at a 0.15 confidence threshold. Decode peaks at 76.9 t/s at 32K input; the official engine with DSpark forced on reached 1278.4 t/s prefill and 22.9 t/s decode at 510K.