148 reports. These are individual setups, not a controlled benchmark. DeepSeek V4 Flash speeds by hardware →
M3 Ultra · MLX
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 34 t/s at the start of a run, rising to 43 t/s by the end. The cached tokens were the default chat prompt and the query was 13k.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at about 50 t/s prefill and about 1 t/s decode on an M5 Air with 32 GB. Setup uses the streamed experts trick with a model of roughly 300B total parameters.
2× NVIDIA RTX 5090 · llama.cpp · 384,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Q8_K_XL against IQ3_XXS on DeepSeek-V4-Flash-0731-UD. Q8_K_XL offloads about 40 GB to system RAM. IQ3_XXS is 2.2x faster at decode and 2.4x faster at prefill, with equal correctness. Q8_K_XL has more empty-answer failures, 8/125 against 5/125. A reasoning budget of 4096 eliminates empty answers, and disabling reasoning degrades trap handling. Q8_K_XL is slower but more reliable in some tasks.
NVIDIA RTX 5090 · vLLM · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports full 1M context on a single RTX 5090 with CPU/RAM offloading, at about 800 t/s prefill and over 15 t/s decode. Setup uses DSpark speculative decoding with dynamic acceptance rates, with two routed MoE layers kept GPU-resident.
8× NVIDIA RTX 5090 · 250,000 ctx
User reports 2000+ t/s aggregate and 30-40 t/s per session across 50 concurrent sessions on 8x RTX 5090. Setup is the REAP mxfp4 image with 250k KV per session.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 (Q4) at ~29 tok/s decode on one RTX PRO 6000 96GB. Setup is Krasis v1.0.19 with INT4 experts, keeping the hottest 6,440 of 11,008 routed experts in VRAM and serving the rest from system RAM during decode. Decode after a roughly 1K prompt was 29.4, 28.2 and 28.5 tok/s for 50, 100 and 250 tokens, and 19.4 tok/s after a 62K prompt. Prefill reaches 1,328 tok/s at 23,348 tokens and 1,204 tok/s at 62,403, in internal timing-disabled results rather than HTTP round trips.
NVIDIA RTX 4090 · llama.cpp · 65,536 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports generation at 12.0-13.1 t/s and prompt processing at 212-224 t/s for 8k-32k tokens. Setup uses --n-cpu-moe 39 to offload experts to CPU, and requires 128 GB RAM. Agentic tasks take 15-26 minutes. No quality benchmarks were run.
NVIDIA RTX 5090 · vLLM · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DSpark K1/K2 at 13.78 t/s for reasoning and 17-18 t/s for content on a single RTX 5090 at full 1M context. Setup is a vLLM fork, guqiong96/Lvllmds4-x reporting vLLM 2.3.9, with phase-adaptive K1/K2 dual CUDA graphs and routed MoE experts in system RAM.
Unknown GPU · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731-REAP at ~20.5 t/s generation with an extreme 2-bit quantization. Setup is IQ2_XXS with mixed quantization (w2Q2K-AProjQ8-OutQ8), reduced from 95GB bf16 to 54GB. Output stays coherent despite the aggressive quantization.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports the MXFP4 version from Bartoswski on a DGX Spark, with prefill at 1K t/s and generation averaging 90 t/s. User calls it very efficient and the best scoring yet, and notes it lacks vision.
NVIDIA DGX Spark · llama.cpp · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 UD-IQ3_XXS at roughly 23-27 t/s on a DGX Spark GB10, stabilizing at about 24 t/s. Setup is llama.cpp with DSpark speculative decoding. The user notes it is faster than Qwen3.6-27B-NVFP4 for codegen.
M5 Pro 48GB · Ollama · 16,384 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares DeepSeek V4 Flash 9B distill against its base model Qwen3.5 9B on a MacBook Pro M5 Pro with 48 GB RAM. Both models gave the same answers on 6/8 tasks, but the distill used fewer tokens (5480 vs 8975). Throughput was near identical (44 vs 41 tok/s). The distill was slower on open-ended tasks (1467 tokens vs 867 for a buffer overflow explanation). The base model failed arithmetic due to a token cap.
M5 Pro 64GB · ds4 · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 10-17 t/s in SSD streaming mode. Setup keeps routed experts partly in RAM cache and pulls them from the GGUF on cache misses. Experts and output head are Q8_0, while router, embeddings and V4 auxiliary blocks are FP16.
NVIDIA RTX 3090 · llama.cpp · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at ~9.9 tok/s with the UD-IQ4_XS quant on a single RTX 3090 at 128K context. Setup is llama.cpp b10291 with Text Generation WebUI, Q8_0 KV cache, 44 GPU layers and --n-cpu-moe 38 keeping expert tensors in 128 GB of DDR5-5600 on a Ryzen 9 9900X. Smaller quants run faster: UD-IQ3_S at ~12.1 tok/s, UD-IQ3_XXS at ~12.5 tok/s and UD-IQ2_M at ~14.9 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731-UD-Q4_K_XL at 45 t/s and 512k context as their primary coding model, across a multi-GPU setup of an RTX 6000 Pro Blackwell 96GB, 2x RTX 5090, an RTX 4090, 3x AMD R9700, and a Strix Halo laptop. Other models in use are GLM-4.7-flash, Gemma-4-12B-it, KAT-Coder-V2.5-Dev, LFM2.5-VL-1.6B, GLM-5.2-UD, Kimi-K2.7-Code, and MiniMax-M3. The user is dissatisfied with the workflow, describing it as a good fast worker or a good thinker but not both, and notes that switching models kills the cache.
2× NVIDIA DGX Spark · vLLM · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 (304B MoE) at 82 tok/s decode and about 1400 tok/s prefill on 2x DGX Spark, each with 128GB unified memory. Setup is vLLM 0.26.1rc1 with TP=2, 1M context and FP8. User asks for advice on reducing VRAM and OS RAM usage.
2× NVIDIA RTX 4090 · llama.cpp · 65,536 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DSpark draft model performance of 1-2 t/s with DeepSeek-V4-Flash on llama-server, compared to 30-40 t/s with the MTP draft model. Setup is an RTX 4090 and an RTX 6000 Pro with 120 GB total VRAM and 32 GB RAM, running the unsloth DeepSeek-V4-Flash-0731 137 GB Q4 UD-IQ4-NL quant with the dspark-DeepSeek-V4-Flash-0731-Q8_0.gguf draft model, --spec-type draft-dspark, --spec-draft-n-max 3, --device-draft CUDA1, --tensor-split 2,3, --n-cpu-moe 13, --main-gpu 1, --split-mode layer, --ctx-size 65536, --flash-attn on, --batch-size 2048 and --ubatch-size 2048. User asks for advice on the correct setup.
Unknown GPU
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports about 3-4 t/s generation on a Xeon w7-3465 CPU with DDR5-4800 RDIMM in a 4-channel configuration. The model is DeepSeek-V4-Flash-0731 with 13B active experts and MXFP4 weights. The user notes the speed is far below theoretical memory bandwidth and discusses thread settings and memory bandwidth issues.
2× NVIDIA RTX 4090 · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at about 30 t/s prompt processing and about 3 t/s generation on CPU with an RTX 4090 and a Tesla P40. Setup is the IQ4_XS quant at roughly 127 GB with MTP enabled, and layer-by-layer assignment because tensor splitting is not supported. The user mentions an Unsloth 4bit K_XL quant at roughly 144 GB but could not run MTP with it due to memory constraints.
2× NVIDIA DGX Spark · vLLM · 1,000,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 60 t/s on a 2x DGX Spark cluster with a vLLM recipe. The setup uses a 1M context window. User praises NVFP4 support and performance improvements, and compares the cluster favorably to Strix and M5 for prompt processing.