148 reports. These are individual setups, not a controlled benchmark. DeepSeek V4 Flash speeds by hardware →
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 1800 t/s prefill and 40 t/s generation on two DGX Spark units. User praises the scalability via ConnectX and the power efficiency.
NVIDIA RTX 5090 · llama.cpp · 1,000,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 1M context on an RTX 5090, with prefill at 159 t/s and decode at 13.7 t/s. Setup is a patched llama.cpp, with peak VRAM around 31 GiB. The user verified the run with a needle-in-haystack test.
NVIDIA RTX 5090 · llama.cpp · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash on an RTX 5090 with a llama.cpp fork, at 22.7 to 21.3 t/s TG and 1105 to 927 t/s PP across prompt lengths. Setup is the Q2_K quant at 1M context, with CPU offloading for the MoE layers.
GTX 1060 6GB
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash at 3.2 t/s generation with a Bartowski MXFP4 quant, running CPU only on an E5-2699v4 Xeon with 512Gb of 4-channel DDR4 2133. The GTX 1060 6Gb in the machine is too weak, so the user runs CPU only. GLM 5.2 in Q4_K_XL runs at 1.8 t/s on the same rig. The user expected more than 5 t/s and suspects the MXFP4 format.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports an optimized DeepSeek V4 Flash 8-bit affine MLX model on oMLX, with prefill improving from ~300-321 to ~533 tok/s and decode from ~7.31 to ~20-22 tok/s. Real runs at 79K-119K context show 19.2-20.7 tok/s. User asks for community review on accuracy and next optimization directions.
NVIDIA DGX Spark · vLLM · 162,816 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports REAP-pruned DeepSeek-V4-Flash served at 262k context on a single DGX Spark (Ascent GX10). Setup is vLLM. Prefill and generation throughput remain stable from 4K to 162K context. Benchmarks include pp and tg at various context lengths and concurrency levels.
M3 Ultra 512GB · llama.cpp · 10,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 8 t/s on a Mac Studio M3 Ultra 512GB with DeepSeek V4 Flash GGUF Q4_K_XL. The user also tried Q8. The user expected better performance.
User reports DeepSeek V4 Flash with DSpark speculative decoding on an HGX-H200 with 4 GPUs at TP=4 via SGLang. Setup compares the marlin and flashinfer MoE backends. DSpark is faster than EAGLE: 3.2x at bs=1 and +46% throughput at bs=24. The user reports TTFT and accept length benchmarks.
5× 2x3090 + 5060 Ti + 2x4060 Ti · llama.cpp · 16,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a fix for a DeepSeek V4 Flash crash and stall in llama.cpp. The model is used for agentic coding with pi, and the fix addresses a seq_rm issue causing the stalls and crashes. User reports the experience improved even at 10 t/s.
Unknown GPU · llama.cpp · 253,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a 29x speedup on DeepSeek V4 Flash after profiling llama.cpp. The gains come from fixing a CUDA attention kernel fallback, MoE tile padding, and using the MTP head for decode. Prefill runs at 443 t/s at 97k context and 256 t/s at 253k context, with decode at 37 t/s. The model is 284B in 2bit, 87GB, and fits in 96GB VRAM.
NVIDIA RTX Pro 6000 Blackwell · vLLM-Moet · 130,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 100-130 t/s on a single RTX 6000 Pro. Setup is vLLM-Moet at 130K context, requiring about 150GB RAM for loading.
NVIDIA RTX 4060 Ti 16GB · llama.cpp · 131,072 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a 98 GB model running on 16 GB VRAM with CPU offload, improving from 2 to 7 t/s with recent llama.cpp updates.
2× NVIDIA RTX 3080 20GB · llama.cpp · 131,072 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 17.53 t/s generation and 273.21 t/s prefill at 128k context on 2x RTX 3080 20GB with 64GB DDR5. Setup is the llama.cpp fork by fairydreaming with an 86.7 GB model, --n-cpu-moe 28 and split-mode layer.
NVIDIA RTX 3090 · llama.cpp · 8,192 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 37.47 t/s pp and 6.62 t/s tg on a dual Xeon E5-2660 v2 with an RTX 3090. Setup is the Q4_K_XL quant at 8k context. The user also tested Cydonia v4.3, a Mistral Small 3.2 finetune, at 308.92 t/s pp and 7.8 t/s tg.
NVIDIA RTX 5090 · llama.cpp · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports prefill of ~650-700 t/s and decode of ~17 t/s. Setup uses CPU offloading for MoE experts, with a loading time of 32 sec. User notes the speed is not as impressive as Qwen models but sees room for optimization.
NVIDIA RTX 3090 · llama.cpp · 196,608 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks DeepSeek V4 Flash GGUF in IQ3_XXS-AS and IQ2_S quants on an RTX 3090 with 128 GB DDR4. Setup is mainline llama.cpp b10064, which matches the fairydreaming fork's performance. User prefers IQ3_XXS-AS as the sweet spot.
M5 Max 128GB · DwarfStar · 100,000 ctx
User compares DeepSeek-V4-Flash on an M5 Max MacBook with 128 GB against 2x DGX Spark. Terminal-Bench 2.1 scores 54% (47/87) on the Mac and 52% (45/86) on the Spark pair. The Mac runs a heavily quantized GGUF at IQ2_XXS/Q2_K, about 2.45 bits per weight. The Spark pair runs the native FP8/FP4 checkpoint. The Mac wins 11 tasks outright and the Spark pair 9, with no significant difference in task-solving. The Spark pair offers higher speed at 54-58 tok/s single-stream and 253 tok/s aggregate at concurrency 8, plus a larger 262K server context window.
3× NVIDIA RTX 3090 · llama.cpp · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 37.91 t/s prompt processing and 9.07 t/s generation on two RTX 3090 24GB cards and one RTX 5090 32GB card, 80GB VRAM total, with 128GB DDR4 RAM. Setup is llama.cpp server with the unsloth Q8_K_XL GGUF, tensor splitting, and MoE layers offloaded to CPU. User says performance is better than Qwen 3.5 v2 on coding tasks and asks for speed improvement suggestions.
User reports a single B300 GPU running vLLM 0.25.0 at batch 256 with about 300 output tokens per item. Setup is tensor_parallel_size=1, block_size=256, max_num_seqs=256, enable_prefix_caching=true, moe_backend=flashinfer_trtllm, reasoning_parser=deepseek_v4, attention_config use_fp4_indexer_cache=true, and compilation_config cudagraph_mode=FULL_AND_PIECEWISE. The user reports deep_gemm_mega_moe requires expert parallel on a single GPU, and that disabling DSpark speculative decoding doubled throughput.
RTX 4090D 48GB · vLLM · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running on two RTX 4090D 48GB GPUs, 96 GB total, with a p2p patch. Setup includes Blackwell-only kernels re-implemented in Triton for sm89, and llama.cpp with the IQ2XXS quant, 262k context and a q8_0 KV cache. The user mentions a 2-3x speedup for parallel agentic workflows versus llama.cpp.