Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User compares Claude Code, OpenCode, and Pi as harnesses for DeepSeek V4 Flash, served via vLLM at ~180 tok/s.
Diff quality was the same across all three harnesses.
Time and token usage varied widely, with Claude Code taking ~4x longer than the fastest.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash on Strix Halo at 32.0 tok/s decode with DSpark speculative draft and 25.31 tok/s autoregressive.
Setup is ROCmFP2 mixed precision at roughly 2.88 bits per parameter, with a Q4RMFP4 draft model. Prefill runs at 245 tok/s sparse.
The user compares this to LocalMaxxing entries of 18.99 tok/s for HipFire and 15.6 tok/s for DwarfStar.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks whether anyone has run DeepSeek-V4-Flash-Strix-Halo-GGUF on a Strix Halo.
The user notes it fits with ~64K context and is reported at 35 t/s.
The user is cautiously optimistic about the result.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports all experts on CPU with only 15.8 GB of 40 GB VRAM used.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks what minimum VRAM GPU is needed to run DeepSeek-V4-Flash-0731 Q4_K_XL at around 30 t/s.
The model is an MoE with 13B active parameters, and the user is considering GPUs up to 48 GB of VRAM.
No specific hardware or engine is mentioned.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks TensorSharp against llama.cpp on 4x A40 GPUs, with TensorSharp's CUDA backend reaching 836 t/s prefill at 16K context and 31.5 t/s decode short.
The model is DeepSeek-V4-Flash-0731-UD-Q8_K_XL from unsloth.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports about 30 t/s on an M5 Max with a quantized DeepSeek V4 Flash running on the DS4 engine.
The user states this is double the speed of llama.cpp.
The user asks for feedback from CUDA and ROCm users.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports llama.cpp running on 4x RTX 5060 Ti 16GB with DDR4 3200 RAM in 4-channel.
Setup uses -ub/-b at 4096.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
creative-writing
User reports DeepSeek V4 Flash 0731 at 27.2 t/s on an RTX 6000 96GB and a W7800 48GB.
Setup is llama.cpp server with the Q3_K_XL quant.
User notes it is cheaper than K3 and GLM 5.2 but very good.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports a dual RTX 3060 setup with RAM offloading, a Ryzen 7500F and 96 GB of 5600 RAM, writing a tetris game.
Prompt eval runs at 3.0 tok/s. Generation is reported as 4.5 tok/s in the summary but 3.5 tok/s in PowerShell, and the user trusts PowerShell.
LM Studio failed to offload to the second GPU, so the user used Unsloth Studio.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports llama-bench results on 3x RTX 3090 with Q3_K_XL quantization.
The model is 119.40 GiB with 284.33 B params.
Prompt processing runs at 116.04 t/s and generation at 7.71 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 UD-IQ3_S running under the llama.cpp loader in text-generation-webui.
Setup uses --n-cpu-moe 39 to offload MoE experts to system RAM, with the loader estimating roughly 136 GB total and relying on 128 GB of DDR5 at 5600 MHz.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks multiple quants on a Bosgame M5 with an RTX PRO 6000 Max-Q eGPU.
Setup is a mixed GPU configuration with CUDA0 and ROCm0, using a DSpark drafter ported from a closed PR. UD-Q8_K_XL (150.8 GiB) reaches 44.0 t/s decode and 564 t/s prefill with draft acceptance 0.535. UD-Q4_K_XL (144.4 GiB) reaches 48.4 t/s decode and 585 t/s prefill with draft acceptance 0.532. UD-Q2_K_XL (90.2 GiB) reaches 59.5 t/s decode and 1513 t/s prefill with no drafter.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User requantized only the 129 routed expert tensors of DeepSeek-V4-Flash-0731 to IQ3_XXS, with down projections at IQ3_S, leaving attention, shared experts and router at Q8_0, BF16 and F32. The result is 111.37 GiB in four shards.
Against the MXFP4 source, KLD is a mean of 0.2386 versus 0.2936 for UD-IQ3_S, top-1 is 84.65% versus 82.78%, and delta PPL is +0.536 versus +0.685, but the requant is 2.12 GiB larger and has a worse max KLD of 12.53 versus 11.13.
On 5 mixed GPUs (2x 3090, 5060 Ti, 2x 4060 Ti, 96 GiB VRAM) with expert spill to CPU, speed is 13.91/13.57/13.26 t/s at depths 0/4096/16384 versus 9.88/9.69/9.51 for the MXFP4 source, roughly 1.4x. antirez's flat Q2 is 80.76 GiB, about 98% resident, at 30.27 t/s. The user notes known DSV4 SWA/rollback stall issues in llama.cpp and used a patched build.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports DeepSeek V4 Flash 0731 at 15-16 t/s generation and 105-110 t/s prompt processing on 3x AMD MI50 32GB GPUs, 96GB of VRAM in total.
Setup is llama-server (llama.cpp) with the UD-IQ2_M quant. The 90.9GB model fits fully in VRAM.
The user tested it with a Rubik's cube HTML coding task and is impressed it runs locally.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 UD-Q8_K_XL at a steady 17.20 t/s on an RTX A6000 48GB.
Setup is an AMD EPYC 74F3 24-core with 8 channel 3200 DDR4.
Prompt processing is in the high 70 t/s, down to mid 30 t/s at 300k context. The user says 48GB VRAM is enough for the full 1mil context but prompt processing would be very slow.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports prompt processing in the high 140s t/s, dropping to the mid 80s at 60k context, with generation at ~11.9 t/s.
Setup uses the ROCm 7.2.4 backend on 1x 7900 XTX 24GB plus 3x MI60 32GB.
One MI60 failed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash 284B-A13B at 4.8 tok/s on a 24 GB M5 Pro with about 5.3 GB of memory in use.
Setup is Mference, an open-source engine. The user also runs Gemma 4 26B-A4B and Qwen 3.6 35B-A3B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at 155.43 GiB FP8 across 48 shards on a DGX Spark (GB10, aarch64, sm_121, 121.7 GiB unified memory, 128 GiB swapfile), with prefill steady at 1000 tps and decode at about 19 tok/s.
Setup is vLLM-Moet (vLLM v0.25.0 plus patch) with 2-bit MoE experts, a KV cache of 4.56M tokens at 512K context and util 0.90, and the MTP head from ycui7/DeepSeek-V4-Flash-MTP.
With the MTP head decode reaches 26.6 tok/s, 48% over plain. Aggregate MTP versus plain is 25.2 versus 19.1 at conc1, 30.9 versus 26.4 at conc2, and 43.2 versus 45.6 at conc4. Per-request MTP versus plain is 25.2 versus 19.1 at conc1, 23.0 versus 17.7 at conc2, and 14.5 versus 13.9 at conc4. The 2-bit planes are 43 layers x 1.69 GiB, about 73 GiB. Boot takes about 10 min warm and 31-46 min cold.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 700 pp/s and 18 tg/s with llama.cpp using --cpu-moe on a Threadripper 5965WX with an RTX 5090.
User notes llama.cpp does not support native FP8 cache for DFV4 Flash, so an FP16 cache is required, and speculative decoding is not working.
User suggests building a rig with an Epyc and 2x 5060 Ti.