Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
vision
User reports training a 40.1M connector to give DeepSeek V4 Flash basic vision, with the model loaded across four B200s in a custom SGLang stack.
Training ran on 5x H200.
User also trained a Laguna XS 2.1 version (33B total / 3B active) with a 30.7M connector, inspired by Baseten's GLM-5.2 Vision NVFP4.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at 74.8 tok/s single-stream on real coding prompts on 2x DGX Spark.
Setup is vLLM 0.25.2 with tensor parallel 2 over RoCE, DSpark speculative decoding (spec7) and NVFP4 MLA KV cache, with KV capacity of 2.74M tokens.
The user puts the speculative speedup at 2.74x over a 27.3 tok/s bandwidth floor. Prefill runs at ~1.7-1.9k tok/s with a 42 s TTFT on a 64k prompt, a random-token benchmark gives 33.8 tok/s, and spec3 wins at 4 concurrent streams with 105.7 aggregate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at 26.76 t/s sustained decode and 236 t/s prefill on a Strix Halo (Ryzen AI MAX+ 395, Radeon 8060S).
Setup is llama.cpp Vulkan v0.6.1 with DSpark speculative decoding, a bf16 draft model, and a q8_0 KV cache.
The user compares this with the DGX Spark and notes that q8_0 KV beats f16, the draft must be bf16, Vulkan beats ROCm, ubatch tops out at 2048, and a v0.6 regression is fixed in v0.6.1.
User compares quants of DeepSeek V4 Flash 0731 on 8x RTX 5090, benchmarking with wikitext-2 at 5632 context and reporting PPL and top-1 accuracy.
The quant named AD-IQ2_M is 104 GB and reaches 83.6% top-1.
The user mentions an H100 for comparison.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports testing multiple models on 4x CMP 170HX 64GB cards, including DeepSeek V4-Flash 0731 with a Q4_K_XL quant, 13B active MoE, and 1M context, plain no-spec.
The cards are cut-down A100 mining cards with 64GB each, PCIe Gen2 x4, and no NVLink.
Other models tested include gpt-oss-120B, Qwen3.6-35B-A3B, GLM-4.5-Air, and MiniMax-M2.7.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports DSpark speculative decoding with the drafter in RAM running faster than in VRAM.
Setup uses a q8_0 KV cache, which allows 768K context with minimal decode loss.
LiveCodeBench scores 28/30 (93.3%).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at about 8 t/s decode and about 30 t/s prefill on an M1 Max 64GB.
Setup is a patched llama.cpp with the IQ3-XXS quant at 104 GB and context limited to 64k.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at 21.06 t/s continuous generation and 30.59 t/s sustained accepted generation on 4x AMD Radeon Pro V620 with 128 GB total.
Setup is the unsloth IQ3_XXS quant with speculative decoding using a DSpark Q8 drafter on CPU RAM and 2 draft tokens. Prompt ingestion is 276.23 t/s at 32K and 379.01 t/s at 4K.
The user calls the model at q3_xxs "fucking stupid" and lobotomized, and notes there is not enough VRAM for the drafter, which affects TG speeds.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash (v4-flash-0731) at about 8 t/s on a single RTX 4090 with 64 GB RAM.
Setup is the Blaze engine with Unsloth's IXQ2/Q3 checkpoint, keeping heavily utilized experts in RAM with the CPU.
Throughput drops to 5 t/s on an expert miss that causes a disk read. Prefill is a work in progress.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 64 t/s baseline and 171 t/s with DSpark k=5, a 2.7x speedup.
Setup is vLLM nightly 0.27.2rc1.dev122, with 88 GB of experts offloaded to LPDDR5x via UVA.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a speculative decoding sweep with DSpark drafters, from a 20.48 t/s baseline to a peak of 28.5 t/s at n_max=3, a 1.39x gain.
Q2_K_S and Q8_0 drafters land within 1-3% of each other.
The user estimates 22-28 t/s for realistic mixed use, with prefill at 200-220 t/s at 64k context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at 10.1 t/s generation and 99.4 t/s prompt processing on 4x RTX 3060 12GB at 368k context.
Setup is llama.cpp build b10181 with a Q8_0 KV cache and a microbatch size of 2048, with the 144 GiB GGUF mostly in 128GB of system RAM.
Runs at 376832 and 360448 context gave similar speeds.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash serving on an M3 Ultra 512GB with the ds4 engine, with cold prefill improved from 392 to 475 t/s at 64k context through kernel optimizations.
Cache prewarming with max_tokens:0 yields roughly 10x speedup for chat turns, from 6-20s down to 1.6s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User asks which model to use for agentic coding.
User reports DeepSeek V4 Flash at 10-11 t/s with 1M context, and Qwen3.8 27B at 15-16 t/s with FP8 quants.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports sustained generation between 67-84 t/s, with prompt eval at 2570 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 2.04 t/s with an 8 GB expert cache and 1.23 t/s with a 32 GB expert cache, streaming from SSD.
Speculative decoding and prefetch both decreased throughput.
Codebook 2-bit quantization failed the quality gate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash (284B) at about 11 t/s on a 64GB MacBook.
The model is 165GB on disk and does not fit in RAM, so it runs via SSD streaming with a 2-bit file.
Quality matches official perplexity (6.1250 vs 6.1262). Higher quality settings give 1-2 t/s. An 8GB cache gave 2.04 t/s versus 1.23 t/s with 32GB. Speculative decoding and prefetching did not help.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 16 t/s on a Blackwell 5000 with CPU offloading.
Setup is the UD-Q3_K_XL quant at roughly 120 GB, on a Blackwell 5000 with 48 GB and an RTX 3090 with 24 GB, plus 96 GB of RAM.
Using both GPUs drops throughput to 11-12 t/s or fails to fit. User asks for help with multi-GPU configuration.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 25.8 t/s, with a 42 t/s peak, on an M2 Ultra with 60 cores and 192gb.
Setup is a custom llama.cpp fork with the model repacked losslessly to 141GiB, smaller than the public GGUFs, no KV cache quant, and an SSD KV cache with dynamic lanes, 1M context total across 8 lanes.
Prompt processing is about 250 t/s at 8k-32k. The user compares this with 16 t/s on an M3 Ultra.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-uselong-context
User reports an exl3 2.52 bpw quant of DeepSeek V4 Flash at about 60 t/s decode on VRAM-only, handling contexts above 200k.
The user compares it to a slower MXFP4 version on unified RAM and notes it performs well in opencode.
The user asks which tasks are most quantization-fragile.