DeepSeek V4 Flash reports

generation:

120.0 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports speculative decoding with the draft model dspark-DeepSeek-V4-Flash-0731-Q8_0.gguf, peaking at 153.9 t/s on code, averaging 120 t/s on code and 80 t/s on non-code. Setup is a 5090 on x8, an RTX PRO 6000 on x8 and an RTX PRO 5000 on x4 over PCIE Gen5, with a tensor split of 22,8,13. Spec draft n-max is 5 and p-min is 0.6.

generation:

30.0 tokens/s

prompt processing (prefill):

230.0 tokens/s

quant:

Q8_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User compares DeepSeek-V4-Flash and Qwen3.8-Flash-Next. Setup is DeepSeek-V4-Flash lossless at 162 GB in Q8_K_XL on 4x RTX 3090 with 192 GB DDR4, reaching about 30 t/s and about 230 pp. User notes Qwen3.8-Flash-Next has no lossless Q8 available and would need 360 GB, and criticizes benchmark misleadingness due to the size difference.

generation:

180.0 tokens/s

prompt processing (prefill):

5000.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports prefill speeds of ~5000 pp/tg for DeepSeek V4 Flash on 4x RTX 4090 with a vLLM fork. User also mentions Qwen3.8 Flash next with ~7500pp/135tg (MTP). User compares to llama.cpp and ik_llama, noting vLLM is much faster for prefill.

User reports DeepSeek V4 Flash on 2x RTX PRO 6000 with vLLM reaches Sonnet quality and faster wall-clock time than Sonnet or Opus via API. User mentions Qwen 3.6 models as anchors.

generation:

10.9 tokens/s

prompt processing (prefill):

132.5 tokens/s

quant:

UD-Q2_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4 Flash at 10.9 t/s generation and 132.5 t/s prompt on an RTX 4090 with 128 GB of DDR5 5600 MT/s. Setup is llama.cpp llama-server with unsloth's UD-Q2_K_XL quant, 64000 context, flash attention off and 8 threads pinned to the P-cores of an Intel Core Ultra 7 270k, on a 24GB GPU plus system RAM. Pinning the P-cores took generation from 6.8 tok/s to 11 tok/s, and the IQ4_NL quant ran at 50.7 t/s prompt and 8.1 t/s generation with only about 10k context. The user finds it smarter than Qwen 3.6 27B Q4_K_XL but still prefers Qwen 3.6 27B for agentic work because it runs so much faster.

generation:

11.5 tokens/s

prompt processing (prefill):

300.0 tokens/s

quant:

IQ2_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a local model reaching 100% on an SQL benchmark, running on dual RTX 3080 20GB cards with 96GB RAM and a 9800X3D. Setup is a custom IQ2_M GGUF with grafted tensors from antirez GGUF, running on a modified ds4 engine. Mainline llama.cpp gives 100pp and 8tg.

quant:

Q3 (GGUF)

User reports running a frontier model on a home PC with 24 GB of VRAM. The user notes the run is slow.

generation:

28.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports decode speeds at various context depths: 28 t/s at the start, 23.5 t/s at 45k, and 18 t/s at 192k. The run was maintained with 8k token output. Prefill performance is mentioned but no numbers are given.