DeepSeek V4.1 Flash

generation:

41.0 tokens/s

quant:

Q2 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek-V4.1-Flash-1017 at 41 tok/s on a single RTX 3090 with 192 GB of system RAM, averaged across multiple prompts. Setup is the user's delta-inference-engine (a Windows port of Xolotl, built on Strata) running nazeshinjite's Q2 GGUF natively on Windows, with the model streamed from a 366 GB download and a 1M context. The user says output matches DeepSeek's API next token 90.2% of the time on general text and 92.4% on code/math. The same table gives 33 tok/s with 128 GB RAM, and the needs list gives 31 tok/s at 16 GB VRAM and 17 tok/s at 12 GB VRAM. Other quants measured 38 to 42 tok/s.

generation:

62.2 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1-Flash decoding at a median 62.2 t/s on four DGX Spark (GX10) units in the original IKEA Kallax setup, with prefill of 4,255 t/s at 8K and 4,268 t/s at 250K. Decode uses speculative decoding, which swings about ±10 t/s between runs, so decode was left out of the cooling comparison. Each run is a single 8-minute sweep from 1K to 250K tokens with the model already loaded. Across six cooling setups the hottest GPU peaked at 82°C with no throttling; clocks stepped down about 3% from roughly 2525 MHz at 53°C to 2450 MHz at 82°C, and 250K prefill stayed between 4,225 and 4,353 t/s. The user's V3 open-faced air duct enclosure was the coolest setup at 65.1°C average under load.

generation:

20.1 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 20.1 tok/s on an unnamed machine, served through Maya. The run read 223 new prompt tokens in 3.9 s with 65 reused, over a 1,155-token conversation; no quant, context length or KV cache setting is given. The user's point is behavioural: the model insisted it is Claude and refused to repeat its prompt verbatim, and the user asks whether Maya diverges from upstream in the scheduler or offload path.

generation:

23.3 tokens/s

prompt processing (prefill):

1077.0 tokens/s

quant:

IQ1_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 23.3 t/s decode and 1,077 t/s prefill on a Strix Halo 128GB (Bosgame M5), fully resident in GPU memory with no SSD streaming. Setup is the user's Gufo fork with IQ1_M GGUF weights, a 112 GiB backbone in GPU memory, IQ4_NL Engram tables (103 GiB) read from NVMe, and a 4.6 GiB Q2_K/Q4_K drafter. The reported figure is plain decode; DSpark speculative decoding with prompt lookup reaches 28.0 t/s on mixed text and 65.3 t/s on repetitive text at depth 0. At 32K depth decode is 22.9 t/s with 993 t/s prefill, and at 128K 21.7 t/s with 700 t/s prefill. Eight concurrent users total 78.5 t/s, and editing a source file through prompt lookup gives 67.6 t/s. The user warns the setup needs the whole machine, leaving about 1.75 GiB of RAM with the drafter loaded.

generation:

106.0 tokens/s

quant:

EXL3 3.5bpw (EXL3)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 106 t/s per stream at concurrency 1 on four DGX Sparks, averaging 93 t/s aggregate across eight categories. Setup is TensorFold's Zig engine with an EXL3 3.5bpw pack, 1M context, a 2M-position KV pool, DSpark speculative drafts on and tensor parallel across all four GB10 nodes over RoCE v2. The sweep runs to 16 streams: per-stream decode falls to 20 t/s while aggregate rises to 250 t/s at C16. Coding reaches 139 t/s per stream at C1 and 343 t/s aggregate at C16; prose 81 and 224. Cold prefill was skipped, previously measured at 1.9-2.3K tok/s.

generation:

35.0 tokens/s

quant:

IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User asks whether 2x AMD R9700 with 128GB RAM running GLM-5.3-Flash or DeepSeek-V4.1-Flash under Strata, Maya or Xolotl would beat a single RTX 5090 with 128GB RAM. The only figure given is the 5090 setup at roughly 35 tok/s with Strata on deepseek-v4.1-flash-1017-gsq-rco IQ2_XXS. No benchmark of the dual R9700 configuration is reported.

generation:

11-18 tokens/s

quant:

IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 11-18 tok/s on an RTX 4090 with 96GB of DDR5-6800 system RAM and a 4TB PCIe 5 NVMe drive. Setup is Strata with an IQ2_XXS quant; the model runs partly from system memory. User calls the speed quite good for the model but says prefill is very bad.

generation:

10.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash running at about 10 t/s at 8K context on an RTX 5090. Setup is Strata with a Lexar 4TB NVMe and 64 GB of DDR5 system RAM. User says the speed makes the API cheaper than the electricity cost, but is glad it runs at all on the PC.

generation:

35.5 tokens/s

prompt processing (prefill):

960.5 tokens/s

quant:

UD-IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 35.5 t/s decode after a ~1.8K-token prompt on 5x NVIDIA A40, with prefill at 960.5 t/s. Setup is TensorSharp with UD-IQ2_XXS GGUF weights, tensor parallel across five GPUs, thinking disabled and greedy decoding; the 125.9 GiB Engram table stayed in host memory, so the run used both GPU and system memory. Decode was 35-36 t/s without a drafter and 51-55 t/s with the DSpark Q8_0 drafter; the same machine's llama.cpp fork reached 29.6 t/s decode and 174.6 t/s prefill. Disabling the approximate YOCO prefill optimization gave 563.9 t/s prefill and perplexity 2.4811 versus 2.4978 with it on and 2.4697 for llama.cpp. A four-request batch took 21.2-25.5 s on TensorSharp and 43.3-46.8 s on llama.cpp.

generation:

7.4 tokens/s

quant:

FP4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports the full DeepSeek-V4.1-Flash-1017 model at up to 7.4 tok/s warm (5.8 cold) on a 160-token code prompt, on an RTX 3060 12GB with 64 GB DDR4 and two NVMe drives. Setup is the user's own PROMETHEVS engine, which streams the 510 GB original FP4 weights through VRAM, RAM and both drives, computing tokens on a 2-bit copy of the experts and recomputing uncertain tokens with the original weights; DeepSeek's draft head proposes blocks of five tokens. Server runs give 5-8 tok/s on code and 4.5-5.4 on reasoning and agent prompts, against 2.14 tok/s for plain exact decoding of the full model on the same PC. Agreement with the full weights is 97.4% for the 2-bit copy alone and 98.3% with corrections over 960 audited tokens. Only tested on this one Windows machine so far, needs about 1.1 TB of NVMe, and has not been compared with DeepSeek's reference implementation.

prompt processing (prefill):

563.9 tokens/s

quant:

UD-IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks DeepSeek-V4.1-Flash-1017 on five NVIDIA A40s over PCIe, reporting 563.9 tok/s prefill at ~1.8K tokens with YOCO disabled. Setup is TensorSharp with UD-IQ2_XXS GGUF shards (~300.5 GB), tensor parallel 5, and the 125.9 GiB Engram table mapped in host memory under a 233.8 GiB host-memory limit. llama.cpp on the same files reached 174.6 tok/s at ~1.8K and 225.7 tok/s at ~7.3K, where TensorSharp gave 549.7 tok/s. Workload was real-text summarization, greedy decoding, 256 output tokens, three measured requests per prompt size.

prompt processing (prefill):

400.0 tokens/s

quant:

EXL3 2.9bpw (EXL3)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash 1017 running at up to 40 tok/s, averaging around 30 tok/s, on 2x CMP 170HX plus an RTX 5060 Ti 16GB (144GB total VRAM) with 128GB DDR4-3000 and an Intel Core i9-10980XE. Setup uses LibertAIDAI's EXL3 2.9 bpw quantization with DSpARK enabled and expert caching: 93% of routed experts run on the GPUs, the rest on the CPU, with frequently used experts migrated to VRAM in the background. Prefill is around 400 tokens/sec, which the user calls slow. The 40 tok/s is a peak; the average is about 30 tok/s.

generation:

7.4 tokens/s

quant:

2-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek-V4.1-Flash-1017 at 7.4 tok/s with a warm cache and 5.8 tok/s cold on an RTX 3060 12GB with 64 GB DDR4 and NVMe storage. Setup is a custom engine that computes tokens on a 2-bit copy of the experts, recomputes flagged tokens with the original FP4 weights, and uses a draft head proposing blocks of five tokens. The 510 GB model streams from disk; plain exact decoding on the same machine gives 2.14 tok/s. Reasoning and agent prompts run at 4.4 to 5.2 tok/s. On 960 audited tokens the 2-bit copy matches the exact path 97.4% of the time, 98.3% with corrections; a 1,101-token agent prompt scores 95.9%. The engine is not yet released.

generation:

25.5 tokens/s

prompt processing (prefill):

542.1 tokens/s

quant:

oQ4e (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash (10-17 checkpoint) running on a 256GB M5 Ultra (64-core GPU, 30-core CPU) at 20-25 tok/s generation and 500 tok/s prefill. Setup is oMLX with an oQ4e MLX build, 65% MoE expert offload, CED prefill on, SSD N-gram offload on and a 16384 prefill chunk size; the 4-bit N-gram table was swapped for the original FP8 shards. Single-request tables give 18.8-30.8 tok/s decode on an English novel and 21.2-25.9 tok/s on Python code, with peak memory up to 203.6 GB; context benchmarks show 532 tok/s prefill at 128k and 516 tok/s at 256k.

generation:

39.3 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek-V4.1-Flash at 39.3 tok/s on code and 24.6 tok/s on prose with 4x RTX 3090 (96 GB VRAM) plus an EPYC 9575F and 768 GB DDR5. Setup is llama.cpp built from PR #30817 with the UD-Q4_K_XL GGUF quant, 65536 context, flash attention on, and the DSpark Q8_0 drafter (8 GB) for speculative decoding with up to 3 draft tokens. All figures are one request at a time with thinking off, single runs. Prefill and generation on an 8K prompt were 326 / 20.6 tok/s and on a 32K prompt 363 / 25.4 tok/s. DSpark roughly doubles code speed with about 80% of guesses accepted, but only about 30% are accepted on prose, where it is sometimes a bit slower; the user keeps it on because they mostly use the model for code.

generation:

26.0 tokens/s

prompt processing (prefill):

170.0 tokens/s

quant:

2.6bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 26 t/s generation and 170 t/s prompt processing on 2x RTX 3090s. Setup is the dsx engine with a 2.6-bit quant (2.6127 bpw experts) and Engram tables, at 262,144 context; the 187.6 GB weights plus 203 GB Engram tables ran from a SATA SSD, and loading from an HDD took 90-110 minutes. The build scores KL 0.1972, 82.59% top-1 and perplexity 9.3626 against the FP8/FP4 teacher; the engine does not support images yet.

generation:

34.0 tokens/s

quant:

GSQ (GGUF)

kv:

FP4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports about 34 tok/s decode with a 763B DeepSeek-V4.1-Flash-1017 build quantized for the Strata-based Xolotl engine. Setup is Xolotl (Strata) with a GSQ mixed-precision quant (Q3_K / Q2_0 / ternary per routed-expert projection) and FP4 KV cache at 512K context; the 153 GiB artifact is spread across VRAM, pinned RAM and NVMe, pulling in only the experts each token needs. The user says throughput is memory-bandwidth-bound and should land in the same class Xolotl advertises for V4.1; the build is clean, with no abliteration, and tuned for agentic and coding workloads.

generation:

27.0 tokens/s

prompt processing (prefill):

180-320 tokens/s

quant:

GSQ Uncensored

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek-V4.1-Flash (552B) at 27 tok/s decode on a single RTX 4090, with prefill of 180-320 tok/s. Setup is the Xolotl engine (based on Strata) with a 152 GiB uncensored GSQ quant, 512K context, weights spread across 24GB VRAM, 90GB DDR4 and NVMe on a Ryzen 7 5700X. User also reports 21/21 HumanEval, 10/10 tool calls and 3/3 agentic bug-fix loops through the real server with thinking on, and says the engine also runs GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B.

generation:

56.8 tokens/s

quant:

EXL3 2.9bpw (EXL3)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek-V4.1-Flash-1017 at 56.8 tok/s decode on text and 92.2 tok/s on code with TensorFold on two DGX Spark, TP=2 over the CX7 cable. Setup is TensorFold with EXL3 2.9bpw weights and 1M context, using a speculative two-node decoder; the same pair with the vLLM kit reached 27.0 tok/s text and 39.9 tok/s code. Four concurrent code requests total 140 tok/s, and a 64K-token needle test took 35.9 s versus 80.4 s on the kit. Long-prompt prefill is 30-37% below the published numbers (1,707 vs 2,421 tok/s at 32K), and the first long prompt after a start costs about 6 s extra.

generation:

60.8 tokens/s

prompt processing (prefill):

2574.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 Flash at 60.8 t/s decode and 2,574 t/s prefill on a 1K prompt using a tuned SGLang build on 4x DGX Spark. Setup is SGLang TP4 with knapcio's recipe, DSpark speculative decoding, Engram tables streamed from NVMe, 262K context, and 100 GbE switch. The model weights are 511 GB and do not fit comfortably on 4 Sparks; memory allocation above 0.92 per Spark crashes. User compares against Patrick Moorhead's results: 62.5 t/s vs ~58 t/s single-request prose, 79.1 t/s vs ~96 t/s single-request code, and 293 t/s vs 494 t/s peak code at 16 vs 32 concurrent requests. User could not reproduce 1M context, 32 slots, or ~0.2 s first token.