Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingmultilinguallong-context
User reports DeepSeek V4.1 Flash Q2 (~340 GiB MoE) running on a single AMD Strix Halo 128GB box with 124 GiB unified memory, reaching a highest completion-token rate of 7.12 tok/s including reasoning.
Setup is the kyuz0 ds4 ROCm runtime with SSD expert streaming (--ssd-streaming-cache-experts), ROCm 7.2.4, 8k context, and a host-side patch. Small coding and Czech tests completed at request-wall medians of 29.2 s (Python), 26.4 s (bugfix), and 20.4 s (Czech); 64k prompts pass but 128k never produced a final answer in a 905 s bounded calibration.
User stopped the experiment, calling it not a practical winner. A Three.js coding test produced 20,575 output tokens over ~83 minutes at ~3.9-4.2 end-to-end tok/s with zero accepted games. The same box measured Qwen3.8 Flash-Next at 44.66 tok/s short, 41.37 @ ~32k, 40.08 @ ~64k, and 38.03 @ ~126k with MTP decode, and GLM-5.3-Flash at 14.63 tok/s on ROCm versus 8.57 tok/s on Vulkan.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4.1 Flash at 41.1 tok/s decode on 8× NVIDIA A40 with layer-split execution.
Setup is the TensorSharp engine with Q2_K weights; prefill measured 532–541 tok/s with GPU-resident Engram, and cold model loading from MooseFS took 144–155 seconds.
Decode improved from 35.6 to 41.1 tok/s after reducing decode graph partitions from 565–577 to 8. These are project benchmarks, not a comparison against llama.cpp.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentictool-usecoding
User reports DeepSeek V4.1 Flash at 300 t/s prefill and 16 t/s decode on an M3 Ultra 512GB, running the Q2 quant on the DS4 engine.
Tool calls were functional but the model made questionable choices, ignoring the code execution tool and searching the local drive for a remote file before finding the right path. The 18k-token Hermes prompt was insufficient to steer it.
At around 110k tokens of context the decode rate only just began to dip below 16 t/s, which the user notes is a key advantage of DS4 over llama.cpp. The GPU ran at about 95% throughout. The user compares it unfavorably to GLM 5.3 Flash, which puts out roughly 21-19 t/s, and is hesitant to try Q4.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcoding
User reports DeepSeek V4.1 Flash at 31.3 t/s decode at 8k context on an M3 Ultra 512GB, up from 16.6 t/s upstream.
Setup is a forked ds4 engine with Q4 GGUF weights, DSpark multi-token speculative decoding, and a disk KV cache. At 300k context decode is 28.3 t/s, prefill on a 62k prompt is 813 t/s, and TTFT on a 23k system prompt is 31.1 s.
With DSpark on, code generation reaches 40.5 t/s and agent answer phases 41.3 t/s. A 91-minute agent turn decoded 101k tokens with 4.6M tokens prefilled at 99.5% cache hit and 56 tool calls. Output is byte-identical to upstream under greedy decode.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4.1 Flash at 1.8-2.2 t/s on an M2 Max 64GB.
Setup is the Whallm app with MLX, loading parts of the model from SSD, around 33 GiB peak MLX memory, with 1K-16K input tokens.
The app also adds built-in throughput benchmarks and an OpenAI-compatible API.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4.1 Flash at 40.3–40.7 t/s single-stream decode on 8× NVIDIA A40 GPUs with Q2_K GGUF weights.
Setup is TensorSharp with layer splitting, F16 KV cache, and 65K context; prefill is 533–539 t/s. Engram tables are ~60 GiB and kept on GPU.
With Q4_K_M, single-stream decode is 31.0–32.5 t/s and prefill 452–492 t/s; aggregate decode is 39.3 t/s at 2 concurrent, 48.9 t/s at 4, and 48.5 t/s at 8. Layer split beats routed-MoE tensor parallelism on this machine, ~32 t/s vs ~22 t/s, because the cards have no NVLink.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4.1 Flash at 32.5 t/s single-stream decode on 8× NVIDIA A40 with Q4_K_M GGUF.
Setup is TensorSharp with layer split, F16 KV cache, and 65,536-token configured context. Prefill measurements use approximately 4.9K-token prompts.
Q2_K achieved 40.72 t/s decode and 539 t/s prefill. Q4_K_M aggregate decode throughput was 39.3 t/s at 2 concurrent requests, 48.9 t/s at 4, and 48.5 t/s at 8. Layer split beat routed-MoE tensor parallelism (31–32.5 vs. 21.4–22 t/s).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports DeepSeek V4.1 Flash on an M3 Ultra 512GB with oQ4e and Engram in RAM, reaching 439.4 t/s prefill and 19.7 t/s generation at 64K context with MTP off, and 435.6 t/s prefill and 39.7 t/s generation with MTP on.
Setup is oMLX 0.7.0.dev2 with Python code prompts, temperature 1.0, 128 generated tokens, and no prefix cache. One measured run per configuration after warm-up.
At 4K context, prefill was 458.0 t/s with MTP off and 452.2 t/s with MTP on, while generation was 20.2 t/s and 32.1 t/s respectively. At 16K, prefill was 459.1 t/s and 454.8 t/s, with generation at 20.0 t/s and 34.7 t/s. At 32K, prefill was 452.2 t/s and 447.7 t/s, with generation at 19.8 t/s and 31.5 t/s. The user also notes experimental MoE Expert SSD Offload support for DeepSeek V4.1, Qwen3.8-Flash-Next, Gemma 4 MoE, and OLMoE, and an M5 prefill speedup from 615.2 to 826.7 t/s at 32K on an M5 Max with Qwen3.8-27B using INT8-activation kernels.
User asks about Artificial Analysis benchmark scores rather than local hardware performance.
The models discussed are DeepSeek V4.1-Flash (552B, AA score 40), Qwen 3.8-Flash-Next (180B, AA score 40), GPT-6 Astra (Max) and Fable 5.1.
DeepSeek V4.1-Flash reportedly matches or exceeds Qwen 3.8-Flash-Next on most evaluations and beats GPT-6 Astra (Max) on AutomationBench-AA (agentic SaaS workflows), but falls behind on AA-Omniscience Non-Hallucination Rate. AA spent $13,129 to independently test Fable 5.1.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentic
User reports about 30 t/s prompt processing and about 6 t/s generation on a Xeon, CPU only.
The n-gram table is offloaded on 50% of threads.
The user describes the code as sloppily vibecoded by Opus 5.0 and unreviewed, and the goal is overnight and over-week agentic jobs.