DeepSeek V4 Flash 284B · 13B active at 4-bit generates a median of 8.3 tokens per second on the NVIDIA RTX 3090, the hardware it is most run on, from 2 runs that kept part of the model in system RAM (community reports, as of 25 Oct 2026).
As of 25 Oct 2026, DeepSeek V4 Flash 284B · 13B active at 4-bit on the hardware it is most run on, with the median of runs that kept part of the model in system RAM (one device, one request):
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at 79 tok/s greedy decode on four Radeon AI PRO R9700 cards.
Setup is the affinity engine with a prebuilt 2.875 bpw quant, all 11,008 experts resident in VRAM with no host pool allocated, on a Threadripper PRO 9955WX with 128 GB RAM.
The same box on two cards gave 36 tok/s with 45% of experts in VRAM and the rest streaming from a 51 GB pinned host pool. Sampled decode at temp 1.0 was 63 tok/s and about 90 tok/s on a chat answer with thinking on. Load takes about a minute, 42 seconds of it pushing 96 GB of experts onto the cards.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 19.9 t/s generation and 270.1 t/s prompt processing on two AMD Strix Halo nodes with 128 GB RAM each, linked by OdinLink USB4/TB5 RDMA.
Setup is the Antirez Q4_K experts build with F16 HC/compressor/indexer and Q8 attention/shared/output, tensor parallel across two Radeon 8060S iGPUs with a balanced 50/50 expert split.
The run used a fixed 2,048-token prompt followed by 300 generated tokens, with ordinary decode and no speculative decoding; the source gives 19.89 t/s and 270.10 t/s exactly.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash IQ3_XXS at 18.6 t/s generation and 140.3 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is llama.cpp build 434ddbb with Vulkan on a single Radeon 8060S, 120GB GTT region, and a 97.05GiB tensor size.
The figures are separate PP512 and TG128 synthetic tests, not a 512-input/128-output client request; the source gives 18.56 +/- 0.01 t/s and 140.30 +/- 4.35 t/s over 3 repeats.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 17.3 t/s generation and 254.3 t/s prompt processing on an AMD Ryzen AI MAX+ 395 with Radeon 8060S and 128 GB unified memory.
Setup is llama.cpp (Nathanw1014/strix-halo-llamacpp v0.7.5, Vulkan backend) with UD-IQ3_XXS, K/V q8_0, flash attention, and no speculative decoding, on one Strix Halo GPU.
The headline figures are the 7,680-input point of a three-point curve; at 30,720 input the rates are 247.7/17.4 and at 122,879 input 194.8/15.8 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 28.5 t/s generation and 241.1 t/s prompt processing on an AMD Ryzen AI MAX+ 395 with Radeon 8060S graphics and 128 GB unified memory.
Setup is llama.cpp (Nathanw1014/strix-halo-llamacpp v0.7.5, Vulkan backend) with UD-IQ3_XXS, K/V q8_0, flash attention and DSpark fixed draft length 4, at 7,680 input tokens.
The same run measured 32.28 t/s at 30,720 input tokens and 28.11 t/s at 122,879 input tokens; the headline figures are the 7,680-input point, not a maximum across contexts.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 21.2 t/s generation and 321.5 t/s prompt processing on one AMD Ryzen AI MAX+ 395 with Radeon 8060S (gfx1151) and 128 GB unified memory.
Setup is Lucebox PR 729 on ROCm/HIP 7.2.4 with ROCmFPX-MIX-STRIX, K/V q4_0, fused decode and verify with F16 KV, expert top-k 6, sparse prefill, and a 131,072-token allocation; speculation is disabled with no drafter loaded.
The headline figures are the 7,680-input point of a native PP/TG curve, not a maximum across contexts; the same curve gives 318.0/18.4 at 30,720 input and 287.4/12.9 at 122,879 input.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 41.9 t/s generation and 283.3 t/s prompt processing on an AMD Ryzen AI MAX+ 395 with Radeon 8060S (gfx1151) and 128 GB unified memory, one Strix Halo GPU.
Setup is Lucebox PR 729 on ROCm/HIP 7.2.4 with ROCmFPX-MIX-STRIX, K/V q4_0, fused decode and verify with F16 KV, expert top-k 6, sparse prefill, and DSpark adaptive q2-q5 speculation, at a 131,072-token allocation.
The headline figures are the 7,680-input point of a native PP/TG curve, not a maximum across contexts; the same curve gives 306.6/41.4 at 30,720 input and 283.9/38.0 at 122,879 input.
2× NVIDIA RTX PRO 5000 Blackwell (one 72GB card and one 48GB card) · llama.cpp
generation:
19.4 tokens/s
prompt processing (prefill):
52.2 tokens/s
quant:
MXFP4 (GGUF)
kv:
F16
flash attention:
on
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at 19.4 t/s generation and 52.2 t/s prompt processing on one RTX PRO 5000 Blackwell 72 GB plus one RTX PRO 5000 Blackwell 48 GB.
Setup is llama.cpp CUDA b11063 with MXFP4, f16 K/V cache, flash attention, layer split 4.5/2 and -ncmoe 11, leaving about 110.6 GiB of weights on the GPUs.
The 52.2 t/s is the PP512 cell and the 19.4 t/s is the separate TG128 cell, one repetition each; the 145.63 GiB GGUF exceeds the 118.39 GiB combined VRAM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash prefill at 454.6 t/s on an AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151) with 128 GB unified memory.
Setup is the DS4 Halo ROCm prefill fork with IQ2_XXS + Q2_K and selected Q8 tensors, 4,352-token context allocation and 4,096-token prefill chunks.
The figure is a warmed prepared-prefill mean of 454.59 t/s from two samples, 454.52 and 454.65, on a complete position-zero 4,096-token request with generation disabled.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 284B-A13B at 17.8 t/s generation and 238 t/s prompt processing on an AMD Gorgon Halo 192GB machine.
Setup is llama.cpp with UD-IQ3_S, Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory.
The 238 t/s and 17.8 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output conversation; the same source reports roughly 32k-depth prompt processing at 106 t/s and generation at 13.2 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-contextsummarization
User reports DeepSeek-V4-Flash at 15.4 tok/s decode and 142.30 tok/s prefill at 2,048 tokens of context on a Ryzen AI Max+ PRO 395 (Strix Halo) with 128 GB unified memory.
Setup is the ds4 engine with an IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 GGUF (81 GB), 32,768-token context, ROCm 7.1.1; decode falls 12% from 2K to 32K while prefill stays flat.
A second model, Qwen3.8-Flash-Next UD-IQ4_XS (88 GB) under Strata 0.1.40, decodes at 39.0 to 40.9 tok/s and prefills a 6,786-token prompt at 343 to 367 tok/s, dropping to 30.4 tok/s each with two requests in parallel. The same engine measured 8.7 tok/s while another model's worker held 41 GiB of VRAM and 23 GB of RAM, against 39.0 tok/s on an exclusive box.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvision
User reports DeepSeek V4 Flash Vision-Exp at 31.7 tok/s decode on a Mac Studio M4 Max 128GB.
Setup is ds4 (DwarfStar) with IQ2_XXS/Q2_K GGUF, DSpark speculative decoding at confidence 0.6, and a 32768 context length.
Thinking-on pagoda run reached 29.84 tok/s over 26,415 completion tokens; 1M context loaded at 97.15 GiB planned total with 32.0 tok/s decode. User notes DSpark adds roughly 10-20% over no speculation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at 36.1 t/s generation and 498.3 t/s prefill on a MacBook Pro M5 Max 128GB 40 GPU.
Setup is oMLX 0.5.7 with a 2.44 bpw mixed MLX quantization (92.8 GB on disk) at 1k prompt, single request, 128 generated tokens, speculative decoding off.
Generation holds 33.8 t/s at 4k and 31.5 t/s at 32k, with peak memory rising 1.03 GB across an 8x context jump; prefill at 32k takes 97 seconds to first token. Continuous batching reaches 83.2 t/s at batch 8 (2.30x).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports DeepSeek-V4-Flash-0731 at 542 tok/s prefill on 64k context on 2x Mac Studio 512GB.
Setup is MLX with mxfp4 routed experts and affine-8 gs64 elsewhere (155 GB), tensor-parallel over Thunderbolt RDMA, 2 nodes.
Decode is ~26-29 tok/s flat at 64-256k context and ~31 tok/s flat at 6k generated tokens, up from ~10 and 17.7 decaying on the stock baseline. Prefill reaches 510 tok/s at 128k and 366 tok/s at 256k, with 128k prefill transient memory cut from +99 GB to ~24 GB per node. Recall is 5/5 at all depths to 256k. User notes 512k+ cold prefill still exceeds the per-node transient budget beside a resident second model.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentictool-uselong-context
User reports DeepSeek-V4-Flash q2-imatrix at 45.9 t/s generation and 362.3 t/s prefill on an RTX Pro 6000 Blackwell 96GB at 2048 context.
Setup is ds4 with q2-imatrix GGUF, fp16 dequant cache disabled and MTP off, fully resident at ~91 GB peak. The 80.76 GB model only fits with DS4_CUDA_Q8_F16_CACHE_MB=0; MTP adds 3.5 GB and pushes over 96 GB.
Generation beats posted M5 Max and DGX Spark GB10 q2 numbers; prefill is about equal to Spark. At 28672 context generation drops to 40.6 t/s. The user notes the card is tight for this model and heavier ds4_test sessions OOM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticlong-context
User reports DeepSeek-V4-Flash-0731 (284B MoE, 13B active) at 28.91 t/s generation and 261.7 t/s prompt processing on a single AMD Ryzen AI MAX+ 395 with 128GB unified memory.
Setup is a llama.cpp fork (Nathanw1014/llama.cpp) with Vulkan/RADV backend, UD-IQ2_XXS quant (2.06 bpw), DSpark draft model at n=3, 16384 context, 120 GiB GTT limit.
Best result is IQ2_XXS + DSpark n=3 at 28.91 t/s, reaching 90% of Lucebox's 32.0 t/s with 6 experts instead of 4. The fork beats upstream by 49% on prompt processing at 8192 context. IQ2 with draft model sits at the memory limit and is a measurement setup, not a production state.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 484.62 t/s prefill and 16.28 t/s decode on a Strix Halo 128GB (Ryzen AI MAX+ 395, Radeon 8060S, gfx1151).
Setup is Gufo with an IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 mixed GGUF, one session, speculative decoding disabled, thinking off, temperature 0, seed 1, nominal pp2048/tg128 (2,035 prompt tokens, 128 generated).
One warmed sample; no standard deviation reported. The maintainer notes target-model quality parity is unestablished and build-sensitive differences are under investigation. Power, KV-cache type, Flash Attention and peak memory were not reported.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash reaching a peak of 40 t/s generation on 8x Radeon AI PRO R9700 under llama.cpp, but the run froze after 200-300 tokens.
Setup is llama.cpp on 8x R9700 with 256GB system RAM and roughly 400GB total VRAM; the user notes llama.cpp on many GPUs was very slow, possibly due to ROCm.
The user also runs MiniMax M2.7 (MXFP4_16, iq4_nl) on 8x R9700 and Qwen3-27b-BF16 on 4x7900xtx under vLLM, and gave up on a REAP GLM-5.1 50% build due to vLLM errors.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash at 13.61 t/s generation and 338.41 t/s prompt eval on 2 x RTX PRO 4000 with 48 GiB VRAM and 160 GiB system RAM.
Setup is llama.cpp with an MXFP4/Q8_0 GGUF (about 146 GiB) at 32768 context, using --fit on and --no-mmap.
A llama-batched-bench sweep from 8K to 1M context shows prompt throughput falling from 448.57 t/s to 65.97 t/s and generation from 19.20 t/s to 6.56 t/s; without hc-ops the same sweep was slower (322.50 t/s prompt, 14.62 t/s generation at 8K). KLD against the base logits is 0.422277 mean for the Q2 quant and 0.115089 for the Q4 quant.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731-MXFP4-MLX decode at 14.89 tok/s on an M3 Ultra 512GB, about 2.8x below the 41.7 tok/s the model card reports for this checkpoint at 23K context with native DSpark MTP.
Setup is oMLX 0.6.4 with MXFP4 MLX weights, native MTP enabled with 3 draft tokens, 23,716-token prompt, no KV cache quant stated. MTP acceptance is 86-88% and prefill is 445.08 tok/s, but decode does not follow.
User notes the same ~14-15 tok/s class on 0.6.3rc3 and 0.6.4, no swap or memory pressure, and that 0.5.4rc2 (the reference build) was not A/B tested.
Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.
The hardware DeepSeek V4 Flash 284B · 13B active at 4-bit is most run on: NVIDIA RTX 3090 (8.3 t/s median, 2 runs); M5 32GB (1.0 t/s median, one run); NVIDIA DGX Spark (20.6 t/s median, one run). Medians of runs that kept part of the model in system RAM, as of 25 Oct 2026.
Which inference engines do people use for DeepSeek V4 Flash?
Reports of DeepSeek V4 Flash name llama.cpp (58 reports), vLLM (26 reports), ds4 (9 reports), MLX (5 reports), as of 25 Oct 2026.
With part of the model in system RAM: DeepSeek V4 Flash 284B · 13B active at 4-bit
Runs on one device that kept some of the model in system RAM or streamed it from an SSD. The CPU and its memory set the pace as much as the card, so these figures are kept apart from runs with the whole model on the device and vary more from machine to machine.
Speed by hardware on DeepSeek V4 Flash 284B · 13B active at 4-bit
The size and quant level the most devices have plain runs of: one device, one request, no speculative decoding, and the whole model in the device's memory. Every speed in this table is the same model.