llamaperf

DeepSeek V4.1 Flash 552B (16B active)

on NVIDIA RTX 5070 · Strata

Tone: positiveNVIDIA hardware
Nov 21, 2026
Throughput
28.0 t/s gen · 519.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
IQ2_XS (GGUF)
System RAM
64 GB
VRAM reported
12 GB

Use cases

codingtool-usemultilingual

Summary

User reports DeepSeek-V4.1-Flash at 28.0 tok/s on an RTX 5070 12 GB with 64 GB DDR5-5600, using a fork of Strata with the IQ2_XS 2.25 bpw quant (266 GB) and experts streamed from an NVMe SSD. Prompt read at 30K tokens is 519 tok/s. Other rows: code at temp 0 32.5 tok/s, short replies about 29 tok/s, and out of the box about 5 tok/s on code. KLD is 0.183 against the same quant with all 6 experts, from skipping light experts; a more conservative preset gets 0.057 with 89.4 % GPQA and 95.7 / 89.6 % HumanEval. GPQA Diamond 89.9 % and HumanEval / HumanEval+ 94.5 / 90.2 % are single runs.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

DeepSeek V4.1 Flash 552B · 16B active at 3-bit by hardware

Bars are medians on one scale, 0 to 73.4 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 5070 →Every DeepSeek V4.1 Flash report →Estimate DeepSeek V4.1 Flash on this card →