Use cases
codingtool-usemultilingual
Summary
User reports DeepSeek-V4.1-Flash at 28.0 tok/s on an RTX 5070 12 GB with 64 GB DDR5-5600, using a fork of Strata with the IQ2_XS 2.25 bpw quant (266 GB) and experts streamed from an NVMe SSD.
Prompt read at 30K tokens is 519 tok/s. Other rows: code at temp 0 32.5 tok/s, short replies about 29 tok/s, and out of the box about 5 tok/s on code.
KLD is 0.183 against the same quant with all 6 experts, from skipping light experts; a more conservative preset gets 0.057 with 89.4 % GPQA and 95.7 / 89.6 % HumanEval. GPQA Diamond 89.9 % and HumanEval / HumanEval+ 94.5 / 90.2 % are single runs.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
DeepSeek V4.1 Flash 552B · 16B active at 3-bit by hardware
- 73.4 t/s · one run
- 40.0 t/s · one run
Bars are medians on one scale, 0 to 73.4 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the NVIDIA RTX 5070 →Every DeepSeek V4.1 Flash report →Estimate DeepSeek V4.1 Flash on this card →