DeepSeek V4.1 Flash 552B (16B active)
on Quadro P2200 · ik_llama.cpp
Use cases
coding
Summary
User reports DeepSeek V4.1 Flash (552B MoE) at 9.95 t/s decode and 101 t/s prefill on a used Dell R820 with 4x Xeon E5-4640 and a Quadro P2200 5GB, with the model held in 768GB of DDR3 system RAM.
Setup is a patched ik_llama.cpp ported to AVX1, with dense weights in Q6_K, NUMA thread pinning per CPU, and the GPU handling half the dense decode work and most of the prompt work.
Stock llama.cpp first gave 0.43 t/s decode and 4.2 t/s prefill. With MTP on, code workloads reach 12-13 t/s.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
DeepSeek V4.1 Flash 552B · 16B active at 3-bit by hardware
- 73.4 t/s · one run
- 40.0 t/s · one run
Bars are medians on one scale, 0 to 73.4 tokens/s. Press ▶ to watch text arrive at a row's speed.