DeepSeek V4.1 Flash 552B (16B active)
on 4× NVIDIA RTX 3090 · llama.cpp · 65,536 ctx
Use cases
coding
Summary
User reports DeepSeek-V4.1-Flash at 39.3 tok/s on code and 24.6 tok/s on prose with 4x RTX 3090 (96 GB VRAM) plus an EPYC 9575F and 768 GB DDR5.
Setup is llama.cpp built from PR #30817 with the UD-Q4_K_XL GGUF quant, 65536 context, flash attention on, and the DSpark Q8_0 drafter (8 GB) for speculative decoding with up to 3 draft tokens. All figures are one request at a time with thinking off, single runs.
Prefill and generation on an 8K prompt were 326 / 20.6 tok/s and on a 32K prompt 363 / 25.4 tok/s. DSpark roughly doubles code speed with about 80% of guesses accepted, but only about 30% are accepted on prose, where it is sometimes a bit slower; the user keeps it on because they mostly use the model for code.