DeepSeek V4 Flash 284B (13B active)
on NVIDIA RTX 3090 · llama.cpp · 128,000 ctx
Summary
User reports DeepSeek-V4-Flash-0731 at ~9.9 tok/s with the UD-IQ4_XS quant on a single RTX 3090 at 128K context.
Setup is llama.cpp b10291 with Text Generation WebUI, Q8_0 KV cache, 44 GPU layers and --n-cpu-moe 38 keeping expert tensors in 128 GB of DDR5-5600 on a Ryzen 9 9900X.
Smaller quants run faster: UD-IQ3_S at ~12.1 tok/s, UD-IQ3_XXS at ~12.5 tok/s and UD-IQ2_M at ~14.9 tok/s.