DeepSeek V4 Flash 284B (13B active)
on 2× NVIDIA RTX 4090 · llama.cpp
Sep 7, 2026
Summary
User reports DeepSeek V4 Flash 0731 at about 30 t/s prompt processing and about 3 t/s generation on CPU with an RTX 4090 and a Tesla P40.
Setup is the IQ4_XS quant at roughly 127 GB with MTP enabled, and layer-by-layer assignment because tensor splitting is not supported.
The user mentions an Unsloth 4bit K_XL quant at roughly 144 GB but could not run MTP with it due to memory constraints.