DeepSeek V4 Flash 284B (13B active)
on NVIDIA RTX 5090 · ktransformers · 8,192 ctx
Use cases
codingagentictool-use
Summary
User reports DeepSeek-V4-Flash at ~28 tok/s single-stream decode on one RTX 5090 with dual Xeon Gold 6138 and 256GB DDR4-2400.
Setup is stock ktransformers + sglang with int4 AMXINT4 CPU-offloaded experts, NUMA-sharded, top-4 adaptive-k, 8192 context, fp8 KV cache. The 284B-A13B model is 142 GB at 4-bit and does not fit the GPU; experts are computed on the CPU.
Re-verified at 27.9-28.0 tok/s steady-state decode. Decode is DDR4-bandwidth-saturated. EAGLE speculative decode (27.2), more GPU experts (27.5), and 80 hyperthreads (2.7) all lost. Coding canary passes 8/8.