llamaperf

DeepSeek V4 Flash 284B (13B active)

on NVIDIA RTX 5090 · ktransformers · 8,192 ctx

Tone: positiveNVIDIA hardware
Oct 6, 2026
Throughput
28.0 t/s gen
Quant
int4 (safetensors)
KV cache
fp8
System RAM
256 GB
VRAM reported
32 GB

Use cases

codingagentictool-use

Summary

User reports DeepSeek-V4-Flash at ~28 tok/s single-stream decode on one RTX 5090 with dual Xeon Gold 6138 and 256GB DDR4-2400. Setup is stock ktransformers + sglang with int4 AMXINT4 CPU-offloaded experts, NUMA-sharded, top-4 adaptive-k, 8192 context, fp8 KV cache. The 284B-A13B model is 142 GB at 4-bit and does not fit the GPU; experts are computed on the CPU. Re-verified at 27.9-28.0 tok/s steady-state decode. Decode is DDR4-bandwidth-saturated. EAGLE speculative decode (27.2), more GPU experts (27.5), and 80 hyperthreads (2.7) all lost. Coding canary passes 8/8.