llamaperf
Sep 27, 2026
Throughput
5.1 t/s gen
Quant
MXFP4 (GGUF)
System RAM
126 GB
VRAM reported
31 GB

Summary

User ports DeepSeek-V4.1-Flash (552B) to a llama.cpp fork and runs it on a single RTX 5090 with 31.8 GiB VRAM and 125.7 GiB RAM, streaming routed experts from NVMe through a VRAM cache and pinned host tier. Reports 5.1 tokens/s on new content and 21 tokens/s on content already resident in the cache. Setup uses MXFP4 experts as a lossless repack of the released blocks, with the engram tables (189 GiB of n-gram memory) kept on disk and read by the host. Logit correlation against the reference at 1401 tokens is 0.9967. Measured ceilings are 6.2 tokens/s with no disk miss and 21.3 tokens/s with everything resident. Per remap call at a 72 GiB host tier: 20% compute, 26% PCIe, 54% NVMe. A prefetch oracle gives +30% only at 5 tokens of look-ahead; block verification with a perfect draft gives +17-20%; an n-gram draft loses 33% on realistic content; the DSpark draft head shows 51-97% acceptance but is neutral on mixed content.