llamaperf
Sep 7, 2026
Throughput
25.0 t/s gen
Quant
Q4_K_XL (GGUF)
System RAM
64 GB
VRAM reported
32 GB

Summary

User reports Qwen 3.8 Flash-Next at about 25 t/s on an RTX 5090 with 64 GB of RAM. Setup is the Q4_K_XL quant, with the n-gram table unloaded to an SSD to fit in memory. User advises that the n-gram table can be offloaded to an SSD with minimal speed cost.