llamaperf
Sep 25, 2026
Throughput
20.0 t/s gen
Quant
UD-Q4_K_XL (GGUF)
System RAM
128 GB

Summary

User reports Qwen3.8 Flash-Next at approximately 20 tok/s single-stream decode over 256 output tokens on an RTX 5090 with 128 GB system RAM. Setup is llama.cpp with GGUF UD-Q4_K_XL, with the model partly offloaded to system RAM. With 4 parallel requests of roughly 4K prompt plus 256 output, aggregate end-to-end throughput is 13.8 tok/s; with roughly 8K prompt plus 256 output it is 8.81 tok/s. The user notes these aggregate figures are a different measurement from the single-request rate and should not be read as single-stream speed.