llamaperf
Oct 18, 2026
Throughput
15.0 t/s gen
Quant
IQ3_S (GGUF)
System RAM
15 GB
VRAM reported
16 GB

Summary

User reports Qwen3.8-Flash-Next IQ3_S (83.6 GB) decoding at ~15 tok/s (~67 ms/token) in steady state on an RTX 5060 Ti 16 GB with 15.38 GiB RAM and a Gen3-class NVMe. Setup is a patched llama.cpp (Sabah-MoE-Accelerator) that streams the expert bank from disk into a 9.8 GB VRAM slab with unbuffered I/O; the 28.8 GB per-layer token embedding is left on CPU via -ot, peak VRAM is 15.36 of 16.31 GB, and host RAM stays flat at 4.78 GiB. Time to first token is 11.2 s for a 53-token prompt (12.68 GB cold fill at 1.11 GB/s), so a 256-token answer is 7.1 tok/s end to end and 10.4 tok/s over the decodes alone; warm-up runs 79-204 ms/token. The author notes speculative decoding and larger batches were 2.8-7.7x slower on this rig, and a second GTX 1080 was measured as no faster than disk.