llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 6× NVIDIA RTX 3090 · llama.cpp-multigpu · 250,000 ctx

Tone: positiveNVIDIA hardware
Oct 7, 2026
Throughput
33.7 t/s gen
Quant
UD-Q4_K_XL (GGUF)
KV cache
q8_0

Use cases

codinglong-context

Summary

User reports Qwen3.8-Flash-Next at 33.7 t/s decode on six RTX 3090s over PCIe without NVLink, up from 10.2 t/s on stock llama.cpp. Setup is a llama.cpp-multigpu fork with UD-Q4_K_XL GGUF and q8_0 KV cache, 5 slots of 262k context, layer split across the cards. Prefill rose from 263 to 2111 t/s at 250k context on the 3090s, and 744 to 7403 t/s on six rented RTX 4090s. Five-session decode went from 2.3 to 27.3 t/s per session on the 3090s. The user notes the 10.8x figure is against a 2.3 t/s baseline and should not be quoted. N-gram speculation gave 2-2.5x on code rewrites and 1.5x on code explanation but was off for the benchmarks. MTP was tried and was slower, so it is not included.