Qwen3.8 125B (6B active) Flash-Next
on 6× NVIDIA RTX 3090 · llama.cpp-multigpu · 250,000 ctx
Use cases
codinglong-context
Summary
User reports Qwen3.8-Flash-Next at 33.7 t/s decode on six RTX 3090s over PCIe without NVLink, up from 10.2 t/s on stock llama.cpp.
Setup is a llama.cpp-multigpu fork with UD-Q4_K_XL GGUF and q8_0 KV cache, 5 slots of 262k context, layer split across the cards.
Prefill rose from 263 to 2111 t/s at 250k context on the 3090s, and 744 to 7403 t/s on six rented RTX 4090s. Five-session decode went from 2.3 to 27.3 t/s per session on the 3090s. The user notes the 10.8x figure is against a 2.3 t/s baseline and should not be quoted. N-gram speculation gave 2-2.5x on code rewrites and 1.5x on code explanation but was off for the benchmarks. MTP was tried and was slower, so it is not included.