Qwen3.8 125B (6B active) Flash-Next
on 6× NVIDIA RTX 3090 · vLLM · 250,000 ctx
Oct 12, 2026
Summary
User reports Qwen3.8-Flash-Next at 57.6 t/s decode and 6574 t/s prefill on six RTX 3090s.
Setup is vLLM with a GPTQ-INT4 checkpoint and int8 KV cache, 250k context, PLE table on SSD, pipeline parallelism across six cards.
Decode falls to 29.4 t/s per user with seven concurrent sessions (206 t/s aggregate); llama.cpp-multigpu holds 30 t/s per user at five sessions. NVFP4 checkpoints are 10-12% slower than GPTQ on these cards. Load to healthy is 4-5 minutes.