Oct 8, 2026
Summary
User reports Qwen3.8-Flash-Next at 60 to 100 t/s decode and over 3000 t/s prefill on a rig with 8x Radeon Pro V620 (256 GB VRAM).
Setup is a custom vLLM fork with W4A16 routed experts and BF16 elsewhere, pipeline parallel 4, MTP enabled with 3 token drafting, concurrency 1.
User says llama.cpp managed only around 350-450 t/s prefill with the same model, and that the new setup is about 800% faster.