Oct 8, 2026
Summary
User reports Qwen3.8-Flash-Next at ~64 tokens/s single-stream decode and ~1,200 tokens/s prefill on four AMD Radeon PRO V620 cards (32 GB each, gfx1030) in one box.
Setup is a vLLM fork with AWQ W4A16 backbone plus a quantised int4 n-gram table, tensor parallel across all four cards, MTP off and cards uncapped; the n-gram table is CPU-offloaded.
At the production 140 W setting decode is ~59.5 tokens/s, and with MTP=3 it ranges ~60-72 tokens/s. Model files total ~105 GB of downloads.