llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 3× AMD Radeon Pro V620 32GB · vLLM · 262,144 ctx

AMD hardware
Oct 8, 2026
Throughput
62-68 t/s gen
Quant
AWQ-W4A16 (AWQ)
System RAM
128 GB
VRAM reported
32 GB

Summary

User reports Qwen3.8-Flash-Next at 62-68 tok/s decode on 3x Radeon PRO V620 32GB in pipeline parallel PP=3. Setup is vLLM with an AWQ-W4A16 checkpoint, int4 n-gram sidecar in host RAM, MTP k=2 speculative decoding, 262K context, 160 W power cap per card. Prefill measured at 1152 / 1705 / 1936 / 1933 tok/s for 4K / 16K / 64K / 130K prompts; a 200 W cap raises prefill to 1306 / 1943 / 2221 / 2254 tok/s with decode unchanged. Multi-stream decode without MTP reaches 157 tok/s at 4 streams and 242 at 8; with MTP 159 at 4 and 91 at 8.