llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 4× AMD Radeon Pro V620 32GB · vLLM · 131,072 ctx

Tone: mixedAMD hardware
Sep 18, 2026
Throughput
70.0 t/s gen · 1300.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
W4A16 (AutoRound)
MTP (Multi-Token Prediction)
on
System RAM
256 GB
VRAM reported
128 GB

Use cases

codingcreative-writing

Summary

User reports Qwen3.8 Flash-Next at 70 t/s generation and 1.3k t/s prefill on 4x V620 (128GB total VRAM) at 128k+ context. Setup is a vLLM fork with AutoRound W4A16 quantization and MTP-2 speculative decoding, on an EPYC 7452 with 256GB DDR4. Prose generation runs at 60 t/s. Power draw is 700-900W during prefill and 500-600W during decode. User was initially disappointed with Qwen3.8 27B speeds but is satisfied with Flash-Next.