llamaperf

Qwen3.8 125B (6B active) Flash-Next

on AMD Strix Halo 128GB · vLLM · 5,600 ctx

Tone: positiveAMD hardware
Oct 31, 2026
Throughput
43.0 t/s gen · 1539.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
AWQ W4A16 (AWQ)
System RAM
128 GB

Summary

User reports Qwen3.8-Flash-Next at 43 tok/s decode and 1,539 tok/s prefill on a Strix Halo 128GB at a 5.6k prompt. Setup is a vLLM fork with the libstrix plugin, AWQ W4A16 weights, 16k prefill chunks and MTP 2, single stream. The 102 GB n-gram table is streamed from disk and prefetched into the page cache. At 26.9k and 100.5k prompts decode is 43 and 40 tok/s with prefill 1,617 and 1,572 tok/s. Before this work the same setup did 684-806 tok/s prefill and about 18 tok/s decode. KLD against stock is 0.008-0.019 with 97-98% top-1 agreement, perplexity within 0.6%, and GSM8K 93 vs 92.