llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× AMD Radeon AI PRO R9700 32GB · R9V · 65,536 ctx

Tone: positiveAMD hardware
Sep 24, 2026
Throughput
74.7 t/s gen · 3600.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ4_XS (GGUF)
MTP (Multi-Token Prediction)
on
System RAM
128 GB

Summary

User reports Qwen3.8 Flash Next Uncensored at 74.7 t/s decode and 3,600 t/s prefill at 64k context on 2x Radeon AI PRO R9700 with 128GB DDR5. Setup is the R9V engine with an IQ4_XS GGUF plus MTP and a KVA projector; concurrent throughput is 70.3 t/s at 8 streams, TTFT P50 338 ms, update P99 51.5 ms. KVA projectors give 1.45-1.85x prefill speedup (1700 to 2500-3150 t/s) for +2.6% to +8% perplexity depending on start layer; a multi-layer variant reaches 1.55x at +2% perplexity. RAM use floats around 50GB for expert storage.