Qwen3.8 125B (6B active) Flash-Next
on 4× AMD Radeon AI PRO R9700 32GB · radiance
Summary
User reports Qwen3.8-Flash-Next at 236 tok/s single-stream aggregate on four R9700 32 GB cards at TP4, with 1,192 tok/s at 16 concurrent streams.
Setup is radiance 1.11.0 with the fp8-iq4r-moe container (113.6 GiB, 4-bit experts, int8 trunk, bf16 protected experts), fp8 KV cache, MTP head depth 3, 262144 max model length, all weights in VRAM and the 47.7 GiB n-gram table pinned in host RAM.
Decode step is 11.27 ms against a 11.2 ms reference; prefill measured 8.4K t/s at 4.4K and 70K prompt tokens, and single stream with thinking on reached 197 tok/s over 4,096 generated tokens. Concurrency rows are thinking off, 512 tokens each, wall-clock over the whole batch; a 2x TP2 attempt lost.