llamaperf

Qwen3.8 125B (6B active) Flash-Next

on AMD Strix Halo 128GB · Rulith Inference

Tone: positiveAMD hardware
Oct 24, 2026
Throughput
52.7 t/s gen · 1250.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
EXL3 4.05 bpw (GGUF)
KV cache
q8_0
System RAM
128 GB

Summary

User reports Qwen3.8-Flash-Next at 52.7 t/s generation with multi-token prediction and about 1,250 t/s prompt processing on a 6K-token prompt, on a Strix Halo 128GB box running Windows. Setup is Rulith Inference 0.6.3 with turboderp's EXL3 4.05 bpw quant repacked as GGUF, q8_0 KV cache, 64 GB VRAM setting with overflow experts in system RAM, 512K context and up to 8 parallel chats. Generation is 28.8 t/s without MTP. The EXL3 build is 104 GB against 111 GB for UD-Q4_K_XL, with mean KLD 0.022 versus 0.024 and code KLD 0.0044 versus 0.0086; chat and multilingual KLD are a tie. The 39 GB n-gram embedding table is read from disk by rows and never goes to the GPU. Prompt processing is a few percent slower than the UD-IQ4_XS build.