Qwen3.8 125B (6B active) Flash-Next
on AMD Strix Halo 128GB · Rulith Inference
Summary
User reports Qwen3.8-Flash-Next at 52.7 t/s generation with multi-token prediction and about 1,250 t/s prompt processing on a 6K-token prompt, on a Strix Halo 128GB box running Windows.
Setup is Rulith Inference 0.6.3 with turboderp's EXL3 4.05 bpw quant repacked as GGUF, q8_0 KV cache, 64 GB VRAM setting with overflow experts in system RAM, 512K context and up to 8 parallel chats. Generation is 28.8 t/s without MTP.
The EXL3 build is 104 GB against 111 GB for UD-Q4_K_XL, with mean KLD 0.022 versus 0.024 and code KLD 0.0044 versus 0.0086; chat and multilingual KLD are a tie. The 39 GB n-gram embedding table is read from disk by rows and never goes to the GPU. Prompt processing is a few percent slower than the UD-IQ4_XS build.