Qwen3.8 125B (6B active) Flash-Next
on AMD Strix Halo 128GB · llama.cpp · 256,000 ctx
Use cases
agenticlong-contextvisioncoding
Summary
User reports Qwen3.8-Flash-Next (125B MoE, ~6B active) at 44.9 tok/s decode after a short question and 1217 t/s prefill on a 95.6K-token prompt, on a single Ryzen AI Max+ 395 (Strix Halo, 128 GB, 96 GB GPU memory in BIOS) running Windows.
Setup is a patched llama.cpp runtime (Strix Llama 0.3.0) with Unsloth's UD-IQ4_XS GGUF quant (94 GB), 256K context, MTP speculative decoding on (~2.9 tokens per pass), and an OpenAI-compatible API.
Decode drops to 35.8 tok/s at 86K context; with MTP off it is ~26 tok/s after a short question and ~23 tok/s at 86K context. Three and four concurrent conversations reach 58.8 and 62.6 tok/s combined. Decode is memory-bandwidth-bound (~5.8 GB of weights read per token at ~215 GB/s).