llamaperf

Qwen3.8 125B (6B active) Flash-Next

on AMD Strix Halo 128GB · llama.cpp · 256,000 ctx

Tone: positiveAMD hardware
Sep 27, 2026
Throughput
44.9 t/s gen · 1217.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-IQ4_XS (GGUF)
System RAM
128 GB
VRAM reported
96 GB

Use cases

agenticlong-contextvisioncoding

Summary

User reports Qwen3.8-Flash-Next (125B MoE, ~6B active) at 44.9 tok/s decode after a short question and 1217 t/s prefill on a 95.6K-token prompt, on a single Ryzen AI Max+ 395 (Strix Halo, 128 GB, 96 GB GPU memory in BIOS) running Windows. Setup is a patched llama.cpp runtime (Strix Llama 0.3.0) with Unsloth's UD-IQ4_XS GGUF quant (94 GB), 256K context, MTP speculative decoding on (~2.9 tokens per pass), and an OpenAI-compatible API. Decode drops to 35.8 tok/s at 86K context; with MTP off it is ~26 tok/s after a short question and ~23 tok/s at 86K context. Three and four concurrent conversations reach 58.8 and 62.6 tok/s combined. Decode is memory-bandwidth-bound (~5.8 GB of weights read per token at ~215 GB/s).