Qwen3.8 125B (6B active) Flash-Next
on M3 Ultra 512GB · llama.cpp · 32,768 ctx
Sep 28, 2026
Summary
User reports Qwen3.8-Flash-Next at 25.7 t/s aggregate decode on an M3 Ultra 512GB, with throughput collapsing as concurrent long sequences are batched.
Setup is llama.cpp (Unsloth prebuilt b10830) with Q8_0 GGUF, 32k context, flash attention on, and 8 slots. At 32k context, N=1 gives 25.7 t/s aggregate, N=2 gives 8.6 t/s, and N=4 gives 5.5 t/s. Two separate -np 1 processes reach 39.4 t/s aggregate, 4.6x faster than one process batching the same two sequences.
User also sees the same cliff with GLM-5.3-Flash, going from 40 to 10.6 t/s aggregate at N=4 between 256 and 8k context, and uses two -np 1 processes behind a load balancer as a production workaround.