Qwen3.8 125B (6B active) Flash-Next
on M3 Max 48GB · oMLX · 65,536 ctx
Use cases
codingmath
Summary
User reports Qwen3.8-Flash-Next at ~38 tok/s decode on a 48 GB M3 Max, with prefill at 210-380 tok/s.
Setup is the user's oMLX fork with a custom Metal kernel for ternary routed expert gate/up (Bonsai T5 packing, ~1.875 bpw), Q3 expert down projections and the 53 GB n-gram table left on SSD as Q8 and mmapped per token, about 35.6 GiB resident. The oMLX memory guard is on the 'safe' profile with a 48 GB limit, one model loaded and one request at a time.
A 65,536-token prompt plus 256 output completed at 30.7 tok/s with a 42.3 GiB physical peak, and 96K trips the prefill guard. Against Unsloth UD-Q4_K_XL, which does not fit in 48 GB, MMLU is 83.0% vs 89.7% and KLD vs Q8_0 is 0.49 vs 0.036.