Qwen3.8 125B (6B active) Flash-Next
M5 Ultra 96GB · oMLX · 262,144 ctx
- generation:
- 90.0 tokens/s
- prompt processing (prefill):
- 1260.0 tokens/s
- quant:
- oQ3.5e (MLX)
User reports Qwen3.8 Flash-Next at 90 tok/s generation and 1,260 tok/s prompt reading on an M5 Ultra Mac Studio with 96 GB, running as a coding agent at 262k context. Setup is oMLX 0.7.2 with the oQ3.5e (~3.5-bit) MLX quant, 60 GB model footprint, 74 GB average and 83 GB peak memory, 2.1 s median time to first token. With Splash as the engine, Flash-Next reached 140 tok/s at Q2_0, 129 tok/s at IQ3_XXS, 132 tok/s at IQ3_S and 79 tok/s at UD-IQ4_XS; Qwen3.8 27B ran at 56 tok/s (oQ8e) and 92 tok/s (UD-Q4_K_M). Two concurrent agents gave 1.52x work per hour at 93 tok/s each, three gave 1.56x at 72 tok/s each. The user notes oMLX crashed out of GPU memory with MTP enabled and multiple agents, and that Flash-Next keeps a ~29 GB lookup table on the SSD.