Qwen3.8 125B (6B active) Flash-Next
M5 Ultra 96GB · mlx-serve · 3,700 ctx
- prompt processing (prefill):
- 1354.0 tokens/s
- quant:
- iQ-MLX 3.3 bpw (MLX)
User reports Qwen3.8-Flash-Next on an M5 Ultra 96GB (64-core GPU) prefilling a cold 3.7k-token prompt at 1,354 tok/s. Setup is mlx-serve 26.10.1 with iQ-MLX 3.3 bpw, 131,072 context, native MTP adaptive decoding, PLD and a hot prefix cache, one request at a time. Decode is given per workload only: 227.5 tok/s on copy, 172.8 on new code and 106.4 on novel prose, with speculation on. Cold prefill rose to 4,083 tok/s at 25.2k. An agent replay ran at 125.8 tok/s text-round decode.