llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Ultra 96GB · mlx-serve · 3,700 ctx

Apple hardware
Oct 8, 2026
Throughput
1354.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
iQ-MLX 3.3 bpw (MLX)
System RAM
96 GB

Use cases

codingagenticlong-context

Summary

User reports Qwen3.8-Flash-Next on an M5 Ultra 96GB (64-core GPU) prefilling a cold 3.7k-token prompt at 1,354 tok/s. Setup is mlx-serve 26.10.1 with iQ-MLX 3.3 bpw, 131,072 context, native MTP adaptive decoding, PLD and a hot prefix cache, one request at a time. Decode is given per workload only: 227.5 tok/s on copy, 172.8 on new code and 106.4 on novel prose, with speculation on. Cold prefill rose to 4,083 tok/s at 25.2k. An agent replay ran at 125.8 tok/s text-round decode.