Use cases
agentictool-use
Summary
User reports Qwen3.8 Flash-Next at oQ8e on an M5 Ultra 256GB, with single-stream prefill of 3k-4k t/s and decode around 100 t/s at 130k-200k context.
Setup is oMLX with turbo quant Q8 KV cache and n-gram offload, running up to 3 concurrent sessions; multi-stream prefill holds at 1.3k-1.8k t/s per stream with aggregate decode of 120 t/s, using barely 200GB of RAM.
User also reports GLM 5.3 Flash Q4 at 1.8k t/s single-stream prefill and 500 t/s prefill across 3 streams with aggregate decode of 130-180 t/s, and DeepSeek V4.1 Flash 1017 at over 3k t/s prefill on a single stream at oQ2e, which they find too degraded for their use. DeepSeek V4 Flash 0817 prefills too slowly on oMLX. All figures are at 130k-200k context.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3.8 125B · 6B active at 4-bit by hardware
- 129.2 t/s · one run
- 105.5 t/s · 6 runs
- 57.2 t/s · 2 runs
- 55.0 t/s · one run
- 36.7 t/s · one run
- NVIDIA DGX Spark128 GB29.8 t/s · 2 runs
Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Models most run on the M5 Ultra 256GB
- 73.4 t/s · one run
- Gemma 4 31B16-bit13.9 t/s · one run
- 74.7 t/s · one run
- 17.0 t/s · one run
- 141.0 t/s · one run
- Qwen3.8 27B4-bit50.0 t/s · one run
Bars are medians on one scale, 0 to 141.0 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the M5 Ultra 256GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →