Use cases
codingagentic
Summary
User reports about 50 tokens/s running Qwen3.8-Flash-Next on a Mac Studio M5 Ultra 256GB, with 8GB KV cache and 1200GB SSD cache.
Setup uses oMLX with the oQ4e quant and MTP speculation; cache efficiency is 96.4% and prefill time drops after a couple of rounds.
User says speed is satisfactory for a single lane, but running two or more lanes with 2-3 models loaded causes prefill and generation to race and slows things dramatically. Other models in the workflow are Laguna-S-2.1 with a DFlash FP8 draft, GLM-5.3-Flash and Qwen 3.8-35B-A3B.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3.8 125B · 6B active at 4-bit by hardware
- 129.2 t/s · one run
- 105.5 t/s · 6 runs
- 57.2 t/s · 2 runs
- 55.0 t/s · one run
- 36.7 t/s · one run
- NVIDIA DGX Spark128 GB29.8 t/s · 2 runs
Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Models most run on the M5 Ultra 256GB
- 73.4 t/s · one run
- Gemma 4 31B16-bit13.9 t/s · one run
- 74.7 t/s · one run
- 17.0 t/s · one run
- 141.0 t/s · one run
- Qwen3.8 27B4-bit50.0 t/s · one run
Bars are medians on one scale, 0 to 141.0 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the M5 Ultra 256GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →