Qwen3.8 125B (6B active) Flash-Next
on M5 Ultra 256GB · oMLX · 130,000 ctx
Summary
User reports an MLX conversion of NVIDIA's Qwen3.8-Flash-Next NVFP4-QAD running on an M5 Ultra 256GB, reaching about 100 token/s with speculative decoding.
The model has 75GB resident memory and was tested with 22 parallel sessions at 130k+ context, pushing the 256GB Mac to about 220GB RAM used.
MTP remained untrained, giving about 75% acceptance rates, and the model scored within the noise floor of the 8-bit Qwen3.8 Flash Next for similarity.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3.8 125B · 6B active at 4-bit by hardware
- 129.2 t/s · one run
- 105.5 t/s · 6 runs
- 57.2 t/s · 2 runs
- 55.0 t/s · one run
- 36.7 t/s · one run
- NVIDIA DGX Spark128 GB29.8 t/s · 2 runs
Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Models most run on the M5 Ultra 256GB
- 73.4 t/s · one run
- Gemma 4 31B16-bit13.9 t/s · one run
- 74.7 t/s · one run
- 17.0 t/s · one run
- 141.0 t/s · one run
- Qwen3.8 27B4-bit50.0 t/s · one run
Bars are medians on one scale, 0 to 141.0 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the M5 Ultra 256GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →