llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Ultra 256GB · oMLX

Tone: positiveApple hardware
Nov 14, 2026
Throughput
100.0 t/s gen · 3000-4000 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
oQ8e
KV cache
Q8
System RAM
256 GB

Use cases

agentictool-use

Summary

User reports Qwen3.8 Flash-Next at oQ8e on an M5 Ultra 256GB, with single-stream prefill of 3k-4k t/s and decode around 100 t/s at 130k-200k context. Setup is oMLX with turbo quant Q8 KV cache and n-gram offload, running up to 3 concurrent sessions; multi-stream prefill holds at 1.3k-1.8k t/s per stream with aggregate decode of 120 t/s, using barely 200GB of RAM. User also reports GLM 5.3 Flash Q4 at 1.8k t/s single-stream prefill and 500 t/s prefill across 3 streams with aggregate decode of 130-180 t/s, and DeepSeek V4.1 Flash 1017 at over 3k t/s prefill on a single stream at oQ2e, which they find too degraded for their use. DeepSeek V4 Flash 0817 prefills too slowly on oMLX. All figures are at 130k-200k context.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the M5 Ultra 256GB

Bars are medians on one scale, 0 to 141.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the M5 Ultra 256GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →