Qwen3.8 125B (6B active) Flash-Next
on M5 Pro 48GB · Strata · 262,144 ctx
Use cases
long-context
Summary
User reports Qwen3.8 Flash-Next at about 32 tok/s decode with IQ3_S on an M5 Pro 48GB MacBook Pro.
Setup is Strata 0.2.13 with the Metal engine, DFlash2 speculative decoding (spec 3), 262,144 context and int8 KV cache, with the resident expert arena pinned to 22 GB.
IQ2_XS averaged about 36 tok/s and IQ3_XXS about 34 tok/s on the same prompt set. Clamping the arena to 10 GB to simulate a 32 GB Mac gave about 27 tok/s with IQ3_XXS, and an M2 Max 32GB with an 11 GB arena at 128K context ran IQ3_XXS at about 21 tok/s. The user notes the expert pool holds only 40-50% of experts at 256K context with the rest streamed from SSD, costing roughly 15-25% speed.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3.8 125B · 6B active at 4-bit by hardware
- 129.2 t/s · one run
- 105.5 t/s · 6 runs
- 57.2 t/s · 2 runs
- 55.0 t/s · one run
- 36.7 t/s · one run
- NVIDIA DGX Spark128 GB29.8 t/s · 2 runs
Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Models most run on the M5 Pro 48GB
- DeepSeek V4 Flash 9B4-bit44.0 t/s · one run
Bars are medians on one scale, 0 to 44.0 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the M5 Pro 48GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →