llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Max 128GB · MTPLX · 262,144 ctx

Tone: positiveApple hardware
Oct 6, 2026
Throughput
73.5 t/s gen · 1453.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
4-bit (MLX)
System RAM
128 GB

Use cases

codinglong-contextvision

Summary

User reports Qwen3.8 Flash-Next at 73.5 tok/s with MTP speculative decoding on an M5 Max 128GB, versus 43.8 tok/s plain autoregressive. Setup is MTPLX with a dynamic 4-bit quant (8-bit attention) MLX pack, 262,144-token context, MTP depth ceiling 3, single stream through mtplx serve. Prompt processing on a 4,061-token prompt is 1,453 tok/s on MTPLX 2.12.0 (786 tok/s on 2.11.3); a 65,502-token prompt reads at 1,094 tok/s with 63.5 tok/s decode. The pack downloads at 115.1 GB including a 32 GB n-gram table streamed from SSD, peaks at 87 GiB while serving, and is recommended for Macs with 128 GB or more.