Qwen3.8 125B (6B active) Flash-Next
on M3 Ultra 256GB · Rapid-MLX
Summary
User reports Qwen3.8 Flash-Next at 25.67 t/s on an M3 Ultra 256GB.
Setup is Rapid-MLX 0.13.2 with mixed 4-bit affine MLX quantization and a 2048-token prefill chunk, at roughly 102.8-103.8GB active unified allocation.
Decode falls to 24.27 / 23.40 / 21.72 t/s at the 2048 / 8192 / 32768 prompt targets, with median TTFT of 2.262 / 9.236 / 44.659 seconds. Each row is the median of three batch-size-one runs with cold prefix cache and 256 requested decode tokens.