Qwen3.8 125B (6B active) Flash-Next
on M5 Pro 64GB · Whallm · 16,384 ctx
Summary
User reports Qwen3.8-Flash-Next (FP8) at 15.0–17.2 tok/s decode and 500–600 tok/s prefill at 16K context on an M5 Pro 64GB Mac.
Setup is Whallm 1.1.11 with MTP on by default, streaming experts from SSD; peak MLX memory stays under 19 GiB at 16K context.
A second row for Swift1.5-Qwen3.8 (MXFP4) gives 14.6–17.5 tok/s decode and 571 tok/s prefill at 18.8 GiB peak. Previous v1.1.7 numbers were ~9 tok/s decode and ~146 tok/s prefill.