llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M4 Max 128GB · strata-mlx

Tone: positiveApple hardware
Oct 11, 2026
Throughput
6-7 t/s gen · 273-287 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q2_0 (GGUF)
System RAM
128 GB

Summary

User reports running Qwen3.8-Flash-Next, a 125B-A6B MoE, on a 128 GB M4 Max MacBook Pro with a new unofficial MLX engine called strata-mlx that reads Strata's GGUF files directly. Setup uses the model's built-in draft head for speculative decoding. The Q2_0 file is 66 GB and the IQ3_S file is 84 GB. On a Mac capped to 16 GB of memory the 66 GB file runs at 6-7 tokens/second, on a 24 GB machine at 19-23, and on a 32 GB machine at 28-47, with experts streamed from SSD. A 1,537-token prompt runs at 273-287 tokens/second in that mode. The user notes a 48 GB Mac holds the Q2_0 file whole and never touches the disk, and that speed is not guaranteed on smaller machines.