M1 Max 32GB reports

generation:

12-15 tokens/s

quant:

IQ2_K (MLX)

kv:

KVarN K4/V4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next (125B) at 12-15 decode tokens per second on a 2021 M1 Max with 32 GB of unified memory. Setup is MoEspresso 3 with IQ2_K routed projections and KVarN K4/V4 KV cache, using Metal execution through the mlx-iqk library. Cache-Prior expert bias is enabled by default; prefill is unbiased. The user notes the speed holds only when no other memory-hungry applications are running. The user also reports an overall score of 84.3% on a reproducible 48-question benchmark, ahead of Claude Opus 4.8 xhigh (80.3%) and other hosted solutions.

generation:

15.8 tokens/s

prompt processing (prefill):

81.8 tokens/s

quant:

4bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User compares MLX and llama.cpp on Qwen3.8-27B. MLX reaches 15.81 t/s generation and 81.76 t/s prompt processing. llama.cpp reaches 9.69 t/s generation and 99.61 t/s prompt processing. The llama.cpp run uses the UD-Q4_K_M quant.