Local model performance on your hardware

generation:

34.5 tokens/s

quant:

6-bit (MLX)

User reports Qwen3-Coder-Next 80B-A3B at 34.5 tok/s decode on an M5 Pro 48GB Mac, running a full agentic coding task in 44s. Setup is the Global Agent engine (MLX-based) with 6-bit weights and expert streaming from SSD, speed mode, peak memory 31.5GB. The post also benchmarks Qwen3.8-Flash-Next 229B-A7B (19.2 tok/s), Qwen3-235B-A22B (9.19 tok/s) and MiniMax-M3 426B-A24B (4.62 tok/s) on the same machine, plus 16GB and 32GB memory-capped runs; the 16GB Qwen3-Coder-Next run reached 41.2 tok/s. Exact mode figures are lower (17.7, 7.88, 2.62, 2.20 tok/s).

generation:

36.8 tokens/s

prompt processing (prefill):

545.8 tokens/s

quant:

UD-Q6_K_XL (GGUF)

kv:

f16

User reports Qwen3-Coder-Next UD-Q6_K_XL at 36.75 t/s on tg32 on an AMD Strix Halo box, with prefill at 545.77 t/s on pp2048. Setup is llama-server with -ngl 999, flash attention on, -c 262144, f16 KV cache and one slot, measured with llama-benchy 0.4.1 in API latency mode. The tg32 rate is 36.75 ± 0.05 t/s with a peak of 37.94 ± 0.06 t/s. A 27B 8-bit XL model was too slow to be workable.