Local model performance on your hardware

generation:

71.2 tokens/s

prompt processing (prefill):

1096.8 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

User reports Qwen3-Coder 30B-A3B at 71.2 t/s generation and 1096.8 t/s prompt processing on a Mac Studio M2 Ultra with 128GB unified memory. Setup is llama.cpp build 6060 with the UD-Q4_K_XL GGUF and flash attention enabled, run with -ngl 100 on one 60-core GPU. The figures are separate synthetic llama-bench tests: PP512 at 1096.84 +/- 5.48 t/s and TG128 at 71.23 +/- 0.08 t/s, with 16 CPU threads and a 16.45 GiB model footprint.

generation:

69.3 tokens/s

prompt processing (prefill):

1080.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

off

mtp (multi-token prediction):

off

User reports Qwen3-Coder 30B-A3B at 69.3 t/s generation and 1080.5 t/s prompt processing on a Mac Studio M2 Ultra with 128GB unified memory. Setup is llama.cpp build 6060 with the UD-Q4_K_XL GGUF and flash attention off, one GPU with -ngl 100 and 16 threads. The figures are separate synthetic llama-bench tests: PP512 and TG128, not a 512-input/128-output request. The same run measured 818.85 t/s PP8192 and 40.35 t/s TG8192.