- generation:
- 77.2 tokens/s
- prompt processing (prefill):
- 780.2 tokens/s
- quant:
- Q4_K_M (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3-30B-A3B-Instruct-2507 at 77.2 t/s generation and 780.2 t/s prompt processing on an M4 Pro 64GB.
Setup is llama.cpp commit 6fdd0ac89 with Q4_K_M, Metal backend, -ngl 99, and 10 CPU threads on one physical GPU.
The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single chat request.
- generation:
- 73.6 tokens/s
- prompt processing (prefill):
- 834.2 tokens/s
- quant:
- MXFP4 (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports gpt-oss-20b at 73.6 t/s generation and 834.2 t/s prompt processing on an M4 Pro with 64 GB unified memory.
Setup is llama.cpp commit 6fdd0ac89 with MXFP4 weights, Metal backend, -ngl 99, 10 CPU threads, and f16 K/V cache on one GPU.
The figures are native llama-bench PP512 and TG128 means over five repetitions from cleared state at depth 0, not a single chat request.
- generation:
- 42.4 tokens/s
- prompt processing (prefill):
- 786.5 tokens/s
- quant:
- Q8_0 (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 42.4 t/s generation and 786.5 t/s prompt processing on an M4 Pro 64GB.
Setup is llama.cpp with Q8_0 weights, Metal backend, -ngl 99 and 10 CPU threads, on one physical GPU.
The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.
- generation:
- 29.1 tokens/s
- prompt processing (prefill):
- 284.5 tokens/s
- quant:
- Q4_K_M (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 12B at 29.1 t/s generation and 284.5 t/s prompt processing on an M4 Pro 64GB.
Setup is llama.cpp with Q4_K_M, Metal backend, -ngl 99, and 10 CPU threads.
The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.
- generation:
- 54.2 tokens/s
- prompt processing (prefill):
- 730.9 tokens/s
- quant:
- UD-Q4_K_M (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 54.2 t/s generation and 730.9 t/s prompt processing on an M4 Pro 64GB.
Setup is llama.cpp with UD-Q4_K_M, Metal backend, -ngl 99, 10 CPU threads, and one physical GPU.
The figures are separate native tests: PP512 and TG128 at depth 0, five repetitions each, not a single 512-input/128-output request.
- generation:
- 12.6 tokens/s
- prompt processing (prefill):
- 123.9 tokens/s
- quant:
- Q4_K_M (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B at 12.6 t/s generation and 123.9 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory.
Setup is llama.cpp commit 6fdd0ac89 with the Q4_K_M GGUF, Metal backend, -ngl 99, 10 CPU threads, and f16 K/V cache.
The figures are native llama-bench means over five repetitions: PP512 at 123.908607 t/s and TG128 at 12.621998 t/s, each starting from cleared state at depth 0, with load and warmup excluded.
- generation:
- 48.6 tokens/s
- prompt processing (prefill):
- 747.1 tokens/s
- quant:
- UD-Q4_K_M (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 48.6 t/s generation and 747.1 t/s prompt processing on an M4 Pro 64GB.
Setup is llama.cpp with UD-Q4_K_M, K/V f16, Flash Attention auto, -ngl 99 and 10 CPU threads on one Metal GPU.
The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single chat request.
Cyber-Tiel-Coder 35B-A3B / AX normal wiring / python-lru 219 input, 256 output / MTP
M4 Pro 64GB · AX Engine
- generation:
- 116.3 tokens/s
- quant:
- AXQ MXFP4 mixed (MLX)
- mtp (multi-token prediction):
- on
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Cyber-Tiel-Coder-35B-A3B at 116.3 t/s on an Apple M4 Pro with 64 GiB unified memory.
Setup is the AX Engine native MLX MXFP4 trunk with a protected BF16 MTP sidecar, expert streaming off, one request at a time with cold KV and no prefix reuse.
The figure is the median decode rate over six measured requests on the python-lru workload at 219 input and 256 output tokens, with a range of 116.19 to 116.35 t/s; completion including TTFT was 94.37 t/s.
- generation:
- 78.3 tokens/s
- prompt processing (prefill):
- 1236.0 tokens/s
- quant:
- MLX affine mixed 3.40 bpw (MLX)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 78.3 t/s generation and 1236 t/s prompt processing on a single M4 Pro Mac mini with 64 GB unified memory.
Setup is MLX 0.32.0 with MLX-LM 0.31.3, an affine mixed 3.40 bpw text-only conversion, batch 1, greedy temperature 0, thinking enabled.
Source medians are 1,236.00 PP and 78.32 TG over 2,048 prompt tokens and 235 generated tokens; ranges are 1,233.55-1,236.60 PP and 78.22-78.52 TG. The author reports quality losses versus its bf16 comparator, including MMLU-Pro-400 -9 percentage points.
- generation:
- 109.1 tokens/s
- quant:
- AXQ-MXFP4 (MLX)
- mtp (multi-token prediction):
- on
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Tiel-Coder 35B-A3B at 109.1 t/s median native decode on a single Apple Mac mini M4 Pro with 64 GiB unified memory.
Setup is AX Engine with the AXQ-MXFP4 MLX export, expert streaming off, and speculative decoding at maximum draft depth three.
The figure is native decode only, measured over six requests on a python-lru prompt with 194 input tokens and 256 total output tokens; separate median completion throughput including TTFT is 90.46 t/s.
- generation:
- 20.5 tokens/s
- prompt processing (prefill):
- 126.9 tokens/s
- quant:
- PQ2_0 (GGUF)
- kv:
- F16
- flash attention:
- on
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a sourced third-party community benchmark of Bonsai 2 27B PQ2_0 on one Apple Mac mini M4 Pro with 64 GB unified memory. Prompt processing pp512 measured 126.89 +/- 0.02 tokens/second and token generation tg128 measured 20.53 +/- 0.01 tokens/second.
Setup is llama.cpp PrismML fork (tag prism-b10709-9a9394a) with PQ2_0 GGUF and default F16 KV cache, 10 threads, -ngl 99, flash attention enabled, Metal/BLAS backend. The two figures are separate synthetic llama-bench tests with three repetitions each, not a paired chat request.
Caveats: Spotlight indexing consumed approximately 1.5-2 CPU cores throughout the runs; an unrelated 664 MB embedding model reappeared between runs; no swap pressure was observed. The 6.70 GiB GGUF size is a file size, not measured resident or peak RAM. Peak RAM, configured maximum context, exact weight revision, binary hash, and exact measurement date are unknown.
- generation:
- 49.3 tokens/s
- quant:
- 8-bit (MLX)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5-35B-A3B at 49.3 t/s on an M4 Pro Mac mini with 64GB unified memory.
Setup is mlx_lm with 8-bit weights, single request, 512-token context and 512 output tokens.
Figure is transcribed from SiliconBench (arXiv:2609.19169v1, Table 4), a single-node baseline, not a multi-node result; the 512-token context is the tested workload, not a maximum. Quantization scheme, runtime version, GPU core count, KV-cache precision and sampling settings are not established for this row.
- generation:
- 72.2 tokens/s
- quant:
- 4bit (MLX)
- kv:
- kv4
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Phi-3.5-mini-instruct-4bit at 72.2 t/s on a Mac Mini M4 Pro 64GB.
Setup is mlx-lm 0.30.7 with 4-bit weights and kv4 KV-cache quantization at 4096 generated tokens.
KV-cache quantization with kv4 uses 0.51 GB and is 1.1% faster than unquantized (71.4 t/s), while kv8 uses 0.91 GB and is 4.9% slower (67.9 t/s). User also tested Qwen2.5-14B-Instruct-4bit and found GQA halves KV-cache per token versus Phi-3.5-mini's MHA.