Mac M4 reports

generation:

60.4 tokens/s

prompt processing (prefill):

818.7 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B-it at 60.4 t/s generation and 818.7 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA 0.6.4 with UD-Q4_K_M weights, f16 KV cache, and one Metal GPU, with 640 positions allocated. The figures are native PP512 and TG128 counts over five repetitions, not a single 512-input/128-output request.

generation:

12.9 tokens/s

prompt processing (prefill):

126.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6 27B at 12.9 t/s generation and 126.4 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q4_K_M weights, f16 K/V cache, and one Metal GPU; the run used 512 synthetic prompt positions and 128 serial decode positions. The user notes these are native processing counts, not one 512-input/128-output chat request, and that no speculative decoding was requested.

generation:

66.4 tokens/s

prompt processing (prefill):

787.1 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 66.4 t/s generation and 787.1 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with UD-Q4_K_M weights, f16 K/V cache, and one Metal GPU at 640 allocated positions. The figures are native PP512 and TG128 counts, not a single 512-input/128-output request.

generation:

67.5 tokens/s

prompt processing (prefill):

1158.7 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5-Omni-3B at 67.5 t/s generation and 1158.7 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is llama.cpp commit 6fdd0ac89 with Q8_0 weights, Metal backend, -ngl 99, 10 CPU threads, and f16 K/V cache. The figures are separate native tests from cleared state: PP512 and TG128, each at zero pre-existing depth, not one 512-input/128-output request.

generation:

53.6 tokens/s

prompt processing (prefill):

878.7 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-VL-4B-Instruct at 53.6 t/s generation and 878.7 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp commit 6fdd0ac89 with Q8_0 weights, Metal backend, -ngl 99, and 10 CPU threads on one physical GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

12.8 tokens/s

prompt processing (prefill):

126.1 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 12.8 t/s generation and 126.1 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory. Setup is llama.cpp commit 6fdd0ac89 with the UD-IQ4_XS GGUF, Metal backend, -ngl 99, 10 CPU threads, and K/V cache in f16. The figures are native llama-bench means over five repetitions: PP512 at 126.094155 t/s and TG128 at 12.841348 t/s, each starting from cleared state at depth 0, with model load and warmup excluded.

generation:

11.5 tokens/s

prompt processing (prefill):

126.3 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 11.5 t/s generation and 126.3 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp commit 6fdd0ac89 with Q4_K_M, Metal backend, -ngl 99, 10 CPU threads, and K/V f16. The native figures are PP512 and TG128 from cleared state at depth 0, not a single 512-input/128-output request.

generation:

78.2 tokens/s

prompt processing (prefill):

1558.9 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 78.2 t/s generation and 1558.9 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with Q8_0 weights, Metal backend, -ngl 99, and 10 CPU threads on one physical GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

17.4 tokens/s

prompt processing (prefill):

146.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Mistral Small 3.1 24B at 17.4 t/s generation and 146.8 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is llama.cpp commit 6fdd0ac89 with Q4_K_M weights, Metal backend, -ngl 99, 10 CPU threads, and f16 K/V cache. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single chat request.

generation:

77.2 tokens/s

prompt processing (prefill):

780.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-30B-A3B-Instruct-2507 at 77.2 t/s generation and 780.2 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp commit 6fdd0ac89 with Q4_K_M, Metal backend, -ngl 99, and 10 CPU threads on one physical GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single chat request.

generation:

73.6 tokens/s

prompt processing (prefill):

834.2 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-20b at 73.6 t/s generation and 834.2 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is llama.cpp commit 6fdd0ac89 with MXFP4 weights, Metal backend, -ngl 99, 10 CPU threads, and f16 K/V cache on one GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions from cleared state at depth 0, not a single chat request.

generation:

42.4 tokens/s

prompt processing (prefill):

786.5 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 42.4 t/s generation and 786.5 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with Q8_0 weights, Metal backend, -ngl 99 and 10 CPU threads, on one physical GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

29.1 tokens/s

prompt processing (prefill):

284.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 29.1 t/s generation and 284.5 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with Q4_K_M, Metal backend, -ngl 99, and 10 CPU threads. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

54.2 tokens/s

prompt processing (prefill):

730.9 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 54.2 t/s generation and 730.9 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with UD-Q4_K_M, Metal backend, -ngl 99, 10 CPU threads, and one physical GPU. The figures are separate native tests: PP512 and TG128 at depth 0, five repetitions each, not a single 512-input/128-output request.

generation:

12.6 tokens/s

prompt processing (prefill):

123.9 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B at 12.6 t/s generation and 123.9 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory. Setup is llama.cpp commit 6fdd0ac89 with the Q4_K_M GGUF, Metal backend, -ngl 99, 10 CPU threads, and f16 K/V cache. The figures are native llama-bench means over five repetitions: PP512 at 123.908607 t/s and TG128 at 12.621998 t/s, each starting from cleared state at depth 0, with load and warmup excluded.

generation:

48.6 tokens/s

prompt processing (prefill):

747.1 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 48.6 t/s generation and 747.1 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with UD-Q4_K_M, K/V f16, Flash Attention auto, -ngl 99 and 10 CPU threads on one Metal GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single chat request.

generation:

116.3 tokens/s

quant:

AXQ MXFP4 mixed (MLX)

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Cyber-Tiel-Coder-35B-A3B at 116.3 t/s on an Apple M4 Pro with 64 GiB unified memory. Setup is the AX Engine native MLX MXFP4 trunk with a protected BF16 MTP sidecar, expert streaming off, one request at a time with cold KV and no prefix reuse. The figure is the median decode rate over six measured requests on the python-lru workload at 219 input and 256 output tokens, with a range of 116.19 to 116.35 t/s; completion including TTFT was 94.37 t/s.

generation:

52.7 tokens/s

prompt processing (prefill):

683.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Falcon-H1 7B-Instruct at 52.7 t/s generation and 683 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers, and no speculative drafter. The figures are separate synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the JSON run measured 682.958202 ± 4.356899 t/s and 52.696786 ± 0.186655 t/s.

generation:

86.4 tokens/s

prompt processing (prefill):

1297.7 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Falcon-H1 3B-Instruct at 86.4 t/s generation and 1297.7 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers, and 12 CPU threads. The figures are separate synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the JSON run measured 1297.664754 ± 4.914553 t/s and 86.354246 ± 0.010372 t/s.

generation:

147.5 tokens/s

prompt processing (prefill):

2535.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Falcon-H1 1.5B-Instruct at 147.5 t/s generation and 2535.2 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers and 12 CPU threads. The figures are standalone synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the source's exact means are 2535.20394 ± 4.882498 t/s and 147.525027 ± 0.873067 t/s.