Mac M4 reports

generation:

141.9 tokens/s

prompt processing (prefill):

2656.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Granite-4.0-H 1B at 141.9 t/s generation and 2656.4 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers, and no speculative drafter. The figures are separate synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the source's exact means are 2656.440968 ± 5.382321 t/s and 141.855767 ± 1.690267 t/s.

generation:

282.5 tokens/s

prompt processing (prefill):

8470.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Granite-4.0-H 350M at 282.5 t/s generation and 8470.4 t/s prompt processing on an M4 Max with 128 GiB unified RAM. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers and 12 CPU threads. The figures are standalone synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the JSON run measured 8470.38629 ± 91.887773 t/s and 282.459008 ± 1.376164 t/s.

generation:

19.7 tokens/s

prompt processing (prefill):

223.8 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3 8B at 19.7 t/s generation and 223.8 t/s prompt processing on a Mac mini M4 with 32 GB unified memory. Setup is llama.cpp b10929 with the Q4_K_M GGUF and Metal, all layers on the GPU, four CPU threads. The figures are separate llama-bench tests (pp512 and tg128), each the mean of three repetitions at zero initial depth, not one 512-input/128-output request; at 8192 prefill depth generation is 16.1 t/s.

generation:

36.2 tokens/s

prompt processing (prefill):

416.6 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3 4B at 36.2 t/s generation and 416.6 t/s prompt processing on a Mac mini M4 with 32 GB unified memory. Setup is llama.cpp b10929 with the Q4_K_M GGUF and Metal, all layers on the 10-core integrated GPU. The figures are separate llama-bench tests, pp512 and tg128, each the mean of three repetitions at zero initial depth, not one 512-input/128-output request.

generation:

13.1 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 13.1 t/s on a Mac mini M4 with 16 GB unified memory. Setup is llama.cpp with Q4_K_M and 8,192 tokens of context, run on the official Mac build on Apple graphics. The 13.1 t/s is the median of three requests after a short warm-up, with roughly 1% agreement among repeats; the source omits measured Mac memory.

generation:

17.7 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.5 9B at 17.7 t/s on a Mac mini M4 with 16 GB unified memory. Setup is llama.cpp with Q4_K_M and a configured context of 8,192 tokens. The figure is the median of three requests after a short warm-up, with roughly 1% agreement among repeats; no prompt-processing rate, TTFT or end-to-end latency is supplied.

generation:

29.7 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.5 4B at 29.7 t/s on a Mac mini M4 with 16 GB unified memory. Setup is llama.cpp with Q4_K_M and a configured context of 8,192 tokens; KV cache precision is not stated. The figure is the median of three requests after a short warm-up, with roughly 1% agreement among repeats; no prompt-processing rate or TTFT is given.

generation:

5.8 tokens/s

prompt processing (prefill):

60.5 tokens/s

quant:

UD-Q4_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 5.8 t/s generation and 60.5 t/s prompt processing on two Apple Mac mini M4 machines with 16 GB unified RAM each. Setup is llama.cpp build 9731ad3 with Metal and RPC, Unsloth UD-Q4_K_M, model split 50/50 over a Thunderbolt 4 bridge, with -fa 1 -ngl 99 -b 1024 -ub 512 and three repetitions. The 5.8 t/s is the tg128 native llama-bench microtest and 60.5 t/s is the separate pp512 phase; one mini cannot fit the 16.5 GB artifact at its 13 GB GPU limit.

generation:

12.7 tokens/s

prompt processing (prefill):

116.0 tokens/s

quant:

4-bit (MLX)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-14B-4bit at 12.7 t/s generation on one Apple Mac mini M4 with 16 GB unified memory. Setup is mlx-lm 0.29.1 native mlx_lm generate with 4-bit MLX safetensors, --temp 0 and --max-tokens 128, no speculative decoding or prompt cache. The separate long run reports 116 t/s prompt processing at 3,796 input tokens; the two short runs both measured 12.7 t/s, and peak memory was 8.4GB.

generation:

23.2 tokens/s

prompt processing (prefill):

219.0 tokens/s

quant:

4-bit (MLX)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-8B-4bit at 23.2 t/s generation on a single Apple Mac mini M4 with 16 GB unified memory. Setup is MLX via mlx-lm 0.29.1 native mlx_lm generate, 4-bit, with --temp 0 and --max-tokens 128; no speculative decoding or prompt cache. A separate long request reports 219 t/s prompt processing at 3,796 input tokens, so the two figures come from different requests. Peak memory is reported at 4.7GB.

generation:

23.9 tokens/s

prompt processing (prefill):

119.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 23.9 t/s generation and 119.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp (llama-server 0.4.0-dev, Metal) with Q4_K_M, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, flash attention on, no prompt cache, and speculation disabled. The headline Quick Chat run used 87 prompt tokens and 96 output tokens, with TG 23.88856 t/s and PP 119.88656 t/s as medians of three measured requests after one discarded warmup.

generation:

76.4 tokens/s

prompt processing (prefill):

319.9 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 76.4 t/s generation and 319.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with UD-Q4_K_XL, one Mac Studio with 40-core GPU and 64 GB unified memory. The headline Quick Chat run used 35 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not filled 4K or long-context benchmarks.

generation:

79.9 tokens/s

prompt processing (prefill):

91.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 79.9 t/s generation and 91 t/s prompt processing on an M4 Max with 64GB unified memory. Setup is ZINC on Metal with the UD-Q4_K_XL GGUF, speculative decoding disabled, context auto-sized from device memory. The headline Quick Chat run used 36 prompt tokens and 96 output tokens; the three rate samples span 78.71 to 79.95 t/s for generation and 90.1 to 92.7 t/s for prefill.

generation:

65.5 tokens/s

prompt processing (prefill):

305.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 9B at 65.5 t/s generation and 305.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with Q4_K_M, 4096 context, flash attention on, and prompt caching disabled. The headline Quick Chat run used 45 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not long-context benchmarks.

generation:

67.1 tokens/s

prompt processing (prefill):

546.3 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 9B at 67.1 t/s generation and 546.3 t/s prompt processing on an M4 Max 64GB. Setup is ZINC with Metal, Q4_K_M, speculative decoding disabled, and an auto-sized context; the run used 48 prompt tokens and 96 output tokens. The stored 67.1 t/s is the median of three samples, with the source's own median listed as 67.05 t/s over a 67 to 67.34 t/s range.

generation:

22.4 tokens/s

prompt processing (prefill):

62.7 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 22.4 t/s generation and 62.7 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with Q4_K_M, 4096 context, flash attention on, and one Mac Studio with a 40-core GPU. The headline Quick Chat run used 52 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not long-context benchmarks.

generation:

20.8 tokens/s

prompt processing (prefill):

123.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 20.8 t/s generation and 123.9 t/s prompt processing on an M4 Max 64GB. Setup is ZINC on Metal with Q4_K_M, serial single-device inference with speculative decoding disabled; configured context is unknown. The headline Quick Chat run used 49 prompt tokens and 96 output tokens, with TG 20.82[20.64,21.14] and PP 123.9[119.2,124.4] t/s as medians of three runs; these are short inputs, not long-context benchmarks.

generation:

81.7 tokens/s

prompt processing (prefill):

379.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 81.7 t/s generation and 379.6 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with UD-Q4_K_M, one Mac Studio with a 40-core GPU and 64 GB unified memory. The figures are medians of three measured nonstreaming requests after one discarded warmup, with 52 prompt and 96 output tokens, and speculation disabled.

generation:

68.4 tokens/s

prompt processing (prefill):

303.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 68.4 t/s generation and 303.6 t/s prompt processing on an M4 Max 64GB. Setup is ZINC with UD-Q4_K_M, Metal, speculative decoding disabled, and unknown context length. The headline Quick Chat run used 49 prompt tokens and 96 output tokens, with TG 68.44[67.17,68.66] and PP 303.6[299.2,318.6] t/s over three measured invocations.

generation:

5.3 tokens/s

prompt processing (prefill):

90.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 5.3 t/s generation and 90 t/s prompt processing on an M4 32GB Mac mini. Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, 100% GPU offload, 20.29 GB loaded memory including KV cache. The figures are three-run averages from a fixed Chinese long-form prompt; 4096 is the configured context, not measured input length, and per-run counters are unpublished.