Mac M4 reports

generation:

238.6 tokens/s

prompt processing (prefill):

4177.1 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ling-mini-2.0 at 238.6 t/s generation and 4177.1 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with Q4_K_M, flash attention and F16 K/V, single sequence, 99 GPU layers requested. Figures are depth-0 microbenchmarks: PP512 and TG128 separately, not an end-to-end run. At prefix depth 4096 generation falls to 213.24 t/s and at 16384 to 172.28 t/s.

generation:

123.1 tokens/s

prompt processing (prefill):

1577.0 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-20b at 123.1 t/s generation and 1577 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with MXFP4 GGUF, flash attention, F16 K/V, requested GPU layers 99, single sequence, five repetitions. The headline is depth 0; at prefix depth 4096 generation falls to 117.20 t/s and at 16384 to 105.36 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

88.1 tokens/s

prompt processing (prefill):

1408.1 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 88.1 t/s generation and 1408.1 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with UD-Q4_K_M, flash attention and F16 K/V cache, single sequence, GPU layers 99. The headline is depth 0; at prefix depth 4096 generation is 79.77 t/s and at 16384 it is 75.49 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

85.4 tokens/s

prompt processing (prefill):

809.5 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-120b at 85.4 t/s generation and 809.5 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with MXFP4 GGUF, flash attention, F16 K/V, requested GPU layers 99, single sequence, five repetitions. Headline is depth 0; at prefix depth 4096 generation falls to 81.13 t/s and at 16384 to 72.80 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

54.5 tokens/s

prompt processing (prefill):

589.0 tokens/s

quant:

Q4_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 54.5 t/s generation and 589 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with Q4_0, flash attention and F16 K/V, single sequence, 99 GPU layers requested. The headline is depth 0; at prefix depth 4096 generation is 50.62 t/s and at 16384 it is 47.37 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

6.5 tokens/s

prompt processing (prefill):

60.8 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

F16

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 6.5 t/s generation and 60.8 t/s prompt processing on a Mac mini M4 with 24 GB unified memory. Setup is llama.cpp build 9544 with the UD-IQ4_XS GGUF, run via llama-bench with -p 512 -n 128, F16 K/V cache and flash attention AUTO. The source's native figures are PP512 = 60.78 +/- 1.60 tokens/s and TG128 = 6.47 +/- 0.03 tokens/s, separate prompt and generation microbenchmarks rather than one request; TG128 is depth-0 generation, not decode after a 512-token prompt.

generation:

90.1 tokens/s

prompt processing (prefill):

2122.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-0.8B at 90.1 t/s generation and 2122 t/s prompt processing on an M4 16GB Mac mini. Setup is llama.cpp 0.6.0 build 11429 with Q8_0 and Metal, flash attention enabled, single-stream. The same campaign's Homebrew llama.cpp 0.4.1 build 10964 baseline reached 87.3 t/s generation and 2049 t/s prompt processing; batched aggregate generation was 87.7 t/s at 1 stream and 322.4 t/s at 8 streams.

generation:

28.3 tokens/s

prompt processing (prefill):

352.4 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-4B at 28.3 t/s generation and 352.4 t/s prompt processing on a Mac mini M4 with 16 GB unified memory. Setup is llama.cpp 0.6.0 build 11429 with Metal, Q4_K_M GGUF, flash attention enabled, run via llama-bench with -p 512 -n 64 -d 0,4096 -r 3. The two figures are separate microbenchmarks, not one 512-input/64-output request; generation context depth is unknown, and PP512 after 4096 existing tokens is 321.5 t/s.

generation:

12.9 tokens/s

prompt processing (prefill):

138.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 12.9 t/s generation and 138.5 t/s prompt processing on a base Apple M4 Mac mini with 24 GiB unified memory. Setup is llama.cpp b10090 with UD-Q4_K_XL, run via llama-bench at 512 prompt and 128 generation tokens over 5 repetitions on one GPU. Later interleaved runs reached 12.71/12.79 t/s and prompt speed drifted 129–134 under sustained load; a depth ladder showed generation falling to 9.26 t/s at 32768.

generation:

47.4 tokens/s

prompt processing (prefill):

556.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5 3B at 47.4 t/s generation and 556.9 t/s prompt processing on a base Apple M4 Mac mini with 16 GB unified memory. Setup is llama.cpp with Q4_K_M, all layers on the GPU via Metal, measured with llama-bench PP512 and TG128 passes. The figures are means of three repetitions, with 10.42 GiB baseline system RAM in use during the run, so it is not an idle-machine result.

generation:

70.9 tokens/s

prompt processing (prefill):

1245.6 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5-Omni-3B at 70.9 t/s generation and 1245.6 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q8_0 weights, f16 K/V cache, and one Metal GPU; the run used 512 synthetic prompt positions and 128 serial decode positions. The figures are engine-side loop wall time, not a 512-input/128-output chat request, and no speculative decoding was used.

generation:

54.2 tokens/s

prompt processing (prefill):

957.9 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-VL-4B-Instruct at 54.2 t/s generation and 957.9 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA with Q8_0 weights, q8 activations, f16 scales and f16 KV cache, one GPU, 640 positions allocated. The figures are native synthetic counts: 512 prompt positions and 128 serial decode positions, not a single chat request.

generation:

13.0 tokens/s

prompt processing (prefill):

125.6 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 13 t/s generation and 125.6 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with UD-IQ4_XS weights and f16 KV cache, one GPU, 640 positions allocated. The figures are native PP512 and TG128 means over five repetitions, not one 512-input/128-output request.

generation:

13.1 tokens/s

prompt processing (prefill):

126.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 13.1 t/s generation and 126.2 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q4_K_M weights, f16 KV cache, and one Metal GPU. The figures are native PP512 and TG128 means over five repetitions, not a single chat request; the session allocates 640 positions.

generation:

86.6 tokens/s

prompt processing (prefill):

1545.7 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 86.6 t/s generation and 1545.7 t/s prompt processing on an M4 Pro with 64GB unified memory. Setup is dasLLAMA with Q8_0 weights, f16 KV cache, and one Metal GPU; the session requested 640 positions and no speculative decoding. The figures are native PP512 and TG128 counts over five repetitions, not one 512-in/128-out request.

generation:

17.6 tokens/s

prompt processing (prefill):

143.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Mistral Small 3.1 24B at 17.6 t/s generation and 143.2 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q4_K_M weights, f16 KV cache, and one Metal GPU at 99 layers offloaded. The figures are native PP512 and TG128 counts, not a single 512-input/128-output request; the session requests 640 positions of context.

generation:

94.5 tokens/s

prompt processing (prefill):

806.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-30B-A3B-Instruct-2507 at 94.5 t/s generation and 806 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q4_K_M weights, f16 KV cache, and one Metal GPU, with 640 positions allocated. The figures are native PP512 and TG128 means over five repetitions, not a single 512-input/128-output request.

generation:

83.2 tokens/s

prompt processing (prefill):

862.6 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-20b at 83.2 t/s generation and 862.6 t/s prompt processing on an Apple M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with MXFP4 weights, q8 activations, f16 scales and f16 KV cache, one Metal GPU with 99 layers offloaded. Figures are native PP512 and TG128 counts over five repetitions, not a single 512-input/128-output request; the session requests 640 positions of context.

generation:

46.2 tokens/s

prompt processing (prefill):

770.2 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 46.2 t/s generation and 770.2 t/s prompt processing on an M4 Pro 64GB. Setup is dasLLAMA with Q8_0 weights, f16 KV cache, and one Metal GPU, with 640 positions allocated. The figures are native PP512 and TG128 counts, not one 512-input/128-output request, and no longer-context performance is inferred.

generation:

30.9 tokens/s

prompt processing (prefill):

298.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 30.9 t/s generation and 298.5 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q4_K_M weights and f16 KV cache, one GPU, 640 positions allocated. The figures are native PP512 and TG128 means over five repetitions, not a single 512-input/128-output request; a separate four-stream run reached 34.18746480210349 t/s summed scheduler throughput.