Mac M5 reports

generation:

80.3 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3-14B at 80.3 t/s on a Mac Studio M5 Ultra with 256 GB unified memory. Setup is LM Studio with 4-bit MLX weights on one machine, 36 CPU cores and 80 GPU cores. The 80.3 t/s is the mean of three runs; TTFT is 3.53 seconds, and the displayed 2,754 prompt tokens/s is prompt-token count divided by TTFT, not native prefill. Configured context capacity is unknown.

generation:

26.3 tokens/s

prompt processing (prefill):

685.4 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports QwQ-32B at 26.3 t/s generation and 685.4 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, flash attention enabled, and n_gpu_layers=99. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request; context length is unspecified.

generation:

124.3 tokens/s

prompt processing (prefill):

3342.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 124.3 t/s generation and 3342.7 t/s prompt processing on a single Apple M5 Max with 48 GB unified memory. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

86.5 tokens/s

prompt processing (prefill):

2369.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 9B at 86.5 t/s generation and 2369.3 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

135.6 tokens/s

prompt processing (prefill):

4099.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 4B at 135.6 t/s generation and 4099.7 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

111.2 tokens/s

prompt processing (prefill):

3378.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-35B-A3B at 111.2 t/s generation and 3378.3 t/s prompt processing on a single Apple M5 Max with 48 GB unified memory. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

29.2 tokens/s

prompt processing (prefill):

730.5 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-27B at 29.2 t/s generation and 730.5 t/s prompt processing on a single Apple M5 Max with 48 GB unified memory. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, flash attention enabled and n_gpu_layers=99; the context length is unspecified. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput.

generation:

97.7 tokens/s

prompt processing (prefill):

2744.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 8B at 97.7 t/s generation and 2744.7 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request; context length is unspecified.

generation:

159.7 tokens/s

prompt processing (prefill):

4972.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 4B at 159.7 t/s generation and 4972.7 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request; context length is unspecified.

generation:

26.4 tokens/s

prompt processing (prefill):

667.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 32B at 26.4 t/s generation and 667 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, flash attention on, and n_gpu_layers=99. The figures are separate llama-bench microtests, tg128=26.38 t/s and pp512=667.0 t/s, not one 512-input/128-output request; context length is unspecified.

generation:

148.5 tokens/s

prompt processing (prefill):

3496.9 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 30B A3B at 148.5 t/s generation and 3496.9 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

57.4 tokens/s

prompt processing (prefill):

1584.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 14B at 57.4 t/s generation and 1584.3 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

288.6 tokens/s

prompt processing (prefill):

11493.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 1.7B at 288.6 t/s generation and 11493.01 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

457.1 tokens/s

prompt processing (prefill):

23288.5 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 3 0.6B at 457.1 t/s generation and 23288.5 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

106.0 tokens/s

prompt processing (prefill):

2901.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 2.5 Coder 7B at 106 t/s generation on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

26.3 tokens/s

prompt processing (prefill):

685.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 2.5 Coder 32B at 26.3 t/s generation and 685 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, flash attention on, and n_gpu_layers=99. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

56.0 tokens/s

prompt processing (prefill):

1546.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen 2.5 Coder 14B at 56.0 t/s generation and 1545.95 t/s prompt processing on a single Apple M5 Max with 48 GB unified memory. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

56.8 tokens/s

prompt processing (prefill):

1537.5 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Phi-4 at 56.8 t/s generation and 1537.5 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

164.9 tokens/s

prompt processing (prefill):

5412.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Phi-4 at 164.9 t/s generation and 5412.7 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.

generation:

165.0 tokens/s

prompt processing (prefill):

5416.2 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Phi-4-mini Q4_K_M at 165 t/s generation and 5416.2 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, n_gpu_layers=99 and flash attention enabled, run as separate llama-bench microtests. The figures are native tg128 and pp512 tests, not one 512-input/128-output request or server concurrency; context length is unspecified.