llamaperf

M2 24GB reports

3 reports. These are individual setups, not a controlled benchmark. M2 24GB speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

LLaMA family 7B / Q4_0 / Mac mini M2 Metal b1550 / PP512 TG128 / exact base and artifact unknown

M2 24GB · llama.cpp b1550 (8e672efe) / Metal

Apple hardware
generation:
21.9 tokens/s
prompt processing (prefill):
179.6 tokens/s
quant:
Q4_0 (GGUF)

User reports a LLaMA-family 7B model at 21.9 t/s generation and 179.6 t/s prompt processing on a Mac mini M2 with 24 GB unified RAM. Setup is llama.cpp build 1550 with Q4_0 quantization, F16 KV cache, Metal backend and ngl 99, running llama-bench PP512 and TG128. The figures are independent synthetic prompt and generation tests, not a 512-token prompt followed by a 128-token answer; the source rates are 179.57 ± 0.04 t/s and 21.91 ± 0.02 t/s.

Oct 13, 2026

LLaMA family 7B / Q8_0 / Mac mini M2 Metal b1550 / PP512 TG128 / exact base and artifact unknown

M2 24GB · llama.cpp b1550 (8e672efe) / Metal

Apple hardware
generation:
12.2 tokens/s
prompt processing (prefill):
181.4 tokens/s
quant:
Q8_0 (GGUF)

User reports a LLaMA-family 7B model mostly Q8_0 at 12.2 t/s generation and 181.4 t/s prompt processing on a Mac mini M2 with 24 GB unified RAM. Setup is llama.cpp build 1550 with Q8_0 and F16 KV cache, Metal backend at ngl 99, one GPU. The figures are native llama-bench PP512 and TG128 synthetic tests, not a 512-token prompt followed by a 128-token answer; the source rates are 181.40 ± 0.05 t/s and 12.21 ± 0.04 t/s.

Oct 13, 2026

LLaMA family 7B / F16 / Mac mini M2 Metal b1550 / PP512 TG128 / exact base and artifact unknown

M2 24GB · llama.cpp b1550 (8e672efe) / Metal

Apple hardware
generation:
6.7 tokens/s
prompt processing (prefill):
201.3 tokens/s
quant:
F16 (GGUF)

User reports a LLaMA-family 7B model at 6.7 t/s generation and 201.3 t/s prompt processing on a Mac mini M2 with 24 GB unified RAM. Setup is llama.cpp build 1550 with F16 weights and F16 KV cache, Metal backend with ngl 99 on one GPU. The figures are native llama-bench PP512 and TG128 results, 201.34 ± 0.21 t/s and 6.72 ± 0.01 t/s, independent synthetic prompt and generation tests rather than a 512-token prompt followed by a 128-token answer.

Oct 13, 2026