llama.cpp on the M4 24GB
llamaperf has 4 llama.cpp reports on the M4 24GB, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
4 llama.cpp reports on the M4 24GB
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- Qwen 2.5 Coder · Q4_K_M22.6 t/s
- Llama 3.2 · Q4_K_M46.0 t/s
- Qwen3.8 · UD-IQ4_XS6.5 t/s
- Gemma 4 · UD-Q4_K_XL12.9 t/s
Frequently asked
How fast is llama.cpp on the M4 24GB?
llamaperf has 4 llama.cpp reports on the M4 24GB, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.