Llama 3.1 8B

on M4 16GB · llama.cpp

Oct 12, 2026

Throughput

21.0 t/s gen · 229.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K_M (GGUF)

System RAM

16 GB

Summary

User reports Llama 3.1 8B at 21 t/s generation and 229.2 t/s prompt processing on an M4 16GB Mac mini. Setup is llama.cpp 0.4.0 with Q4_K_M and Metal, all layers on GPU; context, KV precision and Flash Attention are unknown. The figures come from separate llama-bench tests, pp512 at 229.23 ±0.08 t/s and tg128 at 20.95 ±0.02 t/s over three runs, not a combined 512-input/128-output completion, with 10.29 GiB RAM already in use.