llamaperf
Oct 14, 2026
Throughput
11.1 t/s gen · 68.1 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
KV cache
F16
System RAM
128 GB

Summary

User reports Llama 2 70B Chat at 11.1 t/s generation and 68.1 t/s prompt processing on a Mac Studio M1 Ultra with 128GB unified memory. Setup is llama.cpp build 2164 with Q4_K_M quantization, 512-token context, F16 K and F16 V cache, all 81 layers offloaded to the Metal GPU. The figures are the runtime's separate prompt-eval and eval counters, not wall-clock request throughput; prompt evaluation was 56 tokens and token evaluation was 333 runs.