llamaperf

Bonsai 2 26.9B Ternary-Bonsai-2

on M4 Pro 64GB · llama.cpp

Apple hardware
Oct 9, 2026
Throughput
20.5 t/s gen · 126.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
PQ2_0 (GGUF)
KV cache
F16
Flash Attention
on
System RAM
64 GB

Summary

User reports a sourced third-party community benchmark of Bonsai 2 27B PQ2_0 on one Apple Mac mini M4 Pro with 64 GB unified memory. Prompt processing pp512 measured 126.89 +/- 0.02 tokens/second and token generation tg128 measured 20.53 +/- 0.01 tokens/second. Setup is llama.cpp PrismML fork (tag prism-b10709-9a9394a) with PQ2_0 GGUF and default F16 KV cache, 10 threads, -ngl 99, flash attention enabled, Metal/BLAS backend. The two figures are separate synthetic llama-bench tests with three repetitions each, not a paired chat request. Caveats: Spotlight indexing consumed approximately 1.5-2 CPU cores throughout the runs; an unrelated 664 MB embedding model reappeared between runs; no swap pressure was observed. The 6.70 GiB GGUF size is a file size, not measured resident or peak RAM. Peak RAM, configured maximum context, exact weight revision, binary hash, and exact measurement date are unknown.