Bonsai 2 27B Ternary
on M4 Max 128GB · llama.cpp · 131,072 ctx
Summary
User reports Bonsai 2 27B (Ternary) at 34.5 t/s generation on an Apple Mac Studio M4 Max 128GB.
Setup is llama.cpp (Prism fork build 10683) with PQ2_0 ternary GGUF, Metal, flash attention on, -ngl 999, 131072 configured context, single request, temperature 0, no speculative decoding. The 7.2 GB figure is the model file size, not measured peak RAM.
The 34.5 t/s is the arithmetic mean of three prose runs (35.49, 34.03, 33.87 t/s) over 2097 prompt tokens and 320 output tokens; a related code workload averaged 34.3 t/s. Prefill is not reported because TTFT includes request overhead. KV-cache type, peak memory, CPU core count and power draw are unknown.