llamaperf

Bonsai 2 27B Ternary

on Unknown GPU · llama.cpp

Tone: mixed
Oct 14, 2026
Throughput
2.8 t/s gen
Quant
TQ1_0 (GGUF)

Summary

User reports running a 27B ternary model across two phones over a USB-C cable at ~2.8 tok/s. The setup uses llama.cpp RPC over a localhost tunnel created via Android USB accessory mode, with no Wi-Fi, root, or adb. The model is Ternary-Bonsai-2 (27B) in a Hadamard-rotated basis, run with a custom llama.cpp patch that applies rotation to activations. The two-phone pair is not yet faster than the fastest single phone, which achieves 5.62 tok/s on a Redmi, compared to 0.44 tok/s with the vendor's runtime. The requantization increases perplexity to 14.26 from a 14.05 reference (+1.5%). A lossless repacker from 2-bit ternary GGUF to GGML's TQ1_0 base-3 container reduces the model size from 6.5 GiB to 5.3 GiB (−18%).