Bonsai 2 27B Ternary
on RTX 12GB · LM Studio
Use cases
vision
Summary
User reports Ternary Bonsai 2 27B at ~60 tok/s with PQ2_0 and ~52 tok/s with PTQ1_0 on a 12 GB RTX card.
Setup uses a custom LM Studio runtime built on Prism's llama.cpp fork (b10709) to load the custom ternary GGUF types, with vision (mmproj) working.
On Apple Silicon, an oMLX adapter runs the MLX 2-bit pack at 18 tok/s decode on an M1 Max 64 GB, about 30% slower than the v1 pack because it does not use oMLX's tuned 2-bit kernel.