Bonsai 2 27B Ternary
on Intel Arc B580 12GB · llama.cpp · 131,072 ctx
Oct 3, 2026
Use cases
codingagenticlong-context
Summary
User reports Ternary Bonsai 2 27B running on an Intel Arc B580 12GB at 128K context entirely in VRAM, with about 90 t/s writing new code at temperature 0 (about 80 at temperature 0.6), 250 to 370 t/s on code edits with MTP plus n-gram drafts, and 42 t/s plain.
Setup is a custom llama.cpp fork with ternary weights repacked to 2 bit, q4_0 KV cache, and decode attention on XMX; the fork is compared against PrismML's basic SYCL path on the same card, model and flags.
At 48K tokens of history it still does 63 t/s, and 44 at 115K. Against PrismML's fork: prompt reading 207 vs 895 t/s, plain generation 21 vs 42, generation at 32K depth 8 vs 36, chat with MTP 38 vs 88. Against Gemma 4 12B on the same card, Gemma is a bit better at code (HumanEval+ 152 vs 141 / 164), they tie on GSM8K, and tool calling is close; with Google's QAT assistant as MTP drafter, Gemma 4 12B goes from ~40 to ~100 t/s on new code (edits 64 -> 168).