Bonsai 2 Ternary 27B CRACK / PQ2_0 / idle resident peer / code r1
on AMD Strix Halo 128GB · Prism llama.cpp 1a07bfa5 + input-sync patch / HIP · 262,144 ctx
Summary
User reports Bonsai 2 at 22.3 t/s generation and 214.2 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is PrismML-Eng/llama.cpp with PQ2_0 weights, Q4_0 K/V cache, full GPU offload, flash attention, one slot and 262144 context on a single Ryzen AI MAX+ 395 APU with 128 GB unified memory.
The figure is the idle-peer fixed-prefix run with 524 input and 256 output tokens; the source gives 22.25989819108211 t/s and 214.23028774725407 t/s.