Bonsai 2 27B Swift Katana
on NVIDIA RTX 3060 12GB · llama.cpp · 128,000 ctx
Use cases
tool-usevisionagenticcodinglong-context
Summary
User reports a Qwen3.8-27B derivative (Swift Bonsai 2 27B Katana + MTP) generating around 43-44 t/s on an RTX 3060 12GB, peaking at 45.74 t/s over a three-second window.
Setup is the Prism fork of llama.cpp with a PTQ1_0 ternary GGUF and q4_0 KV cache at 128,000 tokens, MTP speculation with a draft limit of 1, all model layers on the GPU and the vision projector on the CPU; the whole process used 9,144 MiB of VRAM.
In daily use the user sees 28-40 t/s after a few prompts and 17-25 t/s much later with far more context. The user cites another author's Bonsai 2 PTQ1_0 + MTP tests on the same card at 50.1 t/s with MTP versus 39.8 without at 131K, and 26.86 versus 22.23 at 41.8K tokens filled. Figures are described as observations rather than controlled benchmarks.