llamaperf

Bonsai 2 27B Swift Katana

on NVIDIA RTX 3060 12GB · llama.cpp · 128,000 ctx

Tone: positiveNVIDIA hardware
Oct 26, 2026
Throughput
43-44 t/s gen
Quant
PTQ1_0 (GGUF)
KV cache
q4_0
System RAM
32 GB
VRAM reported
12 GB

Use cases

tool-usevisionagenticcodinglong-context

Summary

User reports a Qwen3.8-27B derivative (Swift Bonsai 2 27B Katana + MTP) generating around 43-44 t/s on an RTX 3060 12GB, peaking at 45.74 t/s over a three-second window. Setup is the Prism fork of llama.cpp with a PTQ1_0 ternary GGUF and q4_0 KV cache at 128,000 tokens, MTP speculation with a draft limit of 1, all model layers on the GPU and the vision projector on the CPU; the whole process used 9,144 MiB of VRAM. In daily use the user sees 28-40 t/s after a few prompts and 17-25 t/s much later with far more context. The user cites another author's Bonsai 2 PTQ1_0 + MTP tests on the same card at 50.1 t/s with MTP versus 39.8 without at 131K, and 26.86 versus 22.23 at 41.8K tokens filled. Figures are described as observations rather than controlled benchmarks.