Bonsai 27B
on NVIDIA RTX 3060 Laptop 6GB · mentria.ai · 3,072 ctx
Summary
User reports 25-30 t/s decode in a chat UI and 32 t/s raw in a WebGPU browser inference engine, with prompt processing of 1489 tokens in about 25 seconds.
The model is natively 1-bit, 27B parameters in 3.8 GB, running at 3072 tokens of context on a 6 GB card.
The user also mentions smaller tiers (Qwen3.5 0.8B, 2B, 4B) and a vision tower.