Bonsai 2 27B
on NVIDIA RTX 5060 Ti 16GB · NInfer · 200,000 ctx
Summary
User reports Bonsai 2 27B running on an RTX 5060 Ti 16GB with NInfer, achieving prefill over 1000 t/s (dropping from an initial 1600-1800 t/s) and decode above 50 t/s at up to about 200k context.
Setup uses a modified and optimised version of NInfer, adapted with the help of ChatGPT and Claude to run on the 5060 Ti.
Prefill and decode figures are given as ranges rather than single values.