Bonsai 2
on NVIDIA RTX 5060 Ti 16GB · llama.cpp
Summary
User reports Bonsai 2 at 300 tok/s on an RTX 5060 Ti 16GB using a fork of llama.cpp.
The user states the fork previously ran Bonsai 2 at 140 tok/s on an RTX 4070 Ti Super 16GB, and that after further optimization it reaches 140 tok/s on the 5060 Ti 16GB. The 300 tok/s figure is given as a correction for running without full context.
The fork was also modified to run more than one session in parallel, as it previously ran only one.