Raon-1 29B (4B active) mini-preview
on NVIDIA RTX 5070 Ti · ik_llama.cpp
Summary
User reports Raon-1-mini-preview at 232 tok/s on an RTX 5070 Ti 16GB, up from 50 tok/s.
Setup is ik_llama.cpp with an IQ3_KS GGUF, weights streamed from system RAM on a machine with a Ryzen 5 5600X and 48GB DDR4.
The 232 tok/s is the combined result of stacked optimizations, with weights streaming at about 2/3 of the card's peak bandwidth.