Raon-1 3.1B Nano
on Unknown GPU · ik_llama.cpp
Summary
User reports Raon-1 Nano 3.1B at 23.5 t/s generation and 229.1 t/s prompt processing on a CPU-only Intel Core i5-13420H with 15 GiB RAM.
Setup is ik_llama.cpp with the official Q4_K_M GGUF (2.32 GiB), 4 threads, AVX2 + AVX-VNNI, no GPU.
With run-time repack (-rtr) generation rises to 24.0 t/s and pp256 to 235.6 t/s. Native MTP speculative decoding gives 25.6 t/s (27.7 t/s with -rtr) at about 87% draft acceptance. Perplexity is 1.0976 versus 1.0960 on KRAFTON's fork.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.