llamaperf

Raon-1 3.1B Nano

on Unknown GPU · ik_llama.cpp

Nov 12, 2026
Throughput
23.5 t/s gen · 229.1 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
Q4_K_M (GGUF)
System RAM
15 GB

Summary

User reports Raon-1 Nano 3.1B at 23.5 t/s generation and 229.1 t/s prompt processing on a CPU-only Intel Core i5-13420H with 15 GiB RAM. Setup is ik_llama.cpp with the official Q4_K_M GGUF (2.32 GiB), 4 threads, AVX2 + AVX-VNNI, no GPU. With run-time repack (-rtr) generation rises to 24.0 t/s and pp256 to 235.6 t/s. Native MTP speculative decoding gives 25.6 t/s (27.7 t/s with -rtr) at about 87% draft acceptance. Perplexity is 1.0976 versus 1.0960 on KRAFTON's fork.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Every Raon-1 report →