SmolLM2 0.14B 0.1B
NVIDIA Jetson Orin Nano Super 8GB · llama.cpp · 2,048 ctx
- generation:
- 165.2 tokens/s
- quant:
- Q4_K_M (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks SmolLM2-135M at 165.2 tok/s on a Jetson Orin Nano Super 8GB at 25W power mode. Setup is llama.cpp CUDA with Q4_K_M quant, 2048-token context, 256-token generation, single request, all layers on GPU. 25W beats 15W by 35-47% throughput and edges out MAXN on tokens per joule; llama.cpp is 1.37x faster than Ollama at 25W. The model uses 101 MB and draws about 5.6 W.