Llama 3.2 1B
on Unknown GPU · llama.cpp
Summary
User reports Llama 3.2 1B Instruct at 15.4 tokens/s on a Raspberry Pi 5 (8GB).
Setup is llama.cpp compiled natively for Cortex-A76 with Q8_0 quantization, running CPU-only on 4 cores.
A Q4_K_M build reached 17.8 tokens/s, prompt eval 42.1 tokens/s, and time-to-first-token 0.38 s, compared with 0.85 tokens/s and 18.97 s TTFT for a generic ARM64 build.