Llama 3 8B
on NVIDIA RTX 3090 · llama.cpp · 1,024 ctx
Oct 9, 2026
Summary
User benchmarks LLaMA 3 8B at 111.74 t/s generation and 3865.39 t/s prompt processing on an RTX 3090 24GB.
Setup is llama.cpp with Q4_K_M GGUF weights, 1024-token generation and 1024-token prompt eval, single card.
The same card reaches 115.42 t/s at 512 tokens and drops to 87.49 t/s at 8192 tokens; the F16 build of the same model runs at 46.51 t/s generation and 4239.64 t/s prompt processing.