Llama 3.2 3B

on NVIDIA RTX 5080 · Ollama · 16,384 ctx

Oct 12, 2026

Throughput

298.6 t/s gen · 19397.4 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K_M (GGUF)

VRAM reported

16 GB

Summary

User reports Llama 3.2 3B at 298.6 t/s generation and 19397.4 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at a configured 16384-token context, with an actual recorded prompt of 1657 tokens rather than a full 16384-token prefill. The figures are medians of 3 fresh-prefill runs, derived from Ollama eval_count/eval_duration rather than TTFT division, with temperature 0 and a unique nonce per repetition to avoid prompt-cache hits.