llamaperf

Llama 3

2 reports

Llama 3 8B at 4-bit generates a median of 111.7 tokens per second on the NVIDIA RTX 3090, the hardware it is most run on, from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, Llama 3 8B at 4-bit on each device, fastest first, with the median generation speed of plain runs (one device, one request, no speculative decoding, the whole model in its memory):

Bars are medians on one scale, 0 to 111.7 tokens/s. Press ▶ to watch text arrive at a row's speed.

Llama 3 VRAM requirements by size and quant →

How does Llama 3 run on your hardware?

Pick your GPU or Mac, then read the speeds people reported for Llama 3 on it, or estimate memory fit and speed.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Run Llama 3 yourself? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance for Llama 3

Filter this model’s reports by setup →
2 reports for this model so far, so treat the figures as early. Add yours.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Llama 3 8B

NVIDIA RTX 3090 · llama.cpp · 1,024 ctx

NVIDIA hardware
generation:
111.7 tokens/s
prompt processing (prefill):
3865.4 tokens/s
quant:
Q4_K_M (GGUF)

User benchmarks LLaMA 3 8B at 111.74 t/s generation and 3865.39 t/s prompt processing on an RTX 3090 24GB. Setup is llama.cpp with Q4_K_M GGUF weights, 1024-token generation and 1024-token prompt eval, single card. The same card reaches 115.42 t/s at 512 tokens and drops to 87.49 t/s at 8192 tokens; the F16 build of the same model runs at 46.51 t/s generation and 4239.64 t/s prompt processing.

Oct 9, 2026

Llama 3 8B

2× NVIDIA RTX 3090 · llama.cpp · 65,536 ctx

Tone: negativeNVIDIA hardware
generation:
129.1 tokens/s
prompt processing (prefill):
5124.4 tokens/s
quant:
Q4_0 (GGUF)
kv:
f16
flash attention:
on

User benchmarks the new tensor parallel (split mode tensor) implementation in llama.cpp against graph parallel in ik_llama.cpp on a 2x RTX 3090 system with 48 GiB of VRAM. Setup is llama.cpp with Q4_0 quantized Llama 3 8B, f16 KV cache, 65536 context, 100 GPU layers, and flash attention enabled. The run crashed with CUDA out of memory at 34816 tokens of context. User reports 129.08 t/s generation and 5124.38 t/s prompt processing at zero context, degrading to 26.89 t/s generation and 3092.68 t/s prompt processing at 34816 tokens. User calls the PR a gimmick not ready for prime time, noting memory is not released and performance lags well behind ik_llama.cpp graph parallel.

Oct 8, 2026

Get a weekly email of new Llama 3 reports on any GPU.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Llama 3 8B
NVIDIA RTX 3090
Q4_K_M
llama.cpp
1,024111.7 tokens/s
Llama 3 8B
2× NVIDIA RTX 3090
Q4_0
llama.cpp
65,536129.1 tokens/s

Frequently asked

What hardware do people run Llama 3 on?

The hardware Llama 3 8B at 4-bit is most run on: NVIDIA RTX 3090 (111.7 t/s median, one run). Medians of plain runs, as of 22 Nov 2026.

Which inference engines do people use for Llama 3?

Reports of Llama 3 name llama.cpp (2 reports), as of 22 Nov 2026.