llamaperf

Qwen3 8B

on NVIDIA RTX 5060 8GB · llama.cpp

Tone: positiveNVIDIA hardware
Nov 9, 2026
Throughput
57.0 t/s gen
Quant
Q4_K_M (GGUF)

Use cases

codingagentictool-use

Summary

User reports Qwen3-8B at about 57 tok/s on an RTX 5060, the only machine where the full verification was run. Setup is llama.cpp with Q4_K_M weights; the tool picked the model, quant, context size and KV cache for the hardware. The figure comes from smoke tests, not a quality benchmark: 9 of 9 checks passed and tool calling works. AMD, Apple and the other supported runtimes have had much less testing.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3 8B at 4-bit by hardware

Bars are medians on one scale, 0 to 130.6 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 5060 8GB

Bars are medians on one scale, 0 to 57.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 5060 8GB →Every Qwen3 report →Estimate Qwen3 on this card →