Qwen3 8B
on NVIDIA RTX 5060 8GB · llama.cpp
Use cases
codingagentictool-use
Summary
User reports Qwen3-8B at about 57 tok/s on an RTX 5060, the only machine where the full verification was run.
Setup is llama.cpp with Q4_K_M weights; the tool picked the model, quant, context size and KV cache for the hardware.
The figure comes from smoke tests, not a quality benchmark: 9 of 9 checks passed and tool calling works. AMD, Apple and the other supported runtimes have had much less testing.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3 8B at 4-bit by hardware
- NVIDIA RTX 508016 GB130.6 t/s · one run
- 104.2 t/s · 2 runs
- 98.8 t/s · one run
- 81.2 t/s · one run
- Intel Arc Pro B7032 GB77.7 t/s · one run
- AMD RX 6700 XT12 GB61.0 t/s · one run
Bars are medians on one scale, 0 to 130.6 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Models most run on the NVIDIA RTX 5060 8GB
- Qwen3 8B4-bit57.0 t/s · one run
- Qwen3.8 27B2-bit30.1 t/s · one run
Bars are medians on one scale, 0 to 57.0 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the NVIDIA RTX 5060 8GB →Every Qwen3 report →Estimate Qwen3 on this card →