Qwen3 8B
NVIDIA RTX 5060 8GB · llama.cpp
- generation:
- 57.0 tokens/s
- quant:
- Q4_K_M (GGUF)
User reports Qwen3-8B at about 57 tok/s on an RTX 5060, the only machine where the full verification was run. Setup is llama.cpp with Q4_K_M weights; the tool picked the model, quant, context size and KV cache for the hardware. The figure comes from smoke tests, not a quality benchmark: 9 of 9 checks passed and tool calling works. AMD, Apple and the other supported runtimes have had much less testing.