llamaperf

Qwen3 0.6B

on NVIDIA GT 1030 2GB · llama.cpp

Oct 8, 2026
Throughput
64.8 t/s gen · 494.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
KV cache
q8_0
System RAM
4 GB
VRAM reported
2 GB

Summary

User benchmarks Qwen3 0.6B Q4_K_M on an NVIDIA GT 1030 2GB in an old Core2 Quad Q6600 PC with 4GB DDR2 RAM, comparing Vulkan and CUDA backends. Setup is llama.cpp with Q4_K_M weights, q8_0 KV cache, flash attention on, 99 layers offloaded, and n_batch 1024. Vulkan gives 494.23 t/s pp512 and 64.77 t/s tg128; CUDA gives 834.55 t/s pp512 and 40.72 t/s tg128. A second Vulkan run without int dot support drops pp512 to 242.19 t/s while tg128 stays at 64.36 t/s. User asks why Vulkan decode is faster and CUDA prefill is faster, and whether a mix of ~800 t/s prefill and ~60 t/s decode is possible.