Qwen3 0.6B
on NVIDIA GT 1030 2GB · llama.cpp
Summary
User benchmarks Qwen3 0.6B Q4_K_M on an NVIDIA GT 1030 2GB in an old Core2 Quad Q6600 PC with 4GB DDR2 RAM, comparing Vulkan and CUDA backends.
Setup is llama.cpp with Q4_K_M weights, q8_0 KV cache, flash attention on, 99 layers offloaded, and n_batch 1024. Vulkan gives 494.23 t/s pp512 and 64.77 t/s tg128; CUDA gives 834.55 t/s pp512 and 40.72 t/s tg128.
A second Vulkan run without int dot support drops pp512 to 242.19 t/s while tg128 stays at 64.36 t/s. User asks why Vulkan decode is faster and CUDA prefill is faster, and whether a mix of ~800 t/s prefill and ~60 t/s decode is possible.