llama.cpp on the NVIDIA RTX 3060 12GB

27 llama.cpp reports on the NVIDIA RTX 3060 12GB

Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.

• Qwen3.8 · IQ4_XS12.9 t/s(2 cards)
• Qwen3.8 27B · Q4_K_S30.8 t/s(2 cards)
• Ornith1.5 35B · 3B active · IQ4_XS97.5 t/s(2 cards)
• Qwen3.8 125B · 6B active · IQ3_XXS15.0 t/s(2 cards, part in system RAM)
• Muse 30B · IQ4_XS42.0 t/s(2 cards, speculative decoding)
• Qwen3.8 27B · UD-Q4_K_XL56.0 t/s(2 cards, speculative decoding)
• Qwen3.8 27B · Q4_K_XL47.3 t/s(2 cards, speculative decoding)
• Model not identified · iq2_xxs30.0 t/s(2 cards, part in system RAM)
• Qwen3.8 125B · 6B active · IQ4_XS10.0 t/s(part in system RAM)
• Qwen3.8 27B · Q4_K_S9.7 t/s(speculative decoding, part in system RAM)
• Qwen3.8 27B · Q625.0 t/s(3 cards)
• Qwen3.8 27B · IQ3-XXS29.0 t/s(speculative decoding)
• Qwen3.8 27B · IQ3_XXS29.0 t/s(speculative decoding)
• Qwen3.8 27B · IQ3_XXS11.4 t/s(speculative decoding)
• Qwen3.8 27B · IQ3_XXS38.4 t/s(speculative decoding)
• Qwen3.5 13B12.0 t/s(part in system RAM)
• Qwen3.8 27B · Q4_K_XL40.0 t/s(2 cards, speculative decoding)
• Qwen3.8 27B · Q4_K_XL50.6 t/s(2 cards, speculative decoding)
• Qwen3.8 · Q4_K_XL20.0 t/s
• DeepSeek V4 Flash 284B · 13B active · Q4_K_XL10.1 t/s(4 cards, part in system RAM)
• Qwen3.8 27B · Q4_K_XL26.9 t/s(2 cards, speculative decoding)
• Qwen3.8 27B · Q4_K_M27.7 t/s(2 cards)
• Qwen3.8 125B · 6B active · IQ4_XS12.0 t/s(2 cards, part in system RAM)
• Qwen3.8 125B · 6B active · UD-Q4_K_XLPP 200.0 t/s(part in system RAM)
• Qwen3.6 27B · Q4_K_S43.3 t/s(2 cards, speculative decoding)
• Qwen3.6 27B · Q2-XS70.0 t/s(part in system RAM)
• Qwen3.6 35B · 3B active · IQ4_XS46.8 t/s(part in system RAM)