- throughput:
- 60.0 t/s gen
- quant:
- Q4_K_M (gguf)
text-generation
~60 tok/s on RTX 3060 12GB. E2B runs effortlessly. Source: estimated from compute-market tiers
RTX 3060 12GB
NVIDIA · 12GB · 3 reports
- throughput:
- 25.0 t/s gen
- quant:
- Q4_K_M (gguf)
text-generation
~25 tok/s on RTX 3060 12GB. 26B MoE Q4 fits with ~8K context. Great value option. Source: compute-market.com
Gemma 4 8B E4B Instruct
RTX 3060 12GB · Ollama
- throughput:
- 45.0 t/s gen
- quant:
- Q4_K_M (gguf)
text-generation
~45 tok/s on RTX 3060 12GB. E4B fits easily. Source: compute-market.com