- throughput:
- 30.0 t/s gen
- quant:
- Q4_K_M (gguf)
text-generation
~30 tok/s on RTX 4070 12GB. 26B MoE fits at Q4 with short context. Source: compute-market.com
RTX 4070
NVIDIA · 12GB · 2 reports
This page is thin (2 of 3 reports needed for indexing). Help fill it in.
Gemma 4 8B E4B Instruct
RTX 4070 · Ollama
- throughput:
- 55.0 t/s gen
- quant:
- Q4_K_M (gguf)
text-generation
~55 tok/s on RTX 4070 12GB. Ada Lovelace efficiency. Source: estimated from compute-market tiers