Gemma 4 12B
on NVIDIA RTX 2060 Super 8 GB; Intel Core i9-10900K host · llama.cpp · 8,192 ctx
Summary
User reports Gemma 4 12B-it at 27.7 t/s on one NVIDIA RTX 2060 Super with 8 GB of VRAM and an Intel Core i9-10900K host.
Setup is llama.cpp with a Q4_K_M GGUF, -c 8192, -ngl 99 and --parallel 1, with every model layer forced onto the GPU.
The 27.7 t/s is the median of three requests after a short warm-up, with roughly 1% agreement among repeats. The author separately reports 11.3 t/s when automatic placement leaves some layers in system memory.