Gemma 4 26B (4B active)
on Unknown GPU · llama.cpp · 8,192 ctx
Summary
User reports Gemma 4 26B A4B at around 20.5 tokens/s on an NVIDIA GPU with 8 GB of VRAM and 32 GB of system RAM.
Setup is llama.cpp with Q5_K_P quantization, 8192 context, --parallel 1, --gpu-layers auto, --fit on, --load-mode none, reasoning off.
The user asks whether this is the best performance to expect or if there is room for optimization.