llamaperf

Gemma 4 26B (4B active)

on NVIDIA RTX 5080 · llama.cpp

Tone: positiveNVIDIA hardware
Nov 20, 2026
Throughput
225.0 t/s gen
Quant
Q3_K_M (GGUF)

Summary

User reports a pruned one-subject Gemma 4 26B-A4B pack at 225 tok/s on an RTX 5080, against 223 tok/s for the full model. Setup is unmodified llama.cpp (b12549) with Q3_K_M GGUF; the pack keeps 64 of 128 experts per layer and is 7.4 GB versus 13.3 GB for the full model. The full model at Q2_K produced broken output. On a Ryzen 7 9700X with no GPU the pack runs 20.5 tok/s and the full model 20.0 tok/s. Against the full model on 300 blind-judged questions a pack wins or ties 53% at 64 experts per layer and 77% at 96, and is much weaker outside its subject.