Gemma 4 26B
on RTX 5080 Laptop · Gem16 · 220,000 ctx
Use cases
visionmultilingual
Summary
User reports Gemma 4 26B at 182 t/s decode and 5660 t/s prefill on an RTX 5080 Laptop 16GB GPU.
Setup is the custom Gem16 engine with an EXL3-like custom quantisation, fitting 220k context; the engine is single-user and uses MTP speculative decoding.
The user also reports Gemma 4 12B at 87 t/s decode and 5800 t/s prefill, and notes the 12B can serve 2 sessions. The 12B's native audio understanding degrades past around 8k context, which the user attributes to the model.