Gemma 4 26B (4B active)
on RTX 4070 Laptop 8GB · Unsloth Studio · 65,536 ctx
Summary
User reports Gemma 4 26B A4B at 40-50 t/s on an RTX 4070 Laptop 8GB with 32GB system RAM.
Setup is Unsloth Studio with the QAT GGUF, 65,536 context, q8 KV cache, MTP speculative decoding with 2 draft tokens, and 31 GPU layers.
Initial throughput was around 20 t/s; dropping context length and shifting more layers to the GPU raised it to 40-50 t/s.