Gemma 4 26B (4B active)
AMD MI50 32GB · Ollama
- prompt processing (prefill):
- 95.0 tokens/s
- quant:
- Q4_K_M (GGUF)
- kv:
- q8_0
User reports Gemma 4 26B at 65-68 tok/s on an AMD Instinct MI50 32GB. Setup is Ollama v0.32.14 with Q4_K_M and q8_0 KV cache, flash attention on, all layers offloaded, MTP speculative decoding enabled automatically. The figure is a range; short answers reach ~95 tok/s prompt eval. A QAT Q4_0 build with MTP reaches ~78 tok/s, and Qwen 3.5 35B Q6_K runs at ~52 tok/s with ~541 tok/s prompt eval. Power overdrive to 160W adds another +8-9%.