Gemma 4 26B (4B active)

User reports Gemma 4 26B-A4B QAT at about 170 tokens a second on one RTX 4090, with a full two-call pass taking 5 seconds. Setup is Ollama with QAT weights, one card, 96 GB system RAM; vLLM was faster at 3.5 seconds per pass but held 94% of the card against Ollama's 70%. Sending both calls of a pass together cut it to 3.8 seconds on Ollama and 2.8 seconds on vLLM. MedGemma 27B Q4_K_M took 18 seconds per pass. The user asks about speculative decoding, other engines, other models, and CPU offload.