Gemma 4 26B (4B active)
on AMD Strix Halo 128GB · llama.cpp
Use cases
summarizationtool-usemultilinguallong-context
Summary
User reports Gemma 4 26B-A4B QAT at 102/102/127 tok/s (Swedish prose / English prose / JSON) on a Strix Halo 128GB box, about 45% faster than the regular UD-Q4_K_M at 70/71/90 tok/s.
Setup is llama.cpp Vulkan/RADV build 11235 with QAT UD-Q4_K_XL and the matching QAT MTP drafter, single request, -ctxcp 0, using 17.7 GB versus 20.9 GB for Q4_K_M.
Lower K-quants (Q3_K_XL, IQ3_S) gave no speed gain. Nemotron 3.5 Lightning 30B-A3B and Qwen3.6-35B-A3B made factual errors in 50-80% of summaries versus about 10% for Gemma. Strata v0.1.44.1 with Qwen3.8-Flash-Next IQ3_S decodes at 50.9 tok/s shallow context and passed 256k needle tests up to 242k tokens with prefill flat near 1,200 tok/s.