Oct 12, 2026
Summary
User reports Gemma 4 26B-A4B at 53.2 t/s decode and 3991 t/s prefill on Strix Halo 128GB, compared against llama.cpp b11069 at 44.0 t/s decode and 1685 t/s prefill.
Setup is the Gufo fork with Unsloth UD-Q4_K_XL GGUF, single-user HTTP benchmark, greedy decoding with thinking off, no MTP.
With MTP speculative decoding decode rises to 79.8 t/s, and 254 t/s on repetitive output. At 131K context decode is 38.0 t/s (47.7 with MTP). With 8 concurrent users the 26B serves about 222 t/s aggregate versus 185 on llama.cpp. The 26B QAT quant reaches 81.4 t/s decode and 4131 t/s prefill in gufo bench versus 69.4 and 1546 on llama-bench.