Gemma 4 26B (4B active)
on NVIDIA RTX 5090 · llama.cpp
Summary
User benchmarks an FWHT CUDA implementation for KV cache quantization on Gemma 4 26B.A4B Q4_K_M.
Setup is llama.cpp with -ctk q8_0 -ctv q8_0, reporting pp2048 and tg128 values.
The implementation gives a 1-2% pp boost and a 7-9% tg boost, with the highest t/s taken from the cuda-fwt column.