Gemma 4 12B

User reports Gemma 4 12B QAT at 20 t/s decode and 1100 t/s prefill on a single Intel Arc A770 16GB. Setup is llama.cpp with QAT weights, Q8 KV cache and a Q4 MTP head, giving 128k context with about 5GB VRAM headroom; the card is in a 13-year-old Haswell desktop with under 6GB RAM headroom. Four models run together at 14-15GB VRAM via llama-swap, with LiteLLM routing prompts through Laya to thinking and non-thinking aliases of the 12B. The user notes SYCL/llama.cpp lacks Alchemist optimizations, making the A770 the bottleneck, and that GPU-resident models still use 4-5GB of system RAM as compute buffer.