Gemma 4 26B (4B active)

User benchmarks the Gemma 4 family on an NVIDIA DGX Spark (GB10 Grace Blackwell SoC) with 128 GB unified memory, served via llama.cpp in a Docker container. Setup is llama.cpp build 5d3a4a7 with GGUF weights, flash attention enabled, batch size 2048, and a 256K context window. The 26B-A4B MoE model uses Q4_K_M and reaches 69.9 t/s generation and 2657 t/s prefill at 512 tokens. The 31B dense model reaches 11.0 t/s generation and 743 t/s prefill. At 32K context depth, the 26B drops to 52.1 t/s generation and 1877 t/s prefill. Batched concurrency on the 26B reaches 537 t/s aggregate generation at 32 concurrent requests and 3015 t/s aggregate prefill at batch 16.