Gemma 4 31B

User reports Gemma-4-31B at 36.5 tok/s on a single RTX 3090 with full big context, using mainline llama.cpp built from source with -DGGML_CUDA_FA_ALL_QUANTS=ON. Setup is llama.cpp with Q4_K_S weights and q5_0/q4_1 KV cache on one 24 GB card; the stock prebuilt image only reaches ~12 tok/s because the flag is off by default. The post compares engines: beellama.cpp with DFlash speculative decoding reaches 54-193 tok/s depending on workload (131-200K context), vLLM gets 86-145 tok/s on dual cards at 262K, and ik_llama is fast but context-capped near 24K. Quality on an 8-pack was 109/150 with reasoning off and 114/150 with reasoning on.