Gemma 4 31B
on CUDA0 · llama.cpp · 15,000 ctx
Summary
User reports Gemma4 31B at about 10 t/s, compared with Qwen3.8 27B at 40-70 t/s and a 120B MoE at 10-20 t/s.
Setup is llama.cpp with speculative decoding using an MTP draft model and a Q4 KV cache.
The user notes memory allocation on a second GPU despite free VRAM on CUDA0.