Gemma 4 30.7B
on 2× NVIDIA RTX PRO 5000 Blackwell (one 72GB card and one 48GB card) · llama.cpp
Summary
User reports Gemma 4 31B at 52.9 t/s generation and 3022.2 t/s prompt processing on one RTX PRO 5000 Blackwell 72 GB plus one RTX PRO 5000 Blackwell 48 GB.
Setup is llama.cpp CUDA b10941 with Q8_0, f16 K/V cache, flash attention on, tensor split 1/1, batch 2048 and microbatch 512.
The 52.9 t/s is TG128 and the 3022.2 t/s is PP512, separate tests rather than one combined request; the same run measured PP2048 at 2936.0 t/s and PP8192 at 2881.1 t/s.