Gemma 4 31B
on 2× AMD RX 7900 XT · vLLM
Summary
User reports gemma-4-31B at 43 tok/s decode on 2× RX 7900 XT with tensor parallelism.
Setup is vLLM with w4a16 weights, TP=2, both cards drawing 265 W at the same time.
The machine is a VFIO virtual machine with no P2P and cross-die PCIe 3.0, measured with no PCIe atomics. A 26B MoE reaches 108 tok/s at short context on the same pair.