Gemma 4 31B
on NVIDIA H100 80GB · vLLM · 32,768 ctx
May 13, 2026
Use cases
codingmath
Summary
User benchmarks Gemma 4 31B dense with MTP and DFlash speculative decoding, alongside Gemma 4 26B-A4B MoE (25.2B total, 3.8B active).
At concurrency 1 the dense baseline runs 40.3 tok/s, MTP 125.3 tok/s and DFlash 122.1 tok/s, making MTP 3.11x and DFlash 3.03x faster than baseline. At concurrency 16 the dense baseline reaches 375 tok/s, MTP 953 tok/s and DFlash 725 tok/s.
For the MoE at concurrency 1 the baseline runs 177.1 tok/s, MTP 264.2 tok/s and DFlash 306.4 tok/s. At concurrency 16 the MoE baseline reaches 975 tok/s, MTP 1808 tok/s and DFlash 1957 tok/s.
Coding, math, STEM and reasoning benefited more.