Gemma 4 31B

User reports Gemma 4 31B IT NVFP4 Turbo at 51 tok/s single-stream decode and 15359 tok/s prefill on an RTX PRO 6000. Setup is vLLM with NVFP4 weights, FP8 KV cache, and 16384 tokens of context, using 18.5 GiB of GPU memory. Batched decode reaches 1244 tok/s and concurrency 6.22 req/s at 1K input / 200 output tokens; the same benchmark on an RTX 5090 performed identically because at 16k context performance is not limited by GPU memory.