llamaperf

Gemma 4 26B (4B active)

on NVIDIA RTX 5090 · vLLM

Tone: positiveNVIDIA hardware
May 9, 2026
Throughput
578.0 t/s gen
Quant
AWQ-4bit
VRAM reported
32 GB

Summary

User reports 228 t/s baseline and 578 t/s with DFlash speculative decoding, a 2.56x speedup, on vLLM 0.19.2rc1. Setup uses the z-lab/gemma-4-26B-A4B-it-DFlash draft model, with 256 input tokens and 1024 output tokens.