vLLM on the NVIDIA RTX 2080 Ti 22GB (modded)
llamaperf has 3 vLLM reports on the NVIDIA RTX 2080 Ti 22GB (modded), none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
3 vLLM reports on the NVIDIA RTX 2080 Ti 22GB (modded)
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- Qwen3.8 27B · NVFP4220.8 t/s(2 cards, speculative decoding)
- Qwen3.6 35B · 3B active · AWQ 4-bit120.0 t/s(2 cards)
- Qwen3.6 27B · AWQ101.3 t/s(2 cards, speculative decoding)
Frequently asked
How fast is vLLM on the NVIDIA RTX 2080 Ti 22GB (modded)?
llamaperf has 3 vLLM reports on the NVIDIA RTX 2080 Ti 22GB (modded), none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.