NVIDIA · 80GB · 4 reports
Use the calculator to check which models fit in 80 GB of VRAM. The reports below include different GPU counts and memory setups; a model listed here may need extra cards or CPU offloading.
How to compare these reports →Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 193 t/s on Hopper. Setup is vLLM with Canada-Quant quants and patched MTP code. User also reports roughly 400 t/s with 4 concurrent threads.
User reports a model based on Qwen3.5-4B, trained on 8xH100 for 3 days. It supports Safetensors, GGUF and MLX weights, requires as little as 4GB VRAM, and is available in GPTQ, W8A8, FP8, Q4 and Q6 quantizations. It has been tested with vLLM, SGLang and llama.cpp.
H100 80GB · vLLM · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User benchmarks Gemma 4 31B dense with MTP and DFlash speculative decoding, alongside Gemma 4 26B-A4B MoE (25.2B total, 3.8B active). At concurrency 1 the dense baseline runs 40.3 tok/s, MTP 125.3 tok/s and DFlash 122.1 tok/s, making MTP 3.11x and DFlash 3.03x faster than baseline. At concurrency 16 the dense baseline reaches 375 tok/s, MTP 953 tok/s and DFlash 725 tok/s. For the MoE at concurrency 1 the baseline runs 177.1 tok/s, MTP 264.2 tok/s and DFlash 306.4 tok/s. At concurrency 16 the MoE baseline reaches 975 tok/s, MTP 1808 tok/s and DFlash 1957 tok/s. Coding, math, STEM and reasoning benefited more.
2× H100 80GB · vLLM · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports Qwen3.6-27B at 45 t/s on a rented GPU instance with 2x H100s and 160 GB of VRAM. Setup is vLLM at 128K context, serving multiple agents hitting the API at once, including Claude Code, QwenCode and social media bots. The instance costs about $0.90/hr, with $120 spent last month. The user says the model outperformed a 120B model in tests.