NVIDIA · 11GB · 2 reports
Use the calculator to check which models fit in 11 GB of VRAM. The reports below include different GPU counts and memory setups; a model listed here may need extra cards or CPU offloading.
How to compare these reports →RTX 2080 Ti · NInfer · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Speculative decoding with MTP3 draft window yields ~456 tok/s, ~65% acceptance rate. Standard autoregressive ~25 tok/s. VRAM usage ~17.5 GiB with draft weights, leaving ~4.5-5.0 GiB for KV cache.
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Custom Turing CUDA kernels for W8A8 INT8 matmul. Heterogeneous inference with 4x 11/22GB VRAM and 1TB system RAM. Computation-communication overlap for MoE routing. Open-sourced on GitHub.