Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 193 t/s on Hopper. Setup is vLLM with Canada-Quant quants and patched MTP code. User also reports roughly 400 t/s with 4 concurrent threads.
NVIDIA H100 80GB · vLLM · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Gemma 4 31B dense with MTP and DFlash speculative decoding, alongside Gemma 4 26B-A4B MoE (25.2B total, 3.8B active). At concurrency 1 the dense baseline runs 40.3 tok/s, MTP 125.3 tok/s and DFlash 122.1 tok/s, making MTP 3.11x and DFlash 3.03x faster than baseline. At concurrency 16 the dense baseline reaches 375 tok/s, MTP 953 tok/s and DFlash 725 tok/s. For the MoE at concurrency 1 the baseline runs 177.1 tok/s, MTP 264.2 tok/s and DFlash 306.4 tok/s. At concurrency 16 the MoE baseline reaches 975 tok/s, MTP 1808 tok/s and DFlash 1957 tok/s. Coding, math, STEM and reasoning benefited more.
2× NVIDIA H100 80GB · vLLM · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B at 45 t/s on a rented GPU instance with 2x H100s and 160 GB of VRAM. Setup is vLLM at 128K context, serving multiple agents hitting the API at once, including Claude Code, QwenCode and social media bots. The instance costs about $0.90/hr, with $120 spent last month. The user says the model outperformed a 120B model in tests.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking