What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
NVIDIA 80 GB+· NVIDIA only
Workstation and datacenter cards. 70B-class and larger on one device. Memory speed 1,344 to 8,000 GB/s across the 9 devices with reports, so a typical figure here spans all of them. 74 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly FP8 Gen 50 t/s (3 runs) · PP 3468 t/s (1 run) · with speculation 107 t/s (9) | 20 | 50 3 runs, 3 devices | 3468 1 run | 107 9 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly NVFP4 Gen 105 t/s (6 runs) · PP 5110 t/s (6 runs) · with speculation 240 t/s (3) | 15 | 105 6 runs | 5110 6 runs | 240 3 runs |
| 3 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly Q4_K_XL Gen 46 t/s (3 runs) · PP 312 t/s (2 runs) | 10 | 46 3 runs, 2 devices | 312 2 runs | none |
| 4 | Qwen3.627B Alibaba · mostly Q4_K_M Gen 38 t/s (1 run) · PP 907 t/s (1 run) · with speculation 118 t/s (1) | 3 | 38 1 run | 907 1 run | 118 1 run |
| 5 | GLM-4.5-Air106B · 12B active mostly Q4_K_M Gen 138 t/s (1 run) | 2 | 138 1 run | no PP data | none |
| 6 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly IQ2_XXS Gen 70 t/s (1 run) | 2 | 70 1 run | no PP data | none |
| 7 | Gemma 431B Google DeepMind · mostly NVFP4 Gen 51 t/s (1 run) · PP 15359 t/s (1 run) · with speculation 125 t/s (1) | 2 | 51 1 run | 15359 1 run | 125 1 run |
| 8 | Muse Glimmer30B Meta · mostly BF16 Gen 35 t/s (1 run) · with speculation 57 t/s (1) | 2 | 35 1 run | no PP data | 57 1 run |
| 9 | DeepSeek V4 Pro1600B · 49B active DeepSeek No generation data | 2 | no generation data | no PP data | none |
| 10 | Qwen3.635B · 3B active Alibaba · mostly Q6_K No generation data · with speculation 63 t/s (1) | 2 | no generation data | no PP data | 63 1 run |
| 11 | Qwen3-Next80B · 3B active Alibaba · mostly W4A16 Gen 116 t/s (1 run) | 1 | 116 1 run | no PP data | none |
| 12 | Qwen3.8 Flash-Next Abliterated125B · 6B active Alibaba · mostly Q5 Gen 110 t/s (1 run) | 1 | 110 1 run | no PP data | none |
| 13 | GLM-4.7 Flash30B Zhipu AI No generation data · with speculation 168 t/s (1) | 1 | no generation data | no PP data | 168 1 run |
| 14 | Gemma 34B Google · mostly BF16 No generation data · with speculation 133 t/s (1) | 1 | no generation data | no PP data | 133 1 run |
| 15 | Gemma 4 Gemma 4 31B31B Google DeepMind · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 16 | Llama 3.21B Meta No generation data | 1 | no generation data | no PP data | none |
| 17 | Muse Muse Glimmer 30B28B Meta · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 18 | Nemotron 3.5 Super VL120B · 12B active mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 19 | Qwen38B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 20 | Qwen3.535B · 3B active Alibaba · mostly Q8_0 No generation data · PP 16460 t/s (1 run) | 1 | no generation data | 16460 1 run | none |
| 21 | Qwen3.6 abliterated27B Alibaba · mostly BF16 No generation data | 1 | no generation data | no PP data | none |
| 22 | Qwen3.6 Qwen3.6-35B-A3B35B · 3B active Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 23 | Qwen3.8 Qwen3.8-27B27B Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 24 | Qwen3.8 Sparkle 1.027B Alibaba · mostly FP8 No generation data · with speculation 165 t/s (1) | 1 | no generation data | no PP data | 165 1 run |
Not in the table: 3 reports don't say which model size ran. 41 ran on more than one device, which says nothing about what one of them does.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.