What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
NVIDIA 20-24 GB· NVIDIA only
The community sweet spot: 27B-32B at 4 bits with room for context. Memory speed 300 to 1,008 GB/s across the 11 devices with reports, so a typical figure here spans all of them. 203 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly Q4_K_M Gen 37 t/s (19 runs) · PP 1062 t/s (6 runs) · with speculation 96 t/s (33) | 83 | 37 19 runs, 6 devices | 1062 6 runs | 96 33 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly UD-Q4_K_XL No generation data | 18 | no generation data | no PP data | none |
| 3 | Qwen3.627B Alibaba · mostly Q4_K_M Gen 35 t/s (4 runs) · PP 818 t/s (2 runs) · with speculation 50 t/s (6) | 14 | 35 4 runs, 3 devices | 818 2 runs | 50 6 runs |
| 4 | Qwen3.635B · 3B active Alibaba · mostly IQ4 Gen 85 t/s (6 runs) · PP 982 t/s (1 run) · with speculation 52 t/s (1) | 11 | 85 6 runs, 3 devices | 982 1 run | 52 1 run |
| 5 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 No generation data | 10 | no generation data | no PP data | none |
| 6 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly 3.05bpw No generation data | 6 | no generation data | no PP data | none |
| 7 | DeepSeek V4.1 Flash552B · 16B active mostly 4-bit No generation data | 5 | no generation data | no PP data | none |
| 8 | Gemma 426B · 4B active Google DeepMind · mostly Q4_K_M Gen 144 t/s (4 runs) · PP 5600 t/s (1 run) | 4 | 144 4 runs | 5600 1 run | none |
| 9 | Qwen3.8 Swift-1.527B Alibaba · mostly IQ3_S Gen 67 t/s (1 run) · with speculation 94 t/s (2) | 4 | 67 1 run | no PP data | 94 2 runs |
| 10 | Muse Glimmer30B Meta · mostly Q4_K_M Gen 30 t/s (1 run) · with speculation 89 t/s (2) | 3 | 30 1 run | no PP data | 89 2 runs |
| 11 | Nemotron 3.5 Bolt74B · 9B active mostly 2-bit Gen 86 t/s (2 runs) · PP 828 t/s (2 runs) | 2 | 86 2 runs | 828 2 runs | none |
| 12 | Gemma 431B Google DeepMind · mostly Q4_K_S Gen 41 t/s (2 runs) | 2 | 41 2 runs, 2 devices | no PP data | none |
| 13 | Qwen3.8 Swift-1.5-Uncensored27B Alibaba · mostly IQ4_XS No generation data · with speculation 68 t/s (1) | 2 | no generation data | no PP data | 68 1 run |
| 14 | Qwen30.6B Alibaba Gen 531 t/s (1 run) | 1 | 531 1 run | no PP data | none |
| 15 | Qwen2.5 Coder-7B7B Alibaba · mostly Q4_K_M Gen 159 t/s (1 run) | 1 | 159 1 run | no PP data | none |
| 16 | Llama 38B mostly Q4_K_M Gen 112 t/s (1 run) · PP 3865 t/s (1 run) | 1 | 112 1 run | 3865 1 run | none |
| 17 | Gemma 412B Google DeepMind · mostly QAT Gen 83 t/s (1 run) | 1 | 83 1 run | no PP data | none |
| 18 | Qwen3.8 Escha-W227B Alibaba · mostly escha 2-bit Gen 67 t/s (1 run) | 1 | 67 1 run | no PP data | none |
| 19 | Ornith1.535B · 3B active mostly Q4_0 Gen 61 t/s (1 run) · PP 1104 t/s (1 run) | 1 | 61 1 run | 1104 1 run | none |
| 20 | Qwen3.8 Jev27B Alibaba · mostly 4.0bpw Gen 54 t/s (1 run) · PP 885 t/s (1 run) | 1 | 54 1 run | 885 1 run | none |
| 21 | Qwen3.59B Alibaba · mostly BF16 Gen 41 t/s (1 run) | 1 | 41 1 run | no PP data | none |
| 22 | Qwen3.8 AREX-227B Alibaba · mostly EXL3 4.00 bpw Gen 36 t/s (1 run) | 1 | 36 1 run | no PP data | none |
| 23 | Qwen2.5 Coder-32B32B Alibaba · mostly Q4_K_M Gen 28 t/s (1 run) · PP 450 t/s (1 run) | 1 | 28 1 run | 450 1 run | none |
| 24 | Qwen2.57B Alibaba · mostly bf16 Gen 18 t/s (1 run) | 1 | 18 1 run | no PP data | none |
| 25 | Qwen3 Heretic14B Alibaba · mostly INT4_SYM Gen 3.6 t/s (1 run) | 1 | 3.6 1 run | no PP data | none |
| 26 | Bonsai 2 Ternary27B PrismML · mostly Q4/Q5 No generation data · with speculation 188 t/s (1) | 1 | no generation data | no PP data | 188 1 run |
| 27 | GLM-4.5-Air106B · 12B active No generation data | 1 | no generation data | no PP data | none |
| 28 | GLM-5.2744B · 40B active Zhipu AI · mostly UD-IQ2_M No generation data | 1 | no generation data | no PP data | none |
| 29 | Gemma 31B Google No generation data | 1 | no generation data | no PP data | none |
| 30 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 31 | Nemotron 3 Super120B · 12B active mostly ~3 bits per weight No generation data | 1 | no generation data | no PP data | none |
| 32 | Nemotron 3.5 Lightning30B · 3B active NVIDIA · mostly W4A16 No generation data | 1 | no generation data | no PP data | none |
| 33 | Ornith1.5 BigBang35B · 3B active mostly Q4_K_M No generation data · with speculation 169 t/s (1) | 1 | no generation data | no PP data | 169 1 run |
| 34 | Ornith1.5122B mostly Q4_K_XL No generation data | 1 | no generation data | no PP data | none |
| 35 | Qwen3235B · 22B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 36 | Qwen3-Next80B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 37 | Qwen3.5 Mica v0.14B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 38 | Qwen3.527B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 39 | Qwen3.535B · 3B active Alibaba · mostly IQ4_NL No generation data · PP 980 t/s (1 run) | 1 | no generation data | 980 1 run | none |
| 40 | Qwen3.6 Heretic-v227B Alibaba · mostly Q4_K_M No generation data · with speculation 80 t/s (1) | 1 | no generation data | no PP data | 80 1 run |
| 41 | Qwen3.8 Uncensored-HauHauCS27B Alibaba · mostly INT8 No generation data | 1 | no generation data | no PP data | none |
| 42 | Qwen3.8 heretic-ara27B Alibaba · mostly Q5_K_M No generation data | 1 | no generation data | no PP data | none |
| 43 | Qwen3.8 Bonsai 227B Alibaba · mostly PQ2_0 No generation data · PP 835 t/s (1 run) | 1 | no generation data | 835 1 run | none |
| 44 | Qwen3.8 Swift27B Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 45 | Qwen3.8 Swift Uncensored27B Alibaba · mostly IQ4_XS-M No generation data | 1 | no generation data | no PP data | none |
| 46 | Qwen3.8 Huihui-Abliterated27B Alibaba · mostly Q4_K_S No generation data | 1 | no generation data | no PP data | none |
| 47 | Qwen3.8 Bonsai-Llama-Jev27B Alibaba · mostly Q2_64 No generation data | 1 | no generation data | no PP data | none |
| 48 | Qwen3.8 Uncensored27B Alibaba No generation data · with speculation 20 t/s (1) | 1 | no generation data | no PP data | 20 1 run |
| 49 | Qwen3.8 Defiance-Uncensored27B Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 50 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ3_S No generation data | 1 | no generation data | no PP data | none |
| 51 | Qwen3.8 Swift-1.5 Flash-Next125B · 6B active Alibaba · mostly IQ2_XS No generation data | 1 | no generation data | no PP data | none |
| 52 | gpt-oss21B · 3.6B active OpenAI No generation data | 1 | no generation data | no PP data | none |
Not in the table: 6 reports don't say which model size ran. 130 ran on more than one device, which says nothing about what one of them does.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.