What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
NVIDIA 16 GB· NVIDIA only
Midrange NVIDIA cards. 14B comfortably, 30B-class mixture-of-experts with offload. Memory speed 224 to 960 GB/s across the 13 devices with reports, so a typical figure here spans all of them. 144 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly IQ4_XS Gen 24 t/s (8 runs) · PP 600 t/s (2 runs) · with speculation 56 t/s (17) | 47 | 24 8 runs, 5 devices | 600 2 runs | 56 17 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_S No generation data | 32 | no generation data | no PP data | none |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_XL No generation data | 9 | no generation data | no PP data | none |
| 4 | Qwen3.8 Swift-1.527B Alibaba · mostly IQ3_S Gen 53 t/s (1 run) · with speculation 88 t/s (2) | 6 | 53 1 run | no PP data | 88 2 runs |
| 5 | Gemma 426B · 4B active Google DeepMind · mostly IQ4_XS Gen 116 t/s (2 runs) · PP 4714 t/s (2 runs) | 5 | 116 2 runs, 2 devices | 4714 2 runs | none |
| 6 | Qwen3.627B Alibaba · mostly IQ4_XS Gen 22 t/s (1 run) · PP 760 t/s (1 run) | 3 | 22 1 run | 760 1 run | none |
| 7 | Qwen330B · 3B active Alibaba · mostly Q4_K_M No generation data | 3 | no generation data | no PP data | none |
| 8 | Qwen2.5 Coder-7B7B Alibaba · mostly Q4 Gen 124 t/s (2 runs) · PP 9826 t/s (1 run) | 2 | 124 2 runs, 2 devices | 9826 1 run | none |
| 9 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M Gen 74 t/s (1 run) | 2 | 74 1 run | no PP data | none |
| 10 | Bonsai 227B PrismML · mostly PQ2 Gen 57 t/s (1 run) | 2 | 57 1 run | no PP data | none |
| 11 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly UD-Q2_K_XL No generation data | 2 | no generation data | no PP data | none |
| 12 | Gemma 431B Google DeepMind · mostly NVFP4 No generation data · with speculation 67 t/s (1) | 2 | no generation data | no PP data | 67 1 run |
| 13 | Llama 3.23B Meta · mostly Q4_K_M Gen 299 t/s (1 run) · PP 19397 t/s (1 run) | 1 | 299 1 run | 19397 1 run | none |
| 14 | gpt-oss21B · 3.6B active OpenAI · mostly MXFP4 Gen 178 t/s (1 run) · PP 10329 t/s (1 run) | 1 | 178 1 run | 10329 1 run | none |
| 15 | Qwen2.57B Alibaba · mostly Q4_K_M Gen 177 t/s (1 run) · PP 9984 t/s (1 run) | 1 | 177 1 run | 9984 1 run | none |
| 16 | Qwen38B Alibaba · mostly Q4_K_M Gen 131 t/s (1 run) | 1 | 131 1 run | no PP data | none |
| 17 | Qwen3-VL8B Gen 127 t/s (1 run) | 1 | 127 1 run | no PP data | none |
| 18 | Qwen2.514B Alibaba · mostly Q4_K_M Gen 95 t/s (1 run) · PP 4865 t/s (1 run) | 1 | 95 1 run | 4865 1 run | none |
| 19 | Qwen2.5 Coder14B Alibaba · mostly Q4_K_M Gen 93 t/s (1 run) · PP 4842 t/s (1 run) | 1 | 93 1 run | 4842 1 run | none |
| 20 | MiniCPM-V 4.735B · 3B active mostly mix_q3gud_im Gen 93 t/s (1 run) | 1 | 93 1 run | no PP data | none |
| 21 | Qwen3.8 Escha-W227B Alibaba · mostly 2.469 bpw Gen 59 t/s (1 run) | 1 | 59 1 run | no PP data | none |
| 22 | Mimo 2.6 Distill-Qwen9B Xiaomi · mostly Q8_0 Gen 42 t/s (1 run) · PP 1718 t/s (1 run) | 1 | 42 1 run | 1718 1 run | none |
| 23 | Qwen3.6 APEX-I-Quality35B · 3B active Alibaba Gen 37 t/s (1 run) | 1 | 37 1 run | no PP data | none |
| 24 | Muse Glimmer30B Meta · mostly Q4_K_XL Gen 18 t/s (1 run) | 1 | 18 1 run | no PP data | none |
| 25 | Qwen3.59B Alibaba · mostly Q6_K Gen 4.6 t/s (1 run) · PP 53 t/s (1 run) | 1 | 4.6 1 run | 53 1 run | none |
| 26 | Agents-A1 Uncensored-MTP-APEX35B · 3B active InternScience · mostly APEX Compact No generation data · with speculation 97 t/s (1) | 1 | no generation data | no PP data | 97 1 run |
| 27 | DeepSeek V4.1 Flash552B · 16B active mostly 1.75 bpw No generation data | 1 | no generation data | no PP data | none |
| 28 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 29 | Gemma 412B Google DeepMind · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 30 | Ornith1.535B · 3B active mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 31 | Poolside Laguna S 2.1118B · 8B active mostly IQ3_S No generation data | 1 | no generation data | no PP data | none |
| 32 | Qwen2.50.5B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 33 | Qwen3-Coder30B · 3B active mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 34 | Qwen3-Next80B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 35 | Qwen3.535B · 3B active Alibaba · mostly UD-IQ3_XXS No generation data · PP 1168 t/s (1 run) | 1 | no generation data | 1168 1 run | none |
| 36 | Qwen3.8 Heretic27B Alibaba · mostly UD-IQ4_XS No generation data · with speculation 55 t/s (1) | 1 | no generation data | no PP data | 55 1 run |
| 37 | Qwen3.8 abliterated27B Alibaba · mostly IQ3_S No generation data · with speculation 105 t/s (1) | 1 | no generation data | no PP data | 105 1 run |
| 38 | Qwen3.8 Uncensored27B Alibaba · mostly IQ3_XXS No generation data · with speculation 35 t/s (1) | 1 | no generation data | no PP data | 35 1 run |
| 39 | Qwen3.8 OrcaRouter Uncensored27B Alibaba · mostly IQ3_XXS No generation data · with speculation 83 t/s (1) | 1 | no generation data | no PP data | 83 1 run |
| 40 | Qwen3.8 TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX27B Alibaba No generation data · with speculation 30 t/s (1) | 1 | no generation data | no PP data | 30 1 run |
| 41 | Raon-1 mini-preview29B · 4B active mostly IQ3_KS No generation data | 1 | no generation data | no PP data | none |
Not in the table: 10 reports don't say which model size ran. 85 ran on more than one device, which says nothing about what one of them does.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.