What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
NVIDIA 32-48 GB· NVIDIA only
Single cards with room for 32B at higher precision or 70B at low bits. Memory speed 696 to 1,792 GB/s across the 9 devices with reports, so a typical figure here spans all of them. 169 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly NVFP4 Gen 72 t/s (16 runs) · PP 826 t/s (5 runs) · with speculation 148 t/s (33) | 67 | 72 16 runs, 2 devices | 826 5 runs | 148 33 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly NVFP4 Gen 59 t/s (2 runs) | 34 | 59 2 runs | no PP data | none |
| 3 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly Q8_K_XL No generation data | 10 | no generation data | no PP data | none |
| 4 | Qwen3.627B Alibaba · mostly NVFP4 Gen 39 t/s (2 runs) · PP 2023 t/s (1 run) · with speculation 77 t/s (2) | 6 | 39 2 runs, 2 devices | 2023 1 run | 77 2 runs |
| 5 | Qwen3.8 Flash-Next Uncensored125B · 6B active Alibaba · mostly NVFP4 No generation data | 5 | no generation data | no PP data | none |
| 6 | Muse Glimmer30B Meta · mostly Q5_K_XL No generation data · with speculation 243 t/s (2) | 4 | no generation data | no PP data | 243 2 runs |
| 7 | Qwen3.635B · 3B active Alibaba · mostly Q4 Gen 302 t/s (2 runs) · PP 5310 t/s (1 run) | 3 | 302 2 runs | 5310 1 run | none |
| 8 | Gemma 426B · 4B active Google DeepMind · mostly AWQ-4bit Gen 244 t/s (1 run) · PP 13809 t/s (1 run) · with speculation 578 t/s (1) | 3 | 244 1 run | 13809 1 run | 578 1 run |
| 9 | DeepSeek V4.1 Flash552B · 16B active mostly IQ2_XXS No generation data | 3 | no generation data | no PP data | none |
| 10 | Qwen3.8 Huihui-Abliterated27B Alibaba · mostly NVFP4 No generation data · with speculation 201 t/s (3) | 3 | no generation data | no PP data | 201 3 runs |
| 11 | Qwen38B Alibaba · mostly BF16 Gen 89 t/s (1 run) · PP 6700 t/s (1 run) · with speculation 159 t/s (1) | 2 | 89 1 run | 6700 1 run | 159 1 run |
| 12 | Qwen3.8 Uncensored27B Alibaba · mostly NVFP4 Gen 77 t/s (1 run) · with speculation 175 t/s (1) | 2 | 77 1 run | no PP data | 175 1 run |
| 13 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly EXL3 3.05 bpw No generation data | 2 | no generation data | no PP data | none |
| 14 | Qwen2.57B Alibaba No generation data | 2 | no generation data | no PP data | none |
| 15 | Qwen3.8 Swift27B Alibaba · mostly Q6_K No generation data | 2 | no generation data | no PP data | none |
| 16 | LFM2.58B · 1B active Liquid AI · mostly Q8_0 Gen 563 t/s (1 run) · PP 30681 t/s (1 run) | 1 | 563 1 run | 30681 1 run | none |
| 17 | Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 352 t/s (1 run) | 1 | 352 1 run | no PP data | none |
| 18 | Qwen3.535B · 3B active Alibaba · mostly Q4_K_XL Gen 194 t/s (1 run) | 1 | 194 1 run | no PP data | none |
| 19 | Qwen3.54B Alibaba · mostly Q8_0 Gen 107 t/s (1 run) · PP 4445 t/s (1 run) | 1 | 107 1 run | 4445 1 run | none |
| 20 | Bonsai 227B PrismML · mostly PQ2_0 Gen 101 t/s (1 run) | 1 | 101 1 run | no PP data | none |
| 21 | Gemma 431B Google DeepMind · mostly Q4_K_M Gen 64 t/s (1 run) | 1 | 64 1 run | no PP data | none |
| 22 | Qwen2.532B Alibaba Gen 37 t/s (1 run) | 1 | 37 1 run | no PP data | none |
| 23 | Bonsai 2 Ternary27B PrismML · mostly PTQ1_0 Gen 37 t/s (1 run) · PP 909 t/s (1 run) | 1 | 37 1 run | 909 1 run | none |
| 24 | Qwen3.527B Alibaba · mostly Q4_K_M Gen 30 t/s (1 run) · PP 1516 t/s (1 run) | 1 | 30 1 run | 1516 1 run | none |
| 25 | Gemma 4 E2B5.1B Google DeepMind · mostly bf16 Gen 17 t/s (1 run) | 1 | 17 1 run | no PP data | none |
| 26 | Gemma 412B Google DeepMind No generation data | 1 | no generation data | no PP data | none |
| 27 | Kimi K2.51000B · 32B active Moonshot AI · mostly IQ3_M No generation data | 1 | no generation data | no PP data | none |
| 28 | Mimo 2.6 Flash-RL309B · 15B active Xiaomi · mostly 2.20 bpw No generation data | 1 | no generation data | no PP data | none |
| 29 | Nemotron 3.5 Bolt74B · 9B active mostly IQ2_XXS No generation data · with speculation 343 t/s (1) | 1 | no generation data | no PP data | 343 1 run |
| 30 | Qwen3.59B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 31 | Qwen3.6 Ornith-Agents-A1-3.635B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 32 | Qwen3.8 RVN Heretic (ARA abliterated)27B Alibaba · mostly Q6_K No generation data | 1 | no generation data | no PP data | none |
| 33 | Qwen3.8 QUASAR27B Alibaba · mostly NVFP4 No generation data · with speculation 287 t/s (1) | 1 | no generation data | no PP data | 287 1 run |
| 34 | Qwen3.8 Swift-1.527B Alibaba · mostly NVFP4 No generation data · with speculation 161 t/s (1) | 1 | no generation data | no PP data | 161 1 run |
| 35 | Qwen3.8 RVN27B Alibaba · mostly Q6_K No generation data · with speculation 100 t/s (1) | 1 | no generation data | no PP data | 100 1 run |
| 36 | Qwopus3.6 Coder-Compat27B mostly Q4_K_M No generation data · with speculation 40 t/s (1) | 1 | no generation data | no PP data | 40 1 run |
Not in the table: 10 reports don't say which model size ran. 43 ran on more than one device, which says nothing about what one of them does.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.