What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
NVIDIA 6-12 GB
Entry-level NVIDIA cards. 7B-14B models at 4 bits. Memory speed 160 to 1,493 GB/s across the 26 devices with reports, so a typical figure here spans all of them. 115 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_XXS No generation data | 24 | no generation data | no PP data | none |
| 2 | Qwen3.827B Alibaba · mostly IQ3_XXS Gen 30 t/s (3 runs) · PP 943 t/s (1 run) · with speculation 29 t/s (3) | 19 | 30 3 runs, 3 devices | 943 1 run | 29 3 runs |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_M No generation data | 19 | no generation data | no PP data | none |
| 4 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M Gen 50 t/s (2 runs) | 4 | 50 2 runs, 2 devices | no PP data | none |
| 5 | Bonsai 227B PrismML · mostly PTQ1_0 Gen 29 t/s (3 runs) · PP 614 t/s (2 runs) | 3 | 29 3 runs, 3 devices | 614 2 runs | none |
| 6 | DeepSeek V4.1 Flash552B · 16B active mostly 2-bit No generation data | 3 | no generation data | no PP data | none |
| 7 | Ornith1.535B · 3B active mostly Q4_K_M No generation data | 3 | no generation data | no PP data | none |
| 8 | LFM31.7B mostly Q8 Gen 91 t/s (1 run) · PP 5258 t/s (1 run) | 2 | 91 1 run | 5258 1 run | none |
| 9 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4_K_M Gen 70 t/s (2 runs) | 2 | 70 2 runs, 2 devices | no PP data | none |
| 10 | Qwen2.57B Alibaba · mostly Q4_K_M Gen 39 t/s (2 runs) | 2 | 39 2 runs, 2 devices | no PP data | none |
| 11 | Qwen3.59B Alibaba · mostly Q4_K_M Gen 38 t/s (1 run) · PP 1829 t/s (1 run) | 2 | 38 1 run | 1829 1 run | none |
| 12 | Gemma 412B Google DeepMind · mostly Q6_K Gen 28 t/s (1 run) | 2 | 28 1 run | no PP data | none |
| 13 | Qwen3.627B Alibaba · mostly Q2-XS Gen 24 t/s (1 run) | 2 | 24 1 run | no PP data | none |
| 14 | Qwen2.5 MoE-A0.9B1.5B · 0.9B active Alibaba · mostly Q4_K_M Gen 253 t/s (1 run) | 1 | 253 1 run | no PP data | none |
| 15 | LFM2.51.2B Liquid AI Gen 129 t/s (1 run) · PP 8500 t/s (1 run) | 1 | 129 1 run | 8500 1 run | none |
| 16 | Qwen30.6B Alibaba Gen 76 t/s (1 run) | 1 | 76 1 run | no PP data | none |
| 17 | Llama 27B mostly Q4_0 Gen 62 t/s (1 run) · PP 1084 t/s (1 run) | 1 | 62 1 run | 1084 1 run | none |
| 18 | Ling-3.0 Tiny7.9B · 1.3B active Ant Group · mostly Q4_K_M Gen 54 t/s (1 run) · PP 245 t/s (1 run) | 1 | 54 1 run | 245 1 run | none |
| 19 | GLM-4.7 Flash30B Zhipu AI Gen 43 t/s (1 run) | 1 | 43 1 run | no PP data | none |
| 20 | KAT-Coder V2.5-Dev35B · 3B active Kwaipilot Gen 14 t/s (1 run) | 1 | 14 1 run | no PP data | none |
| 21 | Bonsai27B PrismML · mostly 1-bit No generation data | 1 | no generation data | no PP data | none |
| 22 | Bonsai 2 Swift Katana27B PrismML · mostly PTQ1_0 No generation data | 1 | no generation data | no PP data | none |
| 23 | DeepSeek V4 Flash REAP-150B150B DeepSeek · mostly FP4 No generation data | 1 | no generation data | no PP data | none |
| 24 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly Q4 No generation data | 1 | no generation data | no PP data | none |
| 25 | Gemma 4 Uncensored26B · 4B active Google DeepMind No generation data | 1 | no generation data | no PP data | none |
| 26 | Gemma 4 Ultra-Uncensored-Heretic26B · 4B active Google DeepMind · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 27 | Gemma 426B Google DeepMind No generation data | 1 | no generation data | no PP data | none |
| 28 | Gemma 431B Google DeepMind · mostly Q5_K_M No generation data | 1 | no generation data | no PP data | none |
| 29 | Ornith1.59B No generation data · with speculation 15 t/s (1) | 1 | no generation data | no PP data | 15 1 run |
| 30 | Ornith1.5122B · 10B active mostly UD-IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 31 | Qwen330B · 3B active Alibaba · mostly Q4_K_S No generation data | 1 | no generation data | no PP data | none |
| 32 | Qwen3.54B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 33 | Qwen3.513B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 34 | Qwen3.535B · 3B active Alibaba · mostly Q2 No generation data | 1 | no generation data | no PP data | none |
| 35 | Qwen3.6 HauhauCS uncensored Aggressive35B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 36 | Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active Alibaba · mostly Q3_K_P No generation data | 1 | no generation data | no PP data | none |
| 37 | Qwen3.89B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 38 | Qwen3.8 L0xRE27B Alibaba · mostly IQ3 No generation data · with speculation 40 t/s (1) | 1 | no generation data | no PP data | 40 1 run |
| 39 | Qwen3.8 Swift-1.527B Alibaba · mostly IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 40 | Qwen3.8 Uncensored27B Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 41 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ2_XS No generation data | 1 | no generation data | no PP data | none |
Not in the table: 9 reports don't say which model size ran. 46 ran on more than one device, which says nothing about what one of them does.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.