What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
CPU only
Runs with no GPU at all. Speed follows system memory bandwidth. 54 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.635B · 3B active Alibaba · mostly IQ2_XXS Gen 6.5 t/s (4 runs) · PP 118 t/s (1 run) | 5 | 6.5 4 runs | 118 1 run | none |
| 2 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4 Gen 5.5 t/s (3 runs) · PP 349 t/s (2 runs) | 4 | 5.5 3 runs | 349 2 runs | none |
| 3 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_XXS Gen 31 t/s (2 runs) · PP 1563 t/s (1 run) · with speculation 7.8 t/s (1) | 3 | 31 2 runs | 1563 1 run | 7.8 1 run |
| 4 | Qwen3.827B Alibaba · mostly IQS3_XXS Gen 2.5 t/s (3 runs) | 3 | 2.5 3 runs | no PP data | none |
| 5 | Qwen3.50.8B Alibaba · mostly BF16 Gen 123 t/s (1 run) · PP 2758 t/s (1 run) | 2 | 123 1 run | 2758 1 run | none |
| 6 | Bonsai1.7B PrismML Gen 41 t/s (2 runs) | 2 | 41 2 runs | no PP data | none |
| 7 | Llama 3.21B Meta · mostly Q4_K_M Gen 15 t/s (2 runs) · PP 75 t/s (2 runs) | 2 | 15 2 runs | 75 2 runs | none |
| 8 | DeepSeek R1671B · 37B active DeepSeek · mostly Q2_K_XL Gen 7.1 t/s (2 runs) | 2 | 7.1 2 runs | no PP data | none |
| 9 | Qwen2.50.5B Alibaba · mostly Q4_0 Gen 5.8 t/s (2 runs) | 2 | 5.8 2 runs | no PP data | none |
| 10 | Bonsai 2 Ternary27B PrismML · mostly 2.06 bpw Gen 4.2 t/s (2 runs) | 2 | 4.2 2 runs | no PP data | none |
| 11 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly MXFP4 Gen 3.6 t/s (2 runs) | 2 | 3.6 2 runs | no PP data | none |
| 12 | Gemma 30.3B Google · mostly ternary No generation data | 2 | no generation data | no PP data | none |
| 13 | Llama 3.18B Meta · mostly 1.58-bit ternary No generation data · PP 370 t/s (1 run) · with speculation 12 t/s (1) | 2 | no generation data | 370 1 run | 12 1 run |
| 14 | LFM2.50.2B Liquid AI · mostly Q4_0 Gen 192 t/s (1 run) · PP 1611 t/s (1 run) | 1 | 192 1 run | 1611 1 run | none |
| 15 | Qwen3.535B · 3B active Alibaba Gen 40 t/s (1 run) | 1 | 40 1 run | no PP data | none |
| 16 | Qwen3-Next80B · 3B active Alibaba · mostly Q4_K_M Gen 19 t/s (1 run) · PP 130 t/s (1 run) | 1 | 19 1 run | 130 1 run | none |
| 17 | LFM2.52.6B Liquid AI · mostly Q4_K_M Gen 17 t/s (1 run) | 1 | 17 1 run | no PP data | none |
| 18 | DeepSeek V2 Lite16B · 2.4B active mostly Q4_K_S Gen 14 t/s (1 run) | 1 | 14 1 run | no PP data | none |
| 19 | Ling-3.0 Tiny7.9B · 1.3B active Ant Group · mostly Q6 Gen 10 t/s (1 run) | 1 | 10 1 run | no PP data | none |
| 20 | Gemma 29B Gen 8.2 t/s (1 run) | 1 | 8.2 1 run | no PP data | none |
| 21 | Gemma 426B · 4B active Google DeepMind Gen 8.0 t/s (1 run) | 1 | 8.0 1 run | no PP data | none |
| 22 | Qwen3-Coder-Next80B · 3B active mostly Q4_K_M Gen 7.7 t/s (1 run) | 1 | 7.7 1 run | no PP data | none |
| 23 | Llama 3.23B Meta · mostly q4f16_1 Gen 6.1 t/s (1 run) · PP 8.8 t/s (1 run) | 1 | 6.1 1 run | 8.8 1 run | none |
| 24 | Gemma 31B Google Gen 4.5 t/s (1 run) | 1 | 4.5 1 run | no PP data | none |
| 25 | Llama 3.170B Meta · mostly F16 Gen 2.3 t/s (1 run) · PP 39 t/s (1 run) | 1 | 2.3 1 run | 39 1 run | none |
| 26 | Qwen 2.5 Coder14B mostly Q3_K_M Gen 1.2 t/s (1 run) | 1 | 1.2 1 run | no PP data | none |
| 27 | Qwen3.59B Alibaba Gen 1.0 t/s (1 run) | 1 | 1.0 1 run | no PP data | none |
| 28 | Bonsai Ternary-Bonsai1.7B PrismML · mostly PQ2_0 No generation data · PP 71 t/s (1 run) | 1 | no generation data | 71 1 run | none |
| 29 | DeepSeek V4.1 Flash552B · 16B active No generation data · with speculation 6.0 t/s (1) | 1 | no generation data | no PP data | 6.0 1 run |
| 30 | Falcon31B No generation data | 1 | no generation data | no PP data | none |
| 31 | Gemma 22B mostly FP16 No generation data | 1 | no generation data | no PP data | none |
| 32 | Gemma 4 E4B8B Google DeepMind · mostly Q8 No generation data | 1 | no generation data | no PP data | none |
| 33 | Qwen2.5 Coder-7B7B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 34 | Qwen2.5-Coder3B Alibaba No generation data | 1 | no generation data | no PP data | none |
Not in the table: 4 reports don't say which model size ran.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.