What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
A Mac counts when its report names the product, or when only that product was sold with its chip and memory.
Apple Silicon 48-96 GB· Mac mini only
70B-class models fit at 4 bits. Memory speed 273 to 307 GB/s across the 4 devices with reports, so a typical figure here spans all of them. 43 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly UD-IQ4_XS Gen 13 t/s (4 runs) · PP 126 t/s (4 runs) · with speculation 35 t/s (1) | 8 | 13 4 runs | 126 4 runs | 35 1 run |
| 2 | gpt-oss21B · 3.6B active OpenAI · mostly MXFP4 Gen 74 t/s (3 runs) · PP 834 t/s (3 runs) | 4 | 74 3 runs | 834 3 runs | none |
| 3 | Qwen3.635B · 3B active Alibaba · mostly UD-Q4_K_M Gen 58 t/s (2 runs) · PP 767 t/s (2 runs) · with speculation 90 t/s (1) | 4 | 58 2 runs | 767 2 runs | 90 1 run |
| 4 | Gemma 426B · 4B active Google DeepMind · mostly UD-Q4_K_M Gen 57 t/s (2 runs) · PP 775 t/s (2 runs) | 3 | 57 2 runs | 775 2 runs | none |
| 5 | Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 86 t/s (2 runs) · PP 793 t/s (2 runs) | 2 | 86 2 runs | 793 2 runs | none |
| 6 | Gemma 4 E2B5.1B Google DeepMind · mostly Q8_0 Gen 82 t/s (2 runs) · PP 1552 t/s (2 runs) | 2 | 82 2 runs | 1552 2 runs | none |
| 7 | Qwen2.5-Omni3B mostly Q8_0 Gen 69 t/s (2 runs) · PP 1202 t/s (2 runs) | 2 | 69 2 runs | 1202 2 runs | none |
| 8 | Qwen3-VL4B mostly Q8_0 Gen 54 t/s (2 runs) · PP 918 t/s (2 runs) | 2 | 54 2 runs | 918 2 runs | none |
| 9 | Gemma 4 E4B8B Google DeepMind · mostly Q8_0 Gen 44 t/s (2 runs) · PP 778 t/s (2 runs) | 2 | 44 2 runs | 778 2 runs | none |
| 10 | Gemma 412B Google DeepMind · mostly Q4_K_M Gen 30 t/s (2 runs) · PP 292 t/s (2 runs) | 2 | 30 2 runs | 292 2 runs | none |
| 11 | Mistral Small 3.124B mostly Q4_K_M Gen 18 t/s (2 runs) · PP 145 t/s (2 runs) | 2 | 18 2 runs | 145 2 runs | none |
| 12 | Qwen3.627B Alibaba · mostly Q4_K_M Gen 13 t/s (2 runs) · PP 125 t/s (2 runs) | 2 | 13 2 runs | 125 2 runs | none |
| 13 | Qwen3.59B Alibaba · mostly 4-bit No generation data · with speculation 129 t/s (2) | 2 | no generation data | no PP data | 129 2 runs |
| 14 | Qwen3 30B-A3B, standard 8 experts30B · 3.3B active Alibaba · mostly MLX 4-bit affine, group 64 Gen 98 t/s (1 run) | 1 | 98 1 run | no PP data | none |
| 15 | Phi-3.5 mini3.8B Microsoft · mostly 4bit Gen 72 t/s (1 run) | 1 | 72 1 run | no PP data | none |
| 16 | Qwen3.535B · 3B active Alibaba · mostly 8-bit Gen 49 t/s (1 run) | 1 | 49 1 run | no PP data | none |
| 17 | Bonsai 2 Ternary-Bonsai-2-27B27B PrismML · mostly PQ2_0 Gen 21 t/s (1 run) · PP 127 t/s (1 run) | 1 | 21 1 run | 127 1 run | none |
| 18 | Qwen3-Coder-Next80B · 3B active mostly 6-bit No generation data | 1 | no generation data | no PP data | none |
| 19 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly 4bit No generation data | 1 | no generation data | no PP data | none |
Not in the table: 3 reports don't say which model size ran.
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.