What people run, by hardware
Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.
To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.
A Mac counts when its report names the product, or when only that product was sold with its chip and memory.
Apple Silicon 128 GB+· MacBook Pro only
Max and Ultra Macs with room for large mixture-of-experts models. Memory speed 410 to 614 GB/s across the 3 devices with reports, so a typical figure here spans all of them. 23 single-device reports.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.635B · 3B active Alibaba · mostly 4-bit Gen 79 t/s (1 run) · PP 2153 t/s (1 run) · with speculation 99 t/s (1) | 5 | 79 1 run | 2153 1 run | 99 1 run |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_XXS No generation data · with speculation 12 t/s (1) | 4 | no generation data | no PP data | 12 1 run |
| 3 | Qwen3.827B Alibaba · mostly 8-bit No generation data · with speculation 41 t/s (2) | 3 | no generation data | no PP data | 41 2 runs |
| 4 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly 2.44bit-mixed Gen 31 t/s (2 runs) · PP 344 t/s (2 runs) | 2 | 31 2 runs | 344 2 runs | none |
| 5 | Qwen3.627B Alibaba · mostly Q8 Gen 17 t/s (1 run) · PP 433 t/s (2 runs) | 2 | 17 1 run | 433 2 runs | none |
| 6 | Nemotron-Labs Dory 1.04B mostly 8-bit Gen 101 t/s (1 run) | 1 | 101 1 run | no PP data | none |
| 7 | Qwen3.535B · 3B active Alibaba · mostly 4-bit Gen 91 t/s (1 run) | 1 | 91 1 run | no PP data | none |
| 8 | Nanbeige4.333B mostly 4-bit Gen 29 t/s (1 run) · PP 757 t/s (1 run) | 1 | 29 1 run | 757 1 run | none |
| 9 | DeepSeek V3.1 Terminus671B mostly INT4 No generation data | 1 | no generation data | no PP data | none |
| 10 | Gemma 431B Google DeepMind No generation data · with speculation 7.5 t/s (1) | 1 | no generation data | no PP data | 7.5 1 run |
| 11 | Kimi K32800B · 104B active Moonshot AI · mostly 4-bit No generation data | 1 | no generation data | no PP data | none |
| 12 | Qwen3.5122B · 10B active Alibaba · mostly WinterMix58 No generation data | 1 | no generation data | no PP data | none |
How this list works
Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.
Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.
Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.