What people run, by hardware

Pick the kind of machine you have. You'll see the model sizes people report running on it, most reported first, with the speed they typically get. It tells you what works on hardware like yours. It doesn't tell you which model gives the best answers.

To check a specific model on your exact card, use the calculator. For coding, see the coding models by VRAM.

Maker

All NVIDIA AMD Intel Apple

NVIDIA

NVIDIA ≤12GB

RTX 3060 12GB, 4070, 5070, laptops

NVIDIA 16GB

RTX 4060 Ti 16GB, 5060 Ti, 5070 Ti, 5080

NVIDIA 24GB

RTX 3090, 4090, 3080 20GB

NVIDIA 32-48GB

RTX 5090, RTX A6000, V100 32GB

NVIDIA 80GB+

RTX Pro 6000, H100, H200, A100 80GB

Apple Silicon

Apple ≤16GB

M1-M5 8-16GB, MacBook Air

Apple 18-36GB

M3/M4 Pro, M4 24GB, M-Max 36GB

Apple 48-96GB

M-Pro/M-Max 48-96GB

Apple 128GB+

M-Max 128GB, M-Ultra 192-512GB

Unified-memory PCs

iGPU & boards

Radeon 780M, Jetson Orin

Strix Halo & Spark

AMD Ryzen AI Max+ 395, NVIDIA DGX Spark

AMD and Intel

AMD/Intel ≤16GB

RX 9070, 7600 XT, Arc A770

AMD/Intel 20-32GB

RX 7900 XTX, AI PRO R9700, MI50 32GB, Arc Pro B70

AMD/Intel 48GB+

Instinct MI300X

No GPU

CPU only

Threadripper, EPYC, desktop CPUs

Model size

All sizes Small (<14B)Mid (14-40B)Large (>40B)

Apple Silicon 48-96 GB

70B-class models fit at 4 bits. Memory speed 273 to 1,229 GB/s across the 18 devices with reports, so a typical figure here spans all of them. 197 single-device reports.

#	Model	Reports	Gen t/s	PP t/s	With speculation
1	Qwen3.827B Alibaba · mostly Q4_K_M Gen 14 t/s (8 runs) · PP 126 t/s (5 runs) · with speculation 34 t/s (16)	32	14 8 runs, 5 devices	126 5 runs	34 16 runs
2	Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly oQ4e Gen 42 t/s (3 runs) · PP 621 t/s (3 runs) · with speculation 67 t/s (4)	24	42 3 runs, 2 devices	621 3 runs	67 4 runs
3	Qwen3.635B · 3B active Alibaba · mostly 4bit Gen 76 t/s (7 runs) · PP 787 t/s (5 runs) · with speculation 121 t/s (5)	14	76 7 runs, 4 devices	787 5 runs	121 5 runs
4	Qwen3.627B Alibaba · mostly Q4_K_M Gen 13 t/s (4 runs) · PP 124 t/s (3 runs) · with speculation 23 t/s (4)	9	13 4 runs, 3 devices	124 3 runs	23 4 runs
5	Gemma 426B · 4B active Google DeepMind · mostly UD-Q4_K_M Gen 72 t/s (6 runs) · PP 731 t/s (5 runs)	8	72 6 runs, 5 devices	731 5 runs	none
6	DeepSeek V4 Flash284B · 13B active DeepSeek · mostly IQ3-XXS Gen 13 t/s (1 run)	8	13 1 run	no PP data	none
7	Qwen3.59B Alibaba · mostly Q4_K_M Gen 66 t/s (3 runs) · PP 321 t/s (3 runs) · with speculation 129 t/s (2)	7	66 3 runs, 3 devices	321 3 runs	129 2 runs
8	Qwen3.535B · 3B active Alibaba · mostly 4-bit Gen 89 t/s (5 runs) · PP 1789 t/s (4 runs)	5	89 5 runs, 4 devices	1789 4 runs	none
9	gpt-oss21B · 3.6B active OpenAI · mostly MXFP4 Gen 78 t/s (4 runs) · PP 768 t/s (4 runs)	5	78 4 runs, 2 devices	768 4 runs	none
10	Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 95 t/s (3 runs) · PP 806 t/s (3 runs)	4	95 3 runs, 2 devices	806 3 runs	none
11	Qwen3.54B Alibaba · mostly Q4_K_M Gen 85 t/s (3 runs) · PP 4100 t/s (1 run)	4	85 3 runs, 3 devices	4100 1 run	none
12	Gemma 431B Google DeepMind · mostly Q4_K_M Gen 24 t/s (2 runs) · PP 345 t/s (2 runs)	4	24 2 runs, 2 devices	345 2 runs	none
13	Qwen38B Alibaba · mostly 4-bit Gen 98 t/s (3 runs) · PP 2948 t/s (2 runs)	3	98 3 runs, 2 devices	2948 2 runs	none
14	Gemma 4 E2B5.1B Google DeepMind · mostly Q8_0 Gen 87 t/s (3 runs) · PP 1559 t/s (3 runs)	3	87 3 runs, 2 devices	1559 3 runs	none
15	Gemma 312B Google · mostly 4-bit Gen 64 t/s (2 runs) · PP 1778 t/s (2 runs)	3	64 2 runs	1778 2 runs	none
16	Gemma 4 E4B8B Google DeepMind · mostly Q8_0 Gen 46 t/s (3 runs) · PP 787 t/s (3 runs)	3	46 3 runs, 2 devices	787 3 runs	none
17	Qwen 2.5 Coder32B mostly 4-bit Gen 26 t/s (3 runs) · PP 685 t/s (3 runs)	3	26 3 runs, 2 devices	685 3 runs	none
18	Qwen3.527B Alibaba · mostly 4-bit (source config) Gen 25 t/s (2 runs) · PP 461 t/s (3 runs)	3	25 2 runs, 2 devices	461 3 runs	none
19	Qwen30.6B Alibaba · mostly 4-bit Gen 514 t/s (2 runs) · PP 22991 t/s (2 runs)	2	514 2 runs	22991 2 runs	none
20	Llama 3.21B Meta · mostly 4-bit Gen 440 t/s (1 run) · PP 16968 t/s (1 run)	2	440 1 run	16968 1 run	none
21	Qwen31.7B Alibaba · mostly 4-bit Gen 327 t/s (2 runs) · PP 11928 t/s (2 runs)	2	327 2 runs	11928 2 runs	none
22	Gemma 31B Google · mostly 4-bit Gen 295 t/s (1 run) · PP 16750 t/s (1 run)	2	295 1 run	16750 1 run	none
23	Llama 3.23B Meta · mostly 4-bit Gen 200 t/s (1 run) · PP 6123 t/s (1 run)	2	200 1 run	6123 1 run	none
24	Qwen34B Alibaba · mostly 4-bit Gen 173 t/s (2 runs) · PP 5239 t/s (2 runs)	2	173 2 runs	5239 2 runs	none
25	Gemma 34B Google · mostly 4-bit Gen 163 t/s (2 runs) · PP 5656 t/s (2 runs)	2	163 2 runs	5656 2 runs	none
26	Qwen 2.5 Coder7B mostly 4-bit Gen 114 t/s (2 runs) · PP 3132 t/s (2 runs)	2	114 2 runs	3132 2 runs	none
27	Qwen2.5-Omni3B mostly Q8_0 Gen 69 t/s (2 runs) · PP 1202 t/s (2 runs)	2	69 2 runs	1202 2 runs	none
28	Qwen314B Alibaba · mostly 4-bit Gen 60 t/s (2 runs) · PP 1661 t/s (2 runs)	2	60 2 runs	1661 2 runs	none
29	Qwen 2.5 Coder14B mostly 4-bit Gen 58 t/s (2 runs) · PP 1633 t/s (2 runs)	2	58 2 runs	1633 2 runs	none
30	Qwen3-VL4B mostly Q8_0 Gen 54 t/s (2 runs) · PP 918 t/s (2 runs)	2	54 2 runs	918 2 runs	none
31	Gemma 412B Google DeepMind · mostly Q4_K_M Gen 30 t/s (2 runs) · PP 292 t/s (2 runs)	2	30 2 runs	292 2 runs	none
32	Gemma 327B Google · mostly 4-bit Gen 30 t/s (1 run) · PP 751 t/s (1 run)	2	30 1 run	751 1 run	none
33	Qwen332B Alibaba · mostly 4-bit Gen 27 t/s (2 runs) · PP 699 t/s (2 runs)	2	27 2 runs	699 2 runs	none
34	Mistral Small 3.124B mostly Q4_K_M Gen 18 t/s (2 runs) · PP 145 t/s (2 runs)	2	18 2 runs	145 2 runs	none
35	Kimi K32800B · 104B active Moonshot AI · mostly VQ3R No generation data	2	no generation data	no PP data	none
36	Qwen3.8 Swift-1.527B Alibaba · mostly 4.7bpw No generation data · with speculation 35 t/s (1)	2	no generation data	no PP data	35 1 run
37	Qwen3.8 Uncensored27B Alibaba · mostly 4-bit No generation data · with speculation 59 t/s (2)	2	no generation data	no PP data	59 2 runs
38	Ministral 33B mostly 4-bit (source config) Gen 128 t/s (1 run) · PP 743 t/s (1 run)	1	128 1 run	743 1 run	none
39	LFM224B · 2B active mostly 4-bit mixed (config) Gen 118 t/s (1 run) · PP 616 t/s (1 run)	1	118 1 run	616 1 run	none
40	Qwen3 30B-A3B, standard 8 experts30B · 3.3B active Alibaba · mostly MLX 4-bit affine, group 64 Gen 98 t/s (1 run)	1	98 1 run	no PP data	none
41	Qwen2.57B Alibaba · mostly 4-bit (source config) Gen 77 t/s (1 run) · PP 603 t/s (1 run)	1	77 1 run	603 1 run	none
42	gpt-oss117B · 5.1B active OpenAI · mostly MXFP4 Gen 73 t/s (1 run) · PP 347 t/s (1 run)	1	73 1 run	347 1 run	none
43	Phi-3.5 mini3.8B Microsoft · mostly 4bit Gen 72 t/s (1 run)	1	72 1 run	no PP data	none
44	DeepSeek V4 Flash 9B distill9B DeepSeek · mostly Q4_K_M Gen 44 t/s (1 run)	1	44 1 run	no PP data	none
45	Qwen3.8 27B dense27B Alibaba · mostly Q4_K_M Gen 21 t/s (1 run) · PP 144 t/s (1 run)	1	21 1 run	144 1 run	none
46	Bonsai 2 Ternary-Bonsai-2-27B27B PrismML · mostly PQ2_0 Gen 21 t/s (1 run) · PP 127 t/s (1 run)	1	21 1 run	127 1 run	none
47	Muse Glimmer30B Meta · mostly 8-bit Gen 8.2 t/s (1 run)	1	8.2 1 run	no PP data	none
48	DeepSeek V4.1 Flash552B · 16B active No generation data	1	no generation data	no PP data	none
49	Qwen3-Coder30B · 3B active mostly Q4_K_M No generation data	1	no generation data	no PP data	none
50	Qwen3-Coder-Next80B · 3B active mostly 6-bit No generation data	1	no generation data	no PP data	none
51	Qwen3-Next80B · 3B active Alibaba · mostly 4-bit No generation data	1	no generation data	no PP data	none
52	Qwen3.8 Qwen3.8-27B27B Alibaba · mostly Q4_K_M No generation data	1	no generation data	no PP data	none
53	Qwen3.8 Splash-HQ27B Alibaba · mostly Q8 No generation data · with speculation 37 t/s (1)	1	no generation data	no PP data	37 1 run
54	Qwen3.8 Swift-1.5 Flash-Next125B · 6B active Alibaba · mostly IQ3_S No generation data · with speculation 145 t/s (1)	1	no generation data	no PP data	145 1 run

Not in the table: 70 reports don't say which model size ran.

How this list works

Each row is one model size, because a 27B and a 397B from the same family are different choices. Rows are ordered by how many people report running that size on this hardware.

Generation speed is the median of runs on one device, serving one request, without speculative decoding. PP is the median prompt-processing speed from plain runs that reported it, so its run count can be smaller. Speculative decoding (MTP, a draft model, DFlash, n-gram lookup) can make the same model several times faster, so those runs get their own column. A report that doesn't make clear which kind of run it was counts in neither.

Hardware is grouped by how much it holds and how fast its memory is, since memory speed sets how fast a model writes. Click a model for every report behind its row.