llamaperf

Best local LLMs for 32GB VRAM

How far 32GB of graphics memory goes for local models on cards like NVIDIA RTX 5090, AMD Radeon AI PRO R9700 32GB, NVIDIA V100 32GB or Intel Arc Pro B70: which models fit, and what people actually run on it.

Is 32GB of VRAM enough?

32GB holds a dense model of up to about 39B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.6 35B · 3B active, which needs about 26 GB at Q5_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 32GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

And 106 smaller models. The calculator lists them all for your card.

What people run on 32GB

Model sizes reported on setups with more than 24 GB and up to 32 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.827B
Alibaba · mostly NVFP4
Gen 37 t/s (26 runs) · PP 669 t/s (10 runs) · with speculation 140 t/s (47)
163
2Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly NVFP4
Gen 59 t/s (2 runs) · with speculation 390 t/s (1)
87
3Qwen3.635B · 3B active
Alibaba · mostly Q4_K_M
Gen 63 t/s (14 runs) · PP 977 t/s (9 runs)
23
4DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly Q8_K_XL
No generation data
10
5Gemma 426B · 4B active
Google DeepMind · mostly Q4_K_M
Gen 148 t/s (2 runs) · PP 8505 t/s (2 runs) · with speculation 578 t/s (1)
9
6Qwen3.627B
Alibaba · mostly NVFP4
Gen 39 t/s (2 runs) · PP 2023 t/s (1 run) · with speculation 77 t/s (2)
9
7DeepSeek V4.1 Flash552B · 16B active
mostly IQ2_XXS
No generation data
8
8Ornith1.535B · 3B active
mostly MXFP4
Gen 109 t/s (3 runs) · PP 9699 t/s (1 run) · with speculation 245 t/s (1)
7
9Qwen3.8 Flash-Next Uncensored125B · 6B active
Alibaba · mostly NVFP4
No generation data
6
10Qwen38B
Alibaba · mostly Q4_K_M
Gen 78 t/s (3 runs) · PP 1172 t/s (3 runs) · with speculation 159 t/s (1)
4
11Gemma 431B
Google DeepMind · mostly Q4_K_M
Gen 64 t/s (1 run)
4
12Qwen3.8 Swift-1.527B
Alibaba · mostly MXFP4
Gen 38 t/s (1 run) · with speculation 154 t/s (3)
4
13Qwen3.8 Swift27B
Alibaba · mostly Q4_K_XL
Gen 25 t/s (1 run) · PP 492 t/s (1 run)
4
14GLM-5.3 Flash320B · 18B active
Zhipu AI · mostly EXL3 3.05 bpw
No generation data
4
15Gemma 412B
Google DeepMind · mostly Q4_K_M
No generation data
4
16Muse Glimmer30B
Meta · mostly Q5_K_XL
No generation data · with speculation 243 t/s (2)
4
17Nemotron 3.5 Lightning30B · 3B active
NVIDIA · mostly NVFP4
No generation data
4
18Qwen3.535B · 3B active
Alibaba · mostly Q4_K_M
Gen 172 t/s (2 runs) · PP 3075 t/s (1 run)
3
19Qwen330B · 3B active
Alibaba · mostly Q4_K_M
Gen 79 t/s (3 runs) · PP 1063 t/s (1 run)
3
20LFM2.524B · 2.5B active
Liquid AI · mostly Q4_K_M
Gen 74 t/s (2 runs)
3
21Qwen3.59B
Alibaba · mostly Q4_K_M
No generation data
3
22Qwen3.8 Huihui-Abliterated27B
Alibaba · mostly NVFP4
No generation data · with speculation 201 t/s (3)
3
23Qwen3.8 Uncensored27B
Alibaba · mostly NVFP4
Gen 77 t/s (1 run) · with speculation 175 t/s (1)
2
24Qwen3.54B
Alibaba · mostly Q8_0
Gen 62 t/s (2 runs) · PP 2442 t/s (2 runs)
2
25DeepSeek V4.1 Flash763B
mostly IQ2_XXS
No generation data
2
26Gemma 4 E4B8B
Google DeepMind · mostly Q4_K_M
No generation data
2
27Qwen2.57B
Alibaba · mostly AWQ
No generation data
2
28LFM2.58B · 1B active
Liquid AI · mostly Q8_0
Gen 563 t/s (1 run) · PP 30681 t/s (1 run)
1
29Bonsai 227B
PrismML · mostly PQ2_0
Gen 101 t/s (1 run)
1
30Mellum2.1 Thinking12B · 2.5B active
mostly oQ4e
Gen 82 t/s (1 run)
1
31Qwen3-Coder30B · 3B active
mostly Q4_K_M
Gen 66 t/s (1 run)
1
32Qwen3 Thinking-250730B · 3B active
Alibaba · mostly Q4_0
Gen 63 t/s (1 run) · PP 1224 t/s (1 run)
1
33Qwen2.532B
Alibaba
Gen 37 t/s (1 run)
1
34Bonsai 2 Ternary27B
PrismML · mostly PTQ1_0
Gen 37 t/s (1 run) · PP 909 t/s (1 run)
1
35Qwen34B
Alibaba · mostly Q4_K_M
Gen 36 t/s (1 run) · PP 417 t/s (1 run)
1
36Qwen2.51.5B
Alibaba · mostly FP16
Gen 32 t/s (1 run)
1
37Qwen3.527B
Alibaba · mostly Q4_K_M
Gen 30 t/s (1 run) · PP 1516 t/s (1 run)
1
38Gemma 327B
Google · mostly 4-bit
Gen 8.9 t/s (1 run) · PP 11 t/s (1 run)
1
39Edge0145B · 10B active
mostly NVFP4
No generation data
1
40Gemma 4 E2B5.1B
Google DeepMind · mostly Q4_K_M
No generation data
1
41Inkling 2 Mini26B · 3.4B active
mostly UD-IQ4_XS
No generation data · with speculation 182 t/s (1)
1
42Kimi K2.51000B · 32B active
Moonshot AI · mostly IQ3_M
No generation data
1
43Kimi K2.61000B · 32B active
Moonshot AI
No generation data
1
44Nemotron 3.5 Super VL120B · 12B active
mostly NVFP4
No generation data
1
45Nemotron 3.5 Bolt74B · 9B active
mostly IQ2_XXS
No generation data · with speculation 343 t/s (1)
1
46Ornith1.5 Ornith1.5-35B-A3B Splash35B · 3B active
No generation data
1
47Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active
Alibaba · mostly Q4_K_P
No generation data
1
48Qwen3.8 Swift-1.5 Abliterated27B
Alibaba
No generation data · with speculation 152 t/s (1)
1
49Qwen3.8 Swift-Genesis27B
Alibaba
No generation data
1
50Qwen3.8 RVN Heretic (ARA abliterated)27B
Alibaba · mostly Q6_K
No generation data
1
51Qwen3.8 QUASAR27B
Alibaba · mostly NVFP4
No generation data · with speculation 287 t/s (1)
1
52Qwen3.8 Swift Uncensored27B
Alibaba · mostly Q6_K
No generation data
1
53Qwen3.8 Uncensored-HauhauCS-Aggressive27B
Alibaba · mostly Q5_K_P
No generation data · with speculation 14 t/s (1)
1
54Qwen3.8 Swift-Qwen3.8-27B Splash27B
Alibaba
No generation data
1
55Qwen3.8 Qwen3.8-27B Splash27B
Alibaba
No generation data
1
56Qwen3.8 RVN27B
Alibaba · mostly Q6_K
No generation data · with speculation 100 t/s (1)
1
57Qwen3.835B
Alibaba · mostly Q6
No generation data
1
58Qwen3.8 Swift-1.5-125B-A6B125B · 6B active
Alibaba · mostly IQ2_XS
No generation data
1
59Qwen3.8 Flash-Next abliterated-v2125B · 6B active
Alibaba · mostly UD-Q5_K_XL
No generation data
1
60Qwen3.82400B · 95B active
Alibaba · mostly UD-Q1_0
No generation data
1
61Qwopus3.6 Coder-Compat27B
mostly Q4_K_M
No generation data · with speculation 40 t/s (1)
1
62gpt-oss21B · 3.6B active
OpenAI
No generation data
1

Frequently asked

What do people run on 32GB of VRAM?

On setups with more than 24 GB and up to 32 GB of memory, the model people report most is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.

Does a Mac with 32GB count as 32GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 32GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.