Best local LLMs for 32GB VRAM
How far 32GB of graphics memory goes for local models on cards like NVIDIA RTX 5090, AMD Radeon AI PRO R9700 32GB, NVIDIA V100 32GB or Intel Arc Pro B70: which models fit, and what people actually run on it.
Is 32GB of VRAM enough?
32GB holds a dense model of up to about 39B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.6 35B · 3B active, which needs about 26 GB at Q5_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 32GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| Qwen3.6 35B · 3B active | Q5_K_M | 26 GB |
| Qwen3.5 35B · 3B active | Q5_K_M | 26 GB |
| Ornith1.5 35B · 3B active | Q5_K_M | 26 GB |
| MiniCPM-V 4.7 35B · 3B active | Q5_K_M | 26 GB |
| Agents-A1 35B · 3B active | Q5_K_M | 26 GB |
| KAT-Coder V2.5-Dev 35B · 3B active | Q5_K_M | 26 GB |
| Nex-N2.5-mini 35B · 3B active | Q5_K_M | 26 GB |
| Ornith 1.0 35B · 3B active | Q5_K_M | 26 GB |
| Qwen3.8 35B · 3B active | Q5_K_M | 26.4 GB |
| Qwen3.8 35B | Q5_K_M | 26.4 GB |
| Nanbeige4.3 33B | Q5_K_M | 26.6 GB |
| Qwen3 32B | Q5_K_M | 25.9 GB |
And 106 smaller models. The calculator lists them all for your card.
What people run on 32GB
Model sizes reported on setups with more than 24 GB and up to 32 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly NVFP4 Gen 37 t/s (26 runs) · PP 669 t/s (10 runs) · with speculation 140 t/s (47) | 163 | 37 26 runs, 7 devices | 669 10 runs | 140 47 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly NVFP4 Gen 59 t/s (2 runs) · with speculation 390 t/s (1) | 87 | 59 2 runs | no PP data | 390 1 run |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_M Gen 63 t/s (14 runs) · PP 977 t/s (9 runs) | 23 | 63 14 runs, 7 devices | 977 9 runs | none |
| 4 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly Q8_K_XL No generation data | 10 | no generation data | no PP data | none |
| 5 | Gemma 426B · 4B active Google DeepMind · mostly Q4_K_M Gen 148 t/s (2 runs) · PP 8505 t/s (2 runs) · with speculation 578 t/s (1) | 9 | 148 2 runs, 2 devices | 8505 2 runs | 578 1 run |
| 6 | Qwen3.627B Alibaba · mostly NVFP4 Gen 39 t/s (2 runs) · PP 2023 t/s (1 run) · with speculation 77 t/s (2) | 9 | 39 2 runs, 2 devices | 2023 1 run | 77 2 runs |
| 7 | DeepSeek V4.1 Flash552B · 16B active mostly IQ2_XXS No generation data | 8 | no generation data | no PP data | none |
| 8 | Ornith1.535B · 3B active mostly MXFP4 Gen 109 t/s (3 runs) · PP 9699 t/s (1 run) · with speculation 245 t/s (1) | 7 | 109 3 runs, 3 devices | 9699 1 run | 245 1 run |
| 9 | Qwen3.8 Flash-Next Uncensored125B · 6B active Alibaba · mostly NVFP4 No generation data | 6 | no generation data | no PP data | none |
| 10 | Qwen38B Alibaba · mostly Q4_K_M Gen 78 t/s (3 runs) · PP 1172 t/s (3 runs) · with speculation 159 t/s (1) | 4 | 78 3 runs, 3 devices | 1172 3 runs | 159 1 run |
| 11 | Gemma 431B Google DeepMind · mostly Q4_K_M Gen 64 t/s (1 run) | 4 | 64 1 run | no PP data | none |
| 12 | Qwen3.8 Swift-1.527B Alibaba · mostly MXFP4 Gen 38 t/s (1 run) · with speculation 154 t/s (3) | 4 | 38 1 run | no PP data | 154 3 runs |
| 13 | Qwen3.8 Swift27B Alibaba · mostly Q4_K_XL Gen 25 t/s (1 run) · PP 492 t/s (1 run) | 4 | 25 1 run | 492 1 run | none |
| 14 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly EXL3 3.05 bpw No generation data | 4 | no generation data | no PP data | none |
| 15 | Gemma 412B Google DeepMind · mostly Q4_K_M No generation data | 4 | no generation data | no PP data | none |
| 16 | Muse Glimmer30B Meta · mostly Q5_K_XL No generation data · with speculation 243 t/s (2) | 4 | no generation data | no PP data | 243 2 runs |
| 17 | Nemotron 3.5 Lightning30B · 3B active NVIDIA · mostly NVFP4 No generation data | 4 | no generation data | no PP data | none |
| 18 | Qwen3.535B · 3B active Alibaba · mostly Q4_K_M Gen 172 t/s (2 runs) · PP 3075 t/s (1 run) | 3 | 172 2 runs, 2 devices | 3075 1 run | none |
| 19 | Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 79 t/s (3 runs) · PP 1063 t/s (1 run) | 3 | 79 3 runs, 2 devices | 1063 1 run | none |
| 20 | LFM2.524B · 2.5B active Liquid AI · mostly Q4_K_M Gen 74 t/s (2 runs) | 3 | 74 2 runs, 2 devices | no PP data | none |
| 21 | Qwen3.59B Alibaba · mostly Q4_K_M No generation data | 3 | no generation data | no PP data | none |
| 22 | Qwen3.8 Huihui-Abliterated27B Alibaba · mostly NVFP4 No generation data · with speculation 201 t/s (3) | 3 | no generation data | no PP data | 201 3 runs |
| 23 | Qwen3.8 Uncensored27B Alibaba · mostly NVFP4 Gen 77 t/s (1 run) · with speculation 175 t/s (1) | 2 | 77 1 run | no PP data | 175 1 run |
| 24 | Qwen3.54B Alibaba · mostly Q8_0 Gen 62 t/s (2 runs) · PP 2442 t/s (2 runs) | 2 | 62 2 runs, 2 devices | 2442 2 runs | none |
| 25 | DeepSeek V4.1 Flash763B mostly IQ2_XXS No generation data | 2 | no generation data | no PP data | none |
| 26 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M No generation data | 2 | no generation data | no PP data | none |
| 27 | Qwen2.57B Alibaba · mostly AWQ No generation data | 2 | no generation data | no PP data | none |
| 28 | LFM2.58B · 1B active Liquid AI · mostly Q8_0 Gen 563 t/s (1 run) · PP 30681 t/s (1 run) | 1 | 563 1 run | 30681 1 run | none |
| 29 | Bonsai 227B PrismML · mostly PQ2_0 Gen 101 t/s (1 run) | 1 | 101 1 run | no PP data | none |
| 30 | Mellum2.1 Thinking12B · 2.5B active mostly oQ4e Gen 82 t/s (1 run) | 1 | 82 1 run | no PP data | none |
| 31 | Qwen3-Coder30B · 3B active mostly Q4_K_M Gen 66 t/s (1 run) | 1 | 66 1 run | no PP data | none |
| 32 | Qwen3 Thinking-250730B · 3B active Alibaba · mostly Q4_0 Gen 63 t/s (1 run) · PP 1224 t/s (1 run) | 1 | 63 1 run | 1224 1 run | none |
| 33 | Qwen2.532B Alibaba Gen 37 t/s (1 run) | 1 | 37 1 run | no PP data | none |
| 34 | Bonsai 2 Ternary27B PrismML · mostly PTQ1_0 Gen 37 t/s (1 run) · PP 909 t/s (1 run) | 1 | 37 1 run | 909 1 run | none |
| 35 | Qwen34B Alibaba · mostly Q4_K_M Gen 36 t/s (1 run) · PP 417 t/s (1 run) | 1 | 36 1 run | 417 1 run | none |
| 36 | Qwen2.51.5B Alibaba · mostly FP16 Gen 32 t/s (1 run) | 1 | 32 1 run | no PP data | none |
| 37 | Qwen3.527B Alibaba · mostly Q4_K_M Gen 30 t/s (1 run) · PP 1516 t/s (1 run) | 1 | 30 1 run | 1516 1 run | none |
| 38 | Gemma 327B Google · mostly 4-bit Gen 8.9 t/s (1 run) · PP 11 t/s (1 run) | 1 | 8.9 1 run | 11 1 run | none |
| 39 | Edge0145B · 10B active mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 40 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 41 | Inkling 2 Mini26B · 3.4B active mostly UD-IQ4_XS No generation data · with speculation 182 t/s (1) | 1 | no generation data | no PP data | 182 1 run |
| 42 | Kimi K2.51000B · 32B active Moonshot AI · mostly IQ3_M No generation data | 1 | no generation data | no PP data | none |
| 43 | Kimi K2.61000B · 32B active Moonshot AI No generation data | 1 | no generation data | no PP data | none |
| 44 | Nemotron 3.5 Super VL120B · 12B active mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 45 | Nemotron 3.5 Bolt74B · 9B active mostly IQ2_XXS No generation data · with speculation 343 t/s (1) | 1 | no generation data | no PP data | 343 1 run |
| 46 | Ornith1.5 Ornith1.5-35B-A3B Splash35B · 3B active No generation data | 1 | no generation data | no PP data | none |
| 47 | Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active Alibaba · mostly Q4_K_P No generation data | 1 | no generation data | no PP data | none |
| 48 | Qwen3.8 Swift-1.5 Abliterated27B Alibaba No generation data · with speculation 152 t/s (1) | 1 | no generation data | no PP data | 152 1 run |
| 49 | Qwen3.8 Swift-Genesis27B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 50 | Qwen3.8 RVN Heretic (ARA abliterated)27B Alibaba · mostly Q6_K No generation data | 1 | no generation data | no PP data | none |
| 51 | Qwen3.8 QUASAR27B Alibaba · mostly NVFP4 No generation data · with speculation 287 t/s (1) | 1 | no generation data | no PP data | 287 1 run |
| 52 | Qwen3.8 Swift Uncensored27B Alibaba · mostly Q6_K No generation data | 1 | no generation data | no PP data | none |
| 53 | Qwen3.8 Uncensored-HauhauCS-Aggressive27B Alibaba · mostly Q5_K_P No generation data · with speculation 14 t/s (1) | 1 | no generation data | no PP data | 14 1 run |
| 54 | Qwen3.8 Swift-Qwen3.8-27B Splash27B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 55 | Qwen3.8 Qwen3.8-27B Splash27B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 56 | Qwen3.8 RVN27B Alibaba · mostly Q6_K No generation data · with speculation 100 t/s (1) | 1 | no generation data | no PP data | 100 1 run |
| 57 | Qwen3.835B Alibaba · mostly Q6 No generation data | 1 | no generation data | no PP data | none |
| 58 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ2_XS No generation data | 1 | no generation data | no PP data | none |
| 59 | Qwen3.8 Flash-Next abliterated-v2125B · 6B active Alibaba · mostly UD-Q5_K_XL No generation data | 1 | no generation data | no PP data | none |
| 60 | Qwen3.82400B · 95B active Alibaba · mostly UD-Q1_0 No generation data | 1 | no generation data | no PP data | none |
| 61 | Qwopus3.6 Coder-Compat27B mostly Q4_K_M No generation data · with speculation 40 t/s (1) | 1 | no generation data | no PP data | 40 1 run |
| 62 | gpt-oss21B · 3.6B active OpenAI No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 32GB of VRAM?
On setups with more than 24 GB and up to 32 GB of memory, the model people report most is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 32GB count as 32GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 32GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.