Best local LLM for coding with 32GB VRAM
The models people report running for coding on setups with more than 24 GB and up to 32 GB of memory, such as NVIDIA RTX 5090, AMD Radeon AI PRO R9700 32GB, NVIDIA V100 32GB or Intel Arc Pro B70. Each row is one model size, with the quant most people used and its typical speed.
Ranked from reports whose posters used the model for coding.
Ranked by community reports
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly NVFP4 Gen 80 t/s (7 runs) · PP 22 t/s (1 run) · with speculation 169 t/s (16) | 44 | 80 7 runs, 3 devices | 22 1 run | 169 16 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly NVFP4 Gen 74 t/s (1 run) | 23 | 74 1 run | no PP data | none |
| 3 | Qwen3.635B · 3B active Alibaba · mostly MLX 4-bit Gen 64 t/s (5 runs) · PP 977 t/s (3 runs) | 7 | 64 5 runs, 5 devices | 977 3 runs | none |
| 4 | Qwen3.8 Swift-1.527B Alibaba · mostly MXFP4 Gen 38 t/s (1 run) · with speculation 136 t/s (2) | 3 | 38 1 run | no PP data | 136 2 runs |
| 5 | Qwen3.627B Alibaba · mostly NVFP4 No generation data · with speculation 74 t/s (1) | 3 | no generation data | no PP data | 74 1 run |
| 6 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly int4 No generation data | 2 | no generation data | no PP data | none |
| 7 | Muse Glimmer30B Meta · mostly Q5_K_XL No generation data · with speculation 253 t/s (1) | 2 | no generation data | no PP data | 253 1 run |
| 8 | Ornith1.535B · 3B active mostly MXFP4 No generation data · with speculation 245 t/s (1) | 2 | no generation data | no PP data | 245 1 run |
| 9 | Qwen3.8 Huihui-Abliterated27B Alibaba · mostly NVFP4 No generation data · with speculation 203 t/s (2) | 2 | no generation data | no PP data | 203 2 runs |
| 10 | Qwen3.8 Flash-Next Uncensored125B · 6B active Alibaba · mostly NVFP4 No generation data | 2 | no generation data | no PP data | none |
| 11 | Bonsai 227B PrismML · mostly PQ2_0 Gen 101 t/s (1 run) | 1 | 101 1 run | no PP data | none |
| 12 | Qwen3-Coder30B · 3B active mostly Q4_K_M Gen 66 t/s (1 run) | 1 | 66 1 run | no PP data | none |
| 13 | DeepSeek V4.1 Flash552B · 16B active mostly MXFP4 No generation data | 1 | no generation data | no PP data | none |
| 14 | Gemma 431B Google DeepMind · mostly QAT No generation data | 1 | no generation data | no PP data | none |
| 15 | Nemotron 3.5 Bolt74B · 9B active mostly IQ2_XXS No generation data · with speculation 343 t/s (1) | 1 | no generation data | no PP data | 343 1 run |
| 16 | Nemotron 3.5 Lightning30B · 3B active NVIDIA · mostly Q4_0 No generation data | 1 | no generation data | no PP data | none |
| 17 | Qwen3.535B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 18 | Qwen3.8 RVN27B Alibaba · mostly Q6_K No generation data · with speculation 100 t/s (1) | 1 | no generation data | no PP data | 100 1 run |
| 19 | Qwen3.8 Uncensored27B Alibaba · mostly NVFP4 No generation data · with speculation 175 t/s (1) | 1 | no generation data | no PP data | 175 1 run |
| 20 | Qwen3.8 Swift27B Alibaba · mostly Q6_K No generation data | 1 | no generation data | no PP data | none |
| 21 | Qwen3.8 RVN Heretic (ARA abliterated)27B Alibaba · mostly Q6_K No generation data | 1 | no generation data | no PP data | none |
| 22 | Qwen3.8 QUASAR27B Alibaba · mostly NVFP4 No generation data · with speculation 287 t/s (1) | 1 | no generation data | no PP data | 287 1 run |
| 23 | Qwen3.835B Alibaba · mostly Q6 No generation data | 1 | no generation data | no PP data | none |
| 24 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ2_XS No generation data | 1 | no generation data | no PP data | none |
| 25 | Qwopus3.6 Coder-Compat27B mostly Q4_K_M No generation data · with speculation 40 t/s (1) | 1 | no generation data | no PP data | 40 1 run |
Frequently asked
What is the best local LLM for coding with 32GB of VRAM?
On setups with more than 24 GB and up to 32 GB of memory, the model people report using most for coding is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table shows the quant most of them used and their typical speed, so you can start from a setup that is known to work.
What counts as a coding report?
A community performance report whose poster described using the model for coding: an editor assistant, an agent, or code generation. The memory band is the poster's reported VRAM, or the card's VRAM times the number of cards.
Which quant should I use for coding on 32GB?
Start from the quant most people used for that model, shown under its name. It is one known to fit with room for a coding context. If you want a bigger model in the same memory, step down one quant rung, and the VRAM calculator shows the exact memory at each rung for your card.
How we rank
Model sizes are ordered by how many coding reports use them in this memory band, which measures what people run, not answer quality. Typical speed is the median of runs on one device serving one request without speculative decoding. Runs with MTP or a draft model are counted separately, because they can be several times faster. Reports come from r/LocalLLaMA, the web and direct submissions. Nothing is sponsored.