Best local LLM for coding with 24GB VRAM
The models people report running for coding on setups with more than 16 GB and up to 24 GB of memory, such as NVIDIA RTX 3090, NVIDIA RTX 4090, AMD RX 7900 XTX or NVIDIA Tesla P40 24GB. Each row is one model size, with the quant most people used and its typical speed.
Ranked from reports whose posters used the model for coding.
Ranked by community reports
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly Q4_K_M Gen 31 t/s (10 runs) · PP 997 t/s (5 runs) · with speculation 73 t/s (18) | 57 | 31 10 runs, 6 devices | 997 5 runs | 73 18 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_S No generation data | 14 | no generation data | no PP data | none |
| 3 | Qwen3.627B Alibaba · mostly Q4_K_M Gen 42 t/s (3 runs) · PP 526 t/s (2 runs) · with speculation 50 t/s (5) | 12 | 42 3 runs, 2 devices | 526 2 runs | 50 5 runs |
| 4 | Qwen3.8 Swift-1.527B Alibaba · mostly AWQ-INT4 Gen 67 t/s (1 run) · with speculation 83 t/s (1) | 5 | 67 1 run | no PP data | 83 1 run |
| 5 | Qwen3.635B · 3B active Alibaba · mostly Q3_K_M Gen 120 t/s (3 runs) | 3 | 120 3 runs, 2 devices | no PP data | none |
| 6 | Gemma 426B · 4B active Google DeepMind · mostly Q5_K_M Gen 134 t/s (2 runs) · PP 5600 t/s (1 run) | 2 | 134 2 runs | 5600 1 run | none |
| 7 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly IQ2_M No generation data | 2 | no generation data | no PP data | none |
| 8 | DeepSeek V4.1 Flash552B · 16B active mostly EXL3 3.0 bpw No generation data | 2 | no generation data | no PP data | none |
| 9 | Qwen2.5 Coder-7B7B Alibaba · mostly Q4_K_M Gen 159 t/s (1 run) | 1 | 159 1 run | no PP data | none |
| 10 | DeepSeek-Coder-V2-Lite16B · 2.4B active mostly 4bit Gen 126 t/s (1 run) · PP 364 t/s (1 run) | 1 | 126 1 run | 364 1 run | none |
| 11 | Nemotron 3.5 Bolt74B · 9B active mostly IQ2_XXS Gen 85 t/s (1 run) · PP 833 t/s (1 run) | 1 | 85 1 run | 833 1 run | none |
| 12 | Qwen3.8 Escha-W227B Alibaba · mostly escha 2-bit Gen 67 t/s (1 run) | 1 | 67 1 run | no PP data | none |
| 13 | Gemma 431B Google DeepMind · mostly Q4_K_S Gen 37 t/s (1 run) | 1 | 37 1 run | no PP data | none |
| 14 | Qwen3.8 AREX-227B Alibaba · mostly EXL3 4.00 bpw Gen 36 t/s (1 run) | 1 | 36 1 run | no PP data | none |
| 15 | Qwen2.5 Coder-32B32B Alibaba · mostly Q4_K_M Gen 28 t/s (1 run) · PP 450 t/s (1 run) | 1 | 28 1 run | 450 1 run | none |
| 16 | Bonsai 2 Ternary27B PrismML · mostly Q4/Q5 No generation data · with speculation 188 t/s (1) | 1 | no generation data | no PP data | 188 1 run |
| 17 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly Q2 No generation data | 1 | no generation data | no PP data | none |
| 18 | Muse Glimmer30B Meta · mostly Q5_K_M No generation data · with speculation 85 t/s (1) | 1 | no generation data | no PP data | 85 1 run |
| 19 | Nemotron 3.5 Lightning30B · 3B active NVIDIA · mostly Q4_0 No generation data | 1 | no generation data | no PP data | none |
| 20 | Ornith1.5 BigBang35B · 3B active mostly Q4_K_M No generation data · with speculation 169 t/s (1) | 1 | no generation data | no PP data | 169 1 run |
| 21 | Ornith1.5122B mostly UD-IQ2_XXS No generation data | 1 | no generation data | no PP data | none |
| 22 | Qwen2.532B Alibaba · mostly AWQ No generation data | 1 | no generation data | no PP data | none |
| 23 | Qwen314B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 24 | Qwen3.5 Mica v0.14B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 25 | Qwen3.527B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 26 | Qwen3.535B · 3B active Alibaba · mostly IQ4_NL No generation data · PP 980 t/s (1 run) | 1 | no generation data | 980 1 run | none |
| 27 | Qwen3.6 Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-LOW27B Alibaba · mostly IQ4_XS No generation data · with speculation 42 t/s (1) | 1 | no generation data | no PP data | 42 1 run |
| 28 | Qwen3.8 CODER27B Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 29 | Qwen3.8 Swift-1.5-Uncensored27B Alibaba · mostly IQ4_XS No generation data · with speculation 68 t/s (1) | 1 | no generation data | no PP data | 68 1 run |
| 30 | Qwen3.8 Swift27B Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 31 | Qwen3.8 pi27B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 32 | Qwen3.8 Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-NEO27B Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What is the best local LLM for coding with 24GB of VRAM?
On setups with more than 16 GB and up to 24 GB of memory, the model people report using most for coding is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 27B. The table shows the quant most of them used and their typical speed, so you can start from a setup that is known to work.
What counts as a coding report?
A community performance report whose poster described using the model for coding: an editor assistant, an agent, or code generation. The memory band is the poster's reported VRAM, or the card's VRAM times the number of cards.
Which quant should I use for coding on 24GB?
Start from the quant most people used for that model, shown under its name. It is one known to fit with room for a coding context. If you want a bigger model in the same memory, step down one quant rung, and the VRAM calculator shows the exact memory at each rung for your card.
How we rank
Model sizes are ordered by how many coding reports use them in this memory band, which measures what people run, not answer quality. Typical speed is the median of runs on one device serving one request without speculative decoding. Runs with MTP or a draft model are counted separately, because they can be several times faster. Reports come from r/LocalLLaMA, the web and direct submissions. Nothing is sponsored.