Best local LLMs for 12GB VRAM
How far 12GB of graphics memory goes for local models on cards like NVIDIA RTX 3060 12GB, NVIDIA RTX 4070, Intel Arc B580 12GB or NVIDIA RTX 5070: which models fit, and what people actually run on it.
Is 12GB of VRAM enough?
12GB holds a dense model of up to about 12B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.5 13B, which needs about 9.4 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.8 125B · 6B active, mostly at IQ2_XS, which leaves little room for context or runs partly from system RAM.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 12GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| DeepSeek V2 16B · 2.4B active | Q3_K_M | 9 GB |
| DeepSeek-Coder-V2-Lite 16B · 2.4B active | Q3_K_M | 9 GB |
| Qwen3 14B | Q3_K_M | 9.2 GB |
| Qwen2.5 14B | Q3_K_M | 9.2 GB |
| Qwen 2.5 Coder 14B | Q3_K_M | 9.2 GB |
| Qwen3.5 13B | Q4_K_M | 9.4 GB |
| Gemma 4 12B | Q4_K_M | 8.9 GB |
| Mellum2.1 12B · 2.5B active | Q5_K_M | 10.2 GB |
| Gemma 3 12B | Q4_K_M | 9.6 GB |
| Qwen3.5 9B | Q6_K | 9.4 GB |
| Ornith1.5 9B | Q6_K | 9.3 GB |
| Qwen3.8 9B | Q6_K | 9.4 GB |
And 65 smaller models. The calculator lists them all for your card.
What people run on 12GB
Model sizes reported on setups with more than 8 GB and up to 12 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ2_XS Gen 50 t/s (1 run) · PP 1000 t/s (1 run) | 35 | 50 1 run | 1000 1 run | none |
| 2 | Qwen3.827B Alibaba · mostly IQ3_XXS Gen 20 t/s (6 runs) · PP 532 t/s (2 runs) · with speculation 20 t/s (4) | 26 | 20 6 runs, 4 devices | 532 2 runs | 20 4 runs |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q2_K_XL Gen 114 t/s (1 run) · PP 2154 t/s (1 run) | 23 | 114 1 run | 2154 1 run | none |
| 4 | Qwen3.59B Alibaba · mostly NPU2 Gen 90 t/s (3 runs) | 5 | 90 3 runs, 3 devices | no PP data | none |
| 5 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M Gen 55 t/s (3 runs) | 5 | 55 3 runs, 2 devices | no PP data | none |
| 6 | Bonsai 227B PrismML · mostly PTQ1_0 Gen 47 t/s (4 runs) · PP 1211 t/s (3 runs) | 4 | 47 4 runs, 2 devices | 1211 3 runs | none |
| 7 | DeepSeek V4.1 Flash552B · 16B active mostly 2-bit No generation data | 4 | no generation data | no PP data | none |
| 8 | Gemma 412B Google DeepMind · mostly IQ4_NL Gen 31 t/s (2 runs) · PP 654 t/s (1 run) | 3 | 31 2 runs, 2 devices | 654 1 run | none |
| 9 | Qwen3.627B Alibaba · mostly Q2-XS Gen 24 t/s (1 run) | 3 | 24 1 run | no PP data | none |
| 10 | Llama 27B mostly Q4_0 Gen 62 t/s (1 run) · PP 2769 t/s (2 runs) | 2 | 62 1 run | 2769 2 runs | none |
| 11 | Bonsai 2 Ternary27B PrismML · mostly 2-bit ternary Gen 60 t/s (1 run) | 2 | 60 1 run | no PP data | none |
| 12 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly IQ2_M No generation data | 2 | no generation data | no PP data | none |
| 13 | Qwen3.8 Flash-Next Abliterated125B · 6B active Alibaba · mostly IQ3_S No generation data | 2 | no generation data | no PP data | none |
| 14 | Nemotron 3.5 Lightning Ternary30B · 3B active NVIDIA · mostly ternary 1.58-bit Gen 172 t/s (1 run) · PP 7000 t/s (1 run) | 1 | 172 1 run | 7000 1 run | none |
| 15 | Qwen3.5 Fonzie0.24B Alibaba · mostly Q4_K_M Gen 118 t/s (1 run) | 1 | 118 1 run | no PP data | none |
| 16 | Qwen330B · 3B active Alibaba Gen 96 t/s (1 run) | 1 | 96 1 run | no PP data | none |
| 17 | Qwen3.8 Mirai S27B Alibaba Gen 70 t/s (1 run) | 1 | 70 1 run | no PP data | none |
| 18 | Qwen38B Alibaba · mostly Q4_K_M Gen 61 t/s (1 run) · PP 851 t/s (1 run) | 1 | 61 1 run | 851 1 run | none |
| 19 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4_K_M Gen 60 t/s (1 run) | 1 | 60 1 run | no PP data | none |
| 20 | Qwen2.57B Alibaba · mostly Q4_K_M Gen 49 t/s (1 run) | 1 | 49 1 run | no PP data | none |
| 21 | GLM-4.7 Flash30B Zhipu AI Gen 43 t/s (1 run) | 1 | 43 1 run | no PP data | none |
| 22 | Muse Glimmer30B Meta · mostly EXL3-SC 3.00bpw H4 Gen 30 t/s (1 run) | 1 | 30 1 run | no PP data | none |
| 23 | Agents-A1 Millie35B · 3B active InternScience · mostly ternary Gen 22 t/s (1 run) · PP 261 t/s (1 run) | 1 | 22 1 run | 261 1 run | none |
| 24 | GLM-49B mostly Q4_0 Gen 21 t/s (1 run) | 1 | 21 1 run | no PP data | none |
| 25 | KAT-Coder V2.5-Dev35B · 3B active Kwaipilot Gen 14 t/s (1 run) | 1 | 14 1 run | no PP data | none |
| 26 | Qwen34B Alibaba · mostly Q4_K_M Gen 13 t/s (1 run) · PP 261 t/s (1 run) | 1 | 13 1 run | 261 1 run | none |
| 27 | Llama 3.18B Meta Gen 1.0 t/s (1 run) | 1 | 1.0 1 run | no PP data | none |
| 28 | Bonsai 2 Swift Katana27B PrismML · mostly PTQ1_0 No generation data | 1 | no generation data | no PP data | none |
| 29 | DeepSeek V4 Flash284B DeepSeek · mostly W8A8 No generation data | 1 | no generation data | no PP data | none |
| 30 | DeepSeek V4 Flash REAP-150B150B DeepSeek · mostly FP4 No generation data | 1 | no generation data | no PP data | none |
| 31 | DeepSeek V4.1 Flash1017B mostly Q8_XL No generation data | 1 | no generation data | no PP data | none |
| 32 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly Q4 No generation data | 1 | no generation data | no PP data | none |
| 33 | Gemma 431B Google DeepMind · mostly Q5_K_M No generation data | 1 | no generation data | no PP data | none |
| 34 | Inkling 2 Mini26B · 3.4B active mostly IQ4_NL No generation data | 1 | no generation data | no PP data | none |
| 35 | Ornith1.5122B · 10B active mostly UD-IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 36 | Qwen3.54B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 37 | Qwen3.513B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 38 | Qwen3.5 DR27B Alibaba · mostly 2.50bpw No generation data | 1 | no generation data | no PP data | none |
| 39 | Qwen3.6 Tiel-Coder35B · 3B active Alibaba · mostly Q4_K_XL No generation data | 1 | no generation data | no PP data | none |
| 40 | Qwen3.6 HauhauCS uncensored Aggressive35B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 41 | Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active Alibaba · mostly Q3_K_P No generation data | 1 | no generation data | no PP data | none |
| 42 | Qwen3.8 L0xRE27B Alibaba · mostly IQ3 No generation data · with speculation 40 t/s (1) | 1 | no generation data | no PP data | 40 1 run |
| 43 | Qwen3.8 Swift-1.527B Alibaba · mostly IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 44 | Qwen3.8 Swift-1.5-Uncensored27B Alibaba · mostly 2.3bpw No generation data | 1 | no generation data | no PP data | none |
| 45 | Qwen3.8 Bonsai-Llama-Jev27B Alibaba · mostly Q2_64 No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 12GB of VRAM?
On setups with more than 8 GB and up to 12 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 12GB count as 12GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 12GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.