llamaperf

Best local LLMs for 12GB VRAM

How far 12GB of graphics memory goes for local models on cards like NVIDIA RTX 3060 12GB, NVIDIA RTX 4070, Intel Arc B580 12GB or NVIDIA RTX 5070: which models fit, and what people actually run on it.

Is 12GB of VRAM enough?

12GB holds a dense model of up to about 12B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.5 13B, which needs about 9.4 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.8 125B · 6B active, mostly at IQ2_XS, which leaves little room for context or runs partly from system RAM.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 12GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

ModelBest quantNeeds
DeepSeek V2 16B · 2.4B activeQ3_K_M9 GB
DeepSeek-Coder-V2-Lite 16B · 2.4B activeQ3_K_M9 GB
Qwen3 14BQ3_K_M9.2 GB
Qwen2.5 14BQ3_K_M9.2 GB
Qwen 2.5 Coder 14BQ3_K_M9.2 GB
Qwen3.5 13BQ4_K_M9.4 GB
Gemma 4 12BQ4_K_M8.9 GB
Mellum2.1 12B · 2.5B activeQ5_K_M10.2 GB
Gemma 3 12BQ4_K_M9.6 GB
Qwen3.5 9BQ6_K9.4 GB
Ornith1.5 9BQ6_K9.3 GB
Qwen3.8 9BQ6_K9.4 GB

And 65 smaller models. The calculator lists them all for your card.

What people run on 12GB

Model sizes reported on setups with more than 8 GB and up to 12 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly IQ2_XS
Gen 50 t/s (1 run) · PP 1000 t/s (1 run)
35
2Qwen3.827B
Alibaba · mostly IQ3_XXS
Gen 20 t/s (6 runs) · PP 532 t/s (2 runs) · with speculation 20 t/s (4)
26
3Qwen3.635B · 3B active
Alibaba · mostly Q2_K_XL
Gen 114 t/s (1 run) · PP 2154 t/s (1 run)
23
4Qwen3.59B
Alibaba · mostly NPU2
Gen 90 t/s (3 runs)
5
5Gemma 4 E4B8B
Google DeepMind · mostly Q4_K_M
Gen 55 t/s (3 runs)
5
6Bonsai 227B
PrismML · mostly PTQ1_0
Gen 47 t/s (4 runs) · PP 1211 t/s (3 runs)
4
7DeepSeek V4.1 Flash552B · 16B active
mostly 2-bit
No generation data
4
8Gemma 412B
Google DeepMind · mostly IQ4_NL
Gen 31 t/s (2 runs) · PP 654 t/s (1 run)
3
9Qwen3.627B
Alibaba · mostly Q2-XS
Gen 24 t/s (1 run)
3
10Llama 27B
mostly Q4_0
Gen 62 t/s (1 run) · PP 2769 t/s (2 runs)
2
11Bonsai 2 Ternary27B
PrismML · mostly 2-bit ternary
Gen 60 t/s (1 run)
2
12DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly IQ2_M
No generation data
2
13Qwen3.8 Flash-Next Abliterated125B · 6B active
Alibaba · mostly IQ3_S
No generation data
2
14Nemotron 3.5 Lightning Ternary30B · 3B active
NVIDIA · mostly ternary 1.58-bit
Gen 172 t/s (1 run) · PP 7000 t/s (1 run)
1
15Qwen3.5 Fonzie0.24B
Alibaba · mostly Q4_K_M
Gen 118 t/s (1 run)
1
16Qwen330B · 3B active
Alibaba
Gen 96 t/s (1 run)
1
17Qwen3.8 Mirai S27B
Alibaba
Gen 70 t/s (1 run)
1
18Qwen38B
Alibaba · mostly Q4_K_M
Gen 61 t/s (1 run) · PP 851 t/s (1 run)
1
19Gemma 4 E2B5.1B
Google DeepMind · mostly Q4_K_M
Gen 60 t/s (1 run)
1
20Qwen2.57B
Alibaba · mostly Q4_K_M
Gen 49 t/s (1 run)
1
21GLM-4.7 Flash30B
Zhipu AI
Gen 43 t/s (1 run)
1
22Muse Glimmer30B
Meta · mostly EXL3-SC 3.00bpw H4
Gen 30 t/s (1 run)
1
23Agents-A1 Millie35B · 3B active
InternScience · mostly ternary
Gen 22 t/s (1 run) · PP 261 t/s (1 run)
1
24GLM-49B
mostly Q4_0
Gen 21 t/s (1 run)
1
25KAT-Coder V2.5-Dev35B · 3B active
Kwaipilot
Gen 14 t/s (1 run)
1
26Qwen34B
Alibaba · mostly Q4_K_M
Gen 13 t/s (1 run) · PP 261 t/s (1 run)
1
27Llama 3.18B
Meta
Gen 1.0 t/s (1 run)
1
28Bonsai 2 Swift Katana27B
PrismML · mostly PTQ1_0
No generation data
1
29DeepSeek V4 Flash284B
DeepSeek · mostly W8A8
No generation data
1
30DeepSeek V4 Flash REAP-150B150B
DeepSeek · mostly FP4
No generation data
1
31DeepSeek V4.1 Flash1017B
mostly Q8_XL
No generation data
1
32GLM-5.3 Flash320B · 18B active
Zhipu AI · mostly Q4
No generation data
1
33Gemma 431B
Google DeepMind · mostly Q5_K_M
No generation data
1
34Inkling 2 Mini26B · 3.4B active
mostly IQ4_NL
No generation data
1
35Ornith1.5122B · 10B active
mostly UD-IQ3_XXS
No generation data
1
36Qwen3.54B
Alibaba · mostly Q4_K_M
No generation data
1
37Qwen3.513B
Alibaba
No generation data
1
38Qwen3.5 DR27B
Alibaba · mostly 2.50bpw
No generation data
1
39Qwen3.6 Tiel-Coder35B · 3B active
Alibaba · mostly Q4_K_XL
No generation data
1
40Qwen3.6 HauhauCS uncensored Aggressive35B · 3B active
Alibaba · mostly Q4_K_M
No generation data
1
41Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active
Alibaba · mostly Q3_K_P
No generation data
1
42Qwen3.8 L0xRE27B
Alibaba · mostly IQ3
No generation data · with speculation 40 t/s (1)
1
43Qwen3.8 Swift-1.527B
Alibaba · mostly IQ3_XXS
No generation data
1
44Qwen3.8 Swift-1.5-Uncensored27B
Alibaba · mostly 2.3bpw
No generation data
1
45Qwen3.8 Bonsai-Llama-Jev27B
Alibaba · mostly Q2_64
No generation data
1

Frequently asked

What do people run on 12GB of VRAM?

On setups with more than 8 GB and up to 12 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.

Does a Mac with 12GB count as 12GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 12GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.