llamaperf

Best local LLMs for 16GB VRAM

How far 16GB of graphics memory goes for local models on cards like NVIDIA RTX 5060 Ti 16GB, NVIDIA RTX 5070 Ti, NVIDIA RTX 5080 or AMD RX 9070 XT 16GB: which models fit, and what people actually run on it.

Is 16GB of VRAM enough?

16GB holds a dense model of up to about 16B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is gpt-oss 20B, which needs about 13.3 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.8 27B, mostly at IQ4_XS, which leaves little room for context or runs partly from system RAM.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 16GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

ModelBest quantNeeds
gpt-oss 21B · 3.6B activeQ3_K_M11.1 GB
gpt-oss 20BQ4_K_M13.3 GB
DeepSeek V2 16B · 2.4B activeQ5_K_M13 GB
DeepSeek-Coder-V2-Lite 16B · 2.4B activeQ5_K_M13 GB
Qwen3 14BQ5_K_M12.7 GB
Qwen2.5 14BQ5_K_M12.7 GB
Qwen 2.5 Coder 14BQ5_K_M12.7 GB
Qwen3.5 13BQ6_K12.7 GB
Gemma 4 12BQ6_K11.9 GB
Mellum2.1 12B · 2.5B activeQ6_K11.7 GB
Gemma 3 12BQ6_K12.6 GB
Qwen3.5 9BQ8_011.1 GB

And 67 smaller models. The calculator lists them all for your card.

What people run on 16GB

Model sizes reported on setups with more than 12 GB and up to 16 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.827B
Alibaba · mostly IQ4_XS
Gen 29 t/s (18 runs) · PP 400 t/s (5 runs) · with speculation 58 t/s (22)
96
2Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly IQ3_S
Gen 41 t/s (1 run)
70
3Qwen3.635B · 3B active
Alibaba · mostly Q4_K_M
Gen 60 t/s (2 runs) · PP 457 t/s (2 runs) · with speculation 110 t/s (1)
29
4Qwen3.8 Swift-1.527B
Alibaba · mostly IQ3_S
Gen 42 t/s (2 runs) · with speculation 70 t/s (3)
10
5Qwen3.59B
Alibaba · mostly Q4_K_M
Gen 9.6 t/s (4 runs) · PP 53 t/s (1 run)
8
6DeepSeek V4.1 Flash552B · 16B active
mostly IQ2_XS
No generation data
8
7Gemma 426B · 4B active
Google DeepMind · mostly IQ4_XS
Gen 95 t/s (3 runs) · PP 4714 t/s (2 runs)
7
8Qwen38B
Alibaba · mostly Q4_K_M
Gen 61 t/s (4 runs) · PP 2232 t/s (2 runs) · with speculation 133 t/s (1)
6
9Gemma 4 E4B8B
Google DeepMind · mostly Q4
Gen 51 t/s (3 runs) · PP 326 t/s (1 run)
6
10Gemma 412B
Google DeepMind · mostly Q4_K_M
Gen 64 t/s (2 runs) · PP 3100 t/s (1 run) · with speculation 20 t/s (1)
5
11Qwen3.627B
Alibaba · mostly IQ4_XS
Gen 22 t/s (1 run) · PP 760 t/s (1 run)
4
12Llama 3.23B
Meta · mostly Q4_K_M
Gen 299 t/s (1 run) · PP 19397 t/s (1 run)
3
13Qwen3.54B
Alibaba · mostly Q4_K_M
Gen 19 t/s (2 runs) · PP 352 t/s (1 run)
3
14Qwen3.535B · 3B active
Alibaba · mostly UD-IQ3_XXS
Gen 17 t/s (1 run) · PP 1168 t/s (1 run)
3
15Qwen3.8 Uncensored27B
Alibaba · mostly IQ4_XS
Gen 11 t/s (1 run) · with speculation 35 t/s (1)
3
16GLM-5.3 Flash320B · 18B active
Zhipu AI · mostly IQ4_XS
No generation data
3
17Gemma 426B
Google DeepMind · mostly EXL3 like
No generation data · with speculation 182 t/s (1)
3
18Qwen330B · 3B active
Alibaba · mostly Q4_K_M
No generation data
3
19Ornith1.59B
mostly Q6_K
Gen 175 t/s (1 run) · PP 1450 t/s (1 run)
2
20Qwen2.5 Coder-7B7B
Alibaba · mostly Q4
Gen 124 t/s (2 runs) · PP 9826 t/s (1 run)
2
21Qwen2.50.5B
Alibaba · mostly Q4
Gen 96 t/s (1 run)
2
22Qwen2.5 Coder14B
Alibaba · mostly Q4_K_M
Gen 63 t/s (2 runs) · PP 3026 t/s (2 runs)
2
23Bonsai 227B
PrismML · mostly PQ2
Gen 57 t/s (1 run)
2
24Qwen3.8 Heretic27B
Alibaba · mostly Q3_K_XL
Gen 57 t/s (1 run) · PP 1518 t/s (1 run) · with speculation 55 t/s (1)
2
25Muse Glimmer30B
Meta · mostly Q4_K_XL
Gen 18 t/s (1 run) · with speculation 20 t/s (1)
2
26Qwen314B
Alibaba · mostly 4-bit
Gen 15 t/s (2 runs) · PP 116 t/s (1 run)
2
27Llama 3.18B
Meta · mostly Q4_K_M
Gen 1.5 t/s (1 run)
2
28DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly UD-Q2_K_XL
No generation data
2
29Gemma 431B
Google DeepMind · mostly NVFP4
No generation data · with speculation 67 t/s (1)
2
30gpt-oss21B · 3.6B active
OpenAI · mostly MXFP4
Gen 178 t/s (1 run) · PP 10329 t/s (1 run)
1
31Qwen2.57B
Alibaba · mostly Q4_K_M
Gen 177 t/s (1 run) · PP 9984 t/s (1 run)
1
32Qwen3-VL8B
Gen 127 t/s (1 run)
1
33MiniCPM51B
Gen 125 t/s (1 run)
1
34Qwen2.514B
Alibaba · mostly Q4_K_M
Gen 95 t/s (1 run) · PP 4865 t/s (1 run)
1
35MiniCPM-V 4.735B · 3B active
mostly mix_q3gud_im
Gen 93 t/s (1 run)
1
36Qwen3.50.8B
Alibaba · mostly Q8_0
Gen 90 t/s (1 run) · PP 2122 t/s (1 run)
1
37Bonsai27B
PrismML · mostly Q1_0
Gen 73 t/s (1 run)
1
38Mellum2.1 Thinking12B · 2.5B active
mostly NVFP4
Gen 64 t/s (1 run)
1
39Qwen3.8 Escha-W227B
Alibaba · mostly 2.469 bpw
Gen 59 t/s (1 run)
1
40Qwen2.53B
Alibaba · mostly Q4_K_M
Gen 47 t/s (1 run) · PP 557 t/s (1 run)
1
41Mimo 2.6 Distill-Qwen9B
Xiaomi · mostly Q8_0
Gen 42 t/s (1 run) · PP 1718 t/s (1 run)
1
42Qwen332B
Alibaba · mostly 3.6577 bpw
Gen 38 t/s (1 run)
1
43Qwen3.6 APEX-I-Quality35B · 3B active
Alibaba
Gen 37 t/s (1 run)
1
44Qwen31.7B
Alibaba · mostly Q8_0
Gen 33 t/s (1 run)
1
45Caphlon-227B
mostly PTQ1_SB
Gen 16 t/s (1 run) · PP 278 t/s (1 run)
1
46Qwen3.6 Coder27B · 3B active
Alibaba · mostly Q4_K_M
Gen 14 t/s (1 run)
1
47Agents-A1 Uncensored-MTP-APEX35B · 3B active
InternScience · mostly APEX Compact
No generation data · with speculation 97 t/s (1)
1
48GLM-5.2744B · 40B active
Zhipu AI · mostly fmt=4 grouped int4, gs=64
No generation data
1
49Gemma 4 E2B5.1B
Google DeepMind · mostly Q4_K_M
No generation data
1
50Llama 27B
mostly Q4_0
No generation data
1
51Mellum2.112B · 2.5B active
mostly Q8
No generation data
1
52Metis-1.6 Logos10B · 1.2B active
mostly FP8
No generation data · with speculation 187 t/s (1)
1
53Nemotron 3.5 Bolt74B · 9B active
mostly IQ2_XSS
No generation data
1
54Ornith1.535B · 3B active
mostly Q4_K_M
No generation data
1
55Poolside Laguna S 2.1118B · 8B active
mostly IQ3_S
No generation data
1
56Qwen3-Coder30B · 3B active
mostly Q4_K_M
No generation data
1
57Qwen3-Next80B · 3B active
Alibaba · mostly Q4_K_M
No generation data
1
58Qwen3.52B
Alibaba · mostly int4 GPTQ group 32
No generation data · with speculation 620 t/s (1)
1
59Qwen3.5 DeepSeek-V4-Flash9B
Alibaba · mostly Q6_K
No generation data · with speculation 49 t/s (1)
1
60Qwen3.527B
Alibaba · mostly IQ3_S
No generation data · PP 1049 t/s (1 run)
1
61Qwen3.6 Math-Physics11B
Alibaba · mostly Q4_K_M
No generation data
1
62Qwen3.6 Omnimerge-v427B
Alibaba · mostly IQ3_M
No generation data · with speculation 46 t/s (1)
1
63Qwen3.6 BigBang-v135B · 3B active
Alibaba · mostly IQ3_XXS
No generation data
1
64Qwen3.8 TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX27B
Alibaba
No generation data · with speculation 30 t/s (1)
1
65Qwen3.8 Huihui-Abliterated27B
Alibaba · mostly IQ3_XXS
No generation data
1
66Qwen3.8 OrcaRouter Uncensored27B
Alibaba · mostly IQ3_XXS
No generation data · with speculation 83 t/s (1)
1
67Qwen3.8 QUASAR27B
Alibaba · mostly NVFP4
No generation data
1
68Qwen3.8 ASCII-Condensed27B
Alibaba · mostly UD-IQ4_XS
No generation data
1
69Qwen3.8 abliterated27B
Alibaba · mostly IQ3_S
No generation data · with speculation 105 t/s (1)
1
70Qwen3.8 Unsloth Dynamic v3.127B
Alibaba · mostly UD-Q2_K_XL
No generation data · with speculation 78 t/s (1)
1
71Qwen3.8 Flash-Next Abliterated125B · 6B active
Alibaba · mostly IQ3_XXS
No generation data
1
72Qwen3.8 Swift-1.5 Flash-Next125B · 6B active
Alibaba · mostly IQ2_XS
No generation data
1
73Raon-1 mini-preview29B · 4B active
mostly IQ3_KS
No generation data
1
74gpt-oss117B · 5.1B active
OpenAI
No generation data
1

Frequently asked

What do people run on 16GB of VRAM?

On setups with more than 12 GB and up to 16 GB of memory, the model people report most is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.

Does a Mac with 16GB count as 16GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 16GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.