Best local LLMs for 16GB VRAM
How far 16GB of graphics memory goes for local models on cards like NVIDIA RTX 5060 Ti 16GB, NVIDIA RTX 5070 Ti, NVIDIA RTX 5080 or AMD RX 9070 XT 16GB: which models fit, and what people actually run on it.
Is 16GB of VRAM enough?
16GB holds a dense model of up to about 16B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is gpt-oss 20B, which needs about 13.3 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.8 27B, mostly at IQ4_XS, which leaves little room for context or runs partly from system RAM.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 16GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| gpt-oss 21B · 3.6B active | Q3_K_M | 11.1 GB |
| gpt-oss 20B | Q4_K_M | 13.3 GB |
| DeepSeek V2 16B · 2.4B active | Q5_K_M | 13 GB |
| DeepSeek-Coder-V2-Lite 16B · 2.4B active | Q5_K_M | 13 GB |
| Qwen3 14B | Q5_K_M | 12.7 GB |
| Qwen2.5 14B | Q5_K_M | 12.7 GB |
| Qwen 2.5 Coder 14B | Q5_K_M | 12.7 GB |
| Qwen3.5 13B | Q6_K | 12.7 GB |
| Gemma 4 12B | Q6_K | 11.9 GB |
| Mellum2.1 12B · 2.5B active | Q6_K | 11.7 GB |
| Gemma 3 12B | Q6_K | 12.6 GB |
| Qwen3.5 9B | Q8_0 | 11.1 GB |
And 67 smaller models. The calculator lists them all for your card.
What people run on 16GB
Model sizes reported on setups with more than 12 GB and up to 16 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly IQ4_XS Gen 29 t/s (18 runs) · PP 400 t/s (5 runs) · with speculation 58 t/s (22) | 96 | 29 18 runs, 12 devices | 400 5 runs | 58 22 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_S Gen 41 t/s (1 run) | 70 | 41 1 run | no PP data | none |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_M Gen 60 t/s (2 runs) · PP 457 t/s (2 runs) · with speculation 110 t/s (1) | 29 | 60 2 runs | 457 2 runs | 110 1 run |
| 4 | Qwen3.8 Swift-1.527B Alibaba · mostly IQ3_S Gen 42 t/s (2 runs) · with speculation 70 t/s (3) | 10 | 42 2 runs, 2 devices | no PP data | 70 3 runs |
| 5 | Qwen3.59B Alibaba · mostly Q4_K_M Gen 9.6 t/s (4 runs) · PP 53 t/s (1 run) | 8 | 9.6 4 runs, 3 devices | 53 1 run | none |
| 6 | DeepSeek V4.1 Flash552B · 16B active mostly IQ2_XS No generation data | 8 | no generation data | no PP data | none |
| 7 | Gemma 426B · 4B active Google DeepMind · mostly IQ4_XS Gen 95 t/s (3 runs) · PP 4714 t/s (2 runs) | 7 | 95 3 runs, 3 devices | 4714 2 runs | none |
| 8 | Qwen38B Alibaba · mostly Q4_K_M Gen 61 t/s (4 runs) · PP 2232 t/s (2 runs) · with speculation 133 t/s (1) | 6 | 61 4 runs, 3 devices | 2232 2 runs | 133 1 run |
| 9 | Gemma 4 E4B8B Google DeepMind · mostly Q4 Gen 51 t/s (3 runs) · PP 326 t/s (1 run) | 6 | 51 3 runs, 3 devices | 326 1 run | none |
| 10 | Gemma 412B Google DeepMind · mostly Q4_K_M Gen 64 t/s (2 runs) · PP 3100 t/s (1 run) · with speculation 20 t/s (1) | 5 | 64 2 runs | 3100 1 run | 20 1 run |
| 11 | Qwen3.627B Alibaba · mostly IQ4_XS Gen 22 t/s (1 run) · PP 760 t/s (1 run) | 4 | 22 1 run | 760 1 run | none |
| 12 | Llama 3.23B Meta · mostly Q4_K_M Gen 299 t/s (1 run) · PP 19397 t/s (1 run) | 3 | 299 1 run | 19397 1 run | none |
| 13 | Qwen3.54B Alibaba · mostly Q4_K_M Gen 19 t/s (2 runs) · PP 352 t/s (1 run) | 3 | 19 2 runs | 352 1 run | none |
| 14 | Qwen3.535B · 3B active Alibaba · mostly UD-IQ3_XXS Gen 17 t/s (1 run) · PP 1168 t/s (1 run) | 3 | 17 1 run | 1168 1 run | none |
| 15 | Qwen3.8 Uncensored27B Alibaba · mostly IQ4_XS Gen 11 t/s (1 run) · with speculation 35 t/s (1) | 3 | 11 1 run | no PP data | 35 1 run |
| 16 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly IQ4_XS No generation data | 3 | no generation data | no PP data | none |
| 17 | Gemma 426B Google DeepMind · mostly EXL3 like No generation data · with speculation 182 t/s (1) | 3 | no generation data | no PP data | 182 1 run |
| 18 | Qwen330B · 3B active Alibaba · mostly Q4_K_M No generation data | 3 | no generation data | no PP data | none |
| 19 | Ornith1.59B mostly Q6_K Gen 175 t/s (1 run) · PP 1450 t/s (1 run) | 2 | 175 1 run | 1450 1 run | none |
| 20 | Qwen2.5 Coder-7B7B Alibaba · mostly Q4 Gen 124 t/s (2 runs) · PP 9826 t/s (1 run) | 2 | 124 2 runs, 2 devices | 9826 1 run | none |
| 21 | Qwen2.50.5B Alibaba · mostly Q4 Gen 96 t/s (1 run) | 2 | 96 1 run | no PP data | none |
| 22 | Qwen2.5 Coder14B Alibaba · mostly Q4_K_M Gen 63 t/s (2 runs) · PP 3026 t/s (2 runs) | 2 | 63 2 runs, 2 devices | 3026 2 runs | none |
| 23 | Bonsai 227B PrismML · mostly PQ2 Gen 57 t/s (1 run) | 2 | 57 1 run | no PP data | none |
| 24 | Qwen3.8 Heretic27B Alibaba · mostly Q3_K_XL Gen 57 t/s (1 run) · PP 1518 t/s (1 run) · with speculation 55 t/s (1) | 2 | 57 1 run | 1518 1 run | 55 1 run |
| 25 | Muse Glimmer30B Meta · mostly Q4_K_XL Gen 18 t/s (1 run) · with speculation 20 t/s (1) | 2 | 18 1 run | no PP data | 20 1 run |
| 26 | Qwen314B Alibaba · mostly 4-bit Gen 15 t/s (2 runs) · PP 116 t/s (1 run) | 2 | 15 2 runs, 2 devices | 116 1 run | none |
| 27 | Llama 3.18B Meta · mostly Q4_K_M Gen 1.5 t/s (1 run) | 2 | 1.5 1 run | no PP data | none |
| 28 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly UD-Q2_K_XL No generation data | 2 | no generation data | no PP data | none |
| 29 | Gemma 431B Google DeepMind · mostly NVFP4 No generation data · with speculation 67 t/s (1) | 2 | no generation data | no PP data | 67 1 run |
| 30 | gpt-oss21B · 3.6B active OpenAI · mostly MXFP4 Gen 178 t/s (1 run) · PP 10329 t/s (1 run) | 1 | 178 1 run | 10329 1 run | none |
| 31 | Qwen2.57B Alibaba · mostly Q4_K_M Gen 177 t/s (1 run) · PP 9984 t/s (1 run) | 1 | 177 1 run | 9984 1 run | none |
| 32 | Qwen3-VL8B Gen 127 t/s (1 run) | 1 | 127 1 run | no PP data | none |
| 33 | MiniCPM51B Gen 125 t/s (1 run) | 1 | 125 1 run | no PP data | none |
| 34 | Qwen2.514B Alibaba · mostly Q4_K_M Gen 95 t/s (1 run) · PP 4865 t/s (1 run) | 1 | 95 1 run | 4865 1 run | none |
| 35 | MiniCPM-V 4.735B · 3B active mostly mix_q3gud_im Gen 93 t/s (1 run) | 1 | 93 1 run | no PP data | none |
| 36 | Qwen3.50.8B Alibaba · mostly Q8_0 Gen 90 t/s (1 run) · PP 2122 t/s (1 run) | 1 | 90 1 run | 2122 1 run | none |
| 37 | Bonsai27B PrismML · mostly Q1_0 Gen 73 t/s (1 run) | 1 | 73 1 run | no PP data | none |
| 38 | Mellum2.1 Thinking12B · 2.5B active mostly NVFP4 Gen 64 t/s (1 run) | 1 | 64 1 run | no PP data | none |
| 39 | Qwen3.8 Escha-W227B Alibaba · mostly 2.469 bpw Gen 59 t/s (1 run) | 1 | 59 1 run | no PP data | none |
| 40 | Qwen2.53B Alibaba · mostly Q4_K_M Gen 47 t/s (1 run) · PP 557 t/s (1 run) | 1 | 47 1 run | 557 1 run | none |
| 41 | Mimo 2.6 Distill-Qwen9B Xiaomi · mostly Q8_0 Gen 42 t/s (1 run) · PP 1718 t/s (1 run) | 1 | 42 1 run | 1718 1 run | none |
| 42 | Qwen332B Alibaba · mostly 3.6577 bpw Gen 38 t/s (1 run) | 1 | 38 1 run | no PP data | none |
| 43 | Qwen3.6 APEX-I-Quality35B · 3B active Alibaba Gen 37 t/s (1 run) | 1 | 37 1 run | no PP data | none |
| 44 | Qwen31.7B Alibaba · mostly Q8_0 Gen 33 t/s (1 run) | 1 | 33 1 run | no PP data | none |
| 45 | Caphlon-227B mostly PTQ1_SB Gen 16 t/s (1 run) · PP 278 t/s (1 run) | 1 | 16 1 run | 278 1 run | none |
| 46 | Qwen3.6 Coder27B · 3B active Alibaba · mostly Q4_K_M Gen 14 t/s (1 run) | 1 | 14 1 run | no PP data | none |
| 47 | Agents-A1 Uncensored-MTP-APEX35B · 3B active InternScience · mostly APEX Compact No generation data · with speculation 97 t/s (1) | 1 | no generation data | no PP data | 97 1 run |
| 48 | GLM-5.2744B · 40B active Zhipu AI · mostly fmt=4 grouped int4, gs=64 No generation data | 1 | no generation data | no PP data | none |
| 49 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 50 | Llama 27B mostly Q4_0 No generation data | 1 | no generation data | no PP data | none |
| 51 | Mellum2.112B · 2.5B active mostly Q8 No generation data | 1 | no generation data | no PP data | none |
| 52 | Metis-1.6 Logos10B · 1.2B active mostly FP8 No generation data · with speculation 187 t/s (1) | 1 | no generation data | no PP data | 187 1 run |
| 53 | Nemotron 3.5 Bolt74B · 9B active mostly IQ2_XSS No generation data | 1 | no generation data | no PP data | none |
| 54 | Ornith1.535B · 3B active mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 55 | Poolside Laguna S 2.1118B · 8B active mostly IQ3_S No generation data | 1 | no generation data | no PP data | none |
| 56 | Qwen3-Coder30B · 3B active mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 57 | Qwen3-Next80B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 58 | Qwen3.52B Alibaba · mostly int4 GPTQ group 32 No generation data · with speculation 620 t/s (1) | 1 | no generation data | no PP data | 620 1 run |
| 59 | Qwen3.5 DeepSeek-V4-Flash9B Alibaba · mostly Q6_K No generation data · with speculation 49 t/s (1) | 1 | no generation data | no PP data | 49 1 run |
| 60 | Qwen3.527B Alibaba · mostly IQ3_S No generation data · PP 1049 t/s (1 run) | 1 | no generation data | 1049 1 run | none |
| 61 | Qwen3.6 Math-Physics11B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 62 | Qwen3.6 Omnimerge-v427B Alibaba · mostly IQ3_M No generation data · with speculation 46 t/s (1) | 1 | no generation data | no PP data | 46 1 run |
| 63 | Qwen3.6 BigBang-v135B · 3B active Alibaba · mostly IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 64 | Qwen3.8 TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX27B Alibaba No generation data · with speculation 30 t/s (1) | 1 | no generation data | no PP data | 30 1 run |
| 65 | Qwen3.8 Huihui-Abliterated27B Alibaba · mostly IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 66 | Qwen3.8 OrcaRouter Uncensored27B Alibaba · mostly IQ3_XXS No generation data · with speculation 83 t/s (1) | 1 | no generation data | no PP data | 83 1 run |
| 67 | Qwen3.8 QUASAR27B Alibaba · mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 68 | Qwen3.8 ASCII-Condensed27B Alibaba · mostly UD-IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 69 | Qwen3.8 abliterated27B Alibaba · mostly IQ3_S No generation data · with speculation 105 t/s (1) | 1 | no generation data | no PP data | 105 1 run |
| 70 | Qwen3.8 Unsloth Dynamic v3.127B Alibaba · mostly UD-Q2_K_XL No generation data · with speculation 78 t/s (1) | 1 | no generation data | no PP data | 78 1 run |
| 71 | Qwen3.8 Flash-Next Abliterated125B · 6B active Alibaba · mostly IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 72 | Qwen3.8 Swift-1.5 Flash-Next125B · 6B active Alibaba · mostly IQ2_XS No generation data | 1 | no generation data | no PP data | none |
| 73 | Raon-1 mini-preview29B · 4B active mostly IQ3_KS No generation data | 1 | no generation data | no PP data | none |
| 74 | gpt-oss117B · 5.1B active OpenAI No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 16GB of VRAM?
On setups with more than 12 GB and up to 16 GB of memory, the model people report most is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 16GB count as 16GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 16GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.