Best local LLMs for 8GB VRAM
How far 8GB of graphics memory goes for local models on cards like NVIDIA RTX 4060, NVIDIA RTX 3070, NVIDIA RTX 4060 Laptop 8GB or NVIDIA RTX 5060 8GB: which models fit, and what people actually run on it.
Is 8GB of VRAM enough?
8GB holds a dense model of up to about 6B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Gemma 4 8B, which needs about 6.6 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.6 35B · 3B active, mostly at Q4_K_XL, which leaves little room for context or runs partly from system RAM.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 8GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| Qwen3.5 9B | Q3_K_M | 6 GB |
| Ornith1.5 9B | Q3_K_M | 6 GB |
| Qwen3.8 9B | Q3_K_M | 6 GB |
| DeepSeek V4 Flash 9B | Q3_K_M | 6.1 GB |
| GLM-4 9B | Q3_K_M | 6 GB |
| Gemma 2 9B | Q3_K_M | 6.6 GB |
| Mimo 2.6 9B | Q3_K_M | 5.9 GB |
| Gemma 4 8B | Q4_K_M | 6.6 GB |
| Qwen3 8B | Q3_K_M | 6.5 GB |
| Llama 3.1 8B | Q3_K_M | 6.5 GB |
| Llama 3 8B | Q3_K_M | 6.5 GB |
| Qwen3-VL 8B | Q3_K_M | 6.5 GB |
And 56 smaller models. The calculator lists them all for your card.
What people run on 8GB
Model sizes reported on setups with up to 8 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_XL No generation data | 27 | no generation data | no PP data | none |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ2_XS No generation data | 13 | no generation data | no PP data | none |
| 3 | Qwen3.827B Alibaba · mostly IQ3_S Gen 30 t/s (1 run) | 8 | 30 1 run | no PP data | none |
| 4 | Qwen3.59B Alibaba · mostly Q4_K_M Gen 34 t/s (2 runs) · PP 1829 t/s (1 run) | 7 | 34 2 runs | 1829 1 run | none |
| 5 | Ornith1.535B · 3B active mostly Q4_K_M No generation data | 5 | no generation data | no PP data | none |
| 6 | Bonsai 227B PrismML · mostly PTQ1_0 Gen 18 t/s (4 runs) · PP 31 t/s (3 runs) | 4 | 18 4 runs, 4 devices | 31 3 runs | none |
| 7 | Gemma 426B · 4B active Google DeepMind · mostly Q5_K_P No generation data | 4 | no generation data | no PP data | none |
| 8 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4 Gen 80 t/s (1 run) | 3 | 80 1 run | no PP data | none |
| 9 | LFM31.7B mostly 4-bit Gen 66 t/s (2 runs) · PP 5258 t/s (1 run) | 3 | 66 2 runs, 2 devices | 5258 1 run | none |
| 10 | Qwen30.6B Alibaba · mostly Q4_K_M Gen 70 t/s (2 runs) · PP 494 t/s (1 run) | 2 | 70 2 runs | 494 1 run | none |
| 11 | Gemma 4 E4B8B Google DeepMind · mostly Q3_K_M Gen 52 t/s (1 run) | 2 | 52 1 run | no PP data | none |
| 12 | Qwen2.57B Alibaba Gen 32 t/s (2 runs) | 2 | 32 2 runs | no PP data | none |
| 13 | Gemma 412B Google DeepMind · mostly Q6_K Gen 13 t/s (1 run) | 2 | 13 1 run | no PP data | none |
| 14 | DeepSeek V4.1 Flash552B · 16B active mostly FP8 No generation data | 2 | no generation data | no PP data | none |
| 15 | Qwen2.5-Coder7B Alibaba · mostly Q4_K_M No generation data | 2 | no generation data | no PP data | none |
| 16 | Qwen3.89B Alibaba No generation data | 2 | no generation data | no PP data | none |
| 17 | Qwen2.5 MoE-A0.9B1.5B · 0.9B active Alibaba · mostly Q4_K_M Gen 253 t/s (1 run) | 1 | 253 1 run | no PP data | none |
| 18 | SmolLM20.1B mostly Q4_K_M Gen 165 t/s (1 run) | 1 | 165 1 run | no PP data | none |
| 19 | LFM2.51.2B Liquid AI Gen 129 t/s (1 run) · PP 8500 t/s (1 run) | 1 | 129 1 run | 8500 1 run | none |
| 20 | Qwen2.5 Coder-1.5B1.5B Alibaba Gen 99 t/s (1 run) | 1 | 99 1 run | no PP data | none |
| 21 | Qwen2.5-Coder FFCraft1.5B Alibaba · mostly Q8_0 Gen 57 t/s (1 run) | 1 | 57 1 run | no PP data | none |
| 22 | Qwen38B Alibaba · mostly Q4_K_M Gen 57 t/s (1 run) | 1 | 57 1 run | no PP data | none |
| 23 | Ling-3.0 Tiny7.9B · 1.3B active Ant Group · mostly Q4_K_M Gen 54 t/s (1 run) · PP 245 t/s (1 run) | 1 | 54 1 run | 245 1 run | none |
| 24 | Qwen3.5 reasoning9B Alibaba Gen 50 t/s (1 run) | 1 | 50 1 run | no PP data | none |
| 25 | LFM3 Omni2B Gen 44 t/s (1 run) | 1 | 44 1 run | no PP data | none |
| 26 | Qwen3.52B Alibaba Gen 23 t/s (1 run) | 1 | 23 1 run | no PP data | none |
| 27 | LFM2.52.6B Liquid AI · mostly QAD-Q4_0 Gen 22 t/s (1 run) · PP 200 t/s (1 run) | 1 | 22 1 run | 200 1 run | none |
| 28 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly MXFP4 Gen 3.2 t/s (1 run) | 1 | 3.2 1 run | no PP data | none |
| 29 | LFM3 VL2B mostly OptiQ-4bit Gen 1.0 t/s (1 run) | 1 | 1.0 1 run | no PP data | none |
| 30 | Bonsai27B PrismML · mostly 1-bit No generation data | 1 | no generation data | no PP data | none |
| 31 | Gemma 426B Google DeepMind No generation data | 1 | no generation data | no PP data | none |
| 32 | Gemma 4 Ultra-Uncensored-Heretic26B · 4B active Google DeepMind · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 33 | Gemma 4 Uncensored26B · 4B active Google DeepMind No generation data | 1 | no generation data | no PP data | none |
| 34 | LFM2.524B · 2.5B active Liquid AI · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 35 | Mellum2.112B · 2.5B active mostly Q8 No generation data | 1 | no generation data | no PP data | none |
| 36 | Nemotron 3.5 Bolt E4B8B mostly BF16 No generation data | 1 | no generation data | no PP data | none |
| 37 | Ornith1.59B No generation data · with speculation 15 t/s (1) | 1 | no generation data | no PP data | 15 1 run |
| 38 | Qwen2.51.5B Alibaba · mostly Q2_K No generation data | 1 | no generation data | no PP data | none |
| 39 | Qwen2.514B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 40 | Qwen34B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 41 | Qwen330B · 3B active Alibaba · mostly Q4_K_S No generation data | 1 | no generation data | no PP data | none |
| 42 | Qwen3-Coder30B · 3B active mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 43 | Qwen3.5 Claude-4.6-Opus-Reasoning-Distilled-i127B Alibaba · mostly IQ1_S No generation data | 1 | no generation data | no PP data | none |
| 44 | Qwen3.535B · 3B active Alibaba · mostly Q2 No generation data | 1 | no generation data | no PP data | none |
| 45 | Qwen3.627B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 46 | Qwen3.6 Claude-4.6-Opus-Reasoning-Distilled35B · 3B active Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 47 | Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 48 | Qwen3.6 CyberTiel35B · 3B active Alibaba · mostly APEX-I-MiniPlus-V2.1 No generation data | 1 | no generation data | no PP data | none |
| 49 | Qwen3.8 Swift27B Alibaba · mostly W4A16 No generation data | 1 | no generation data | no PP data | none |
| 50 | Qwen3.835B · 3B active Alibaba · mostly IQ2_M No generation data | 1 | no generation data | no PP data | none |
| 51 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ2_XS No generation data | 1 | no generation data | no PP data | none |
| 52 | Qwen3.8 Flash-Next Gyro S125B · 6B active Alibaba · mostly TQ1_0 No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 8GB of VRAM?
On setups with up to 8 GB of memory, the model people report most is Qwen3.6 35B · 3B active, followed by Qwen3.8 125B · 6B active and Qwen3.8 27B. The table lists each with the quant most people used and the typical speed.
Does a Mac with 8GB count as 8GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 8GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.