llamaperf

Best local LLMs for 8GB VRAM

How far 8GB of graphics memory goes for local models on cards like NVIDIA RTX 4060, NVIDIA RTX 3070, NVIDIA RTX 4060 Laptop 8GB or NVIDIA RTX 5060 8GB: which models fit, and what people actually run on it.

Is 8GB of VRAM enough?

8GB holds a dense model of up to about 6B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Gemma 4 8B, which needs about 6.6 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.6 35B · 3B active, mostly at Q4_K_XL, which leaves little room for context or runs partly from system RAM.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 8GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

ModelBest quantNeeds
Qwen3.5 9BQ3_K_M6 GB
Ornith1.5 9BQ3_K_M6 GB
Qwen3.8 9BQ3_K_M6 GB
DeepSeek V4 Flash 9BQ3_K_M6.1 GB
GLM-4 9BQ3_K_M6 GB
Gemma 2 9BQ3_K_M6.6 GB
Mimo 2.6 9BQ3_K_M5.9 GB
Gemma 4 8BQ4_K_M6.6 GB
Qwen3 8BQ3_K_M6.5 GB
Llama 3.1 8BQ3_K_M6.5 GB
Llama 3 8BQ3_K_M6.5 GB
Qwen3-VL 8BQ3_K_M6.5 GB

And 56 smaller models. The calculator lists them all for your card.

What people run on 8GB

Model sizes reported on setups with up to 8 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.635B · 3B active
Alibaba · mostly Q4_K_XL
No generation data
27
2Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly IQ2_XS
No generation data
13
3Qwen3.827B
Alibaba · mostly IQ3_S
Gen 30 t/s (1 run)
8
4Qwen3.59B
Alibaba · mostly Q4_K_M
Gen 34 t/s (2 runs) · PP 1829 t/s (1 run)
7
5Ornith1.535B · 3B active
mostly Q4_K_M
No generation data
5
6Bonsai 227B
PrismML · mostly PTQ1_0
Gen 18 t/s (4 runs) · PP 31 t/s (3 runs)
4
7Gemma 426B · 4B active
Google DeepMind · mostly Q5_K_P
No generation data
4
8Gemma 4 E2B5.1B
Google DeepMind · mostly Q4
Gen 80 t/s (1 run)
3
9LFM31.7B
mostly 4-bit
Gen 66 t/s (2 runs) · PP 5258 t/s (1 run)
3
10Qwen30.6B
Alibaba · mostly Q4_K_M
Gen 70 t/s (2 runs) · PP 494 t/s (1 run)
2
11Gemma 4 E4B8B
Google DeepMind · mostly Q3_K_M
Gen 52 t/s (1 run)
2
12Qwen2.57B
Alibaba
Gen 32 t/s (2 runs)
2
13Gemma 412B
Google DeepMind · mostly Q6_K
Gen 13 t/s (1 run)
2
14DeepSeek V4.1 Flash552B · 16B active
mostly FP8
No generation data
2
15Qwen2.5-Coder7B
Alibaba · mostly Q4_K_M
No generation data
2
16Qwen3.89B
Alibaba
No generation data
2
17Qwen2.5 MoE-A0.9B1.5B · 0.9B active
Alibaba · mostly Q4_K_M
Gen 253 t/s (1 run)
1
18SmolLM20.1B
mostly Q4_K_M
Gen 165 t/s (1 run)
1
19LFM2.51.2B
Liquid AI
Gen 129 t/s (1 run) · PP 8500 t/s (1 run)
1
20Qwen2.5 Coder-1.5B1.5B
Alibaba
Gen 99 t/s (1 run)
1
21Qwen2.5-Coder FFCraft1.5B
Alibaba · mostly Q8_0
Gen 57 t/s (1 run)
1
22Qwen38B
Alibaba · mostly Q4_K_M
Gen 57 t/s (1 run)
1
23Ling-3.0 Tiny7.9B · 1.3B active
Ant Group · mostly Q4_K_M
Gen 54 t/s (1 run) · PP 245 t/s (1 run)
1
24Qwen3.5 reasoning9B
Alibaba
Gen 50 t/s (1 run)
1
25LFM3 Omni2B
Gen 44 t/s (1 run)
1
26Qwen3.52B
Alibaba
Gen 23 t/s (1 run)
1
27LFM2.52.6B
Liquid AI · mostly QAD-Q4_0
Gen 22 t/s (1 run) · PP 200 t/s (1 run)
1
28DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly MXFP4
Gen 3.2 t/s (1 run)
1
29LFM3 VL2B
mostly OptiQ-4bit
Gen 1.0 t/s (1 run)
1
30Bonsai27B
PrismML · mostly 1-bit
No generation data
1
31Gemma 426B
Google DeepMind
No generation data
1
32Gemma 4 Ultra-Uncensored-Heretic26B · 4B active
Google DeepMind · mostly IQ4_XS
No generation data
1
33Gemma 4 Uncensored26B · 4B active
Google DeepMind
No generation data
1
34LFM2.524B · 2.5B active
Liquid AI · mostly Q4_K_M
No generation data
1
35Mellum2.112B · 2.5B active
mostly Q8
No generation data
1
36Nemotron 3.5 Bolt E4B8B
mostly BF16
No generation data
1
37Ornith1.59B
No generation data · with speculation 15 t/s (1)
1
38Qwen2.51.5B
Alibaba · mostly Q2_K
No generation data
1
39Qwen2.514B
Alibaba · mostly Q4_K_M
No generation data
1
40Qwen34B
Alibaba · mostly Q4_K_M
No generation data
1
41Qwen330B · 3B active
Alibaba · mostly Q4_K_S
No generation data
1
42Qwen3-Coder30B · 3B active
mostly Q4_K_M
No generation data
1
43Qwen3.5 Claude-4.6-Opus-Reasoning-Distilled-i127B
Alibaba · mostly IQ1_S
No generation data
1
44Qwen3.535B · 3B active
Alibaba · mostly Q2
No generation data
1
45Qwen3.627B
Alibaba · mostly Q4_K_M
No generation data
1
46Qwen3.6 Claude-4.6-Opus-Reasoning-Distilled35B · 3B active
Alibaba · mostly IQ4_XS
No generation data
1
47Qwen3.6 Uncensored-HauhauCS-Aggressive35B · 3B active
Alibaba · mostly Q4_K_M
No generation data
1
48Qwen3.6 CyberTiel35B · 3B active
Alibaba · mostly APEX-I-MiniPlus-V2.1
No generation data
1
49Qwen3.8 Swift27B
Alibaba · mostly W4A16
No generation data
1
50Qwen3.835B · 3B active
Alibaba · mostly IQ2_M
No generation data
1
51Qwen3.8 Swift-1.5-125B-A6B125B · 6B active
Alibaba · mostly IQ2_XS
No generation data
1
52Qwen3.8 Flash-Next Gyro S125B · 6B active
Alibaba · mostly TQ1_0
No generation data
1

Frequently asked

What do people run on 8GB of VRAM?

On setups with up to 8 GB of memory, the model people report most is Qwen3.6 35B · 3B active, followed by Qwen3.8 125B · 6B active and Qwen3.8 27B. The table lists each with the quant most people used and the typical speed.

Does a Mac with 8GB count as 8GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 8GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.