llamaperf

Best local LLMs for 48GB VRAM

How far 48GB of graphics memory goes for local models on cards like NVIDIA A40 48GB, AMD Radeon Pro W7900, NVIDIA L40S or NVIDIA RTX 6000 Ada: which models fit, and what people actually run on it.

Is 48GB of VRAM enough?

48GB holds a dense model of up to about 64B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.6 35B · 3B active, which needs about 36.9 GB at Q8_0 with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 48GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

And 113 smaller models. The calculator lists them all for your card.

What people run on 48GB

Model sizes reported on setups with more than 32 GB and up to 48 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.827B
Alibaba · mostly 4-bit
Gen 23 t/s (4 runs) · with speculation 38 t/s (11)
56
2Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly IQ3_S
Gen 90 t/s (1 run)
42
3Qwen3.635B · 3B active
Alibaba · mostly 4-bit
Gen 124 t/s (5 runs) · PP 3333 t/s (3 runs) · with speculation 123 t/s (5)
16
4DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly Q8_K_XL
No generation data
11
5Qwen3.59B
Alibaba · mostly 4-bit
Gen 83 t/s (2 runs) · PP 2369 t/s (1 run) · with speculation 116 t/s (3)
7
6Qwen3.54B
Alibaba · mostly Q4_K_M
Gen 109 t/s (4 runs) · PP 4100 t/s (1 run)
5
7Qwen3.627B
Alibaba · mostly 4-bit
No generation data · with speculation 49 t/s (2)
5
8Qwen34B
Alibaba · mostly 4-bit
Gen 186 t/s (3 runs) · PP 5239 t/s (2 runs)
3
9Gemma 4 E2B5.1B
Google DeepMind · mostly Q4_K_M
Gen 95 t/s (2 runs) · PP 7021 t/s (1 run)
3
10Qwen3.8 Uncensored27B
Alibaba · mostly 4-bit
No generation data · with speculation 59 t/s (2)
3
11Qwen3.8 Swift-1.527B
Alibaba · mostly Q3S
No generation data
3
12Qwen30.6B
Alibaba · mostly 4-bit
Gen 514 t/s (2 runs) · PP 22991 t/s (2 runs)
2
13Llama 3.21B
Meta · mostly 4-bit
Gen 440 t/s (1 run) · PP 16968 t/s (1 run)
2
14Qwen31.7B
Alibaba · mostly 4-bit
Gen 327 t/s (2 runs) · PP 11928 t/s (2 runs)
2
15Gemma 31B
Google · mostly 4-bit
Gen 295 t/s (1 run) · PP 16750 t/s (1 run)
2
16Llama 3.23B
Meta · mostly 4-bit
Gen 200 t/s (1 run) · PP 6123 t/s (1 run)
2
17Gemma 34B
Google · mostly 4-bit
Gen 163 t/s (2 runs) · PP 5656 t/s (2 runs)
2
18Qwen330B · 3B active
Alibaba · mostly 4-bit
Gen 149 t/s (1 run) · PP 3497 t/s (1 run)
2
19Gemma 426B · 4B active
Google DeepMind · mostly 4-bit
Gen 117 t/s (1 run) · PP 3423 t/s (1 run)
2
20Qwen 2.5 Coder7B
mostly 4-bit
Gen 114 t/s (2 runs) · PP 3132 t/s (2 runs)
2
21Qwen38B
Alibaba · mostly 4-bit
Gen 104 t/s (2 runs) · PP 2948 t/s (2 runs)
2
22Gemma 312B
Google · mostly 4-bit
Gen 64 t/s (2 runs) · PP 1778 t/s (2 runs)
2
23Qwen314B
Alibaba · mostly 4-bit
Gen 60 t/s (2 runs) · PP 1661 t/s (2 runs)
2
24Qwen 2.5 Coder14B
mostly 4-bit
Gen 58 t/s (2 runs) · PP 1633 t/s (2 runs)
2
25Gemma 327B
Google · mostly 4-bit
Gen 30 t/s (1 run) · PP 751 t/s (1 run)
2
26Qwen3.527B
Alibaba · mostly Q4_K_M
Gen 29 t/s (1 run) · PP 596 t/s (2 runs)
2
27Qwen332B
Alibaba · mostly 4-bit
Gen 27 t/s (2 runs) · PP 699 t/s (2 runs)
2
28Qwen 2.5 Coder32B
mostly 4-bit
Gen 27 t/s (2 runs) · PP 710 t/s (2 runs)
2
29DeepSeek V4.1 Flash552B · 16B active
mostly 2.6bit
No generation data
2
30GLM-5.3 Flash320B · 18B active
Zhipu AI · mostly GSQ-RCO 3.0bit Q4_K attention
No generation data
2
31Inkling 2 Mini26B · 3.4B active
mostly IQ4_XS
No generation data
2
32Qwen3.535B · 3B active
Alibaba · mostly Q4_K_M
Gen 111 t/s (1 run) · PP 3378 t/s (1 run)
1
33Gemma 4 E4B8B
Google DeepMind · mostly Q4_K_M
Gen 111 t/s (1 run) · PP 4217 t/s (1 run)
1
34RWKV72.9B
mostly F16
Gen 49 t/s (1 run) · PP 4316 t/s (1 run)
1
35DeepSeek V4 Flash 9B distill9B
DeepSeek · mostly Q4_K_M
Gen 44 t/s (1 run)
1
36Kolibri-178B · 3.5B active
mostly IQ2_M
Gen 27 t/s (1 run) · PP 430 t/s (1 run)
1
37Gemma 431B
Google DeepMind · mostly Q4_K_M
Gen 25 t/s (1 run) · PP 627 t/s (1 run)
1
38BitNet b1.582B
mostly ternary
Gen 14 t/s (1 run)
1
39Muse Glimmer30B
Meta · mostly 8-bit
Gen 8.2 t/s (1 run)
1
40DeepSeek R1671B · 37B active
DeepSeek · mostly Q3
No generation data
1
41Llama 38B
mostly Q4_0
No generation data
1
42Llama 3.370B
Meta · mostly Q4_K_M
No generation data
1
43Qwen2.57B
Alibaba
No generation data
1
44Qwen3-Coder-Next80B · 3B active
mostly 6-bit
No generation data
1
45Qwen3.5122B · 10B active
Alibaba · mostly UD-IQ4_XS
No generation data
1
46Qwen3.8 Swift27B
Alibaba · mostly Q4_K_M
No generation data
1
47Qwen3.8 Swift-1.5-125B-A6B125B · 6B active
Alibaba · mostly IQ3-XXS
No generation data
1
48Qwen3.8 Flash-Next OrcaRouter125B · 6B active
Alibaba · mostly IQ4_XS
No generation data
1
49Qwen3.8 Flash-Next Uncensored-177B177B
Alibaba · mostly IQ3_XXS
No generation data
1
50gpt-oss21B · 3.6B active
OpenAI · mostly MXFP4-Q8
No generation data
1

Frequently asked

What do people run on 48GB of VRAM?

On setups with more than 32 GB and up to 48 GB of memory, the model people report most is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.

Does a Mac with 48GB count as 48GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 48GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.