Best local LLMs for 48GB VRAM
How far 48GB of graphics memory goes for local models on cards like NVIDIA A40 48GB, AMD Radeon Pro W7900, NVIDIA L40S or NVIDIA RTX 6000 Ada: which models fit, and what people actually run on it.
Is 48GB of VRAM enough?
48GB holds a dense model of up to about 64B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.6 35B · 3B active, which needs about 36.9 GB at Q8_0 with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 48GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| Qwen3-Next 80B · 3B active | Q3_K_M | 37.5 GB |
| Qwen3-Coder-Next 80B · 3B active | Q3_K_M | 36.9 GB |
| Nemotron 3.5 74B · 9B active | Q3_K_M | 36.9 GB |
| Nemotron 3.5 Bolt 74B · 9B active | Q3_K_M | 34.2 GB |
| Qwen2.5 72B | Q3_K_M | 36 GB |
| Llama 3.3 70B | Q3_K_M | 35.1 GB |
| Llama 3.1 70B | Q3_K_M | 35.1 GB |
| Qwen3.6 35B · 3B active | Q8_0 | 36.9 GB |
| Qwen3.5 35B · 3B active | Q8_0 | 36.9 GB |
| Ornith1.5 35B · 3B active | Q8_0 | 36.9 GB |
| MiniCPM-V 4.7 35B · 3B active | Q8_0 | 36.9 GB |
| Agents-A1 35B · 3B active | Q8_0 | 36.9 GB |
And 113 smaller models. The calculator lists them all for your card.
What people run on 48GB
Model sizes reported on setups with more than 32 GB and up to 48 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly 4-bit Gen 23 t/s (4 runs) · with speculation 38 t/s (11) | 56 | 23 4 runs, 3 devices | no PP data | 38 11 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ3_S Gen 90 t/s (1 run) | 42 | 90 1 run | no PP data | none |
| 3 | Qwen3.635B · 3B active Alibaba · mostly 4-bit Gen 124 t/s (5 runs) · PP 3333 t/s (3 runs) · with speculation 123 t/s (5) | 16 | 124 5 runs, 4 devices | 3333 3 runs | 123 5 runs |
| 4 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly Q8_K_XL No generation data | 11 | no generation data | no PP data | none |
| 5 | Qwen3.59B Alibaba · mostly 4-bit Gen 83 t/s (2 runs) · PP 2369 t/s (1 run) · with speculation 116 t/s (3) | 7 | 83 2 runs, 2 devices | 2369 1 run | 116 3 runs |
| 6 | Qwen3.54B Alibaba · mostly Q4_K_M Gen 109 t/s (4 runs) · PP 4100 t/s (1 run) | 5 | 109 4 runs, 3 devices | 4100 1 run | none |
| 7 | Qwen3.627B Alibaba · mostly 4-bit No generation data · with speculation 49 t/s (2) | 5 | no generation data | no PP data | 49 2 runs |
| 8 | Qwen34B Alibaba · mostly 4-bit Gen 186 t/s (3 runs) · PP 5239 t/s (2 runs) | 3 | 186 3 runs, 2 devices | 5239 2 runs | none |
| 9 | Gemma 4 E2B5.1B Google DeepMind · mostly Q4_K_M Gen 95 t/s (2 runs) · PP 7021 t/s (1 run) | 3 | 95 2 runs, 2 devices | 7021 1 run | none |
| 10 | Qwen3.8 Uncensored27B Alibaba · mostly 4-bit No generation data · with speculation 59 t/s (2) | 3 | no generation data | no PP data | 59 2 runs |
| 11 | Qwen3.8 Swift-1.527B Alibaba · mostly Q3S No generation data | 3 | no generation data | no PP data | none |
| 12 | Qwen30.6B Alibaba · mostly 4-bit Gen 514 t/s (2 runs) · PP 22991 t/s (2 runs) | 2 | 514 2 runs | 22991 2 runs | none |
| 13 | Llama 3.21B Meta · mostly 4-bit Gen 440 t/s (1 run) · PP 16968 t/s (1 run) | 2 | 440 1 run | 16968 1 run | none |
| 14 | Qwen31.7B Alibaba · mostly 4-bit Gen 327 t/s (2 runs) · PP 11928 t/s (2 runs) | 2 | 327 2 runs | 11928 2 runs | none |
| 15 | Gemma 31B Google · mostly 4-bit Gen 295 t/s (1 run) · PP 16750 t/s (1 run) | 2 | 295 1 run | 16750 1 run | none |
| 16 | Llama 3.23B Meta · mostly 4-bit Gen 200 t/s (1 run) · PP 6123 t/s (1 run) | 2 | 200 1 run | 6123 1 run | none |
| 17 | Gemma 34B Google · mostly 4-bit Gen 163 t/s (2 runs) · PP 5656 t/s (2 runs) | 2 | 163 2 runs | 5656 2 runs | none |
| 18 | Qwen330B · 3B active Alibaba · mostly 4-bit Gen 149 t/s (1 run) · PP 3497 t/s (1 run) | 2 | 149 1 run | 3497 1 run | none |
| 19 | Gemma 426B · 4B active Google DeepMind · mostly 4-bit Gen 117 t/s (1 run) · PP 3423 t/s (1 run) | 2 | 117 1 run | 3423 1 run | none |
| 20 | Qwen 2.5 Coder7B mostly 4-bit Gen 114 t/s (2 runs) · PP 3132 t/s (2 runs) | 2 | 114 2 runs | 3132 2 runs | none |
| 21 | Qwen38B Alibaba · mostly 4-bit Gen 104 t/s (2 runs) · PP 2948 t/s (2 runs) | 2 | 104 2 runs | 2948 2 runs | none |
| 22 | Gemma 312B Google · mostly 4-bit Gen 64 t/s (2 runs) · PP 1778 t/s (2 runs) | 2 | 64 2 runs | 1778 2 runs | none |
| 23 | Qwen314B Alibaba · mostly 4-bit Gen 60 t/s (2 runs) · PP 1661 t/s (2 runs) | 2 | 60 2 runs | 1661 2 runs | none |
| 24 | Qwen 2.5 Coder14B mostly 4-bit Gen 58 t/s (2 runs) · PP 1633 t/s (2 runs) | 2 | 58 2 runs | 1633 2 runs | none |
| 25 | Gemma 327B Google · mostly 4-bit Gen 30 t/s (1 run) · PP 751 t/s (1 run) | 2 | 30 1 run | 751 1 run | none |
| 26 | Qwen3.527B Alibaba · mostly Q4_K_M Gen 29 t/s (1 run) · PP 596 t/s (2 runs) | 2 | 29 1 run | 596 2 runs | none |
| 27 | Qwen332B Alibaba · mostly 4-bit Gen 27 t/s (2 runs) · PP 699 t/s (2 runs) | 2 | 27 2 runs | 699 2 runs | none |
| 28 | Qwen 2.5 Coder32B mostly 4-bit Gen 27 t/s (2 runs) · PP 710 t/s (2 runs) | 2 | 27 2 runs | 710 2 runs | none |
| 29 | DeepSeek V4.1 Flash552B · 16B active mostly 2.6bit No generation data | 2 | no generation data | no PP data | none |
| 30 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly GSQ-RCO 3.0bit Q4_K attention No generation data | 2 | no generation data | no PP data | none |
| 31 | Inkling 2 Mini26B · 3.4B active mostly IQ4_XS No generation data | 2 | no generation data | no PP data | none |
| 32 | Qwen3.535B · 3B active Alibaba · mostly Q4_K_M Gen 111 t/s (1 run) · PP 3378 t/s (1 run) | 1 | 111 1 run | 3378 1 run | none |
| 33 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M Gen 111 t/s (1 run) · PP 4217 t/s (1 run) | 1 | 111 1 run | 4217 1 run | none |
| 34 | RWKV72.9B mostly F16 Gen 49 t/s (1 run) · PP 4316 t/s (1 run) | 1 | 49 1 run | 4316 1 run | none |
| 35 | DeepSeek V4 Flash 9B distill9B DeepSeek · mostly Q4_K_M Gen 44 t/s (1 run) | 1 | 44 1 run | no PP data | none |
| 36 | Kolibri-178B · 3.5B active mostly IQ2_M Gen 27 t/s (1 run) · PP 430 t/s (1 run) | 1 | 27 1 run | 430 1 run | none |
| 37 | Gemma 431B Google DeepMind · mostly Q4_K_M Gen 25 t/s (1 run) · PP 627 t/s (1 run) | 1 | 25 1 run | 627 1 run | none |
| 38 | BitNet b1.582B mostly ternary Gen 14 t/s (1 run) | 1 | 14 1 run | no PP data | none |
| 39 | Muse Glimmer30B Meta · mostly 8-bit Gen 8.2 t/s (1 run) | 1 | 8.2 1 run | no PP data | none |
| 40 | DeepSeek R1671B · 37B active DeepSeek · mostly Q3 No generation data | 1 | no generation data | no PP data | none |
| 41 | Llama 38B mostly Q4_0 No generation data | 1 | no generation data | no PP data | none |
| 42 | Llama 3.370B Meta · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 43 | Qwen2.57B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 44 | Qwen3-Coder-Next80B · 3B active mostly 6-bit No generation data | 1 | no generation data | no PP data | none |
| 45 | Qwen3.5122B · 10B active Alibaba · mostly UD-IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 46 | Qwen3.8 Swift27B Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 47 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ3-XXS No generation data | 1 | no generation data | no PP data | none |
| 48 | Qwen3.8 Flash-Next OrcaRouter125B · 6B active Alibaba · mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 49 | Qwen3.8 Flash-Next Uncensored-177B177B Alibaba · mostly IQ3_XXS No generation data | 1 | no generation data | no PP data | none |
| 50 | gpt-oss21B · 3.6B active OpenAI · mostly MXFP4-Q8 No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 48GB of VRAM?
On setups with more than 32 GB and up to 48 GB of memory, the model people report most is Qwen3.8 27B, followed by Qwen3.8 125B · 6B active and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 48GB count as 48GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 48GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.