llamaperf

Best local LLMs for 64GB VRAM

How far 64GB of graphics memory goes for local models on cards like NVIDIA CMP 170HX 64GB (unlocked) or AMD Instinct MI210: which models fit, and what people actually run on it.

Is 64GB of VRAM enough?

64GB holds a dense model of up to about 87B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3-Next 80B · 3B active, which needs about 47.5 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.8 125B · 6B active, mostly at IQ4_XS, which leaves little room for context or runs partly from system RAM.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 64GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

And 118 smaller models. The calculator lists them all for your card.

What people run on 64GB

Model sizes reported on setups with more than 48 GB and up to 64 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly IQ4_XS
Gen 42 t/s (3 runs) · PP 621 t/s (3 runs) · with speculation 43 t/s (2)
40
2Qwen3.827B
Alibaba · mostly Q4_K_M
Gen 15 t/s (8 runs) · PP 126 t/s (6 runs) · with speculation 38 t/s (12)
37
3Qwen3.635B · 3B active
Alibaba · mostly 4-bit
Gen 71 t/s (7 runs) · PP 659 t/s (4 runs) · with speculation 92 t/s (4)
12
4DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly IQ3-XXS
No generation data
11
5DeepSeek V4.1 Flash552B · 16B active
mostly IQ2_XXS
No generation data
7
6Gemma 426B · 4B active
Google DeepMind · mostly UD-Q4_K_M
Gen 62 t/s (5 runs) · PP 598 t/s (4 runs)
6
7Qwen3.627B
Alibaba · mostly Q4_K_M
Gen 22 t/s (4 runs) · PP 126 t/s (3 runs) · with speculation 63 t/s (1)
6
8gpt-oss21B · 3.6B active
OpenAI · mostly MXFP4
Gen 78 t/s (4 runs) · PP 768 t/s (4 runs)
4
9Qwen3.59B
Alibaba · mostly Q4_K_M
Gen 64 t/s (2 runs) · PP 314 t/s (2 runs)
4
10Qwen3.8 Swift-1.527B
Alibaba · mostly FP8
No generation data · with speculation 35 t/s (2)
4
11Qwen3.535B · 3B active
Alibaba · mostly 4-bit (source config)
Gen 49 t/s (3 runs) · PP 1261 t/s (2 runs)
3
12Gemma 431B
Google DeepMind · mostly Q4_K_M
Gen 22 t/s (1 run) · PP 63 t/s (1 run)
3
13Qwen330B · 3B active
Alibaba · mostly Q4_K_M
Gen 86 t/s (2 runs) · PP 793 t/s (2 runs)
2
14Gemma 4 E2B5.1B
Google DeepMind · mostly Q8_0
Gen 82 t/s (2 runs) · PP 1552 t/s (2 runs)
2
15Qwen2.5-Omni3B
mostly Q8_0
Gen 69 t/s (2 runs) · PP 1202 t/s (2 runs)
2
16Qwen3-VL4B
mostly Q8_0
Gen 54 t/s (2 runs) · PP 918 t/s (2 runs)
2
17Gemma 4 E4B8B
Google DeepMind · mostly Q8_0
Gen 44 t/s (2 runs) · PP 778 t/s (2 runs)
2
18Gemma 412B
Google DeepMind · mostly Q4_K_M
Gen 30 t/s (2 runs) · PP 292 t/s (2 runs)
2
19Mistral Small 3.124B
mostly Q4_K_M
Gen 18 t/s (2 runs) · PP 145 t/s (2 runs)
2
20GLM-5.3 Flash320B · 18B active
Zhipu AI · mostly 2.05bpw EXL3
No generation data
2
21Kimi K32800B · 104B active
Moonshot AI · mostly VQ3R
No generation data
2
22Ministral 33B
mostly 4-bit (source config)
Gen 128 t/s (1 run) · PP 743 t/s (1 run)
1
23LFM224B · 2B active
mostly 4-bit mixed (config)
Gen 118 t/s (1 run) · PP 616 t/s (1 run)
1
24Qwen3-Next80B · 3B active
Alibaba · mostly W4A16
Gen 116 t/s (1 run)
1
25Qwen3 30B-A3B, standard 8 experts30B · 3.3B active
Alibaba · mostly MLX 4-bit affine, group 64
Gen 98 t/s (1 run)
1
26Qwen2.57B
Alibaba · mostly 4-bit (source config)
Gen 77 t/s (1 run) · PP 603 t/s (1 run)
1
27Phi-3.5 mini3.8B
Microsoft · mostly 4bit
Gen 72 t/s (1 run)
1
28Qwen3.54B
Alibaba · mostly Q4_K_M
Gen 70 t/s (1 run)
1
29Qwen3.8 27B dense27B
Alibaba · mostly Q4_K_M
Gen 21 t/s (1 run) · PP 144 t/s (1 run)
1
30Qwen3.527B
Alibaba · mostly 4-bit (source config)
Gen 21 t/s (1 run) · PP 91 t/s (1 run)
1
31Bonsai 2 Ternary-Bonsai-2-27B27B
PrismML · mostly PQ2_0
Gen 21 t/s (1 run) · PP 127 t/s (1 run)
1
32Qwen 2.5 Coder32B
mostly 4-bit (source config)
Gen 18 t/s (1 run) · PP 127 t/s (1 run)
1
33DeepSeek V4 Flash284B
DeepSeek · mostly q2
No generation data
1
34DeepSeek V4.1 Flash763B
mostly IQ2_XXS
No generation data
1
35Gemma 312B
Google · mostly QAT 4-bit
No generation data
1
36Llama 3.370B
Meta · mostly Q4_K_M
No generation data
1
37Muse Glimmer30B
Meta · mostly Q6_K_XL
No generation data
1
38Qwen3-Coder30B · 3B active
mostly Q4_K_M
No generation data
1
39Qwen3.8 Splash-HQ27B
Alibaba · mostly Q8
No generation data · with speculation 37 t/s (1)
1
40Qwen3.8 Uncensored27B
Alibaba · mostly Q8_0
No generation data
1
41Qwen3.8 Speed-Optimized27B
Alibaba
No generation data
1
42Qwen3.82400B · 95B active
Alibaba · mostly Q1_0
No generation data
1
43Qwopus3.8 Coder-Flash27B
mostly Q5_K_M
No generation data
1

Frequently asked

What do people run on 64GB of VRAM?

On setups with more than 48 GB and up to 64 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.

Does a Mac with 64GB count as 64GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 64GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.