Best local LLMs for 64GB VRAM
How far 64GB of graphics memory goes for local models on cards like NVIDIA CMP 170HX 64GB (unlocked) or AMD Instinct MI210: which models fit, and what people actually run on it.
Is 64GB of VRAM enough?
64GB holds a dense model of up to about 87B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3-Next 80B · 3B active, which needs about 47.5 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top. People do push further: the model reported most on these cards is Qwen3.8 125B · 6B active, mostly at IQ4_XS, which leaves little room for context or runs partly from system RAM.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 64GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| Nemotron 3 120B · 12B active | Q3_K_M | 54.3 GB |
| Nemotron 3.5 Super 120B · 12B active | Q3_K_M | 54.3 GB |
| Poolside Laguna S 2.1 118B · 8B active | Q3_K_M | 53.9 GB |
| gpt-oss 117B · 5.1B active | Q3_K_M | 53.2 GB |
| GLM-4.5-Air 106B · 12B active | Q3_K_M | 49.7 GB |
| Qwen3-Next 80B · 3B active | Q4_K_M | 47.5 GB |
| Qwen3-Coder-Next 80B · 3B active | Q4_K_M | 46.9 GB |
| Nemotron 3.5 74B · 9B active | Q4_K_M | 46.1 GB |
| Nemotron 3.5 Bolt 74B · 9B active | Q5_K_M | 52.7 GB |
| Qwen2.5 72B | Q5_K_M | 54 GB |
| Llama 3.3 70B | Q5_K_M | 52.6 GB |
| Llama 3.1 70B | Q5_K_M | 52.6 GB |
And 118 smaller models. The calculator lists them all for your card.
What people run on 64GB
Model sizes reported on setups with more than 48 GB and up to 64 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly IQ4_XS Gen 42 t/s (3 runs) · PP 621 t/s (3 runs) · with speculation 43 t/s (2) | 40 | 42 3 runs, 2 devices | 621 3 runs | 43 2 runs |
| 2 | Qwen3.827B Alibaba · mostly Q4_K_M Gen 15 t/s (8 runs) · PP 126 t/s (6 runs) · with speculation 38 t/s (12) | 37 | 15 8 runs, 5 devices | 126 6 runs | 38 12 runs |
| 3 | Qwen3.635B · 3B active Alibaba · mostly 4-bit Gen 71 t/s (7 runs) · PP 659 t/s (4 runs) · with speculation 92 t/s (4) | 12 | 71 7 runs, 4 devices | 659 4 runs | 92 4 runs |
| 4 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly IQ3-XXS No generation data | 11 | no generation data | no PP data | none |
| 5 | DeepSeek V4.1 Flash552B · 16B active mostly IQ2_XXS No generation data | 7 | no generation data | no PP data | none |
| 6 | Gemma 426B · 4B active Google DeepMind · mostly UD-Q4_K_M Gen 62 t/s (5 runs) · PP 598 t/s (4 runs) | 6 | 62 5 runs, 4 devices | 598 4 runs | none |
| 7 | Qwen3.627B Alibaba · mostly Q4_K_M Gen 22 t/s (4 runs) · PP 126 t/s (3 runs) · with speculation 63 t/s (1) | 6 | 22 4 runs, 3 devices | 126 3 runs | 63 1 run |
| 8 | gpt-oss21B · 3.6B active OpenAI · mostly MXFP4 Gen 78 t/s (4 runs) · PP 768 t/s (4 runs) | 4 | 78 4 runs, 2 devices | 768 4 runs | none |
| 9 | Qwen3.59B Alibaba · mostly Q4_K_M Gen 64 t/s (2 runs) · PP 314 t/s (2 runs) | 4 | 64 2 runs, 2 devices | 314 2 runs | none |
| 10 | Qwen3.8 Swift-1.527B Alibaba · mostly FP8 No generation data · with speculation 35 t/s (2) | 4 | no generation data | no PP data | 35 2 runs |
| 11 | Qwen3.535B · 3B active Alibaba · mostly 4-bit (source config) Gen 49 t/s (3 runs) · PP 1261 t/s (2 runs) | 3 | 49 3 runs, 2 devices | 1261 2 runs | none |
| 12 | Gemma 431B Google DeepMind · mostly Q4_K_M Gen 22 t/s (1 run) · PP 63 t/s (1 run) | 3 | 22 1 run | 63 1 run | none |
| 13 | Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 86 t/s (2 runs) · PP 793 t/s (2 runs) | 2 | 86 2 runs | 793 2 runs | none |
| 14 | Gemma 4 E2B5.1B Google DeepMind · mostly Q8_0 Gen 82 t/s (2 runs) · PP 1552 t/s (2 runs) | 2 | 82 2 runs | 1552 2 runs | none |
| 15 | Qwen2.5-Omni3B mostly Q8_0 Gen 69 t/s (2 runs) · PP 1202 t/s (2 runs) | 2 | 69 2 runs | 1202 2 runs | none |
| 16 | Qwen3-VL4B mostly Q8_0 Gen 54 t/s (2 runs) · PP 918 t/s (2 runs) | 2 | 54 2 runs | 918 2 runs | none |
| 17 | Gemma 4 E4B8B Google DeepMind · mostly Q8_0 Gen 44 t/s (2 runs) · PP 778 t/s (2 runs) | 2 | 44 2 runs | 778 2 runs | none |
| 18 | Gemma 412B Google DeepMind · mostly Q4_K_M Gen 30 t/s (2 runs) · PP 292 t/s (2 runs) | 2 | 30 2 runs | 292 2 runs | none |
| 19 | Mistral Small 3.124B mostly Q4_K_M Gen 18 t/s (2 runs) · PP 145 t/s (2 runs) | 2 | 18 2 runs | 145 2 runs | none |
| 20 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly 2.05bpw EXL3 No generation data | 2 | no generation data | no PP data | none |
| 21 | Kimi K32800B · 104B active Moonshot AI · mostly VQ3R No generation data | 2 | no generation data | no PP data | none |
| 22 | Ministral 33B mostly 4-bit (source config) Gen 128 t/s (1 run) · PP 743 t/s (1 run) | 1 | 128 1 run | 743 1 run | none |
| 23 | LFM224B · 2B active mostly 4-bit mixed (config) Gen 118 t/s (1 run) · PP 616 t/s (1 run) | 1 | 118 1 run | 616 1 run | none |
| 24 | Qwen3-Next80B · 3B active Alibaba · mostly W4A16 Gen 116 t/s (1 run) | 1 | 116 1 run | no PP data | none |
| 25 | Qwen3 30B-A3B, standard 8 experts30B · 3.3B active Alibaba · mostly MLX 4-bit affine, group 64 Gen 98 t/s (1 run) | 1 | 98 1 run | no PP data | none |
| 26 | Qwen2.57B Alibaba · mostly 4-bit (source config) Gen 77 t/s (1 run) · PP 603 t/s (1 run) | 1 | 77 1 run | 603 1 run | none |
| 27 | Phi-3.5 mini3.8B Microsoft · mostly 4bit Gen 72 t/s (1 run) | 1 | 72 1 run | no PP data | none |
| 28 | Qwen3.54B Alibaba · mostly Q4_K_M Gen 70 t/s (1 run) | 1 | 70 1 run | no PP data | none |
| 29 | Qwen3.8 27B dense27B Alibaba · mostly Q4_K_M Gen 21 t/s (1 run) · PP 144 t/s (1 run) | 1 | 21 1 run | 144 1 run | none |
| 30 | Qwen3.527B Alibaba · mostly 4-bit (source config) Gen 21 t/s (1 run) · PP 91 t/s (1 run) | 1 | 21 1 run | 91 1 run | none |
| 31 | Bonsai 2 Ternary-Bonsai-2-27B27B PrismML · mostly PQ2_0 Gen 21 t/s (1 run) · PP 127 t/s (1 run) | 1 | 21 1 run | 127 1 run | none |
| 32 | Qwen 2.5 Coder32B mostly 4-bit (source config) Gen 18 t/s (1 run) · PP 127 t/s (1 run) | 1 | 18 1 run | 127 1 run | none |
| 33 | DeepSeek V4 Flash284B DeepSeek · mostly q2 No generation data | 1 | no generation data | no PP data | none |
| 34 | DeepSeek V4.1 Flash763B mostly IQ2_XXS No generation data | 1 | no generation data | no PP data | none |
| 35 | Gemma 312B Google · mostly QAT 4-bit No generation data | 1 | no generation data | no PP data | none |
| 36 | Llama 3.370B Meta · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 37 | Muse Glimmer30B Meta · mostly Q6_K_XL No generation data | 1 | no generation data | no PP data | none |
| 38 | Qwen3-Coder30B · 3B active mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 39 | Qwen3.8 Splash-HQ27B Alibaba · mostly Q8 No generation data · with speculation 37 t/s (1) | 1 | no generation data | no PP data | 37 1 run |
| 40 | Qwen3.8 Uncensored27B Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 41 | Qwen3.8 Speed-Optimized27B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 42 | Qwen3.82400B · 95B active Alibaba · mostly Q1_0 No generation data | 1 | no generation data | no PP data | none |
| 43 | Qwopus3.8 Coder-Flash27B mostly Q5_K_M No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 64GB of VRAM?
On setups with more than 48 GB and up to 64 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 64GB count as 64GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 64GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.