Best local LLMs for 96GB VRAM
How far 96GB of graphics memory goes for local models on cards like NVIDIA RTX Pro 6000 Blackwell or NVIDIA RTX PRO 6000 Max-Q: which models fit, and what people actually run on it.
Is 96GB of VRAM enough?
96GB holds a dense model of up to about 137B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.8 125B · 6B active, which needs about 72.3 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 96GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| Qwen3.8 177B | Q3_K_M | 79.8 GB |
| DeepSeek V4 Flash 162B · 13B active | Q3_K_M | 73 GB |
| DeepSeek V4 Flash 150B | Q3_K_M | 67.8 GB |
| Edge0 145B · 10B active | Q3_K_M | 67.4 GB |
| Edge0 145B | Q3_K_M | 67.4 GB |
| Qwen3.8 125B · 6B active | Q4_K_M | 72.3 GB |
| Ling-3.0 124B · 5.1B active | Q4_K_M | 71.6 GB |
| Qwen3.5 122B · 10B active | Q4_K_M | 71.2 GB |
| Ornith1.5 122B | Q4_K_M | 71.2 GB |
| Ornith1.5 122B · 10B active | Q4_K_M | 71.2 GB |
| Qwen3.8 122B · 10B active | Q4_K_M | 71.2 GB |
| Nemotron 3 120B · 12B active | Q4_K_M | 69.3 GB |
And 131 smaller models. The calculator lists them all for your card.
What people run on 96GB
Model sizes reported on setups with more than 64 GB and up to 96 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly NVFP4 Gen 105 t/s (6 runs) · PP 5110 t/s (6 runs) · with speculation 90 t/s (11) | 52 | 105 6 runs | 5110 6 runs | 90 11 runs |
| 2 | Qwen3.827B Alibaba · mostly FP8 Gen 18 t/s (3 runs) · with speculation 100 t/s (11) | 30 | 18 3 runs, 3 devices | no PP data | 100 11 runs |
| 3 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly Q4_K_XL Gen 45 t/s (4 runs) · PP 312 t/s (2 runs) | 17 | 45 4 runs, 3 devices | 312 2 runs | none |
| 4 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly 3 Gen 70 t/s (1 run) | 9 | 70 1 run | no PP data | none |
| 5 | Qwen3.627B Alibaba · mostly 8bit Gen 7.4 t/s (1 run) · PP 121 t/s (1 run) · with speculation 28 t/s (3) | 7 | 7.4 1 run | 121 1 run | 28 3 runs |
| 6 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_M Gen 290 t/s (1 run) · PP 11486 t/s (1 run) | 6 | 290 1 run | 11486 1 run | none |
| 7 | DeepSeek V4.1 Flash552B · 16B active mostly VQ2 No generation data | 6 | no generation data | no PP data | none |
| 8 | Qwen3.535B · 3B active Alibaba · mostly Q8_0 Gen 92 t/s (2 runs) · PP 1808 t/s (3 runs) | 3 | 92 2 runs | 1808 3 runs | none |
| 9 | GLM-4.5-Air106B · 12B active mostly Q4_K_M Gen 138 t/s (1 run) | 2 | 138 1 run | no PP data | none |
| 10 | gpt-oss117B · 5.1B active OpenAI · mostly MXFP4 Gen 125 t/s (2 runs) · PP 347 t/s (1 run) | 2 | 125 2 runs, 2 devices | 347 1 run | none |
| 11 | Gemma 431B Google DeepMind · mostly NVFP4 Gen 51 t/s (1 run) · PP 15359 t/s (1 run) · with speculation 125 t/s (1) | 2 | 51 1 run | 15359 1 run | 125 1 run |
| 12 | Qwen38B Alibaba · mostly Q4 Gen 43 t/s (1 run) | 2 | 43 1 run | no PP data | none |
| 13 | Muse Glimmer30B Meta · mostly BF16 Gen 35 t/s (1 run) · with speculation 57 t/s (1) | 2 | 35 1 run | no PP data | 57 1 run |
| 14 | DeepSeek V4 Pro1600B · 49B active DeepSeek No generation data | 2 | no generation data | no PP data | none |
| 15 | Qwen3.8 Swift-1.527B Alibaba · mostly 4.7bpw No generation data | 2 | no generation data | no PP data | none |
| 16 | Qwen3.8 Qwen3.8-27B27B Alibaba · mostly Q4_K_M No generation data | 2 | no generation data | no PP data | none |
| 17 | Qwen3.84B Alibaba · mostly Q4_K_M Gen 291 t/s (1 run) | 1 | 291 1 run | no PP data | none |
| 18 | Qwen3.8 Flash-Next Abliterated125B · 6B active Alibaba · mostly Q5 Gen 110 t/s (1 run) | 1 | 110 1 run | no PP data | none |
| 19 | Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 59 t/s (1 run) · PP 920 t/s (1 run) | 1 | 59 1 run | 920 1 run | none |
| 20 | DeepSeek V4.1 Flash763B mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 21 | GLM-4.7 Flash30B Zhipu AI No generation data · with speculation 168 t/s (1) | 1 | no generation data | no PP data | 168 1 run |
| 22 | GLM-5.1744B · 40B active Zhipu AI · mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 23 | Gemma 34B Google · mostly BF16 No generation data · with speculation 133 t/s (1) | 1 | no generation data | no PP data | 133 1 run |
| 24 | Gemma 4 Gemma 4 31B31B Google DeepMind · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 25 | Kimi K3.1 Pro-Coder2800B mostly 2.81/2.56/1.44 bpw 3-tier MoE (~1.92 bpw avg) No generation data | 1 | no generation data | no PP data | none |
| 26 | Llama 3.21B Meta No generation data | 1 | no generation data | no PP data | none |
| 27 | Mimo 2.5310B · 15B active Xiaomi No generation data | 1 | no generation data | no PP data | none |
| 28 | Mimo 2.6 Flash-RL309B · 15B active Xiaomi · mostly 2.20 bpw No generation data | 1 | no generation data | no PP data | none |
| 29 | MiniMax M3.1 Flash428B · 23B active mostly 2-bit No generation data | 1 | no generation data | no PP data | none |
| 30 | Minimax M2.1230B · 10B active mostly UD-Q4_K_XL No generation data | 1 | no generation data | no PP data | none |
| 31 | Minimax M3 uncensored428B · 23B active MiniMax · mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 32 | Minimax M3428B · 23B active MiniMax · mostly MXFP4 No generation data | 1 | no generation data | no PP data | none |
| 33 | Muse Muse Glimmer 30B28B Meta · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 34 | Nemotron 3.5 Super VL120B · 12B active mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 35 | Qwen3-Next80B · 3B active Alibaba · mostly 4-bit No generation data | 1 | no generation data | no PP data | none |
| 36 | Qwen3.5 NuExtract34B Alibaba No generation data | 1 | no generation data | no PP data | none |
| 37 | Qwen3.6 abliterated27B Alibaba · mostly BF16 No generation data | 1 | no generation data | no PP data | none |
| 38 | Qwen3.6 Qwen3.6-35B-A3B35B · 3B active Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 39 | Qwen3.6 Ornith-Agents-A1-3.635B · 3B active Alibaba · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 40 | Qwen3.8 sabi-noema27B Alibaba · mostly 6bit No generation data | 1 | no generation data | no PP data | none |
| 41 | Qwen3.8 Swift-1.5 Flash-Next125B · 6B active Alibaba · mostly IQ3_S No generation data · with speculation 145 t/s (1) | 1 | no generation data | no PP data | 145 1 run |
| 42 | Step-3.5-Flash196B · 11B active mostly IQ4_XS No generation data | 1 | no generation data | no PP data | none |
| 43 | Tencent-HY3295B · 21B active Tencent · mostly Q6_K No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 96GB of VRAM?
On setups with more than 64 GB and up to 96 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and DeepSeek V4 Flash 284B · 13B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 96GB count as 96GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 96GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.