llamaperf

Best local LLMs for 128GB VRAM

How far 128GB of graphics memory goes for local models on cards like AMD Instinct MI250 or AMD Instinct MI250X 128GB: which models fit, and what people actually run on it.

Is 128GB of VRAM enough?

128GB holds a dense model of up to about 186B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.8 177B, which needs about 101.9 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.

The calculator checks your exact card, including longer contexts and offloading.

Models that fit in 128GB

Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.

And 136 smaller models. The calculator lists them all for your card.

What people run on 128GB

Model sizes reported on setups with more than 96 GB and up to 128 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.

#ModelReports
1Qwen3.8 Flash-Next125B · 6B active
Alibaba · mostly UD-Q4_K_XL
Gen 35 t/s (28 runs) · PP 1200 t/s (20 runs) · with speculation 43 t/s (35)
121
2Qwen3.827B
Alibaba · mostly NVFP4
Gen 15 t/s (19 runs) · PP 251 t/s (16 runs) · with speculation 41 t/s (25)
62
3Qwen3.635B · 3B active
Alibaba · mostly Q4_K_M
Gen 58 t/s (20 runs) · PP 1029 t/s (17 runs) · with speculation 122 t/s (8)
38
4DeepSeek V4 Flash284B · 13B active
DeepSeek · mostly UD-IQ3_XXS
Gen 35 t/s (3 runs) · PP 344 t/s (2 runs) · with speculation 29 t/s (5)
24
5Qwen3.627B
Alibaba · mostly Q8
Gen 13 t/s (3 runs) · PP 241 t/s (4 runs) · with speculation 21 t/s (3)
13
6Qwen3-Coder30B · 3B active
mostly UD-Q4_K_XL
Gen 70 t/s (9 runs) · PP 406 t/s (8 runs)
12
7Gemma 426B · 4B active
Google DeepMind · mostly NVFP4
Gen 62 t/s (6 runs) · PP 2657 t/s (5 runs) · with speculation 102 t/s (1)
9
8gpt-oss117B · 5.1B active
OpenAI · mostly MXFP4
Gen 54 t/s (7 runs) · PP 645 t/s (7 runs)
9
9DeepSeek V4.1 Flash552B · 16B active
mostly Q2
Gen 25 t/s (1 run) · PP 1228 t/s (1 run) · with speculation 38 t/s (1)
8
10Gemma 431B
Google DeepMind · mostly QAT Q4_0
Gen 6.4 t/s (5 runs) · PP 545 t/s (5 runs) · with speculation 7.5 t/s (1)
8
11Qwen3.535B · 3B active
Alibaba · mostly UD-Q4_K_XL
Gen 60 t/s (3 runs) · PP 1017 t/s (1 run)
6
12Qwen3.59B
Alibaba · mostly Q5_K_XL
Gen 48 t/s (3 runs) · PP 832 t/s (3 runs)
6
13Qwen3.54B
Alibaba · mostly Q8_0
Gen 90 t/s (2 runs) · PP 1522 t/s (2 runs)
5
14Qwen330B · 3B active
Alibaba · mostly Q4_K_M
Gen 86 t/s (3 runs) · PP 993 t/s (2 runs)
5
15gpt-oss21B · 3.6B active
OpenAI · mostly MXFP4
Gen 73 t/s (5 runs) · PP 1216 t/s (5 runs)
5
16Qwen3.5122B · 10B active
Alibaba · mostly UD-Q4_K_XL
Gen 19 t/s (3 runs) · PP 287 t/s (1 run)
5
17Llama 3.23B
Meta · mostly BF16
Gen 77 t/s (4 runs) · PP 1725 t/s (4 runs)
4
18Ornith1.535B · 3B active
mostly NVFP4
Gen 71 t/s (2 runs) · PP 1505 t/s (1 run) · with speculation 79 t/s (1)
4
19GLM-5.3 Flash320B · 18B active
Zhipu AI · mostly 3.05bpw
No generation data
4
20Gemma 4 E4B8B
Google DeepMind · mostly Q4_K_M
Gen 70 t/s (1 run) · PP 1173 t/s (1 run)
3
21Ling-3.0 Flash124B · 5.1B active
Ant Group · mostly Q5_K_M
Gen 39 t/s (3 runs)
3
22Qwen3.8 Flash-Next Uncensored125B · 6B active
Alibaba · mostly IQ4
No generation data · with speculation 72 t/s (1)
3
23SmolLM33B
mostly Q4_K_M
Gen 144 t/s (2 runs) · PP 2190 t/s (2 runs)
2
24Nex-N2.5-mini35B · 3B active
mostly MLX-4bit
Gen 105 t/s (2 runs)
2
25Nemotron 3 Super120B · 12B active
mostly Q4_K
Gen 17 t/s (2 runs) · PP 769 t/s (1 run)
2
26Qwen3.527B
Alibaba · mostly UD-Q4_K_XL
Gen 12 t/s (1 run) · PP 310 t/s (1 run)
2
27Gemma 327B
Google · mostly BF16
Gen 8.4 t/s (2 runs) · PP 243 t/s (2 runs)
2
28Qwen332B
Alibaba · mostly Q8_0
Gen 6.4 t/s (2 runs) · PP 68 t/s (2 runs)
2
29Gemma 4 E2B5.1B
Google DeepMind · mostly 4bit
Gen 205 t/s (1 run)
1
30Nemotron 3 Nano30B · 3B active
mostly 4-bit
Gen 160 t/s (1 run)
1
31Falcon-H11.5B
mostly Q4_K_M
Gen 148 t/s (1 run) · PP 2535 t/s (1 run)
1
32Qwen3 30B-A3B, standard 8 experts30B · 3.3B active
Alibaba · mostly MLX 4-bit affine, group 64
Gen 144 t/s (1 run)
1
33Granite-4.0-H1B
mostly Q4_K_M
Gen 142 t/s (1 run) · PP 2656 t/s (1 run)
1
34Qwen34B
Alibaba · mostly 4-bit
Gen 117 t/s (1 run) · PP 942 t/s (1 run)
1
35LFM2.52.6B
Liquid AI · mostly Q4_K_M
Gen 113 t/s (1 run)
1
36Nemotron-Labs Dory 1.04B
mostly 8-bit
Gen 101 t/s (1 run)
1
37Qwen3.50.8B
Alibaba · mostly BF16
Gen 87 t/s (1 run)
1
38Falcon-H13B
mostly Q4_K_M
Gen 86 t/s (1 run) · PP 1298 t/s (1 run)
1
39Qwen38B
Alibaba · mostly 4-bit
Gen 81 t/s (1 run) · PP 518 t/s (1 run)
1
40LFM31.7B
mostly Q8_0
Gen 78 t/s (1 run) · PP 4100 t/s (1 run)
1
41Ornith 1.035B · 3B active
mostly Q4_K_M
Gen 74 t/s (1 run) · PP 1103 t/s (1 run)
1
42Granite Switch 4.18B
mostly 8-bit
Gen 56 t/s (1 run)
1
43Gemma 412B
Google DeepMind · mostly Q4_0
Gen 55 t/s (1 run) · PP 589 t/s (1 run)
1
44Falcon-H17B
mostly Q4_K_M
Gen 53 t/s (1 run) · PP 683 t/s (1 run)
1
45Ornith1.5 Ornith-1.5-35B-A3B35B · 3B active
mostly Q6_K
Gen 50 t/s (1 run) · PP 563 t/s (1 run)
1
46Qwen3-Coder-Next80B · 3B active
mostly UD-Q6_K_XL
Gen 37 t/s (1 run) · PP 546 t/s (1 run)
1
47Qwen3.8 Swift-1.5-125B-A6B125B · 6B active
Alibaba · mostly IQ4_XS
Gen 36 t/s (1 run) · PP 1634 t/s (1 run)
1
48Bonsai 2 Ternary27B
PrismML · mostly PQ2_0
Gen 34 t/s (1 run)
1
49StepFun 3.7 Flash198B · 11B active
StepFun · mostly Q4_K_S
Gen 34 t/s (1 run)
1
50Tencent-HY3295B · 21B active
Tencent · mostly UD128
Gen 32 t/s (1 run)
1
51Nemotron 3.5 Super VL120B · 12B active
mostly Q4_K_M
Gen 31 t/s (1 run)
1
52Nanbeige4.333B
mostly 4-bit
Gen 29 t/s (1 run) · PP 757 t/s (1 run)
1
53Mimo 2.6 Flash-RL309B · 15B active
Xiaomi · mostly IQ2_M
Gen 26 t/s (1 run) · PP 310 t/s (1 run)
1
54Muse Glimmer30B
Meta · mostly UD-Q2_K_XL
Gen 25 t/s (1 run)
1
55DeepSeek V4 Flash162B · 13B active
DeepSeek · mostly NVFP4
Gen 14 t/s (1 run) · PP 539 t/s (1 run)
1
56Llama 3.370B
Meta · mostly Q4_K_XL
Gen 5.1 t/s (1 run) · PP 78 t/s (1 run)
1
57DeepSeek V3.1 Terminus671B
mostly INT4
No generation data
1
58DeepSeek V4 Flash Vision-Exp285B · 13B active
DeepSeek · mostly IQ2_XXS/Q2_K
No generation data · with speculation 32 t/s (1)
1
59DeepSeek V4.1 Flash763B
mostly IQ1_M
No generation data
1
60GLM-5.3 Flash / Maya-S-v2321B · 18B active
Zhipu AI · mostly Maya-S-v2-IQ2_XXS
No generation data
1
61Gemma 4 DiffusionGemma26B · 4B active
Google DeepMind · mostly NVFP4
No generation data
1
62Kimi K32800B · 104B active
Moonshot AI · mostly 4-bit
No generation data
1
63Laguna S 2.1118B
No generation data
1
64Llama27B
mostly Q4_0
No generation data
1
65Mimo 2.6 Pro1020B · 42B active
Xiaomi · mostly FP8
No generation data
1
66Nemotron 3.5 Super VL120B · 12B active
mostly NVFP4
No generation data · with speculation 28 t/s (1)
1
67Nemotron 3.5 Lightning30B · 3B active
NVIDIA · mostly Q4_K_M
No generation data
1
68Qwen3.52B
Alibaba · mostly Q8_0
No generation data
1
69Qwen3.8 oQ4e-mtp restored-prefix B3 / PR1 61bd8a0627B
Alibaba · mostly oQ4e
No generation data · with speculation 8.4 t/s (1)
1
70Qwen3.8 Swift27B
Alibaba · mostly 4bit
No generation data · with speculation 88 t/s (1)
1
71Qwen3.8 Flash-Next Heretic122B · 10B active
Alibaba · mostly Q8_0
No generation data
1
72Qwen3.8 Flash-Next Uncensored (vmlinux Gufo Q4Mix)125B · 6B active
Alibaba · mostly Q4 Mix (Q4_K/Q5_1/IQ4_NL/Q8_0)
No generation data
1

Frequently asked

What do people run on 128GB of VRAM?

On setups with more than 96 GB and up to 128 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.

Does a Mac with 128GB count as 128GB of VRAM?

No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 128GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.