Best local LLMs for 128GB VRAM
How far 128GB of graphics memory goes for local models on cards like AMD Instinct MI250 or AMD Instinct MI250X 128GB: which models fit, and what people actually run on it.
Is 128GB of VRAM enough?
128GB holds a dense model of up to about 186B parameters at Q4_K_M with an 8K context, by the calculator's memory estimate. The largest model people report here that fits comfortably is Qwen3.8 177B, which needs about 101.9 GB at Q4_K_M with an 8K context. Anything bigger needs a smaller quant, part of the model in system RAM, or more memory, and a longer context needs more room on top.
The calculator checks your exact card, including longer contexts and offloading.
Models that fit in 128GB
Models people report on llamaperf, largest first, at the best quant whose memory with an 8K context stays within the calculator's comfortable band (85% of the card). Estimated from each model's shape.
| Model | Best quant | Needs |
|---|---|---|
| Qwen3 235B · 22B active | Q3_K_M | 106.7 GB |
| MiniMax M2.7 230B · 10B active | Q3_K_M | 104.5 GB |
| Minimax M2.1 230B · 10B active | Q3_K_M | 104.5 GB |
| StepFun 3.7 198B · 11B active | Q3_K_M | 88.9 GB |
| Step-3.5-Flash 196B · 11B active | Q3_K_M | 88 GB |
| Qwen3.8 177B | Q4_K_M | 101.9 GB |
| DeepSeek V4 Flash 162B · 13B active | Q4_K_M | 93.3 GB |
| DeepSeek V4 Flash 150B | Q5_K_M | 105.3 GB |
| Edge0 145B · 10B active | Q5_K_M | 103.6 GB |
| Edge0 145B | Q5_K_M | 103.6 GB |
| Qwen3.8 125B · 6B active | Q6_K | 103.5 GB |
| Ling-3.0 124B · 5.1B active | Q6_K | 102.6 GB |
And 136 smaller models. The calculator lists them all for your card.
What people run on 128GB
Model sizes reported on setups with more than 96 GB and up to 128 GB of memory, most reported first. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly UD-Q4_K_XL Gen 35 t/s (28 runs) · PP 1200 t/s (20 runs) · with speculation 43 t/s (35) | 121 | 35 28 runs, 5 devices | 1200 20 runs | 43 35 runs |
| 2 | Qwen3.827B Alibaba · mostly NVFP4 Gen 15 t/s (19 runs) · PP 251 t/s (16 runs) · with speculation 41 t/s (25) | 62 | 15 19 runs, 4 devices | 251 16 runs | 41 25 runs |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_M Gen 58 t/s (20 runs) · PP 1029 t/s (17 runs) · with speculation 122 t/s (8) | 38 | 58 20 runs, 4 devices | 1029 17 runs | 122 8 runs |
| 4 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly UD-IQ3_XXS Gen 35 t/s (3 runs) · PP 344 t/s (2 runs) · with speculation 29 t/s (5) | 24 | 35 3 runs, 2 devices | 344 2 runs | 29 5 runs |
| 5 | Qwen3.627B Alibaba · mostly Q8 Gen 13 t/s (3 runs) · PP 241 t/s (4 runs) · with speculation 21 t/s (3) | 13 | 13 3 runs, 2 devices | 241 4 runs | 21 3 runs |
| 6 | Qwen3-Coder30B · 3B active mostly UD-Q4_K_XL Gen 70 t/s (9 runs) · PP 406 t/s (8 runs) | 12 | 70 9 runs, 2 devices | 406 8 runs | none |
| 7 | Gemma 426B · 4B active Google DeepMind · mostly NVFP4 Gen 62 t/s (6 runs) · PP 2657 t/s (5 runs) · with speculation 102 t/s (1) | 9 | 62 6 runs, 3 devices | 2657 5 runs | 102 1 run |
| 8 | gpt-oss117B · 5.1B active OpenAI · mostly MXFP4 Gen 54 t/s (7 runs) · PP 645 t/s (7 runs) | 9 | 54 7 runs, 4 devices | 645 7 runs | none |
| 9 | DeepSeek V4.1 Flash552B · 16B active mostly Q2 Gen 25 t/s (1 run) · PP 1228 t/s (1 run) · with speculation 38 t/s (1) | 8 | 25 1 run | 1228 1 run | 38 1 run |
| 10 | Gemma 431B Google DeepMind · mostly QAT Q4_0 Gen 6.4 t/s (5 runs) · PP 545 t/s (5 runs) · with speculation 7.5 t/s (1) | 8 | 6.4 5 runs, 2 devices | 545 5 runs | 7.5 1 run |
| 11 | Qwen3.535B · 3B active Alibaba · mostly UD-Q4_K_XL Gen 60 t/s (3 runs) · PP 1017 t/s (1 run) | 6 | 60 3 runs, 2 devices | 1017 1 run | none |
| 12 | Qwen3.59B Alibaba · mostly Q5_K_XL Gen 48 t/s (3 runs) · PP 832 t/s (3 runs) | 6 | 48 3 runs, 2 devices | 832 3 runs | none |
| 13 | Qwen3.54B Alibaba · mostly Q8_0 Gen 90 t/s (2 runs) · PP 1522 t/s (2 runs) | 5 | 90 2 runs | 1522 2 runs | none |
| 14 | Qwen330B · 3B active Alibaba · mostly Q4_K_M Gen 86 t/s (3 runs) · PP 993 t/s (2 runs) | 5 | 86 3 runs, 2 devices | 993 2 runs | none |
| 15 | gpt-oss21B · 3.6B active OpenAI · mostly MXFP4 Gen 73 t/s (5 runs) · PP 1216 t/s (5 runs) | 5 | 73 5 runs, 2 devices | 1216 5 runs | none |
| 16 | Qwen3.5122B · 10B active Alibaba · mostly UD-Q4_K_XL Gen 19 t/s (3 runs) · PP 287 t/s (1 run) | 5 | 19 3 runs, 2 devices | 287 1 run | none |
| 17 | Llama 3.23B Meta · mostly BF16 Gen 77 t/s (4 runs) · PP 1725 t/s (4 runs) | 4 | 77 4 runs | 1725 4 runs | none |
| 18 | Ornith1.535B · 3B active mostly NVFP4 Gen 71 t/s (2 runs) · PP 1505 t/s (1 run) · with speculation 79 t/s (1) | 4 | 71 2 runs, 2 devices | 1505 1 run | 79 1 run |
| 19 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly 3.05bpw No generation data | 4 | no generation data | no PP data | none |
| 20 | Gemma 4 E4B8B Google DeepMind · mostly Q4_K_M Gen 70 t/s (1 run) · PP 1173 t/s (1 run) | 3 | 70 1 run | 1173 1 run | none |
| 21 | Ling-3.0 Flash124B · 5.1B active Ant Group · mostly Q5_K_M Gen 39 t/s (3 runs) | 3 | 39 3 runs | no PP data | none |
| 22 | Qwen3.8 Flash-Next Uncensored125B · 6B active Alibaba · mostly IQ4 No generation data · with speculation 72 t/s (1) | 3 | no generation data | no PP data | 72 1 run |
| 23 | SmolLM33B mostly Q4_K_M Gen 144 t/s (2 runs) · PP 2190 t/s (2 runs) | 2 | 144 2 runs | 2190 2 runs | none |
| 24 | Nex-N2.5-mini35B · 3B active mostly MLX-4bit Gen 105 t/s (2 runs) | 2 | 105 2 runs, 2 devices | no PP data | none |
| 25 | Nemotron 3 Super120B · 12B active mostly Q4_K Gen 17 t/s (2 runs) · PP 769 t/s (1 run) | 2 | 17 2 runs | 769 1 run | none |
| 26 | Qwen3.527B Alibaba · mostly UD-Q4_K_XL Gen 12 t/s (1 run) · PP 310 t/s (1 run) | 2 | 12 1 run | 310 1 run | none |
| 27 | Gemma 327B Google · mostly BF16 Gen 8.4 t/s (2 runs) · PP 243 t/s (2 runs) | 2 | 8.4 2 runs | 243 2 runs | none |
| 28 | Qwen332B Alibaba · mostly Q8_0 Gen 6.4 t/s (2 runs) · PP 68 t/s (2 runs) | 2 | 6.4 2 runs | 68 2 runs | none |
| 29 | Gemma 4 E2B5.1B Google DeepMind · mostly 4bit Gen 205 t/s (1 run) | 1 | 205 1 run | no PP data | none |
| 30 | Nemotron 3 Nano30B · 3B active mostly 4-bit Gen 160 t/s (1 run) | 1 | 160 1 run | no PP data | none |
| 31 | Falcon-H11.5B mostly Q4_K_M Gen 148 t/s (1 run) · PP 2535 t/s (1 run) | 1 | 148 1 run | 2535 1 run | none |
| 32 | Qwen3 30B-A3B, standard 8 experts30B · 3.3B active Alibaba · mostly MLX 4-bit affine, group 64 Gen 144 t/s (1 run) | 1 | 144 1 run | no PP data | none |
| 33 | Granite-4.0-H1B mostly Q4_K_M Gen 142 t/s (1 run) · PP 2656 t/s (1 run) | 1 | 142 1 run | 2656 1 run | none |
| 34 | Qwen34B Alibaba · mostly 4-bit Gen 117 t/s (1 run) · PP 942 t/s (1 run) | 1 | 117 1 run | 942 1 run | none |
| 35 | LFM2.52.6B Liquid AI · mostly Q4_K_M Gen 113 t/s (1 run) | 1 | 113 1 run | no PP data | none |
| 36 | Nemotron-Labs Dory 1.04B mostly 8-bit Gen 101 t/s (1 run) | 1 | 101 1 run | no PP data | none |
| 37 | Qwen3.50.8B Alibaba · mostly BF16 Gen 87 t/s (1 run) | 1 | 87 1 run | no PP data | none |
| 38 | Falcon-H13B mostly Q4_K_M Gen 86 t/s (1 run) · PP 1298 t/s (1 run) | 1 | 86 1 run | 1298 1 run | none |
| 39 | Qwen38B Alibaba · mostly 4-bit Gen 81 t/s (1 run) · PP 518 t/s (1 run) | 1 | 81 1 run | 518 1 run | none |
| 40 | LFM31.7B mostly Q8_0 Gen 78 t/s (1 run) · PP 4100 t/s (1 run) | 1 | 78 1 run | 4100 1 run | none |
| 41 | Ornith 1.035B · 3B active mostly Q4_K_M Gen 74 t/s (1 run) · PP 1103 t/s (1 run) | 1 | 74 1 run | 1103 1 run | none |
| 42 | Granite Switch 4.18B mostly 8-bit Gen 56 t/s (1 run) | 1 | 56 1 run | no PP data | none |
| 43 | Gemma 412B Google DeepMind · mostly Q4_0 Gen 55 t/s (1 run) · PP 589 t/s (1 run) | 1 | 55 1 run | 589 1 run | none |
| 44 | Falcon-H17B mostly Q4_K_M Gen 53 t/s (1 run) · PP 683 t/s (1 run) | 1 | 53 1 run | 683 1 run | none |
| 45 | Ornith1.5 Ornith-1.5-35B-A3B35B · 3B active mostly Q6_K Gen 50 t/s (1 run) · PP 563 t/s (1 run) | 1 | 50 1 run | 563 1 run | none |
| 46 | Qwen3-Coder-Next80B · 3B active mostly UD-Q6_K_XL Gen 37 t/s (1 run) · PP 546 t/s (1 run) | 1 | 37 1 run | 546 1 run | none |
| 47 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ4_XS Gen 36 t/s (1 run) · PP 1634 t/s (1 run) | 1 | 36 1 run | 1634 1 run | none |
| 48 | Bonsai 2 Ternary27B PrismML · mostly PQ2_0 Gen 34 t/s (1 run) | 1 | 34 1 run | no PP data | none |
| 49 | StepFun 3.7 Flash198B · 11B active StepFun · mostly Q4_K_S Gen 34 t/s (1 run) | 1 | 34 1 run | no PP data | none |
| 50 | Tencent-HY3295B · 21B active Tencent · mostly UD128 Gen 32 t/s (1 run) | 1 | 32 1 run | no PP data | none |
| 51 | Nemotron 3.5 Super VL120B · 12B active mostly Q4_K_M Gen 31 t/s (1 run) | 1 | 31 1 run | no PP data | none |
| 52 | Nanbeige4.333B mostly 4-bit Gen 29 t/s (1 run) · PP 757 t/s (1 run) | 1 | 29 1 run | 757 1 run | none |
| 53 | Mimo 2.6 Flash-RL309B · 15B active Xiaomi · mostly IQ2_M Gen 26 t/s (1 run) · PP 310 t/s (1 run) | 1 | 26 1 run | 310 1 run | none |
| 54 | Muse Glimmer30B Meta · mostly UD-Q2_K_XL Gen 25 t/s (1 run) | 1 | 25 1 run | no PP data | none |
| 55 | DeepSeek V4 Flash162B · 13B active DeepSeek · mostly NVFP4 Gen 14 t/s (1 run) · PP 539 t/s (1 run) | 1 | 14 1 run | 539 1 run | none |
| 56 | Llama 3.370B Meta · mostly Q4_K_XL Gen 5.1 t/s (1 run) · PP 78 t/s (1 run) | 1 | 5.1 1 run | 78 1 run | none |
| 57 | DeepSeek V3.1 Terminus671B mostly INT4 No generation data | 1 | no generation data | no PP data | none |
| 58 | DeepSeek V4 Flash Vision-Exp285B · 13B active DeepSeek · mostly IQ2_XXS/Q2_K No generation data · with speculation 32 t/s (1) | 1 | no generation data | no PP data | 32 1 run |
| 59 | DeepSeek V4.1 Flash763B mostly IQ1_M No generation data | 1 | no generation data | no PP data | none |
| 60 | GLM-5.3 Flash / Maya-S-v2321B · 18B active Zhipu AI · mostly Maya-S-v2-IQ2_XXS No generation data | 1 | no generation data | no PP data | none |
| 61 | Gemma 4 DiffusionGemma26B · 4B active Google DeepMind · mostly NVFP4 No generation data | 1 | no generation data | no PP data | none |
| 62 | Kimi K32800B · 104B active Moonshot AI · mostly 4-bit No generation data | 1 | no generation data | no PP data | none |
| 63 | Laguna S 2.1118B No generation data | 1 | no generation data | no PP data | none |
| 64 | Llama27B mostly Q4_0 No generation data | 1 | no generation data | no PP data | none |
| 65 | Mimo 2.6 Pro1020B · 42B active Xiaomi · mostly FP8 No generation data | 1 | no generation data | no PP data | none |
| 66 | Nemotron 3.5 Super VL120B · 12B active mostly NVFP4 No generation data · with speculation 28 t/s (1) | 1 | no generation data | no PP data | 28 1 run |
| 67 | Nemotron 3.5 Lightning30B · 3B active NVIDIA · mostly Q4_K_M No generation data | 1 | no generation data | no PP data | none |
| 68 | Qwen3.52B Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 69 | Qwen3.8 oQ4e-mtp restored-prefix B3 / PR1 61bd8a0627B Alibaba · mostly oQ4e No generation data · with speculation 8.4 t/s (1) | 1 | no generation data | no PP data | 8.4 1 run |
| 70 | Qwen3.8 Swift27B Alibaba · mostly 4bit No generation data · with speculation 88 t/s (1) | 1 | no generation data | no PP data | 88 1 run |
| 71 | Qwen3.8 Flash-Next Heretic122B · 10B active Alibaba · mostly Q8_0 No generation data | 1 | no generation data | no PP data | none |
| 72 | Qwen3.8 Flash-Next Uncensored (vmlinux Gufo Q4Mix)125B · 6B active Alibaba · mostly Q4 Mix (Q4_K/Q5_1/IQ4_NL/Q8_0) No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What do people run on 128GB of VRAM?
On setups with more than 96 GB and up to 128 GB of memory, the model people report most is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 35B · 3B active. The table lists each with the quant most people used and the typical speed.
Does a Mac with 128GB count as 128GB of VRAM?
No. macOS keeps part of a Mac's unified memory back from the GPU, so a Mac with 128GB can use less than that for a model. The calculator's Mac tab grades every Mac against its real limit.