Best local LLM for coding with 128GB VRAM
The models people report running for coding on setups with more than 96 GB and up to 128 GB of memory, such as AMD Instinct MI250 or AMD Instinct MI250X 128GB. Each row is one model size, with the quant most people used and its typical speed.
Ranked from reports whose posters used the model for coding.
Ranked by community reports
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly UD-Q4_K_XL Gen 48 t/s (12 runs) · PP 791 t/s (7 runs) · with speculation 39 t/s (12) | 43 | 48 12 runs, 4 devices | 791 7 runs | 39 12 runs |
| 2 | Qwen3.827B Alibaba · mostly NVFP4 Gen 12 t/s (2 runs) · PP 806 t/s (1 run) · with speculation 42 t/s (8) | 14 | 12 2 runs | 806 1 run | 42 8 runs |
| 3 | Qwen3.627B Alibaba · mostly NVFP4 Gen 15 t/s (2 runs) · PP 706 t/s (1 run) · with speculation 18 t/s (2) | 7 | 15 2 runs, 2 devices | 706 1 run | 18 2 runs |
| 4 | Qwen3.635B · 3B active Alibaba · mostly 4bit Gen 50 t/s (3 runs) · PP 323 t/s (1 run) · with speculation 105 t/s (1) | 5 | 50 3 runs, 2 devices | 323 1 run | 105 1 run |
| 5 | Qwen3-Coder30B · 3B active mostly Q4_K_M Gen 70 t/s (1 run) | 4 | 70 1 run | no PP data | none |
| 6 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly 2-bit expert planes No generation data · with speculation 17 t/s (2) | 4 | no generation data | no PP data | 17 2 runs |
| 7 | Ornith1.535B · 3B active mostly NVFP4 Gen 69 t/s (1 run) · with speculation 79 t/s (1) | 2 | 69 1 run | no PP data | 79 1 run |
| 8 | DeepSeek V4.1 Flash552B · 16B active mostly Q2 No generation data | 2 | no generation data | no PP data | none |
| 9 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly 3.05bpw No generation data | 2 | no generation data | no PP data | none |
| 10 | Nex-N2.5-mini35B · 3B active mostly MLX-4bit Gen 134 t/s (1 run) | 1 | 134 1 run | no PP data | none |
| 11 | Qwen3.535B · 3B active Alibaba · mostly Q8 Gen 50 t/s (1 run) | 1 | 50 1 run | no PP data | none |
| 12 | Qwen3-Coder-Next80B · 3B active mostly UD-Q6_K_XL Gen 37 t/s (1 run) · PP 546 t/s (1 run) | 1 | 37 1 run | 546 1 run | none |
| 13 | Qwen3.8 Swift-1.5-125B-A6B125B · 6B active Alibaba · mostly IQ4_XS Gen 36 t/s (1 run) · PP 1634 t/s (1 run) | 1 | 36 1 run | 1634 1 run | none |
| 14 | Nemotron 3.5 Super VL120B · 12B active mostly Q4_K_M Gen 31 t/s (1 run) | 1 | 31 1 run | no PP data | none |
| 15 | Nanbeige4.333B mostly 4-bit Gen 29 t/s (1 run) · PP 757 t/s (1 run) | 1 | 29 1 run | 757 1 run | none |
| 16 | DeepSeek V4 Flash Vision-Exp285B · 13B active DeepSeek · mostly IQ2_XXS/Q2_K No generation data · with speculation 32 t/s (1) | 1 | no generation data | no PP data | 32 1 run |
| 17 | DeepSeek V4.1 Flash763B mostly IQ1_M No generation data | 1 | no generation data | no PP data | none |
| 18 | Gemma 426B · 4B active Google DeepMind · mostly QAT Q4_0 No generation data | 1 | no generation data | no PP data | none |
Frequently asked
What is the best local LLM for coding with 128GB of VRAM?
On setups with more than 96 GB and up to 128 GB of memory, the model people report using most for coding is Qwen3.8 125B · 6B active, followed by Qwen3.8 27B and Qwen3.6 27B. The table shows the quant most of them used and their typical speed, so you can start from a setup that is known to work.
What counts as a coding report?
A community performance report whose poster described using the model for coding: an editor assistant, an agent, or code generation. The memory band is the poster's reported VRAM, or the card's VRAM times the number of cards.
Which quant should I use for coding on 128GB?
Start from the quant most people used for that model, shown under its name. It is one known to fit with room for a coding context. If you want a bigger model in the same memory, step down one quant rung, and the VRAM calculator shows the exact memory at each rung for your card.
How we rank
Model sizes are ordered by how many coding reports use them in this memory band, which measures what people run, not answer quality. Typical speed is the median of runs on one device serving one request without speculative decoding. Runs with MTP or a draft model are counted separately, because they can be several times faster. Reports come from r/LocalLLaMA, the web and direct submissions. Nothing is sponsored.