Best GPUs for 30B local LLMs
Models around 30B parameters are the most popular size for local use: good enough for real work and small enough for one consumer card. This page covers hardware with 16 GB or more and ranks it on the 30B-class model the most of it has been measured on.
What is the best GPU for 30B models?
Measured on Qwen3.8 27B at 4-bit, the fastest card people have reported is NVIDIA RTX 5090, at 80.0 tokens per second (median of 7 runs). NVIDIA RTX Pro 6000 Blackwell follows at 67.0 tokens per second (one run), then NVIDIA RTX 5070 Ti at 54.7 tokens per second (one run). The table lists every card with a run of this model.
- Up to 16 GBNVIDIA RTX 5070 TiGen 54.7 t/s
- 17-24 GBAMD RX 7900 XTXGen 42.5 t/s
- 25-32 GBNVIDIA RTX 5090Gen 80.0 t/s
- 33-96 GBNVIDIA RTX Pro 6000 BlackwellGen 67.0 t/s
- Over 96 GBM5 Ultra 256GBGen 50.0 t/s
Speed only matters if the model fits. Check what fits on your card.
Ranked on Qwen3.8 27B at 4-bit
Median tokens per second from plain runs: one GPU, one request, no speculative decoding, and the whole model in its memory. A run that kept part of the model in system RAM or on disk isn't ranked, because then the CPU and its memory set the pace as much as the GPU. The runs still differ in engine and context, and a long context can cut decode speed sharply, so each row shows what its median came from. Read a small gap between two rows as noise. Every Qwen3.8 report
| # | GPU | Memory | Gen t/s | PP t/s | Gen runs | Engine and context |
|---|---|---|---|---|---|---|
| 1 | NVIDIA RTX 5090 | 32 GB | 80.0 | - | 7 | llama.cpp, TensorSharp +1, 2 not stated40K to 256K context, 2 not stated |
| 2 | NVIDIA RTX Pro 6000 Blackwell | 96 GB | 67.0 | - | 1 | engine not statedcontext not stated |
| 3 | NVIDIA RTX 5070 Ti | 16 GB | 54.7 | - | 1 | llama.cpp110K context |
| 4 | M5 Ultra 256GB | 256 GBunified | 50.0 | 1800.01 run | 1 | oMLX8K context |
| 5 | AMD RX 7900 XTX | 24 GB | 42.5 | 640.04 runs | 6 | llama.cpp, Ollama, 2 not stated64K to 220K context, 3 not stated |
| 6 | NVIDIA RTX 4090 | 24 GB | 40.0 | 1949.32 runs | 3 | llama.cpp8K to 194K context |
| 7 | NVIDIA RTX 3090 | 24 GB | 37.4 | 561.02 runs | 6 | llama.cpp, vLLM, 1 not stated128K to 256K context, 1 not stated |
| 8 | AMD RX 7800 XT 16GB | 16 GB | 35.0 | - | 2 | llama.cpp, 1 not stated100K to 120K context |
| 9 | NVIDIA RTX 5090 Laptop 24GB | 24 GB | 33.5 | - | 2 | llama.cpp, 1 not stated64K to 100K context |
| 10 | M3 Ultra 512GB | 512 GBunified | 31.8 | 391.21 run | 1 | MLXcontext not stated |
| 11 | NVIDIA V100 32GB | 32 GB | 31.4 | 940.52 runs | 2 | llama.cpp, 1 not stated32K to 128K context |
| 12 | AMD Radeon AI PRO R9700 32GB | 32 GB | 29.8 | 1287.01 run | 3 | llama.cpp, 1 not stated2K context, 2 not stated |
| 13 | M5 Max 36GB | 36 GBunified | 25.8 | - | 1 | engine not statedcontext not stated |
| 14 | NVIDIA RTX 3080 20GB | 20 GB | 25.7 | 997.01 run | 2 | llama.cpp, 1 not stated96K context, 1 not stated |
| 15 | Intel Arc Pro B70 | 32 GB | 25.6 | 512.11 run | 1 | llama.cppcontext not stated |
| 16 | M1 Ultra 128GB | 128 GBunified | 25.1 | 153.31 run | 1 | engine not statedcontext not stated |
| 17 | M4 Max 64GB | 64 GBunified | 23.9 | 119.91 run | 1 | llama.cppcontext not stated |
| 18 | M4 Max 128GB | 128 GBunified | 21.8 | 236.61 run | 1 | llama.cppcontext not stated |
| 19 | Intel Arc Pro B60 24GB | 24 GB | 18.8 | 339.43 runs | 4 | llama.cpp, 1 not stated512 to 150K context, 1 not stated |
| 20 | M1 Max 64GB | 64 GBunified | 16.9 | - | 1 | LM Studio64K context |
| 21 | M1 Max 32GB | 32 GBunified | 15.8 | 81.81 run | 1 | MLXcontext not stated |
| 22 | NVIDIA L4 | 24 GB | 14.6 | - | 1 | llama.cppcontext not stated |
| 23 | M3 Ultra 256GB | 256 GBunified | 14.0 | 93.11 run | 1 | Ollamacontext not stated |
| 24 | AMD Strix Halo 128GB | 128 GBunified | 13.1 | 378.23 runs | 4 | llama.cpp, 1 not stated32K context, 3 not stated |
| 25 | M4 Pro 64GB | 64 GBunified | 12.9 | 126.24 runs | 4 | llama.cpp, 2 not statedcontext not stated |
| 26 | M2 Pro 32GB | 32 GBunified | 8.6 | 21.91 run | 1 | llama.cpp128K context |
| 27 | M4 24GB | 24 GBunified | 6.5 | 60.81 run | 1 | llama.cppcontext not stated |
| 28 | NVIDIA RTX 4070 Ti Super | 16 GB | 5.7 | - | 1 | llama.cpp4K context |
| 29 | NVIDIA DGX Spark | 128 GBunified | - | 899.91 run | 0 | llama.cpp16K context |
No speed reports yet
These match the profile, but nobody has reported a plain run on them.
What to look for
24 GB is the comfortable size
A 4-bit 30B-class model fits on a 24 GB card with room left for a useful context. That's why 24 GB cards are the usual recommendation for this size.
16 GB works with trade-offs
On a 16 GB card you'll use a smaller quant, a shorter context or partial offloading. Each costs something, and the calculator shows which quant fits your card and how fast it should run.
Frequently asked
Can a 16 GB GPU run 30B models?
Yes, with a smaller quant or a shorter context. The calculator lists what fits on your exact card and what it costs in quality.
How is this list ranked?
Every card is ranked on its median speed for one 30B-class reference model at one quant level, counting plain single-GPU runs only. The reference is the model between 24B and 40B parameters that the most of this hardware has reports of.