Local LLM: the best models to run on your hardware
The best local LLM is the biggest one that fits your memory, run at a quant you can live with. Pick your memory below and see which models people run on it, and the speed they get.
What is a local LLM?
A local LLM is a large language model that runs on your own computer instead of a company's server. Your prompts and files stay on your machine, and you pay for hardware once instead of for every request. What limits you is memory: the whole model has to fit in your graphics card's memory, or in a Mac's unified memory.
Speeds on this page come from reports people measured on their own hardware. Newest report: 2026-11-11.
Pick your memory
Graphics memory on a PC, or unified memory on a Mac. Each card shows the most-reported model that fits whole in that much memory, with the median speed of its plain runs.
How much memory you need
The largest dense model that fits with room to spare, by the calculator's estimate at Q4_K_M with an 8K context. A longer context needs more.
| Memory | Largest dense model | Models people report |
|---|---|---|
| 8GB | about 6B parameters | Models that fit in 8GB |
| 12GB | about 12B parameters | Models that fit in 12GB |
| 16GB | about 16B parameters | Models that fit in 16GB |
| 24GB | about 28B parameters | Models that fit in 24GB |
| 32GB | about 39B parameters | Models that fit in 32GB |
| 48GB | about 64B parameters | Models that fit in 48GB |
| 64GB | about 87B parameters | Models that fit in 64GB |
| 96GB | about 137B parameters | Models that fit in 96GB |
| 128GB | about 186B parameters | Models that fit in 128GB |
What people run most
Model sizes ranked by how many reports name them. Typical speed is the median of plain runs, and speculative runs are counted apart.
| # | Model | Reports | Gen t/s | PP t/s | With speculation |
|---|---|---|---|---|---|
| 1 | Qwen3.827B Alibaba · mostly Q4_K_M Gen 27 t/s (140 runs) · PP 433 t/s (65 runs) · with speculation 67 t/s (185) | 661 | 27 140 runs, 52 devices | 433 65 runs | 67 185 runs |
| 2 | Qwen3.8 Flash-Next125B · 6B active Alibaba · mostly UD-Q4_K_XL Gen 40 t/s (63 runs) · PP 1200 t/s (41 runs) · with speculation 47 t/s (62) | 574 | 40 63 runs, 13 devices | 1200 41 runs | 47 62 runs |
| 3 | Qwen3.635B · 3B active Alibaba · mostly Q4_K_M Gen 61 t/s (68 runs) · PP 1013 t/s (44 runs) · with speculation 121 t/s (21) | 205 | 61 68 runs, 23 devices | 1013 44 runs | 121 21 runs |
| 4 | DeepSeek V4 Flash284B · 13B active DeepSeek · mostly MXFP4 Gen 29 t/s (18 runs) · PP 312 t/s (10 runs) · with speculation 29 t/s (7) | 133 | 29 18 runs, 9 devices | 312 10 runs | 29 7 runs |
| 5 | Qwen3.627B Alibaba · mostly Q4_K_M Gen 26 t/s (20 runs) · PP 341 t/s (14 runs) · with speculation 50 t/s (19) | 77 | 26 20 runs, 13 devices | 341 14 runs | 50 19 runs |
| 6 | DeepSeek V4.1 Flash552B · 16B active mostly IQ2_XXS Gen 22 t/s (6 runs) · PP 1119 t/s (4 runs) · with speculation 31 t/s (3) | 76 | 22 6 runs, 4 devices | 1119 4 runs | 31 3 runs |
| 7 | Gemma 426B · 4B active Google DeepMind · mostly Q4_K_M Gen 79 t/s (33 runs) · PP 2595 t/s (25 runs) · with speculation 340 t/s (2) | 56 | 79 33 runs, 17 devices | 2595 25 runs | 340 2 runs |
| 8 | GLM-5.3 Flash320B · 18B active Zhipu AI · mostly NVFP4 Gen 37 t/s (5 runs) · PP 875 t/s (2 runs) · with speculation 70 t/s (4) | 51 | 37 5 runs, 3 devices | 875 2 runs | 70 4 runs |
| 9 | Qwen3.59B Alibaba · mostly Q4_K_M Gen 48 t/s (19 runs) · PP 826 t/s (8 runs) · with speculation 116 t/s (3) | 46 | 48 19 runs, 14 devices | 826 8 runs | 116 3 runs |
| 10 | Qwen3.8 Swift-1.527B Alibaba · mostly IQ3_S Gen 45 t/s (4 runs) · with speculation 83 t/s (13) | 36 | 45 4 runs, 4 devices | no PP data | 83 13 runs |
Find the right ranking
New to local LLMs?
Start with the guide to running an LLM locally. It covers what you need, which runtime to pick and how to check that a model fits before you download it.
Frequently asked
What is a local LLM?
A local LLM is a large language model that runs on your own computer instead of a company's server. Your prompts and files stay on your machine, and you pay for hardware once instead of for every request.
What is the best local LLM?
It depends on your memory. Across every setup, the model people report most is Qwen3.8 27B. That ranks how many reports name a model, so it shows what's popular and says nothing about quality. Pick your memory size above to see what people report on a card like yours.
How much memory do I need to run a local LLM?
Memory decides which models load. By the calculator's estimate, a dense model of up to about 6B parameters fits in 8GB with room to spare, 16B in 16GB and 28B in 24GB, at Q4_K_M with an 8K context. A longer context needs more room. The VRAM requirements guide has the full tables.
Can I run a local LLM without a GPU?
Yes. A model can run on the CPU and system memory, and it generates more slowly than on a GPU. The leaderboard has a CPU-only tier that shows what people get.
How do I get started?
Install a runtime such as Ollama or LM Studio, pick a model that fits your memory, and run it. The guide to running an LLM locally walks through the choices.