How to run an LLM locally on Windows
On a Windows PC the number to check is your graphics card's own memory, called VRAM. It decides which models run fast and which ones crawl.
1. Find your VRAM
Open Task Manager, go to the Performance tab and pick your GPU. The figure labelled dedicated GPU memory is your VRAM. At Q4_K_M with a 8K context, a dense model of up to about 6B parameters fits in 8GB with room to spare, 12B in 12GB, 16B in 16GB and 28B in 24GB. The VRAM requirements guide has the full tables, and the LLM VRAM calculator checks your card against a specific model.
2. Pick a runtime
LM Studio gives you a window with a model browser and a chat box. Ollama runs from a terminal. Both download models for you, and each runtime page shows the speeds people report on it.
3. Choose a model that fits
Pick the biggest model that fits in your VRAM with room left for the context. A model that does not fit can still run by spilling into system RAM, and it generates text more slowly. The local LLM picker shows what people run at each memory size, and the best GPU for local LLMs page ranks cards by the speed people report. For code, see the best local LLM for coding.
4. Share your result
Once a model runs, your speed helps the next person choose a card. Benchmark your own machine and it joins the reports on this site. For the basics that apply to every computer, see how to run an LLM locally.
Frequently asked
How much VRAM do I need to run an LLM on Windows?
By the calculator's estimate at Q4_K_M with a 8K context, a dense model of up to about 6B parameters fits in 8GB with room to spare, 12B in 12GB, 16B in 16GB and 28B in 24GB. A longer context needs more.
Can I run an LLM on Windows without a GPU?
Yes. The model runs on the CPU and system memory, and it generates text more slowly. The leaderboard has a CPU-only tier that shows what people get.
What happens if the model is bigger than my VRAM?
Most runtimes can place part of the model in system RAM, and that part runs slower. Reports on this site mark runs that spill to system memory, so you can compare them with runs that fit whole.
Which runtime should I use on Windows?
LM Studio gives you a window and a model browser, and Ollama runs from a terminal. Each has its own page with the speeds people report on it.