Best hardware for local LLMs
Each ranking compares hardware on one model that most of it has been measured on, by the median speed people report on it.
Best local LLMs by VRAM
How far your graphics memory goes: what fits, and what people run on it.
- Best local LLM for coding, by VRAM
The models people run for coding on 8GB, 12GB, 16GB, 24GB, 32GB and larger setups, ranked from community reports.
- Running a local LLM without a GPU
What people run on the CPU alone, and the speeds they report.
- Running a local LLM on two or more GPUs
What people run on multi-card setups, and the same card on one GPU beside it.
- Best GPUs for running local LLMs
Which GPU runs local LLMs fastest, ranked by the median tokens per second people report on the same model, with the fastest card at each memory size.
- Best Mac for running local LLMs
Which Apple Silicon Mac runs local LLMs fastest, ranked by the median tokens per second people report on the same model, with the fastest Mac at each memory size.
- Best NVIDIA GPUs for local LLMs
Which NVIDIA GPU runs local LLMs fastest, ranked by the median tokens per second people report on the same model, from GeForce to data center cards.
- Best GPUs for 70B and larger local LLMs
Which GPUs and Macs run large local LLMs fastest, ranked by the median tokens per second people report on the same large model.
- Best GPUs for 30B local LLMs
Which GPUs and Macs run 30B-class local LLMs fastest, ranked by the median tokens per second people report on the same model.
- Best Mac for 70B and larger local LLMs
Which Apple Silicon Macs run large local LLMs fastest, ranked by the median tokens per second people report on the same large model.