Ollama benchmarks
A local model runner built on llama.cpp's GGML library, with a model registry and a one-line install.
45 community reports
Ollama results by GPU
Hardware with reported Ollama results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup.
Ollama on NVIDIA RTX 50809 reports
- Qwen3.8 27B · Q8_040.0 t/s(2 cards)
- Model not named · Q4_K_M34.9 t/s, 5,335 t/s prompt
- Qwen3-Coder 30B · 3B active · Q4_K_M51.7 t/s, 1,928 t/s prompt(part in system RAM)
- Qwen2.5 14B · Q4_K_M93.4 t/s, 4,842 t/s prompt
- Qwen2.5 7B · Q4_K_M171.4 t/s, 9,826 t/s prompt
Ollama on M4 32GB7 reports
- Gemma 4 31B · Q4_K_M5.3 t/s, 90 t/s prompt
- Gemma 4 26B · 4B active · Q4_K_M32.0 t/s, 657 t/s prompt
- Gemma 4 12B · Q4_K_M12.5 t/s, 236 t/s prompt
- Gemma 4 8B · Q4_K_M29.9 t/s, 566 t/s prompt
- Gemma 4 5.1B · Q4_K_M55.0 t/s, 1,089 t/s prompt
Ollama on NVIDIA RTX 3060 12GB6 reports
- Qwen3.8 27B · Q4_K_M17.9 t/s, 230 t/s prompt(2 cards)
- Gemma 4 31B · Q5_K_M2.3 t/s(speculative, part in system RAM)
- GLM-4.7 30B43.2 t/s
- Qwen2.5 7B · Q4_K_M49.0 t/s
- Gemma 4 5.1B · Q4_K_M60.0 t/s
Ollama on NVIDIA RTX 5070 Ti6 reports
- Gemma 4 31B · Q4_K_M9.3 t/s(part in system RAM)
- Gemma 4 26B · 4B active · Q4_K_M102.9 t/s(part in system RAM)
- Gemma 4 12B · Q4_K_M78.6 t/s
- Gemma 4 8B · Q4_K_M148.2 t/s
- Gemma 4 5.1B · Q4_K_M226.3 t/s
Ollama on NVIDIA RTX 50904 reports
- Qwen3.8 27B144.0 t/s(part in system RAM)
- Gemma 4 31B · Q4_K_M64.0 t/s
- Qwen3.8 27B15.0 t/s
- Qwen3.8 27B · Q4_K_M171.0 t/s
Ollama on NVIDIA RTX 30903 reports
- Qwen3.8 27B · IQ3_S83.0 t/s(speculative)
- Qwen3.8 27B · Q460.0 t/s(2 cards, speculative)
- Qwen3.6 35B · 3B active112.4 t/s(part in system RAM)
Ollama on M4 16GB2 reports
- Llama 3.2 3B · Q4_K_M46.7 t/s
- Llama 3.2 3B · Q4_K_M45.4 t/s
Ollama on M5 Pro 48GB2 reports
- Model not named112.0 t/s
- DeepSeek V4 Flash 9B · Q4_K_M44.0 t/s
Ollama on NVIDIA DGX Spark2 reports
- gpt-oss 117B · 5.1B active58.1 t/s(2 cards)
- Qwen3.5 397B · 17B active · UD-IQ3_XXS15.3 t/s(2 cards)
Ollama on NVIDIA RTX 40702 reports
- Model not named29.6 t/s(speculative, part in system RAM)
- Gemma 4 8B · Q4_K_M55.0 t/s
Ollama on NVIDIA RTX 40902 reports
- Gemma 4 26B · 4B active · QAT170.0 t/s
- Gemma 4 26B · 4B active · Q4_K_M149.6 t/s
Ollama on AMD MI50 32GB1 report
- Gemma 4 26B · 4B active · Q4_K_M95 t/s prompt(speculative)
Ollama on AMD Radeon 780M1 report
- Qwen3.8 27B3.2 t/s, 32 t/s prompt(2 cards)
Ollama on AMD RX 7800 XT 16GB1 report
- Llama 3.1 8B · Q4_K_M1.5 t/s
Ollama on AMD RX 7900 XTX1 report
- Qwen3.8 27B · Q4_K_M34.4 t/s, 216 t/s prompt
Ollama on M2 Max 64GB1 report
- Qwen3.6 35B · 3B active · NVFP471.3 t/s
Ollama on M3 Ultra 256GB1 report
- Qwen3.8 27B · Q4_K_M14.0 t/s, 93 t/s prompt
Ollama on M4 Max 128GB1 report
- Qwen3.5 35B · 3B active · Q4_K_M43.5 t/s
Ollama on M4 Max 36GB1 report
- Qwen3.8 27B · NVFP448.0 t/s, 198 t/s prompt
Ollama on NVIDIA RTX 2000 Ada1 report
- Qwen3.5 35B · 3B active · Q4_K_M58.9 t/s, 2,318 t/s prompt(2 cards)
Ollama on NVIDIA RTX 4070 Ti Super1 report
- Qwen3.8 27B · UD-Q3_K_XL36.5 t/s(part in system RAM)
Ollama on NVIDIA RTX 5060 Ti 16GB1 report
- Qwen3.6 27B · Q4_K_M46.0 t/s, 1,050 t/s prompt
Ollama on NVIDIA RTX A2000 12GB1 report
- Gemma 4 12B28.2 t/s
Ollama on NVIDIA RTX Pro 4000 Blackwell1 report
- Qwen3.8 27B33.9 t/s
Ollama, GPU not identified9 reports
- LFM3 2B43.7 t/s
- Qwen3.6 35B · 3B active155.0 t/s
- Qwen3.5 9B50.0 t/s
- Qwen3.6 35B · 3B active10.0 t/s(part in system RAM)
- Qwen3.5 27B · IQ1_S16.0 t/s(part in system RAM)
Ollama adds a model registry, a background server and an OpenAI-compatible API to a local runtime, so one command downloads a model and starts it.
It's built on GGML, the library under llama.cpp. Ollama picks most settings for you, which is quick to start with and leaves fewer knobs than llama.cpp's own flags. The matched comparisons below show how its reported speeds compare with other engines on the same card and model.
Compare GPUs running Ollama
Every card is compared on one model, Qwen2.5 7B at 4-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.
| GPU | VRAM | Reports | Median speed, Qwen2.5 7B 4-bit |
|---|---|---|---|
| NVIDIA RTX 5080nvidia | 16GB | 9 | Gen 176.6 t/sPP 9984.4 t/s · n=1 |
| M4 32GBapple | 32GB | 7 | no Qwen2.5 7B 4-bit run reported |
| NVIDIA RTX 3060 12GBnvidia | 12GB | 7 | Gen 49.0 t/s |
| NVIDIA RTX 5070 Tinvidia | 16GB | 7 | no Qwen2.5 7B 4-bit run reported |
| NVIDIA RTX 5090nvidia | 32GB | 5 | no Qwen2.5 7B 4-bit run reported |
| NVIDIA RTX 3090nvidia | 24GB | 4 | no Qwen2.5 7B 4-bit run reported |
| M4 16GBapple | 16GB | 3 | no Qwen2.5 7B 4-bit run reported |
| NVIDIA DGX Sparknvidia | 128GB | 3 | no Qwen2.5 7B 4-bit run reported |
Ollama against other engines
Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.
| Model and GPU | Ollama | Other engine |
|---|---|---|
| Qwen3.8 27B 4-bitAMD RX 7900 XTX | 34.4 t/sPrompt processing (prefill) 215.8 tokens/s (1)one generation report, 64K context | llama.cpp37.1 t/sPrompt processing (prefill) 877.8 tokens/s (1)median of 3 generation reports, 64K context, 2 not stated |
| Qwen3.6 35B · 3B active 4-bitM2 Max 64GB | 71.3 t/sone generation report, 32K context | LM Studio83.8 t/sone generation report, 256K context |
Top models on Ollama
Frequently asked
Is Ollama faster than other engines?
llamaperf has 2 matched pairs of Ollama and other-engine runs on the same card, model size and bit class, but none is comparable enough to say which engine is faster: the sides have too few reports or were run at different context lengths. A pair counts toward a direction only when each side has at least 3 generation reports with a stated context, and the two sides' contexts overlap. The measured figures and their conditions are in the table on this page.
Is Ollama the same as llama.cpp?
No, but they share a foundation. Ollama is built on GGML, the library under llama.cpp, and adds a model registry, a background server and an API on top.
What's the best GPU for Ollama?
The question is the same as for any engine: which card has room for the models you want, and how fast its memory is. The VRAM calculator shows what fits a given card, and the best GPU ranking orders cards by measured speed.
Does Ollama support AMD or Apple Silicon GPUs?
Yes. It uses ROCm on AMD cards, Metal on Apple Silicon, and CUDA on NVIDIA, with the CPU as a fallback.