llamaperf
Submit

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

Join the community

r/llamaperf on RedditNew reports as they land, and questions about what runs on what.llamaperf on DiscordChat live with people running local models, and ask what fits your card.

Follow along:XLinkedIn

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Buying hardware? Compare GPUs →

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: M5 Pro 24GB
Compare setup details (1 active)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: positiveApple hardware
generation:
125.8 tokens/s
prompt processing (prefill):
363.9 tokens/s
quant:
4bit (MLX)
codingagentic

User benchmarks DeepSeek-Coder-V2-Lite-Instruct at 125.8 tok/s generation and 363.9 tok/s prompt processing on an Apple M5 Pro with 24GB unified memory. Setup is MLX with a 4bit quant (8.2GB) on a single machine, running a fixed coding task capped at 1500 output tokens. The model produced correct working code; the user notes that 24GB has a hard ceiling around 20GB models and that background downloads measurably affect tok/s.

Oct 3, 2026
Showing 1–1 of 1
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB323NVIDIA RTX 3090221NVIDIA RTX 5090171NVIDIA DGX Spark92NVIDIA RTX 5060 Ti 16GB72NVIDIA RTX Pro 6000 Blackwell71M5 Max 48GB67NVIDIA RTX 3060 12GB66AMD Radeon AI PRO R9700 32GB63NVIDIA RTX 409057

Records by model

2730 total
Qwen3.81293
Qwen3.6285
Gemma 4168
DeepSeek V4 Flash149
Qwen3.5123
Qwen375
other637

Records by engine

1999 total
llama.cpp1015
vLLM209
Strata152
MLX107
Ollama84
other432

Use cases

coding 745agentic 474long-context 346tool-use 210vision 144summarization 61math 58creative-writing 47multilingual 30rp 13text-generation 9reasoning 3
coding745agentic474long-context346tool-use210vision144summarization61math58creative-writing47

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RX 7900 XTX43RTX 409040RTX 309037RX 7800 XT 16GB35RTX 5090 Laptop 24GB34M3 Ultra 512GB32

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX640RTX 40901949RTX 3090561M3 Ultra 512GB391V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153

Reports by model size

Qwen3.8 27B717Qwen3.8 125B · 6B active512Qwen3.6 35B · 3B active194DeepSeek V4 Flash 284B · 13B active132Qwen3.6 27B80Gemma 4 26B · 4B active59DeepSeek V4.1 Flash 552B · 16B active51GLM-5.3 320B · 18B active45

Quants

Q4_K_M344UD-Q4_K_XL159NVFP4149Q8_01074-bit94IQ4_XS76Q4_K_XL75MXFP466IQ3_XXS66IQ3_S60