llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

Join the community

Follow along:XLinkedIn

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD Radeon Pro W6600
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

No AMD Radeon Pro W6600 reports match these filters yet.

Estimate what fits on the AMD Radeon Pro W6600 →Share your own numbers →

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB311NVIDIA RTX 3090215NVIDIA RTX 5090158NVIDIA DGX Spark86NVIDIA RTX Pro 6000 Blackwell70NVIDIA RTX 5060 Ti 16GB67M5 Max 48GB66NVIDIA RTX 3060 12GB63AMD Radeon AI PRO R9700 32GB60NVIDIA RTX 409053

Records by model

2550 total
Qwen3.81194
Qwen3.6268
Gemma 4164
DeepSeek V4 Flash148
Qwen3.5116
Qwen371
other589

Records by engine

1884 total
llama.cpp977
vLLM201
Strata126
MLX106
Ollama79
other395

Use cases

coding 672agentic 427long-context 315tool-use 185vision 124summarization 60math 50creative-writing 42multilingual 26rp 12text-generation 9reasoning 3
coding672agentic427long-context315tool-use185vision124summarization60math50creative-writing42

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509073RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX43RTX 409040RTX 309037RX 7800 XT 16GB35RTX 5090 Laptop 24GB34V100 32GB31Radeon AI PRO R9700 32GB30

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX640RTX 40901949RTX 3090561V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153Arc Pro B60 24GB380

Reports by model size

Qwen3.8 27B642Qwen3.8 125B · 6B active465Qwen3.6 35B · 3B active155DeepSeek V4 Flash 284B · 13B active131Qwen3.6 27B75Gemma 4 26B · 4B active42GLM-5.3 320B · 18B active40DeepSeek V4.1 Flash 552B · 16B active40

Quants

Q4_K_M330UD-Q4_K_XL150NVFP4141Q8_01044-bit86IQ4_XS75Q4_K_XL74MXFP464IQ3_XXS61Q454