Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

Join the community

r/llamaperf on RedditNew reports as they land, and questions about what runs on what.llamaperf on DiscordChat live with people running local models, and ask what fits your card.

Follow along:X LinkedIn

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Buying hardware? Compare GPUs →

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

Model: DeepSeek V3.1

Compare setup details

QuantizationEngineRecorded GPU countReported context (tokens)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

DeepSeek V3.1 671B Terminus

M5 Max 128GB · llama.cpp

generation:: 3.6 tokens/s
quant:: INT4 (GGUF)

User reports running DeepSeek v3.1 671B Terminus at 3.6 tokens/sec on a MacBook Pro with M5 Max 128GB and 8TB storage. Setup uses a heavily modified slipstream llama.cpp base that caches MoE experts to disk and calls them in as they change, with a -0.1 bias in the MTP router for in-RAM experts. A custom GGUF-derived format uses BF16 for fixed layers and INT4 for expert layers. The user says it is working extremely well and generating a somewhat usable 3.6 tokens/sec.

Showing 1–1 of 1

Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB303NVIDIA RTX 3090196NVIDIA RTX 5090141NVIDIA DGX Spark81M5 Max 48GB66NVIDIA RTX Pro 6000 Blackwell64NVIDIA RTX 5060 Ti 16GB61NVIDIA RTX 3060 12GB58AMD Radeon AI PRO R9700 32GB56M4 Max 128GB49

Records by model

Qwen3.81086

Qwen3.6245

Gemma 4154

DeepSeek V4 Flash148

Qwen3.5103

Qwen370

other532

Records by engine

llama.cpp913

vLLM189

MLX104

Strata96

Ollama68

other370

Use cases

coding 598agentic 379long-context 291tool-use 164vision 114summarization 55math 46creative-writing 39multilingual 25text-generation 9rp 9reasoning 3

coding598agentic379long-context291tool-use164vision114summarization55math46creative-writing39

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509082RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX48RTX 309036RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RTX 409032V100 32GB31Radeon AI PRO R9700 32GB30

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX530RTX 3090561RTX 40901949V100 32GB941RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153Arc Pro B60 24GB380M1 Max 32GB82

Reports by model size

Qwen3.8 27B587Qwen3.8 125B · 6B active414Qwen3.6 35B · 3B active136DeepSeek V4 Flash 284B · 13B active131Qwen3.6 27B72Gemma 4 26B · 4B active37GLM-5.3 320B · 18B active36DeepSeek V4.1 Flash 552B · 16B active30

Quants

Q4_K_M305UD-Q4_K_XL146NVFP4127Q8_0994-bit83IQ4_XS71Q4_K_XL71MXFP462IQ3_XXS58Q452