Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

Join the community

r/llamaperf on RedditNew reports as they land, and questions about what runs on what.llamaperf on DiscordChat live with people running local models, and ask what fits your card.

Follow along:X LinkedIn

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Buying hardware? Compare GPUs →

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

Model: DeepSeek V4 Pro

Compare setup details

QuantizationEngineRecorded GPU countReported context (tokens)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

DeepSeek V4 Pro 1600B (49B active)

NVIDIA RTX PRO 6000 Max-Q · llama.cpp · 1,048,576 ctx

User benchmarks DeepSeek V4 Pro GGUF at 794 GB on an Epyc 9374F with 12x96 GB DDR5 and an RTX PRO 6000 Max-Q. Setup is llama.cpp on a branch with expert offloading. Prompt processing ranges from 192 t/s at 8K context to 66 t/s at 1M context, and generation ranges from 11.73 t/s to 5.83 t/s. RAM usage is 69.3% of 1152 GB and VRAM usage is 78986 MiB of 96 GB, with power around 500 W during prompt processing. The user notes mainline llama.cpp issues with memory waste, a broken quantized KV cache, and bugs with prompt cache reuse.

DeepSeek V4 Pro 1600B (49B active)

NVIDIA RTX PRO 6000 Max-Q · ktransformers · 65,536 ctx

generation:: 7.1 tokens/s
prompt processing (prefill):: 46.2 tokens/s

User benchmarks at various context depths on an Epyc 9374F. GPU VRAM usage is 90815MiB/97887MiB and RAM usage is 907.5GB/1152GB. Power draw is about 100W prefill and 150W token generation for the GPU, and about 400W for the CPU and motherboard. The original model files are used with no conversion.

Showing 1–2 of 2

Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB313NVIDIA RTX 3090217NVIDIA RTX 5090161NVIDIA DGX Spark87NVIDIA RTX Pro 6000 Blackwell70NVIDIA RTX 5060 Ti 16GB67M5 Max 48GB67NVIDIA RTX 3060 12GB63AMD Radeon AI PRO R9700 32GB61NVIDIA RTX 409055

Records by model

Qwen3.81215

Qwen3.6274

Gemma 4165

DeepSeek V4 Flash148

Qwen3.5116

Qwen371

other595

Records by engine

llama.cpp983

vLLM201

Strata132

MLX106

Ollama79

other407

Use cases

coding 685agentic 434long-context 323tool-use 189vision 126summarization 61math 51creative-writing 43multilingual 26rp 12text-generation 9reasoning 3

coding685agentic434long-context323tool-use189vision126summarization61math51creative-writing43

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509073RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX43RTX 409040RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34M3 Ultra 512GB32V100 32GB31

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX640RTX 40901949RTX 3090561M3 Ultra 512GB391V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153

Reports by model size

Qwen3.8 27B682Qwen3.8 125B · 6B active472Qwen3.6 35B · 3B active184DeepSeek V4 Flash 284B · 13B active131Qwen3.6 27B79Gemma 4 26B · 4B active59GLM-5.3 320B · 18B active42DeepSeek V4.1 Flash 552B · 16B active42

Quants

Q4_K_M332UD-Q4_K_XL153NVFP4144Q8_01054-bit87IQ4_XS75Q4_K_XL74MXFP465IQ3_XXS61Q454