Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

Join the community

r/llamaperf on RedditNew reports as they land, and questions about what runs on what.llamaperf on DiscordChat live with people running local models, and ask what fits your card.

Follow along:X LinkedIn

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Buying hardware? Compare GPUs →

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: M5 Max 128GB

Compare setup details (1 active)

QuantizationEngineRecorded GPU countReported context (tokens)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

DeepSeek V4 Flash 284B (13B active)

M5 Max 128GB · LM Studio · 1,048,576 ctx

generation:: 26.0 tokens/s
prompt processing (prefill):: 190.0 tokens/s
quant:: IQ2XXS (GGUF)
kv:: F16
flash attention:: on

User reports DeepSeek V4 Flash at ~26 tok/s generation and ~190 tok/s prompt processing on a MacBook Pro M5 Max with 128 GB unified memory. Setup is LM Studio with antirez's deepseek_v4 llama.cpp fork sideloaded, IQ2XXS GGUF weights, F16 KV cache, and the full 1048576 token context. The model is a 256-expert MoE with ~80 GB resident. The figures are approximate and the run is a workaround: upstream llama.cpp does not yet support the deepseek_v4 architecture and LM Studio's MLX runtime rejects V4 GGUFs. Long prompts beyond 16k tokens are flushed in 16k chunks due to the n_batch cap.

Showing 1–1 of 1

Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB318NVIDIA RTX 3090219NVIDIA RTX 5090170NVIDIA DGX Spark91NVIDIA RTX 5060 Ti 16GB72NVIDIA RTX Pro 6000 Blackwell70M5 Max 48GB67NVIDIA RTX 3060 12GB65AMD Radeon AI PRO R9700 32GB63NVIDIA RTX 409057

Records by model

Qwen3.81270

Qwen3.6282

Gemma 4168

DeepSeek V4 Flash149

Qwen3.5121

Qwen375

other627

Records by engine

llama.cpp1007

vLLM205

Strata149

MLX106

Ollama83

other426

Use cases

coding 733agentic 466long-context 341tool-use 207vision 134summarization 61math 57creative-writing 46multilingual 29rp 13text-generation 9reasoning 3

coding733agentic466long-context341tool-use207vision134summarization61math57creative-writing46

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX43RTX 409040RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34M3 Ultra 512GB32V100 32GB31

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX640RTX 40901949RTX 3090561M3 Ultra 512GB391V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153

Reports by model size

Qwen3.8 27B709Qwen3.8 125B · 6B active498Qwen3.6 35B · 3B active192DeepSeek V4 Flash 284B · 13B active132Qwen3.6 27B79Gemma 4 26B · 4B active59DeepSeek V4.1 Flash 552B · 16B active48GLM-5.3 320B · 18B active45

Quants

Q4_K_M341UD-Q4_K_XL156NVFP4148Q8_01074-bit91IQ4_XS76Q4_K_XL74MXFP466IQ3_XXS65IQ3_S59