Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: NVIDIA RTX 4060

Compare setup details (1 active)

QuantizationEngineRecorded GPU countReported context (tokens)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Gemma 4 26B

2× NVIDIA RTX 4060 · LM Studio

generation:: 75.0 tokens/s
prompt processing (prefill):: 1500.0 tokens/s
quant:: IQ (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B at 75 t/s generation and 1500 t/s prompt processing on 2x RTX 4060 8GB. Setup is LM Studio with an IQ quant, serving to Hermes. User credits mradermacher's IQ quants for getting the model running.

Showing 1–1 of 1

Page 1 of 1

Community benchmarks snapshot

Records by GPU

NVIDIA RTX 3090185NVIDIA RTX 5090131AMD Strix Halo 128GB120NVIDIA DGX Spark74NVIDIA RTX Pro 6000 Blackwell59NVIDIA RTX 5060 Ti 16GB57NVIDIA RTX 3060 12GB50AMD Radeon AI PRO R9700 32GB50NVIDIA RTX 409042NVIDIA RTX 5070 Ti34

Records by model

Qwen3.8924

Qwen3.6193

DeepSeek V4 Flash138

Gemma 473

Qwen3.546

GLM-5.334

other268

Records by engine

llama.cpp649

vLLM182

Strata66

NInfer44

Ollama40

other294

Use cases

coding 532agentic 347long-context 257tool-use 148vision 104summarization 53math 42creative-writing 36multilingual 21text-generation 9rp 8reasoning 3

coding532agentic347long-context257tool-use148vision104summarization53math42creative-writing36

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509082RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX48RTX 309036RTX 5090 Laptop 24GB34RTX 409032V100 32GB31Radeon AI PRO R9700 32GB30RX 7800 XT 16GB30

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX530RTX 3090561RTX 40901949V100 32GB941RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153Arc Pro B60 24GB380M1 Max 32GB82

Reports by model size

Qwen3.8 27B537Qwen3.8 125B · 6B active355DeepSeek V4 Flash 284B · 13B active129Qwen3.6 35B · 3B active120Qwen3.6 27B71GLM-5.3 320B · 18B active30Gemma 4 26B · 4B active30DeepSeek V4.1 Flash 552B · 16B active27

Quants

Q4_K_M146NVFP4116Q4_K_XL66IQ4_XS63UD-Q4_K_XL62IQ3_XXS44Q442Q8_039MXFP4334bit32