Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

Model: Falcon3

Compare setup details

QuantizationEngineRecorded GPU countReported context (tokens)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Falcon3 1B

Unknown GPU · BitRun

generation:: 98-127 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Falcon3 1B Instruct at 98-127 tokens/sec on an Apple M4 Pro CPU (12 cores), running pure CPU SIMD with 0% GPU usage. Setup is the BitRun runner (patched bitnet.cpp) with a ~1.30 GB GGUF model using ~1.9 GB RAM. The same model runs at 39-45 tokens/sec on an Intel Core i5-12600K (AVX2) with 209.6 t/s prompt processing, and 40-45 tokens/sec on WSL2 with 4 cores. BitNet b1.58 2.4B reaches ~40 tokens/sec and BitNet 70M 1,350+ tokens/sec on the M4 Pro.

Showing 1–1 of 1

Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB299NVIDIA RTX 3090193NVIDIA RTX 5090138NVIDIA DGX Spark80M5 Max 48GB66NVIDIA RTX Pro 6000 Blackwell63NVIDIA RTX 5060 Ti 16GB60NVIDIA RTX 3060 12GB55AMD Radeon AI PRO R9700 32GB53M4 Max 128GB49

Records by model

Qwen3.81060

Qwen3.6240

Gemma 4154

DeepSeek V4 Flash147

Qwen3.5101

Qwen368

other522

Records by engine

llama.cpp903

vLLM189

MLX102

Strata85

Ollama66

other360

Use cases

coding 578agentic 370long-context 281tool-use 157vision 113summarization 54math 43creative-writing 39multilingual 25text-generation 9rp 8reasoning 3

coding578agentic370long-context281tool-use157vision113summarization54math43creative-writing39

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509082RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX48RTX 309036RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RTX 409032V100 32GB31Radeon AI PRO R9700 32GB30

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX530RTX 3090561RTX 40901949V100 32GB941RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153Arc Pro B60 24GB380M1 Max 32GB82

Reports by model size

Qwen3.8 27B581Qwen3.8 125B · 6B active395Qwen3.6 35B · 3B active131DeepSeek V4 Flash 284B · 13B active130Qwen3.6 27B72Gemma 4 26B · 4B active37GLM-5.3 320B · 18B active35DeepSeek V4.1 Flash 552B · 16B active29

Quants

Q4_K_M302UD-Q4_K_XL144NVFP4123Q8_0994-bit83IQ4_XS71Q4_K_XL71MXFP461IQ3_XXS57Q451