llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: M6 32GB
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Qwen3.8 27B

M6 32GB · ANEMLL Forge · 96,000 ctx

Tone: positiveApple hardware
generation:
27.0 tokens/s
prompt processing (prefill):
123.0 tokens/s
quant:
INT8
kv:
INT8

User reports Qwen3.8-27B running on the M6 Neural Engine via ANEMLL Forge 0.2.1, with about 27 tok/s decode and 123 tok/s prefill at 96K context on a 32GB M6 Mac Mini. Setup uses an INT8 K/V cache and a smaller drafter sharing the target's output head, with about 21.4 GB total memory at 96K context. Decode varies by content: JSON 35–62 tok/s, code 18–32, prose 10–17. Prefill is 260 tok/s at short context and 62 at 192K, where a smaller prefill batch of 32 rows is used. The user calls 96K the sweet spot for size and speed on 32GB.

Oct 24, 2026
Tone: positiveApple hardware
generation:
10.0 tokens/s
prompt processing (prefill):
200.0 tokens/s
quant:
Q2_0 (GGUF)

User reports Qwen3.8-Flash-Next at about 10 tok/s generation and about 200 tok/s prefill on a Mac Mini M6 with 32 GB. Setup is Strata with a Q2_0 quant at 64K context. User says quality seems on par with 4-bit Qwen3.8-27B and that the fans almost never turn on with Strata, unlike the 27B run with Splash.

Oct 23, 2026

Gemma 3 27B

M6 32GB · MLX-LM 0.31.3 / MLX 0.32.2 / Metal

Apple hardware
generation:
8.9 tokens/s
prompt processing (prefill):
11.2 tokens/s
quant:
4-bit (MLX)

User reports Gemma 3 27B at 8.9 t/s generation and 11.2 t/s prompt processing on a Mac mini M6 with 32GB unified memory. Setup is MLX-LM with the mlx-community 4-bit quantized model, run through the mlx_lm.generate CLI with a 20-token prompt and 500 output tokens. Peak memory was 16.296 GB. The rates are native MLX-LM phase timings from a single displayed run, not a benchmark average.

Oct 13, 2026

Qwen3.5 9B

M6 32GB · LM Studio · 165,120 ctx

Apple hardware
generation:
30.3 tokens/s

User reports Qwen3.5 9B at 30.3 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with the MLX backend and a loaded context of 165,120 tokens, one machine with one model loaded alone. The 30.3 t/s is the median of three English-prose runs, a client-observed streaming decode rate rather than a pooled or peak figure; exact quantization is unknown.

Oct 13, 2026

Gemma 4 31B

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
7.5 tokens/s

User reports Gemma 4 31B at 7.5 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with a 32,768-token loaded context; quantization and backend are unrecorded. The 7.5 t/s is the median of three English-prose runs, measured as client-observed streaming decode throughput rather than a pooled or peak rate.

Oct 13, 2026

Gemma 4 12B

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
18.8 tokens/s

User reports Gemma 4 12B at 18.8 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with a QAT-labeled model at 32,768 tokens of loaded context; exact quantization and backend are unknown. The 18.8 t/s is the median of three English-prose runs, client-observed streaming decode throughput, not a pooled rate or prompt-processing rate.

Oct 13, 2026

Gemma 4 26B (4B active)

M6 32GB · LM Studio · 80,128 ctx

Apple hardware
generation:
48.2 tokens/s

User reports Gemma 4 26B-A4B at 48.2 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with an unreported backend and unknown quantization, loaded context 80,128 tokens, one model loaded alone. The 48.2 t/s is the median of three English-prose runs, client-observed streaming decode throughput, not a pooled rate or peak; the five workload medians range from 45.5 to 53.3 t/s.

Oct 13, 2026

gpt-oss 21B (3.6B active)

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
45.2 tokens/s

User reports gpt-oss 21B-A3.6B at 45.2 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with an unreported backend and quantization, loaded at 32,768 tokens of context on one machine. The 45.2 t/s is the median of three English-prose runs and is client-observed streaming decode throughput, not a pooled rate or prompt-processing rate; the JSON extraction task reached a 53.3 t/s median.

Oct 13, 2026

Muse Glimmer

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
8.6 tokens/s

User reports Muse at 8.6 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with an unknown quantization and a loaded context of 32,768 tokens on one machine. The 8.6 t/s is the median of three English-prose runs, client-observed streaming decode throughput rather than a pooled or peak rate; the JSON task passed 0 of 3 automatic checks.

Oct 13, 2026

Nemotron 3 Nano Omni

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
45.0 tokens/s

User reports Nemotron 3 at 45 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with an unknown quantization and a loaded context of 32,768 tokens on one machine. The 45 t/s is the median of three English-prose runs and is client-observed streaming decode throughput, not a pooled rate or prompt-processing rate.

Oct 13, 2026

Ornith1.5 35B (3B active) Splash

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
61.5 tokens/s

User reports Ornith1.5-35B-A3B Splash at 61.5 t/s on an Apple Mac mini M6 with 32GB unified memory. Setup is LM Studio with the Splash backend and 32,768 tokens of loaded context, one model loaded alone. The 61.5 t/s is the median of three English-prose runs, client-observed streaming decode throughput rather than a pooled rate or peak; the three runs measured 64.7, 59.4 and 61.5 t/s.

Oct 13, 2026

Qwen3.8 27B Splash

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
20.2 tokens/s

User reports Qwen3.8-27B Splash at 20.2 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with the Splash backend, 32,768 tokens of loaded context, and speculative decoding with DFlash2 drafts; quantization is unknown. The 20.2 t/s is the median of three English-prose runs, with individual runs of 20.6, 20.2 and 20 t/s; it is client-observed streaming decode throughput, not a pooled rate or prompt-processing rate.

Oct 13, 2026

Qwen3.8 27B Swift Splash

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
19.5 tokens/s

User reports Swift-Qwen3.8-27B Splash at 19.5 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with the Splash backend and speculative decoding using DFlash2 drafts, loaded at 32,768 tokens of context. The 19.5 t/s is the median of three English-prose runs and is client-observed streaming decode throughput, not a pooled rate or prompt-processing rate; exact quantization is unknown.

Oct 13, 2026

Qwen3.8 27B

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
11.8 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3.8 27B at 11.8 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with its llama.cpp GGUF backend, GGUF Q4_K_M quantization, and a loaded context of 32,768 tokens. The 11.8 t/s is the median of three English-prose runs (Ruhr economy prompt), client-observed streaming decode throughput, not an overall mean or prefill rate.

Oct 13, 2026

Ornith1.5 35B (3B active)

M6 32GB · LM Studio · 32,768 ctx

Apple hardware
generation:
43.7 tokens/s
quant:
Q4_K_M (GGUF)

User reports Ornith1.5 35B-A3B at 43.7 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with its llama.cpp GGUF backend, GGUF Q4_K_M, and a loaded context of 32,768 tokens. The 43.7 t/s is the median of three English-prose runs (Ruhr economy prompt) and is client-observed streaming decode throughput, not an overall mean, peak speed, or prefill rate.

Oct 13, 2026
Apple hardware
generation:
42.3 tokens/s
quant:
UD-Q3_K_XL (GGUF)

User reports Nemotron 3.5 Lightning 30B-A3B at 42.3 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with its llama.cpp GGUF backend, UD-Q3_K_XL quant, and 32,768 tokens of loaded context. The 42.3 t/s is the median of three English-prose runs (Ruhr economy prompt), client-observed streaming decode throughput, not an overall mean or prefill rate.

Oct 13, 2026
Apple hardware
generation:
45.1 tokens/s
quant:
Q4_K_M (GGUF)

User reports Nemotron 3.5 Lightning 30B-A3B at 45.1 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with its llama.cpp GGUF backend, GGUF Q4_K_M, and a loaded context of 32,768 tokens. The 45.1 t/s is the median of three English-prose runs and is client-observed streaming decode throughput, not an overall mean, peak, or prefill rate.

Oct 13, 2026

Qwen3.5 9B

M6 32GB · llama.cpp

Apple hardware
generation:
27.0 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3.5-9B at approximately 27 tokens/second decode on an Apple Mac mini M6 with 32GB unified memory. Setup is llama.cpp with Q4_K_M weights (5.5GB file) at empty prior context. The figure is from a Tweakers review sweep of prior context from empty to 90,000 tokens in 10,000-token increments; the sweep endpoint is not a verified maximum context capacity. Prefill, KV cache, offload and speculation settings are not reported for this result.

Oct 9, 2026
Apple hardware
generation:
16.2 tokens/s
prompt processing (prefill):
439.0 tokens/s
quant:
affine Q4, group size 64 (MLX)

User reports Qwen3.5-4B at 16.24 t/s generation and 438.97 t/s prefill on an Apple M6 Mac mini with 32GB unified memory. Setup is MLX 0.32.2 with MLX-LM 0.31.3, a controlled affine 4-bit (Q4) MLX package with group size 64 and 16-bit KV cache, single request, 2048 input tokens and 128 generated tokens. Measurements are from Precisit's published benchmark, not an independent rerun; decode excludes first-token production and model loading, and the 2048-token input is not a configured context limit.

Oct 8, 2026
Apple hardware
generation:
52.2 tokens/s
quant:
4-bit (MLX)

User reports Gemma-4-26B-A4B 4-bit at 52.2 tok/s decode on a base Mac mini M6 with 32 GB unified memory. Setup is SwiftLM with MLX 4-bit weights, full GPU offload, peak GPU memory 19.5 GB; the longest prompt that passed was 80.7K tokens at 622 tok/s prefill and 24.3 tok/s decode. Also benchmarks Qwen3.6-35B-A3B 4-bit at 46.7 tok/s (GPU) and 13.2 tok/s (--stream-experts), Qwen3.8-27B 4-bit dense at 9.3 tok/s with 200 tok/s prefill, and Gemma-4-26B-A4B 8-bit at 8.8 tok/s with --stream-experts. An A/B against the prior revision shows 998 tok/s prefill at ~2.3K tokens versus 508 tok/s with swap, and 914 tok/s at ~9.5K tokens where the earlier build aborted on swap.

Oct 5, 2026
Showing 1–20 of 21
Page 1 of 2

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3265 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other781

Records by engine

2307 total
llama.cpp1133
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 280vision 173summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use280vision173summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480