llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD Radeon 780M
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Unknown family 26B (4B active)

AMD Radeon 780M · llama.cpp · 16,384 ctx

Tone: mixedAMD hardware
generation:
25.0 tokens/s
quant:
Q4 QAT (GGUF)
flash attention:
on
agenticlong-context

User reports a fully local Italian voice assistant on a mini PC with a Radeon 780M iGPU and 64GB DDR5 shared memory, reaching a median ~2.2s from end of speech to first audio. The LLM is a 26B-A4B MoE at Q4 QAT running in llama.cpp on Vulkan at ~25 tok/s, with 16k context, one slot, flash attention and cache reuse. Prefill of the turn costs ~1.0s (~0.35s fixed per request plus ~3ms/token), ASR ~0.75s and first sentence plus TTS first chunk ~0.55s. The user puts the calculated floor on this box at ~1.2-1.5s. Smaller dense models were benchmarked and rejected for factual fidelity, and KV-cache warming during speech was net neutral-to-negative on one slot. The user asks what else could be done before buying a 24GB dGPU.

Nov 16, 2026
Tone: positiveAMD hardware
generation:
37.0 tokens/s
agentictool-use

User reports LFM2.5-24B-A2B at about 37 t/s at zero context on an AMD Radeon 780M iGPU. No engine, quant or context settings are stated beyond the GGUF build linked. User says it is not a coding model but a task executor, and that it pairs well with Hermes for handing off low-to-mid complexity tasks.

Nov 15, 2026

Qwen3.8 27B

AMD Radeon 780M · llama.cpp · 32,768 ctx

Tone: mixedAMD hardware
prompt processing (prefill):
74.0 tokens/s
quant:
IQ3_S (GGUF)
kv:
q8_0
codingmultilingual

User reports Qwen3.8-27B at 5.1-8.3 t/s generation and ~74 t/s prompt on a Radeon 780M laptop with 32GB unified RAM. Setup is llama.cpp with IQ3_S quant, q8_0 KV cache, 32768 context, and DFlash2 speculative decoding. User also ran Gemma 4 26B-A4B Q4_K_M at 21.8-25.8 t/s, Qwen3.6-35B-A3B IQ4_XS at 24.4-33.3 t/s, and LFM2.5-24B-A2B Q4_0 at 38.7-40.3 t/s. User plans to use Qwen3.8-27B for coding and Gemma 4 26B-A4B for other tasks.

Oct 14, 2026
Tone: positiveAMD hardware
generation:
25.0 tokens/s
prompt processing (prefill):
208.7 tokens/s
quant:
Q4_K_M (GGUF)

User reports Gemma4 26B Q4_K_M at ~25.00 t/s generation and ~208.72 t/s prompt processing on an AMD Radeon 780M iGPU. Setup is llama.cpp with Vulkan backend, Q4_K_M GGUF, -ngl 99, on a MINISFORUM UM890 Pro mini PC running Ubuntu 24.04. A CLI smoke test gave ~23.4 t/s generation and ~37.3 t/s prompt; a no-reasoning run gave ~24.4 t/s generation and ~117.6 t/s prompt. Ollama on the same box was around 4.5 t/s generation, roughly a 5x-6x uplift. Ollama's installed Gemma4 blob could not be loaded directly in upstream llama.cpp due to a tensor count mismatch.

Sep 26, 2026
Tone: mixedAMD hardware
generation:
3.2 tokens/s
prompt processing (prefill):
32.5 tokens/s
coding

User reports Qwen3.8 27B at 3.20 t/s generation and 32.45 t/s prompt processing on a Ryzen 9 7940HS with Radeon 780M iGPU and 24GB RAM. Setup is Ollama on two machines: a Minisforum UM790 Pro and a home build with the same Ryzen 9 7940HS, both with 24GB RAM. The Minisforum run reached 4.73 t/s generation and 20.52 t/s prompt processing. Both runs stopped mid-task and follow-up prompts returned no output. User expected low throughput but not the failures.

Sep 18, 2026
Tone: positiveAMD hardware
generation:
21.1 tokens/s
prompt processing (prefill):
287.3 tokens/s
quant:
Q8_0
kv:
q8_0

User benchmarks Qwen3.6 35B-A3B Q8_0 on a Ryzen 7 260 with a 780M iGPU and 64 GB DDR5, reaching pp8192 287.33 t/s and tg128 21.06 t/s. Setup is the Vulkan backend. Gemma 4 31B Q8_0 on the same hardware gives pp8192 51.59 t/s and tg128 2.46 t/s. With MTP, Gemma 4 31B reaches tg ~5.76 t/s, and Qwen3.6 35B-A3B with MTP and partial offloading reaches tg ~34.85 t/s. The user mentions an RTX 5060 8GB as a bonus for MoE partial offloading.

Sep 7, 2026
AMD hardware
generation:
18.4 tokens/s
prompt processing (prefill):
311.4 tokens/s
quant:
Q8 (GGUF)
kv:
Q8

User benchmarks ROCm against Vulkan on a Radeon 780m iGPU. ROCm gives a 50% pp speedup for dense models. The user also reports Qwen3.8 27B results.

Sep 7, 2026
Tone: mixedAMD hardware
prompt processing (prefill):
100.0 tokens/s

User reports ROCm 7.14 on a Radeon 780M with llama.cpp is fast but unstable and crashes. Setting AMD_SERIALIZE_KERNEL=3 stabilizes it but drops prompt processing to about 100 t/s, down from 200-300 t/s without the workaround. User asks for advice.

Sep 7, 2026
Showing 1–8 of 8
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB89NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3270 total
Qwen3.81569
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other784

Records by engine

2312 total
llama.cpp1136
Strata236
vLLM227
MLX107
Ollama99
other507

Use cases

coding 938agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding938agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active678Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M397UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480