llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: Intel Arc A770 16GB
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: mixedIntel hardware
generation:
69.7 tokens/s
prompt processing (prefill):
2074.2 tokens/s
quant:
Q4_0 (GGUF)

User benchmarks Llama 2 7B Q4_0 on an Arc A770 16GB with arcint's llama.cpp patches and flash attention on, reaching 2074.21 t/s prefill and 69.71 t/s decode in llama-bench (512 prefill, 128 decode, 5 repetitions). Both builds are at llama.cpp commit 3e4dfa8 on a Ryzen 5700X, Debian 13, xe driver; the A770 sits on PCIe 3.0 x4 behind the chipset with its clock pinned at 2000 MHz. Stock SYCL fp16 with flash attention decodes faster at 73.66 t/s but prefills at 1457.24 t/s; arcint leads prefill by 10.5% on the A770 and 31% on the B60, while SYCL wins decode by 5.4% and 4.4%. Through arcint's own server the A770 gives 1963 / 70.5. On the B60, Qwen3.6-35B-A3B Q4_K_M runs at 2332 / 84.9 and Qwen3.8-27B Q4_K_M at 844 / 19.9; the 27B at full 262,144 context with MTP drafts decodes 47.1 t/s at 3,599 prompt tokens down to 12.5 t/s at 234,553.

Nov 5, 2026

Gemma 4 12B

Intel Arc A770 16GB · llama.cpp · 128,000 ctx

Tone: positiveIntel hardware
generation:
20.0 tokens/s
prompt processing (prefill):
1100.0 tokens/s
quant:
QAT
kv:
Q8
multilingualrp

User reports Gemma 4 12B QAT at 20 t/s decode and 1100 t/s prefill on a single Intel Arc A770 16GB. Setup is llama.cpp with QAT weights, Q8 KV cache and a Q4 MTP head, giving 128k context with about 5GB VRAM headroom; the card is in a 13-year-old Haswell desktop with under 6GB RAM headroom. Four models run together at 14-15GB VRAM via llama-swap, with LiteLLM routing prompts through Laya to thinking and non-thinking aliases of the 12B. The user notes SYCL/llama.cpp lacks Alchemist optimizations, making the A770 the bottleneck, and that GPU-resident models still use 4-5GB of system RAM as compute buffer.

Oct 26, 2026
Tone: positiveIntel hardware
generation:
49.0 tokens/s
prompt processing (prefill):
48.0 tokens/s
quant:
Q6_K (GGUF)
kv:
q4_1
flash attention:
on
codingtool-usevisionlong-contextagentic

User reports Qwen3.5-9B at 49 t/s generation and 48 t/s prompt processing on an Intel Arc A770 16GB. Setup is llama.cpp b9521 (Vulkan) with Q6_K weights, q4_1 KV cache, 256K context, flash attention, vision mmproj and MTP speculative decoding on a single slot. The same model under WSL2 with Q8_0 KV cache and 128K context reached about 40 t/s without vision. Qwopus3.5-4B-Coder hit 64 t/s generation and 100 t/s prompt at 96K context, and Gemma 4 12B reached 22 t/s generation and 74 t/s prompt at 128K context.

Oct 7, 2026
Intel hardware
generation:
14.4 tokens/s
quant:
Q4_K_M (GGUF)
coding

User reports Qwen3.6-27B-A3B-Coder at 14.4 t/s decode on an Intel Arc A770 16GB, scoring 10/10 on a Lua CSV parser acceptance task. Setup is llama.cpp with the SYCL backend and GGUF Q4_K_M weights, a comparison point against arcint's own AWQ IR serving on the same card. The figure is described as a rough bound rather than a directly comparable measurement, since the quantisation and engine differ from arcint's production configuration.

Oct 5, 2026
Intel hardware
generation:
33.2 tokens/s
quant:
Q8_0 (GGUF)

User benchmarks Qwen3-1.7B at 33.2 tok/s on an Intel Arc A770 16GB using llama.cpp with the SYCL backend. Setup is llama.cpp SYCL with Q8_0 quantisation, single-stream (n_parallel=1). OpenVINO via OVMS reached 65.4 tok/s on the same model. Ten models were tested in total, with OpenVINO faster than SYCL on every model.

Sep 23, 2026
Tone: mixedIntel hardware
generation:
12.0 tokens/s
quant:
Q3_K_S (GGUF)

User reports Qwen3.8 27B at 12 t/s on an Intel Arc A770 using pure K-quants. Setup uses the SYCL backend, where I-quant mixes like Unsloth's UD-Q3_K_XL run at 7 t/s while pure K-quants like Bartowski's Q3_K_S run at 12 t/s. The user notes SYCL is bottlenecked by inefficient I-quant handling. The user is looking for the smallest K-quant-only Qwen3.8 27B at q3, with Bartowski's Q3_K_S at 12.7 GB as the current top contender.

Sep 14, 2026
Showing 1–6 of 6
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3266 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other782

Records by engine

2308 total
llama.cpp1134
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480