llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD RX 7600 XT 16GB
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Qwen3.8 27B

2× AMD RX 7600 XT 16GB · llama.cpp · 35,000 ctx

AMD hardware
generation:
30.0 tokens/s
prompt processing (prefill):
550.0 tokens/s
quant:
UD-Q4_K_XL (GGUF)
kv:
f16

User reports Qwen3.8 27B at 550 t/s prompt processing and 30 t/s generation on an RX 7600 XT 16GB plus RX 9060 XT 16GB, in a moderate context of 35k tokens. Setup is llama.cpp (latest git) with UD-Q4_K_XL GGUF weights, f16 KV cache, tensor split 60/40, flash attention on, and draft-MTP plus ngram-mod speculative decoding. ROCm was considerably faster than Vulkan. Adaptive KV streaming reached over 800 t/s prefill but limited the user to lower quantizations and left the second card unused. The user asks whether Strata or another fork would do better on this 32GB RAM system.

Nov 16, 2026

Qwen3.8 27B

AMD RX 7600 XT 16GB · llama.cpp · 114,688 ctx

Tone: positiveAMD hardware
quant:
GSQ-RCO-IQ3_XXS (GGUF)
kv:
Q4_0
mtp (multi-token prediction):
on
codingagentic

User benchmarks Qwen3.8 27B on an RX 7600 XT 16GB, completing 15/15 tasks with 1.000 correctness in 348.5 seconds. Setup is llama.cpp HIP ROCm with the GSQ-RCO-IQ3_XXS GGUF and a Q4_0 KV cache at 114,688 context, fully in VRAM with no CPU offload. The same benchmark also ran Ornith-1.5-9B (15/15, 0.983, 184.9s) and K2-Horizon-7B (11/15, 0.909, 498.1s); the user calls Qwen3.8 27B the undisputed winner for agentic coding.

Sep 23, 2026

Qwen3.8 27B

AMD RX 7600 XT 16GB · llama.cpp · 163,840 ctx

Tone: positiveAMD hardware
generation:
18.0 tokens/s
prompt processing (prefill):
141.0 tokens/s
quant:
IQ3_XXS (GGUF)
kv:
q8_0
mtp (multi-token prediction):
off
coding

User reports Qwen3.8 27B at 18 t/s decode and 141 t/s prefill on a 1.8k-token prompt, running on an AMD Radeon RX 7600 XT 16 GB at 163,840 context. Setup is llama.cpp llama-server build 10480 with the unsloth Qwen3.8-27B-UD-IQ3_XXS GGUF (10.2 GiB), q8_0 KV cache, flash attention on, and Vulkan (RADV) backend. The model is a hybrid architecture where only 16 of 64 layers keep a full KV cache, so KV is about 5.3 GiB at this context. With MTP on the same model runs about 24 t/s at 98k context and 39 t/s at 124k. The user notes 160k is a VRAM-math ceiling, not a quality claim, and that decode drops to about 6 t/s if the GPU spills to system RAM.

Sep 23, 2026

Muse 30B Glimmer

AMD RX 7600 XT 16GB · llama.cpp · 62,144 ctx

Tone: positiveAMD hardware
generation:
20.0 tokens/s
prompt processing (prefill):
308.0 tokens/s
quant:
UD-Q2-K-XL
coding

User reports Muse Glimmer 30B at about 308 t/s prompt processing and about 20 t/s generation on an RX 7600 XT 16GB. Setup is llama.cpp with ROCm, the UD-Q2-K-XL quant and DFlash speculative decoding. The run completed a coding task.

Sep 8, 2026
Showing 1–4 of 4
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB89NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3270 total
Qwen3.81569
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other784

Records by engine

2312 total
llama.cpp1136
Strata236
vLLM227
MLX107
Ollama99
other507

Use cases

coding 938agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding938agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active678Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M397UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480