llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD RX 9060 XT 16GB
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: mixedAMD hardware
generation:
45.5 tokens/s
prompt processing (prefill):
161.9 tokens/s
quant:
Q2_0 (GGUF)
kv:
int8
chat

User reports Qwen3.8-Flash-Next Q2_0 at 45.5 t/s decode and 161.9 t/s prefill (pp4k) on an RX 9060 XT 16GB with 32GB DDR4, but says prefill is usually under 150 t/s and asks how to fix it. Setup is Strata 0.1.52 (HIP backend, Windows) with a 64K context and int8 KV cache; the 37.6 GB GGUF exceeds the card's VRAM, so weights run partly from system RAM. Strata's tuner ran 501.5 s and confirmed the defaults; two tuning arms were skipped for lack of free memory (25.6 GiB free, 28.4 GiB needed).

Oct 28, 2026

Qwen3.6 35B (3B active)

AMD RX 9060 XT 16GB · PolyStrata-HIP · 31,296 ctx

Tone: positiveAMD hardware
generation:
70.7 tokens/s
prompt processing (prefill):
1326.0 tokens/s
quant:
UD-Q3_K_M (GGUF)
kv:
FP16

User reports Qwen3.6-35B-A3B at 70.7 tok/s generation on a 31,296-token prompt on an RX 9060 XT 16GB with a Ryzen 5 5600X and 32GB RAM on Windows 11. Setup is the PolyStrata-HIP fork of the strata-poly GGUF engine with Unsloth's UD-Q3_K_M quant, FP16 KV cache and MTP speculation enabled; experts not resident in VRAM are computed on the CPU from system RAM. Prefill on that run was 1,326 tok/s. Shorter warm prompts reached 93.4 tok/s at 2,068 tokens and 85.8 tok/s at 8,235 tokens, while a 45,355-token prompt gave 61.9 tok/s. With Q8_0 KV and 262,144 tokens of allocated capacity, a 400-token response ran at 33.8 tok/s capped at 2,000 resident experts versus 47.6 tok/s uncapped with 5,097 resident. Streaming KV through pinned RAM at roughly 261K tokens dropped generation to 3.64 tok/s.

Oct 18, 2026

Qwen3.8 27B

AMD RX 9060 XT 16GB · llama.cpp · 20,000 ctx

Tone: positiveAMD hardware
generation:
42.7 tokens/s
prompt processing (prefill):
658.0 tokens/s
quant:
IQ4_XS (GGUF)
kv:
Q4/Q8
mtp (multi-token prediction):
on

User reports Qwen 3.8 27B at 42.7 t/s generation and 658 t/s prefill at 20k context on a single RX 9060 XT 16GB. Setup is a fork of llama.cpp (achyudh/llama.cpp) with an unsloth IQ4_XS GGUF, Q4 KV values with Q8 keys, MTP plus n-gram prediction, and KV cache streaming; 24GB of DDR5 system RAM, 16GB of it used for this model. At 200k context the user reports 35 t/s generation and 300 t/s prefill, with a 262k context target. The user notes the build needed PCIe atomics fixes.

Oct 18, 2026
AMD hardware
generation:
66.0 tokens/s
quant:
UD-Q4_K_XL (GGUF)
kv:
q8_0

User reports Qwen3.6-35B-A3B at 66.04 tokens/s on an AMD Radeon RX 9060 XT 16 GB with 32 GB of system RAM. Setup is llama.cpp (Vulkan backend) with UD-Q4_K_XL GGUF weights, q8_0 KV cache, 40k context, and MTP speculative decoding drafting up to 3 tokens per step. The 35B MoE model does not fit in 16 GB of VRAM, so the expert weights of the first 20 layers run on the CPU. Run on a Ryzen 7 5800X3D desktop under SteamOS, single slot, with flash attention and prefix cache reuse enabled.

Oct 6, 2026
AMD hardware
generation:
30.0 tokens/s
quant:
IQ3_S (GGUF)

User asks whether Qwen3.8 Flash-Next can run on an RX 9060 XT 16GB with 32GB DDR5-6000. They currently run Swift-1.5-Qwen3.8 at IQ3_S at 30 t/s on the same machine.

Oct 3, 2026
AMD hardware
generation:
50.0 tokens/s
prompt processing (prefill):
850.0 tokens/s
quant:
IQ3_S (GGUF)

User reports 850 t/s prefill and 50 t/s generation with MTP at temperature 0 on an AMD RX 9060 XT 16GB, running Qwen3.8-27B-GSQ-RCO IQ3_S in a custom inference engine. MTP is used for the generation figure; the user notes MTP with temperature above 0 is not developed yet. User asks for ideas to systematically test the engine, having already run KL divergence against BF12 on CPU, bit correctness checks, needle-in-a-haystack at 25/50/75/90% key positions with distractors (also against llama.cpp), and HumanEval (92/93 passed so far).

Oct 3, 2026
AMD hardware
generation:
33.5 tokens/s
prompt processing (prefill):
1210.3 tokens/s
quant:
Q4_K_M (GGUF)

User benchmarks Qwen 2.5-Coder 14B Instruct Q4_K_M at 33.51 t/s generation and 1210.29 t/s prompt processing on an AMD RX 9060 XT 16GB via Vulkan. Setup is llama.cpp build c4ae9a88f8 with -ngl 99 and -fa 1; the same model on ROCm reaches 30.79 t/s generation and 1211.32 t/s prompt processing. A Qwen 3.5 9B Q4_K_M run reaches 50.15 t/s generation and 1904.16 t/s prompt processing on Vulkan, and a 14B quant sweep gives about 32, 28 and 20 tok/s for Q4_K_M, Q5_K_M and Q8_0.

Sep 29, 2026

Ornith1.5 9B

AMD RX 9060 XT 16GB · llama.cpp · 262,144 ctx

Tone: positiveAMD hardware
generation:
36.0 tokens/s
prompt processing (prefill):
950.0 tokens/s
quant:
Q6_K (GGUF)
kv:
Q8
codingagentic

User reports Ornith-1.5-9B at about 36 t/s generation and about 950 t/s prompt eval on an AMD RX 9060 XT 16GB with llama.cpp. Throughput drops to about 25 t/s generation and about 500 t/s prompt eval during long tasks. The run lasted about 3.5 hours on a coding agent task. The user contrasts this with Qwen 3.8 27B, which was frustratingly slow on the same hardware.

Sep 7, 2026
Showing 1–8 of 8
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3265 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other781

Records by engine

2307 total
llama.cpp1133
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 280vision 173summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use280vision173summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480