llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD RX 6900 XT 16GB
Compare setup details (1 active)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

AMD hardware
generation:
40-45 tokens/s
quant:
Q2

User reports Qwen3.8 Flash-Next at around 40-45 tok/s on an RX 6900 XT 16GB, depending on the prompt. Setup is Strata with a Q2 quant on a 32GB DDR4 system (Ryzen 7 5700X, B450 board). User asks whether to upgrade to 64GB RAM with an X570 board or add a second 6800/6900 XT for 16GB more VRAM, noting the current board's second slot is PCIe 2.0 x4. A dual-GPU goal would be Qwen3.8 27B at Q4 or higher.

Oct 21, 2026
Tone: positiveAMD hardware
generation:
41.5 tokens/s
quant:
Q2_0 (GGUF)
kv:
Q8
coding

User reports Qwen3.8 Flash-Next at 41.5 t/s on an RX 6900 XT 16GB, generating a roughly 4500-token animated SVG response; a Python task ran at about 32 tok/s. Setup is Strata with Q2_0 weights, 64k context and Q8 KV cache, using around 13.2GB VRAM on a Ryzen 7 5700X with 32GB DDR4 3600. The user has not filled the 64k context yet and wonders whether Q4 KV would allow more; the plan is 64GB RAM with IQ3_S, and IQ3_XXS is also being considered.

Oct 17, 2026
Tone: mixedAMD hardware
generation:
25.0 tokens/s
quant:
GSQ-RCO

User reports Qwen3.8-Flash-Next at 25 tok/s on an RX 6900 XT installed in a retired HP DL380p server with 172 GB DDR3 and 2x 10-core CPUs. Setup is Strata with llama.cpp and an HTTP router, GSQ-RCO quant at 64k context. The GPU is powered by an external desktop PSU; total draw is 250 W idle and about 450 W while inferring. User also reports Qwen3.8-27B with GSQ-RCO-IQ3_XXS and MTP at 192k context reaching 45 tok/s. The machine is a prototype; user plans a flexible riser and a proper GPU platform, and may add Nvidia P40s in the remaining slots someday.

Oct 6, 2026
Showing 1–3 of 3
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3266 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other782

Records by engine

2308 total
llama.cpp1134
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480