llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD RX 6700 XT
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: mixedAMD hardware
generation:
10.0 tokens/s
quant:
IQ3_XXS (GGUF)
coding

User reports Qwen3.8 27B at IQ3_XXS averaging around 10 t/s on an RX 6700 XT 12GB, with 32GB DDR4 3200 and a Ryzen 5 5600X. No engine or context length is stated for the run. The user says the MoE model they also run gets 35 t/s or more, and is weighing the DeepSeek harness against pi dev and opencode for finding a useful local workload.

Oct 28, 2026
Tone: mixedAMD hardware
generation:
30.0 tokens/s
quant:
Q4_K_XL (GGUF)
kv:
Q8_0
flash attention:
on
coding

User reports 30 t/s generation on an RX 6700 XT 12GB with a Qwen3.6 35B-A3B Tiel-Coder model at Q4_K_XL, using llama.cpp with the Vulkan backend on Windows 11. Setup uses 100,000 context, Q8_0 KV cache, flash attention, 28 MoE layers offloaded to CPU, 6 threads, and MTP speculative decoding with draft-n-max 3. User is asking for advice to reach 40 t/s without changing the quant, and notes 6 threads was best out of 4, 8, and 12.

Oct 11, 2026
Tone: positiveAMD hardware
generation:
61.0 tokens/s
prompt processing (prefill):
851.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
f16

User reports Qwen3-8B Q4_K_M at 61 t/s generation and 851 t/s prompt on an RX 6700 XT 12 GB. Setup is the bundled llama.cpp (0.85.5) with the AMD Flash Attention kernel, KV cache f16, measured with pp512 / tg128. The project's headline claim is that the same card goes from 0.6-2.6 t/s under stock llama.cpp to ~61 t/s with ToshLLM. The page also lists Qwen3.6-35B-A3B Q4_K_S at 475 t/s prompt / 29 t/s generation with --n-cpu-moe 24, and community rows including a 122B-A10B MoE across four W6800X GPUs at 226 t/s prompt / 26 t/s generation.

Oct 9, 2026

Gemma 4 12B

AMD RX 6700 XT · llama.cpp · 8,192 ctx

Tone: mixedAMD hardware
generation:
34.6 tokens/s
prompt processing (prefill):
653.9 tokens/s
quant:
IQ4_NL (GGUF)
kv:
q8_0
long-context

User benchmarks Gemma 4 12B (IQ4_NL, 6.24 GiB) on an AMD RX 6700 XT under llama.cpp, comparing the ROCm and Vulkan backends at 8192-token prefill and 512-token generation with q8_0 KV cache and flash-attention on. ROCm averages 653.9 t/s prefill and 34.60 t/s decode over 3 runs; Vulkan averages 354.4 t/s prefill and 40.92 t/s decode. ROCm is 84.5% faster on prefill but 15.4% slower on decode, giving a net wall-clock win of about 23% for a full 8192-prefill plus 512-generate cycle, with a crossover near 1760 prompt tokens. ROCm required two workarounds on gfx1031: building for gfx1030 with HSA_OVERRIDE_GFX_VERSION=10.3.0, and patching a flash-attention assert in fattn-common.cuh. A separate TOP_K sampler gap on ROCm is noted as under investigation.

Oct 6, 2026

Qwen3.8 27B

2× AMD RX 6700 XT · llama.cpp · 144,000 ctx

Tone: positiveAMD hardware
generation:
18.0 tokens/s
prompt processing (prefill):
150.0 tokens/s
quant:
UD-IQ3_XXS (GGUF)
agenticlong-context

User reports Qwen3.8 27B at 18 t/s generation and 150 t/s prompt processing on a 2007 Dell Precision T5400 with dual RX 6700 XT / RX 6700 (22GB total VRAM). Setup is llama.cpp with UD-IQ3_XXS quant at 144k context, using MTP q4_0 speculative decoding, on 24GB DDR2 and dual Xeon X5460. User compares five systems and argues older dual-Xeon dual-GPU setups beat a 2025 HP Omen with RTX 5070 in context length and speed, concluding DDR5 is not worth the cost for agentic tasks.

Sep 28, 2026
Showing 1–5 of 5
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3266 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other782

Records by engine

2308 total
llama.cpp1134
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480