llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

Model: Qwen3-Coder
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: positiveNVIDIA hardware
generation:
107.0 tokens/s
prompt processing (prefill):
3706.5 tokens/s
quant:
Q4_K_M (GGUF)
kv:
Q8_0
codingagentic

User reports Qwen3-Coder-30B-A3B-Instruct at 107.0 tok/s generation and 3,706.5 tok/s prefill on an RTX 5090 Laptop 24GB in an MSI Raider A18. Setup is llama.cpp with CUDA, Q4_K_M GGUF (17.28 GiB), Q8_0 KV cache, FlashAttention on, 65,536-token context, all layers on the GPU, one slot. The figures come from a repository-analysis workload with an 8,638-token prompt and 371 generated tokens; the user notes the screenshot captures one strong interval, not an average across the audit.

Nov 17, 2026

Qwen3-Coder 30B (3B active)

GTX 1060 6GB · llama.cpp · 131,072 ctx

Tone: mixed
generation:
4.5 tokens/s
prompt processing (prefill):
25.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
q8_0
codingagentictool-use

User reports Qwen3-Coder-30B-A3B at 25 t/s prompt processing and 4.5 t/s generation on a GTX 1060 6GB, with an 11K-token initial context including MCP tools. Setup is llama.cpp (llama serve) with Q4_K_M weights, q8_0 KV cache, 131072 context, -cmoe CPU offload and 10 threads on a Xeon E5-2666 v3 with 16GB RAM; the run uses 4.5GB VRAM and up to 13GB system RAM. User finds the model struggles with short prompts and asks for alternatives good at agentic coding and tool calling.

Nov 11, 2026
AMD hardware
generation:
97.7 tokens/s
prompt processing (prefill):
1115.3 tokens/s
quant:
Q4_K_S (GGUF)
kv:
f16
coding

User reports 97.73 t/s generation and 1115.3 t/s prompt processing. Source evidence (verbatim): Source: https://www.soothill.io/blog/2026/08/03/llamacpp-vulkan-vs-rocm-strix-halo/ # llama.cpp: Vulkan vs ROCm on Strix Halo Host: EVO-X3, 128GB integrated-memory Strix Halo desktop. llama.cpp Vulkan b10216 and ROCm b10085. 6 threads pinned, 32,768 context, flash attention on, batch/ubatch 2048/512, K/V cache f16, mmap off. Qwen3-Coder-30B-A3B-Instruct Q4_K_S (17.46 GB), prompt 512, generation 128, single stream: Vulkan 5 runs prompt 1,115.30 tok/s, generation 97.73 tok/s. ROCm 50 runs prompt 1,344.65, generation 73.65. Qwen3.5-122B-A10B Q4_K_XL (77.03 GB), ROCm: pp512 339.9, pp4096 523.0, generation 21.3 tok/s. Vulkan: pp512 285.0, pp4096 374.3, generation 22.9 tok/s. No speculative decoding stated.

Oct 16, 2026
Tone: positiveAMD hardware
generation:
173.0 tokens/s
quant:
BF16 (safetensors)
coding

User benchmarks Qwen3-Coder-30B-A3B in BF16 on a single MI300X 192GB, reporting 173 tok/s decode on a 32K prompt with SGLang defaults. Setup is SGLang 0.5.21 with a 131K context window, prefix caching off, and 256-token replies; vLLM 0.29.1 with AITER attention reached 164 tok/s and llama.cpp (HIP) 108 tok/s with a 20.5 s first token. SGLang was fastest from 1 to 64 users; vLLM only pulled ahead at 128 users. Prefix caching was the only tuning knob that helped, adding 8 to 11 percent. The user notes vLLM without AITER attention collapsed to 27 tok/s total on 16 users with 32K prompts versus 133 on SGLang.

Oct 16, 2026
AMD hardware
generation:
50.1 tokens/s
prompt processing (prefill):
442.1 tokens/s
quant:
Q4_K_M (GGUF)
kv:
Q8_0
coding

User reports 50.070235 t/s generation and 442.1505415 t/s prompt processing. Source evidence (verbatim): Source: https://www.soothill.io/blog/2026/08/19/vulkan-064-rocm-714-strix-halo/ # Strix Halo Qwen3-Coder Q4_K_M Vulkan measured 32K-depth benchmark First-hand Darren Soothill benchmark, measured/published 19 August 2026. Hardware GMKtec EVO-X3, Radeon 8060S gfx1151, 128GB unified memory (Strix Halo). CPU SKU and core counts not established by this article alone. Selected model Qwen3-Coder-30B-A3B-Q4_K_M, GGUF, llama.cpp Vulkan v0.6.4 build 10565 commit baf6360be (author short commit; full revision unknown). Selected measured context depth 32768 tokens already present before the operation; this is occupied depth, not allocated capacity. Microbatch 2048, batch2048, all available layers offloaded, Flash Attention enabled, Q8_0 K and V cache, one server slot, no-host loading. Native generation tg128 at that measured depth: 50.070235 tokens per second. Native prompt processing pp512 at the same depth and microbatch: 442.1505415 tokens per second. These are separate operation cells of the same matched synthetic configuration, not a single combined request or end-to-end rate. The author reports median of launch means. This 32K comparison used two launches per backend in ABBA order. Mesa26.3.0-devel RADV, libdrm2.4.134, shaderc2026.3-dev. IOMMU stayed enabled; no thermal, firmware or power-policy changes. Server slot count does not prove request concurrency; no concurrent request rate is inferred. Exact tested quant conversion repository, filename, checksum and revision are not published. Resident weight allocation and RAM consumption are unspecified. Original article https://www.soothill.io/blog/2026/08/19/vulkan-064-rocm-714-strix-halo/ . Original CSV https://www.soothill.io/assets/data/strix-halo-vulkan-064-rocm-714-2026-08-19.csv . Verified publisher model link https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct is a base-model reference, not the tested Q4_K_M artifact. Other backends, shallow cells, microbatches and served MTP requests are separate tests and must not be combined with the selected 32K synthetic cells.

Oct 14, 2026
AMD hardware
generation:
67.5 tokens/s
quant:
Q4_K_S (GGUF)
coding

User reports 67.5288 t/s generation. Source evidence (verbatim): Source: https://www.soothill.io/blog/2026/08/14/coding-model-benchmark-strix-halo/ # Qwen3-Coder 30B-A3B Q4_K_S: first-hand EVO-X3 coding decode measurement Original author Darren Soothill measured this deployment on 14 August 2026, published 14 August and updated 15 August. Hardware: GMKtec EVO-X3, Ryzen AI MAX+ 395, Radeon 8060S, 128GB unified memory (Strix Halo). Selected native decode result: Qwen3-Coder-30B-A3B-Instruct-Q4_K_S, llama.cpp, 67.5288 tokens per second. The author's CSV records this separately from wall_seconds_per_task 3.9879 and completion_tokens_per_task 250.8. This is an aggregate deployment result across ten first-response coding tasks, not a standalone llama-bench tg128 test. The publication does not specify whether the decode aggregate is arithmetic or token-weighted; do not reinterpret its statistic. Runtime build/version, CPU/GPU core counts, OS, concurrency, actual per-task prompt tokens and prefill rate are not supplied. Every coding request used a configured 32768-token context capacity and 2048-token output ceiling, temperature 0, top-p 1, presence penalty 0, thinking disabled and no repair turn. Configured context capacity is not the actual input length or occupied context depth. The model has 30.5B total and 3.3B active parameters. CSV installed_gib 16.26 describes installed storage, not resident RAM or GPU allocation. The 5/10 task pass score is quality, not throughput. Other model/quant rows and their speeds are separate deployments and must not be combined with this selected row. Original article: https://www.soothill.io/blog/2026/08/14/coding-model-benchmark-strix-halo/ . Author's machine-readable source: https://www.soothill.io/assets/data/evox3-coding-model-benchmark-2026-08-14.csv . Verified Hugging Face link supplied by the author: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct . This is a base-model reference, not the exact tested Q4_K_S quantization artifact. Tested conversion repository, filename checksum and revision are not established; never infer them from the base repository.

Oct 14, 2026

Qwen3-Coder 30B (3B active)

AMD Radeon 8060S integrated GPU (Strix Halo; installed RAM unreported) · llama.cpp b6884 / 229bf6862 / ROCm

generation:
62.6 tokens/s
prompt processing (prefill):
577.5 tokens/s
quant:
UD-Q4_K_XL (GGUF)
flash attention:
off

User reports Qwen3-Coder 30B-A3B at 62.6 t/s generation and 577.5 t/s prompt processing on a Minisforum M1-S1 MAX with Ryzen AI Max+ 395 and Radeon 8060S (gfx1151). Setup is llama-bench with UD-Q4_K_XL, flash attention off, and ngl=100. The 62.6 t/s is the TG128 figure; at 8192-token work counts the same run drops to 33.23 t/s generation and 376.64 t/s prompt processing.

Oct 14, 2026

Qwen3-Coder 30B (3B active)

AMD Strix Halo 128GB · llama.cpp / Vulkan KHR cooperative matrices / build unknown

AMD hardware
generation:
9.2 tokens/s
prompt processing (prefill):
154.0 tokens/s
quant:
BF16

User reports Qwen3-Coder 30B at 9.2 t/s generation and 154 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp with Vulkan on the Radeon 8060S iGPU, BF16 quantization, 6 threads and maximum GPU offload, run as llama-bench with -p 512 -n 128. The source gives 9.19 t/s TG and 154 t/s PP as separate synthetic phase tests, not an end-to-end rate; context depth, KV precision and memory use are unreported.

Oct 13, 2026
AMD hardware
generation:
69.6 tokens/s
prompt processing (prefill):
812.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
F16
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder-30B-A3B-Instruct at 69.6 t/s generation and 812 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp with Q4_K_M weights, f16 K/V cache, flash attention, and one GPU (Radeon 8060S) with n_gpu_layers=999. The figures are separate synthetic llama-bench PP512 and TG128 tests at n_depth=0, not generation after a 512-token prompt and not HTTP end-to-end throughput; no long-context rate is implied.

Oct 13, 2026
AMD hardware
generation:
65.9 tokens/s
prompt processing (prefill):
401.5 tokens/s
quant:
UD-Q4_K_XL (GGUF)
flash attention:
on

User reports Qwen3-Coder-30B-A3B-Instruct at 65.9 t/s generation and 401.5 t/s prompt processing on an EVO-X2 with AMD Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama.cpp with the UD-Q4_K_XL GGUF, ROCm 6.4.0, Flash Attention on, and one gfx1151 iGPU at ngl=100. The source gives 65.93 +/- 0.18 t/s for tg128 and 401.48 +/- 2.72 t/s for pp512, with tg8192 at 63.27 t/s; these are separate synthetic microbenchmarks, not a same-request prompt and answer.

Oct 13, 2026
AMD hardware
generation:
59.8 tokens/s
prompt processing (prefill):
409.5 tokens/s
quant:
UD-Q4_K_XL (GGUF)
flash attention:
off

User reports Qwen3-Coder-30B-A3B-Instruct at 59.8 t/s generation and 409.5 t/s prompt processing on an EVO-X2 with AMD Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama.cpp with the UD-Q4_K_XL GGUF, ROCm 6.4.0, Flash Attention off, and one gfx1151 iGPU at ngl=100. The rates are separate llama-bench microbenchmarks, not a same-request prompt and answer; longer tests in the same arm give pp8192 254.28 t/s and tg8192 30.78 t/s.

Oct 13, 2026
AMD hardware
generation:
71.3 tokens/s
prompt processing (prefill):
399.8 tokens/s
quant:
UD-Q4_K_XL (GGUF)
kv:
F16
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder-30B-A3B-Instruct at 71.3 t/s generation and 399.8 t/s prompt processing on an EVO-X2 with AMD Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama.cpp build 6060 with the UD-Q4_K_XL GGUF, Flash Attention on, Vulkan backend, and ngl=100 on the single integrated Radeon 8060S. The figures are separate llama-bench microbenchmarks: tg128 at 71.29 +/- 0.11 t/s and pp512 at 399.76 +/- 1.41 t/s, with longer runs of tg8192 at 59.91 +/- 1.95 t/s and pp8192 at 315.36 +/- 0.33 t/s.

Oct 13, 2026
AMD hardware
generation:
73.0 tokens/s
prompt processing (prefill):
391.5 tokens/s
quant:
UD-Q4_K_XL (GGUF)
kv:
F16
flash attention:
off
mtp (multi-token prediction):
off

User reports Qwen3-Coder-30B-A3B-Instruct at 73 t/s generation and 391.5 t/s prompt processing on an EVO-X2 with AMD Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama.cpp build 6060 with the UD-Q4_K_XL GGUF, Flash Attention off, Vulkan backend, and ngl=100 on the integrated Radeon 8060S. The source-native rates are separate llama-bench microbenchmarks: pp512 at 391.54 +/- 1.50 t/s and tg128 at 72.99 +/- 0.31 t/s, with longer tests at pp8192 284.71 t/s and tg8192 46.60 t/s.

Oct 13, 2026
Apple hardware
generation:
71.2 tokens/s
prompt processing (prefill):
1096.8 tokens/s
quant:
UD-Q4_K_XL (GGUF)
kv:
F16
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder 30B-A3B at 71.2 t/s generation and 1096.8 t/s prompt processing on a Mac Studio M2 Ultra with 128GB unified memory. Setup is llama.cpp build 6060 with the UD-Q4_K_XL GGUF and flash attention enabled, run with -ngl 100 on one 60-core GPU. The figures are separate synthetic llama-bench tests: PP512 at 1096.84 +/- 5.48 t/s and TG128 at 71.23 +/- 0.08 t/s, with 16 CPU threads and a 16.45 GiB model footprint.

Oct 13, 2026
Apple hardware
generation:
69.3 tokens/s
prompt processing (prefill):
1080.5 tokens/s
quant:
UD-Q4_K_XL (GGUF)
kv:
F16
flash attention:
off
mtp (multi-token prediction):
off

User reports Qwen3-Coder 30B-A3B at 69.3 t/s generation and 1080.5 t/s prompt processing on a Mac Studio M2 Ultra with 128GB unified memory. Setup is llama.cpp build 6060 with the UD-Q4_K_XL GGUF and flash attention off, one GPU with -ngl 100 and 16 threads. The figures are separate synthetic llama-bench tests: PP512 and TG128, not a 512-input/128-output request. The same run measured 818.85 t/s PP8192 and 40.35 t/s TG8192.

Oct 13, 2026

Qwen3-Coder Next / b8118 Vulkan / PP512 TG128 at prepared depth8000

AMD Radeon 8060S gfx1151 (Strix Halo; CPU SKU and installed RAM unknown) · llama.cpp

generation:
31.2 tokens/s
prompt processing (prefill):
341.3 tokens/s
quant:
UD-Q6_K_XL (GGUF)
kv:
Q8
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder-Next UD-Q6_K_XL at 31.2 t/s generation and 341.3 t/s prompt processing on a single Radeon 8060S gfx1151 device. Setup is llama.cpp b8118 with q8_0 KV cache, -ngl 999, -fa 1, -ub 2048 -b 2048, and 16 CPU threads; the Vulkan/RADV BLAS arm produced the headline 341.27/31.20 tok/s. The figures are means of five repetitions with warmup, measured at a 8000-token depth prefix that is excluded from timing; the ROCm/HIP arm reached 303.02/24.74 tok/s.

Oct 13, 2026

Qwen3-Coder Next / b8118 Vulkan / PP512 TG128 at prepared depth8000

AMD Radeon 8060S gfx1151 (Strix Halo; CPU SKU and installed RAM unknown) · llama.cpp

generation:
30.0 tokens/s
prompt processing (prefill):
358.9 tokens/s
quant:
Q8_0 (GGUF)
kv:
Q8
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder-Next Q8_0 at 30 t/s generation and 358.9 t/s prompt processing on a single Radeon 8060S gfx1151 device. Setup is llama.cpp b8118 with q8_0 KV cache, 16 CPU threads, -ngl 999, -fa 1 and a 8000-token depth; no speculative draft is used. The figures are the Vulkan/RADV arm, measured as 29.98 ± 0.51 t/s TG128 and 358.92 ± 2.25 PP512; the 8000-token prefix is excluded from timing.

Oct 13, 2026

Qwen3-Coder Next / b8180 self-build, PP512 TG128 at prepared depth32768

HP Zbook Ultra G1a / Radeon 8060S (Strix Halo; 128 GB unified RAM; CPU SKU unknown) · llama.cpp

generation:
24.2 tokens/s
prompt processing (prefill):
344.0 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder-Next Q8_0 at 24.2 t/s generation and 344 t/s prompt processing on an HP Zbook Ultra G1a Strix Halo laptop with 128 GB unified RAM and a Radeon 8060S gfx1151 GPU. Setup is a self-compiled llama.cpp b8180 with ROCm 7.2, flash attention, ngl 99, and no speculative decoding, run at 32768 depth. The figures are separate synthetic PP512 and TG128 tests, not one sampled request; the same run measured 343.99 t/s PP and 24.21 t/s TG at depth 32768, and 251.15 t/s PP and 22.09 t/s TG at depth 65536.

Oct 13, 2026

Qwen3-Coder 30B (3B active)

AMD Ryzen AI Max+ 395 / Radeon 8060S (Framework Desktop; 64 GB unified) · llama.cpp

generation:
69.2 tokens/s
prompt processing (prefill):
1242.4 tokens/s
quant:
UD-Q6_K_XL (GGUF)
kv:
F16
flash attention:
on
mtp (multi-token prediction):
off

User reports Qwen3-Coder-30B-A3B-Instruct at 69.2 t/s generation and 1242.4 t/s prompt processing on a Framework Desktop with AMD Ryzen AI Max+ 395 and Radeon 8060S, 64 GB unified memory. Setup is llama.cpp with UD-Q6_K_XL, f16 K/V cache, flash attention, one Vulkan device and 99 GPU layers. The two figures are separate depth-0 arms, PP512 and TG32, not one 512-input/32-output run.

Oct 13, 2026

Qwen3-Coder 30B (3B active)

AMD Radeon 8060S integrated GPU (Strix Halo, gfx1151) · 1bit engine lean ROCm / ROCmFPX (W4A4)

generation:
79.2 tokens/s
prompt processing (prefill):
2156.0 tokens/s
quant:
Q4_0-H32 W4A4 (GGUF)

User reports Qwen3-Coder 30B-A3B at 79.2 t/s generation and 2,156 t/s prompt processing on one Strix Halo integrated GPU. Setup is ROCmFPX/lean ROCm (HIP) with rotated Q4_0 W4A4 attention and experts, GGML_W4A4_TENSORS=all, no MTP or drafter identified. The figures are separate native llama-bench phases, -r 3, pp512 2,156 t/s and tg128 79.2 t/s, not a client request or TTFT; decode does not use W4A4.

Oct 13, 2026
Showing 1–20 of 27
Page 1 of 2

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3266 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other782

Records by engine

2308 total
llama.cpp1134
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480