llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: AMD Gorgon Halo 192GB
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: negativeAMD hardware
generation:
9.0 tokens/s
prompt processing (prefill):
463.0 tokens/s
quant:
Q6_K_XL (GGUF)
kv:
q8_0
agenticlong-context

User benchmarks Qwen3.8 27B (dense) at 9 t/s generation and 463 t/s prompt reading on a MINISFORUM MS-S1 MAX P495 (Ryzen AI Max+ PRO 495, 192GB) at empty context. Setup is llama.cpp build 23f7aab with Vulkan, Q6_K_XL weights, q8_0 KV cache and flash attention, ubatch 512, all layers on the GPU. With MTP speculative decoding (--spec-type draft-mtp) the same model reaches 24 t/s with 82.9% draft acceptance. At 32k context it reads at 286 t/s and writes at 9 t/s; at 128k, 131 t/s and 8 t/s. The user could not reproduce an f16 KV collapse at depth and says the machine is not worth $8k.

Oct 18, 2026
Tone: positiveAMD hardware
generation:
103.0 tokens/s
coding

User reports DeepSeek V4.1 Flash at up to 103 tok/s writing code on a cluster of two Lucebox workstations, each with a Gorgon Halo 495 128GB plus an R9700 AI PRO 32GB. The two boxes are joined by a ConnectX-5 Ex card in each with a single 100 GbE cable and no switch, talking over RDMA (RoCE v2) and running as one model, with every token passing through every layer on both boxes. Each step sends about 95 small messages between the boxes. The user says this is 1.9 times faster than a single Lucebox Zero 495 with 192 GB running the same engine, and that the first two-box run was only a quarter faster than one box. A single 128 GB box cannot keep all of the model's experts in memory, while two boxes have room for it.

Oct 14, 2026
Tone: positiveAMD hardware
generation:
55.7 tokens/s
prompt processing (prefill):
1135.0 tokens/s
quant:
MXFP4 (MXFP4)
agenticcodinglong-context

User benchmarks gpt-oss 120B at 55.7 t/s generation and 1135 t/s prompt processing on a Minisforum MS-S1 MAX P495 (Ryzen AI Max+ PRO 495, Radeon 8065S, 192GB). Setup is llama.cpp build 23b0202a1 with Vulkan backend, MXFP4 quant, pp512 and tg128, all layers on GPU. At 32k context the model still reaches 42.3 t/s generation and 385 t/s prompt processing. Other models tested: Qwen3 Coder 30B A3B Q8_0 at 62.8 t/s, Qwen3.8 27B Q8_0 at 8.1 t/s, MiniMax M2.7 230B A10B UD IQ4_XS at 30.2 t/s, and DeepSeek V4 Flash 0731 284B A13B UD IQ3_S at 17.8 t/s. MiniMax crashed at 32k depth with default ubatch and required ubatch 256, which lowered prompt processing.

Oct 12, 2026
AMD hardware
generation:
17.8 tokens/s
prompt processing (prefill):
238.0 tokens/s
quant:
UD-IQ3_S
mtp (multi-token prediction):
off

User reports DeepSeek V4 Flash 284B-A13B at 17.8 t/s generation and 238 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with UD-IQ3_S, Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 238 t/s and 17.8 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output conversation; the same source reports roughly 32k-depth prompt processing at 106 t/s and generation at 13.2 t/s.

Oct 11, 2026
AMD hardware
generation:
30.2 tokens/s
prompt processing (prefill):
309.0 tokens/s
quant:
UD-IQ4_XS
mtp (multi-token prediction):
off

User reports MiniMax M2.7 230B-A10B at 30.2 t/s generation and 309 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with UD-IQ4_XS, Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 309 t/s and 30.2 t/s come from separate llama-bench pp512 and tg128 tests at empty prior context, not a paired 512-input/128-output run; the source also reports about 51 t/s prompt and 14.7 t/s generation at roughly 32k depth.

Oct 11, 2026
AMD hardware
generation:
8.1 tokens/s
prompt processing (prefill):
466.0 tokens/s
quant:
Q8_0
mtp (multi-token prediction):
off

User reports Qwen3.8 27B at 8.1 t/s generation and 466 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with Q8_0 and all layers on the GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 466 t/s and 8.1 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output run; the same source separately reports about 100 t/s prompt and 7.5 t/s generation at roughly 32k depth.

Oct 11, 2026
AMD hardware
generation:
62.8 tokens/s
prompt processing (prefill):
1692.0 tokens/s
quant:
Q8_0
mtp (multi-token prediction):
off

User reports Qwen3-Coder 30B-A3B at 62.8 t/s generation and 1692 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with Q8_0 and the Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 1692 t/s and 62.8 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output run; the same source reports about 205 t/s prompt and 33.1 t/s generation at roughly 32k depth.

Oct 11, 2026
AMD hardware
generation:
42.3 tokens/s
prompt processing (prefill):
385.0 tokens/s
quant:
MXFP4
mtp (multi-token prediction):
off

User reports gpt-oss 120B MXFP4 at 42.3 t/s generation and 385 t/s prompt processing on a Minisforum MS-S1 MAX-P495 with AMD Ryzen AI Max+ PRO 495 and 192 GB unified memory. Setup is llama.cpp build 23b0202a1 with the Vulkan backend and all layers on GPU, at a reported 32k test depth. Figures come from separate llama-bench pp512 and tg128 tests with two repetitions each, not a paired conversation; the source reports 59 GiB model size.

Oct 11, 2026
Showing 1–8 of 8
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3266 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other782

Records by engine

2308 total
llama.cpp1134
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480