llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: NVIDIA GTX 1080 Ti
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Llama 3.1 8B

NVIDIA GTX 1080 Ti · text-generation-webui

Tone: positiveNVIDIA hardware
generation:
1.0 tokens/s
rp

User recounts their AI tooling history, starting with Ollama, LM Studio, Msty and text-generation-webui on a mini PC with an i5 6400T and 24GB DDR4 using integrated graphics, where Llama 3.1 8B generated about 1 tok/s. They later moved to a Xeon E3-1270 V2 with 32GB DDR3 and a GTX 1080 Ti, then SillyTavern with KoboldCpp, Llamafile and finally llama.cpp. The 1 tok/s figure is from the earlier integrated-graphics mini PC, not the 1080 Ti; the user says they enjoyed it despite the limits and that roleplay started there.

Nov 17, 2026
Tone: mixedNVIDIA hardware
generation:
7-10 tokens/s
quant:
Q4_K_M (GGUF)
codingagentic

User asks which model to use as a coding builder agent on a GTX 1080 Ti 11GB with 32GB RAM, running llama.cpp with the Vulkan backend. They report Qwen3.6 35B A3B UD Q4_K_M at about 7 to 10 t/s on a few small tasks, and Qwen3.5-9B UD Q4_K_XL with MTP at about 25 to 30 t/s but not usable as a builder agent. They also tried Sharp Spark X2.5 4B Q6_K_XL, which overthinks, and Sharp Spark X2.6 4B Q6_K_XL, which is fine for simple tasks but not real work. They want 100k to 128k context and note CUDA is unstable on the 1080 Ti.

Nov 7, 2026
NVIDIA hardware
generation:
62.0 tokens/s
prompt processing (prefill):
1084.0 tokens/s
quant:
Q4_0 (GGUF)

User benchmarks Llama-2 7B at 62 t/s generation and 1084 t/s prompt processing on an NVIDIA GTX 1080 Ti 11GB. Setup is the CUDA build of llama.cpp with Q4_0 GGUF weights, run via llama-bench with -ngl 100. Figures are pp512 and tg128, a plain single-stream run with no speculative decoding.

Oct 20, 2026
Tone: positiveNVIDIA hardware
generation:
52.0 tokens/s
prompt processing (prefill):
340.9 tokens/s
quant:
Q6_K_XL (GGUF)

User reports Gemma 4 26B-A4B at 52.03 t/s generation and 340.86 t/s prompt processing on a dual-GPU setup of GTX 1080 Ti and Radeon MI50 16GB, totaling 27GB VRAM. Setup is llama.cpp Vulkan pre-built binary build 851cb34f2 (11055) with the UD-Q6_K_XL GGUF, flash attention on, and 99 GPU layers offloaded. Single-GPU GTX 1080 Ti runs of the same model reached 11.76 t/s generation and 162.12 t/s prompt processing. The user also benchmarked Qwen3.6-35B-A3B MXFP4 MoE, Nemotron 31B-A3.5B Q5_K_M, Qwen3.8 27B Q6_K, and medgemma 27B Q6_K_XL, noting dense models benefited most from the second GPU.

Sep 19, 2026
NVIDIA hardware
generation:
19.7 tokens/s
prompt processing (prefill):
332.6 tokens/s
quant:
MXFP4 (GGUF)

User benchmarks Qwen3.6 35B-A3B at 19.7 t/s generation and 332.57 t/s prompt processing on a mixed GTX 1080 Ti and Radeon VII Vulkan setup. Setup is llama.cpp Vulkan build b29c606e2 with the MXFP4 MoE GGUF and Flash Attention enabled, across two GPUs. User also reports results for twelve other models including llama 7B, bailingmoe2 16B-A1B, gpt-oss 20B, Gemma 4 26B-A4B, Qwen3.5 27B, Qwen3-Coder 30B-A3B, granite 4.0, and Phi-3.5-MoE.

Sep 18, 2026
Showing 1–5 of 5
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB348NVIDIA RTX 3090271NVIDIA RTX 5090197NVIDIA DGX Spark99NVIDIA RTX 3060 12GB96NVIDIA RTX 5060 Ti 16GB88NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB81NVIDIA RTX 5070 Ti77NVIDIA RTX 409068

Records by model

3266 total
Qwen3.81567
Qwen3.6334
Gemma 4185
DeepSeek V4 Flash150
Qwen3.5145
DeepSeek V4.1 Flash103
other782

Records by engine

2308 total
llama.cpp1134
Strata234
vLLM227
MLX107
Ollama99
other507

Use cases

coding 937agentic 640long-context 442tool-use 281vision 174summarization 81math 77creative-writing 58multilingual 35rp 22text-generation 9reasoning 3
coding937agentic640long-context442tool-use281vision174summarization81math77creative-writing58

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B819Qwen3.8 125B · 6B active676Qwen3.6 35B · 3B active242DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active92Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M395UD-Q4_K_XL189NVFP4174Q8_01194-bit106IQ4_XS86IQ3_XXS85IQ3_S84Q4_K_XL83Q480