llamaperf
Submit

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Buying hardware? Compare GPUs →

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

Model: RWKV7
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

reported speed:
48.7 tokens/s generation · 4315.7 tokens/s prompt processing
quant:
F16 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports RWKV7 2.9B at 48.71 t/s generation and 4315.67 t/s prompt processing on an AMD Radeon Pro W7900. Setup is llama.cpp with F16 weights and ROCm backend, 99 GPU layers, 512-token prompt and 128-token generation. Q8_0 on ROCm reaches 58.59 t/s generation and 4033.24 t/s prompt; Vulkan backend gives 39.49 t/s (F16) and 45.21 t/s (Q8_0) generation.

Oct 8, 2026
Showing 1–1 of 1
Page 1 of 1

Community benchmarks snapshot

Records by GPU

NVIDIA RTX 3090167NVIDIA RTX 5090115AMD Strix Halo 128GB84NVIDIA DGX Spark65NVIDIA RTX 5060 Ti 16GB54NVIDIA RTX Pro 6000 Blackwell51NVIDIA RTX 3060 12GB47AMD Radeon AI PRO R9700 32GB44NVIDIA RTX 409037NVIDIA RTX 5070 Ti31

Records by model

1439 total
Qwen3.8793
Qwen3.6174
DeepSeek V4 Flash127
Gemma 463
Qwen3.535
Qwen325
other222

Records by engine

1109 total
llama.cpp581
vLLM161
Strata48
NInfer41
Ollama35
other243

Use cases

coding 467agentic 296long-context 219tool-use 125vision 89summarization 47math 37creative-writing 31multilingual 19text-generation 9rp 6reasoning 3
coding467agentic296long-context219tool-use125vision89summarization47math37creative-writing31

Median t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509093RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX41RTX 309036V100 32GB33RTX 409032RX 7800 XT 16GB30RTX 5090 Laptop 24GB30Radeon AI PRO R9700 32GB29

Reports by model size

Qwen3.8 27B476Qwen3.8 125B · 6B active287Qwen3.6 35B · 3B active103DeepSeek V4 Flash 284B · 13B active102Qwen3.6 27B69Gemma 4 26B · 4B active26DeepSeek V4.1 Flash 552B · 16B active23GLM-5.3 320B · 18B active18

Quants

Q4_K_M124NVFP499IQ4_XS60Q4_K_XL58UD-Q4_K_XL50IQ3_XXS38Q437Q8_0354-bit27MXFP425