llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

Join the community

Follow along:XLinkedIn

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

Model: Tencent-HY4
Compare setup details

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Tone: mixedNVIDIA hardware
prompt processing (prefill):
3450.0 tokens/s
quant:
NVFP4 (NVFP4)
visionlong-contextmath

User reports Tencent Hy4 Flash running in Basalt 0.5.0 on an RTX 5090 plus RTX 5060 Ti, at about 84 t/s on prose, 172 t/s on structured output and around 3,450 t/s prefill. Setup is Basalt with an NVFP4 build, a single .basalt file needing roughly 95 to 100 GB of memory total, with more of it in VRAM being better. Speculative decoding uses DFly2. User says throughput is lower than Flash Next because Hy4 Flash has almost twice the weights per token at NVFP4, with more cache misses that are expensive; they expect maybe 10 or 20% more performance to unlock. Quantization hits it hard: the official Q4_K_M measured just below 87% same top 1 against BF16 and the NVFP4 build about 86%, versus above 91% for Flash Next NVFP4. Math felt better on Hy4, and 1M context and dynamic vision both work.

Nov 19, 2026

Tencent-HY4 Flash

NVIDIA RTX 3060 12GB · RustingStrata · 20,000 ctx

NVIDIA hardware
generation:
3.3 tokens/s
prompt processing (prefill):
11.6 tokens/s
quant:
IQ2_S/IQ2_XXS (GGUF)

User reports Tencent Hy4-flash at 3.3 t/s generation after reading a 20,000-token prompt at 11.6 t/s on an RTX 3060 12GB with 32 GB RAM. Setup is RustingStrata, the user's own Rust GGUF engine, with an IQ2_S/IQ2_XXS 85 GB GGUF in two files; the model is far larger than the card and the RAM, so weights ran from system memory and NVMe. The 20,000-token prompt took 29 minutes to read. After a short prompt the model writes 4.3 to 5.1 t/s on CUDA. llama.cpp comparison was not possible because Hy4-flash support there is still a draft pull request.

Nov 19, 2026
Tone: positiveApple hardware
generation:
26.0 tokens/s
quant:
oQh8-BF16

User reports Tencent Hy4-flash running at about 26 tokens per second on an M5 Ultra with 256 GB of memory. Setup is a preliminary oMLX pull request, run from the command line, using an 8-bit oQh8-BF16 conversion of the model that needs about 175 GB of memory free. Long prompts work but get very slow, and the image part is not connected yet, so it is text only. A 128GB version is planned.

Nov 19, 2026
Showing 1–3 of 3
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB347NVIDIA RTX 3090264NVIDIA RTX 5090197NVIDIA DGX Spark98NVIDIA RTX 3060 12GB93NVIDIA RTX 5060 Ti 16GB86NVIDIA RTX Pro 6000 Blackwell81AMD Radeon AI PRO R9700 32GB77NVIDIA RTX 5070 Ti76M5 Max 48GB67

Records by model

3214 total
Qwen3.81554
Qwen3.6333
Gemma 4182
DeepSeek V4 Flash150
Qwen3.5143
DeepSeek V4.1 Flash102
other750

Records by engine

2278 total
llama.cpp1120
Strata231
vLLM221
MLX107
Ollama97
other502

Use cases

coding 923agentic 624long-context 428tool-use 269vision 166summarization 78math 75creative-writing 57multilingual 34rp 20text-generation 9reasoning 3
coding923agentic624long-context428tool-use269vision166summarization78math75creative-writing57

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509080RTX Pro 600067RTX 5070 Ti55M5 Ultra 256GB50RTX 409040RX 7900 XTX40RTX 309038RX 7800 XT 16GB35RTX 5090 Laptop 24GB34RX 7900 XT32

Prompt processing, same model

M5 Ultra 256GB1800RTX 40901949RX 7900 XTX640RTX 3090561RX 7900 XT800M3 Ultra 512GB372V100 32GB941Radeon AI PRO R9700 32GB1287RTX 3080 20GB997Arc Pro B70512

Reports by model size

Qwen3.8 27B815Qwen3.8 125B · 6B active667Qwen3.6 35B · 3B active241DeepSeek V4 Flash 284B · 13B active133DeepSeek V4.1 Flash 552B · 16B active91Qwen3.6 27B81Gemma 4 26B · 4B active61GLM-5.3 320B · 18B active57

Quants

Q4_K_M392UD-Q4_K_XL185NVFP4170Q8_01194-bit106IQ4_XS85IQ3_S84Q4_K_XL83IQ3_XXS82Q480