Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
User reports GLM-4.5-Air at about 138 tok/s on an RTX PRO 6000 96GB. Setup is llama.cpp with Q4_K_M, measured at empty context. The figure comes from a site of 941 measured rows, including 540 llama-bench rows the user ran on rented GPUs from an RTX 3060 12GB through an RTX 5090 and an RTX PRO 6000 96GB.
4× NVIDIA RTX Pro 6000 Blackwell · SGLang · 128,000 ctx
User reports GLM-4.5-Air FP8 at 133 tokens/sec with SGLang on an RTX 6000 Pro Blackwell, compared with about 78 tokens/sec in vLLM. Setup is SGLang with FP8 weights, fp8_e5m2 KV cache, 128000 context length and tensor parallelism across 4 GPUs. The user asks how to reach the same 133 tokens/sec in vLLM with Triton, and whether the Cutlass design is slower than Triton.
NVIDIA RTX Pro 6000 Blackwell · ik_llama.cpp · 16,384 ctx
User benchmarks GLM-4.5-Air with Unsloth UD-Q3_K_XL quant on an RTX PRO 6000 96GB with CPU offload for some experts, using ik_llama.cpp at 16k context with q8_0 KV cache. Setup is ik_llama.cpp with UD-Q3_K_XL and q8_0 KV cache, offloading some expert layers to a Ryzen 9 7950X with 128GB DDR5-3600. At 12288 tokens of context, prompt processing reaches 444.23 t/s and generation 8.25 t/s. The user also compares mainline llama.cpp and an Ubergarm IQ3_KT quant, noting a batch warm-up quirk in the forked llama-sweep-bench.
NVIDIA RTX 3090 · llama.cpp
User recommends enabling MTP in llama.cpp for GLM-4.5-Air, a 106B MoE with 12B active params, on 3090s. User notes MTP also works for full GLM-4.5. User mentions creative writing and RP finetunes and Intellect 3.x.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking