How to compare local LLM benchmarks
A tokens-per-second number describes a particular run. Before using it to choose a model or buy a GPU, check what was running and how it was measured.
Start with the whole hardware setup
One GPU and four GPUs are different setups, even if the card name is the same. Check GPU count, total memory and system RAM. CPU offloading lets a model use memory outside the GPU, but changes its performance. Some reports describe mixed GPU models in their notes.
Separate generation from prompt processing
Generation speed measures output tokens. Prompt processing measures how quickly the input is processed. Neither number alone tells you how long a complete answer takes. A long prompt can mean a substantial wait before the first output token.
Check how many requests ran at once
Aggregate throughput adds up tokens across concurrent requests. It is useful for serving many users, but it is not the speed one person sees. If a report does not specify concurrency, do not assume its figure is a single conversation.
Match the model and settings
Compare the same model size and quantization. Context length, engine version, KV-cache settings and speculative decoding can also change the result. Missing settings mean the comparison is incomplete.
Distinguish an estimate from a measurement
The VRAM calculator estimates memory use and, where hardware data is available, speed. It also shows community measurements where available. A community report is evidence of one setup, not a guarantee for yours. Open the report and its source to check the conditions.