About llamaperf
llamaperf collects reports of how fast open-weight models run on real hardware, so you can see what a card or Mac does before you buy it or download a model for it.
Where the reports come from
Most reports start as posts on r/LocalLLaMA. We read the opening post of a thread and never the replies, because a reply's speed attached to the poster's hardware would make a wrong record.
Others come from benchmark write-ups people publish on GitHub, Hugging Face and hardware forums, and from people who submit their own numbers on the submit page. A report that came from a public post links back to it, so you can read the author's own words.
How a post becomes a record
A language model reads each post and fills in the hardware, model, quant, engine and speeds it states. A report is published on its own only when it names a model or a GPU and gives a speed or a rating. Anything else waits for a person to review it.
Some rules are enforced in code, because asking the model nicely wasn't enough. A range such as "30-40 tok/s" stays a range and is never averaged into a figure nobody measured. Each speed records whether it came from a plain run or used speculative decoding, several machines or system RAM for part of the model, and every median on the site counts plain single-device runs only. Offloaded runs get their own figures, never mixed in. A CPU-only run is filed with no GPU.
How we check it
We audit the whole dataset regularly. A plain run faster than the hardware's memory allows fails the audit, and so does a run filed as fitting on a device too small to hold the model, or a record whose source says something different. The original post text is kept, so every correction is checked against what the author wrote.
The calculator labels every speed as measured, when it comes from community reports, or estimated, when it comes from the card's memory bandwidth, and shows the inputs behind each estimate.
How the site is paid for
llamaperf is free to use and carries sponsored placements. Every one is labelled Sponsored, and sponsors have no say in which hardware or models rank where. The rankings are computed from reports alone. The privacy page lists what we collect and who else handles it.
Corrections
If a record gets your post wrong, or you know a figure is off, tell us through the contact form. We'll check it against the source and fix it.