llamaperf for agents

Ask what runs on your hardware

The read server needs no account and no key. It answers from the same data and the same code as the website, so an agent and a reader see the same numbers.

claude mcp add --transport http llamaperf-catalog https://llamaperf.com/api/mcp/public

Any client that speaks MCP over HTTP can use the same address. Then ask something like:

• What is the fastest plain run reported on an RTX 3090 for a 30B model?
• Does a 64 GB M4 Max hold a 70B model at 32k context, and how fast?
• Show me llama.cpp reports for this card at Q4_K_M.

Tools

find_hardware

Looks up a GPU or Mac by name and returns the slug the other tools take.

find_models

Looks up a model family by name and returns its slug.

search_reports

Lists individual reports for a card, model, engine or quant, newest or fastest first. Each has its generation and prompt speed, its run kind and a link to the report page.

hardware_performance

For one card or Mac, every model size and quant reported on it, with the plain-run median, the speculative median beside it, and how many reports sit behind each.

check_fit

The VRAM calculator. For a card and a context length, which model sizes fit, at which quant, how much context, and a speed labelled measured or estimated.

What a number means

A median counts plain runs only: one device, one request at a time, no speculative decoding and the whole model in the device's memory. Runs that used speculation or kept part of the model in system RAM get their own figures beside it. A range in a source stays a range.

A calculator speed says whether it is measured, a community median at the same bit class, or estimated, from the card's memory bandwidth. A figure with one or two reports behind it is one person's run, so read the count next to it. These are community reports and not lab tests, and the about page explains how each one is read from its source and checked.

Measure your own setup

A single file with no dependencies times a local model server and prints the result. It works with llama-server, Ollama, vLLM, LM Studio and any other server that speaks the OpenAI chat API.

curl -O https://llamaperf.com/llamaperf-bench.mjs
node llamaperf-bench.mjs --endpoint http://localhost:8080/v1 --model my-model

It throws away a warmup run, measures five more, and reports time to first token, prompt speed and generation speed for each, with the median. Token counts come from the server's own usage report, so a server that sends none is refused instead of guessed. A failed run is listed and never counted. Nothing leaves your machine unless you add --submit.

Add it to llamaperf

The tool can send its result for you, or your coding agent can run a benchmark and submit it over MCP. Either way it goes through the same extraction and checks as a pasted report, and needs a free account for its key. You name your hardware yourself, because it is never guessed. Setup is on the agent submission page.

For crawlers and language models

/llms.txt describes the site in plain text. Every report has a stable address of the form /report/<id>.