llamaperf

Gufo benchmarks

An inference engine for running open-weight LLMs locally.

58 community reports

Gufo results by GPU

Hardware with reported Gufo results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Gufo, GPU not identified5 reports

This engine doesn't yet have an editorial profile on llamaperf. The community reports below show how it's been used in practice across different hardware.

Compare GPUs running Gufo

Every card is compared on one model, Qwen3.8 125B · 6B active at 4-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.

GPUVRAMReportsMedian speed, Qwen3.8 125B · 6B active 4-bit
AMD Strix Halo 128GBamd128GB58Gen 35.0 t/sPP 1794.3 t/s · n=4

Gufo against other engines

Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.

Model and GPUGufoOther engine
Gemma 4 31B 4-bitAMD Strix Halo 128GB11.5 t/sPrompt processing (prefill) 559.3 tokens/s (2)median of 2 generation reports, context not statedllama.cpp-Prompt processing (prefill) 724.0 tokens/s (1)no generation reports, context not stated
Qwen3.6 35B · 3B active 8-bitAMD Strix Halo 128GB50.8 t/sPrompt processing (prefill) 2384.0 tokens/s (1)one generation report, context not statedllama.cpp24.1 t/sPrompt processing (prefill) 280.3 tokens/s (2)one generation report, 512K context, 1 not stated
Qwen3.8 125B · 6B active 4-bitAMD Strix Halo 128GB35.0 t/sPrompt processing (prefill) 1794.3 tokens/s (4)median of 5 generation reports, 16K to 134K context, 1 not statedllama.cpp23.0 t/sPrompt processing (prefill) 251.9 tokens/s (2)median of 3 generation reports, 128K to 160K context, 1 not stated

Top models on Gufo

Frequently asked

Is Gufo faster than other engines?

llamaperf has 2 matched pairs of Gufo and other-engine runs on the same card, model size and bit class, but none is comparable enough to say which engine is faster: the sides have too few reports or were run at different context lengths. A pair counts toward a direction only when each side has at least 3 generation reports with a stated context, and the two sides' contexts overlap. The measured figures and their conditions are in the table on this page.