llamaperf
Sep 10, 2026
Throughput
6.0 t/s gen · 30.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Use cases

agentic

Summary

User reports about 30 t/s prompt processing and about 6 t/s generation on a Xeon, CPU only. The n-gram table is offloaded on 50% of threads. The user describes the code as sloppily vibecoded by Opus 5.0 and unreviewed, and the goal is overnight and over-week agentic jobs.