llamaperf
Oct 13, 2026
Throughput
73.0 t/s gen · 961.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
MXFP4 (GGUF)
Flash Attention
on
MTP (Multi-Token Prediction)
off
System RAM
128 GB

Summary

Primary result: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/bench/gpt-oss-20b.md Pinned campaign README: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/README.md Model: gpt-oss-20b, native gpt-oss 20B MXFP4 MoE,20.91B parameters. Tensor-storage size11.27 GiB is not disk-file bytes or resident memory. Tested GGUF publisher, filename, revision and hash are unknown. Observed PP512 961.24 ±390.50 and TG128 73.01 ±0.10 tok/s. CRITICAL: author explicitly attributes the enormous PP variability to GPU contention from live services; preserve the PP as a noisy observed measurement, not a representative uncontended speed. TG is described by the author as more reliable, but still comes from the contended working machine. No confidence interval or independent repeats beyond the printed summary are supplied. Requested n_ctx512 for PP and128 for TG; effective allocation unknown. First-party bkpaine1 measurements dated 2026-07-23. Source repository https://github.com/bkpaine1/halo-llamacpp, capture revision 3617c72a325d03f73da0764d6b64d386e2db1a9c. Historical qualification of previously deferred evidence; no new measurement or source-change claim. Hardware: source README identifies AMD Ryzen AI Max+ 395, Radeon 8060S gfx1151 and 128 GB unified memory. Its explicit statement that every run enumerates one ROCm device, with Device 0 AMD Radeon Graphics/gfx1151, binds GPU quantity 1 to this native campaign. This is a common campaign-level device snippet, not a repeated device block in each result file. Vendor SKU specification confirms 16 CPU cores, 32 hardware threads and 40 graphics compute units: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Actual benchmark thread count is not reported. Host label AI-Strix; manufacturer, machine model, distribution, kernel, memory speed and tested power setting unknown. The enumerated 65536 MiB VRAM is driver-reported GPU-addressable capacity in the unified pool, not 64 GiB additional physical memory or measured residency. Model/KV/process resident memory and exact disk bytes unknown. Runtime: llama.cpp commit 1425386fd996511e1f3295e7366c38289a92a271, ROCm/HIP 7.2.4 (hipconfig 7.2.53211), ROCm clang-22. Source common command: ROCBLAS_USE_HIPBLASLT=1 llama-bench -m <model> -ngl 999 -fa 1; build enables HIP, gfx1151, rocWMMA FATTN and HIP_NO_VMM. The raw tables report build 1425386 (1), ROCm, ngl999 and fa1. Build number 1 is reported literally, not a modern release tag. No speculative drafter in these native runs. Live services shared this working box, so no isolated hardware or backend causal comparison is established. Exact historical timer source: https://github.com/ggml-org/llama.cpp/blob/1425386fd996511e1f3295e7366c38289a92a271/tools/llama-bench/llama-bench.cpp . PP512 runs 512 synthetic random-token decode work units, then synchronizes. TG128 runs 128 synchronized single-token decode work units; these are not client-visible generated text tokens. Rates are per-trial (n_prompt+n_gen)/elapsed, then arithmetic mean and sample SD where supplied. Depth is prepared/restored before timing and excluded from both rate numerator and denominator. TTFT, client latency, request wall time and model-load time are not measured. No rate-derived TTFT proxy is supplied. The exact source defaults are batch2048, ubatch512, f16 K/V, five repetitions and warmup enabled; full per-run command lines are absent, so defaults are labeled source defaults rather than independently observed settings. bench.sh uses --no-mmap while the common README command omits it; load mode is left unknown rather than grafted from that recipe. Effective allocated context capacity is not reported. HF link https://huggingface.co/openai/gpt-oss-20b is a canonical parent-model/architecture reference ONLY, verified public metadata revision 6cee5e81ee83917806bbde320786a8fb61efebee. It is not the tested GGUF or evidence of that quant publisher; no reference-model bytes are claimed to match the benchmark. No weights downloaded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source rates and standard deviations are retained above.