llamaperf
Oct 13, 2026
Throughput
85.1 t/s gen · 1917.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
Flash Attention
on
MTP (Multi-Token Prediction)
off
System RAM
128 GB

Summary

Primary result: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/bench/ollama-headtohead-llama3.2-3b.md Pinned campaign README: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/README.md Model: source llama3.2:3b Q4_K_M. Exact tested Ollama GGUF blob SHA256 supplied by source: dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff. Public current metadata https://ollama.com/library/llama3.2:3b/blobs/dde5aa3fc5ff verifies the matching short blob prefix, llama architecture and Q4_K_M. We did not download/re-hash weights; full hash remains author-reported. The same source reports direct llama-bench PP5121917.92/TG12885.07 tok/s; this is a first-party textual native summary, not the complete raw table, and no SD/per-trial output is preserved. Requested n_ctx512/128 under the documented d0/default convention; effective allocation unknown. Source also gives Ollama prompt-eval880.53 on38 actual prompt tokens and eval82.08, but Ollama version, output count and comparable timing/workload are missing. Those companion values are excluded from structured native metrics; do not call the comparison apples-to-apples or claim a causal3.6% runtime gain. No PP/TG cross-pairing. First-party bkpaine1 measurements dated 2026-07-23. Source repository https://github.com/bkpaine1/halo-llamacpp, capture revision 3617c72a325d03f73da0764d6b64d386e2db1a9c. Historical qualification of previously deferred evidence; no new measurement or source-change claim. Hardware: source README identifies AMD Ryzen AI Max+ 395, Radeon 8060S gfx1151 and 128 GB unified memory. Its explicit statement that every run enumerates one ROCm device, with Device 0 AMD Radeon Graphics/gfx1151, binds GPU quantity 1 to this native campaign. This is a common campaign-level device snippet, not a repeated device block in each result file. Vendor SKU specification confirms 16 CPU cores, 32 hardware threads and 40 graphics compute units: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Actual benchmark thread count is not reported. Host label AI-Strix; manufacturer, machine model, distribution, kernel, memory speed and tested power setting unknown. The enumerated 65536 MiB VRAM is driver-reported GPU-addressable capacity in the unified pool, not 64 GiB additional physical memory or measured residency. Model/KV/process resident memory and exact disk bytes unknown. Runtime: llama.cpp commit 1425386fd996511e1f3295e7366c38289a92a271, ROCm/HIP 7.2.4 (hipconfig 7.2.53211), ROCm clang-22. Source common command: ROCBLAS_USE_HIPBLASLT=1 llama-bench -m <model> -ngl 999 -fa 1; build enables HIP, gfx1151, rocWMMA FATTN and HIP_NO_VMM. The raw tables report build 1425386 (1), ROCm, ngl999 and fa1. Build number 1 is reported literally, not a modern release tag. No speculative drafter in these native runs. Live services shared this working box, so no isolated hardware or backend causal comparison is established. Exact historical timer source: https://github.com/ggml-org/llama.cpp/blob/1425386fd996511e1f3295e7366c38289a92a271/tools/llama-bench/llama-bench.cpp . PP512 runs 512 synthetic random-token decode work units, then synchronizes. TG128 runs 128 synchronized single-token decode work units; these are not client-visible generated text tokens. Rates are per-trial (n_prompt+n_gen)/elapsed, then arithmetic mean and sample SD where supplied. Depth is prepared/restored before timing and excluded from both rate numerator and denominator. TTFT, client latency, request wall time and model-load time are not measured. No rate-derived TTFT proxy is supplied. The exact source defaults are batch2048, ubatch512, f16 K/V, five repetitions and warmup enabled; full per-run command lines are absent, so defaults are labeled source defaults rather than independently observed settings. bench.sh uses --no-mmap while the common README command omits it; load mode is left unknown rather than grafted from that recipe. Effective allocated context capacity is not reported. HF link https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct is a canonical parent-model/architecture reference ONLY, verified public metadata revision 0cb88a4f764b7a12671c53f0838cd831a0843b95. It is not the tested GGUF or evidence of that quant publisher; no reference-model bytes are claimed to match the benchmark. No weights downloaded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source rates and standard deviations are retained above.