llamaperf
Oct 13, 2026
Throughput
18.6 t/s gen · 140.3 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
MTP (Multi-Token Prediction)
off
System RAM
128 GB

Summary

Primary raw source: https://github.com/ThinkOffApp/StrixLink/blob/5bc93e6c5ad136bc72f822be6c3ec95e78e67de8/docs/measurements-2026-09-10-raw.md Campaign description: https://github.com/ThinkOffApp/StrixLink/blob/5bc93e6c5ad136bc72f822be6c3ec95e78e67de8/README.md Author: ThinkOffApp. Source snapshot 5bc93e6c5ad136bc72f822be6c3ec95e78e67de8. Historical September 10–11, 2026 campaign, recovered October 13. Source names the September 10 rerun; raw log crosses UTC midnight. Native tables captured, not independently reproduced. Selected result: DeepSeek V4 Flash IQ3_XXS, SOLO Bosgame M5. PP512 140.30 +/- 4.35 t/s and TG128 18.56 +/- 0.01 t/s, 3 repeats, arithmetic mean/sample SD according to exact native runtime. These are separate synthetic tests, not a512-input/128-output client request; no RPC split or speculative-server rate is imported. Hardware: Bosgame M5, AMD Ryzen AI Max+395 / Radeon8060S, nominal128GB unified RAM, Ubuntu26.04 as stated for the rig. CPU/GPU physical core counts, native thread count, exact September kernel/driver versions and active power limit unknown. GPU count 1 is source-attributed at same-host/campaign scope: raw table labels 'M5 alone, Vulkan ngl99', native backend Vulkan with noRPC; the same file's later00:37:55Z DeepSeek section says the Strix-alone row was already recorded and skipped, followed by bench-tail output 'Found 1 Vulkan devices' and device0 Radeon8060S RADV. This is explicit same-host/campaign enumeration, not a distinct per-selected-process count log. Source also describes two machines' GPUs(Metal+Vulkan). No count inferred merely from one device name. Memory configuration: September nominal1GB BIOS carve and120GB GTT addressable region; these are configuration limits, not measured model-resident memory. Native tensor size 97.05GiB; not diskbytes or residency. The raw 'gtt after' value is post-benchmark and is not a peak/resident model measurement. Native parameter count 284.33B is rounded. Model/artifact: Direct IO (dio) explicitly shown in both native rows. Four shards stated; filenames and exact disk bytes unpublished. Tested GGUF revision, SHA256 and quant publisher unknown. HF link below is only the verified canonical base-model card, NOT the measured quant artifact. It cannot establish historical tested bytes. HF reference only: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/60d8d70770c6776ff598c94bb586a859a38244f1/README.md Runtime: campaign README explicitly binds every September10 row to llama.cpp434ddbb, Vulkan+RPC build on Strix but selected execution Vulkan only; the later split row also has build434ddbb(345). Per-selected-row binary hashes/footers unavailable. Exact434ddbb llama-bench source inspected, no source software run. Core native timer excludes model loading/context creation; times synchronized batch prefill or sequential single-token decode; numerator n_prompt+n_gen. PP512 andTG128 labels contain no @d suffix, consistent with depth0 in exact formatter. Full commands unavailable, so batch/KV/FA/warmup/configuration values are unknown rather than silently inherited from defaults. Model-loaded performance does not include cold-load or RPC upload. Measured TTFT unknown; no reciprocal rate or other proxy supplied. Requested context would be512/128 for separate zero-depth arms in this exact timer; effective allocated capacity not logged. No quality/correctness/generalization claim based on the separate sky-blue smoke. Timer reference: https://github.com/ggml-org/llama.cpp/blob/434ddbb/tools/llama-bench/llama-bench.cpp Caveats: source narrative gives a same-campaign runtime rather than process-local footer; soleGPU proof is same-campaign rather than perprocess. Different Mac-only and RPC arms, August superseded runs, live-server/speculative rates, transport throughput and load durations are excluded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.