Oct 13, 2026
Summary
Source-reported historical ZINC project campaign, target generated_at 2026-09-13T03:37:48.775Z; recovered 2026-10-13, not locally rerun. This report is the llama.cpp arm. Pinned primary data: https://github.com/zolotukhin/zinc/blob/29bc350ac4cbf9f110ec628b9e177ea04ac816fb/site/src/data/zinc-performance.json (targets[id=metal], models[id=gemma4-31b-q4k-m]). Data publication commit is 29bc350ac4cbf9f110ec628b9e177ea04ac816fb; main and pinned JSON were byte-identical at recovery. Site: https://zolotukhin.ai/zinc/benchmarks/#metal . The September 25 top-level artifact date covers multiple targets; it is not this Mac campaign's measurement date.
Hardware: one Apple Mac Studio, Mac16,9, M4 Max, 40-core GPU, 64 GB installed unified memory, Darwin 24.6.0. Forty GPU cores and Mac Studio identity are explicitly stated on the project's hardware/landing page: https://zolotukhin.ai/zinc/ . CPU core count, power mode, temperature/clocks, swap, other-process activity, and storage are not recorded. Unified RAM is not dedicated VRAM. No measured memory usage or model peak is published; leave VRAM/RSS fields blank.
Model identity: source model ID gemma4-31b-q4k-m; GGUF gemma-4-31B-it-Q4_K_M.gguf from https://huggingface.co/unsloth/gemma-4-31B-it-GGUF. Pinned managed catalog: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/src/model/catalog.zig . The harness resolves this ID to the managed cache's model.gguf and passes the same file to the baseline. This binds the intended artifact by code/catalog; raw on-run path/hash logs and measured HF repository revision are absent. Current or historical HF metadata verification: https://huggingface.co/unsloth/gemma-4-31B-it-GGUF/blob/main/gemma-4-31B-it-Q4_K_M.gguf .
The pinned catalog has no checksum. Independently observed current HF SHA256=38bd64c852c4b460434cc7162fa9bdcf242faf86502581a754cb72956bb17f84; historical loaded bytes are not hash-verified. Nominal size labels are rounded source labels, not exact total/active parameter counts.
Runtime provenance: llama-server 0.4.0-dev, reported commit 790cf51aa, Metal; runtime binary checksum unknown. Paired harness: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/tools/performance_suite.mjs at ZINC 68a03ec6c6415efaeabbac7d4dec83ffdaea9608. It prefers one resident local llama-server per model for all scenarios, and can fall back to llama-cli on server-launch failure. The stored binary name and nonempty completion previews support the server path here; a per-run full command/startup log is not retained. Server method specifies -ngl 999, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, --flash-attn on, --no-cache-prompt. These settings are code-derived protocol evidence, not individually logged runtime readbacks; context and FA form fields remain blank rather than claiming independently verified allocations. GPU offload request 999 is not measured layer count. CPU threads and KV precision unknown.
One discarded warmup plus three measured nonstreaming requests per scenario. /v1/chat/completions uses the runtime chat template, temperature 0, source max_tokens. PP/TG are engine-native response timings.prompt_per_second and predicted_per_second, not client input/TTFT or wall generation throughput. Native prompt_ms/prompt_n are preferred over usage when present. The harness reconstructs decode_ms as completion_tokens/TG*1000; it is not a direct raw decode duration. Do not equate its token-count convention with ZINC's N-over-N-minus-one-loop timing. JSON marks speculation disabled; no draft/MTP flags are in this launch. No first-token timestamp is captured.
Headline is Quick Chat/core: recorded prompt/prefill count 52/52, output 96; PP=62.710139, TG=22.422308 tokens/s. Counters are the first measured row saved by createStats, not a per-run count distribution; all three rate samples are retained. These are short actual inputs, not filled 4K or long-context benchmarks. Other scenario labels describe task style; Incident Context reaches only a few hundred input tokens.
All scenario rates are separate medians[min,max], tokens/s, n=3; ms where labeled. Displayed to 8 significant digits; pinned JSON retains full precision.
Quick Chat (core): input/prefill/output=52/52/96; cap=96; PP=62.710139[61.559078,63.866921]; TG=22.422308[21.949139,22.436373]; PPms=829.212[814.193,844.717]; wrapper wall ms=5066.3128[5053.1146,5194.592]; wrapper E2E=29.212566[28.491169,29.288867]; paired engine input=49.
Coding Review (context-medium): input/prefill/output=195/195/160; cap=160; PP=138.60093[136.99958,138.84716]; TG=21.656018[21.470694,21.975446]; PPms=1406.917[1404.422,1423.362]; wrapper wall ms=8767.8085[8643.5612,8812.4465]; wrapper E2E=40.489023[40.283932,41.071035]; paired engine input=192.
Incident Context (context-long): input/prefill/output=345/345/128; cap=128; PP=158.63185[158.0124,159.09578]; TG=21.716882[21.4377,21.983295]; PPms=2174.847[2168.505,2183.373]; wrapper wall ms=8018.295[7953.7387,8109.2845]; wrapper E2E=58.990097[58.328204,59.468888]; paired engine input=346.
Long Coding Draft (decode-extended): input/prefill/output=73/73/256; cap=256; PP=90.175781[90.175335,90.186922]; TG=22.786656[22.63885,22.841738]; PPms=809.53[809.43,809.534]; wrapper wall ms=12001.456[11974.577,12098.31]; wrapper E2E=27.41334[27.19388,27.474875]; paired engine input=70.
Wrapper wall/E2E above are a separate measurement. For ZINC they enclose process startup, model loading, inference and termination; for resident llama-server they enclose request/response and exclude server startup. E2E=(prompt+output)/wrapper wall. The site's 'overall' uses inferred phase durations from median rates, excluding overhead, and is neither the saved wrapper E2E nor TTFT. Never put wall-derived or overall rates into native PP/TG fields.
Cross-engine comparison caveat: all 20 Mac model/scenario pairs in the pinned JSON have different prompt counts. The rendered dashboard instead says 4/20 same; retain this source inconsistency and use the engine-specific counters above. Templates/tokenization and Qwen thinking output can differ; no matched-token speedup or equivalent-quality claim follows. Source preview checks are only a simple repetition/control-token screen, not independent task scoring. ZINC's Gemma 31B Coding Review and Incident Context previews are flagged repetitive; those rates remain labeled and are excluded from any clean-quality conclusion. Exact thermal/power conditions, repeated-run count variation, benchmark command environment and independent correctness validation remain unknown.