Local model performance on your hardware

generation:

23.9 tokens/s

prompt processing (prefill):

119.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical ZINC project campaign, target generated_at 2026-09-13T03:37:48.775Z; recovered 2026-10-13, not locally rerun. This report is the llama.cpp arm. Pinned primary data: https://github.com/zolotukhin/zinc/blob/29bc350ac4cbf9f110ec628b9e177ea04ac816fb/site/src/data/zinc-performance.json (targets[id=metal], models[id=qwen38-27b-q4k-m]). Data publication commit is 29bc350ac4cbf9f110ec628b9e177ea04ac816fb; main and pinned JSON were byte-identical at recovery. Site: https://zolotukhin.ai/zinc/benchmarks/#metal . The September 25 top-level artifact date covers multiple targets; it is not this Mac campaign's measurement date. Hardware: one Apple Mac Studio, Mac16,9, M4 Max, 40-core GPU, 64 GB installed unified memory, Darwin 24.6.0. Forty GPU cores and Mac Studio identity are explicitly stated on the project's hardware/landing page: https://zolotukhin.ai/zinc/ . CPU core count, power mode, temperature/clocks, swap, other-process activity, and storage are not recorded. Unified RAM is not dedicated VRAM. No measured memory usage or model peak is published; leave VRAM/RSS fields blank. Model identity: source model ID qwen38-27b-q4k-m; GGUF Qwen3.8-27B-Q4_K_M.gguf from https://huggingface.co/unsloth/Qwen3.8-27B-GGUF. Pinned managed catalog: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/src/model/catalog.zig . The harness resolves this ID to the managed cache's model.gguf and passes the same file to the baseline. This binds the intended artifact by code/catalog; raw on-run path/hash logs and measured HF repository revision are absent. Current or historical HF metadata verification: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/blob/4121cb19390a7984a0e6dc0f46bea9177b846f15/Qwen3.8-27B-Q4_K_M.gguf . Catalog intended SHA256=7e78da5d7e3ae28d178121f58646953305f3e5bd3cb46f4a75584e8b6c6fe169; independently matched HF metadata. That file was deleted from HF main on August 19; the historical revision linked above still exposes it. The tested HF revision is unknown; do not substitute today's differently named UD-Q4_K_M file. Runtime provenance: llama-server 0.4.0-dev, reported commit 790cf51aa, Metal; runtime binary checksum unknown. Paired harness: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/tools/performance_suite.mjs at ZINC 68a03ec6c6415efaeabbac7d4dec83ffdaea9608. It prefers one resident local llama-server per model for all scenarios, and can fall back to llama-cli on server-launch failure. The stored binary name and nonempty completion previews support the server path here; a per-run full command/startup log is not retained. Server method specifies -ngl 999, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, --flash-attn on, --no-cache-prompt. These settings are code-derived protocol evidence, not individually logged runtime readbacks; context and FA form fields remain blank rather than claiming independently verified allocations. GPU offload request 999 is not measured layer count. CPU threads and KV precision unknown. One discarded warmup plus three measured nonstreaming requests per scenario. /v1/chat/completions uses the runtime chat template, temperature 0, source max_tokens. PP/TG are engine-native response timings.prompt_per_second and predicted_per_second, not client input/TTFT or wall generation throughput. Native prompt_ms/prompt_n are preferred over usage when present. The harness reconstructs decode_ms as completion_tokens/TG*1000; it is not a direct raw decode duration. Do not equate its token-count convention with ZINC's N-over-N-minus-one-loop timing. JSON marks speculation disabled; no draft/MTP flags are in this launch. No first-token timestamp is captured. Headline is Quick Chat/core: recorded prompt/prefill count 87/87, output 96; PP=119.88656, TG=23.88856 tokens/s. Counters are the first measured row saved by createStats, not a per-run count distribution; all three rate samples are retained. These are short actual inputs, not filled 4K or long-context benchmarks. Other scenario labels describe task style; Incident Context reaches only a few hundred input tokens. All scenario rates are separate medians[min,max], tokens/s, n=3; ms where labeled. Displayed to 8 significant digits; pinned JSON retains full precision. Quick Chat (core): input/prefill/output=87/87/96; cap=96; PP=119.88656[119.30932,120.06624]; TG=23.88856[23.866115,23.94904]; PPms=725.686[724.6,729.197]; wrapper wall ms=4702.6213[4693.6425,4743.3275]; wrapper E2E=38.914467[38.580511,38.988909]; paired engine input=48. Coding Review (context-medium): input/prefill/output=212/212/160; cap=160; PP=170.2006[170.17956,170.2959]; TG=23.738075[23.351272,23.869401]; PPms=1245.589[1244.892,1245.743]; wrapper wall ms=7945.3056[7908.2199,8055.2348]; wrapper E2E=46.8201[46.181149,47.039663]; paired engine input=193. Incident Context (context-long): input/prefill/output=378/378/128; cap=128; PP=196.85675[196.8342,197.60635]; TG=22.782255[22.780968,23.159377]; PPms=1920.178[1912.894,1920.398]; wrapper wall ms=7489.3004[7405.5469,7499.1648]; wrapper E2E=67.563053[67.47418,68.327161]; paired engine input=345. Long Coding Draft (decode-extended): input/prefill/output=109/109/256; cap=256; PP=148.71491[143.54533,149.01235]; TG=23.763902[22.944596,23.876832]; PPms=732.946[731.483,759.342]; wrapper wall ms=11496.663[11412.688,11874.348]; wrapper E2E=31.748342[30.738529,31.981949]; paired engine input=70. Wrapper wall/E2E above are a separate measurement. For ZINC they enclose process startup, model loading, inference and termination; for resident llama-server they enclose request/response and exclude server startup. E2E=(prompt+output)/wrapper wall. The site's 'overall' uses inferred phase durations from median rates, excluding overhead, and is neither the saved wrapper E2E nor TTFT. Never put wall-derived or overall rates into native PP/TG fields. Cross-engine comparison caveat: all 20 Mac model/scenario pairs in the pinned JSON have different prompt counts. The rendered dashboard instead says 4/20 same; retain this source inconsistency and use the engine-specific counters above. Templates/tokenization and Qwen thinking output can differ; no matched-token speedup or equivalent-quality claim follows. Source preview checks are only a simple repetition/control-token screen, not independent task scoring. Exact thermal/power conditions, repeated-run count variation, benchmark command environment and independent correctness validation remain unknown.

generation:

76.4 tokens/s

prompt processing (prefill):

319.9 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical ZINC project campaign, target generated_at 2026-09-13T03:37:48.775Z; recovered 2026-10-13, not locally rerun. This report is the llama.cpp arm. Pinned primary data: https://github.com/zolotukhin/zinc/blob/29bc350ac4cbf9f110ec628b9e177ea04ac816fb/site/src/data/zinc-performance.json (targets[id=metal], models[id=qwen36-35b-a3b-q4k-xl]). Data publication commit is 29bc350ac4cbf9f110ec628b9e177ea04ac816fb; main and pinned JSON were byte-identical at recovery. Site: https://zolotukhin.ai/zinc/benchmarks/#metal . The September 25 top-level artifact date covers multiple targets; it is not this Mac campaign's measurement date. Hardware: one Apple Mac Studio, Mac16,9, M4 Max, 40-core GPU, 64 GB installed unified memory, Darwin 24.6.0. Forty GPU cores and Mac Studio identity are explicitly stated on the project's hardware/landing page: https://zolotukhin.ai/zinc/ . CPU core count, power mode, temperature/clocks, swap, other-process activity, and storage are not recorded. Unified RAM is not dedicated VRAM. No measured memory usage or model peak is published; leave VRAM/RSS fields blank. Model identity: source model ID qwen36-35b-a3b-q4k-xl; GGUF Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf from https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF. Pinned managed catalog: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/src/model/catalog.zig . The harness resolves this ID to the managed cache's model.gguf and passes the same file to the baseline. This binds the intended artifact by code/catalog; raw on-run path/hash logs and measured HF repository revision are absent. Current or historical HF metadata verification: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/main/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf . The pinned catalog has no checksum. Independently observed current HF SHA256=707a55a8a4397ecde44de0c499d3e68c1ad1d240d1da65826b4949d1043f4450; historical loaded bytes are not hash-verified. Nominal size labels are rounded source labels, not exact total/active parameter counts. Runtime provenance: llama-server 0.4.0-dev, reported commit 790cf51aa, Metal; runtime binary checksum unknown. Paired harness: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/tools/performance_suite.mjs at ZINC 68a03ec6c6415efaeabbac7d4dec83ffdaea9608. It prefers one resident local llama-server per model for all scenarios, and can fall back to llama-cli on server-launch failure. The stored binary name and nonempty completion previews support the server path here; a per-run full command/startup log is not retained. Server method specifies -ngl 999, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, --flash-attn on, --no-cache-prompt. These settings are code-derived protocol evidence, not individually logged runtime readbacks; context and FA form fields remain blank rather than claiming independently verified allocations. GPU offload request 999 is not measured layer count. CPU threads and KV precision unknown. One discarded warmup plus three measured nonstreaming requests per scenario. /v1/completions uses raw prompt, temperature 0, source max_tokens. PP/TG are engine-native response timings.prompt_per_second and predicted_per_second, not client input/TTFT or wall generation throughput. Native prompt_ms/prompt_n are preferred over usage when present. The harness reconstructs decode_ms as completion_tokens/TG*1000; it is not a direct raw decode duration. Do not equate its token-count convention with ZINC's N-over-N-minus-one-loop timing. JSON marks speculation disabled; no draft/MTP flags are in this launch. No first-token timestamp is captured. Headline is Quick Chat/core: recorded prompt/prefill count 35/35, output 96; PP=319.90056, TG=76.377242 tokens/s. Counters are the first measured row saved by createStats, not a per-run count distribution; all three rate samples are retained. These are short actual inputs, not filled 4K or long-context benchmarks. Other scenario labels describe task style; Incident Context reaches only a few hundred input tokens. All scenario rates are separate medians[min,max], tokens/s, n=3; ms where labeled. Displayed to 8 significant digits; pinned JSON retains full precision. Quick Chat (core): input/prefill/output=35/35/96; cap=96; PP=319.90056[317.63606,323.43619]; TG=76.377242[75.621349,78.354845]; PPms=109.409[108.213,110.189]; wrapper wall ms=1363.4107[1321.1642,1366.2505]; wrapper E2E=96.082566[95.882859,99.154972]; paired engine input=36. Coding Review (context-medium): input/prefill/output=153/153/160; cap=160; PP=870.72321[870.32697,881.78847]; TG=78.683969[74.377161,78.973641]; PPms=175.716[173.511,175.796]; wrapper wall ms=2197.158[2187.5179,2322.4165]; wrapper E2E=142.45676[134.77342,143.08455]; paired engine input=174. Incident Context (context-long): input/prefill/output=315/315/128; cap=128; PP=1233.7797[1231.5367,1239.85]; TG=78.98118[77.796509,79.745167]; PPms=255.313[254.063,255.778]; wrapper wall ms=1864.1756[1847.5418,1889.0465]; wrapper E2E=237.63855[234.50985,239.77806]; paired engine input=322. Long Coding Draft (decode-extended): input/prefill/output=63/63/256; cap=256; PP=578.58147[578.40617,580.43118]; TG=78.415912[77.755755,79.638174]; PPms=108.887[108.54,108.92]; wrapper wall ms=3361.358[3319.0944,3388.6211]; wrapper E2E=94.90212[94.138586,96.110553]; paired engine input=64. Wrapper wall/E2E above are a separate measurement. For ZINC they enclose process startup, model loading, inference and termination; for resident llama-server they enclose request/response and exclude server startup. E2E=(prompt+output)/wrapper wall. The site's 'overall' uses inferred phase durations from median rates, excluding overhead, and is neither the saved wrapper E2E nor TTFT. Never put wall-derived or overall rates into native PP/TG fields. Cross-engine comparison caveat: all 20 Mac model/scenario pairs in the pinned JSON have different prompt counts. The rendered dashboard instead says 4/20 same; retain this source inconsistency and use the engine-specific counters above. Templates/tokenization and Qwen thinking output can differ; no matched-token speedup or equivalent-quality claim follows. Source preview checks are only a simple repetition/control-token screen, not independent task scoring. Exact thermal/power conditions, repeated-run count variation, benchmark command environment and independent correctness validation remain unknown.

generation:

65.5 tokens/s

prompt processing (prefill):

305.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical ZINC project campaign, target generated_at 2026-09-13T03:37:48.775Z; recovered 2026-10-13, not locally rerun. This report is the llama.cpp arm. Pinned primary data: https://github.com/zolotukhin/zinc/blob/29bc350ac4cbf9f110ec628b9e177ea04ac816fb/site/src/data/zinc-performance.json (targets[id=metal], models[id=qwen35-9b-q4k-m]). Data publication commit is 29bc350ac4cbf9f110ec628b9e177ea04ac816fb; main and pinned JSON were byte-identical at recovery. Site: https://zolotukhin.ai/zinc/benchmarks/#metal . The September 25 top-level artifact date covers multiple targets; it is not this Mac campaign's measurement date. Hardware: one Apple Mac Studio, Mac16,9, M4 Max, 40-core GPU, 64 GB installed unified memory, Darwin 24.6.0. Forty GPU cores and Mac Studio identity are explicitly stated on the project's hardware/landing page: https://zolotukhin.ai/zinc/ . CPU core count, power mode, temperature/clocks, swap, other-process activity, and storage are not recorded. Unified RAM is not dedicated VRAM. No measured memory usage or model peak is published; leave VRAM/RSS fields blank. Model identity: source model ID qwen35-9b-q4k-m; GGUF Qwen3.5-9B-Q4_K_M.gguf from https://huggingface.co/unsloth/Qwen3.5-9B-GGUF. Pinned managed catalog: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/src/model/catalog.zig . The harness resolves this ID to the managed cache's model.gguf and passes the same file to the baseline. This binds the intended artifact by code/catalog; raw on-run path/hash logs and measured HF repository revision are absent. Current or historical HF metadata verification: https://huggingface.co/unsloth/Qwen3.5-9B-GGUF/blob/main/Qwen3.5-9B-Q4_K_M.gguf . Catalog intended SHA256=03b74727a860a56338e042c4420bb3f04b2fec5734175f4cb9fa853daf52b7e8; independently matched HF metadata. Today's main matching the catalog checksum is identity support, not a hash check of the measured local file. Runtime provenance: llama-server 0.4.0-dev, reported commit 790cf51aa, Metal; runtime binary checksum unknown. Paired harness: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/tools/performance_suite.mjs at ZINC 68a03ec6c6415efaeabbac7d4dec83ffdaea9608. It prefers one resident local llama-server per model for all scenarios, and can fall back to llama-cli on server-launch failure. The stored binary name and nonempty completion previews support the server path here; a per-run full command/startup log is not retained. Server method specifies -ngl 999, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, --flash-attn on, --no-cache-prompt. These settings are code-derived protocol evidence, not individually logged runtime readbacks; context and FA form fields remain blank rather than claiming independently verified allocations. GPU offload request 999 is not measured layer count. CPU threads and KV precision unknown. One discarded warmup plus three measured nonstreaming requests per scenario. /v1/chat/completions uses the runtime chat template, temperature 0, source max_tokens. PP/TG are engine-native response timings.prompt_per_second and predicted_per_second, not client input/TTFT or wall generation throughput. Native prompt_ms/prompt_n are preferred over usage when present. The harness reconstructs decode_ms as completion_tokens/TG*1000; it is not a direct raw decode duration. Do not equate its token-count convention with ZINC's N-over-N-minus-one-loop timing. JSON marks speculation disabled; no draft/MTP flags are in this launch. No first-token timestamp is captured. Headline is Quick Chat/core: recorded prompt/prefill count 45/45, output 96; PP=305.94346, TG=65.528178 tokens/s. Counters are the first measured row saved by createStats, not a per-run count distribution; all three rate samples are retained. These are short actual inputs, not filled 4K or long-context benchmarks. Other scenario labels describe task style; Incident Context reaches only a few hundred input tokens. All scenario rates are separate medians[min,max], tokens/s, n=3; ms where labeled. Displayed to 8 significant digits; pinned JSON retains full precision. Quick Chat (core): input/prefill/output=45/45/96; cap=96; PP=305.94346[303.94316,313.21343]; TG=65.528178[59.937778,66.031697]; PPms=147.086[143.672,148.054]; wrapper wall ms=1599.1964[1589.47,1733.2868]; wrapper E2E=88.169282[81.348338,88.708817]; paired engine input=48. Coding Review (context-medium): input/prefill/output=170/170/160; cap=160; PP=567.59563[566.61945,572.07853]; TG=65.755845[64.309514,65.957506]; PPms=299.509[297.162,300.025]; wrapper wall ms=2718.8588[2708.9772,2773.5643]; wrapper E2E=121.37445[118.98048,121.8172]; paired engine input=193. Incident Context (context-long): input/prefill/output=336/336/128; cap=128; PP=689.59856[688.29751,693.53138]; TG=65.193834[62.860209,66.185096]; PPms=487.24[484.477,488.161]; wrapper wall ms=2436.802[2408.3797,2506.2923]; wrapper E2E=190.4135[185.13404,192.66065]; paired engine input=345. Long Coding Draft (decode-extended): input/prefill/output=67/67/256; cap=256; PP=456.57433[447.60365,456.79223]; TG=65.709812[65.564733,65.741694]; PPms=146.745[146.675,149.686]; wrapper wall ms=4028.4777[4026.6608,4048.4858]; wrapper E2E=80.179171[79.782915,80.215349]; paired engine input=70. Wrapper wall/E2E above are a separate measurement. For ZINC they enclose process startup, model loading, inference and termination; for resident llama-server they enclose request/response and exclude server startup. E2E=(prompt+output)/wrapper wall. The site's 'overall' uses inferred phase durations from median rates, excluding overhead, and is neither the saved wrapper E2E nor TTFT. Never put wall-derived or overall rates into native PP/TG fields. Cross-engine comparison caveat: all 20 Mac model/scenario pairs in the pinned JSON have different prompt counts. The rendered dashboard instead says 4/20 same; retain this source inconsistency and use the engine-specific counters above. Templates/tokenization and Qwen thinking output can differ; no matched-token speedup or equivalent-quality claim follows. Source preview checks are only a simple repetition/control-token screen, not independent task scoring. Exact thermal/power conditions, repeated-run count variation, benchmark command environment and independent correctness validation remain unknown.

generation:

22.4 tokens/s

prompt processing (prefill):

62.7 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical ZINC project campaign, target generated_at 2026-09-13T03:37:48.775Z; recovered 2026-10-13, not locally rerun. This report is the llama.cpp arm. Pinned primary data: https://github.com/zolotukhin/zinc/blob/29bc350ac4cbf9f110ec628b9e177ea04ac816fb/site/src/data/zinc-performance.json (targets[id=metal], models[id=gemma4-31b-q4k-m]). Data publication commit is 29bc350ac4cbf9f110ec628b9e177ea04ac816fb; main and pinned JSON were byte-identical at recovery. Site: https://zolotukhin.ai/zinc/benchmarks/#metal . The September 25 top-level artifact date covers multiple targets; it is not this Mac campaign's measurement date. Hardware: one Apple Mac Studio, Mac16,9, M4 Max, 40-core GPU, 64 GB installed unified memory, Darwin 24.6.0. Forty GPU cores and Mac Studio identity are explicitly stated on the project's hardware/landing page: https://zolotukhin.ai/zinc/ . CPU core count, power mode, temperature/clocks, swap, other-process activity, and storage are not recorded. Unified RAM is not dedicated VRAM. No measured memory usage or model peak is published; leave VRAM/RSS fields blank. Model identity: source model ID gemma4-31b-q4k-m; GGUF gemma-4-31B-it-Q4_K_M.gguf from https://huggingface.co/unsloth/gemma-4-31B-it-GGUF. Pinned managed catalog: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/src/model/catalog.zig . The harness resolves this ID to the managed cache's model.gguf and passes the same file to the baseline. This binds the intended artifact by code/catalog; raw on-run path/hash logs and measured HF repository revision are absent. Current or historical HF metadata verification: https://huggingface.co/unsloth/gemma-4-31B-it-GGUF/blob/main/gemma-4-31B-it-Q4_K_M.gguf . The pinned catalog has no checksum. Independently observed current HF SHA256=38bd64c852c4b460434cc7162fa9bdcf242faf86502581a754cb72956bb17f84; historical loaded bytes are not hash-verified. Nominal size labels are rounded source labels, not exact total/active parameter counts. Runtime provenance: llama-server 0.4.0-dev, reported commit 790cf51aa, Metal; runtime binary checksum unknown. Paired harness: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/tools/performance_suite.mjs at ZINC 68a03ec6c6415efaeabbac7d4dec83ffdaea9608. It prefers one resident local llama-server per model for all scenarios, and can fall back to llama-cli on server-launch failure. The stored binary name and nonempty completion previews support the server path here; a per-run full command/startup log is not retained. Server method specifies -ngl 999, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, --flash-attn on, --no-cache-prompt. These settings are code-derived protocol evidence, not individually logged runtime readbacks; context and FA form fields remain blank rather than claiming independently verified allocations. GPU offload request 999 is not measured layer count. CPU threads and KV precision unknown. One discarded warmup plus three measured nonstreaming requests per scenario. /v1/chat/completions uses the runtime chat template, temperature 0, source max_tokens. PP/TG are engine-native response timings.prompt_per_second and predicted_per_second, not client input/TTFT or wall generation throughput. Native prompt_ms/prompt_n are preferred over usage when present. The harness reconstructs decode_ms as completion_tokens/TG*1000; it is not a direct raw decode duration. Do not equate its token-count convention with ZINC's N-over-N-minus-one-loop timing. JSON marks speculation disabled; no draft/MTP flags are in this launch. No first-token timestamp is captured. Headline is Quick Chat/core: recorded prompt/prefill count 52/52, output 96; PP=62.710139, TG=22.422308 tokens/s. Counters are the first measured row saved by createStats, not a per-run count distribution; all three rate samples are retained. These are short actual inputs, not filled 4K or long-context benchmarks. Other scenario labels describe task style; Incident Context reaches only a few hundred input tokens. All scenario rates are separate medians[min,max], tokens/s, n=3; ms where labeled. Displayed to 8 significant digits; pinned JSON retains full precision. Quick Chat (core): input/prefill/output=52/52/96; cap=96; PP=62.710139[61.559078,63.866921]; TG=22.422308[21.949139,22.436373]; PPms=829.212[814.193,844.717]; wrapper wall ms=5066.3128[5053.1146,5194.592]; wrapper E2E=29.212566[28.491169,29.288867]; paired engine input=49. Coding Review (context-medium): input/prefill/output=195/195/160; cap=160; PP=138.60093[136.99958,138.84716]; TG=21.656018[21.470694,21.975446]; PPms=1406.917[1404.422,1423.362]; wrapper wall ms=8767.8085[8643.5612,8812.4465]; wrapper E2E=40.489023[40.283932,41.071035]; paired engine input=192. Incident Context (context-long): input/prefill/output=345/345/128; cap=128; PP=158.63185[158.0124,159.09578]; TG=21.716882[21.4377,21.983295]; PPms=2174.847[2168.505,2183.373]; wrapper wall ms=8018.295[7953.7387,8109.2845]; wrapper E2E=58.990097[58.328204,59.468888]; paired engine input=346. Long Coding Draft (decode-extended): input/prefill/output=73/73/256; cap=256; PP=90.175781[90.175335,90.186922]; TG=22.786656[22.63885,22.841738]; PPms=809.53[809.43,809.534]; wrapper wall ms=12001.456[11974.577,12098.31]; wrapper E2E=27.41334[27.19388,27.474875]; paired engine input=70. Wrapper wall/E2E above are a separate measurement. For ZINC they enclose process startup, model loading, inference and termination; for resident llama-server they enclose request/response and exclude server startup. E2E=(prompt+output)/wrapper wall. The site's 'overall' uses inferred phase durations from median rates, excluding overhead, and is neither the saved wrapper E2E nor TTFT. Never put wall-derived or overall rates into native PP/TG fields. Cross-engine comparison caveat: all 20 Mac model/scenario pairs in the pinned JSON have different prompt counts. The rendered dashboard instead says 4/20 same; retain this source inconsistency and use the engine-specific counters above. Templates/tokenization and Qwen thinking output can differ; no matched-token speedup or equivalent-quality claim follows. Source preview checks are only a simple repetition/control-token screen, not independent task scoring. ZINC's Gemma 31B Coding Review and Incident Context previews are flagged repetitive; those rates remain labeled and are excluded from any clean-quality conclusion. Exact thermal/power conditions, repeated-run count variation, benchmark command environment and independent correctness validation remain unknown.

generation:

81.7 tokens/s

prompt processing (prefill):

379.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical ZINC project campaign, target generated_at 2026-09-13T03:37:48.775Z; recovered 2026-10-13, not locally rerun. This report is the llama.cpp arm. Pinned primary data: https://github.com/zolotukhin/zinc/blob/29bc350ac4cbf9f110ec628b9e177ea04ac816fb/site/src/data/zinc-performance.json (targets[id=metal], models[id=gemma4-26b-a4b-q4k-m]). Data publication commit is 29bc350ac4cbf9f110ec628b9e177ea04ac816fb; main and pinned JSON were byte-identical at recovery. Site: https://zolotukhin.ai/zinc/benchmarks/#metal . The September 25 top-level artifact date covers multiple targets; it is not this Mac campaign's measurement date. Hardware: one Apple Mac Studio, Mac16,9, M4 Max, 40-core GPU, 64 GB installed unified memory, Darwin 24.6.0. Forty GPU cores and Mac Studio identity are explicitly stated on the project's hardware/landing page: https://zolotukhin.ai/zinc/ . CPU core count, power mode, temperature/clocks, swap, other-process activity, and storage are not recorded. Unified RAM is not dedicated VRAM. No measured memory usage or model peak is published; leave VRAM/RSS fields blank. Model identity: source model ID gemma4-26b-a4b-q4k-m; GGUF gemma-4-26B-A4B-it-UD-Q4_K_M.gguf from https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF. Pinned managed catalog: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/src/model/catalog.zig . The harness resolves this ID to the managed cache's model.gguf and passes the same file to the baseline. This binds the intended artifact by code/catalog; raw on-run path/hash logs and measured HF repository revision are absent. Current or historical HF metadata verification: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/blob/main/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf . The dashboard abbreviates the quant as Q4_K_M, but the catalog/file identify Unsloth UD-Q4_K_M; retain the UD prefix. Catalog SHA256 is empty; independently observed current HF SHA256=f2c28b3dc4776931ac6f879e11f203dec637ea0f14267a86ec8f6165f63f293f is not a measured-run hash. Runtime provenance: llama-server 0.4.0-dev, reported commit 790cf51aa, Metal; runtime binary checksum unknown. Paired harness: https://github.com/zolotukhin/zinc/blob/68a03ec6c6415efaeabbac7d4dec83ffdaea9608/tools/performance_suite.mjs at ZINC 68a03ec6c6415efaeabbac7d4dec83ffdaea9608. It prefers one resident local llama-server per model for all scenarios, and can fall back to llama-cli on server-launch failure. The stored binary name and nonempty completion previews support the server path here; a per-run full command/startup log is not retained. Server method specifies -ngl 999, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, --flash-attn on, --no-cache-prompt. These settings are code-derived protocol evidence, not individually logged runtime readbacks; context and FA form fields remain blank rather than claiming independently verified allocations. GPU offload request 999 is not measured layer count. CPU threads and KV precision unknown. One discarded warmup plus three measured nonstreaming requests per scenario. /v1/chat/completions uses the runtime chat template, temperature 0, source max_tokens. PP/TG are engine-native response timings.prompt_per_second and predicted_per_second, not client input/TTFT or wall generation throughput. Native prompt_ms/prompt_n are preferred over usage when present. The harness reconstructs decode_ms as completion_tokens/TG*1000; it is not a direct raw decode duration. Do not equate its token-count convention with ZINC's N-over-N-minus-one-loop timing. JSON marks speculation disabled; no draft/MTP flags are in this launch. No first-token timestamp is captured. Headline is Quick Chat/core: recorded prompt/prefill count 52/52, output 96; PP=379.59806, TG=81.673639 tokens/s. Counters are the first measured row saved by createStats, not a per-run count distribution; all three rate samples are retained. These are short actual inputs, not filled 4K or long-context benchmarks. Other scenario labels describe task style; Incident Context reaches only a few hundred input tokens. All scenario rates are separate medians[min,max], tokens/s, n=3; ms where labeled. Displayed to 8 significant digits; pinned JSON retains full precision. Quick Chat (core): input/prefill/output=52/52/96; cap=96; PP=379.59806[366.08633,380.55928]; TG=81.673639[81.497635,81.961556]; PPms=136.987[136.641,142.043]; wrapper wall ms=1303.4373[1297.2103,1308.6213]; wrapper E2E=113.54593[113.09613,114.09098]; paired engine input=49. Coding Review (context-medium): input/prefill/output=195/195/160; cap=160; PP=900.55557[892.57106,902.8572]; TG=80.4983[80.435261,80.819828]; PPms=216.533[215.981,218.47]; wrapper wall ms=2193.0673[2187.1633,2194.1188]; wrapper E2E=161.87374[161.79616,162.3107]; paired engine input=192. Incident Context (context-long): input/prefill/output=345/345/128; cap=128; PP=1131.3775[1116.075,1137.7052]; TG=81.536324[81.355394,82.016548]; PPms=304.938[303.242,309.119]; wrapper wall ms=1864.3052[1853.6036,1872.0063]; wrapper E2E=253.71382[252.67009,255.17862]; paired engine input=346. Long Coding Draft (decode-extended): input/prefill/output=73/73/256; cap=256; PP=501.64581[498.29692,512.67285]; TG=77.914745[72.653692,81.432913]; PPms=145.521[142.391,146.499]; wrapper wall ms=3422.5844[3278.1453,3657.5608]; wrapper E2E=96.12619[89.950659,100.36163]; paired engine input=70. Wrapper wall/E2E above are a separate measurement. For ZINC they enclose process startup, model loading, inference and termination; for resident llama-server they enclose request/response and exclude server startup. E2E=(prompt+output)/wrapper wall. The site's 'overall' uses inferred phase durations from median rates, excluding overhead, and is neither the saved wrapper E2E nor TTFT. Never put wall-derived or overall rates into native PP/TG fields. Cross-engine comparison caveat: all 20 Mac model/scenario pairs in the pinned JSON have different prompt counts. The rendered dashboard instead says 4/20 same; retain this source inconsistency and use the engine-specific counters above. Templates/tokenization and Qwen thinking output can differ; no matched-token speedup or equivalent-quality claim follows. Source preview checks are only a simple repetition/control-token screen, not independent task scoring. Exact thermal/power conditions, repeated-run count variation, benchmark command environment and independent correctness validation remain unknown.

generation:

36.0 tokens/s

prompt processing (prefill):

517.9 tokens/s

quant:

AD-3.84bpw-M64 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a 85 GB GGUF quant of Qwen 3.8 Flash Next running on a 64 GB MacBook with the ngram table offloaded to SSD, at 517.9 t/s prefill and 36 t/s decode. The user notes the quant quality is far from perfect and better versions are planned.