Oct 13, 2026
Summary
Public source report by SpyderMYK, posted 2026-05-12T01:56:59Z: https://github.com/ggml-org/llama.cpp/discussions/4167#discussioncomment-16886673 . Historical result recovered and checked 2026-10-13; not an independent rerun. Exact measurement date is unknown. The source gives separate native llama-bench PP512 and TG128 measurements on an Apple M4 Mac mini with 24 GB RAM and macOS 26.4. This packet concerns llama3.2:3b only.
Hardware: one participating physical Apple M4 GPU. The source's ambiguous phrase "10 cores" is preserved, rather than itself being treated as a GPU count. Apple's product specification for this exact M4 Mac mini class establishes 10 GPU cores, 10 CPU cores (4 performance + 6 efficiency) and 120 GB/s nominal memory bandwidth: https://support.apple.com/en-us/121555 . These are product-spec enrichments, not observed clocks or utilization. The benchmark explicitly reports four CPU execution threads. Installed 24 GB is unified system RAM, not dedicated VRAM. Peak/resident RAM, per-run GPU memory, swap, SSD capacity, clocks, power, thermals, host isolation and exact OS build are unreported. The native 1.87 GiB model-size column is a model tensor footprint, not measured process memory or disk capacity.
Artifact: Llama-3.2-3B-Instruct, GGUF Q4_K_M; native description Q4_K - Medium, 3.21 B parameters (rounded). Source names Ollama tag llama3.2:3b and the tested GGUF blob sha256-dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff. The official Ollama current manifest https://registry.ollama.ai/v2/library/llama3.2/manifests/3b independently lists the identical model-layer SHA256 and 2,019,377,376 bytes. Metadata page https://ollama.com/library/llama3.2:3b/blobs/dde5aa3fc5ff confirms Q4_K_M. Only manifests and rendered metadata were read; no weight bytes were downloaded. The exact source-reported tested content hash is therefore identified, even though the download route and original quantization author are unreported. Ollama is the verified public distributor/reference; do not attribute this run to a different GGUF publisher. Current manifest SHA256 a80c4f17acd55265feec403c7aef86be0c25983ab279d83f3bcd3abbcb5b8b72 identifies the current wrapper, not the historical tested wrapper. This distinction matters because templates can change while the model blob remains identical.
Hugging Face base/reference only: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct . This HF link is not claimed to be a quantized GGUF repository, the benchmark's download source, or its tested HF revision. Tested HF revision is unknown. The exact quantized artifact reference is the Ollama model metadata page and matching digest above. The benchmark runtime is llama.cpp, not Ollama: the Ollama tag identifies model packaging only.
Runtime: llama.cpp llama-bench build 9100, source short hash 2e97c5f, resolved source commit 2e97c5f96f9fe2bb26f794a348e05d7a1c74baa1. Source settings are -p 512 -n 128 -ngl 99 -r 3, explicitly described as Metal backend with full GPU offload. Native table backend BLAS,MTL, threads 4. Native mean and sample standard deviation in tokens/s: PP512 530.01 +/- 0.86; TG128 45.95 +/- 0.02. Three repetitions per phase. Raw per-repeat timings and a binary hash are not available.
Method audited against the exact pinned file https://github.com/ggml-org/llama.cpp/blob/2e97c5f96f9fe2bb26f794a348e05d7a1c74baa1/tools/llama-bench/llama-bench.cpp . These are separate synthetic tests, not a paired 512-token natural-language prompt followed by a 128-token answer. PP performs 512 synthetic token evaluations in batches. TG performs 128 sequential one-token evaluations, sampling random token IDs, and synchronizes each one; no generated prose is counted. Prompt and generation operation counts are native phase labels, not application requested-output caps. The clock is std::chrono::high_resolution_clock, converted to nanoseconds. Arithmetic mean of the three per-run rates is printed with sample standard deviation. Model loading and warmup are outside timing; the memory/cache is cleared at each repetition. Under unchanged pinned defaults, depth zero, batch 2048, microbatch 512, F16 K/V, FA disabled, mmap enabled, warmup enabled (full prompt for PP; one token for TG), no delay. These omitted flags are implementation-bound defaults, not separately recorded per-run telemetry. Source provides a settings summary rather than a complete shell invocation. Pinned line1219 requests n_ctx = n_prompt + n_gen + n_depth: 512 for the separate PP case and 128 for the separate TG case. These are requested benchmark context sizes, not verified effective allocation. Effective allocated context and configured application context capacity remain unknown; model metadata maximum context is not substituted. The synthetic loop contains no draft model, MTP or speculative decoding.
GPU quantity proof is source protocol plus pinned implementation, not merely the compiled backend column. The source explicitly claims full Metal offload with positive -ngl99. Device selection is in https://github.com/ggml-org/llama.cpp/blob/2e97c5f96f9fe2bb26f794a348e05d7a1c74baa1/src/llama.cpp (lines125-269); GPU-layer assignment is in https://github.com/ggml-org/llama.cpp/blob/2e97c5f96f9fe2bb26f794a348e05d7a1c74baa1/src/llama-model.cpp (lines1222-1245). In https://github.com/ggml-org/llama.cpp/blob/2e97c5f96f9fe2bb26f794a348e05d7a1c74baa1/ggml/src/ggml-metal/ggml-metal.cpp the backend defaults to one logical device (line18) and permits GGML_METAL_DEVICES logical instances (lines916-934). Crucially, every such instance binds MTLCreateSystemDefaultDevice in https://github.com/ggml-org/llama.cpp/blob/2e97c5f96f9fe2bb26f794a348e05d7a1c74baa1/ggml/src/ggml-metal/ggml-metal-device.m (lines631-647), so those logical contexts do not imply multiple physical GPUs. Combined with the stated single M4 host, the tested positive offload path uses one physical Apple GPU. No independently captured startup allocation trace, utilization log or exact per-layer residency is claimed.
TTFT is not reported and is left blank. No reciprocal PP, prompt duration, model-loading time, full response latency or generation time is promoted to TTFT. The rates are native synthetic benchmark throughput, without quality validation. Reproduction limits include absent raw repetitions, unspecified run date/conditions and possible local binary modifications. No inference benchmark or downloaded source program was executed during this audit.
LlamaPerf structured numeric fields are rounded to the supported 0.1 tokens/s step; the exact native rates and standard deviations remain above. Source and artifact references are public; no authentication or license acceptance was needed for reading the cited metadata.