Llama 2 7B Chat / Q4_0 GGJT v3 / M1 Ultra 64GB / b944 CPU-Accelerate PP and Metal TG

Michael Tedder's August 4, 2023 first-party demonstration: https://speakerdeck.com/falken/llama-dot-cpp-plus-metal-on-m1-ultra . Slides 3, 9, 10 identify a 2022 Mac Studio M1 Ultra, 64 GB unified RAM, macOS Ventura 13.4.1, and Llama 2 7B Chat Q4_0. GPU core variant unknown. Runtime uses 16 CPU threads of 20 available logical threads. Model filename llama-2-7b-chat.ggmlv3.q4_0.bin, legacy GGML/GGJT v3, 3,791,725,184 bytes. The file size is not RAM. Historical download origin, checksum and revision unknown. https://huggingface.co/meta-llama/Llama-2-7b-chat-hf is a verified base/reference-only link, not the tested quantized artifact. Native llama.cpp build 944, commit 8183159cf3def112f6d1fe94815fce70e1bffa12. Source PP 37.75 tokens/s from 19 tokens in 503.32 ms. Source TG 72.39 tokens/s from 27 evaluation runs in 372.99 ms. There are 28 sampling runs, not a verified 28-token output count. Context 512, batch 512, ngl 32, n_predict=-1, n_keep 0, temperature 0.1, seed 1691074574. Load 4943.32 ms, sampling 20.34 ms and total 899.05 ms are separately reported. Prompt asks Mount Fuji's height; displayed response ends normally. No measured TTFT. Only one displayed run; repetition count and hardware frequencies, thermals, power and swap are unknown. Exactly one participating GPU is established by the positive-offload command, built-in unified-memory Metal startup, and the same build's implementation. https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/llama.cpp#L3321-L3324 invokes one Metal initializer when n_gpu_layers>0; https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/ggml-metal.m#L95-L104 selects one MTLCreateSystemDefaultDevice and creates its command queue. GPU core quantity and GPU device quantity are separate. The printed 49152.00 MB recommended working-set limit is not installed RAM or measured memory use. Important phase distinction: https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/llama.cpp#L1832-L1864 dispatches single-token evaluation to Metal, while multi-token prompt evaluation falls back to the CPU/Accelerate graph implementation. These are native phases from that hybrid 2023 runtime. The PP number must not be described as GPU-prefill throughput. Metal completion is awaited before returning: https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/ggml-metal.m#L1112-L1120 . Timing uses ggml_time_us, backed by clock_gettime(CLOCK_MONOTONIC) and converted to microseconds on this platform: https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/ggml.c#L544-L555 . llama_eval_internal starts its timer before graph construction and accumulates after execution and logits/embedding extraction, counting N>1 tokens as prompt evaluation and N=1 calls as evaluation runs: https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/llama.cpp#L1785-L1787 and #L1889-L1918. Sampling and user-interface delivery have different boundaries. The pinned main implementation performs a one-BOS-token warmup and resets timings before generation: https://github.com/ggml-org/llama.cpp/blob/8183159cf3def112f6d1fe94815fce70e1bffa12/examples/main/main.cpp#L409-L414 . That is code-derived behavior, not proof of a cold application start. The unmodified context default is F16 KV; the source excerpt does not independently print KV precision. Local patches and executable hash are unknown. Later slides' aggregate 7B/13B durations are not promoted to PP, TG or TTFT. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.