llamaperf

ALMA 7B-Ja / Q4_K_M / M2 Pro 19-core / native phases

on M2 Pro 32GB · llama.cpp

Apple hardware
Oct 13, 2026
Throughput
34.8 t/s gen · 227.6 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
System RAM
32 GB

Summary

First-party measurements by kun432 at https://zenn.dev/kun432/scraps/ea13185fd8a086, dated 2023-10-18. The author explicitly identifies Mac mini 2023, Apple M2 Pro with 12 CPU cores, 19 GPU cores and 32 GB installed unified memory. GPU quantity is exactly one participating Metal device: this specific GPU-enabled run prints device discovery and explicit selection of Apple M2 Pro as the default device. This is run-bound selection, not an assumption based on the Mac model; 19 GPU cores are not 19 GPUs. The source also shows CPU-only measurements, which are not merged into this GPU profile. Model ALMA-7B-Ja, quantization Q4_K_M, filename webbigdata-ALMA-7B-Ja-q4_K_M.gguf, GGUF. Source links mmnga/webbigdata-ALMA-7B-Ja-gguf and names the exact file. Current metadata independently verifies it at https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/blob/40e7546c469f9e6d0dac307dbfa90bb3c9c88b07/webbigdata-ALMA-7B-Ja-q4_K_M.gguf; the historical tested revision/hash is unknown. This is the original ALMA-7B-Ja run, not the later v2 mentioned elsewhere in the page. Runtime llama.cpp ./main with Metal; source commit, build version, binary hash and compiler flags are unknown. Command uses the given GGUF and -n 128 with the shown Japanese-to-English translation template. The GPU-enabled command omits -ngl but logs establish active Metal default-device initialization. Exact layer count, context capacity, batch/microbatch, CPU threads, KV precision and flash attention are unreported and not inferred from a current version. Native prompt eval 347.14 ms / 79 tokens = 227.58 tokens/s. Native eval 1265.47 ms / 44 runs = 34.77 tokens/s. Sample count 45; exact output token count is not substituted for native eval-run count. These are actual request phase timers, not an end-to-end average. One displayed run, with no raw repetitions or warmup/cache protocol. No measured TTFT. Native prompt-eval duration and load duration must not be relabelled as TTFT. OS, clock speeds, power, thermals, swap state, disk size and measured peak process/GPU allocation are unknown. Installed 32 GB unified RAM is explicit and separate from file size or runtime buffer figures. No software, weights or benchmarks were executed to verify the author's result. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.