- generation:
- 15.7 tokens/s
- prompt processing (prefill):
- 255.7 tokens/s
- quant:
- Q8_0 (GGUF)
- kv:
- F16
- flash attention:
- on
- mtp (multi-token prediction):
- off
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
Historical public-source recovery from Qiu et al., arXiv:2609.14864v1, submitted 2026-09-14. Source: https://arxiv.org/html/2609.14864v1 . These are selected protocol-complete reference-probe measurements from Supplement section 1.2. The paper excluded them from prediction fitting and scoring, not from its measured cohort. This is not an independently executed benchmark. Hardware: Mac Studio M4 Max, 128 GB unified system memory, Darwin-arm64, BLAS/MTL. Source specifies 12 performance-core threads, not total physical CPU cores. GPU core count, full CPU core count, macOS version, clocks, power and temperature are unreported. RAM is not dedicated VRAM or measured residency. Model M14: Qwen3.8-27B, Q8_0, GGUF. Exact source repository/file: unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q8_0.gguf; 29,047,086,048 bytes. The paper's coarser metadata quant label is Q8_0; the full filename's dynamic UD-/XL identity is preserved when present. Verified current reference: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/blob/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-Q8_0.gguf . Pinned reference revision 4ca720788d1e01f1bff70c033e0d0028fd02e502; LFS SHA256 a680f44a06920e5d689774823782006aa3acc8db95750323373b24139b67e348. Current filename and byte count match the paper. The paper says the Studio files were SHA-verified, but it does not expose the historical per-file hash/revision in its tables; it names companion provenance files whose repository is not linked. Thus this current hash is reference metadata, not asserted as the measured historical bytes. Publisher model card identifies base/reference Qwen/Qwen3.8-27B; https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/raw/4ca720788d1e01f1bff70c033e0d0028fd02e502/README.md . No model weights were downloaded. Runtime: source identifies the official llama.cpp b10794 archive for this Studio; upstream tag resolves to f9f09f02cc44d87d842dbd2d578857d92d4bb63b. The source does not publish the archive hash itself in its displayed table. The MacBook's unrecovered runtime revision and RTX directory label are different hosts and are not transferred here. Headline is depth 0: PP512 255.725 and TG128 15.697 tokens/s. All three same-model depth conditions: Prefix depth 0: native PP512 255.725 +/- 0.175 tokens/s (within-run CV 0.068%); native TG128 15.697 +/- 0.063 tokens/s (CV 0.403%). Prefix depth 4096: native PP512 244.576 +/- 0.168 tokens/s (within-run CV 0.069%); native TG128 15.505 +/- 0.041 tokens/s (CV 0.266%). Prefix depth 16384: native PP512 215.717 +/- 0.047 tokens/s (within-run CV 0.022%); native TG128 14.931 +/- 0.022 tokens/s (CV 0.150%). These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end 512-input/128-output conversation. Existing prefix depth is not configured serving context capacity. The depth-0 generation result does not follow the PP512 prompt. No TTFT was measured or reconstructed. The table's rates are rounded to three decimals; published +/- values are native within-run SD; per-repetition raw timings are unavailable in the paper. Protocol: single sequence; fresh process for each configuration; five repetitions; 45-second settling; requested GPU layers 99; flash attention; F16 K/V. The chosen reference-probe rows have 1 acquisition attempt per depth; pre-run load is not printed in this probe table. Selection prefers complete protocol rows, then lower worst decode CV and lower pre-run load, without reference to prediction residuals. The gate applies to decode; prefill is retained without that gate. No between-run Studio repeatability campaign was performed; execution order was not randomized. No device-memory trace or resident-layer count was recorded. Maximal offload requested does not establish full accelerator residency. Physical GPU quantity 1 is source-protocol and pinned-implementation bound, not inferred from GPU cores or a form default: the paper names the local arm/Metal backend on this Studio and requested offload99. At b10794, Metal initializes MTLCreateSystemDefaultDevice; its implementation expressly uses one physical device, with any GGML_METAL_DEVICES additions being virtual devices on the same physical GPU. This establishes the participating physical Metal device count for the reported Metal protocol, not independent utilization telemetry or proof that all layers reside there. Sources: https://github.com/ggml-org/llama.cpp/blob/f9f09f02cc44d87d842dbd2d578857d92d4bb63b/ggml/src/ggml-metal/ggml-metal-device.m#L1091-L1114 ; https://github.com/ggml-org/llama.cpp/blob/f9f09f02cc44d87d842dbd2d578857d92d4bb63b/src/llama.cpp . Pinned llama-bench semantics: high_resolution_clock timestamps in nanoseconds bracket the timed test; per-repetition rate is (n_prompt+n_gen)*1e9/elapsed_ns, and headline avg_ts averages these rates. Existing depth is preprocessed or restored before the timer. Memory is cleared for each repetition; the requested context is n_prompt+n_gen+n_depth, so there is no single fixed serving-capacity field to fill. Warmup is enabled by the upstream default, but the paper does not print a complete command or separately attest warmup, batch/microbatch, mmap or every environment override; these remain unreported for this campaign. Sources: https://github.com/ggml-org/llama.cpp/blob/f9f09f02cc44d87d842dbd2d578857d92d4bb63b/tools/llama-bench/llama-bench.cpp . No speculative-decoding/MTP workload is described; source uses plain llama-bench PP/TG tests. The paper's frozen metadata snapshot reports 27,320,697,856 total parameters and 27,320,697,856 active parameters (100.0%), 65 layers, and modeled KV bytes at prefix0/4096/16384 of 0/1,090,519,040/4,362,076,160. These are author-derived metadata/modeling quantities, not independent tensor recounts or observed RAM/VRAM usage. They must not be used to assert measured memory residency. Architecture caveat for the paper’s Qwen3.8-27B ladder: its UD-IQ2_XXS artifact M15 has 64 layers and 26,895,998,464 parameters, while the other ladder files have 65 layers and 27,320,697,856 parameters. The IQ2 file therefore has one fewer layer and about 1.55% fewer parameters. This is not an architecture-controlled quantization comparison. The GGUF architecture label qwen35 does not rename the tested Qwen3.8 model family. Classification and separate calibration observation: this file was one of the three Studio reference-probe configurations omitted from fitted/scored prediction cohorts; its six selected protocol-complete PP/TG observations above are still measured benchmarks. Supplement section 6.1 separately lists an initial calibration decode result of 15.364 +/- 0.080 tokens/s (CV 0.52%, effective bandwidth 425.530 GB/s). That separate calibration row lacks the full depth/prompt/output conditions in its table, so it is context only and is not substituted for the protocol-complete headline or blended with its repetitions. No prediction accuracy statistic is assigned to this probe report. The form rounds the structured PP/TG rates to one decimal; the exact source values are retained above.