Local model performance on your hardware

generation:

80.9 tokens/s

prompt processing (prefill):

2779.0 tokens/s

quant:

MXFP4_MOE (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party historical benchmark by uchkw, published 2026-04-21 at 19:32:04 +09:00 (JSON-LD; modified 2026-04-23 at 00:23:18 +09:00): https://note.com/uchkw/n/nb2265e0ef7f5 . Recovered and audited 2026-10-13, not independently rerun. This record concerns Qwen3.5-35B-A3B, source quantization label MXFP4_MOE, on Mac Studio M3 Ultra. Article explicitly separates llama-bench from its llama-server, quality, VLM and long-context application tests. Native separate-phase PP2048=2778.95 tokens/s and TG32=80.93 tokens/s are transcribed from the article's Mac values; the paired DGX Spark values are not part of this report. Raw repetitions, standard deviations and startup logs are not published. Hardware and software table was downloaded and visually inspected: https://assets.st-note.com/img/1776870911-H1KJ45zgtFmWoqTV6ncDYNiX.png?width=1200 . It reports M3 Ultra Mac Studio, 512 GB memory, macOS Tahoe 26.4.1, Metal backend, llama.cpp b8783. The text calls it Mac Studio; no form-factor inference from chip alone. CPU and GPU core counts, CPU thread count, SSD, clocks, thermals, power and run isolation are unreported. Installed 512 GB is unified system memory, not dedicated VRAM or measured resident memory. Runtime build b8783 resolves via public GitHub tag to e21cdc11a0461d8b0cbd28cc356d993bf6be7282. Binary hash, compiler flags and exact OS build are unknown. Exact native settings given by the article: -p 2048 -n 32 -ub 2048 -fa 1 -ngl 999. These are described as the principal parameters, not a complete shell transcript. Under the pinned harness's unchanged omitted defaults: batch 2048, context depth 0, five repetitions, F16 K and V cache, mmap enabled, no delay, warmup enabled, one sequence. These are implementation-bound defaults and not independently recorded run telemetry. The summary rates do not independently prove repetition count or omitted flags. The application server's --ctx-size262144, ubatch512, temperature1/top-p0.95/top-k64 must not be transferred to llama-bench. Configured application context and effective benchmark allocation remain unknown. Pinned native n_ctx request is n_prompt+n_gen+n_depth, hence 2048 for standalone PP and32 for standalone TG at default depth0; requested capacity is not effective allocation. Native semantics audited in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/tools/llama-bench/llama-bench.cpp : separate PP and TG instances are constructed at lines1263-1335. PP evaluates2048 synthetic token IDs; TG evaluates32 sequential single tokens with synchronization after each. These are synthetic evaluation counts, not a paired2048-token user prompt followed by32 generated prose tokens. Actual natural-language output-token count is therefore not applicable/unknown. Model loading and warmup occur before timing, memory/cache is cleared for each repetition, and any configured depth is processed/restored outside the measured phase. Timer is high_resolution_clock in nanoseconds; displayed mean is arithmetic mean of per-repetition native token rates (lines1469-1478,2063-2117,2297-2396). No model output or throughput is reconstructed from request wall time. Native harness has no sampling quality test, draft model, MTP or speculative decoding in these phase loops. GPU quantity is one participating physical Apple GPU, established from the stated local Metal run with explicit positive-ngl999 plus the pinned implementation, not merely from the M3 name or a compiled backend column. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama.cpp#L943-L1064 selects GPU devices, while https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama-model.cpp#L3019-L3044 assigns eligible layers to them for positive GPU offload. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal.cpp#L16-L18 defaults to one logical Metal device; lines913-928 permit virtual contexts via GGML_METAL_DEVICES. Every such logical instance uses the same physical MTLCreateSystemDefaultDevice in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal-device.m#L624-L646 . Virtual contexts are not extra physical GPUs. No independently captured allocation trace, exact per-layer residency or utilization is claimed. Model provenance is limited to source model family, size class, GGUF context and the verbatim MXFP4_MOE label. Exact filename, quantization publisher, artifact bytes, tested SHA256 and tested HF revision are unknown. HF https://huggingface.co/Qwen/Qwen3.5-35B-A3B is verified as a public current base/reference model only; it is not asserted to be the tested quantized GGUF. The separately linked nabe2030 DGX repository uses different b8665/b8672 builds, different rates and Gemma F16 rather than this article's BF16 label, so its publisher/filenames/hashes cannot repair this Mac run's artifact provenance. Do not replace BF16 with F16, UD-Q5_K_M with plain Q5_K_M, or MXFP4_MOE with a generic MLX4bit quant. The source does not explicitly write '-it' for the Mac checkpoint; the HF instruction-model suffix is reference-only. No TTFT is measured in the native benchmark. Neither reciprocalTG nor2048/PP is TTFT. Application average response latencies, long-context headline generation rates and plotted curves have separate protocols and are excluded. No source software executed, weights downloaded, author contacted or external state changed. Unknowns remain explicit. Article date is publication timing, not a verified measurement timestamp. No direct cross-hardware causal speedup is asserted because the article itself records different llama.cpp builds on the Mac and DGX Spark. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

87.8 tokens/s

prompt processing (prefill):

2595.3 tokens/s

quant:

MXFP4_MOE (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party historical benchmark by uchkw, published 2026-04-21 at 19:32:04 +09:00 (JSON-LD; modified 2026-04-23 at 00:23:18 +09:00): https://note.com/uchkw/n/nb2265e0ef7f5 . Recovered and audited 2026-10-13, not independently rerun. This record concerns Gemma-4-26B-A4B, source quantization label MXFP4_MOE, on Mac Studio M3 Ultra. Article explicitly separates llama-bench from its llama-server, quality, VLM and long-context application tests. Native separate-phase PP2048=2595.28 tokens/s and TG32=87.81 tokens/s are transcribed from the article's Mac values; the paired DGX Spark values are not part of this report. Raw repetitions, standard deviations and startup logs are not published. Hardware and software table was downloaded and visually inspected: https://assets.st-note.com/img/1776870911-H1KJ45zgtFmWoqTV6ncDYNiX.png?width=1200 . It reports M3 Ultra Mac Studio, 512 GB memory, macOS Tahoe 26.4.1, Metal backend, llama.cpp b8783. The text calls it Mac Studio; no form-factor inference from chip alone. CPU and GPU core counts, CPU thread count, SSD, clocks, thermals, power and run isolation are unreported. Installed 512 GB is unified system memory, not dedicated VRAM or measured resident memory. Runtime build b8783 resolves via public GitHub tag to e21cdc11a0461d8b0cbd28cc356d993bf6be7282. Binary hash, compiler flags and exact OS build are unknown. Exact native settings given by the article: -p 2048 -n 32 -ub 2048 -fa 1 -ngl 999. These are described as the principal parameters, not a complete shell transcript. Under the pinned harness's unchanged omitted defaults: batch 2048, context depth 0, five repetitions, F16 K and V cache, mmap enabled, no delay, warmup enabled, one sequence. These are implementation-bound defaults and not independently recorded run telemetry. The summary rates do not independently prove repetition count or omitted flags. The application server's --ctx-size262144, ubatch512, temperature1/top-p0.95/top-k64 must not be transferred to llama-bench. Configured application context and effective benchmark allocation remain unknown. Pinned native n_ctx request is n_prompt+n_gen+n_depth, hence 2048 for standalone PP and32 for standalone TG at default depth0; requested capacity is not effective allocation. Native semantics audited in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/tools/llama-bench/llama-bench.cpp : separate PP and TG instances are constructed at lines1263-1335. PP evaluates2048 synthetic token IDs; TG evaluates32 sequential single tokens with synchronization after each. These are synthetic evaluation counts, not a paired2048-token user prompt followed by32 generated prose tokens. Actual natural-language output-token count is therefore not applicable/unknown. Model loading and warmup occur before timing, memory/cache is cleared for each repetition, and any configured depth is processed/restored outside the measured phase. Timer is high_resolution_clock in nanoseconds; displayed mean is arithmetic mean of per-repetition native token rates (lines1469-1478,2063-2117,2297-2396). No model output or throughput is reconstructed from request wall time. Native harness has no sampling quality test, draft model, MTP or speculative decoding in these phase loops. GPU quantity is one participating physical Apple GPU, established from the stated local Metal run with explicit positive-ngl999 plus the pinned implementation, not merely from the M3 name or a compiled backend column. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama.cpp#L943-L1064 selects GPU devices, while https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama-model.cpp#L3019-L3044 assigns eligible layers to them for positive GPU offload. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal.cpp#L16-L18 defaults to one logical Metal device; lines913-928 permit virtual contexts via GGML_METAL_DEVICES. Every such logical instance uses the same physical MTLCreateSystemDefaultDevice in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal-device.m#L624-L646 . Virtual contexts are not extra physical GPUs. No independently captured allocation trace, exact per-layer residency or utilization is claimed. Model provenance is limited to source model family, size class, GGUF context and the verbatim MXFP4_MOE label. Exact filename, quantization publisher, artifact bytes, tested SHA256 and tested HF revision are unknown. HF https://huggingface.co/google/gemma-4-26B-A4B-it is verified as a public current base/reference model only; it is not asserted to be the tested quantized GGUF. The separately linked nabe2030 DGX repository uses different b8665/b8672 builds, different rates and Gemma F16 rather than this article's BF16 label, so its publisher/filenames/hashes cannot repair this Mac run's artifact provenance. Do not replace BF16 with F16, UD-Q5_K_M with plain Q5_K_M, or MXFP4_MOE with a generic MLX4bit quant. The source does not explicitly write '-it' for the Mac checkpoint; the HF instruction-model suffix is reference-only. No TTFT is measured in the native benchmark. Neither reciprocalTG nor2048/PP is TTFT. Application average response latencies, long-context headline generation rates and plotted curves have separate protocols and are excluded. No source software executed, weights downloaded, author contacted or external state changed. Unknowns remain explicit. Article date is publication timing, not a verified measurement timestamp. No direct cross-hardware causal speedup is asserted because the article itself records different llama.cpp builds on the Mac and DGX Spark. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

84.9 tokens/s

prompt processing (prefill):

2634.8 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party historical benchmark by uchkw, published 2026-04-21 at 19:32:04 +09:00 (JSON-LD; modified 2026-04-23 at 00:23:18 +09:00): https://note.com/uchkw/n/nb2265e0ef7f5 . Recovered and audited 2026-10-13, not independently rerun. This record concerns Gemma-4-26B-A4B, source quantization label UD-Q8_K_XL, on Mac Studio M3 Ultra. Article explicitly separates llama-bench from its llama-server, quality, VLM and long-context application tests. Native separate-phase PP2048=2634.81 tokens/s and TG32=84.87 tokens/s are transcribed from the article's Mac values; the paired DGX Spark values are not part of this report. Raw repetitions, standard deviations and startup logs are not published. Hardware and software table was downloaded and visually inspected: https://assets.st-note.com/img/1776870911-H1KJ45zgtFmWoqTV6ncDYNiX.png?width=1200 . It reports M3 Ultra Mac Studio, 512 GB memory, macOS Tahoe 26.4.1, Metal backend, llama.cpp b8783. The text calls it Mac Studio; no form-factor inference from chip alone. CPU and GPU core counts, CPU thread count, SSD, clocks, thermals, power and run isolation are unreported. Installed 512 GB is unified system memory, not dedicated VRAM or measured resident memory. Runtime build b8783 resolves via public GitHub tag to e21cdc11a0461d8b0cbd28cc356d993bf6be7282. Binary hash, compiler flags and exact OS build are unknown. Exact native settings given by the article: -p 2048 -n 32 -ub 2048 -fa 1 -ngl 999. These are described as the principal parameters, not a complete shell transcript. Under the pinned harness's unchanged omitted defaults: batch 2048, context depth 0, five repetitions, F16 K and V cache, mmap enabled, no delay, warmup enabled, one sequence. These are implementation-bound defaults and not independently recorded run telemetry. The summary rates do not independently prove repetition count or omitted flags. The application server's --ctx-size262144, ubatch512, temperature1/top-p0.95/top-k64 must not be transferred to llama-bench. Configured application context and effective benchmark allocation remain unknown. Pinned native n_ctx request is n_prompt+n_gen+n_depth, hence 2048 for standalone PP and32 for standalone TG at default depth0; requested capacity is not effective allocation. Native semantics audited in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/tools/llama-bench/llama-bench.cpp : separate PP and TG instances are constructed at lines1263-1335. PP evaluates2048 synthetic token IDs; TG evaluates32 sequential single tokens with synchronization after each. These are synthetic evaluation counts, not a paired2048-token user prompt followed by32 generated prose tokens. Actual natural-language output-token count is therefore not applicable/unknown. Model loading and warmup occur before timing, memory/cache is cleared for each repetition, and any configured depth is processed/restored outside the measured phase. Timer is high_resolution_clock in nanoseconds; displayed mean is arithmetic mean of per-repetition native token rates (lines1469-1478,2063-2117,2297-2396). No model output or throughput is reconstructed from request wall time. Native harness has no sampling quality test, draft model, MTP or speculative decoding in these phase loops. GPU quantity is one participating physical Apple GPU, established from the stated local Metal run with explicit positive-ngl999 plus the pinned implementation, not merely from the M3 name or a compiled backend column. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama.cpp#L943-L1064 selects GPU devices, while https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama-model.cpp#L3019-L3044 assigns eligible layers to them for positive GPU offload. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal.cpp#L16-L18 defaults to one logical Metal device; lines913-928 permit virtual contexts via GGML_METAL_DEVICES. Every such logical instance uses the same physical MTLCreateSystemDefaultDevice in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal-device.m#L624-L646 . Virtual contexts are not extra physical GPUs. No independently captured allocation trace, exact per-layer residency or utilization is claimed. Model provenance is limited to source model family, size class, GGUF context and the verbatim UD-Q8_K_XL label. Exact filename, quantization publisher, artifact bytes, tested SHA256 and tested HF revision are unknown. HF https://huggingface.co/google/gemma-4-26B-A4B-it is verified as a public current base/reference model only; it is not asserted to be the tested quantized GGUF. The separately linked nabe2030 DGX repository uses different b8665/b8672 builds, different rates and Gemma F16 rather than this article's BF16 label, so its publisher/filenames/hashes cannot repair this Mac run's artifact provenance. Do not replace BF16 with F16, UD-Q5_K_M with plain Q5_K_M, or MXFP4_MOE with a generic MLX4bit quant. The source does not explicitly write '-it' for the Mac checkpoint; the HF instruction-model suffix is reference-only. No TTFT is measured in the native benchmark. Neither reciprocalTG nor2048/PP is TTFT. Application average response latencies, long-context headline generation rates and plotted curves have separate protocols and are excluded. No source software executed, weights downloaded, author contacted or external state changed. Unknowns remain explicit. Article date is publication timing, not a verified measurement timestamp. No direct cross-hardware causal speedup is asserted because the article itself records different llama.cpp builds on the Mac and DGX Spark. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

88.1 tokens/s

prompt processing (prefill):

2554.2 tokens/s

quant:

UD-Q5_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party historical benchmark by uchkw, published 2026-04-21 at 19:32:04 +09:00 (JSON-LD; modified 2026-04-23 at 00:23:18 +09:00): https://note.com/uchkw/n/nb2265e0ef7f5 . Recovered and audited 2026-10-13, not independently rerun. This record concerns Gemma-4-26B-A4B, source quantization label UD-Q5_K_M, on Mac Studio M3 Ultra. Article explicitly separates llama-bench from its llama-server, quality, VLM and long-context application tests. Native separate-phase PP2048=2554.16 tokens/s and TG32=88.08 tokens/s are transcribed from the article's Mac values; the paired DGX Spark values are not part of this report. Raw repetitions, standard deviations and startup logs are not published. Hardware and software table was downloaded and visually inspected: https://assets.st-note.com/img/1776870911-H1KJ45zgtFmWoqTV6ncDYNiX.png?width=1200 . It reports M3 Ultra Mac Studio, 512 GB memory, macOS Tahoe 26.4.1, Metal backend, llama.cpp b8783. The text calls it Mac Studio; no form-factor inference from chip alone. CPU and GPU core counts, CPU thread count, SSD, clocks, thermals, power and run isolation are unreported. Installed 512 GB is unified system memory, not dedicated VRAM or measured resident memory. Runtime build b8783 resolves via public GitHub tag to e21cdc11a0461d8b0cbd28cc356d993bf6be7282. Binary hash, compiler flags and exact OS build are unknown. Exact native settings given by the article: -p 2048 -n 32 -ub 2048 -fa 1 -ngl 999. These are described as the principal parameters, not a complete shell transcript. Under the pinned harness's unchanged omitted defaults: batch 2048, context depth 0, five repetitions, F16 K and V cache, mmap enabled, no delay, warmup enabled, one sequence. These are implementation-bound defaults and not independently recorded run telemetry. The summary rates do not independently prove repetition count or omitted flags. The application server's --ctx-size262144, ubatch512, temperature1/top-p0.95/top-k64 must not be transferred to llama-bench. Configured application context and effective benchmark allocation remain unknown. Pinned native n_ctx request is n_prompt+n_gen+n_depth, hence 2048 for standalone PP and32 for standalone TG at default depth0; requested capacity is not effective allocation. Native semantics audited in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/tools/llama-bench/llama-bench.cpp : separate PP and TG instances are constructed at lines1263-1335. PP evaluates2048 synthetic token IDs; TG evaluates32 sequential single tokens with synchronization after each. These are synthetic evaluation counts, not a paired2048-token user prompt followed by32 generated prose tokens. Actual natural-language output-token count is therefore not applicable/unknown. Model loading and warmup occur before timing, memory/cache is cleared for each repetition, and any configured depth is processed/restored outside the measured phase. Timer is high_resolution_clock in nanoseconds; displayed mean is arithmetic mean of per-repetition native token rates (lines1469-1478,2063-2117,2297-2396). No model output or throughput is reconstructed from request wall time. Native harness has no sampling quality test, draft model, MTP or speculative decoding in these phase loops. GPU quantity is one participating physical Apple GPU, established from the stated local Metal run with explicit positive-ngl999 plus the pinned implementation, not merely from the M3 name or a compiled backend column. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama.cpp#L943-L1064 selects GPU devices, while https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama-model.cpp#L3019-L3044 assigns eligible layers to them for positive GPU offload. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal.cpp#L16-L18 defaults to one logical Metal device; lines913-928 permit virtual contexts via GGML_METAL_DEVICES. Every such logical instance uses the same physical MTLCreateSystemDefaultDevice in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal-device.m#L624-L646 . Virtual contexts are not extra physical GPUs. No independently captured allocation trace, exact per-layer residency or utilization is claimed. Model provenance is limited to source model family, size class, GGUF context and the verbatim UD-Q5_K_M label. Exact filename, quantization publisher, artifact bytes, tested SHA256 and tested HF revision are unknown. HF https://huggingface.co/google/gemma-4-26B-A4B-it is verified as a public current base/reference model only; it is not asserted to be the tested quantized GGUF. The separately linked nabe2030 DGX repository uses different b8665/b8672 builds, different rates and Gemma F16 rather than this article's BF16 label, so its publisher/filenames/hashes cannot repair this Mac run's artifact provenance. Do not replace BF16 with F16, UD-Q5_K_M with plain Q5_K_M, or MXFP4_MOE with a generic MLX4bit quant. The source does not explicitly write '-it' for the Mac checkpoint; the HF instruction-model suffix is reference-only. No TTFT is measured in the native benchmark. Neither reciprocalTG nor2048/PP is TTFT. Application average response latencies, long-context headline generation rates and plotted curves have separate protocols and are excluded. No source software executed, weights downloaded, author contacted or external state changed. Unknowns remain explicit. Article date is publication timing, not a verified measurement timestamp. No direct cross-hardware causal speedup is asserted because the article itself records different llama.cpp builds on the Mac and DGX Spark. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

62.4 tokens/s

prompt processing (prefill):

2649.3 tokens/s

quant:

BF16 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party historical benchmark by uchkw, published 2026-04-21 at 19:32:04 +09:00 (JSON-LD; modified 2026-04-23 at 00:23:18 +09:00): https://note.com/uchkw/n/nb2265e0ef7f5 . Recovered and audited 2026-10-13, not independently rerun. This record concerns Gemma-4-26B-A4B, source quantization label BF16, on Mac Studio M3 Ultra. Article explicitly separates llama-bench from its llama-server, quality, VLM and long-context application tests. Native separate-phase PP2048=2649.31 tokens/s and TG32=62.43 tokens/s are transcribed from the article's Mac values; the paired DGX Spark values are not part of this report. Raw repetitions, standard deviations and startup logs are not published. Hardware and software table was downloaded and visually inspected: https://assets.st-note.com/img/1776870911-H1KJ45zgtFmWoqTV6ncDYNiX.png?width=1200 . It reports M3 Ultra Mac Studio, 512 GB memory, macOS Tahoe 26.4.1, Metal backend, llama.cpp b8783. The text calls it Mac Studio; no form-factor inference from chip alone. CPU and GPU core counts, CPU thread count, SSD, clocks, thermals, power and run isolation are unreported. Installed 512 GB is unified system memory, not dedicated VRAM or measured resident memory. Runtime build b8783 resolves via public GitHub tag to e21cdc11a0461d8b0cbd28cc356d993bf6be7282. Binary hash, compiler flags and exact OS build are unknown. Exact native settings given by the article: -p 2048 -n 32 -ub 2048 -fa 1 -ngl 999. These are described as the principal parameters, not a complete shell transcript. Under the pinned harness's unchanged omitted defaults: batch 2048, context depth 0, five repetitions, F16 K and V cache, mmap enabled, no delay, warmup enabled, one sequence. These are implementation-bound defaults and not independently recorded run telemetry. The summary rates do not independently prove repetition count or omitted flags. The application server's --ctx-size262144, ubatch512, temperature1/top-p0.95/top-k64 must not be transferred to llama-bench. Configured application context and effective benchmark allocation remain unknown. Pinned native n_ctx request is n_prompt+n_gen+n_depth, hence 2048 for standalone PP and32 for standalone TG at default depth0; requested capacity is not effective allocation. Native semantics audited in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/tools/llama-bench/llama-bench.cpp : separate PP and TG instances are constructed at lines1263-1335. PP evaluates2048 synthetic token IDs; TG evaluates32 sequential single tokens with synchronization after each. These are synthetic evaluation counts, not a paired2048-token user prompt followed by32 generated prose tokens. Actual natural-language output-token count is therefore not applicable/unknown. Model loading and warmup occur before timing, memory/cache is cleared for each repetition, and any configured depth is processed/restored outside the measured phase. Timer is high_resolution_clock in nanoseconds; displayed mean is arithmetic mean of per-repetition native token rates (lines1469-1478,2063-2117,2297-2396). No model output or throughput is reconstructed from request wall time. Native harness has no sampling quality test, draft model, MTP or speculative decoding in these phase loops. GPU quantity is one participating physical Apple GPU, established from the stated local Metal run with explicit positive-ngl999 plus the pinned implementation, not merely from the M3 name or a compiled backend column. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama.cpp#L943-L1064 selects GPU devices, while https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/src/llama-model.cpp#L3019-L3044 assigns eligible layers to them for positive GPU offload. https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal.cpp#L16-L18 defaults to one logical Metal device; lines913-928 permit virtual contexts via GGML_METAL_DEVICES. Every such logical instance uses the same physical MTLCreateSystemDefaultDevice in https://github.com/ggml-org/llama.cpp/blob/e21cdc11a0461d8b0cbd28cc356d993bf6be7282/ggml/src/ggml-metal/ggml-metal-device.m#L624-L646 . Virtual contexts are not extra physical GPUs. No independently captured allocation trace, exact per-layer residency or utilization is claimed. Model provenance is limited to source model family, size class, GGUF context and the verbatim BF16 label. Exact filename, quantization publisher, artifact bytes, tested SHA256 and tested HF revision are unknown. HF https://huggingface.co/google/gemma-4-26B-A4B-it is verified as a public current base/reference model only; it is not asserted to be the tested quantized GGUF. The separately linked nabe2030 DGX repository uses different b8665/b8672 builds, different rates and Gemma F16 rather than this article's BF16 label, so its publisher/filenames/hashes cannot repair this Mac run's artifact provenance. Do not replace BF16 with F16, UD-Q5_K_M with plain Q5_K_M, or MXFP4_MOE with a generic MLX4bit quant. The source does not explicitly write '-it' for the Mac checkpoint; the HF instruction-model suffix is reference-only. No TTFT is measured in the native benchmark. Neither reciprocalTG nor2048/PP is TTFT. Application average response latencies, long-context headline generation rates and plotted curves have separate protocols and are excluded. No source software executed, weights downloaded, author contacted or external state changed. Unknowns remain explicit. Article date is publication timing, not a verified measurement timestamp. No direct cross-hardware causal speedup is asserted because the article itself records different llama.cpp builds on the Mac and DGX Spark. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

25.7 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 25.7 t/s aggregate decode on an M3 Ultra 512GB, with throughput collapsing as concurrent long sequences are batched. Setup is llama.cpp (Unsloth prebuilt b10830) with Q8_0 GGUF, 32k context, flash attention on, and 8 slots. At 32k context, N=1 gives 25.7 t/s aggregate, N=2 gives 8.6 t/s, and N=4 gives 5.5 t/s. Two separate -np 1 processes reach 39.4 t/s aggregate, 4.6x faster than one process batching the same two sequences. User also sees the same cliff with GLM-5.3-Flash, going from 40 to 10.6 t/s aggregate at N=4 between 256 and 8k context, and uses two -np 1 processes behind a load balancer as a production workaround.

generation:

31.1 tokens/s

prompt processing (prefill):

558.8 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 31.05 t/s generation and 558.83 t/s prompt processing on an M3 Ultra 512GB. Setup is llama.cpp through llama-swap with the unsloth Q4_K_XL GGUF at 256,000 context, tested with llama-benchy. Prompt processing measured at pp1000 @ d1000 with 3312 ms time to first response; generation measured at tg500 @ d1000 with a peak of 31.67 t/s.

generation:

8.0 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 8 t/s on a Mac Studio M3 Ultra 512GB with DeepSeek V4 Flash GGUF Q4_K_XL. The user also tried Q8. The user expected better performance.