Qwen 3

generation:

97.7 tokens/s

prompt processing (prefill):

2744.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:29:16Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-8b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-8B-GGUF, file Qwen_Qwen3-8B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision 0b69f75b7472688e6808490aa2b85efdb81b5ce7 and SHA256 54fffa050078e984116639c83dfb64b5aa6d4cd474e018b076777c632bbccccd. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-8B-GGUF/blob/0b69f75b7472688e6808490aa2b85efdb81b5ce7/Qwen_Qwen3-8B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 8B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=97.73 tokens/s; prompt processing is pp512=2744.69 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=1848.1; pp256=2439.6; pp512=2744.69; tg128=97.73; tg256=97.77. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=4.68 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=4.84 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=20.5 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

159.7 tokens/s

prompt processing (prefill):

4972.7 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:26:55Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-4b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-4B-GGUF, file Qwen_Qwen3-4B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision cb76885dc66d50759b207c5a48c4e78dfa00c638 and SHA256 fbe1d5edd4ce802ae3ae7c7e4ab7d09789d697fdac1fc7929f8df4ca3c41bae3. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-4B-GGUF/blob/cb76885dc66d50759b207c5a48c4e78dfa00c638/Qwen_Qwen3-4B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 4B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=159.73 tokens/s; prompt processing is pp512=4972.74 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=2670.96; pp256=3761.94; pp512=4972.74; tg128=159.73; tg256=159.46. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=2.33 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=2.48 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=24.4 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

26.4 tokens/s

prompt processing (prefill):

667.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:25:38Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-32b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-32B-GGUF, file Qwen_Qwen3-32B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision 533fbb1ae5f2ce96c9171acc169e1f68f9352eca and SHA256 e41ec56ddd376963a116da97506fadfccb50fb402bb6f3cb4be0bc179a582bd6. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-32B-GGUF/blob/533fbb1ae5f2ce96c9171acc169e1f68f9352eca/Qwen_Qwen3-32B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 32B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=26.38 tokens/s; prompt processing is pp512=667.0 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=461.28; pp256=631.47; pp512=667.0; tg128=26.38; tg256=26.37. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=18.4 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=18.59 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=13.6 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

148.5 tokens/s

prompt processing (prefill):

3496.9 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:16:49Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-30b-a3b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-30B-A3B-GGUF, file Qwen_Qwen3-30B-A3B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision 46f17e079cba70b04390bef39b57d2783e9fd015 and SHA256 a015794bfb1d69cb03dbb86b185fb2b9b339f757df5f8f9dd9ebdab8f6ed5d32. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-30B-A3B-GGUF/blob/46f17e079cba70b04390bef39b57d2783e9fd015/Qwen_Qwen3-30B-A3B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 30B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=148.53 tokens/s; prompt processing is pp512=3496.85 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=1798.44; pp256=2669.29; pp512=3496.85; tg128=148.53; tg256=148.64. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=17.35 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=17.51 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=14.3 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

57.4 tokens/s

prompt processing (prefill):

1584.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:10:40Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-14b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-14B-GGUF, file Qwen_Qwen3-14B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision bd080f768a6401c2d5a7fa53a2e50cd8218a9ce2 and SHA256 915913e22399475dbe6c968ac014d9f1fbe08975e489279aede9d5c7b2c98eb6. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-14B-GGUF/blob/bd080f768a6401c2d5a7fa53a2e50cd8218a9ce2/Qwen_Qwen3-14B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 14B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=57.37 tokens/s; prompt processing is pp512=1584.26 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=1044.44; pp256=1463.63; pp512=1584.26; tg128=57.37; tg256=56.61. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=8.38 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=8.55 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=18.8 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

288.6 tokens/s

prompt processing (prefill):

11493.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:06:29Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-1.7b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-1.7B-GGUF, file Qwen_Qwen3-1.7B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision dcb19155b962dbb6389f4691a982043a8e651022 and SHA256 72c5c3cb38fa32d5256e2fe30d03e7a64c6c79e668ad84057e3bd66e250b24fb. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-1.7B-GGUF/blob/dcb19155b962dbb6389f4691a982043a8e651022/Qwen_Qwen3-1.7B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 1.7B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=288.6 tokens/s; prompt processing is pp512=11493.01 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=6190.75; pp256=8638.62; pp512=11493.01; tg128=288.6; tg256=287.03. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=1.19 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=1.34 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=21.6 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

457.1 tokens/s

prompt processing (prefill):

23288.5 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, saved 2026-09-27T01:05:45Z; recovered 2026-10-13. No local model run or reproduction performed. Primary raw result: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf/qwen3-0.6b_Q4_K_M_ngl99.json . Source publication commit is 34fee99b6df6129fb3c64255c1bfb179a54ca044. The author's upstream submission, including corrections and limitations, is https://github.com/enescingoz/mac-llm-bench/pull/12 . Hardware: one Apple Mac Studio Mac17,14, M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified memory; macOS 27.0 build 26A428, bare metal, AC power, High Power energy mode. Source corrects its JSON core labels: the 6/12 split denotes 6 Super and 12 Performance cores, not 12 Efficiency cores. iogpu.wired_limit_mb=32768 is a session limit, not installed RAM or dedicated VRAM. PR reports ambient about 20.6 C, external WD_BLACK SN7100 NVMe in OWC Express 1M2 over USB4, idle Ollama with no model loaded and no other heavy apps. Swap was at most 2 MB when checked every 60 s. No macOS thermal/performance warnings were recorded, but per-model thermal state was not captured. Do not import zero-swap/per-cell thermal claims from the author's separate bandwidth harness. Exact model: bartowski/Qwen_Qwen3-0.6B-GGUF, file Qwen_Qwen3-0.6B-Q4_K_M.gguf, GGUF Q4_K_M. Manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/gguf-revisions.tsv . It records measured repository revision 60b85c0e3d8fe0f6474f406922a26d12aca4550d and SHA256 9acfc1e001311f34b4252001b626f2e466d592a42065f66571bff3790d4e1b14. The exact revision's HF file page was independently read and its displayed SHA256 matched: https://huggingface.co/bartowski/Qwen_Qwen3-0.6B-GGUF/blob/60b85c0e3d8fe0f6474f406922a26d12aca4550d/Qwen_Qwen3-0.6B-Q4_K_M.gguf . This binds model identity and quantized artifact; today's main is not substituted for the measured revision. Source nominal model size label is 0.6B. Runtime: llama.cpp Homebrew build 11146, commit 7fe450e19, ggml 0.25.3, Metal. The raw runtime.version accidentally stores a load_backend BLAS log line; the author explicitly corrects it in PR 12 and reports Metal 4 tensor API active. This is not a CPU-only BLAS benchmark. Each model was run in a separate ./bench_gguf.sh --model process because --auto stopped after the first model. Source parameters are n_gpu_layers=99, flash_attention=true, threads=0. The helper omits -t when threads=0; this means runtime default rather than zero running threads. Requested 99 GPU layers is not a measured layer-offload count. Quantization above describes weights, not KV cache. Headline generation is tg128=457.09 tokens/s; prompt processing is pp512=23288.49 tokens/s. These are separate native llama-bench microtests from the same campaign, not one 512-input/128-output request, an average across lengths, or server concurrency throughput. All retained native rates in tokens/s: pp128=10629.59; pp256=18270.0; pp512=23288.49; tg128=457.09; tg256=456.61. The pp tests process 128/256/512 synthetic token IDs; the tg tests perform 128/256 one-token decode steps without a preceding 512-token prompt. The context field is intentionally unspecified: these test lengths are not an advertised context window or a long-context setting. Metric method: published mac-llm-bench bench_gguf.sh entrypoint passes -p 128,256,512 -n 128,256 -ngl 99 -fa 1 to llama-bench. It first runs human-readable output, then a separate -o json invocation for saved speed results; those two invocations are not combined. This entrypoint has inline speed/memory logic; the separate lib/run_bench_gguf.sh helper is not the saved-speed entrypoint, so its different display/capture ordering must not be substituted. lib/collect_results.sh maps native n_prompt/n_gen and avg_ts into these speed keys and rounds to 2 decimals. Native llama.cpp code at the reported 7fe450e19 revision times synchronized inference, computes each repetition's token count divided by elapsed nanoseconds, then takes the arithmetic mean of those rates. This is not input divided by TTFT, total wall time, or a pooled count/total-duration calculation. The pinned llama-bench defaults are 5 repetitions, warmup enabled, depth 0, with model memory cleared before each timed repetition; PP uses synthetic random IDs and TG sequential random IDs, without sampling meaningful chat output. This establishes native benchmark semantics. The exact executed shell-harness commit and full raw llama-bench samples, durations, dispersion and command log were not preserved in this result, so the 5-repetition/default-cache interpretation is code/protocol-derived rather than independently counted from raw samples. Current upstream shell code is supporting method evidence, not proof of byte-for-byte historical reproduction. KV precision, actual CPU thread count, realized batch settings, speculation/MTP and quality scores are not separately captured in this JSON; no quality validation or conversational thinking-mode conclusion follows. Memory: file_size_gb=0.45 is the on-disk file size divided by 2^30, despite the GB key. Source peak_rss_gb=0.6 is separately measured process maximum RSS. The published entrypoint obtains RSS with /usr/bin/time -l on a separate llama-bench -p 32 -n 1 invocation, then divides bytes by 2^30. It is not the timed speed test's peak, dedicated VRAM, model-only allocation or installed capacity. The per-record available_ram_gb=22.2 is available system memory at collection, not model usage. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/bench_gguf.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp . Source data/documentation license is CC BY 4.0. All values remain source-reported single-machine results; these short synthetic tests do not establish long-context, multi-user or application end-to-end performance.

generation:

186.0 tokens/s

prompt processing (prefill):

5505.7 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T20:55:11Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-4B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=31.1 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-4B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: 4dcb3d101c2a062e5c1d4bb173588c54ea6c4d25; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-4B-4bit . Current card identifies conversion from Qwen/Qwen3-4B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=2990.39; pp256=4214.14; pp512=5505.67; tg128=185.98; tg256=188.5. Headline uses pp512=5505.67 and tg128=185.98, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=2.993. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.

generation:

28.4 tokens/s

prompt processing (prefill):

730.9 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T20:53:41Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-32B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=31.2 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-32B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: bcaaf7f538adf166c1080a2befdb4f6019f66639; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-32B-4bit . Current card identifies conversion from Qwen/Qwen3-32B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=491.47; pp256=602.46; pp512=730.94; tg128=28.41; tg256=28.54. Headline uses pp512=730.94 and tg128=28.41, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=19.151. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.

generation:

147.2 tokens/s

prompt processing (prefill):

3458.2 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T20:43:07Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-30B-A3B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=31.3 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-30B-A3B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: d388dead1515f5e085ef7a0431dd8fadf0886c57; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-30B-A3B-4bit . Current card identifies conversion from Qwen/Qwen3-30B-A3B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=1488.0; pp256=2626.72; pp512=3458.21; tg128=147.2; tg256=148.65. Headline uses pp512=3458.21 and tg128=147.2, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=17.755. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.

generation:

62.6 tokens/s

prompt processing (prefill):

1738.6 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T20:36:00Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-14B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=31.3 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-14B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: a4d9b2df59d2c150bef02fcbe0d91046b7ca33a4; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-14B-4bit . Current card identifies conversion from Qwen/Qwen3-14B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=1145.62; pp256=1393.0; pp512=1738.59; tg128=62.55; tg256=62.71. Headline uses pp512=1738.59 and tg128=62.55, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=8.825. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.

generation:

366.0 tokens/s

prompt processing (prefill):

12362.2 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T20:34:18Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-1.7B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=31.6 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-1.7B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: 3b1b1768f8f8cf8351c712464f906e86c2b8269e; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-1.7B-4bit . Current card identifies conversion from Qwen/Qwen3-1.7B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=4806.23; pp256=8365.25; pp512=12362.19; tg128=365.99; tg256=371.8. Headline uses pp512=12362.19 and tg128=365.99, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=1.713. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.

generation:

570.6 tokens/s

prompt processing (prefill):

22693.8 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T20:33:31Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-0.6B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=31.6 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-0.6B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: 73e3e38d981303bc594367cd910ea6eb48349da8; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-0.6B-4bit . Current card identifies conversion from Qwen/Qwen3-0.6B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=9042.7; pp256=12175.94; pp512=22693.81; tg128=570.57; tg256=594.11. Headline uses pp512=22693.81 and tg128=570.57, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=0.861. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.

generation:

110.6 tokens/s

prompt processing (prefill):

3150.6 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported historical campaign by emihiggins, measured 2026-09-27T19:13:08Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/Qwen3-8B-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=25.5 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/Qwen3-8B-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: 545dc4251c05440727734bcd94334791f6ab0192; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Qwen3-8B-4bit . Current card identifies conversion from Qwen/Qwen3-8B. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=1876.59; pp256=2405.12; pp512=3150.56; tg128=110.55; tg256=111.03. Headline uses pp512=3150.56 and tg128=110.55, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=5.217. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.