Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 at 16.5 t/s generation and 518.3 t/s prompt processing on an M4 with 16 GB unified memory.
Setup is LiteRT-LM on the GPU backend with dynamic_wi8_afp32 quantization and a 4,096-token maximum context, using a 256-token prefill and 256-token decode benchmark.
The source gives exact rates of 16.53 t/s decode and 518.28 t/s prefill, and does not establish whether the phases were separate tests or one request.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 3.6 t/s generation and 21 t/s prefill on an M4 Mac mini with 16 GB unified memory.
Setup is turboquant-mlx 0.24.0 with TurboQuant 3-bit g64 weights and 4-bit affine embeddings, lm_head and vision tower, at a 2014-token prompt with 13.14 GiB measured peak.
The source's four rows span 220 to 5017 prompt tokens, with decode 3.6 to 3.9 t/s and prefill 19.0 to 21.0 t/s; generated-token counts and KV precision are unstated.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Tiel-Coder 35B-A3B at 109.1 t/s median native decode on a single Apple Mac mini M4 Pro with 64 GiB unified memory.
Setup is AX Engine with the AXQ-MXFP4 MLX export, expert streaming off, and speculative decoding at maximum draft depth three.
The figure is native decode only, measured over six requests on a python-lru prompt with 194 input tokens and 256 total output tokens; separate median completion throughput including TTFT is 90.46 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports gpt-oss 21B-A3.6B at 77.2 t/s on an M4 Pro 48GB.
Setup is Rapid-MLX 0.15.4 with MXFP4-Q8, single-stream generation, median of three runs at temperature 0 over a 512-token decode workload.
Four concurrent streams deliver 126.7 t/s aggregate, and the reported 15.7 GB Metal peak covers the full sweep, including up to approximately 16K prompt and four streams with prefix caching.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 78.5 t/s on an M4 Pro with 48 GB unified memory.
Setup is Rapid-MLX 0.15.4 with MLX 4-bit on one Mac mini, vision disabled, decoding a 512-token workload.
The figure is single-stream, a median of three runs at temperature 0; a separate four-stream result of 121.6 t/s is aggregate and not single-request speed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 90 tok/s on an M4 Pro Mac mini with 48GB unified memory.
Setup is oMLX 0.7.0 with a custom oQ4 quantised model using MTP headers.
User notes performance had recently fallen as low as 10-11 tok/s before this improvement.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-use
User reports Qwen3-8B at ~10 tok/s decode on a 16GB Mac (M4) using MLX with 4-bit quantization.
Setup runs a persistent local agent with Qwen3-8B on the GPU, Qwen3-4B as 36 int8 layer shards on the Neural Engine, and Qwen2.5-0.5B for routing. The ANE toolchain is custom Rust with no Python.
User notes 16GB is a hard ceiling and the mesh has only run on one machine so far.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports running Qwen3.8-Flash-Next, a 125B-A6B MoE, on a 128 GB M4 Max MacBook Pro with a new unofficial MLX engine called strata-mlx that reads Strata's GGUF files directly.
Setup uses the model's built-in draft head for speculative decoding. The Q2_0 file is 66 GB and the IQ3_S file is 84 GB. On a Mac capped to 16 GB of memory the 66 GB file runs at 6-7 tokens/second, on a 24 GB machine at 19-23, and on a 32 GB machine at 28-47, with experts streamed from SSD. A 1,537-token prompt runs at 273-287 tokens/second in that mode.
The user notes a 48 GB Mac holds the Q2_0 file whole and never touches the disk, and that speed is not guaranteed on smaller machines.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Llama 3.2 3B at 45.4 t/s generation on a Mac mini M4 16GB.
Setup is Ollama 0.30.8 at Q4_K_M on Metal, one request, speculative decoding off.
Mean of two runs by ai-local-lab, measured 2026-06-16. A source-reported measurement, not independently reproduced.
IQ2_XXS gate/up + Q2_K down; BF16 n-grams on SSD (GGUF)
mtp (multi-token prediction):
off
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 41.5 t/s generation and 621.3 t/s prefill on an M4 Max 64GB.
Setup is ds4 with IQ2_XXS gate/up + Q2_K down; BF16 n-grams on SSD, one request, speculative decoding off.
First row (2,048-token prompt) of pswai's 64gb-mac-llm-bench, measured 2026-09-28. A source-reported measurement, not independently reproduced.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 21.3 t/s generation and 143.7 t/s prefill on an M4 Max 64GB Mac Studio.
Setup is ZINC with Metal backend, GGUF Q4_K_M, 48 prompt tokens and 96 generated tokens. Figures are source-published medians from three CLI invocations after one discarded warmup.
No speculative decoding is reported. The source warns that 16 of 20 Metal rows have different prompt-token counts between ZINC and llama.cpp, and this submission makes no matched-workload comparison claim.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvision
User reports DeepSeek V4 Flash Vision-Exp at 31.7 tok/s decode on a Mac Studio M4 Max 128GB.
Setup is ds4 (DwarfStar) with IQ2_XXS/Q2_K GGUF, DSpark speculative decoding at confidence 0.6, and a 32768 context length.
Thinking-on pagoda run reached 29.84 tok/s over 26,415 completion tokens; 1M context loaded at 97.15 GiB planned total with 32.0 tok/s decode. User notes DSpark adds roughly 10-20% over no speculation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports GLM-5.3-Flash Q4_K at 6.21 t/s decode and 2.60 t/s prefill on an M4 Max 128 GiB via SSD streaming.
Setup is ds4 with Q4_K GGUF, 4096 context, 128-token protocol, cold and warm runs. The 177.77 GiB model streams from SSD; GPU time is only ~24 ms of ~161 ms per token (15%), with host-side per-layer expert-cache bookkeeping dominating.
Removing the synchronous selected-id readback changed nothing (6.22/6.24 t/s, byte-identical output). MTP was ~15% slower (5.28/5.22 t/s). Cache budgets of 48/56/85 GB gave 5.3/5.3/4.6 t/s; the automatic 70.1 GiB budget was best. All tested flags stayed within noise of 6.2 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a sourced third-party community benchmark of Bonsai 2 27B PQ2_0 on one Apple Mac mini M4 Pro with 64 GB unified memory. Prompt processing pp512 measured 126.89 +/- 0.02 tokens/second and token generation tg128 measured 20.53 +/- 0.01 tokens/second.
Setup is llama.cpp PrismML fork (tag prism-b10709-9a9394a) with PQ2_0 GGUF and default F16 KV cache, 10 threads, -ngl 99, flash attention enabled, Metal/BLAS backend. The two figures are separate synthetic llama-bench tests with three repetitions each, not a paired chat request.
Caveats: Spotlight indexing consumed approximately 1.5-2 CPU cores throughout the runs; an unrelated 664 MB embedding model reappeared between runs; no swap pressure was observed. The 6.70 GiB GGUF size is a file size, not measured resident or peak RAM. Peak RAM, configured maximum context, exact weight revision, binary hash, and exact measurement date are unknown.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 27.33 tok/s decode on an M4 Max 128GB.
Setup is llama.cpp (current main, 62acc89) with UD-Q3_K_XL GGUF at 32768 context, -ngl 99, single stream. The MTP head is not exported in any GGUF of this model, so no speculative decoding.
On the same box an MLX pack with MTP reached 90.07 tok/s and 45.01 tok/s with MTP off. The GGUF is 90 GB of weights versus 46.9-49.1 GB resident for the MLX pack. Quants are not matched and the MLX pack is an abliterated fine-tune, so this is a throughput comparison only. LM Studio cannot load the model on either its GGUF or MLX runtime.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5:9b at 17.42 tokens/s on an Apple M4 Mac mini with 24GB unified memory.
Setup is Ollama 0.33.0 with Q4_K_M at 4096 tokens of context, single request, no KV-cache quantization reported.
Generation speed is the median of 3 trials (range 17.33-17.57 t/s), measured with a ~88-token prompt and output capped at 300 tokens; model load time 2.3s, first-content latency 2.9s cold and 0.64s warm.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5-35B-A3B at 49.3 t/s on an M4 Pro Mac mini with 64GB unified memory.
Setup is mlx_lm with 8-bit weights, single request, 512-token context and 512 output tokens.
Figure is transcribed from SiliconBench (arXiv:2609.19169v1, Table 4), a single-node baseline, not a multi-node result; the 512-token context is the tested workload, not a maximum. Quantization scheme, runtime version, GPU core count, KV-cache precision and sampling settings are not established for this row.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B Uncensored at 25.48 tok/s with DFlash2 speculative decoding on an M4 Pro 48GB, a 2.06x speedup over the 12.35 tok/s baseline.
Setup is dflash 0.1.0 on MLX with a 4-bit MLX target model and a 4-bit draft, block size 5, greedy decoding on HumanEval with 20 samples and 256 max new tokens.
Mean acceptance length was 4.87 tokens with 94.9% of blocks accepted in full. An initial 3-sample smoke test gave 13.76 tok/s baseline and 29.11 tok/s with DFlash. Weights and caches were on an external HDD; the 825.84s wall time includes model loading and is not generation throughput.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Bonsai 2 27B (Ternary) at 34.5 t/s generation on an Apple Mac Studio M4 Max 128GB.
Setup is llama.cpp (Prism fork build 10683) with PQ2_0 ternary GGUF, Metal, flash attention on, -ngl 999, 131072 configured context, single request, temperature 0, no speculative decoding. The 7.2 GB figure is the model file size, not measured peak RAM.
The 34.5 t/s is the arithmetic mean of three prose runs (35.49, 34.03, 33.87 t/s) over 2097 prompt tokens and 320 output tokens; a related code workload averaged 34.3 t/s. Prefill is not reported because TTFT includes request overhead. KV-cache type, peak memory, CPU core count and power draw are unknown.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports the AtomicChat Qwen3.8-Flash-Next GGUF quant on an M4 Max 128GB Mac Studio with cold-start prefill of about 500 t/s; no decode speed is given.
Setup is llama.cpp (qwen4exp branch) with the AtomicChat GGUF, whose PLE n-gram table is paged from its file via mmap, so the model takes about 65GB of RAM instead of 106GB.
For comparison, on oMLX offloading PLE to SSD slows prefill from 600 t/s to 180 t/s, and an open oMLX PR makes that cold prefill almost 3 times faster.