Local model performance on your hardware

generation:

23.9 tokens/s

prompt processing (prefill):

119.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.8 27B at 23.9 t/s generation and 119.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp (llama-server 0.4.0-dev, Metal) with Q4_K_M, --ctx-size 4096, --parallel 1, -b 4096, -ub 1024, flash attention on, no prompt cache, and speculation disabled. The headline Quick Chat run used 87 prompt tokens and 96 output tokens, with TG 23.88856 t/s and PP 119.88656 t/s as medians of three measured requests after one discarded warmup.

generation:

76.4 tokens/s

prompt processing (prefill):

319.9 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.6 35B-A3B at 76.4 t/s generation and 319.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with UD-Q4_K_XL, one Mac Studio with 40-core GPU and 64 GB unified memory. The headline Quick Chat run used 35 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not filled 4K or long-context benchmarks.

generation:

79.9 tokens/s

prompt processing (prefill):

91.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.6 35B-A3B at 79.9 t/s generation and 91 t/s prompt processing on an M4 Max with 64GB unified memory. Setup is ZINC on Metal with the UD-Q4_K_XL GGUF, speculative decoding disabled, context auto-sized from device memory. The headline Quick Chat run used 36 prompt tokens and 96 output tokens; the three rate samples span 78.71 to 79.95 t/s for generation and 90.1 to 92.7 t/s for prefill.

generation:

65.5 tokens/s

prompt processing (prefill):

305.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.5 9B at 65.5 t/s generation and 305.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with Q4_K_M, 4096 context, flash attention on, and prompt caching disabled. The headline Quick Chat run used 45 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not long-context benchmarks.

generation:

67.1 tokens/s

prompt processing (prefill):

546.3 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.5 9B at 67.1 t/s generation and 546.3 t/s prompt processing on an M4 Max 64GB. Setup is ZINC with Metal, Q4_K_M, speculative decoding disabled, and an auto-sized context; the run used 48 prompt tokens and 96 output tokens. The stored 67.1 t/s is the median of three samples, with the source's own median listed as 67.05 t/s over a 67 to 67.34 t/s range.

generation:

22.4 tokens/s

prompt processing (prefill):

62.7 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Gemma 4 31B at 22.4 t/s generation and 62.7 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with Q4_K_M, 4096 context, flash attention on, and one Mac Studio with a 40-core GPU. The headline Quick Chat run used 52 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not long-context benchmarks.

generation:

20.8 tokens/s

prompt processing (prefill):

123.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Gemma 4 31B at 20.8 t/s generation and 123.9 t/s prompt processing on an M4 Max 64GB. Setup is ZINC on Metal with Q4_K_M, serial single-device inference with speculative decoding disabled; configured context is unknown. The headline Quick Chat run used 49 prompt tokens and 96 output tokens, with TG 20.82[20.64,21.14] and PP 123.9[119.2,124.4] t/s as medians of three runs; these are short inputs, not long-context benchmarks.

generation:

81.7 tokens/s

prompt processing (prefill):

379.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Gemma 4 26B-A4B at 81.7 t/s generation and 379.6 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with UD-Q4_K_M, one Mac Studio with a 40-core GPU and 64 GB unified memory. The figures are medians of three measured nonstreaming requests after one discarded warmup, with 52 prompt and 96 output tokens, and speculation disabled.

generation:

68.4 tokens/s

prompt processing (prefill):

303.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Gemma 4 26B-A4B at 68.4 t/s generation and 303.6 t/s prompt processing on an M4 Max 64GB. Setup is ZINC with UD-Q4_K_M, Metal, speculative decoding disabled, and unknown context length. The headline Quick Chat run used 49 prompt tokens and 96 output tokens, with TG 68.44[67.17,68.66] and PP 303.6[299.2,318.6] t/s over three measured invocations.

generation:

50.9 tokens/s

prompt processing (prefill):

511.9 tokens/s

quant:

MLX affine 4-bit, group 64 (MLX)

User reports Grug-12B at 50.9 t/s generation and 511.9 t/s prompt processing on an M4 Max with 64 GB unified memory. Setup is oMLX with MLX affine 4-bit, group 64, at a pp8192/tg128 workload, with 8.26 GB peak memory. The same row lists TTFT 16003.6 ms, TPOT 19.80 ms and E2E 18.540 s; the 8192-token prompt is a workload label, not a configured maximum context.

generation:

103.9 tokens/s

prompt processing (prefill):

1807.2 tokens/s

quant:

MLX affine 4-bit g64; gate modules 8-bit (MLX)

User reports Grug-35B-A3B at 103.9 t/s generation and 1807.2 t/s prompt processing on an M4 Max 64GB. Setup is oMLX with MLX affine 4-bit group 64, gate modules at 8-bit, at 1024 prompt and 128 generated tokens. The run is single-request, with 566.6 ms TTFT, 9.70 ms TPOT, 1.805 s end-to-end and 19.35 GB peak memory; warm-up, repetition count and KV precision are not stated.

generation:

41.5 tokens/s

prompt processing (prefill):

621.3 tokens/s

quant:

IQ2_XXS gate/up + Q2_K down; BF16 n-grams on SSD (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.8 Flash-Next at 41.5 t/s generation and 621.3 t/s prefill on an M4 Max 64GB. Setup is ds4 with IQ2_XXS gate/up + Q2_K down; BF16 n-grams on SSD, one request, speculative decoding off. First row (2,048-token prompt) of pswai's 64gb-mac-llm-bench, measured 2026-09-28. A source-reported measurement, not independently reproduced.

generation:

21.3 tokens/s

prompt processing (prefill):

143.7 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

User reports Qwen3.8-27B at 21.3 t/s generation and 143.7 t/s prefill on an M4 Max 64GB Mac Studio. Setup is ZINC with Metal backend, GGUF Q4_K_M, 48 prompt tokens and 96 generated tokens. Figures are source-published medians from three CLI invocations after one discarded warmup. No speculative decoding is reported. The source warns that 16 of 20 Metal rows have different prompt-token counts between ZINC and llama.cpp, and this submission makes no matched-workload comparison claim.

generation:

82.5 tokens/s

quant:

4-bit

User reports Qwen3.5-9B at 82.53 tokens/second generation on a Mac Studio M4 Max with 64GB unified memory. Setup is vllm-mlx with a 4-bit MLX quant, measured by llama-benchy over 2048 prompt tokens and 32 generated tokens across three runs. A generation peak of 87.99 tokens/second and a TTFT of 2.41 seconds are also reported. Exact quantization scheme, model artifact, runtime versions, configured context limit, concurrency, speculative decoding status and resident memory usage are not stated. The 64GB is system memory capacity, not measured memory use.

generation:

52.6 tokens/s

prompt processing (prefill):

762.0 tokens/s

quant:

iQ-MLX 3.3bpw (MLX)

User reports Qwen3.8-Flash-Next at 52.6 tok/s decode and 762 tok/s prefill on an M4 Max 64GB. Setup is mlx-serve with an iQ-MLX 3.3bpw pack, 64k context, 52 GB resident; the 32 GB n-gram table is mmapped and not resident. The same run measured the mixed-4-8bit pack at 55.5 tok/s decode, 754 tok/s prefill and 278 ms TTFT; MTP is in the pack but default-off and not re-measured.

generation:

11.0 tokens/s

User reports DeepSeek-V4-Flash (284B) at about 11 t/s on a 64GB MacBook. The model is 165GB on disk and does not fit in RAM, so it runs via SSD streaming with a 2-bit file. Quality matches official perplexity (6.1250 vs 6.1262). Higher quality settings give 1-2 t/s. An 8GB cache gave 2.04 t/s versus 1.23 t/s with 32GB. Speculative decoding and prefetching did not help.

generation:

36.0 tokens/s

prompt processing (prefill):

517.9 tokens/s

quant:

AD-3.84bpw-M64 (GGUF)

User reports a 85 GB GGUF quant of Qwen 3.8 Flash Next running on a 64 GB MacBook with the ngram table offloaded to SSD, at 517.9 t/s prefill and 36 t/s decode. The user notes the quant quality is far from perfect and better versions are planned.