M4 Max 128GB reports

generation:

91.8 tokens/s

prompt processing (prefill):

1277.1 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron 3 Nano at 91.8 t/s generation and 1277.1 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with Q4_K_M, flash attention and F16 K/V, single sequence, GPU layers 99. The headline is depth 0; at prefix depth 4096 generation is 90.27 t/s and at 16384 it is 86.07 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

28.5 tokens/s

prompt processing (prefill):

253.9 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse-Glimmer-30B at 28.5 t/s generation and 253.9 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q4_K_XL and F16 K/V cache, flash attention, 99 GPU layers requested, single sequence. Figures are separate 512-token prefill and 128-token generation microbenchmarks at prefix depth 0; at depth 4096 generation is 27.82 t/s and at 16384 it is 27.07 t/s.

generation:

238.6 tokens/s

prompt processing (prefill):

4177.1 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ling-mini-2.0 at 238.6 t/s generation and 4177.1 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with Q4_K_M, flash attention and F16 K/V, single sequence, 99 GPU layers requested. Figures are depth-0 microbenchmarks: PP512 and TG128 separately, not an end-to-end run. At prefix depth 4096 generation falls to 213.24 t/s and at 16384 to 172.28 t/s.

generation:

123.1 tokens/s

prompt processing (prefill):

1577.0 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-20b at 123.1 t/s generation and 1577 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with MXFP4 GGUF, flash attention, F16 K/V, requested GPU layers 99, single sequence, five repetitions. The headline is depth 0; at prefix depth 4096 generation falls to 117.20 t/s and at 16384 to 105.36 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

88.1 tokens/s

prompt processing (prefill):

1408.1 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 88.1 t/s generation and 1408.1 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with UD-Q4_K_M, flash attention and F16 K/V cache, single sequence, GPU layers 99. The headline is depth 0; at prefix depth 4096 generation is 79.77 t/s and at 16384 it is 75.49 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

85.4 tokens/s

prompt processing (prefill):

809.5 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-120b at 85.4 t/s generation and 809.5 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with MXFP4 GGUF, flash attention, F16 K/V, requested GPU layers 99, single sequence, five repetitions. Headline is depth 0; at prefix depth 4096 generation falls to 81.13 t/s and at 16384 to 72.80 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

54.5 tokens/s

prompt processing (prefill):

589.0 tokens/s

quant:

Q4_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 54.5 t/s generation and 589 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with Q4_0, flash attention and F16 K/V, single sequence, 99 GPU layers requested. The headline is depth 0; at prefix depth 4096 generation is 50.62 t/s and at 16384 it is 47.37 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

52.7 tokens/s

prompt processing (prefill):

683.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Falcon-H1 7B-Instruct at 52.7 t/s generation and 683 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers, and no speculative drafter. The figures are separate synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the JSON run measured 682.958202 ± 4.356899 t/s and 52.696786 ± 0.186655 t/s.

generation:

86.4 tokens/s

prompt processing (prefill):

1297.7 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Falcon-H1 3B-Instruct at 86.4 t/s generation and 1297.7 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers, and 12 CPU threads. The figures are separate synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the JSON run measured 1297.664754 ± 4.914553 t/s and 86.354246 ± 0.010372 t/s.

generation:

147.5 tokens/s

prompt processing (prefill):

2535.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Falcon-H1 1.5B-Instruct at 147.5 t/s generation and 2535.2 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers and 12 CPU threads. The figures are standalone synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the source's exact means are 2535.20394 ± 4.882498 t/s and 147.525027 ± 0.873067 t/s.

generation:

141.9 tokens/s

prompt processing (prefill):

2656.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Granite-4.0-H 1B at 141.9 t/s generation and 2656.4 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers, and no speculative drafter. The figures are separate synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the source's exact means are 2656.440968 ± 5.382321 t/s and 141.855767 ± 1.690267 t/s.

generation:

282.5 tokens/s

prompt processing (prefill):

8470.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Granite-4.0-H 350M at 282.5 t/s generation and 8470.4 t/s prompt processing on an M4 Max with 128 GiB unified RAM. Setup is llama.cpp build 8680 with Q4_K_M, f16 K/V cache, Flash Attention on, 99 GPU layers and 12 CPU threads. The figures are standalone synthetic PP512 and TG256 tests at depth 0, single sequence, three repetitions each; the JSON run measured 8470.38629 ± 91.887773 t/s and 282.459008 ± 1.376164 t/s.

generation:

12.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 12 t/s generation on an M4 Max 128GB, with the source's 128K benchmark point given as 11.98 tokens/s. Setup is llama.cpp with UD-Q4_K_XL and f16 K/V cache, all layers offloaded, flash attention on, batch 1024 and microbatch 256. The figure is generation after cache prefilling, not prompt processing or TTFT; the source's f16 KV series is 16.62 t/s at 64K, 14.93 at 81K and 11.98 at 128K.

generation:

15.6 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.5-122B-A10B at 15.6 t/s generation on an M4 Max 128GB. Setup is llama.cpp with Q4_K_M and f16 K/V cache, all layers offloaded, batch 1024 and flash attention on. The figure is the 128K benchmark point after cache prefilling, not newly processed input tokens; the source's f16 KV series is 24.51 t/s at 64K, 22.25 at 81K and 15.59 at 128K, with a reported Metal/VRAM peak of 75.16 GB at 128K.

generation:

26.7 tokens/s

prompt processing (prefill):

269.0 tokens/s

quant:

int4hu block32 symmetric

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse at 26.7 t/s generation and 269 t/s prompt processing on an M4 Max 128GB. Setup is Apple's coreai-pipelined GPU engine with int4hu block32 symmetric quantization, text decoder only, 16.35 GB bundle size. The source table pairs 269.0 tok/s prompt with 26.69 tok/s decode under llm-benchmark at 512 prompt / 1024 generation, 3 trials; aggregation, cache state and batch configuration for that row are not stated.

generation:

6-7 tokens/s

prompt processing (prefill):

273-287 tokens/s

quant:

Q2_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports running Qwen3.8-Flash-Next, a 125B-A6B MoE, on a 128 GB M4 Max MacBook Pro with a new unofficial MLX engine called strata-mlx that reads Strata's GGUF files directly. Setup uses the model's built-in draft head for speculative decoding. The Q2_0 file is 66 GB and the IQ3_S file is 84 GB. On a Mac capped to 16 GB of memory the 66 GB file runs at 6-7 tokens/second, on a 24 GB machine at 19-23, and on a 32 GB machine at 28-47, with experts streamed from SSD. A 1,537-token prompt runs at 273-287 tokens/second in that mode. The user notes a 48 GB Mac holds the Q2_0 file whole and never touches the disk, and that speed is not guaranteed on smaller machines.

generation:

31.7 tokens/s

quant:

IQ2_XXS/Q2_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4 Flash Vision-Exp at 31.7 tok/s decode on a Mac Studio M4 Max 128GB. Setup is ds4 (DwarfStar) with IQ2_XXS/Q2_K GGUF, DSpark speculative decoding at confidence 0.6, and a 32768 context length. Thinking-on pagoda run reached 29.84 tok/s over 26,415 completion tokens; 1M context loaded at 97.15 GiB planned total with 32.0 tok/s decode. User notes DSpark adds roughly 10-20% over no speculation.

generation:

6.2 tokens/s

prompt processing (prefill):

2.6 tokens/s

quant:

Q4_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash Q4_K at 6.21 t/s decode and 2.60 t/s prefill on an M4 Max 128 GiB via SSD streaming. Setup is ds4 with Q4_K GGUF, 4096 context, 128-token protocol, cold and warm runs. The 177.77 GiB model streams from SSD; GPU time is only ~24 ms of ~161 ms per token (15%), with host-side per-layer expert-cache bookkeeping dominating. Removing the synchronous selected-id readback changed nothing (6.22/6.24 t/s, byte-identical output). MTP was ~15% slower (5.28/5.22 t/s). Cache budgets of 48/56/85 GB gave 5.3/5.3/4.6 t/s; the automatic 70.1 GiB budget was best. All tested flags stayed within noise of 6.2 t/s.

generation:

27.3 tokens/s

quant:

UD-Q3_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next at 27.33 tok/s decode on an M4 Max 128GB. Setup is llama.cpp (current main, 62acc89) with UD-Q3_K_XL GGUF at 32768 context, -ngl 99, single stream. The MTP head is not exported in any GGUF of this model, so no speculative decoding. On the same box an MLX pack with MTP reached 90.07 tok/s and 45.01 tok/s with MTP off. The GGUF is 90 GB of weights versus 46.9-49.1 GB resident for the MLX pack. Quants are not matched and the MLX pack is an abliterated fine-tune, so this is a throughput comparison only. LM Studio cannot load the model on either its GGUF or MLX runtime.

generation:

34.5 tokens/s

quant:

PQ2_0 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Bonsai 2 27B (Ternary) at 34.5 t/s generation on an Apple Mac Studio M4 Max 128GB. Setup is llama.cpp (Prism fork build 10683) with PQ2_0 ternary GGUF, Metal, flash attention on, -ngl 999, 131072 configured context, single request, temperature 0, no speculative decoding. The 7.2 GB figure is the model file size, not measured peak RAM. The 34.5 t/s is the arithmetic mean of three prose runs (35.49, 34.03, 33.87 t/s) over 2097 prompt tokens and 320 output tokens; a related code workload averaged 34.3 t/s. Prefill is not reported because TTFT includes request overhead. KV-cache type, peak memory, CPU core count and power draw are unknown.