M1 Max 64GB reports

generation:

51.9 tokens/s

prompt processing (prefill):

491.7 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 51.9 t/s generation and 491.7 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ4_XS and Metal, ngl 99, using the author's locally converted weights. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

53.2 tokens/s

prompt processing (prefill):

520.6 tokens/s

quant:

IQ4_NL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 53.2 t/s generation and 520.6 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ4_NL and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

61.4 tokens/s

prompt processing (prefill):

536.4 tokens/s

quant:

Q4_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 61.4 t/s generation and 536.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q4_0 and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output chat request; TTFT was not measured.

generation:

52.0 tokens/s

prompt processing (prefill):

474.3 tokens/s

quant:

Q4_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 52 t/s generation and 474.3 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q4_K_S and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

49.2 tokens/s

prompt processing (prefill):

469.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 49.2 t/s generation and 469.9 t/s prompt processing on an M1 Max Mac Studio with 64GB unified memory. Setup is llama.cpp b2296 with Q4_K_M and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

57.2 tokens/s

prompt processing (prefill):

534.1 tokens/s

quant:

Q4_1 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 57.2 t/s generation and 534.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q4_1 and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

40.3 tokens/s

prompt processing (prefill):

422.9 tokens/s

quant:

Q5_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 40.3 t/s generation and 422.9 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q5_K_S and Metal, ngl 99, on one GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

39.7 tokens/s

prompt processing (prefill):

428.6 tokens/s

quant:

Q5_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 39.7 t/s generation and 428.6 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q5_K_M and Metal, ngl 99, on one GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

41.9 tokens/s

prompt processing (prefill):

455.7 tokens/s

quant:

Q5_1 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 41.9 t/s generation and 455.7 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q5_1 quantization and Metal backend at ngl 99, using one GPU. The figures come from separate llama-bench phases, PP512 and TG128, not a single chat request; TTFT was not measured.

generation:

39.0 tokens/s

prompt processing (prefill):

444.3 tokens/s

quant:

Q6_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 39 t/s generation and 444.3 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q6_K and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

40.4 tokens/s

prompt processing (prefill):

547.3 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 40.4 t/s generation and 547.3 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q8_0 and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

23.2 tokens/s

prompt processing (prefill):

603.1 tokens/s

quant:

F16 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 23.2 t/s generation and 603.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with F16 weights and Metal, ngl 99, on one participating GPU. The figures are separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

12.6 tokens/s

prompt processing (prefill):

549.3 tokens/s

quant:

F32 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 12.6 t/s generation and 549.3 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with F32 weights and Metal backend at ngl 99, using one GPU. The figures come from separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

12.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 12 t/s on an M1 Max 64GB. Setup is Ollama with Q4_K_M and a 65,536-token context, one parallel request. The 12.0 tok/s is the second 6,872-token prompt run, not an aggregate; separate retrieval tests report prefill of roughly 90 tok/s at about 1,900 and 7,400 tokens, about 83 at 22,200, and 71-74 at 55,600.

generation:

64.4 tokens/s

prompt processing (prefill):

645.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-Coder 30B-A3B at 64.4 t/s generation and 645 t/s prompt processing on an M1 Max 64GB. Setup is llama.cpp with Q4_K_M, measured via llama-bench on Metal. The figure is a native prefill/decode rate, not a TTFT-derived estimate; exact test lengths, context, batch settings and KV precision are not given.

generation:

0.9 tokens/s

quant:

IQ2_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports IQ2_XS quantized 7B model at 0.95 t/s on an M1 Max 64GB, with 25 tokens generated in 26.30 seconds at context 1523. Setup is llama.cpp with IQ2_XS quantization; the user also mentions running Mixtral 8x7B fine-tunes at Q5_K_M getting 8-15 t/s. User also reports an IQ2_XS 120B model taking about 9 minutes for prompt evaluation and roughly 0.1 t/s generation, and questions whether the slow IQ quant performance on Apple Silicon is expected or a bug.

generation:

33.1 tokens/s

quant:

4-bit group-64 (MLX)

kv:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B at 33.1 tok/s decode on an M1 Max 64GB, comparing Splash, MTPLX M1 fork, MTPLX upstream, oMLX, and TensorFold across 2K to 256K context lengths. Setup is Splash 1.1.0-m1 with 4-bit group-64 MLX weights and 8-bit KV cache, speculative decoding enabled, single stream. Splash decodes fastest up to 64K (33.1 tok/s at 2K, 24.1 at 64K) and uses the least memory (26.2 GB peak at 128K). The MTPLX M1 fork had the shortest whole 3-turn conversation at every length (15% shorter than Splash at 128K) but peaked at 43.5 GB. Upstream MTPLX drops to 5.0 tok/s at 128K. The user forked MTPLX and wrote the benchmark.

generation:

44.0 tokens/s

prompt processing (prefill):

328.0 tokens/s

quant:

Q2_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next Q2_0 at 44.0 tok/s decode and 328 tok/s prefill at 4K context on a 2021 M1 Max with 64 GB unified memory. Setup is antirez's ds4 engine (forked for Apple Silicon) with Q2_0 GGUF weights, MTP speculative decoding on, single request, model fully in unified memory. Decode holds 38.8 tok/s at 256K and 35.4 tok/s at 398K; prefill falls from 328 to 292 tok/s over the same range. IQ3_XXS (44 GiB) gives 35.2 tok/s at 4K and 31.7 at 259K. Sustained five-prompt loop for 5 minutes: 43.2 tok/s first quarter, 43.1 last, 73.1 C, 0.70 J per token. In OpenCode sessions (162 requests, context up to 371K) median decode is 44.7 tok/s under 64K and 38.1 at 128-192K. User compares against a Splash port of Qwen3.8-27B (14 tok/s at 128K+ vs 38) and a DGX Spark running vLLM NVFP4 + MTP (36.8 tok/s prose, 45.8 code). DeepSeek V4 Flash (81 GiB Q2, experts streamed from SSD) runs at 11.8 tok/s at 4K on the same Mac.

generation:

144.0 tokens/s

quant:

4-bit

kv:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 144 tok/s on an M1 Max 64GB, up from 99 tok/s with the previous Splash build. Setup is Inco Splash 1.0.2-m1.1 with 4-bit weights, an 8-bit KV cache and DFlash speculative decoding with 8 draft tokens; time to first token for a 38K prompt is 72 s, down from 106 s. In a head-to-head on the same Mac with Qwen3.8-27B, Splash decodes at 41.5 tok/s versus 25.9 for MTPLX and 19.8 for oMLX, at 75.3 C and 1.06 J per token; prefill at 44K tokens is 102 tok/s against 125-127 tok/s for the MLX engines.

generation:

32.8 tokens/s

prompt processing (prefill):

101.0 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 32.8 t/s average on an M1 Max 64GB MacBook Pro, using the Splash-M1 engine with a 4-bit MLX quant. Setup is the prebuilt splash-m1 1.0.2-m1 with incoai Splash packages, compared against oMLX 0.6.4. Prefill on an 8K prompt reached 101 t/s. Four parallel requests totalled 62.3 t/s. Splash was about 2x faster than the 4-bit MLX quant on code, math and parallel requests, but slower on prose (18.3 vs 16.1 t/s) and 32K prompts. Draft acceptance was about 94% on code but only 19% on prose. Time to first token at 32K was about 400 s. Accuracy matched the 4-bit quant on the user's test suites.