M1 Max 64GB

generation:

18.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next 125B at ~18 tok/s on an M1 Max 64GB with the Espetero server, an inference server for Apple silicon written in C++20 and Metal. Setup streams experts from the SSD; the server loads MLX and GGUF files and serves OpenAI, Anthropic and Ollama APIs. DeepSeek-V4.1-Flash 552B runs at ~10 tok/s on a 128 GB Mac Studio, described as experimental.

generation:

16.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks 48 local models as agents on an M1 Max 64GB, with qwen3.8-27b Q4_K_M reaching 16.9 tok/s at 64k context. Setup is LM Studio with GGUF weights, 15.7 GB of memory at 64k context, and 16 multi-step tool tasks plus 10 real-world failure cases run 3 times each. The model scored 99% quality, 100% agent and 98% on the real cases with 0 serious slips in 30 runs at 51 s per task. Ornith-1.5-35B-A3B is the faster pick at 52.4 tok/s with 94% quality and 98% agent. Prompt cache made a follow-up turn on the same long agent prefix 152x faster than cold start on GGUF models (median of 41).

generation:

62.0 tokens/s

prompt processing (prefill):

464.0 tokens/s

quant:

4-bit group-64 (MLX)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.6-35B-A3B on an M1 Max 64GB, reporting 464 tok/s prefill and 62 tok/s decode at a 128K prompt with the MTPLX M1 fork. Setup is the MTPLX M1 fork v2.12.2-m1.11 with mlx-community 4-bit group-64 weights plus an MTP head, 8-bit paged affine KV cache, and speculative decoding on. Splash-M1 1.3.2-m1.1 reached 302 tok/s prefill and 41 tok/s decode at 128K, and oMLX 385 tok/s prefill and 33 tok/s decode. The fork was faster from 8K up on agent logs, 1.46x prefill and 1.5x decode at 128K, while Splash-M1 used less memory (23.5 GB vs 28.1 GB at 128K). The same post also compares Qwen3.8-27B, where the fork reached 132 tok/s prefill and 25.0 tok/s decode at 128K. Only the 2K, 32K and 128K agent-log cells were run three times; the rest are single runs.

generation:

45.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports about 45 tokens/sec on a 7B Llama model in MLX on an M1 Max 64GB, but finds 7B too limited for most tasks. Setup is LM Studio; the user also tried a 32B Qwen model there and hit errors. User asks which models suit offline research, legal and medical analysis, light coding, CAD/PCB, Blender and image generation, and is looking to move up to 32B, 70B and 120B models.

generation:

12.0 tokens/s

prompt processing (prefill):

121.0 tokens/s

quant:

oQ4e (MLX)

kv:

6bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 121 prompt t/s and 12 generation t/s on an M1 Max 64GB. Setup is MLX with oQ4e quant, 6-bit KV cache, 50% expert offload and MTP on. A later fp16 build of the same model gave OOM errors and, at 20% expert offload, only 90 prompt t/s and 6 generation t/s; the user asks whether the model or the settings are at fault.

generation:

33.0 tokens/s

quant:

QAT 4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 33 t/s generation. Source evidence (verbatim): Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ # Mac Studio M1Max64GB native generation after Gemma3QAT MLXdtype conversion Original author article March20,2026. Measured Mac Studio AppleM1 Max 64GB unified RAM,24GPUcores; CPUcorecount unknown. Gemma3-12B-it QAT4-bit MLX in LMStudio: author states generation increases from 28 to 33 tok/s after converting nonquantized dtype frombf16 tofloat16. No averaging or calculatedspeed used. Tested sourcequant repository verified https://huggingface.co/mlx-community/gemma-3-12b-it-qat-4bit ; convertedartifact generated locally with mlx_lm.convert --dtype float16, not a separate verifiedHFartifact. Testedrevision/runtimeversion/prompt/outputcounts/power/KVquant/speculation/residentRAM unknown. Separate reportedprefilltimes at655tokens6.2s→3.7s;8K114.4s→68.9s; do not derive nativeprefillT/s from approximate8K. Other charts report effectiveoutputtokens/totalrequesttime includingprefill, not generation; keep separate. Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ . Rawlogs linkGitHub and were not fetched under sourceaccessstop. CurrentHFcard contains unrelated-looking multimodalmetadata; no testedhistoricalrevision established, so preserve artifact caveat. Selected run: locally converted float16 nonquantized dtype artifact, native generation 33 tok/s. Baseline before conversion is a separate run at 28 tok/s. Prompt processing speed unknown; do not derive it from prefill times.

generation:

53.0 tokens/s

prompt processing (prefill):

499.1 tokens/s

quant:

IQ1_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 53 t/s generation and 499.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ1_S quantization and Metal backend at ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

50.5 tokens/s

prompt processing (prefill):

462.1 tokens/s

quant:

IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 50.5 t/s generation and 462.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_XXS and Metal, ngl 99, on one GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

48.4 tokens/s

prompt processing (prefill):

472.2 tokens/s

quant:

IQ2_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 48.4 t/s generation and 472.2 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_XS weights and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

47.3 tokens/s

prompt processing (prefill):

469.5 tokens/s

quant:

IQ2_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 47.3 t/s generation and 469.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_S quant, Metal backend and ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

62.5 tokens/s

prompt processing (prefill):

512.2 tokens/s

quant:

Q2_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 62.5 t/s generation and 512.2 t/s prompt processing on an M1 Max Mac Studio with 64GB unified memory. Setup is llama.cpp b2296 with Q2_K_S and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; the source rates are 512.21 and 62.49 t/s.

generation:

38.4 tokens/s

prompt processing (prefill):

462.4 tokens/s

quant:

IQ2_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 38.4 t/s generation and 462.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_M quant, Metal backend and ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

51.6 tokens/s

prompt processing (prefill):

493.4 tokens/s

quant:

Q2_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 51.6 t/s generation and 493.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q2_K and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

46.4 tokens/s

prompt processing (prefill):

470.0 tokens/s

quant:

IQ3_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 46.4 t/s generation and 470 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ3_XXS and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

47.9 tokens/s

prompt processing (prefill):

464.9 tokens/s

quant:

IQ3_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 47.9 t/s generation and 464.9 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with the IQ3_XS quant, Metal backend and ngl 99, using the author's locally converted weights. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

42.6 tokens/s

prompt processing (prefill):

466.8 tokens/s

quant:

Q3_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 42.6 t/s generation and 466.8 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q3_K_S and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

47.7 tokens/s

prompt processing (prefill):

456.1 tokens/s

quant:

IQ3_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 47.7 t/s generation and 456.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ3_S quant, Metal backend and ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.

generation:

47.7 tokens/s

prompt processing (prefill):

459.5 tokens/s

quant:

IQ3_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 47.7 t/s generation and 459.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ3_M and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

44.2 tokens/s

prompt processing (prefill):

469.6 tokens/s

quant:

Q3_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 44.2 t/s generation and 469.6 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q3_K_M and Metal, ngl 99, on one GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.

generation:

40.9 tokens/s

prompt processing (prefill):

449.5 tokens/s

quant:

Q3_K_L (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 2 7B Chat at 40.9 t/s generation and 449.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q3_K_L and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.