Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next 125B at ~18 tok/s on an M1 Max 64GB with the Espetero server, an inference server for Apple silicon written in C++20 and Metal.
Setup streams experts from the SSD; the server loads MLX and GGUF files and serves OpenAI, Anthropic and Ollama APIs.
DeepSeek-V4.1-Flash 552B runs at ~10 tok/s on a 128 GB Mac Studio, described as experimental.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-usecoding
User benchmarks 48 local models as agents on an M1 Max 64GB, with qwen3.8-27b Q4_K_M reaching 16.9 tok/s at 64k context.
Setup is LM Studio with GGUF weights, 15.7 GB of memory at 64k context, and 16 multi-step tool tasks plus 10 real-world failure cases run 3 times each.
The model scored 99% quality, 100% agent and 98% on the real cases with 0 serious slips in 30 runs at 51 s per task. Ornith-1.5-35B-A3B is the faster pick at 52.4 tok/s with 94% quality and 98% agent. Prompt cache made a follow-up turn on the same long agent prefix 152x faster than cold start on GGUF models (median of 41).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User benchmarks Qwen3.6-35B-A3B on an M1 Max 64GB, reporting 464 tok/s prefill and 62 tok/s decode at a 128K prompt with the MTPLX M1 fork.
Setup is the MTPLX M1 fork v2.12.2-m1.11 with mlx-community 4-bit group-64 weights plus an MTP head, 8-bit paged affine KV cache, and speculative decoding on. Splash-M1 1.3.2-m1.1 reached 302 tok/s prefill and 41 tok/s decode at 128K, and oMLX 385 tok/s prefill and 33 tok/s decode.
The fork was faster from 8K up on agent logs, 1.46x prefill and 1.5x decode at 128K, while Splash-M1 used less memory (23.5 GB vs 28.1 GB at 128K). The same post also compares Qwen3.8-27B, where the fork reached 132 tok/s prefill and 25.0 tok/s decode at 128K. Only the 2K, 32K and 128K agent-log cells were run three times; the rest are single runs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvisionlong-context
User reports about 45 tokens/sec on a 7B Llama model in MLX on an M1 Max 64GB, but finds 7B too limited for most tasks.
Setup is LM Studio; the user also tried a 32B Qwen model there and hit errors.
User asks which models suit offline research, legal and medical analysis, light coding, CAD/PCB, Blender and image generation, and is looking to move up to 32B, 70B and 120B models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 121 prompt t/s and 12 generation t/s on an M1 Max 64GB.
Setup is MLX with oQ4e quant, 6-bit KV cache, 50% expert offload and MTP on.
A later fp16 build of the same model gave OOM errors and, at 20% expert offload, only 90 prompt t/s and 6 generation t/s; the user asks whether the model or the settings are at fault.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 33 t/s generation.
Source evidence (verbatim):
Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/
# Mac Studio M1Max64GB native generation after Gemma3QAT MLXdtype conversion
Original author article March20,2026. Measured Mac Studio AppleM1 Max 64GB unified RAM,24GPUcores; CPUcorecount unknown. Gemma3-12B-it QAT4-bit MLX in LMStudio: author states generation increases from 28 to 33 tok/s after converting nonquantized dtype frombf16 tofloat16. No averaging or calculatedspeed used. Tested sourcequant repository verified https://huggingface.co/mlx-community/gemma-3-12b-it-qat-4bit ; convertedartifact generated locally with mlx_lm.convert --dtype float16, not a separate verifiedHFartifact. Testedrevision/runtimeversion/prompt/outputcounts/power/KVquant/speculation/residentRAM unknown. Separate reportedprefilltimes at655tokens6.2s→3.7s;8K114.4s→68.9s; do not derive nativeprefillT/s from approximate8K. Other charts report effectiveoutputtokens/totalrequesttime includingprefill, not generation; keep separate. Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ . Rawlogs linkGitHub and were not fetched under sourceaccessstop. CurrentHFcard contains unrelated-looking multimodalmetadata; no testedhistoricalrevision established, so preserve artifact caveat. Selected run: locally converted float16 nonquantized dtype artifact, native generation 33 tok/s. Baseline before conversion is a separate run at 28 tok/s. Prompt processing speed unknown; do not derive it from prefill times.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 53 t/s generation and 499.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ1_S quantization and Metal backend at ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 50.5 t/s generation and 462.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_XXS and Metal, ngl 99, on one GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 48.4 t/s generation and 472.2 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_XS weights and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 47.3 t/s generation and 469.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_S quant, Metal backend and ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 62.5 t/s generation and 512.2 t/s prompt processing on an M1 Max Mac Studio with 64GB unified memory.
Setup is llama.cpp b2296 with Q2_K_S and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one chat request; the source rates are 512.21 and 62.49 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 38.4 t/s generation and 462.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_M quant, Metal backend and ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 51.6 t/s generation and 493.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with Q2_K and Metal, ngl 99, on one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 46.4 t/s generation and 470 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ3_XXS and Metal, ngl 99, on one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 47.9 t/s generation and 464.9 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with the IQ3_XS quant, Metal backend and ngl 99, using the author's locally converted weights.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 42.6 t/s generation and 466.8 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with Q3_K_S and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 47.7 t/s generation and 456.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ3_S quant, Metal backend and ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 47.7 t/s generation and 459.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ3_M and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 44.2 t/s generation and 469.6 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with Q3_K_M and Metal, ngl 99, on one GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 40.9 t/s generation and 449.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with Q3_K_L and Metal, ngl 99, on one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.