Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-usecoding
User benchmarks 48 local models as agents on an M1 Max 64GB, with qwen3.8-27b Q4_K_M reaching 16.9 tok/s at 64k context.
Setup is LM Studio with GGUF weights, 15.7 GB of memory at 64k context, and 16 multi-step tool tasks plus 10 real-world failure cases run 3 times each.
The model scored 99% quality, 100% agent and 98% on the real cases with 0 serious slips in 30 runs at 51 s per task. Ornith-1.5-35B-A3B is the faster pick at 52.4 tok/s with 94% quality and 98% agent. Prompt cache made a follow-up turn on the same long agent prefix 152x faster than cold start on GGUF models (median of 41).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-useagentic
User compares LFM2.5-24B-A2B against Qwen3.6-35B-A3B on 75 multi-step tool-use tasks on an M1 Pro 32GB, with LFM2.5 solving 37/75 (49%) and Qwen 68/75 (91%).
Both ran in llama.cpp b10964 with --jinja at 64k context, LFM2.5 at Q4_K_M (14.5 GB) and Qwen at Q4_K_M (22.3 GB), thinking on.
LFM2.5 was faster per token (~52 vs 34 tok/s) but used about twice the tokens per solved task, 418 s per correct result vs Qwen's 328. In at least 23 of its 38 failures it reported the task as done. The user notes this is 4-bit llama.cpp rather than Liquid's vLLM setup, and that the two tie on BFCLv4.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User benchmarks Qwen3.6-35B-A3B on an M1 Max 64GB, reporting 464 tok/s prefill and 62 tok/s decode at a 128K prompt with the MTPLX M1 fork.
Setup is the MTPLX M1 fork v2.12.2-m1.11 with mlx-community 4-bit group-64 weights plus an MTP head, 8-bit paged affine KV cache, and speculative decoding on. Splash-M1 1.3.2-m1.1 reached 302 tok/s prefill and 41 tok/s decode at 128K, and oMLX 385 tok/s prefill and 33 tok/s decode.
The fork was faster from 8K up on agent logs, 1.46x prefill and 1.5x decode at 128K, while Splash-M1 used less memory (23.5 GB vs 28.1 GB at 128K). The same post also compares Qwen3.8-27B, where the fork reached 132 tok/s prefill and 25.0 tok/s decode at 128K. Only the 2K, 32K and 128K agent-log cells were run three times; the rest are single runs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvisionlong-context
User reports about 45 tokens/sec on a 7B Llama model in MLX on an M1 Max 64GB, but finds 7B too limited for most tasks.
Setup is LM Studio; the user also tried a 32B Qwen model there and hit errors.
User asks which models suit offline research, legal and medical analysis, light coding, CAD/PCB, Blender and image generation, and is looking to move up to 32B, 70B and 120B models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-use
User reports Qwen3.5-35B-A3B at roughly 115-120 tok/s on a Mac Studio M1 Ultra 128GB plus an RTX 3090 24GB over a Thunderbolt 5 eGPU.
Setup is llama.cpp with Q4_K_M, about 17GB of the model on the 3090 and about 5GB on the Mac, with the KV cache quantized to Q8.
The user says the Q8 KV cache lets them run about 200K context entirely in VRAM, and that this model replaced a two-model agentic setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 121 prompt t/s and 12 generation t/s on an M1 Max 64GB.
Setup is MLX with oQ4e quant, 6-bit KV cache, 50% expert offload and MTP on.
A later fp16 build of the same model gave OOM errors and, at 20% expert offload, only 90 prompt t/s and 6 generation t/s; the user asks whether the model or the settings are at fault.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 33 t/s generation.
Source evidence (verbatim):
Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/
# Mac Studio M1Max64GB native generation after Gemma3QAT MLXdtype conversion
Original author article March20,2026. Measured Mac Studio AppleM1 Max 64GB unified RAM,24GPUcores; CPUcorecount unknown. Gemma3-12B-it QAT4-bit MLX in LMStudio: author states generation increases from 28 to 33 tok/s after converting nonquantized dtype frombf16 tofloat16. No averaging or calculatedspeed used. Tested sourcequant repository verified https://huggingface.co/mlx-community/gemma-3-12b-it-qat-4bit ; convertedartifact generated locally with mlx_lm.convert --dtype float16, not a separate verifiedHFartifact. Testedrevision/runtimeversion/prompt/outputcounts/power/KVquant/speculation/residentRAM unknown. Separate reportedprefilltimes at655tokens6.2s→3.7s;8K114.4s→68.9s; do not derive nativeprefillT/s from approximate8K. Other charts report effectiveoutputtokens/totalrequesttime includingprefill, not generation; keep separate. Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ . Rawlogs linkGitHub and were not fetched under sourceaccessstop. CurrentHFcard contains unrelated-looking multimodalmetadata; no testedhistoricalrevision established, so preserve artifact caveat. Selected run: locally converted float16 nonquantized dtype artifact, native generation 33 tok/s. Baseline before conversion is a separate run at 28 tok/s. Prompt processing speed unknown; do not derive it from prefill times.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 69.73 t/s generation and 1172.93 t/s prompt processing.
Source evidence (verbatim):
Source: https://blog.majid.info/llama-bench/utumno.txt
# Mac Studio M1 Ultra 128GB: original Gemma4 E4B Q4_K_M llama-bench log
First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: Mac Studio M1 Ultra 128GB. Gemma4 E4B Q4_K_M GGUF, llama.cpp ff5ef8278 (8763), backend Metal/BLAS. pp512 1172.93 ±0.36 tokens/s; tg128 69.73 ±0.19 tokens/s. 16 CPU threads; recommendedMaxWorkingSetSize 103079.22 MB is a runtime recommendation, not resident memory. Unified system RAM 128GB, GPU core count unknown. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/utumno.txt .
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 70B Chat at 11.1 t/s generation and 68.1 t/s prompt processing on a Mac Studio M1 Ultra with 128GB unified memory.
Setup is llama.cpp build 2164 with Q4_K_M quantization, 512-token context, F16 K and F16 V cache, all 81 layers offloaded to the Metal GPU.
The figures are the runtime's separate prompt-eval and eval counters, not wall-clock request throughput; prompt evaluation was 56 tokens and token evaluation was 333 runs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports gpt-oss 120B MXFP4 at 65.5 t/s generation and 392.4 t/s prompt processing on a Mac Studio M1 Ultra with 128 GB unified memory.
Setup is llama.cpp build 6470 with MXFP4 and F16 K/V cache, one Metal device, all layers offloaded, batch and microbatch 4096, mmap disabled.
The two figures are separate synthetic tests, not one 65536-input/128-output request: prompt processing ran 65536 tokens and generation 128 tokens at depth 0, so generation is not measured at 65K depth.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 53 t/s generation and 499.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ1_S quantization and Metal backend at ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 50.5 t/s generation and 462.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_XXS and Metal, ngl 99, on one GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 48.4 t/s generation and 472.2 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_XS weights and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 47.3 t/s generation and 469.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_S quant, Metal backend and ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 62.5 t/s generation and 512.2 t/s prompt processing on an M1 Max Mac Studio with 64GB unified memory.
Setup is llama.cpp b2296 with Q2_K_S and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one chat request; the source rates are 512.21 and 62.49 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 38.4 t/s generation and 462.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ2_M quant, Metal backend and ngl 99, one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 51.6 t/s generation and 493.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with Q2_K and Metal, ngl 99, on one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 46.4 t/s generation and 470 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with IQ3_XXS and Metal, ngl 99, on one participating GPU.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 47.9 t/s generation and 464.9 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with the IQ3_XS quant, Metal backend and ngl 99, using the author's locally converted weights.
The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Llama 2 7B Chat at 42.6 t/s generation and 466.8 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory.
Setup is llama.cpp b2296 with Q3_K_S and Metal, ngl 99, on one GPU.
The figures are separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.