Local model performance on your hardware

generation:

16.9 tokens/s

quant:

Q4_K_M (GGUF)

User benchmarks 48 local models as agents on an M1 Max 64GB, with qwen3.8-27b Q4_K_M reaching 16.9 tok/s at 64k context. Setup is LM Studio with GGUF weights, 15.7 GB of memory at 64k context, and 16 multi-step tool tasks plus 10 real-world failure cases run 3 times each. The model scored 99% quality, 100% agent and 98% on the real cases with 0 serious slips in 30 runs at 51 s per task. Ornith-1.5-35B-A3B is the faster pick at 52.4 tok/s with 94% quality and 98% agent. Prompt cache made a follow-up turn on the same long agent prefix 152x faster than cold start on GGUF models (median of 41).

generation:

45.0 tokens/s

User reports about 45 tokens/sec on a 7B Llama model in MLX on an M1 Max 64GB, but finds 7B too limited for most tasks. Setup is LM Studio; the user also tried a 32B Qwen model there and hit errors. User asks which models suit offline research, legal and medical analysis, light coding, CAD/PCB, Blender and image generation, and is looking to move up to 32B, 70B and 120B models.

generation:

33.0 tokens/s

quant:

QAT 4-bit (MLX)

User reports 33 t/s generation. Source evidence (verbatim): Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ # Mac Studio M1Max64GB native generation after Gemma3QAT MLXdtype conversion Original author article March20,2026. Measured Mac Studio AppleM1 Max 64GB unified RAM,24GPUcores; CPUcorecount unknown. Gemma3-12B-it QAT4-bit MLX in LMStudio: author states generation increases from 28 to 33 tok/s after converting nonquantized dtype frombf16 tofloat16. No averaging or calculatedspeed used. Tested sourcequant repository verified https://huggingface.co/mlx-community/gemma-3-12b-it-qat-4bit ; convertedartifact generated locally with mlx_lm.convert --dtype float16, not a separate verifiedHFartifact. Testedrevision/runtimeversion/prompt/outputcounts/power/KVquant/speculation/residentRAM unknown. Separate reportedprefilltimes at655tokens6.2s→3.7s;8K114.4s→68.9s; do not derive nativeprefillT/s from approximate8K. Other charts report effectiveoutputtokens/totalrequesttime includingprefill, not generation; keep separate. Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ . Rawlogs linkGitHub and were not fetched under sourceaccessstop. CurrentHFcard contains unrelated-looking multimodalmetadata; no testedhistoricalrevision established, so preserve artifact caveat. Selected run: locally converted float16 nonquantized dtype artifact, native generation 33 tok/s. Baseline before conversion is a separate run at 28 tok/s. Prompt processing speed unknown; do not derive it from prefill times.