M5 Pro 24GB

reported speed:

73.9 tokens/s generation

quant:

IQ3_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports decode speed improvements on Qwen3-30B-A3B with IQ3_XXS quantization on an M5 MacBook Pro 24GB. The decode speed increased from 65.6 to 73.9 tok/s using a llama.cpp PR for Metal optimization. Setup uses llama.cpp with IQ3_XXS quantization. The user also tested on an M1 Pro 32GB. The user is looking for testers for Qwen3.8-Flash-Next and provides benchmark commands.

reported speed:

125.8 tokens/s generation · 363.9 tokens/s prompt processing

quant:

4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks DeepSeek-Coder-V2-Lite-Instruct at 125.8 tok/s generation and 363.9 tok/s prompt processing on an Apple M5 Pro with 24GB unified memory. Setup is MLX with a 4bit quant (8.2GB) on a single machine, running a fixed coding task capped at 1500 output tokens. The model produced correct working code; the user notes that 24GB has a hard ceiling around 20GB models and that background downloads measurably affect tok/s.

reported speed:

4.8 tokens/s generation

quant:

2-bit dynamic

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports DeepSeek-V4-Flash 284B-A13B at 4.8 tok/s on a 24 GB M5 Pro with about 5.3 GB of memory in use. Setup is Mference, an open-source engine. The user also runs Gemma 4 26B-A4B and Qwen 3.6 35B-A3B.