llamaperf

M5 Pro 24GB reports

6 reports. These are individual setups, not a controlled benchmark. M5 Pro 24GB speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: positiveApple hardware
generation:
61.8 tokens/s
quant:
4bit (MLX)
visionlong-contextagentic

User reports Qwen3.8-27B at 61.8 tok/s on an M5 Pro 24GB Mac, up from 57.3 tok/s with draft-tree speculative decoding off. Setup is the native Rust/Metal JustFit 2.6 engine with stock MLX 4-bit weights and Inco AI's DFlash2 draft, on a 24 GB Apple Silicon machine. The 7-prompt greedy suite also gave 40.9 tok/s on M6 (35.6 chain) and 50.9-51.3 tok/s on M4 Pro with a fixed 8-row tree. On NVIDIA SPEED-Bench at 16K prompts the M5 Pro median was 59.5 tok/s, M6 38.5 tok/s and M4 Pro 48.3 tok/s. Dual 128K+32K requests peaked at 21,563 MiB on M5 Pro.

Nov 22, 2026
Tone: positiveApple hardware
generation:
14.7 tokens/s
prompt processing (prefill):
374.1 tokens/s
quant:
IQ3_XXS (GGUF)
agentic

User benchmarks Qwen3.8-Flash-Next 125B at 14.69 tok/s decode on an M5 Pro 24GB, with 374.15 tok/s prefill on a 6,060-token prompt. Setup is the MoEShift backend with an IQ3_XXS quant (76GB on disk) and SSD offloading, peak process memory 8.5GB. Medians of 3 runs with prompt caching disabled. On the same setup an M4 Pro 24GB reached 9.84 tok/s short-prompt decode, 199.09 tok/s prefill and 9.57 tok/s long-prompt decode; long-prompt inference took 55.1s on the M5 Pro versus 83.9s on the M4 Pro.

Nov 9, 2026

Qwen3.8 27B

M5 Pro 24GB · Inco Splash · 120,000 ctx

Tone: positiveApple hardware
generation:
55.0 tokens/s
prompt processing (prefill):
448.0 tokens/s
quant:
UD-IQ3_XXS (GGUF)
agentictool-use

User reports Qwen3.8-27B at about 55 t/s generation and 448 t/s prefill on an M5 Pro MacBook Pro with 24GB unified memory. Setup is the Inco Splash engine (v1.4.1) with the UD-IQ3_XXS GGUF (10.9GB), capped at 18GB memory for around 120k context. User's own KLD measurements give 89.5% top-1 agreement with mean KLD 0.0542, and they say Safari and Discord still run comfortably alongside it.

Oct 27, 2026
Tone: positiveApple hardware
generation:
73.9 tokens/s
quant:
IQ3_XXS (GGUF)

User reports decode speed improvements on Qwen3-30B-A3B with IQ3_XXS quantization on an M5 MacBook Pro 24GB. The decode speed increased from 65.6 to 73.9 tok/s using a llama.cpp PR for Metal optimization. Setup uses llama.cpp with IQ3_XXS quantization. The user also tested on an M1 Pro 32GB. The user is looking for testers for Qwen3.8-Flash-Next and provides benchmark commands.

Oct 6, 2026
Tone: positiveApple hardware
generation:
125.8 tokens/s
prompt processing (prefill):
363.9 tokens/s
quant:
4bit (MLX)
codingagentic

User benchmarks DeepSeek-Coder-V2-Lite-Instruct at 125.8 tok/s generation and 363.9 tok/s prompt processing on an Apple M5 Pro with 24GB unified memory. Setup is MLX with a 4bit quant (8.2GB) on a single machine, running a fixed coding task capped at 1500 output tokens. The model produced correct working code; the user notes that 24GB has a hard ceiling around 20GB models and that background downloads measurably affect tok/s.

Oct 3, 2026
Tone: positiveApple hardware
generation:
4.8 tokens/s
quant:
2-bit dynamic

User reports DeepSeek-V4-Flash 284B-A13B at 4.8 tok/s on a 24 GB M5 Pro with about 5.3 GB of memory in use. Setup is Mference, an open-source engine. The user also runs Gemma 4 26B-A4B and Qwen 3.6 35B-A3B.

Sep 7, 2026