Local model performance on your hardware

generation:

52.0 tokens/s

quant:

Q4_K_M (GGUF)

User compares LFM2.5-24B-A2B against Qwen3.6-35B-A3B on 75 multi-step tool-use tasks on an M1 Pro 32GB, with LFM2.5 solving 37/75 (49%) and Qwen 68/75 (91%). Both ran in llama.cpp b10964 with --jinja at 64k context, LFM2.5 at Q4_K_M (14.5 GB) and Qwen at Q4_K_M (22.3 GB), thinking on. LFM2.5 was faster per token (~52 vs 34 tok/s) but used about twice the tokens per solved task, 418 s per correct result vs Qwen's 328. In at least 23 of its 38 failures it reported the task as done. The user notes this is 4-bit llama.cpp rather than Liquid's vLLM setup, and that the two tie on BFCLv4.

generation:

25.4 tokens/s

prompt processing (prefill):

38.6 tokens/s

quant:

Q4_K_S (GGUF)

kv:

q8_0/q4_0

User reports Qwen3.6-35B-A3B at 25.4 tok/s decode and 38.6 tok/s prefill on a MacBook Pro M1 Pro 32GB. Setup is llama.cpp build b9466 with Q4_K_S GGUF and q8_0/q4_0 KV cache at 32K context, single sequence, all layers on the Metal GPU. User compares the MoE against dense Qwen3.6-27B on the same host: the dense model decodes at 5.3 tok/s and prefills at 12.9 tok/s, uses 17.9 GB wired versus 22.4 GB, and swaps ~1 GB versus ~7.6 GB. Prefill is the bottleneck; cold TTFT on a 7.5K prompt is 92.8 s versus 0.2 s warm. Tool-calling was clean in 10/10 episodes.

generation:

0.1 tokens/s

quant:

FP8-E4M3 (FP8)

User reports GLM-5.3-Flash at ~0.085 tok/s decode (~11-12 s/token) on a MacBook Pro M1 Pro 32 GB. Setup is the apus engine with the official FP8-E4M3 checkpoint, 2 GiB expert cache, --tiered streaming of routed experts from NVMe, MTP speculative decoding on by default (draft depth 3). Prefill was ~76 s for a 26-token prompt, model load ~7 s, peak footprint ~22.3 GiB, weights ~306 GiB, context up to 1,048,576 tokens. The engine reads ~7.9 GiB of experts per generated token cold; the author notes the wall at 32 GB is memory capacity, not disk.