M4 Max 48GB reports

generation:

85.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.5 4B at ~85 tok/s decode on an M4 Max 48GB, running a Mac meeting notetaker stack. Setup is llama.cpp with Q4_K_M weights; the full on-device stack totals 6.8 GB across five models. A 26-minute meeting produced finished notes in 33 s warm. Speaker error dropped from 43.4% to 14.6% DER on AMI, and autocomplete p95 improved from 1.83 s to 0.49 s. MiniCPM5 2B was tried for notes but repeated itself and misassigned action items, so Qwen3.5 4B stayed the default.