M1 Pro 16GB reports

generation:

33.0 tokens/s

quant:

QAT (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 e4b-it-qat at 33 tok/s on a 16 GB M1 Pro, up from 27 tok/s with the non-QAT build. Setup is Ollama with a 6.4 GiB footprint; the 12B deep model cannot stay co-resident, so each swap costs a 4 to 7 s cold load. Moving per-message content out of the system prompt cut follow-up time to first token from 1.8-2.0 s to 0.4-0.5 s. Tool-calling accuracy on 9 prompts was 8/9 bare, 5/9 with a 700-token system prompt, and 9/9 with a one-line nudge placed after the question. Memory injection of 109 facts cost 2,300 tokens per turn; recalling the nearest 10 costs about 150 tokens.