Local model performance on your hardware

generation:

69.7 tokens/s

prompt processing (prefill):

1172.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 69.73 t/s generation and 1172.93 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/utumno.txt # Mac Studio M1 Ultra 128GB: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: Mac Studio M1 Ultra 128GB. Gemma4 E4B Q4_K_M GGUF, llama.cpp ff5ef8278 (8763), backend Metal/BLAS. pp512 1172.93 ±0.36 tokens/s; tg128 69.73 ±0.19 tokens/s. 16 CPU threads; recommendedMaxWorkingSetSize 103079.22 MB is a runtime recommendation, not resident memory. Unified system RAM 128GB, GPU core count unknown. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/utumno.txt .

generation:

69.7 tokens/s

prompt processing (prefill):

1172.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 69.7 t/s generation and 1172.9 t/s prefill on an M1 Ultra 128GB. Setup is llama.cpp at Q4_K_M, one request, speculative decoding off. Fazal Majid's llama-bench run published 2026-04-13. A source-reported measurement, not independently reproduced.

generation:

31.2 tokens/s

prompt processing (prefill):

530.2 tokens/s

quant:

IQ4_NL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks a custom llama.cpp branch with faster Metal inference and n-gram SSD offload at pp512 530.18 t/s and tg128 31.17 t/s with resident n-gram. The model is 68.37 GiB and 125.74 B params. With SSD read mode, pp512 is 197.92 t/s and tg128 is 28.47 t/s.