Gemma 4 reports

generation:

46.2 tokens/s

prompt processing (prefill):

770.2 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 46.2 t/s generation and 770.2 t/s prompt processing on an M4 Pro 64GB. Setup is dasLLAMA with Q8_0 weights, f16 KV cache, and one Metal GPU, with 640 positions allocated. The figures are native PP512 and TG128 counts, not one 512-input/128-output request, and no longer-context performance is inferred.

generation:

30.9 tokens/s

prompt processing (prefill):

298.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 30.9 t/s generation and 298.5 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA version 16 with Q4_K_M weights and f16 KV cache, one GPU, 640 positions allocated. The figures are native PP512 and TG128 means over five repetitions, not a single 512-input/128-output request; a separate four-stream run reached 34.18746480210349 t/s summed scheduler throughput.

generation:

60.4 tokens/s

prompt processing (prefill):

818.7 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B-it at 60.4 t/s generation and 818.7 t/s prompt processing on an M4 Pro with 64 GB unified memory. Setup is dasLLAMA 0.6.4 with UD-Q4_K_M weights, f16 KV cache, and one Metal GPU, with 640 positions allocated. The figures are native PP512 and TG128 counts over five repetitions, not a single 512-input/128-output request.

generation:

78.2 tokens/s

prompt processing (prefill):

1558.9 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 78.2 t/s generation and 1558.9 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with Q8_0 weights, Metal backend, -ngl 99, and 10 CPU threads on one physical GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

42.4 tokens/s

prompt processing (prefill):

786.5 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 42.4 t/s generation and 786.5 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with Q8_0 weights, Metal backend, -ngl 99 and 10 CPU threads, on one physical GPU. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

29.1 tokens/s

prompt processing (prefill):

284.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 29.1 t/s generation and 284.5 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with Q4_K_M, Metal backend, -ngl 99, and 10 CPU threads. The figures are native llama-bench PP512 and TG128 means over five repetitions at depth 0, not a single 512-input/128-output request.

generation:

54.2 tokens/s

prompt processing (prefill):

730.9 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 54.2 t/s generation and 730.9 t/s prompt processing on an M4 Pro 64GB. Setup is llama.cpp with UD-Q4_K_M, Metal backend, -ngl 99, 10 CPU threads, and one physical GPU. The figures are separate native tests: PP512 and TG128 at depth 0, five repetitions each, not a single 512-input/128-output request.

generation:

61.6 tokens/s

prompt processing (prefill):

465.2 tokens/s

quant:

4-bit mixed (config) (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 61.6 t/s generation and 465.2 t/s prompt processing on an Apple M2 Max with 64GB of shared unified memory. Setup is MLX via the mlx_vlm text-only helper with a 4-bit mixed config, batch_size=1, requested 512-token prompt and 128-token generation cap, no HTTP server. The figures are three-trial means after discarding Trial 1; retained generation values were 61.389, 61.732 and 61.742 t/s, and peak memory was 15.255 GiB.

generation:

27.7 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 12B-it at 27.7 t/s on one NVIDIA RTX 2060 Super with 8 GB of VRAM and an Intel Core i9-10900K host. Setup is llama.cpp with a Q4_K_M GGUF, -c 8192, -ngl 99 and --parallel 1, with every model layer forced onto the GPU. The 27.7 t/s is the median of three requests after a short warm-up, with roughly 1% agreement among repeats. The author separately reports 11.3 t/s when automatic placement leaves some layers in system memory.

generation:

13.1 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 13.1 t/s on a Mac mini M4 with 16 GB unified memory. Setup is llama.cpp with Q4_K_M and 8,192 tokens of context, run on the official Mac build on Apple graphics. The 13.1 t/s is the median of three requests after a short warm-up, with roughly 1% agreement among repeats; the source omits measured Mac memory.

generation:

22.4 tokens/s

prompt processing (prefill):

62.7 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 22.4 t/s generation and 62.7 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with Q4_K_M, 4096 context, flash attention on, and one Mac Studio with a 40-core GPU. The headline Quick Chat run used 52 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not long-context benchmarks.

generation:

20.8 tokens/s

prompt processing (prefill):

123.9 tokens/s

quant:

Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 20.8 t/s generation and 123.9 t/s prompt processing on an M4 Max 64GB. Setup is ZINC on Metal with Q4_K_M, serial single-device inference with speculative decoding disabled; configured context is unknown. The headline Quick Chat run used 49 prompt tokens and 96 output tokens, with TG 20.82[20.64,21.14] and PP 123.9[119.2,124.4] t/s as medians of three runs; these are short inputs, not long-context benchmarks.

generation:

81.7 tokens/s

prompt processing (prefill):

379.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 81.7 t/s generation and 379.6 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with UD-Q4_K_M, one Mac Studio with a 40-core GPU and 64 GB unified memory. The figures are medians of three measured nonstreaming requests after one discarded warmup, with 52 prompt and 96 output tokens, and speculation disabled.

generation:

68.4 tokens/s

prompt processing (prefill):

303.6 tokens/s

quant:

UD-Q4_K_M (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 68.4 t/s generation and 303.6 t/s prompt processing on an M4 Max 64GB. Setup is ZINC with UD-Q4_K_M, Metal, speculative decoding disabled, and unknown context length. The headline Quick Chat run used 49 prompt tokens and 96 output tokens, with TG 68.44[67.17,68.66] and PP 303.6[299.2,318.6] t/s over three measured invocations.

generation:

110.5 tokens/s

prompt processing (prefill):

4217.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 110.5 t/s generation and 4217.3 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput.

generation:

173.7 tokens/s

prompt processing (prefill):

7021.2 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 173.7 t/s generation and 7021.2 t/s prompt processing on an M5 Max with 48GB unified memory. Setup is llama.cpp build 11146 with Metal, Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; the context field is intentionally unspecified.

generation:

25.2 tokens/s

prompt processing (prefill):

626.5 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 25.2 t/s generation and 626.5 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Q4_K_M weights and Metal, n_gpu_layers=99 and flash attention enabled. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput.

generation:

116.9 tokens/s

prompt processing (prefill):

3423.2 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 116.9 t/s generation and 3423.2 t/s prompt processing on an M5 Max 48GB. Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash_attention=true. The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request; context length is unspecified and KV cache precision was not captured.

generation:

13.9 tokens/s

prompt processing (prefill):

1237.0 tokens/s

quant:

BF16 (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 13.9 t/s generation and 1237 t/s prompt processing on an M5 Ultra 256GB. Setup is MLX with BF16 weights at a 32768-token input and 128-token output, batch 1 on one device, with KV capacity unconfigured and blank. The 13.9 t/s is the median of five trials, with a range of 13.821 to 13.917 t/s; peak allocator memory was 71.978GB.

generation:

9.2 tokens/s

prompt processing (prefill):

295.9 tokens/s

quant:

BF16 (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 9.2 t/s generation and 295.9 t/s prompt processing on an M3 Ultra 512GB. Setup is MLX with BF16 weights at 32768 input tokens and 128 output tokens, batch 1 on one device, with a fresh KV cache per call and no draft or KV-quant flags. The 32768-token row is a five-trial median, not pooled across lengths, with a peak allocator high-water of 71.714GB; a stuck indexer consumed one CPU core during the tests and its effect was not measured.