Gemma 4 reports

generation:

9.3 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 9.3 t/s on an NVIDIA RTX 5070 Ti. Setup is Ollama with Q4_K_M at 4096 tokens of context, with partial offload to host memory since the weights do not fit entirely in 16 GB. The source also reports 199 t/s prefill and 15.7 GB peak GPU memory; the run is a GPU-plus-host measurement, not all-GPU speed.

generation:

102.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 102.9 t/s on an NVIDIA RTX 5070 Ti. Setup is Ollama with Q4_K_M at num_ctx=4096, with partial offload since the weights do not fit entirely in 16 GB and spill into host memory. The source also reports prefill at 1485 tokens/s and peak GPU memory of 15.7 GB; the figure is a GPU-plus-host measurement, not all-GPU speed.

generation:

78.6 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 78.6 t/s on an NVIDIA RTX 5070 Ti. Setup is Ollama with Q4_K_M and num_ctx=4096, on one 16 GB card with reported peak GPU memory of 8.9 GB. The source also reports prefill at 1771 tokens/s; the article does not quantify GPU layer split or host offload, and longer-context behavior was not measured.

generation:

148.2 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 148.2 t/s on an NVIDIA RTX 5070 Ti. Setup is Ollama with Q4_K_M and a configured context of 4096, on one 16 GB card. The source also reports prefill at 2549 tokens/s and peak GPU memory of 5.1 GB; the figure is source-reported decode, not independently recomputed.

generation:

226.3 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 226.3 t/s on an NVIDIA RTX 5070 Ti. Setup is Ollama with Q4_K_M and num_ctx=4096, on one 16 GB card with reported peak GPU memory of 3.6 GB. The article also reports prefill at 3461 t/s; prompt and output token counts, run count and aggregation for this row are not published, and longer-context behavior was not measured.

generation:

5.3 tokens/s

prompt processing (prefill):

90.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 5.3 t/s generation and 90 t/s prompt processing on an M4 32GB Mac mini. Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, 100% GPU offload, 20.29 GB loaded memory including KV cache. The figures are three-run averages from a fixed Chinese long-form prompt; 4096 is the configured context, not measured input length, and per-run counters are unpublished.

generation:

32.0 tokens/s

prompt processing (prefill):

657.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 32 t/s generation and 657 t/s prompt processing on an M4 32GB Mac mini. Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, 100% GPU offload, 17.37 GB loaded memory including KV cache. Both figures are three-run averages; the 400-character prompt instruction is not an output-token count, and 4096 is the configured context, not measured input length.

generation:

12.5 tokens/s

prompt processing (prefill):

236.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 12.5 t/s generation and 236 t/s prompt processing on an Apple Mac mini M4 with 32 GB unified memory. Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, reporting 100% GPU offload and 8.05 GB loaded memory including KV cache. Both figures are three-run averages; the 4096 is the configured context, not measured input length, and the prompt asks for at least 400 Chinese characters.

generation:

29.9 tokens/s

prompt processing (prefill):

566.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 29.9 t/s generation and 566 t/s prompt processing on an Apple Mac mini M4 with 32 GB unified memory. Setup is Ollama 0.30.5 with GGUF Q4_K_M at num_ctx=4096, with 100% GPU offload and 9.55 GB loaded memory including KV cache. The figures are source-reported three-run averages, with decode from eval_count/eval_duration and prompt processing from prompt_eval_count/prompt_eval_duration, excluding client and model-loading overhead.

generation:

55.0 tokens/s

prompt processing (prefill):

1089.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 55 t/s generation and 1089 t/s prompt processing on an Apple Mac mini M4 with 32 GB unified memory. Setup is Ollama 0.30.5 with GGUF Q4_K_M at num_ctx=4096 and 100% GPU offload, using the Ollama library gemma4:e2b tag. The figures are a three-run average (54.97 t/s decode, 1089 t/s prompt) from a fixed Chinese long-form prompt; reported loaded memory is 7.06 GB including KV cache.

generation:

8.7 tokens/s

prompt processing (prefill):

110.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 8.7 t/s generation and 110.4 t/s prompt processing on an M4 32GB Mac mini. Setup is Ollama 0.30.5 with Q4_K_M and q8_0 KV cache, num_ctx=8192, flash attention enabled, one card. The row is the first long-context tier: 5713 actual input tokens and 512 actual output tokens, not a full 8192-token prompt or an average. The source TTFT estimate is 64.5 s and request wall time 123.4 s.

generation:

16.7 tokens/s

prompt processing (prefill):

245.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 16.7 t/s generation and 245.5 t/s prompt processing on an M4 32GB Mac mini. Setup is Ollama 0.30.5 with Q4_K_M and q8_0 KV cache at 8192 context, one Mac mini M4 with 32 GB unified memory. The figure is a single long-context row with 5709 actual input and 505 actual output tokens, not an average or a full 8192-token prompt; the source TTFT estimate is 23.53 s.

generation:

7.5 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 7.5 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with a 32,768-token loaded context; quantization and backend are unrecorded. The 7.5 t/s is the median of three English-prose runs, measured as client-observed streaming decode throughput rather than a pooled or peak rate.

generation:

18.8 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 18.8 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with a QAT-labeled model at 32,768 tokens of loaded context; exact quantization and backend are unknown. The 18.8 t/s is the median of three English-prose runs, client-observed streaming decode throughput, not a pooled rate or prompt-processing rate.

generation:

48.2 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 48.2 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with an unreported backend and unknown quantization, loaded context 80,128 tokens, one model loaded alone. The 48.2 t/s is the median of three English-prose runs, client-observed streaming decode throughput, not a pooled rate or peak; the five workload medians range from 45.5 to 53.3 t/s.

prompt processing (prefill):

2901.0 tokens/s

quant:

BF16 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 at 2901 t/s prompt processing on one AMD Ryzen AI Max+ 395 / Radeon 8060S integrated GPU (Strix Halo). Setup is the Gufo native HIP engine with BF16 safetensors and a configured context of 131072 tokens, one request at a time. The figure is native prefill for a 2,261-input-token book prompt with 512 output tokens, at TTFT 7.0 s and 184 ms per denoising step; prefill falls to 1,344 t/s at 121,294 input tokens.

prompt processing (prefill):

3180.0 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 at 3180 t/s prompt processing on one AMD Ryzen AI Max+ 395 / Radeon 8060S integrated GPU (Strix Halo). Setup is the Gufo native HIP engine with Q8_0 GGUF and a 131072-token context, one request at a time. The 3180 t/s is native prefill for a 2,261-input-token book prompt producing 512 output tokens, with TTFT 6.0 s and 141 ms per denoising step; prefill falls to 1,339 t/s at 121,294 input tokens.

prompt processing (prefill):

3116.0 tokens/s

quant:

FP8 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 at 3116 t/s prompt processing on one AMD Ryzen AI Max+ 395 / Radeon 8060S integrated GPU (Strix Halo). Setup is the Gufo native HIP engine with FP8 safetensors and a configured context of 131072, one request at a time. The 3116 t/s is native prefill for a 2,261-input-token book prompt with 512 output tokens; the same sweep drops to 1329 t/s at 121,294 input tokens.

generation:

52.9 tokens/s

prompt processing (prefill):

3022.2 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 52.9 t/s generation and 3022.2 t/s prompt processing on one RTX PRO 5000 Blackwell 72 GB plus one RTX PRO 5000 Blackwell 48 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K/V cache, flash attention on, tensor split 1/1, batch 2048 and microbatch 512. The 52.9 t/s is TG128 and the 3022.2 t/s is PP512, separate tests rather than one combined request; the same run measured PP2048 at 2936.0 t/s and PP8192 at 2881.1 t/s.

generation:

33.2 tokens/s

prompt processing (prefill):

2145.5 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 33.2 t/s generation and 2145.5 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on a single card. The 33.2 t/s is TG128 and the 2145.5 t/s is PP512, each the mean of five repetitions; the author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.