Gemma 4 reports

generation:

165.6 tokens/s

prompt processing (prefill):

9555.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 165.64 t/s generation and 9555.89 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/golgotha.txt # NVIDIA GeForce RTX 5090 Laptop GPU 24GB: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: NVIDIA GeForce RTX 5090 Laptop GPU 24GB. Gemma4 E4B Q4_K_M GGUF, llama.cpp e3f542d8 (4236), backend CUDA. pp512 9555.89 ±101.42 tokens/s; tg128 165.64 ±0.19 tokens/s. One device, ngl -1; 23983 MiB visible VRAM, not resident use. Laptop GPU, not desktop RTX5090. CPU and system RAM unknown. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/golgotha.txt .

generation:

51.9 tokens/s

prompt processing (prefill):

1831.8 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 51.92 t/s generation and 1831.78 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/zanzibar.txt # NVIDIA RTX A2000 12GB: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: NVIDIA RTX A2000 12GB. Gemma4 E4B Q4_K_M GGUF, llama.cpp 82764d8f4 (8770), backend CUDA. pp512 1831.78 ±13.79 tokens/s; tg128 51.92 ±0.39 tokens/s. One device, ngl99; 11906 MiB visible VRAM, not resident use. CPU and system RAM unknown. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/zanzibar.txt .

generation:

21.2 tokens/s

prompt processing (prefill):

568.5 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 21.16 t/s generation and 568.54 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/ai-x1-pro.txt # AMD Ryzen AI 9 HX 370 / Radeon 890M, 96GB system RAM: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: AMD Ryzen AI 9 HX 370 / Radeon 890M, 96GB system RAM. Gemma4 E4B Q4_K_M GGUF, llama.cpp 69c28f1 (1), backend ROCm. pp512 568.54 ±1.96 tokens/s; tg128 21.16 ±0.03 tokens/s. gfx1150; ngl99; runtime reports 47072 MiB device VRAM visibility; integrated unified memory, not dedicated VRAM or measured resident use. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/ai-x1-pro.txt .

generation:

69.7 tokens/s

prompt processing (prefill):

1172.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 69.73 t/s generation and 1172.93 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/utumno.txt # Mac Studio M1 Ultra 128GB: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: Mac Studio M1 Ultra 128GB. Gemma4 E4B Q4_K_M GGUF, llama.cpp ff5ef8278 (8763), backend Metal/BLAS. pp512 1172.93 ±0.36 tokens/s; tg128 69.73 ±0.19 tokens/s. 16 CPU threads; recommendedMaxWorkingSetSize 103079.22 MB is a runtime recommendation, not resident memory. Unified system RAM 128GB, GPU core count unknown. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/utumno.txt .

generation:

69.7 tokens/s

prompt processing (prefill):

6757.3 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 69.66 t/s generation and 6757.3 t/s prompt processing on an NVIDIA RTX 4070 Ti. Setup is llama.cpp with Q8_0 and requested GPU layers 999, a direct full-offload run on a 12GB card. The figures come from a synthetic pp512/tg128 test at zero context depth, so they do not measure generation after a filled context.

generation:

48.3 tokens/s

prompt processing (prefill):

1376.8 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 48.3 t/s generation and 1376.8 t/s prompt processing on a GMKtec EVO-X2 Strix Halo running bare-metal. Setup is llama-bench with the Unsloth gemma-4-26B-A4B-it-Q8_0.gguf quant, Vulkan backend, ngl 99 and FA 1, on one AMD Radeon 8060S Graphics device. The native rows give PP512 at 1376.78 +/- 4.61 t/s and TG128 at 48.31 +/- 0.02 t/s; these are separate synthetic prompt-processing and generation phases, not a same-request pair.

generation:

17.2 tokens/s

prompt processing (prefill):

63.7 tokens/s

quant:

4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 17.2 t/s generation and 63.7 t/s prompt processing on a Mac Studio M2 Max with 64 GB unified memory. Setup is mlx-vlm 0.4.4 with 4-bit affine group-64 weights and MLX 0.31.1 on one GPU, running the text-only long-text request with 55 prompt tokens and 500 generated tokens. The 29.9-second whole generate call is not TTFT, and the 63.7 t/s prompt figure is a single request rather than a llama-bench pairing.

generation:

19.1 tokens/s

prompt processing (prefill):

920.7 tokens/s

quant:

BF16 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 19.1 t/s generation and 920.7 t/s prompt processing on a Minisforum MS-S1 with an AMD Ryzen AI Max+ 395 (Strix Halo) and Radeon 8060S Graphics. Setup is llama.cpp with BF16 and flash attention, one CPU thread, batch and ubatch 2048, ngl 99, on Vulkan/RADV. The 19.07 t/s TG128 and 920.65 t/s PP2048 are separate synthetic llama-bench tests, not one 2048-in/128-out request; TG was not measured after the largest PP context.

generation:

44.3 tokens/s

prompt processing (prefill):

1561.8 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B A4B at 44.3 t/s generation and 1561.8 t/s prompt processing on a Minisforum MS-S1 with an AMD Ryzen AI Max+ 395 (Strix Halo) and Radeon 8060S Graphics. Setup is llama.cpp b8665 with UD-Q8_K_XL, flash attention on, one CPU thread, batch and ubatch 2048, and ngl 99 on Vulkan. The figures are separate synthetic llama-bench tests, not one 2048-in/128-out request; TG128 was not measured after the largest prompt context.

generation:

18.3 tokens/s

prompt processing (prefill):

627.1 tokens/s

quant:

BF16 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B A4B at 18.3 t/s generation and 627.1 t/s prompt processing on a Minisforum MS-S1 with an AMD Ryzen AI Max+ 395 (Strix Halo) and Radeon 8060S Graphics. Setup is llama.cpp with BF16 weights, flash attention on, one CPU thread, batch and ubatch 2048, and ngl 99 on Vulkan. The figures are separate synthetic llama-bench tests, not one 2048-in/128-out request; TG128 was not measured after the largest prompt context.

generation:

6.2 tokens/s

prompt processing (prefill):

293.1 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 6.2 t/s generation and 293.1 t/s prompt processing on a Minisforum MS-S1 with an AMD Ryzen AI Max+ 395 (Strix Halo) and Radeon 8060S Graphics. Setup is llama.cpp with UD-Q8_K_XL, flash attention on, one CPU thread, batch and ubatch 2048, and ngl 99 on Vulkan. The pp2048 and tg128 figures are separate synthetic tests, not one 2048-in/128-out request, and TG was not measured after the largest PP context.

generation:

3.5 tokens/s

prompt processing (prefill):

148.3 tokens/s

quant:

BF16 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 31B at 3.5 t/s generation and 148.3 t/s prompt processing on a Minisforum MS-S1 with an AMD Ryzen AI Max+ 395 (Strix Halo) and Radeon 8060S Graphics. Setup is llama.cpp b8640 with BF16 weights, flash attention on, batch and ubatch 2048, and ngl 99 on Vulkan/RADV. The figures are separate synthetic llama-bench tests, not one 2048-in/128-out request; TG128 was 3.46 t/s and was not measured after the largest prompt context.

generation:

87.8 tokens/s

prompt processing (prefill):

2595.3 tokens/s

quant:

MXFP4_MOE (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 87.8 t/s generation and 2595.3 t/s prompt processing on an M3 Ultra Mac Studio with 512 GB of unified memory. Setup is llama.cpp build b8783 with the MXFP4_MOE quantization, Metal backend, F16 K and V cache, and -ngl 999, running one GPU. The figures come from separate llama-bench phases: PP2048 at 2595.28 t/s and TG32 at 87.81 t/s, each the mean of five repetitions at context depth 0.

generation:

84.9 tokens/s

prompt processing (prefill):

2634.8 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 84.9 t/s generation and 2634.8 t/s prompt processing on an M3 Ultra Mac Studio with 512 GB of unified memory. Setup is llama.cpp build b8783 with the UD-Q8_K_XL quant, Metal backend, F16 K and V cache, and -ngl 999 offloading to one GPU. The figures come from llama-bench with -p 2048 -n 32 -ub 2048 -fa 1, five repetitions, batch 2048 and context depth 0; the article's llama-server, quality, VLM and long-context tests are separate.

generation:

88.1 tokens/s

prompt processing (prefill):

2554.2 tokens/s

quant:

UD-Q5_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 88.1 t/s generation and 2554.2 t/s prompt processing on an M3 Ultra Mac Studio with 512 GB of unified memory. Setup is llama.cpp build b8783 with UD-Q5_K_M and F16 K and V cache, Metal backend, one GPU, -ngl 999, batch 2048, five repetitions. The figures are separate llama-bench phases: PP2048 at 2554.16 t/s and TG32 at 88.08 t/s, not a paired 2048-token prompt followed by 32 generated tokens.

generation:

62.4 tokens/s

prompt processing (prefill):

2649.3 tokens/s

quant:

BF16 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 62.4 t/s generation and 2649.3 t/s prompt processing on an M3 Ultra Mac Studio with 512 GB of unified memory. Setup is llama.cpp build b8783 with BF16 weights, F16 K and V cache, Metal backend, and -ngl 999, using -p 2048 -n 32 -ub 2048 -fa 1. The figures are separate llama-bench phases: PP evaluates 2048 synthetic tokens and TG 32 sequential single tokens, with five repetitions and no speculative decoding.

generation:

88.1 tokens/s

prompt processing (prefill):

1408.1 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 26B-A4B at 88.1 t/s generation and 1408.1 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with UD-Q4_K_M, flash attention and F16 K/V cache, single sequence, GPU layers 99. The headline is depth 0; at prefix depth 4096 generation is 79.77 t/s and at 16384 it is 75.49 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

54.5 tokens/s

prompt processing (prefill):

589.0 tokens/s

quant:

Q4_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 54.5 t/s generation and 589 t/s prompt processing on an M4 Max 128GB. Setup is llama.cpp with Q4_0, flash attention and F16 K/V, single sequence, 99 GPU layers requested. The headline is depth 0; at prefix depth 4096 generation is 50.62 t/s and at 16384 it is 47.37 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

12.9 tokens/s

prompt processing (prefill):

138.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 12B at 12.9 t/s generation and 138.5 t/s prompt processing on a base Apple M4 Mac mini with 24 GiB unified memory. Setup is llama.cpp b10090 with UD-Q4_K_XL, run via llama-bench at 512 prompt and 128 generation tokens over 5 repetitions on one GPU. Later interleaved runs reached 12.71/12.79 t/s and prompt speed drifted 129–134 under sustained load; a depth ladder showed generation falling to 9.26 t/s at 32768.

generation:

86.6 tokens/s

prompt processing (prefill):

1545.7 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E2B at 86.6 t/s generation and 1545.7 t/s prompt processing on an M4 Pro with 64GB unified memory. Setup is dasLLAMA with Q8_0 weights, f16 KV cache, and one Metal GPU; the session requested 640 positions and no speculative decoding. The figures are native PP512 and TG128 counts over five repetitions, not one 512-in/128-out request.