Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-usevisionagentic
User reports Gemma 4 E2B at 80 tok/s on an RTX 3050 6GB laptop with 16GB RAM.
No engine, quant or context length is stated.
User is satisfied with the speed but says the model sometimes mishandles tools and hallucinates text in screenshots, and asks whether any good fine-tune of Gemma E2B exists for a local Jarvis-like workflow.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DiffusionGemma 26B-A4B at up to 114.6 t/s on Strix Halo 128GB, compared against GB10.
Setup is Gufo with NVFP4 weights, 2K context, single request. Prefill reaches 3443 t/s at 2K and 1379 t/s at 121K. Time to first token is 5.5 s at 2K and 104.7 s at 121K.
User compares to a llama.cpp Q8_0 run on the same hardware, where prefill is 978 t/s and TTFT 18.8 s at 2K. Against GB10 on SPEED-Bench prompts, Strix Halo is slower at 2K (114.6 vs 148.9 t/s) but faster at 32K (80.4 vs 77.9 t/s). User notes remaining headroom: FP8 denoising step at 134 ms against a 95 ms floor, expert weights streaming at 180-200 GB/s of 242 GB/s, dense products at 72% of matrix-unit floor, and 32K prefill at 40% of its floor.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 31B It at 6.35 tokens/s generation and 515.74 tokens/s prefill on a Strix Halo 128GB unified-memory system (Ryzen AI MAX+ 395, Radeon 8060S).
Setup is the Gufo Gemma development fork (franzmoca/gufo feat/gemma, not stock Gufo 0.12) with UD-Q8_K_XL GGUF weights, one warmed HTTP single-session prose request, greedy temperature 0, thinking off, autoregressive with no speculation. Actual prompt_n 2038, predicted_n 128.
Rates are engine-stage figures from the harness (prefill_tokens*1000/prefill_ms, completion_tokens*1000/decode_ms), not prompt/TTFT or whole-request throughput, and were not independently recomputed or reproduced; one sample, no SD. Explicit KV precision and Flash Attention setting unknown.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 53.2 t/s decode and 3991 t/s prefill on Strix Halo 128GB, compared against llama.cpp b11069 at 44.0 t/s decode and 1685 t/s prefill.
Setup is the Gufo fork with Unsloth UD-Q4_K_XL GGUF, single-user HTTP benchmark, greedy decoding with thinking off, no MTP.
With MTP speculative decoding decode rises to 79.8 t/s, and 254 t/s on repetitive output. At 131K context decode is 38.0 t/s (47.7 with MTP). With 8 concurrent users the 26B serves about 222 t/s aggregate versus 185 on llama.cpp. The 26B QAT quant reaches 81.4 t/s decode and 4131 t/s prefill in gufo bench versus 69.4 and 1546 on llama-bench.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports Gemma 4 31B at 21.4 t/s generation and 235.1 t/s prompt processing on a single Mac Studio with an M3 Ultra and 512 GB unified memory.
Setup is mlx-vlm with 4bit MLX, 36,777 actual input tokens and 128 generated tokens per run, three runs, peak memory 29.7 GB.
The figures are medians; prompt processing p10/p90 are 234.8 and 236.1 t/s, and KV quantization, flash attention and speculative decoding are not exposed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports Gemma 4 26B-A4B at 67.6 t/s generation and 1,645.0 t/s prompt processing on a Mac Studio with an M3 Ultra and 512 GB unified memory.
Setup is mlx-vlm with 4bit MLX, run on a 36,777-token prompt with 128 generated tokens per run over three runs; both figures are medians.
Peak memory was 20.0 GB and median wall time 27 seconds; the configured context cap and KV quantization are not stated.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 41.6 t/s generation and 1698.9 t/s prefill on an AMD Strix Halo 128GB machine.
Setup is llama.cpp with Q8_K_XL, ROCm 7.2.3 HIP backend, Flash Attention enabled, -ngl 99, -b 2048 and -ub 2048, on one Ryzen AI MAX+ 395 with 128 GB unified memory.
The headline combines two separate phase tests, pp512 and tg128, not one paired request; the same model's baseline control reports pp512 = 1369.67 and tg128 = 41.61 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 78.3 t/s generation and 1236 t/s prompt processing on a single M4 Pro Mac mini with 64 GB unified memory.
Setup is MLX 0.32.0 with MLX-LM 0.31.3, an affine mixed 3.40 bpw text-only conversion, batch 1, greedy temperature 0, thinking enabled.
Source medians are 1,236.00 PP and 78.32 TG over 2,048 prompt tokens and 235 generated tokens; ranges are 1,233.55-1,236.60 PP and 78.22-78.52 TG. The author reports quality losses versus its bf16 comparator, including MMLU-Pro-400 -9 percentage points.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 at 16.5 t/s generation and 518.3 t/s prompt processing on an M4 with 16 GB unified memory.
Setup is LiteRT-LM on the GPU backend with dynamic_wi8_afp32 quantization and a 4,096-token maximum context, using a 256-token prefill and 256-token decode benchmark.
The source gives exact rates of 16.53 t/s decode and 518.28 t/s prefill, and does not establish whether the phases were separate tests or one request.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 81.4 t/s generation and 4131 t/s prompt processing on AMD Strix Halo with 128GB unified memory.
Setup is the franzmoca/gufo Gemma fork with native HIP kernels for gfx1151, QAT UD-Q4_K_XL (Q4_0), MTP off.
The figures are separate native gufo bench synthetic tests (pp2048 and tg128), not a paired chat request or HTTP end-to-end throughput; the author states three repetitions but not the aggregate statistic.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 31B at 12.1 t/s generation and 573.6 t/s prefill on an AMD Strix Halo 128GB-class unified-memory system.
Setup is the Gufo Gemma development fork with QAT UD-Q4_K_XL (Q4_0) weights, one warmed HTTP single-session prose request, greedy temperature 0/seed 1, thinking off, autoregressive with draft_n=0.
The row is one sample with no SD, prompt_n 2038, cache_n 0, predicted_n 128; rates are engine-stage, not whole-request throughput, and were not independently recomputed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 31B at 6.4 t/s generation and 515.7 t/s prompt on an AMD Strix Halo 128GB unified-memory system.
Setup is the Gufo Gemma development fork with UD-Q8_K_XL, one warmed HTTP single-session request, greedy temperature 0/seed 1, thinking off, autoregressive with draft_n=0.
The run used prompt_n 2038, cache_n 0 and predicted_n 128; the rates are engine-stage figures, not prompt/TTFT or whole-request throughput, and come from a single sample.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 31B at 10.95 t/s generation and 544.93 t/s prefill on an AMD Strix Halo 128GB-class unified-memory system.
Setup is the Gufo Gemma development fork with UD-Q4_K_XL, one warmed HTTP single-session prose request, greedy temperature 0/seed 1, thinking off, autoregressive with draft_n=0.
The run used prompt_n 2038, cache_n 0 and predicted_n 128; the rates are engine-stage figures from one sample, not prompt/TTFT or whole-request throughput.
Xbox Series S, Zen 2 CPU-only (6 threads) · llama.cpp · 2,048 ctx
generation:
15.3 tokens/s
prompt processing (prefill):
28.5 tokens/s
quant:
Q3_K_S (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E2B at 15.3 t/s generation and 28.5 t/s prefill on an Xbox Series S, CPU-only with six inference threads.
Setup is llama.cpp in xllama 1.6.0.1163 with Q3_K_S and a configured context of 2048, one sequential text request per fresh app process.
The figures are separate medians of three recorded runs, with generation ranging 15.27–15.31 t/s; peak working set was 2741 MB per run.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 78.5 t/s on an M4 Pro with 48 GB unified memory.
Setup is Rapid-MLX 0.15.4 with MLX 4-bit on one Mac mini, vision disabled, decoding a 512-token workload.
The figure is single-stream, a median of three runs at temperature 0; a separate four-stream result of 121.6 t/s is aggregate and not single-request speed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma 4 26B A4B at around 20.5 tokens/s on an NVIDIA GPU with 8 GB of VRAM and 32 GB of system RAM.
Setup is llama.cpp with Q5_K_P quantization, 8192 context, --parallel 1, --gpu-layers auto, --fit on, --load-mode none, reasoning off.
The user asks whether this is the best performance to expect or if there is room for optimization.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 53.2 t/s generation and 3991.2 t/s prefill on a 2,038-token prompt on an AMD Strix Halo 128GB.
Setup is Gufo at UD-Q4_K_XL, one request, speculative decoding off.
franzmoca's depth-zero row of 2026-10-07 from the Gemma 4 Gufo fork. A source-reported maintainer measurement, not independently reproduced.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 69.7 t/s generation and 1172.9 t/s prefill on an M1 Ultra 128GB.
Setup is llama.cpp at Q4_K_M, one request, speculative decoding off.
Fazal Majid's llama-bench run published 2026-04-13. A source-reported measurement, not independently reproduced.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma 4 31B at about 6.0 t/s on a DGX Spark without MTP, and 11.4 to 19.3 t/s with MTP speculative decoding enabled (--spec-draft-n-max 4).
Setup is llama.cpp with the am17an/Gemma4-31B-it-GGUF build.
With MTP the draft accept rate was 0.5881 over 9 requests.
The user notes no speed-up for the MoE variant, over 2x average speedup on the dense model, and that AIME-26 results (~87%) were replicated.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at ~57.6 tok/s at low context on a mixed AMD Vega setup (RX Vega 64 8GB gfx900 + MI50 16GB gfx906).
Setup is llama.cpp built for gfx900;gfx906 under ROCm 7.2.1 with Q4_K_M, 8192 context, layer split and tensor-split 1,4; a 1,2 split overloaded the 8GB Vega64 and caused multi-second stalls.
Decode falls to ~52.8 tok/s at 8191/8192 tokens. The same model on the single MI50 alone ran roughly 58-59 tok/s at low context and 52-53 tok/s near full 8K context.