Local model performance on your hardware

generation:

9.3 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, section headed discrete GPU RTX 5070 Ti (16GB). Page lists June 7, 2026 publication and August 13, 2026 update. The exact NVIDIA measurement date is unknown. These are historical source-reported measurements recovered October 13, not new runs or local reproduction. Headline: 9.32 tokens/s, the article's reported decode speed for Gemma 4 31B-it on one NVIDIA RTX 5070 Ti with 16 GB GDDR7, Windows 11, configured num_ctx=4096. Actual prompt/output token counts, per-run values, repetition count and aggregation procedure for the NVIDIA rows are not published. Do not transfer the Mac section's explicit three-run averages to this NVIDIA row as an established fact. Model: historical Ollama library gemma4:31b, GGUF Q4_K_M, pulled from the registry according to the article's methodology. Runtime is the article's Ollama-based setup; the exact NVIDIA Ollama version, bundled llama.cpp revision, CUDA/driver and harness build are unknown. The explicit Ollama 0.30.5 version in the article is adjacent to the Mac table and is not independently bound to the NVIDIA campaign. Verify the model tag identity at https://ollama.com/library/gemma4:31b . Its upstream model is https://huggingface.co/google/gemma-4-31B-it . This HF link is upstream identity only; the exact tested GGUF's HF equivalent, Ollama manifest/blob digest, weight hash, model revision and quantizer remain unknown. Today's tags and defaults are not the historical tested configuration. The article reports prefill 199 tokens/s for this NVIDIA row. Its earlier methodology defines native rates as Ollama prompt_eval_count/prompt_eval_duration and eval_count/eval_duration and excludes network/loading costs. However, the NVIDIA section does not separately expose the harness or native counters or explicitly bind the earlier timing method to its rerun. The generic PP field is therefore left blank; retain 199 tokens/s here as source-reported PP only. It must not be reconstructed from context, TTFT, memory or wall time. The headline is source-reported decode, not independently recomputed phase speed or end-to-end throughput. Partial offload is explicitly reported for this model: the article says the 26B and 31B weight sets do not fit entirely in 16 GB and some spill into host memory. Preserve this as a GPU-plus-host/offloaded measurement, not all-GPU speed. Exact layer split, host CPU, installed system RAM, host-memory allocation and PCIe link are unknown. Do not infer those quantities from model size or the apparent fit. Reported peak GPU memory: 15.7 GB, with no monitoring tool or sampling method supplied. This is a source-reported allocation/usage figure, separate from the card's 16 GB capacity. System RAM is unknown. GPU board variant, power limit, clock, temperatures, CPU, motherboard and storage are also unknown. Configured context is 4096, not an actual input count. The source's fixed Chinese article-generation workload and the Mac methodology provide useful campaign background, but exact NVIDIA prompt text, output cap, sampling, thinking mode, warmup, cache state/reuse, batch/concurrency, FA, KV precision and speculative decoding/MTP are not separately documented. All remain unknown. Separate accuracy/coding-evaluation counts and settings are not the speed benchmark's actual counts or denominator. The article discusses possible long-context slowdown from host offload as an inference. It explicitly does not measure longer-context NVIDIA behavior, so no extrapolated throughput or context ceiling is recorded. This row also must not be merged with the Mac mini results or the independent SymbolStar campaign.

generation:

102.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, section headed discrete GPU RTX 5070 Ti (16GB). Page lists June 7, 2026 publication and August 13, 2026 update. The exact NVIDIA measurement date is unknown. These are historical source-reported measurements recovered October 13, not new runs or local reproduction. Headline: 102.88 tokens/s, the article's reported decode speed for Gemma 4 26B-A4B-it on one NVIDIA RTX 5070 Ti with 16 GB GDDR7, Windows 11, configured num_ctx=4096. Actual prompt/output token counts, per-run values, repetition count and aggregation procedure for the NVIDIA rows are not published. Do not transfer the Mac section's explicit three-run averages to this NVIDIA row as an established fact. Model: historical Ollama library gemma4:26b, GGUF Q4_K_M, pulled from the registry according to the article's methodology. Runtime is the article's Ollama-based setup; the exact NVIDIA Ollama version, bundled llama.cpp revision, CUDA/driver and harness build are unknown. The explicit Ollama 0.30.5 version in the article is adjacent to the Mac table and is not independently bound to the NVIDIA campaign. Verify the model tag identity at https://ollama.com/library/gemma4:26b . Its upstream model is https://huggingface.co/google/gemma-4-26B-A4B-it . This HF link is upstream identity only; the exact tested GGUF's HF equivalent, Ollama manifest/blob digest, weight hash, model revision and quantizer remain unknown. Today's tags and defaults are not the historical tested configuration. The article reports prefill 1485 tokens/s for this NVIDIA row. Its earlier methodology defines native rates as Ollama prompt_eval_count/prompt_eval_duration and eval_count/eval_duration and excludes network/loading costs. However, the NVIDIA section does not separately expose the harness or native counters or explicitly bind the earlier timing method to its rerun. The generic PP field is therefore left blank; retain 1485 tokens/s here as source-reported PP only. It must not be reconstructed from context, TTFT, memory or wall time. The headline is source-reported decode, not independently recomputed phase speed or end-to-end throughput. Partial offload is explicitly reported for this model: the article says the 26B and 31B weight sets do not fit entirely in 16 GB and some spill into host memory. Preserve this as a GPU-plus-host/offloaded measurement, not all-GPU speed. Exact layer split, host CPU, installed system RAM, host-memory allocation and PCIe link are unknown. Do not infer those quantities from model size or the apparent fit. Reported peak GPU memory: 15.7 GB, with no monitoring tool or sampling method supplied. This is a source-reported allocation/usage figure, separate from the card's 16 GB capacity. System RAM is unknown. GPU board variant, power limit, clock, temperatures, CPU, motherboard and storage are also unknown. Configured context is 4096, not an actual input count. The source's fixed Chinese article-generation workload and the Mac methodology provide useful campaign background, but exact NVIDIA prompt text, output cap, sampling, thinking mode, warmup, cache state/reuse, batch/concurrency, FA, KV precision and speculative decoding/MTP are not separately documented. All remain unknown. Separate accuracy/coding-evaluation counts and settings are not the speed benchmark's actual counts or denominator. The article discusses possible long-context slowdown from host offload as an inference. It explicitly does not measure longer-context NVIDIA behavior, so no extrapolated throughput or context ceiling is recorded. This row also must not be merged with the Mac mini results or the independent SymbolStar campaign.

generation:

78.6 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, section headed discrete GPU RTX 5070 Ti (16GB). Page lists June 7, 2026 publication and August 13, 2026 update. The exact NVIDIA measurement date is unknown. These are historical source-reported measurements recovered October 13, not new runs or local reproduction. Headline: 78.61 tokens/s, the article's reported decode speed for Gemma 4 12B-it on one NVIDIA RTX 5070 Ti with 16 GB GDDR7, Windows 11, configured num_ctx=4096. Actual prompt/output token counts, per-run values, repetition count and aggregation procedure for the NVIDIA rows are not published. Do not transfer the Mac section's explicit three-run averages to this NVIDIA row as an established fact. Model: historical Ollama library gemma4:12b, GGUF Q4_K_M, pulled from the registry according to the article's methodology. Runtime is the article's Ollama-based setup; the exact NVIDIA Ollama version, bundled llama.cpp revision, CUDA/driver and harness build are unknown. The explicit Ollama 0.30.5 version in the article is adjacent to the Mac table and is not independently bound to the NVIDIA campaign. Verify the model tag identity at https://ollama.com/library/gemma4:12b . Its upstream model is https://huggingface.co/google/gemma-4-12B-it . This HF link is upstream identity only; the exact tested GGUF's HF equivalent, Ollama manifest/blob digest, weight hash, model revision and quantizer remain unknown. Today's tags and defaults are not the historical tested configuration. The article reports prefill 1771 tokens/s for this NVIDIA row. Its earlier methodology defines native rates as Ollama prompt_eval_count/prompt_eval_duration and eval_count/eval_duration and excludes network/loading costs. However, the NVIDIA section does not separately expose the harness or native counters or explicitly bind the earlier timing method to its rerun. The generic PP field is therefore left blank; retain 1771 tokens/s here as source-reported PP only. It must not be reconstructed from context, TTFT, memory or wall time. The headline is source-reported decode, not independently recomputed phase speed or end-to-end throughput. The article does not quantify this model's GPU layer split or host offload. Do not copy the earlier Mac's 100% offload statement to the NVIDIA machine. Host CPU, installed system RAM and PCIe link are unknown. Reported peak GPU memory: 8.9 GB, with no monitoring tool or sampling method supplied. This is a source-reported allocation/usage figure, separate from the card's 16 GB capacity. System RAM is unknown. GPU board variant, power limit, clock, temperatures, CPU, motherboard and storage are also unknown. Configured context is 4096, not an actual input count. The source's fixed Chinese article-generation workload and the Mac methodology provide useful campaign background, but exact NVIDIA prompt text, output cap, sampling, thinking mode, warmup, cache state/reuse, batch/concurrency, FA, KV precision and speculative decoding/MTP are not separately documented. All remain unknown. Separate accuracy/coding-evaluation counts and settings are not the speed benchmark's actual counts or denominator. The article discusses possible long-context slowdown from host offload as an inference. It explicitly does not measure longer-context NVIDIA behavior, so no extrapolated throughput or context ceiling is recorded. This row also must not be merged with the Mac mini results or the independent SymbolStar campaign.

generation:

148.2 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, section headed discrete GPU RTX 5070 Ti (16GB). Page lists June 7, 2026 publication and August 13, 2026 update. The exact NVIDIA measurement date is unknown. These are historical source-reported measurements recovered October 13, not new runs or local reproduction. Headline: 148.21 tokens/s, the article's reported decode speed for Gemma 4 E4B-it on one NVIDIA RTX 5070 Ti with 16 GB GDDR7, Windows 11, configured num_ctx=4096. Actual prompt/output token counts, per-run values, repetition count and aggregation procedure for the NVIDIA rows are not published. Do not transfer the Mac section's explicit three-run averages to this NVIDIA row as an established fact. Model: historical Ollama library gemma4:e4b, GGUF Q4_K_M, pulled from the registry according to the article's methodology. Runtime is the article's Ollama-based setup; the exact NVIDIA Ollama version, bundled llama.cpp revision, CUDA/driver and harness build are unknown. The explicit Ollama 0.30.5 version in the article is adjacent to the Mac table and is not independently bound to the NVIDIA campaign. Verify the model tag identity at https://ollama.com/library/gemma4:e4b . Its upstream model is https://huggingface.co/google/gemma-4-E4B-it . This HF link is upstream identity only; the exact tested GGUF's HF equivalent, Ollama manifest/blob digest, weight hash, model revision and quantizer remain unknown. Today's tags and defaults are not the historical tested configuration. The article reports prefill 2549 tokens/s for this NVIDIA row. Its earlier methodology defines native rates as Ollama prompt_eval_count/prompt_eval_duration and eval_count/eval_duration and excludes network/loading costs. However, the NVIDIA section does not separately expose the harness or native counters or explicitly bind the earlier timing method to its rerun. The generic PP field is therefore left blank; retain 2549 tokens/s here as source-reported PP only. It must not be reconstructed from context, TTFT, memory or wall time. The headline is source-reported decode, not independently recomputed phase speed or end-to-end throughput. The article does not quantify this model's GPU layer split or host offload. Do not copy the earlier Mac's 100% offload statement to the NVIDIA machine. Host CPU, installed system RAM and PCIe link are unknown. Reported peak GPU memory: 5.1 GB, with no monitoring tool or sampling method supplied. This is a source-reported allocation/usage figure, separate from the card's 16 GB capacity. System RAM is unknown. GPU board variant, power limit, clock, temperatures, CPU, motherboard and storage are also unknown. Configured context is 4096, not an actual input count. The source's fixed Chinese article-generation workload and the Mac methodology provide useful campaign background, but exact NVIDIA prompt text, output cap, sampling, thinking mode, warmup, cache state/reuse, batch/concurrency, FA, KV precision and speculative decoding/MTP are not separately documented. All remain unknown. Separate accuracy/coding-evaluation counts and settings are not the speed benchmark's actual counts or denominator. The article discusses possible long-context slowdown from host offload as an inference. It explicitly does not measure longer-context NVIDIA behavior, so no extrapolated throughput or context ceiling is recorded. This row also must not be merged with the Mac mini results or the independent SymbolStar campaign.

generation:

226.3 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, section headed discrete GPU RTX 5070 Ti (16GB). Page lists June 7, 2026 publication and August 13, 2026 update. The exact NVIDIA measurement date is unknown. These are historical source-reported measurements recovered October 13, not new runs or local reproduction. Headline: 226.3 tokens/s, the article's reported decode speed for Gemma 4 E2B-it on one NVIDIA RTX 5070 Ti with 16 GB GDDR7, Windows 11, configured num_ctx=4096. Actual prompt/output token counts, per-run values, repetition count and aggregation procedure for the NVIDIA rows are not published. Do not transfer the Mac section's explicit three-run averages to this NVIDIA row as an established fact. Model: historical Ollama library gemma4:e2b, GGUF Q4_K_M, pulled from the registry according to the article's methodology. Runtime is the article's Ollama-based setup; the exact NVIDIA Ollama version, bundled llama.cpp revision, CUDA/driver and harness build are unknown. The explicit Ollama 0.30.5 version in the article is adjacent to the Mac table and is not independently bound to the NVIDIA campaign. Verify the model tag identity at https://ollama.com/library/gemma4:e2b . Its upstream model is https://huggingface.co/google/gemma-4-E2B-it . This HF link is upstream identity only; the exact tested GGUF's HF equivalent, Ollama manifest/blob digest, weight hash, model revision and quantizer remain unknown. Today's tags and defaults are not the historical tested configuration. The article reports prefill 3461 tokens/s for this NVIDIA row. Its earlier methodology defines native rates as Ollama prompt_eval_count/prompt_eval_duration and eval_count/eval_duration and excludes network/loading costs. However, the NVIDIA section does not separately expose the harness or native counters or explicitly bind the earlier timing method to its rerun. The generic PP field is therefore left blank; retain 3461 tokens/s here as source-reported PP only. It must not be reconstructed from context, TTFT, memory or wall time. The headline is source-reported decode, not independently recomputed phase speed or end-to-end throughput. The article does not quantify this model's GPU layer split or host offload. Do not copy the earlier Mac's 100% offload statement to the NVIDIA machine. Host CPU, installed system RAM and PCIe link are unknown. Reported peak GPU memory: 3.6 GB, with no monitoring tool or sampling method supplied. This is a source-reported allocation/usage figure, separate from the card's 16 GB capacity. System RAM is unknown. GPU board variant, power limit, clock, temperatures, CPU, motherboard and storage are also unknown. Configured context is 4096, not an actual input count. The source's fixed Chinese article-generation workload and the Mac methodology provide useful campaign background, but exact NVIDIA prompt text, output cap, sampling, thinking mode, warmup, cache state/reuse, batch/concurrency, FA, KV precision and speculative decoding/MTP are not separately documented. All remain unknown. Separate accuracy/coding-evaluation counts and settings are not the speed benchmark's actual counts or denominator. The article discusses possible long-context slowdown from host offload as an inference. It explicitly does not measure longer-context NVIDIA behavior, so no extrapolated throughput or context ceiling is recorded. This row also must not be merged with the Mac mini results or the independent SymbolStar campaign.

generation:

180.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ornith 1.5 35B-A3B at about 180 tok/s on two RTX 5070 Ti 16GB cards, roughly 3x the 55-70 tok/s of the Qwen3.8 27B it replaced. Setup is Ollama on Windows with Q4_K_M at 128K context, 24.4GB fully on the two cards; the agent runs on pi in WSL. Two runs gave 176 and 183 tok/s with a short prompt and thinking off. At 256K about 1.2GB spills to system RAM and speed drops to about 139 tok/s, so 128K stays the default. Both of the user's agent tests max out at 9/9 and 10/10, which the user notes only shows it is not worse than the 27B, not that it is smarter. Vendor numbers of 79 on SWE-bench Verified and 68.5 on Terminal-Bench 2.1 are cited but untested.

generation:

108-110 tokens/s

prompt processing (prefill):

1683-4323 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3 30B-A3B at 108-110 t/s decode and 1683-4323 t/s prefill on an RTX 5070 Ti 16 GB. Setup is Ollama 0.35.0 with Q4_K_M at 8192 context, median of 2 runs, measured idle. The model produced no compilable code (0/2), failed one-word obedience with 899 characters, and returned 0 characters on the reasoning test after burning its 4000-token budget on 15088 characters of thinking. User notes prefill swings 2x with background load and that the fastest model here measures verbosity.