llamaperf

Local model performance on your hardware

Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.

What runs on your hardware?

Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Looking for a particular model?

Search for a model to see its reported speeds across GPUs and Macs.

Local model performance reports from the community

These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →

GPU: M4 32GB
Compare setup details (1 active)

Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.

Apple hardware
generation:
5.3 tokens/s
prompt processing (prefill):
90.0 tokens/s
quant:
Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, sections describing benchmark methodology and Mac mini M4 token speed. Page lists June 7, 2026 publication and August 13, 2026 update. Exact benchmark date is not given. Recovered October 13; no local reproduction performed. One Apple Mac mini M4 with 32 GB unified memory, Ollama 0.30.5, GGUF Q4_K_M, num_ctx=4096, source reports 100% GPU offload. Exact CPU/GPU core counts, macOS, power, clocks, thermals and storage are not reported. Do not borrow hardware details or flags from SymbolStar's independent M4 mini campaign. Measured model is Ollama library gemma4:31b, obtained through ollama pull; source explicitly says its benchmarks use the library's GGUF Q4_K_M builds rather than downloaded HF originals or a self-made quant. Current tag identity verified at https://ollama.com/library/gemma4:31b . Upstream model identity verified at https://huggingface.co/google/gemma-4-31B-it . This HF link identifies the original 31B-it model, not the exact tested GGUF. The historical Ollama manifest/blob digest, tested model hash, upstream revision, quantizer and HF artifact-equivalent are unknown. Today's tag can contain newer weights, drafting and multiple runtime formats and must not substitute for the source's historical GGUF test. Headline is the source-reported three-run average for this model: native decode 5.28 tokens/s and native prompt processing 90 tokens/s. The speed test uses a fixed Chinese long-form prompt asking for an article of at least 400 Chinese characters. That 400-character instruction is not an actual output-token count. Both actual prompt/output counts, per-run measurements, unrounded timings, dispersion, output cap and mean weighting are unavailable. Source explicitly calculates decode from Ollama eval_count/eval_duration, and PP from prompt_eval_count/prompt_eval_duration; it excludes client/network and model-loading overhead. PP is not prompt length divided by TTFT, and 4096 is the configured context, not measured input length. The individual native counters are not published, so these averages cannot be independently recalculated. Raw harness/build commit is also unknown. Reported loaded memory for this model is 20.29 GB including KV cache at configured 4096. This is a source-reported usage figure, not installed capacity or dedicated VRAM. Unified memory is 32 GB. Cache warmness/reuse, cached-token counts, batch size/concurrency, warmup, repetitions beyond the three speed runs, FA/KV precision, speculation/MTP, thinking mode and sampling parameters for the speed test are unspecified. Do not infer them from current Ollama defaults. The article's separate 200-question accuracy tests and 84 coding problems use other generation settings. Their average output lengths, thinking switches, 8192/6912 and 2048/256 contexts/budgets are not this speed benchmark's counts or configuration. The independent SymbolStar E4B/12B results use different Unsloth files, tasks, context settings and method; do not merge the campaigns. These measurements do not establish larger-context behavior, multi-user throughput or independently validated model quality.

Oct 13, 2026
Apple hardware
generation:
32.0 tokens/s
prompt processing (prefill):
657.0 tokens/s
quant:
Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, sections describing benchmark methodology and Mac mini M4 token speed. Page lists June 7, 2026 publication and August 13, 2026 update. Exact benchmark date is not given. Recovered October 13; no local reproduction performed. One Apple Mac mini M4 with 32 GB unified memory, Ollama 0.30.5, GGUF Q4_K_M, num_ctx=4096, source reports 100% GPU offload. Exact CPU/GPU core counts, macOS, power, clocks, thermals and storage are not reported. Do not borrow hardware details or flags from SymbolStar's independent M4 mini campaign. Measured model is Ollama library gemma4:26b, obtained through ollama pull; source explicitly says its benchmarks use the library's GGUF Q4_K_M builds rather than downloaded HF originals or a self-made quant. Current tag identity verified at https://ollama.com/library/gemma4:26b . Upstream model identity verified at https://huggingface.co/google/gemma-4-26B-A4B-it . This HF link identifies the original 26B-A4B-it model, not the exact tested GGUF. The historical Ollama manifest/blob digest, tested model hash, upstream revision, quantizer and HF artifact-equivalent are unknown. Today's tag can contain newer weights, drafting and multiple runtime formats and must not substitute for the source's historical GGUF test. Headline is the source-reported three-run average for this model: native decode 32.04 tokens/s and native prompt processing 657 tokens/s. The speed test uses a fixed Chinese long-form prompt asking for an article of at least 400 Chinese characters. That 400-character instruction is not an actual output-token count. Both actual prompt/output counts, per-run measurements, unrounded timings, dispersion, output cap and mean weighting are unavailable. Source explicitly calculates decode from Ollama eval_count/eval_duration, and PP from prompt_eval_count/prompt_eval_duration; it excludes client/network and model-loading overhead. PP is not prompt length divided by TTFT, and 4096 is the configured context, not measured input length. The individual native counters are not published, so these averages cannot be independently recalculated. Raw harness/build commit is also unknown. Reported loaded memory for this model is 17.37 GB including KV cache at configured 4096. This is a source-reported usage figure, not installed capacity or dedicated VRAM. Unified memory is 32 GB. Cache warmness/reuse, cached-token counts, batch size/concurrency, warmup, repetitions beyond the three speed runs, FA/KV precision, speculation/MTP, thinking mode and sampling parameters for the speed test are unspecified. Do not infer them from current Ollama defaults. The article's separate 200-question accuracy tests and 84 coding problems use other generation settings. Their average output lengths, thinking switches, 8192/6912 and 2048/256 contexts/budgets are not this speed benchmark's counts or configuration. The independent SymbolStar E4B/12B results use different Unsloth files, tasks, context settings and method; do not merge the campaigns. These measurements do not establish larger-context behavior, multi-user throughput or independently validated model quality.

Oct 13, 2026
Apple hardware
generation:
12.5 tokens/s
prompt processing (prefill):
236.0 tokens/s
quant:
Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, sections describing benchmark methodology and Mac mini M4 token speed. Page lists June 7, 2026 publication and August 13, 2026 update. Exact benchmark date is not given. Recovered October 13; no local reproduction performed. One Apple Mac mini M4 with 32 GB unified memory, Ollama 0.30.5, GGUF Q4_K_M, num_ctx=4096, source reports 100% GPU offload. Exact CPU/GPU core counts, macOS, power, clocks, thermals and storage are not reported. Do not borrow hardware details or flags from SymbolStar's independent M4 mini campaign. Measured model is Ollama library gemma4:12b, obtained through ollama pull; source explicitly says its benchmarks use the library's GGUF Q4_K_M builds rather than downloaded HF originals or a self-made quant. Current tag identity verified at https://ollama.com/library/gemma4:12b . Upstream model identity verified at https://huggingface.co/google/gemma-4-12B-it . This HF link identifies the original 12B-it model, not the exact tested GGUF. The historical Ollama manifest/blob digest, tested model hash, upstream revision, quantizer and HF artifact-equivalent are unknown. Today's tag can contain newer weights, drafting and multiple runtime formats and must not substitute for the source's historical GGUF test. Headline is the source-reported three-run average for this model: native decode 12.5 tokens/s and native prompt processing 236 tokens/s. The speed test uses a fixed Chinese long-form prompt asking for an article of at least 400 Chinese characters. That 400-character instruction is not an actual output-token count. Both actual prompt/output counts, per-run measurements, unrounded timings, dispersion, output cap and mean weighting are unavailable. Source explicitly calculates decode from Ollama eval_count/eval_duration, and PP from prompt_eval_count/prompt_eval_duration; it excludes client/network and model-loading overhead. PP is not prompt length divided by TTFT, and 4096 is the configured context, not measured input length. The individual native counters are not published, so these averages cannot be independently recalculated. Raw harness/build commit is also unknown. Reported loaded memory for this model is 8.05 GB including KV cache at configured 4096. This is a source-reported usage figure, not installed capacity or dedicated VRAM. Unified memory is 32 GB. Cache warmness/reuse, cached-token counts, batch size/concurrency, warmup, repetitions beyond the three speed runs, FA/KV precision, speculation/MTP, thinking mode and sampling parameters for the speed test are unspecified. Do not infer them from current Ollama defaults. The article's separate 200-question accuracy tests and 84 coding problems use other generation settings. Their average output lengths, thinking switches, 8192/6912 and 2048/256 contexts/budgets are not this speed benchmark's counts or configuration. The independent SymbolStar E4B/12B results use different Unsloth files, tasks, context settings and method; do not merge the campaigns. These measurements do not establish larger-context behavior, multi-user throughput or independently validated model quality.

Oct 13, 2026
Apple hardware
generation:
29.9 tokens/s
prompt processing (prefill):
566.0 tokens/s
quant:
Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, sections describing benchmark methodology and Mac mini M4 token speed. Page lists June 7, 2026 publication and August 13, 2026 update. Exact benchmark date is not given. Recovered October 13; no local reproduction performed. One Apple Mac mini M4 with 32 GB unified memory, Ollama 0.30.5, GGUF Q4_K_M, num_ctx=4096, source reports 100% GPU offload. Exact CPU/GPU core counts, macOS, power, clocks, thermals and storage are not reported. Do not borrow hardware details or flags from SymbolStar's independent M4 mini campaign. Measured model is Ollama library gemma4:e4b, obtained through ollama pull; source explicitly says its benchmarks use the library's GGUF Q4_K_M builds rather than downloaded HF originals or a self-made quant. Current tag identity verified at https://ollama.com/library/gemma4:e4b . Upstream model identity verified at https://huggingface.co/google/gemma-4-E4B-it . This HF link identifies the original E4B-it model, not the exact tested GGUF. The historical Ollama manifest/blob digest, tested model hash, upstream revision, quantizer and HF artifact-equivalent are unknown. Today's tag can contain newer weights, drafting and multiple runtime formats and must not substitute for the source's historical GGUF test. Headline is the source-reported three-run average for this model: native decode 29.87 tokens/s and native prompt processing 566 tokens/s. The speed test uses a fixed Chinese long-form prompt asking for an article of at least 400 Chinese characters. That 400-character instruction is not an actual output-token count. Both actual prompt/output counts, per-run measurements, unrounded timings, dispersion, output cap and mean weighting are unavailable. Source explicitly calculates decode from Ollama eval_count/eval_duration, and PP from prompt_eval_count/prompt_eval_duration; it excludes client/network and model-loading overhead. PP is not prompt length divided by TTFT, and 4096 is the configured context, not measured input length. The individual native counters are not published, so these averages cannot be independently recalculated. Raw harness/build commit is also unknown. Reported loaded memory for this model is 9.55 GB including KV cache at configured 4096. This is a source-reported usage figure, not installed capacity or dedicated VRAM. Unified memory is 32 GB. Cache warmness/reuse, cached-token counts, batch size/concurrency, warmup, repetitions beyond the three speed runs, FA/KV precision, speculation/MTP, thinking mode and sampling parameters for the speed test are unspecified. Do not infer them from current Ollama defaults. The article's separate 200-question accuracy tests and 84 coding problems use other generation settings. Their average output lengths, thinking switches, 8192/6912 and 2048/256 contexts/budgets are not this speed benchmark's counts or configuration. The independent SymbolStar E4B/12B results use different Unsloth files, tasks, context settings and method; do not merge the campaigns. These measurements do not establish larger-context behavior, multi-user throughput or independently validated model quality.

Oct 13, 2026
Apple hardware
generation:
55.0 tokens/s
prompt processing (prefill):
1089.0 tokens/s
quant:
Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary source: https://klab.tw/2026/06/gemma4-benchmark/ , KodeLab author Kyle, sections describing benchmark methodology and Mac mini M4 token speed. Page lists June 7, 2026 publication and August 13, 2026 update. Exact benchmark date is not given. Recovered October 13; no local reproduction performed. One Apple Mac mini M4 with 32 GB unified memory, Ollama 0.30.5, GGUF Q4_K_M, num_ctx=4096, source reports 100% GPU offload. Exact CPU/GPU core counts, macOS, power, clocks, thermals and storage are not reported. Do not borrow hardware details or flags from SymbolStar's independent M4 mini campaign. Measured model is Ollama library gemma4:e2b, obtained through ollama pull; source explicitly says its benchmarks use the library's GGUF Q4_K_M builds rather than downloaded HF originals or a self-made quant. Current tag identity verified at https://ollama.com/library/gemma4:e2b . Upstream model identity verified at https://huggingface.co/google/gemma-4-E2B-it . This HF link identifies the original E2B-it model, not the exact tested GGUF. The historical Ollama manifest/blob digest, tested model hash, upstream revision, quantizer and HF artifact-equivalent are unknown. Today's tag can contain newer weights, drafting and multiple runtime formats and must not substitute for the source's historical GGUF test. Headline is the source-reported three-run average for this model: native decode 54.97 tokens/s and native prompt processing 1089 tokens/s. The speed test uses a fixed Chinese long-form prompt asking for an article of at least 400 Chinese characters. That 400-character instruction is not an actual output-token count. Both actual prompt/output counts, per-run measurements, unrounded timings, dispersion, output cap and mean weighting are unavailable. Source explicitly calculates decode from Ollama eval_count/eval_duration, and PP from prompt_eval_count/prompt_eval_duration; it excludes client/network and model-loading overhead. PP is not prompt length divided by TTFT, and 4096 is the configured context, not measured input length. The individual native counters are not published, so these averages cannot be independently recalculated. Raw harness/build commit is also unknown. Reported loaded memory for this model is 7.06 GB including KV cache at configured 4096. This is a source-reported usage figure, not installed capacity or dedicated VRAM. Unified memory is 32 GB. Cache warmness/reuse, cached-token counts, batch size/concurrency, warmup, repetitions beyond the three speed runs, FA/KV precision, speculation/MTP, thinking mode and sampling parameters for the speed test are unspecified. Do not infer them from current Ollama defaults. The article's separate 200-question accuracy tests and 84 coding problems use other generation settings. Their average output lengths, thinking switches, 8192/6912 and 2048/256 contexts/budgets are not this speed benchmark's counts or configuration. The independent SymbolStar E4B/12B results use different Unsloth files, tasks, context settings and method; do not merge the campaigns. These measurements do not establish larger-context behavior, multi-user throughput or independently validated model quality.

Oct 13, 2026
Apple hardware
generation:
8.7 tokens/s
prompt processing (prefill):
110.4 tokens/s
quant:
Q4_K_M (GGUF)
kv:
Q8
flash attention:
on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported campaign measured 2026-06-05 by Kelly, published in SymbolStar/gemma4-mac-mini-bench; recovered 2026-10-13. This is newly recovered historical evidence, not a new run or local reproduction. Primary report: https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/REPORT-longctx.md . Data: https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/data/longctx-results.json . Short-task report/data: https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/REPORT.md and https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/data/results.json . Headline is one source row, the first long-context tier: 8.7 tokens/s decode and 110.4 tokens/s native prompt processing, 5713 actual input tokens and 512 actual output tokens, with num_ctx=8192. This is not an average, maximum, concurrency throughput or a full 8192-token prompt. The same-row source TTFT estimate is 64.5 s and request wall time 123.4 s. Hardware: one Apple Mac mini M4, Mac16,10, 10 CPU cores (4P+6E), 10 GPU cores, 32 GB unified memory; macOS 26.5.1. Ollama detected Metal and 25.0 GiB available in the unified pool; that available-memory value is not dedicated VRAM or installed RAM. Power, clock, temperature, storage model and GPU offload layer count are unknown. Model: unsloth/gemma-4-12b-it-GGUF, gemma-4-12b-it-Q4_K_M.gguf, GGUF Q4_K_M. The source says it downloaded Unsloth GGUF files and registered local aliases gemma4-e4b-q4 and gemma4-12b-q4 after the registry pull stalled. The generic README reproduction command uses the Ollama registry; do not substitute today's registry build for these measured Unsloth files. HF identity and currently available file were verified at https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/blob/main/gemma-4-12b-it-Q4_K_M.gguf. Historical measured HF revision, file hash and complete Modelfile are unknown. Current HF main is not the tested revision. Runtime: Ollama 0.30.5 macOS cask build. Source flags: OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0. Text-only local /api/generate calls, serial request loop, stream=false. Long-context request explicitly uses think=false, temperature=0.3, num_predict=512 and the listed num_ctx. Speculative decoding/MTP and other sampling settings are not reported and must remain unknown. Long-context workload: prefixes of Project Gutenberg's The Adventures of Sherlock Holmes plus a fixed Chinese structured-summary request. Input construction uses 3.5 characters per target token, while saved actual input counts come from Ollama prompt_eval_count; nominal targets are not measured token counts. Each of the listed tiers has one stored run. The script changes num_ctx per request. The report says KV allocation changes between tiers; no explicit cache flush, cached-token counter or independent cold/warm validation is provided, so do not label these as proven cold-prefill measurements. Native metric definitions verified in the published longctx.py: prompt_eval_count / (prompt_eval_duration in seconds) and eval_count / (eval_duration in seconds). It stores rounded PP to 1 decimal and TG to 2 decimals. The source's TTFT is (load_duration + prompt_eval_duration), a server-phase estimate from a nonstreaming request, not measured first-token arrival. PP is not input divided by TTFT or request wall time. Original unrounded phase durations are not retained in longctx-results.json. All long-context rows for this model (configured context; actual input/output; native PP/TG; source phase-TTFT estimate; request wall): 8192; 5713/512; 110.4/8.7 tokens/s; 64.5 s; 123.4 s 16384; 12707/512; 99.8/9.95 tokens/s; 129.35 s; 180.9 s 32768; 26279/489; 92.8/8.38 tokens/s; 284.78 s; 343.2 s Each row records swap_after=27 MB and swap_delta=0 MB. Thus there was no measured increase, not zero pre-existing swap. The script's mem_pct_after is a vm_stat non-free-page calculation and not a reliable macOS memory-pressure or model-allocation measurement; it is not used as VRAM/RAM capacity. Separate short-task campaign, same source setup and configured context8192, one warmup 'Say OK.' call per model then a2-second pause, then translation, SwiftUI coding and pandas analysis in sequence. The published bench.py omits explicit think/temperature options, so their measured settings remain unknown; do not copy long-context think=false into these rows. TG is eval_count/eval_duration. In each line below: task; actual input/output; prompt/eval durations in seconds; TG; load duration; source TTFT estimate; request wall. translate; 94/1100; 1.400089000/136.793086000; 8.04134209 tokens/s; 0.258447625 s; 1.658536625 s; 138.45989299 s swift_code; 74/2232; 0.871346000/267.384022000; 8.34754442 tokens/s; 0.265564333 s; 1.136910333 s; 268.54477787 s data_analysis; 174/1689; 1.975874000/215.092092000; 7.85245048 tokens/s; 0.259922250 s; 2.235796250 s; 217.35145402 s Short-task prompt rates can be recomputed from the retained native counts/durations, but are not combined with the long-context headline. Short-task 'peak_rss_gb' is a post-request ps RSS sample over matching processes, not a continuously monitored GPU peak. Repository provenance: data files show commit f6cffa8f4680ce1e0dcab8b21ee65b281c88d2ce (June6). Published harness currently shows 2bc218b3fa7b931916f837a5b3cb406571d1155a, whose commit description says it sanitizes local paths. The historical executable/harness revision is not pinned to the run, and the saved preview length differs from the current script; use the published script to establish the reported metric method, not a claim of byte-for-byte reproduction. Model quality is not independently validated by this throughput campaign.

Oct 13, 2026
Apple hardware
generation:
16.7 tokens/s
prompt processing (prefill):
245.5 tokens/s
quant:
Q4_K_M (GGUF)
kv:
Q8
flash attention:
on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source-reported campaign measured 2026-06-05 by Kelly, published in SymbolStar/gemma4-mac-mini-bench; recovered 2026-10-13. This is newly recovered historical evidence, not a new run or local reproduction. Primary report: https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/REPORT-longctx.md . Data: https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/data/longctx-results.json . Short-task report/data: https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/REPORT.md and https://github.com/SymbolStar/gemma4-mac-mini-bench/blob/main/data/results.json . Headline is one source row, the first long-context tier: 16.66 tokens/s decode and 245.5 tokens/s native prompt processing, 5709 actual input tokens and 505 actual output tokens, with num_ctx=8192. This is not an average, maximum, concurrency throughput or a full 8192-token prompt. The same-row source TTFT estimate is 23.53 s and request wall time 53.9 s. Hardware: one Apple Mac mini M4, Mac16,10, 10 CPU cores (4P+6E), 10 GPU cores, 32 GB unified memory; macOS 26.5.1. Ollama detected Metal and 25.0 GiB available in the unified pool; that available-memory value is not dedicated VRAM or installed RAM. Power, clock, temperature, storage model and GPU offload layer count are unknown. Model: unsloth/gemma-4-E4B-it-GGUF, gemma-4-E4B-it-Q4_K_M.gguf, GGUF Q4_K_M. The source says it downloaded Unsloth GGUF files and registered local aliases gemma4-e4b-q4 and gemma4-12b-q4 after the registry pull stalled. The generic README reproduction command uses the Ollama registry; do not substitute today's registry build for these measured Unsloth files. HF identity and currently available file were verified at https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF/blob/main/gemma-4-E4B-it-Q4_K_M.gguf. Historical measured HF revision, file hash and complete Modelfile are unknown. Current HF main is not the tested revision. Runtime: Ollama 0.30.5 macOS cask build. Source flags: OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0. Text-only local /api/generate calls, serial request loop, stream=false. Long-context request explicitly uses think=false, temperature=0.3, num_predict=512 and the listed num_ctx. Speculative decoding/MTP and other sampling settings are not reported and must remain unknown. Long-context workload: prefixes of Project Gutenberg's The Adventures of Sherlock Holmes plus a fixed Chinese structured-summary request. Input construction uses 3.5 characters per target token, while saved actual input counts come from Ollama prompt_eval_count; nominal targets are not measured token counts. Each of the listed tiers has one stored run. The script changes num_ctx per request. The report says KV allocation changes between tiers; no explicit cache flush, cached-token counter or independent cold/warm validation is provided, so do not label these as proven cold-prefill measurements. Native metric definitions verified in the published longctx.py: prompt_eval_count / (prompt_eval_duration in seconds) and eval_count / (eval_duration in seconds). It stores rounded PP to 1 decimal and TG to 2 decimals. The source's TTFT is (load_duration + prompt_eval_duration), a server-phase estimate from a nonstreaming request, not measured first-token arrival. PP is not input divided by TTFT or request wall time. Original unrounded phase durations are not retained in longctx-results.json. All long-context rows for this model (configured context; actual input/output; native PP/TG; source phase-TTFT estimate; request wall): 8192; 5709/505; 245.5/16.66 tokens/s; 23.53 s; 53.9 s 16384; 12703/512; 216.6/14.85 tokens/s; 60.31 s; 94.9 s 32768; 26275/479; 186.6/12.72 tokens/s; 142.51 s; 180.3 s 65536; 52358/512; 175/12.24 tokens/s; 300.84 s; 342.9 s Each row records swap_after=27 MB and swap_delta=0 MB. Thus there was no measured increase, not zero pre-existing swap. The script's mem_pct_after is a vm_stat non-free-page calculation and not a reliable macOS memory-pressure or model-allocation measurement; it is not used as VRAM/RAM capacity. Separate short-task campaign, same source setup and configured context8192, one warmup 'Say OK.' call per model then a2-second pause, then translation, SwiftUI coding and pandas analysis in sequence. The published bench.py omits explicit think/temperature options, so their measured settings remain unknown; do not copy long-context think=false into these rows. TG is eval_count/eval_duration. In each line below: task; actual input/output; prompt/eval durations in seconds; TG; load duration; source TTFT estimate; request wall. translate; 94/864; 0.420291000/44.762873000; 19.30171015 tokens/s; 0.260641833 s; 0.680932833 s; 45.44966912 s swift_code; 74/1926; 0.379493000/100.263726000; 19.20933998 tokens/s; 0.274004667 s; 0.653497667 s; 100.92706800 s data_analysis; 174/1256; 0.620109000/70.736921000; 17.75593258 tokens/s; 0.275672542 s; 0.895781542 s; 71.64154196 s Short-task prompt rates can be recomputed from the retained native counts/durations, but are not combined with the long-context headline. Short-task 'peak_rss_gb' is a post-request ps RSS sample over matching processes, not a continuously monitored GPU peak. Repository provenance: data files show commit f6cffa8f4680ce1e0dcab8b21ee65b281c88d2ce (June6). Published harness currently shows 2bc218b3fa7b931916f837a5b3cb406571d1155a, whose commit description says it sanitizes local paths. The historical executable/harness revision is not pinned to the run, and the saved preview length differs from the current script; use the published script to establish the reported metric method, not a claim of byte-for-byte reproduction. Model quality is not independently validated by this throughput campaign.

Oct 13, 2026
Showing 1–7 of 7
Page 1 of 1

Community benchmarks snapshot

Records by GPU

AMD Strix Halo 128GB294NVIDIA RTX 3090190NVIDIA RTX 5090134NVIDIA DGX Spark77M5 Max 48GB65NVIDIA RTX Pro 6000 Blackwell62NVIDIA RTX 5060 Ti 16GB58NVIDIA RTX 3060 12GB54AMD Radeon AI PRO R9700 32GB50M4 Max 128GB49

Records by model

2211 total
Qwen3.81021
Qwen3.6238
DeepSeek V4 Flash147
Gemma 4144
Qwen3.597
Qwen353
other511

Records by engine

1645 total
llama.cpp875
vLLM189
MLX100
Strata70
Ollama62
other349

Use cases

coding 556agentic 357long-context 272tool-use 153vision 109summarization 53math 43creative-writing 37multilingual 23text-generation 9rp 8reasoning 3
coding556agentic357long-context272tool-use153vision109summarization53math43creative-writing37

Median generation t/s by GPU

On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking

RTX 509082RTX Pro 600067M5 Ultra 256GB50RX 7900 XTX48RTX 309036RTX 5090 Laptop 24GB34RTX 409032V100 32GB31Radeon AI PRO R9700 32GB30RX 7800 XT 16GB30

Prompt processing, same model

M5 Ultra 256GB1800RX 7900 XTX530RTX 3090561RTX 40901949V100 32GB941RTX 3080 20GB997Arc Pro B70512M1 Ultra 128GB153Arc Pro B60 24GB380M1 Max 32GB82

Reports by model size

Qwen3.8 27B566Qwen3.8 125B · 6B active371DeepSeek V4 Flash 284B · 13B active130Qwen3.6 35B · 3B active129Qwen3.6 27B72Gemma 4 26B · 4B active36GLM-5.3 320B · 18B active33DeepSeek V4.1 Flash 552B · 16B active28

Quants

Q4_K_M290UD-Q4_K_XL140NVFP4120Q8_0984-bit79Q4_K_XL71IQ4_XS70MXFP458IQ3_XXS52Q450