Gemma 4 31B-it; 16-bit (bf16); native 32768-prompt/128-generation

NetworkChuck, measured 2026-09-20; recovered 2026-10-13, not locally rerun. Measurements/docs: CC BY 4.0. Native data: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/engine.json ; HTTP summary/raw: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/summary.json ; https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/runs.jsonl . Method: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/docs/method.md . Merged suite window 14:41:54.718–16:55:25.045 UTC; native cells have no individual timestamps. Native 32768-input/128-output headline: PP=295.894, TG=9.212 tokens/s, separate five-trial medians, not pooled across lengths. Input is exactly 32768 synthetic tokens. Configured KV capacity unknown and blank. Batch 1; one device. Hardware: Apple Mac Studio M3 Ultra (32-core CPU, 80-core GPU); 512GB installed unified memory; macOS26.5.1; applegpu_g15d; CPU composition 24 performance+8 efficiency; 8TB storage. Unified RAM and allocator peaks are not dedicated VRAM. M3 was Chuck's own; a stuck indexer consumed one CPU core during these tests. Effect unmeasured, no corrective rerun. OS/background effects were not isolated. Power, thermals, clocks, swap and offload count unknown. Model: mlx-community/gemma-4-31b-it-bf16; 16-bit (bf16); MLX Safetensors. Public identity/precision verified 2026-10-13: https://huggingface.co/mlx-community/gemma-4-31b-it-bf16 . Source labels dense, nominal total/active 31/31B; rounded labels, not exact architecture counts. Tested HF revision/hash and group/mixed-precision details unknown; weights not hash-checked. Current HF does not identify the historical snapshot/runtime. Runtime: mlx-lm0.31.3/MLX0.32.2; fingerprint 0.31.3-0.32.2-macOS-26.5.1-arm64-arm-64bit-applegpu_g15d. Harness command: mlx_lm.benchmark --model <folder> -p <input> -g 128 -n 5. Command/parser: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/bench/ssh.ts ; https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/bench/engine.ts . Model already loaded; one discarded warm-up, five trials. Rates printed to 3 decimals; harness uses their median/min/max, not tool means. Versioned runtime semantics: https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py and https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . Seed 0, random IDs rather than HTTP essay, EOS disabled, batch 1, default prefill step 2048, no trial delay. Native prompt_tps=prompt.size/time through first generated token; includes first-token work, not pure kernels. generation_tps=count/time after that boundary through final detokenization, count 128. Fresh KV each call: no prompt_cache passed. No draft/KV-quant/activation-quant flags; flash selection and build/source commit unknown. Native rows: input/output; PP/TG median[min,max] tokens/s; peak decimal GB; n trials: 512/128: PP=308.02[307.627,308.462]; TG=10.3[10.297,10.3]; peak=62.193GB; n=5 4096/128: PP=340.678[340.53,340.983]; TG=10.095[10.091,10.098]; peak=65.364GB; n=5 16384/128: PP=321.324[321.243,321.448]; TG=9.702[9.701,9.703]; peak=68.079GB; n=5 32768/128: PP=295.894[295.434,296.123]; TG=9.212[9.209,9.216]; peak=71.714GB; n=5 Peak=max printed MLX peak_memory, mx.get_peak_memory()/1e9 allocator high-water, not RSS/disk/RAM/VRAM. No trial reset shown; warm-up/earlier allocations may contribute. Load-time/energy unknown. HTTP: serial LAN streaming from separate controller; mlx_lm.server prompt-cache-size 0, thinking disabled, temperature 0, cap 128. Repeated sample.txt essay+summary instruction, unique run-ID prefix. One discarded warm-up+five measured requests/cell, 30-second rests; 100K has one measured run. Raw: cachedTokens 0, output 128, firstWordField=content. Uncached input, warm-loaded model; not cold loading. Controller monotonic timing starts before request; first nonempty text ends TTFT. Client read=input/TTFT; write=(output−1)/(last−first text time); total ends at stream completion. Network/HTTP included; not native PP/TG. Code: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/lane.ts . HTTP rows: nominal; actual in/out; read/write tokens/s; first/total seconds; median[min,max], n. Shown to 10 significant digits; raw retains full precision. 512: in=512[512,513]; out=128[128,128]; read=253.6292968[253.47664,253.7269868]; write=9.950550465[9.943366655,9.950901313]; first=2.019176[2.017917,2.022637]; total=14.783122[14.780995,14.791931]; n=5 4K: in=4097[4096,4099]; out=128[128,128]; read=329.8423873[329.5933841,330.3287307]; write=9.815066374[9.787594555,9.817879115]; first=12.421084[12.408851,12.433502]; total=25.358448[25.350521,25.409205]; n=5 16K: in=16256[16254,16257]; out=128[128,128]; read=318.583264[318.5137296,318.6403799]; write=9.446346648[9.434043556,9.453159298]; first=51.025691[51.019899,51.040186]; total=64.466356[64.460458,64.501646]; n=5 32K: in=32479[32479,32479]; out=128[128,128]; read=294.2110642[294.0358735,294.5408545]; write=8.960484335[8.955359437,8.961596525]; first=110.393537[110.269932,110.459311]; total=124.567014[124.451506,124.632582]; n=5 Current HF card names upstream google/gemma-4-31B-it revision3548789868c5356dbf307c98e6f609007b82b3eb; this is not the measured MLX snapshot revision. Separate earlier M3 cohort: MLX 0.31.2 versus main 0.32.2; mlx-lm 0.31.3; timestamps 2026-09-20T12:40:26.693Z to 2026-09-20T13:04:39.847Z. Source: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/baseline-m3-ultra/runs.jsonl and https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/baseline-m3-ultra/engine.json . Later Sept20 baseline manifest does not cover all earlier timestamps. Earlier results excluded from headline. HTTP cachedTokens 0/output 128; one warm-up+five repeats, except 100K one measured. Same formulas; medians/ranges recomputed from raw times/counts. 512: in=512[511,513]; out=128[128,128]; read=245.1500275[242.273689,245.5957379]; write=9.944198171[9.927911636,9.945870968]; first=2.088517[2.080655,2.11744]; total=14.863785[14.853517,14.887785]; n=5 4K: in=4097[4095,4098]; out=128[128,128]; read=328.0047178[327.5073216,328.347874]; write=9.803344145[9.797806449,9.808081334]; first=12.490674[12.47762,12.509644]; total=25.450789[25.42626,25.464178]; n=5 16K: in=16251[16251,16253]; out=128[128,128]; read=317.8314348[317.710641,317.9204948]; write=9.455167208[9.443309471,9.456347156]; first=51.130877[51.119699,51.150317]; total=64.562643[64.551698,64.599118]; n=5 32K: in=32470[32467,32471]; out=128[128,128]; read=293.8402701[293.705526,294.0009899]; write=8.981949746[8.977686138,8.984867714]; first=110.492003[110.441805,110.552908]; total=124.631643[124.584001,124.687886]; n=5 Earlier native MLX 0.31.2, same method, exact input/output; PP/TG median[min,max]; allocator peak GB; n: 512/128: PP=307.831[307.443,308.187]; TG=10.293[10.291,10.293]; peak=62.193GB; n=5 4096/128: PP=340.54[340.425,340.738]; TG=10.092[10.089,10.092]; peak=65.364GB; n=5 16384/128: PP=321.169[321.052,321.367]; TG=9.714[9.712,9.715]; peak=68.079GB; n=5 32768/128: PP=295.488[295.188,295.734]; TG=9.249[9.248,9.25]; peak=71.714GB; n=5 No independent model-quality validation.