Qwen3.8 27B; 4-bit; native 32768-prompt/128-generation

NetworkChuck, measured 2026-09-20; recovered 2026-10-13, not locally rerun. Measurements/docs: CC BY 4.0. Native data: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/engine.json ; HTTP summary/raw: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/summary.json ; https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/runs.jsonl . Method: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/docs/method.md . Merged suite window 14:41:54.718–16:55:25.045 UTC; native cells have no individual timestamps. Native 32768-input/128-output headline: PP=391.157, TG=31.768 tokens/s, separate five-trial medians, not pooled across lengths. Input is exactly 32768 synthetic tokens. Configured KV capacity unknown and blank. Batch 1; one device. Hardware: Apple Mac Studio M3 Ultra (32-core CPU, 80-core GPU); 512GB installed unified memory; macOS26.5.1; applegpu_g15d; CPU composition 24 performance+8 efficiency; 8TB storage. Unified RAM and allocator peaks are not dedicated VRAM. M3 was Chuck's own; a stuck indexer consumed one CPU core during these tests. Effect unmeasured, no corrective rerun. OS/background effects were not isolated. Power, thermals, clocks, swap and offload count unknown. Model: mlx-community/Qwen3.8-27B-4bit; 4-bit; MLX Safetensors. Public identity/precision verified 2026-10-13: https://huggingface.co/mlx-community/Qwen3.8-27B-4bit . Source labels dense, nominal total/active 27/27B; rounded labels, not exact architecture counts. Tested HF revision/hash and group/mixed-precision details unknown; weights not hash-checked. Current HF does not identify the historical snapshot/runtime. Runtime: mlx-lm0.31.3/MLX0.32.2; fingerprint 0.31.3-0.32.2-macOS-26.5.1-arm64-arm-64bit-applegpu_g15d. Harness command: mlx_lm.benchmark --model <folder> -p <input> -g 128 -n 5. Command/parser: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/bench/ssh.ts ; https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/bench/engine.ts . Model already loaded; one discarded warm-up, five trials. Rates printed to 3 decimals; harness uses their median/min/max, not tool means. Versioned runtime semantics: https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py and https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . Seed 0, random IDs rather than HTTP essay, EOS disabled, batch 1, default prefill step 2048, no trial delay. Native prompt_tps=prompt.size/time through first generated token; includes first-token work, not pure kernels. generation_tps=count/time after that boundary through final detokenization, count 128. Fresh KV each call: no prompt_cache passed. No draft/KV-quant/activation-quant flags; flash selection and build/source commit unknown. Native rows: input/output; PP/TG median[min,max] tokens/s; peak decimal GB; n trials: 512/128: PP=426.402[426.113,426.797]; TG=37.613[37.61,37.625]; peak=16.391GB; n=5 4096/128: PP=440.166[439.832,440.531]; TG=36.662[36.613,36.681]; peak=18.549GB; n=5 16384/128: PP=419.404[419.193,419.763]; TG=34.253[34.226,34.29]; peak=20.562GB; n=5 32768/128: PP=391.157[391.007,391.261]; TG=31.768[31.767,31.787]; peak=23.246GB; n=5 Peak=max printed MLX peak_memory, mx.get_peak_memory()/1e9 allocator high-water, not RSS/disk/RAM/VRAM. No trial reset shown; warm-up/earlier allocations may contribute. Load-time/energy unknown. HTTP: serial LAN streaming from separate controller; mlx_lm.server prompt-cache-size 0, thinking disabled, temperature 0, cap 128. Repeated sample.txt essay+summary instruction, unique run-ID prefix. One discarded warm-up+five measured requests/cell, 30-second rests; 100K has one measured run. Raw: cachedTokens 0, output 128, firstWordField=content. Uncached input, warm-loaded model; not cold loading. Controller monotonic timing starts before request; first nonempty text ends TTFT. Client read=input/TTFT; write=(output−1)/(last−first text time); total ends at stream completion. Network/HTTP included; not native PP/TG. Code: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/lane.ts . HTTP rows: nominal; actual in/out; read/write tokens/s; first/total seconds; median[min,max], n. Shown to 10 significant digits; raw retains full precision. 512: in=510[509,511]; out=128[128,128]; read=365.5672888[364.9190224,366.5407252]; write=32.8371052[32.78281437,32.87826705]; first=1.395092[1.391387,1.399296]; total=5.26326[5.257062,5.27352]; n=5 4K: in=4095[4095,4096]; out=128[128,128]; read=428.5118271[428.4438146,428.6736283]; write=31.91895501[31.87186206,31.96037115]; first=9.556329[9.552722,9.558784]; total=13.535332[13.530005,13.543677]; n=5 16K: in=16263[16262,16264]; out=128[128,128]; read=414.3083238[414.2026107,414.3782211]; write=29.52265585[29.45467795,29.76088651]; first=39.253375[39.248068,39.260979]; total=43.560296[43.52102,43.562865]; n=5 32K: in=32493[32492,32493]; out=128[128,128]; read=388.4928822[388.4450068,388.667172]; write=27.79181958[27.72220603,27.82305414]; first=83.638598[83.601092,83.646332]; total=88.207674[88.170197,88.222767]; n=5 Separate earlier M3 cohort: MLX 0.31.2 versus main 0.32.2; mlx-lm 0.31.3; timestamps 2026-09-20T11:59:09.611Z to 2026-09-20T12:15:59.604Z. Source: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/baseline-m3-ultra/runs.jsonl and https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/baseline-m3-ultra/engine.json . Later Sept20 baseline manifest does not cover all earlier timestamps. Earlier results excluded from headline. HTTP cachedTokens 0/output 128; one warm-up+five repeats, except 100K one measured. Same formulas; medians/ranges recomputed from raw times/counts. 512: in=511[509,511]; out=128[128,128]; read=357.2807356[354.9542866,358.1398315]; write=32.43838304[32.42378241,32.50086754]; first=1.428243[1.42465,1.439622]; total=5.341095[5.339991,5.356814]; n=5 4K: in=4096[4095,4097]; out=128[128,128]; read=416.5061266[416.1040813,416.8738735]; write=31.55226197[31.47394985,31.59191406]; first=9.835378[9.823115,9.843691]; total=13.858506[13.855747,13.869052]; n=5 16K: in=16269[16268,16269]; out=128[128,128]; read=403.654217[403.4194416,403.7293685]; write=29.24961158[29.18044313,29.46557794]; first=40.301821[40.296796,40.327754]; total=44.643474[44.612797,44.683133]; n=5 32K: in=32501[32500,32502]; out=128[128,128]; read=378.8949741[378.8573912,379.0772012]; write=27.46601567[27.29535682,27.52788933]; first=85.7798[85.739791,85.786897]; total=90.39477[90.383003,90.415871]; n=5 Earlier native MLX 0.31.2, same method, exact input/output; PP/TG median[min,max]; allocator peak GB; n: 512/128: PP=414.475[414.35,414.64]; TG=37.509[37.487,37.521]; peak=16.391GB; n=5 4096/128: PP=427.738[427.299,427.923]; TG=36.652[36.642,36.671]; peak=18.549GB; n=5 16384/128: PP=408.222[408.101,408.317]; TG=34.263[34.239,34.267]; peak=20.562GB; n=5 32768/128: PP=381.406[381.333,381.436]; TG=31.781[31.779,31.801]; peak=23.246GB; n=5 No independent model-quality validation.