Qwen3 14B; 4-bit; native 32768-prompt/128-generation

NetworkChuck campaign measured 2026-09-20, recovered 2026-10-13; historical source evidence, no new/local run. Merged window 2026-09-20T14:41:54.718Z–2026-09-20T16:55:25.045Z; native-cell timestamps unavailable. Credit NetworkChuck, CC BY 4.0 data/docs. Repository: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/ . Native data: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/engine.json . Client summary: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/summary.json . Client requests: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/runs.jsonl . Method: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/docs/method.md . Headline: native 32768-input/128-generation cell, PP median 570.075, TG median 35.801 tokens/s across five trials; not pooled-duration or client rates. Context 32768 is actual native input length, not supported maximum or configured KV capacity. Batch 1. Hardware: one Apple Mac Studio M3 Ultra (32-core CPU, 80-core GPU), CPU core composition 24 performance+8 efficiency, 512 GB installed unified memory, macOS 26.5.1, GPU architecture applegpu_g15d. Both review units had 8 TB storage. Installed unified memory is not dedicated VRAM. A stuck background indexing process occupied 100% of one of this M3 Ultra's 32 CPU cores throughout this campaign, until 2026-09-21 10:10 CT. The effect was not measured by rerunning; do not silently correct the rates. This was Chuck's own machine. Power, energy, thermals, clock rates, exact storage part, explicit GPU-offload count and swap state are not reported for these rows. Model: mlx-community/Qwen3-14B-4bit, 4-bit MLX Safetensors; public identity verified 2026-10-13: https://huggingface.co/mlx-community/Qwen3-14B-4bit . Source calls it dense, nominal 14B total/active, local folder ~/bench-models/Qwen3-14B-4bit. Source disk label 7.8 GB is not runtime memory. Tested HF revision/hash unrecorded; author says weights were not hash-verified. Current HF main, conversion version and download size do not identify the historical snapshot/runtime. Quant group size/mixed-precision exceptions unverified. Runtime: mlx-lm 0.31.3 and MLX 0.32.2; server fingerprint 0.31.3-0.32.2-macOS-26.5.1-arm64-arm-64bit-applegpu_g15d. Native harness command is mlx_lm.benchmark --model <local-model-folder> -p <prompt-count> -g 128 -n 5, verified in https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/bench/ssh.ts . Model loaded before timing; one discarded warm-up then five trials. Harness median/min/max uses three-decimal printed trials; raw output retained. Warm-loaded, excludes cold loading. Native method: versioned https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py seeds RNG 0, uses random token IDs, disables EOS stopping and calls stream_generate; no HTTP essay/chat workload. Published command leaves batch 1, prefill step 2048 and delay 0 defaults. https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py defines prompt_tps=prompt.size/runtime time to first generated token, including that boundary rather than pure matrix-kernel time. Generation timing restarts there; generation_tps=generated count/runtime generation-window time, including final detokenizer handling, final count 128. This differs from client (128−1)/first-to-last text. No prompt_cache passed, so generate_step creates fresh KV. No draft model, KV-quantization or activation-quantization flag supplied. Flash/kernel selection and runtime build commit unrecorded; hardware alone does not establish a toggle. All native rows, input/output; PP median [min–max]; TG median [min–max], tokens/s; source peak-memory GB; trial count: 512/128; PP 813.779 [812.584–814.936]; TG 69.365 [69.32–69.391]; peak 8.826 GB; n=5 4096/128; PP 814.254 [813.998–815.156]; TG 62.896 [62.85–62.918]; peak 9.701 GB; n=5 16384/128; PP 690.643 [690.481–691.238]; TG 47.455 [47.42–47.465]; peak 11.602 GB; n=5 32768/128; PP 570.075 [569.782–570.199]; TG 35.801 [35.795–35.812]; peak 14.332 GB; n=5 Memory semantics: source peakMemoryGb is the maximum of printed native peak_memory values across five trials. Upstream peak_memory is mx.get_peak_memory()/1e9, an MLX allocator high-water mark in decimal GB; it is not RSS, installed RAM, dedicated VRAM, or a separately reset per-trial allocation. Warm-up and earlier allocation in that benchmark process can contribute to the high-water mark. No model-load timing or energy number is supplied by these rows. Separate HTTP arm, same model/Mac: serial streaming /v1/chat/completions from LAN controller to mlx_lm.server; --prompt-cache-size 0 and --chat-template-args '{"enable_thinking":false}' , temperature 0, max_tokens 128. Prompt repeats prompts/sample.txt with unique run-ID prefix and summary instruction. One discarded warm-up, five measured requests/cell, 30 s between cells. Single 100K requests, if listed, are not five-repeat cells. Actual counts from usage. All checked raw requests in this group: cachedTokens 0, outputTokens 128, firstWordField content. HTTP cache evidence does not mean cold model loading. Controller monotonic timing begins before request; first text is first nonempty delta. Reading=input tokens/client first-text seconds, including LAN/HTTP. Writing=(output−1)/(last-text−first-text seconds). Total ends at stream completion. These differ from native metrics. Implementation: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/lane.ts and https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/timing.ts . All HTTP rows, each metric shown as median [min–max]; nominal length; actual token counts; client reading/writing tokens/s; first-text and total seconds; repetitions: 512 (512 nominal); actual input 507 [507–508], output 128 [128–128]; TTFT-derived reading 681.304886619 [680.15894523–683.748234999]; client writing 61.7620769176 [61.5733692754–61.8337956262]; first text 0.744678 [0.741501–0.745628] s; total 2.801364 [2.797872–2.808342] s; n=5 4K (4096 nominal); actual input 4097 [4095–4097], output 128 [128–128]; TTFT-derived reading 776.541288513 [774.603443497–777.677154077]; client writing 55.7032446921 [55.5113601595–55.8309246173]; first text 5.275959 [5.266477–5.289158] s; total 7.555642 [7.54761–7.577066] s; n=5 16K (16384 nominal); actual input 16280 [16279–16281], output 128 [128–128]; TTFT-derived reading 669.146411504 [668.529108244–669.423496892]; client writing 41.7188837078 [41.4444331315–41.7777862576]; first text 24.330998 [24.320927–24.350473] s; total 27.382021 [27.368085–27.394455] s; n=5 32K (32768 nominal); actual input 32521 [32520–32521], output 128 [128–128]; TTFT-derived reading 550.950577802 [550.6959282–551.18283725]; client writing 31.5317764581 [31.4994611608–31.5856519565]; first text 59.025261 [59.0022–59.054368] s; total 63.05821 [63.031217–63.090287] s; n=5 Independent M3/M5 invocations were merged with shared settings checked. Different OS, M3 indexing load and unverified weight snapshots confound silicon-only comparisons. No independent model-quality evaluation.