Qwen3 14B; 4-bit; native 32768-prompt/128-generation

NetworkChuck campaign measured 2026-09-20, recovered 2026-10-13; historical source evidence, no new/local run. Merged window 2026-09-20T14:41:54.718Z–2026-09-20T16:55:25.045Z; native-cell timestamps unavailable. Credit NetworkChuck, CC BY 4.0 data/docs. Repository: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/ . Native data: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/engine.json . Client summary: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/summary.json . Client requests: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/data/language-models/runs.jsonl . Method: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/docs/method.md . Headline: native 32768-input/128-generation cell, PP median 2529.82, TG median 66.962 tokens/s across five trials; not pooled-duration or client rates. Context 32768 is actual native input length, not supported maximum or configured KV capacity. Batch 1. Hardware: one Apple Mac Studio M5 Ultra (36-core CPU, 80-core GPU), CPU core composition 12 super+24 performance, 256 GB installed unified memory, macOS 27.0, GPU architecture applegpu_g17d. Both review units had 8 TB storage. Installed unified memory is not dedicated VRAM. This M5 Ultra was loaned by Apple for the review. The comparison M3 Ultra used macOS 26.5.1 and had an indexing process occupying one CPU core; neither OS nor background-load effects were isolated. Power, energy, thermals, clock rates, exact storage part, explicit GPU-offload count and swap state are not reported for these rows. Model: mlx-community/Qwen3-14B-4bit, 4-bit MLX Safetensors; public identity verified 2026-10-13: https://huggingface.co/mlx-community/Qwen3-14B-4bit . Source calls it dense, nominal 14B total/active, local folder ~/bench-models/Qwen3-14B-4bit. Source disk label 7.8 GB is not runtime memory. Tested HF revision/hash unrecorded; author says weights were not hash-verified. Current HF main, conversion version and download size do not identify the historical snapshot/runtime. Quant group size/mixed-precision exceptions unverified. Runtime: mlx-lm 0.31.3 and MLX 0.32.2; server fingerprint 0.31.3-0.32.2-macOS-27.0-arm64-arm-64bit-applegpu_g17d. Native harness command is mlx_lm.benchmark --model <local-model-folder> -p <prompt-count> -g 128 -n 5, verified in https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/bench/ssh.ts . Model loaded before timing; one discarded warm-up then five trials. Harness median/min/max uses three-decimal printed trials; raw output retained. Warm-loaded, excludes cold loading. Native method: versioned https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py seeds RNG 0, uses random token IDs, disables EOS stopping and calls stream_generate; no HTTP essay/chat workload. Published command leaves batch 1, prefill step 2048 and delay 0 defaults. https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py defines prompt_tps=prompt.size/runtime time to first generated token, including that boundary rather than pure matrix-kernel time. Generation timing restarts there; generation_tps=generated count/runtime generation-window time, including final detokenizer handling, final count 128. This differs from client (128−1)/first-to-last text. No prompt_cache passed, so generate_step creates fresh KV. No draft model, KV-quantization or activation-quantization flag supplied. Flash/kernel selection and runtime build commit unrecorded; hardware alone does not establish a toggle. All native rows, input/output; PP median [min–max]; TG median [min–max], tokens/s; source peak-memory GB; trial count: 512/128; PP 2885.105 [2879.185–2887.773]; TG 102.517 [102.496–102.535]; peak 8.826 GB; n=5 4096/128; PP 3542.723 [3365.751–3587.111]; TG 96.019 [95.364–96.031]; peak 9.701 GB; n=5 16384/128; PP 3073.941 [3041.921–3074.689]; TG 80.093 [80.066–80.115]; peak 11.602 GB; n=5 32768/128; PP 2529.82 [2529.664–2530.48]; TG 66.962 [66.938–66.972]; peak 14.332 GB; n=5 Memory semantics: source peakMemoryGb is the maximum of printed native peak_memory values across five trials. Upstream peak_memory is mx.get_peak_memory()/1e9, an MLX allocator high-water mark in decimal GB; it is not RSS, installed RAM, dedicated VRAM, or a separately reset per-trial allocation. Warm-up and earlier allocation in that benchmark process can contribute to the high-water mark. No model-load timing or energy number is supplied by these rows. Separate HTTP arm, same model/Mac: serial streaming /v1/chat/completions from LAN controller to mlx_lm.server; --prompt-cache-size 0 and --chat-template-args '{"enable_thinking":false}' , temperature 0, max_tokens 128. Prompt repeats prompts/sample.txt with unique run-ID prefix and summary instruction. One discarded warm-up, five measured requests/cell, 30 s between cells. Single 100K requests, if listed, are not five-repeat cells. Actual counts from usage. All checked raw requests in this group: cachedTokens 0, outputTokens 128, firstWordField content. HTTP cache evidence does not mean cold model loading. Controller monotonic timing begins before request; first text is first nonempty delta. Reading=input tokens/client first-text seconds, including LAN/HTTP. Writing=(output−1)/(last-text−first-text seconds). Total ends at stream completion. These differ from native metrics. Implementation: https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/lane.ts and https://github.com/theNetworkChuck/mac-studio-m5-ultra-local-ai/blob/main/harness/src/timing.ts . All HTTP rows, each metric shown as median [min–max]; nominal length; actual token counts; client reading/writing tokens/s; first-text and total seconds; repetitions: 512 (512 nominal); actual input 507 [506–509], output 128 [128–128]; TTFT-derived reading 1902.61750633 [1856.64104818–1909.08816516]; client writing 92.2522627228 [92.1225767047–92.3128810632]; first text 0.266475 [0.265048–0.274151] s; total 1.644405 [1.64122–1.650881] s; n=5 4K (4096 nominal); actual input 4100 [4098–4101], output 128 [128–128]; TTFT-derived reading 3170.20057838 [3160.40663539–3203.87090119]; client writing 86.3066072805 [86.3005665258–86.3724065325]; first text 1.293609 [1.279702–1.296985] s; total 2.76521 [2.751289–2.767449] s; n=5 16K (16384 nominal); actual input 16282 [16280–16283], output 128 [128–128]; TTFT-derived reading 2799.66424665 [2796.76769856–2800.17191081]; client writing 68.2793129273 [68.0394929076–68.5252661289]; first text 5.815585 [5.814643–5.821363] s; total 7.681837 [7.671321–7.683826] s; n=5 32K (32768 nominal); actual input 32527 [32526–32528], output 128 [128–128]; TTFT-derived reading 2277.80207751 [2276.97725949–2278.9217499]; client writing 55.420495372 [55.3967563672–55.5510842522]; first text 14.279994 [14.272978–14.285606] s; total 16.57192 [16.567309–16.57732] s; n=5 Independent M3/M5 invocations were merged with shared settings checked. Different OS, M3 indexing load and unverified weight snapshots confound silicon-only comparisons. No independent model-quality evaluation.