Oct 13, 2026
Summary
Source-reported historical llama-bench campaign by karlhendrik, added 2026-09-30. Pinned primary result: https://gist.github.com/karlhendrik/355bb6cbf98861b21177a6b7857e79f7/7ff331f5e10c68699eac04848ca3840708f41061#file-results-round2-md , section Added 30 Sep 2026: Qwen3.8-27B. Gist revision 7ff331f5e10c68699eac04848ca3840708f41061, last updated 2026-09-30T10:05:52Z; recovered 2026-10-13, not rerun. Exact common benchmark harness: https://gist.github.com/karlhendrik/355bb6cbf98861b21177a6b7857e79f7/7ff331f5e10c68699eac04848ca3840708f41061#file-run_llama_bench-sh .
Hardware: TWO Apple Mac mini M4 machines, 16 GB installed unified RAM each, with model inference split 50/50 over a Thunderbolt 4 Bridge using llama.cpp RPC and Metal. Installed aggregate memory is 32 GB across separate hosts, not a contiguous 32 GB unified-memory device. Report hardware count must remain 2 and RAM 16 GB per device. M4a runs llama-bench; M4b runs ggml-rpc-server on MTL0. An M1 gateway mentioned elsewhere serves HTTP production clients and is not a third inference device in this native benchmark. Source GPU wired limit is 13,312 MB per mini, an allocation limit rather than measured memory. CPU/GPU core counts, macOS version, power mode, clocks, temperature, swap and precise background activity are unknown.
Model: Qwen3.8-27B dense, source GGUF architecture qwen35, Unsloth UD-Q4_K_M, source size 16.5 GB. Source publisher/model/quant/size identify the matching artifact: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-UD-Q4_K_M.gguf . Verified current pinned file page: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/blob/287999af67436bfaa4f4ee09dc34af328a495ba6/Qwen3.8-27B-UD-Q4_K_M.gguf ; current observed SHA256 322e194ff79741c7baa497c240f677f54b201b0efab44ca8e50f122b39123482. Neither that current revision nor hash is stated in the measured source, so tested revision and measured file checksum remain unknown. Source does not publish this model's exact local path or raw run log. Retain UD prefix; do not relabel as ordinary Q4_K_M or infer a uniform 4-bit tensor layout. Exact parameter counts, offloaded memory and KV allocation are absent.
Runtime: source-reported llama.cpp build 9731ad3 (18 Aug 2026), Metal and RPC. September 30 addition says same flags and build as above, with three repetitions and50/50 Thunderbolt split. Published common llama-bench command uses -fa 1 -ngl 99 -mmp 0 -b 1024 -ub 512 -p 512,4096 -n 128 -r 3 -o json, plus --rpc and -ts 50/50. Source notes comma-separated -ts 50,50 accidentally ran one host in an earlier discarded pass; the retained measurements use slash-separated split values. Big-model default 2048 batch failed compute-buffer allocation at the 13 GB limit; retained common flags reduce batch/microbatch to 1024/512. This is a serial single-stream benchmark across two devices, not two independent requests.
Native results, source means over 3 repetitions: pp512=60.5 tokens/s; pp4096=99.4 tokens/s; tg128=5.8 tokens/s. Structured PP selects pp512 60.5; structured TG selects tg128 5.8. The pp512, pp4096 and tg128 are SEPARATE native llama-bench phase tests, not a combined 512-input/128-output or 4096-input/128-output completion. PP tests process synthetic prompt lengths 512 and 4096 respectively; TG measures the native 128-token generation microtest. No standard deviation or individual samples are published. Raw llama-bench JSON is available only on request according to the author and was not requested.
Configured context capacity is unknown and left blank. The 131,072-context/q8_0-KV/temp 0.6 server config in README belongs to the production 35B RPC server, not these llama-bench settings, and must not be inherited. The benchmark harness does not set a KV dtype, CPU thread count or reusable request-prefix cache. Native bench rates are not HTTP end-to-end, TTFT or word-estimated rates. Exact timing internals of this author-reported build have not been independently reproduced. No speculative draft options are present in the native benchmark; the separate Ornith MTP experiment is not this model.
Source says one mini cannot fit this artifact at its 13 GB GPU limit; there is no one-device speed result. The 16.5 GB source size refers to the model artifact, not peak RAM/VRAM. Keep measured memory fields blank. Synthetic throughput does not establish answer quality. All rates are historical source claims, not an expectation for a current build or current HF revision.
Dedup: all 97 live approved M4-family catalog rows and all-status gist search read at recovery. Existing gist-linked report 1292 represents the 35B cluster campaign and is excluded; no Qwen3.8-27B two-M4 campaign was present. Existing report 1979 uses one 16 GB M4, TurboQuant 3-bit MLX, and is a different runtime/quant/device arrangement. No author outreach, model download, benchmark execution or external write was performed.