Qwen3.8 27B

Source: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks Raw measured rows: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/3990a8e0fa5f2762911689439ce12e1c28d4c989/data/speed_2026-10-03.jsonl Quantization/hardware/method README: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/cb99870e05aa9f5783c5c6987ddd9154834123ab/README.md Loaded context metadata: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/main/data/loaded_context_2026-10-03.jsonl Original author measurements dated October 3–4, 2026, recovered and checked October 13. The raw file has 240 measured rows plus one separate error record. This report includes only the 15 runs for the exact model alias below. No local reproduction was performed. Headline: 11.8 tokens/s is the median of three English-prose runs (Ruhr economy prompt). It is client-observed streaming decode throughput. It is not an overall mean, pooled throughput, peak speed, native kernel benchmark or prefill throughput. Other workloads are kept separate. Hardware: Apple Mac mini M6, Mac18,5, 12 CPU cores (2 Super + 4 Performance + 6 Efficiency), 12 GPU cores, 32 GB unified memory, macOS 27.0.1. One computer; one model loaded alone. Models were stored on an ORICO 2 TB external NVMe with an ASMedia ASM246x enclosure (source reports 40 Gb/s, PCIe 4.0 x4). Fan control was Apple automatic. Per-run temperature, clock, power, swap, peak resident RAM and exact GPU offload are not reported. Measured model alias: qwen/qwen3.8-27b. Source quantization: GGUF Q4_K_M. Runtime: LM Studio using its llama.cpp GGUF backend; exact application and backend versions/builds are not recorded. The model name/parameter class is source-reported, not an independently counted tensor total. The benchmark identifies this model alias and quantization but not its GGUF filename, Hugging Face publisher/repository, revision or hash. No model link is guessed from the base model or from a similarly named quant. Loaded context is verified as 32,768 tokens for this exact alias in loaded_context_2026-10-03.jsonl. This is allocated/loaded capacity, not a filled prompt length. Actual prompt token counts are not retained. Native prompt-processing rate and prefill duration are unknown and the PP field must stay blank. Method: one stream, max_tokens 600, temperature 0.7, reasoning_effort="none", three repetitions for each of five prompts. The script first sends a short Hallo warm-up request, then uses the OpenAI-compatible chat-completions stream. Thinking tokens count if a model emits them anyway. Requested reasoning-off is not independent proof that all model reasoning was disabled. MTP/speculative settings for these ordinary GGUF rows, KV type, batch/microbatch sizes and thread counts are not recorded. The harness computes decode_tps = (tokens - 1) / (stream_end - first_nonempty_content_time). tokens uses API completion_tokens when present, with a nonempty-stream-chunk-count fallback. The saved record includes both tokens and chunks, but not the original usage object, total elapsed time or finish_reason. Values below are the saved, rounded client rates; token counts are preserved as recorded. They are not re-derived from TTFT. TTFT is client wall latency, not native prefill time. Prompts repeat between the three runs; there is no nonce or explicit cache clearing in this script. Prefix-cache state and cached-token counts are unknown, so do not call these cold-prefill tests. Runs at 600 recorded output tokens reached the requested ceiling numerically; finish_reason was not saved. All 15 runs follow. Each run is shown as decode tokens/s; recorded output tokens; nonempty stream chunks; client TTFT seconds. Repetition order is 0, 1, 2. German prose, Rhine geology (wissen_de), median 11.2 tokens/s: Run 0: 11.2 tokens/s; 546 output tokens; 545 chunks; TTFT 0.79 s. Run 1: 11.7 tokens/s; 600 output tokens; 600 chunks; TTFT 0.26 s. Run 2: 11 tokens/s; 543 output tokens; 542 chunks; TTFT 0.29 s. German prose, bead-based immunoassay (beads_de), median 11.4 tokens/s: Run 0: 11.4 tokens/s; 600 output tokens; 600 chunks; TTFT 0.7 s. Run 1: 11.4 tokens/s; 590 output tokens; 589 chunks; TTFT 0.27 s. Run 2: 10.6 tokens/s; 549 output tokens; 548 chunks; TTFT 0.24 s. English prose, Ruhr economy (prosa_en), median 11.8 tokens/s: Run 0: 11.8 tokens/s; 406 output tokens; 405 chunks; TTFT 0.69 s. Run 1: 11.6 tokens/s; 413 output tokens; 412 chunks; TTFT 0.25 s. Run 2: 12 tokens/s; 448 output tokens; 447 chunks; TTFT 0.25 s. Python ISO-8601 duration parser (code_py), median 17.5 tokens/s: Run 0: 17.7 tokens/s; 580 output tokens; 579 chunks; TTFT 0.73 s. Run 1: 17 tokens/s; 443 output tokens; 442 chunks; TTFT 0.31 s. Run 2: 17.5 tokens/s; 542 output tokens; 541 chunks; TTFT 0.31 s. JSON antigen/disease extraction (json_extr), median 19.3 tokens/s: Run 0: 19.3 tokens/s; 87 output tokens; 86 chunks; TTFT 0.82 s. Run 1: 19.2 tokens/s; 87 output tokens; 86 chunks; TTFT 0.25 s. Run 2: 19.3 tokens/s; 87 output tokens; 86 chunks; TTFT 0.2 s. Quality scope: these tables measure speed. The author reports the JSON task passed all three runs for this model, with automated checking; prose/code quality was not generally graded. Do not infer broader task accuracy from throughput.