- generation:
- 12.7 tokens/s
- prompt processing (prefill):
- 116.0 tokens/s
- quant:
- 4-bit (MLX)
- mtp (multi-token prediction):
- off
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
Source-reported historical native MLX campaign by karlhendrik. Measurement section is dated 2026-09-29, 22:20–23:40 Tallinn; recovered 2026-10-13, not locally rerun. Pinned primary results: https://gist.github.com/karlhendrik/355bb6cbf98861b21177a6b7857e79f7/7ff331f5e10c68699eac04848ca3840708f41061#file-results-round2-md . Gist revision 7ff331f5e10c68699eac04848ca3840708f41061, last updated 2026-09-30T10:05:52Z. Exact invocation script: https://gist.github.com/karlhendrik/355bb6cbf98861b21177a6b7857e79f7/7ff331f5e10c68699eac04848ca3840708f41061#file-mlx_bench-sh . Hardware: ONE Apple Mac mini M4, host M4a, with 16 GB installed unified memory. The comparison section and script explicitly run MLX on one mini; the surrounding two-mini RPC cluster is a different experiment and must not become this report's GPU count. Source cluster configuration raises the GPU wired limit to 13,312 MB per mini, but no independent MLX effective-limit readback is published. CPU/GPU core counts, macOS version, power mode, clocks, thermal state, swap activity and background load are not recorded. Unified RAM is not dedicated VRAM. Model: mlx-community/Qwen3-14B-4bit, MLX safetensors, 4-bit. Exact source-linked artifact repository: https://huggingface.co/mlx-community/Qwen3-14B-4bit . Current metadata/config verification at https://huggingface.co/mlx-community/Qwen3-14B-4bit/blob/a4d9b2df59d2c150bef02fcbe0d91046b7ca33a4/config.json identifies Qwen3ForCausalLM and 4-bit/group-size 64 quantization; metadata revision observed at recovery is a4d9b2df59d2c150bef02fcbe0d91046b7ca33a4. This is an independently observed current revision, NOT proof of the snapshot tested on September 29. Tested HF revision, weight-file hashes and tokenizer revision are unknown. The model-card conversion-tool version is not the runtime version. The 14B label is the source's nominal model name; exact parameter/active-parameter totals are not supplied as measured metadata. Runtime: source states mlx-lm 0.29.1, using native mlx_lm generate on Apple Silicon. Exact command selects the HF ID above, --temp 0 and --max-tokens 128 for each of two short invocations. Short prose asks for an explanation of mixture-of-experts versus dense models and includes /no_think. A separate long invocation uses --max-tokens 32 and a deterministic seed 7 filler of 3,400 space-separated technical words, followed by a one-sentence summarization instruction. Long input is reported as 3,796 tokens; this is actual source-reported prompt count, not context capacity. No speculative draft-model flag or reusable prompt-cache file is supplied. The script uses serial fresh CLI processes, not HTTP, RPC or concurrent users. No explicit warmup/discard policy is recorded. Effective reasoning behavior was not separately validated; /no_think is an instruction, not proof of output semantics. All performance points: the two SHORT generation rates are 12.7 and 12.7 tokens/s. The headline selects the first reported value, 12.7, without inventing a mean or extra precision. The separate LONG prompt-processing rate is 116 tokens/s at 3,796 reported input tokens. The long request's generation rate is not published; short prompt-token count is not published. Source heading calls the short test 128 tokens and the command caps it at 128, but raw per-run token counters are absent, so actual generated counts are not independently established. Long output cap 32 is not an observed output count. These PP and TG fields come from DIFFERENT requests, not a3,796-input/128-output completion. Configured context/KV capacity is unknown and left blank. Native timing audit: official PyPI mlx-lm 0.29.1 source distribution https://pypi.org/project/mlx-lm/0.29.1/ , SHA256 b99180d8f33d33a077b814e550bfb2d8a59ae003d668fd1f4b3fff62a381d34b, mlx_lm/generate.py stream_generate/generate. The script greps its Prompt, Generation and Peak memory output. PP uses encoded prompt.size divided by elapsed time until the first generated token is yielded, then resets the timer; it is native CLI prompt-phase throughput with that first-token boundary, not HTTP TTFT or isolated kernel throughput. TG divides n+1 yielded-token iterations by elapsed time after that reset; the counter includes the first token even though its production preceded the TG timer, and can include terminal EOS. Detokenization/output-loop overhead can contribute. Preserve the native reported convention rather than silently correcting it. The package was only read, not installed or executed. Source reports peak memory 8.4GB. The native printer uses mx.get_peak_memory()/1e9, i.e. decimal GB from the MLX allocator, not installed RAM, model-file size, dedicated VRAM or whole-system RSS. The table does not identify which short/long invocation supplied that peak, and no per-run memory trace exists; retain it in Notes and leave structured measured-VRAM fields blank. Unrecorded: actual short input/output counters, actual long output, cache precision/dtype and effective cache capacity, Flash Attention selection, underlying MLX-library version, OS/build provenance, physical power, per-run durations, raw stdout, and answer correctness. No quality score or broad model recommendation follows from this timing test. Current model config does not prove run-time settings. No HTTP wall time or configured capacity has been relabeled native PP or actual input. Dedup: all 97 live approved M4-family rows and all-status exact gist search were read at recovery. Existing gist-linked report 1292 is the 35B RPC campaign. Existing Qwen3-8B M4-16GB report 1927 is another author's persistent-agent/ANE experiment (~10 tok/s), not this native CLI campaign. These two MLX model campaigns remain distinct from each other and from the source's GGUF arms. No author outreach, model download or local benchmark run was performed.