- generation:
- 190.8 tokens/s
- prompt processing (prefill):
- 6281.4 tokens/s
- quant:
- 4-bit (MLX)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
Source-reported historical campaign by emihiggins, measured 2026-09-27T20:14:47Z; recovered 2026-10-13. No new run or local reproduction. Raw source: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx/phi-4-mini-instruct-4bit_4bit_ngl99.json . Companion provenance: https://github.com/enescingoz/mac-llm-bench/pull/12 . The source states these are unedited copies of its upstream submission. Data files show public-release commit 34fee99b6df6129fb3c64255c1bfb179a54ca044 (September 27, 2026); the measurement predates discovery. One report groups this model, precision, runtime and hardware; all retained lengths are below. Hardware: one Mac Studio (source PR identifies Mac17,14), Apple M5 Max, 18 CPU cores and 40 GPU cores, 48 GB unified RAM, macOS 27.0 (PR build 26A428), AC power and High Power energy mode. Source CPU JSON labels performance_cores=6 and efficiency_cores=12; author clarifies these are 6 Super plus 12 Performance cores. Reported 614 GB/s is theoretical bandwidth. Installed unified RAM is not dedicated VRAM. The raw available_ram_gb=30.2 is a system-availability snapshot, not installed capacity or model allocation. The source records a 32768 MB iogpu wired limit, swap <=2 MB checked every 60 seconds, no system thermal/performance warning, ambient about 20.6 C, external NVMe model storage, and idle Ollama with no model loaded. Per-model temperature, clock and wall power are unknown. Model: mlx-community/phi-4-mini-instruct-4bit, source quant=4bit, MLX weights in Safetensors. Recorded fetched HF snapshot: ac1c269cb4222a4e136a3d09edad301056c1f36a; manifest: https://github.com/emihiggins/m5-max-llm-bench/blob/main/data/mac-llm-bench/m5-max_18c-40g_48gb/mlx-revisions.tsv . Identity verified individually at https://huggingface.co/mlx-community/Phi-4-mini-instruct-4bit . Current card identifies conversion from microsoft/Phi-4-mini-instruct. The snapshot is source-reported; model files were not downloaded or hashed, and current main must not be substituted for the recorded revision. Source spells the repo mlx-community/phi-4-mini-instruct-4bit; the current HF canonical URL uses mlx-community/Phi-4-mini-instruct-4bit. The canonical repository directly exposes the same recorded snapshot, verified at https://huggingface.co/mlx-community/Phi-4-mini-instruct-4bit/tree/ac1c269cb4222a4e136a3d09edad301056c1f36a; lowercase alias/redirect behavior remains unverified. Source file_size_gb=0 is a serializer placeholder for MLX repo strings, not a zero-byte model. Conversion-library versions on HF describe conversion, not the measured runtime. Runtime: source JSON mlx-lm 0.31.3; source PR additionally reports mlx 0.32.2 and Python 3.12. One model invocation per process using ./bench_mlx.sh --repo <repo> --cleanup. The author reports cleanup path bugs; this does not establish a deleted model cache or cold load. Source's generic n_gpu_layers=99, flash_attention=true, threads=0 fields are hard-coded by the MLX serializer and do not verify actual MLX offload layers, Flash Attention setting or thread count. These feature fields stay unknown rather than borrowing llama.cpp flags. Configured context capacity is not recorded; prompt length is not context-window capacity. All retained source speed values, tokens/s: pp128=3728.47; pp256=5040.15; pp512=6281.44; tg128=190.81; tg256=192.31. Headline uses pp512=6281.44 and tg128=190.81, not a best-of sweep or an average across prompt lengths. Linkage is inferred from the published wrapper's overwrite order, not independently retained per-trial logs. The current wrapper runs (prompt,generation)=(128,128),(128,256),(256,128),(256,256),(512,128), skipping (512,256); it overwrites each pp and tg key on every success. Therefore if all documented calls completed, the retained pp128 is from (128,256), pp256 and tg256 from (256,256), and pp512 and tg128 from (512,128). The exported dictionary loses the other paired rates, trial-level variance and failure details; do not reconstruct them. The historical executable/harness commit is unpinned. The source claims documented commands without deviations; current upstream code establishes method and interpretation, not byte-for-byte historical reproduction. Timing method: mlx-lm v0.31.3 benchmark.py uses seeded random token IDs, batch size 1 by default, disables EOS stopping, performs one warmup then defaults to five measured trials for each configuration, and prints arithmetic means of native response rates. The wrapper defaults to five trials, parses those means and rounds speed to two decimals. The benchmark is synthetic text-token throughput, not conversational quality or end-to-end request throughput. Published versioned stream_generate measures prompt.size divided by elapsed time to its first generated token, then resets the timer and reports (n+1)/elapsed for generation. Thus PP is a native engine phase rate with first-token work included; it is not input tokens divided by client/network TTFT, load time or total request wall time. TG follows the library's own numerator/timer convention, not a newly recomputed rate. Each generate call makes a new prompt cache when none is supplied, while the model is already loaded and warmed. The source logs do not independently validate cache state; avoid a claim of proven cold prefill. No chat template, --no-think, speculative draft, MTP or quantized KV option is passed by this published benchmark. Default greedy sampler, unquantized KV and prefill step 2048 are code-derived defaults, not separately recorded run flags; corresponding form fields remain unknown. Memory: source memory.peak_rss_gb=2.97. Despite that field name, the MLX exporter passes the highest native peak_memory mean among configurations. v0.31.3 stream_generate uses mx.get_peak_memory()/1e9, an MLX allocator high-water measurement in decimal GB, not operating-system process RSS, installed RAM, dedicated VRAM or necessarily a reset per-trial peak. No RSS/model-size claim is made. Method sources: https://github.com/enescingoz/mac-llm-bench/blob/main/lib/run_bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/bench_mlx.sh ; https://github.com/enescingoz/mac-llm-bench/blob/main/lib/collect_results.sh ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/benchmark.py ; https://github.com/ml-explore/mlx-lm/blob/v0.31.3/mlx_lm/generate.py . No HumanEval+ or other model-quality score is included for this campaign.