llamaperf
Oct 13, 2026
Throughput
29.7 t/s gen
Quant
Q4_K_M (GGUF)
System RAM
16 GB

Summary

Source-reported measurement by Mustafa.net, measured 2026-10-12 and published 2026-10-13. Primary source: https://mustafa.net/2026/10/13/local-llm-calculator-tested-four-machines/ . This report covers Qwen3.5 4B Q4_K_M on ONE Mac mini M4 with 16 GB installed unified memory. Source says the official Mac llama.cpp build ran directly on Apple graphics; exact build, macOS version and CPU/GPU core counts are unstated. Installed unified memory is not dedicated VRAM. Model and quantization are source-backed. Verified upstream reference: https://huggingface.co/Qwen/Qwen3.5-4B . This is the upstream model, NOT the tested GGUF download. Quantizer/publisher, GGUF filename, tested revision, checksum and conversion details are unknown; do not substitute a guessed Unsloth or other GGUF link. Generation headline is 29.7 tokens/s, the median of three requests after a short warm-up. Source reproduction extracts timings.predicted_per_second from llama.cpp /completion, so this is the server's generation-phase rate, not client wall throughput, TTFT, prefill or word-count estimation. Configured context is 8,192 tokens; actual prompt-token count is absent. The fixed reproduction prompt asks what a network switch does. Temperature is 0 and n_predict is 256. The prose calls these 256 generated tokens, but no raw per-request counters are published, so the command independently establishes the requested cap, not verified realized output count. Source reports roughly 1% agreement among repeats without raw samples or formal spread statistics. No prompt-processing rate, TTFT or end-to-end latency is supplied. KV precision, Flash Attention, speculative decoding/MTP, effective prefix-cache reuse, batch/microbatch, thread settings, power mode, thermals, background load and GPU placement count are unknown. The source explicitly omits Mac measured memory because its reading was unreliable; leave measured RAM/VRAM blank. This is a short warm-run source claim, not a long-context, sustained-load or quality test.