Qwen3.8 27B
on M5 Pro 64GB · TensorFold
Summary
User compares six local runtimes on an Apple M5 Pro with 64 GB of unified memory, all running Qwen3.8-27B with a short lighthouse-keeper prompt of about 44 input tokens and 400 output tokens. TensorFold app beta.8 with engine 1.1.0 and an MLX 4-bit model plus DFlash2 drafter was fastest at a mean 36.0 tok/s (36.2 and 35.7 across two runs).
Other means were LM Studio 0.4.26+4 at 33.5 tok/s, MLX-Serve 26.10.2 at 30.1 tok/s, Ollama 0.40.6 at 27.0 tok/s, Magnitude 0.2.9 at 21.0 tok/s and oMLX 0.7.0 at 16.0 tok/s.
User notes the comparison is not apples-to-apples since model packaging, quantization, speculative decoding and runtime defaults differ, and that TensorFold's hardware qualification does not list the M5 Pro.