Qwen3.8 27B
M5 Pro 24GB · JustFit
- generation:
- 61.8 tokens/s
- quant:
- 4bit (MLX)
User reports Qwen3.8-27B at 61.8 tok/s on an M5 Pro 24GB Mac, up from 57.3 tok/s with draft-tree speculative decoding off. Setup is the native Rust/Metal JustFit 2.6 engine with stock MLX 4-bit weights and Inco AI's DFlash2 draft, on a 24 GB Apple Silicon machine. The 7-prompt greedy suite also gave 40.9 tok/s on M6 (35.6 chain) and 50.9-51.3 tok/s on M4 Pro with a fixed 8-row tree. On NVIDIA SPEED-Bench at 16K prompts the M5 Pro median was 59.5 tok/s, M6 38.5 tok/s and M4 Pro 48.3 tok/s. Dual 128K+32K requests peaked at 21,563 MiB on M5 Pro.