llamaperf

Qwen3.5 27B

on M5 Pro 48GB · mlx-lm · 4,096 ctx

Tone: negativeApple hardware
Oct 8, 2026
Throughput
461.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
affine 4bit (MLX)
System RAM
48 GB

Summary

User reports Qwen3.5-27B affine 4bit prefilling a 4096-token prompt at 461 tok/s on an M5 Pro 48GB (20-core GPU) with stock mlx 0.32.0 and mlx_lm 0.31.3 in a clean venv. The post is a bug report against oMLX 0.5.0, whose bundled MLX build fails a NAX metallib scan, so its M5 dispatch never engages: the same model prefilled at only 180 tok/s at 4096 tokens there, and 262 tok/s with OMLX_NAX=1. No decode figure is given beyond saying decode is normal. Qwen3.5-35B-A3B 4bit prefilled at 1,798 tok/s in the clean venv against about 900 on oMLX 0.5.0.