Qwen3.8 27B
on M5 Max 128GB · MLTF · 2,048 ctx
Summary
User reports Qwen3.8-27B at 70+ tok/s generation on an M5 Max 128GB MacBook Pro.
Setup is the MLTF engine with 8-bit weights and an INT8 KV cache, using Metal TensorOps W8A8 kernels.
Cold prefill is 1,000+ tok/s on a 2Ki prompt, and aggregate generation throughput is 200+ tok/s at concurrency 4. The user chose 8-bit over 4-bit for quality.