llamaperf

Qwen3.8 27B Swift

on M5 Max 128GB · Inco Splash · 65,536 ctx

Tone: positiveApple hardware
Sep 23, 2026
Throughput
87.8 t/s gen
Quant
4bit
System RAM
128 GB

Summary

User reports a Swift-Qwen3.8-27B 4bit model running on the Splash engine on an M5 Max 128GB, with decode of 79–129 tok/s. At 64k context the model reaches 87.8 tok/s versus 36.6 tok/s for the same weights in oMLX (oQ4e-mtp, mixed 4/5-bit), and the gap widens with context. The user notes Splash's /v1 defaults to greedy while Swift wants temp 1.0 / top_k 20 / top_p 0.95, and that the format cannot carry generation_config.json.