Qwen3 8B
on M1 Ultra 128GB · mlxcel
Summary
User reports Qwen3 8B 4-bit at 81.2 t/s generation and 518 t/s prompt processing on an M1 Ultra 128GB Mac Studio.
Setup is mlxcel 0.7.0-beta.1 with FP16 KV cache, one Metal GPU, 512 input tokens and 128 output tokens, greedy temperature 0.
The decode figure covers 128 output tokens over 127 forward steps; prefill excludes loading, tokenization and transport, so it is not end-to-end TTFT.