Qwen3 4B
on M1 Ultra 128GB · mlxcel
Summary
User reports Qwen3 4B 4-bit at 116.9 t/s generation and 942.3 t/s prompt processing on an M1 Ultra 128GB Mac Studio.
Setup is mlxcel 0.7.0-beta.1 with FP16 KV cache, one request of 512 input tokens and 128 output tokens, greedy temperature 0, no speculative decoding.
The decode figure covers 128 output tokens over 127 forward steps; prefill excludes loading, tokenization and transport, so it is not end-to-end TTFT.