SmolLM 135M-Instruct / mlxcel native512-128 / single pass

Oct 13, 2026

Throughput

370.7 t/s gen · 12605.1 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

4-bit (MLX)

MTP (Multi-Token Prediction)

off

System RAM

128 GB

Summary

User reports SmolLM-135M-Instruct-4bit at 370.7 t/s generation and 12605.1 t/s prompt processing on an M1 Ultra 128GB. Setup is mlxcel 0.7.0-beta.1 with 4-bit weights and FP16 KV cache, one Metal GPU, 512 input tokens and 128 output tokens, greedy, no speculative decoding. Prefill excludes loading, tokenization and client delivery, so it is not end-to-end TTFT; the decode rate covers 128 output tokens over 127 forward steps.