llamaperf
Oct 26, 2026
Throughput
159.7 t/s gen
Quant
4-bit (MLX)
System RAM
128 GB

Summary

User reports Nemotron-3 Nano 30B-A3B at 159.7 t/s decode on an M4 Max with 128 GB, batch 1 and greedy, measured with mlx-lm 0.31.3 on mlx 0.32.2. The model was the mlx-community 4-bit build at 17.8 GB; the figure is mlx_lm generate's own Generation rate over 256 tokens, taken as the median of three processes. The post compares mlx-lm against llama.cpp b8680 on the same machine for hybrid Mamba-2 models, but only the MLX row for Nemotron-3 Nano is given here.