Nemotron 3 Nano 30B (3B active)
on M4 Max 128GB · mlx-lm
Summary
User reports Nemotron-3 Nano 30B-A3B at 159.7 t/s decode on an M4 Max with 128 GB, batch 1 and greedy, measured with mlx-lm 0.31.3 on mlx 0.32.2.
The model was the mlx-community 4-bit build at 17.8 GB; the figure is mlx_lm generate's own Generation rate over 256 tokens, taken as the median of three processes.
The post compares mlx-lm against llama.cpp b8680 on the same machine for hybrid Mamba-2 models, but only the MLX row for Nemotron-3 Nano is given here.