Nanbeige4.3 33B
M5 Max 128GB · mlx-lm · 512 ctx
- generation:
- 28.8 tokens/s
- prompt processing (prefill):
- 757.0 tokens/s
- quant:
- 4-bit (MLX)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Nanbeige4.3-33B (dense, 4-bit) on a MacBook Pro M5 Max with 128 GB, comparing mlx-lm, llama.cpp and Kapitsa at prompt lengths from 512 to 32K tokens. At 512 tokens mlx-lm prefills at 757 t/s and decodes at 28.8 t/s, llama.cpp at 691 t/s and 27.3 t/s, and Kapitsa at 693 t/s and 34.9 t/s. mlx-lm and Kapitsa read the same 4-bit MLX AWQ pack (group 64); llama.cpp reads Q4_K_M. Kapitsa's decode figures use KFlash, its tree-based speculative decoding with a small drafter, while mlx-lm and llama.cpp run plain decoding. On a 512-token Python coding task Kapitsa reaches 36.6 tok/s greedy and 32.3 tok/s sampled, against 27.1 and 26.7 for mlx-lm and 24.9 and 24.8 for llama.cpp. At 29.5K context mlx-lm has the fastest cold TTFT at 59.4 s, Kapitsa the fastest follow-up TTFT at 300 ms and the lowest peak memory at 28.1 GB. The author works on Kapitsa and notes the KV cache is large, 26 GB for 128K tokens.