Unknown family
on M1 Max 64GB · llama.cpp · 1,523 ctx
Oct 9, 2026
Summary
User reports IQ2_XS quantized 7B model at 0.95 t/s on an M1 Max 64GB, with 25 tokens generated in 26.30 seconds at context 1523.
Setup is llama.cpp with IQ2_XS quantization; the user also mentions running Mixtral 8x7B fine-tunes at Q5_K_M getting 8-15 t/s.
User also reports an IQ2_XS 120B model taking about 9 minutes for prompt evaluation and roughly 0.1 t/s generation, and questions whether the slow IQ quant performance on Apple Silicon is expected or a bug.