llamaperf
Oct 8, 2026
Throughput
33.4 t/s gen
Quant
4bit (MLX)

Summary

User reports Qwen3.8 27B abliterated MLX 4-bit at 33.374 tok/s median C1 decode with DFlash 2 block 3 speculative decoding on an Apple M5 Max. Setup is MLX 0.32.1 with MLX-LM 0.31.3 and MLX-VLM 0.6.16, affine 4-bit group size 64, 256-token prompt and 256 fixed generation tokens, greedy decoding, one warmup and two timed trials. Autoregressive C1 median was 27.011 tok/s, giving a 1.2356x speedup from DFlash 2. Peak memory was 18.435 GB with the 4-bit DFlash draft and 16.062 GB in the quality suite. The 33.374 tok/s figure is a same-run M5 Max C1 measurement, not a guarantee for every Apple chip or thermal state.