Oct 12, 2026
Summary
User reports Qwen3.5-122B-A10B at 15.6 t/s generation on an M4 Max 128GB.
Setup is llama.cpp with Q4_K_M and f16 K/V cache, all layers offloaded, batch 1024 and flash attention on.
The figure is the 128K benchmark point after cache prefilling, not newly processed input tokens; the source's f16 KV series is 24.51 t/s at 64K, 22.25 at 81K and 15.59 at 128K, with a reported Metal/VRAM peak of 75.16 GB at 128K.