Qwen2.5 72B

User compares GGUF and MLX inference speeds on a Mac Studio M2 Ultra with 192GB, using LM Studio 0.3.10 with MLX 0.6. For Qwen2.5 72B 8-bit, GGUF reaches 7.68 tokens/sec while MLX drops to 0.42 tokens/sec. Smaller models show the opposite: Qwen2.5 0.5B 4-bit runs at 79 tokens/sec in GGUF versus 317 tokens/sec in MLX, and Mistral Small 22B 8-bit is roughly equal at 25.01 versus 25.96 tokens/sec. User reports MLX is 4-5x faster for very small models but about 10x slower for large ones, with the inflection point around 22B parameters, and notes MLX does not use much GPU for larger models while LM Studio lacks a layer-offload option for MLX.