Llama 3.1 8B

User benchmarks Llama 3.1 8B on a base M4 Mac, comparing MLX and llama.cpp. Best observed decode is MLX at 23.1 t/s on a quiet machine, with llama.cpp at 11.5 t/s; llama.cpp prefill is 118 t/s with flash-attention and 71 t/s without. Setup uses MLX 4-bit and llama.cpp GGUF Q4_K_M, with a 32K KV cache pre-allocated by llama.cpp. The user notes the memory gap between engines is an artifact of that pre-allocation, with a real weight delta of about 0.3 GB. The user reports decode is bandwidth-bound at roughly 18-22 t/s either way, and that the speed winner flips depending on background load. The MLX checkpoint is lighter at 4.50 bpw versus 4.90 bpw for llama.cpp.