Qwen3 8B

User reports Qwen3-8B at ~10 tok/s decode on a 16GB Mac (M4) using MLX with 4-bit quantization. Setup runs a persistent local agent with Qwen3-8B on the GPU, Qwen3-4B as 36 int8 layer shards on the Neural Engine, and Qwen2.5-0.5B for routing. The ANE toolchain is custom Rust with no Python. User notes 16GB is a hard ceiling and the mesh has only run on one machine so far.