Grug 12B VLM MLX 4-bit, forced mlx-lm pp8192/tg128

Oct 11, 2026

Throughput

50.9 t/s gen · 511.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

MLX affine 4-bit, group 64 (MLX)

System RAM

64 GB

Summary

User reports Grug-12B at 50.9 t/s generation and 511.9 t/s prompt processing on an M4 Max with 64 GB unified memory. Setup is oMLX with MLX affine 4-bit, group 64, at a pp8192/tg128 workload, with 8.26 GB peak memory. The same row lists TTFT 16003.6 ms, TPOT 19.80 ms and E2E 18.540 s; the 8192-token prompt is a workload label, not a configured maximum context.