Grug 35B-A3B MLX mixed 4-bit, single-request pp1024/tg128

Oct 11, 2026

Throughput

103.9 t/s gen · 1807.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

MLX affine 4-bit g64; gate modules 8-bit (MLX)

System RAM

64 GB

Summary

User reports Grug-35B-A3B at 103.9 t/s generation and 1807.2 t/s prompt processing on an M4 Max 64GB. Setup is oMLX with MLX affine 4-bit group 64, gate modules at 8-bit, at 1024 prompt and 128 generated tokens. The run is single-request, with 566.6 ms TTFT, 9.70 ms TPOT, 1.805 s end-to-end and 19.35 GB peak memory; warm-up, repetition count and KV precision are not stated.