Gemma 4 31B

on M2 Max 64GB · mlx-vlm 0.4.4 / MLX 0.31.1 / Metal

Oct 14, 2026

Throughput

17.2 t/s gen · 63.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

4bit (MLX)

System RAM

64 GB

Summary

User reports Gemma 4 31B at 17.2 t/s generation and 63.7 t/s prompt processing on a Mac Studio M2 Max with 64 GB unified memory. Setup is mlx-vlm 0.4.4 with 4-bit affine group-64 weights and MLX 0.31.1 on one GPU, running the text-only long-text request with 55 prompt tokens and 500 generated tokens. The 29.9-second whole generate call is not TTFT, and the 63.7 t/s prompt figure is a single request rather than a llama-bench pairing.