M4 Pro 48GB

quant:

8-bit (MLX)

User ports LingBot-World-V2 1.3B causal-fast video world model to a single M4 Pro 48GB Mac via PyTorch MPS, generating 109 frames of 480x832 video (6.8 seconds at 16 fps) in 14.4 minutes with a 16.1 GB peak footprint. Setup uses an MLX 8-bit umt5-xxl text encoder (5.6 GB on disk, ~1 s per prompt) and a spatially tiled VAE decode; the text encoder, DiT and VAE are never resident simultaneously. Peak footprint stays at ~16 GB whether generating 121 or 361 frames. User notes the port is not real-time, only tested on one machine, and that 4-bit text encoder quantization was rejected due to 17-28% relative error. A torch bf16 run of 361 frames took 33.5 minutes.

reported speed:

1.8 tokens/s generation

quant:

JANG

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks MoE streaming in oMLX on an M4 Pro 48GB across four models. GLM-5.3-Flash-JANG-MTP reaches 1.82 tok/s with 12.36s TTFT, 10.52 GiB after load and 14.68 GiB peak. Qwen3.8-JANG 4S reaches 3.63 tok/s with 8.56s TTFT, 7.04 GiB after load and 11.19 GiB peak. Qwen3.8-JANG 4M reaches 3.19 tok/s with 11.23s TTFT, 7.05 GiB after load and 11.11 GiB peak. DeepSeek-V4-Flash-0731-JANG reaches 2.71 tok/s with 6.52s TTFT, 8.25 GiB after load and 16.87 GiB peak. The primary record uses GLM-5.3-Flash-JANG-MTP. The other models are Qwen3.8 in JANG 4S and 4M quants and DeepSeek V4 Flash in the 0731 variant with the JANG quant. MoE streaming allows larger MoE models to run with a lower memory footprint at the cost of speed.

reported speed:

18.0 tokens/s generation

quant:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports speculative decoding on Apple Silicon, with an 8-bit model going from 8.2 tok/s baseline to 18-26 tok/s. The speedups are 3.27x on math, 2.5x on code and 2.22x on chat, with output byte-identical. The 4-bit model gets about 1.7x at roughly 25 tok/s and needs about 18 GB, while the 8-bit model peaks at about 40 GB and needs a 48 GB Mac. Meta's DFlash numbers on Mac are 1.5x on an M4 Max and 1.8x on an M5 Max on 4-bit.

reported speed:

8.2 tokens/s generation

quant:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports mlx-dspark, a speculative decoding project, running an 8-bit model at 8.2 tok/s baseline and 18-26 tok/s with speculative decoding. Speedups are 3.27x on math, 2.5x on code and 2.22x on chat. The 4-bit model reaches about 1.7x at about 25 tok/s and needs about 18 GB. The 8-bit model peaks at about 40 GB and requires a 48 GB Mac. Output is byte-identical to normal decoding.

reported speed:

20.3 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports speculative decoding with mlx-dspark on an 8-bit target at a 2.45x mean speedup, 8.3 to 20.3 tok/s. Setup is mlx-dspark with a 4-bit target at 1.74x and 25.3 tok/s. The 8-bit target with drafter beats plain 4-bit at 14.6 tok/s.