llamaperf

gpt-oss 21B (3.6B active)

on M4 Pro 48GB · Rapid-MLX

Apple hardware
Oct 11, 2026
Throughput
77.2 t/s gen
Quant
MXFP4-Q8 (MLX)
VRAM reported
48 GB

Summary

User reports gpt-oss 21B-A3.6B at 77.2 t/s on an M4 Pro 48GB. Setup is Rapid-MLX 0.15.4 with MXFP4-Q8, single-stream generation, median of three runs at temperature 0 over a 512-token decode workload. Four concurrent streams deliver 126.7 t/s aggregate, and the reported 15.7 GB Metal peak covers the full sweep, including up to approximately 16K prompt and four streams with prefix caching.