gpt-oss 21B (3.6B active)
on M4 Pro 48GB · Rapid-MLX
Summary
User reports gpt-oss 21B-A3.6B at 77.2 t/s on an M4 Pro 48GB.
Setup is Rapid-MLX 0.15.4 with MXFP4-Q8, single-stream generation, median of three runs at temperature 0 over a 512-token decode workload.
Four concurrent streams deliver 126.7 t/s aggregate, and the reported 15.7 GB Metal peak covers the full sweep, including up to approximately 16K prompt and four streams with prefix caching.