MLX benchmarks
Apple's machine-learning framework for Apple Silicon, with its own LLM runtime, mlx-lm.
79 community reports
MLX results by GPU
Hardware with reported MLX results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup.
MLX on M5 Max 48GB27 reports
- Phi-4 · 4-bit190.8 t/s, 6,281 t/s prompt
- Phi-4 · 4-bit62.1 t/s, 1,686 t/s prompt
- Gemma 3 4B · 4-bit176.2 t/s, 5,550 t/s prompt
- Gemma 3 27B · 4-bit31.8 t/s, 854 t/s prompt
- Gemma 3 1B · 4-bit374.4 t/s, 17,262 t/s prompt
MLX on M3 Ultra 512GB15 reports
- Llama 3.1 · 4-bit14.9 t/s
- Qwen2.5 72B · 4-bit14.3 t/s
- Llama 3.3 70B · Q86.8 t/s
- Gemma 4 31B · BF169.2 t/s, 296 t/s prompt
- Qwen3.8 27B · Q431.8 t/s, 391 t/s prompt
MLX on M2 Max 64GB14 reports
- Nemotron 3 · 4-bit (source config)93.0 t/s, 469 t/s prompt
- Qwen 2.5 Coder 32B · 4-bit (source config)18.0 t/s, 127 t/s prompt
- Qwen3.5 35B · 3B active · 4-bit (source config)88.9 t/s, 522 t/s prompt
- Qwen3.5 27B · 4-bit (source config)20.7 t/s, 91 t/s prompt
- GLM-4.7 · 4-bit (source config)52.6 t/s, 414 t/s prompt
MLX on M5 Ultra 256GB7 reports
- GLM-5.3 320B · 18B active · 4-bit36.8 t/s(2 cards, speculative)
- Gemma 4 31B · BF1613.9 t/s, 1,237 t/s prompt
- Qwen3.6 35B · 3B active · Q4130.4 t/s, 7,155 t/s prompt
- Qwen3.5 122B · 10B active · Q474.7 t/s, 2,711 t/s prompt
- Qwen3 14B · Q467.0 t/s, 2,530 t/s prompt
MLX on M5 Max 128GB6 reports
- Nemotron-Labs Dory 1.0 4B · 8-bit101.0 t/s
- Qwen3 30B · 3.3B active · MLX 4-bit affine, group 64144.4 t/s
- Qwen3 30B · 3B active130.6 t/s, 5,402 t/s prompt
- Nex-N2.5-mini 35B · 3B active · MLX-4bit133.6 t/s
- Qwen3.6 27B · Q817.2 t/s, 706 t/s prompt
MLX on M4 16GB4 reports
- Qwen3 14B · 4-bit12.7 t/s, 116 t/s prompt
- Qwen3 8B · 4-bit23.2 t/s, 219 t/s prompt
- Mellum2.1 12B · 2.5B active · NVFP464.0 t/s
- Qwen3 8B · 4bit10.0 t/s
MLX on M3 Ultra 96GB3 reports
- gpt-oss 117B · 5.1B active · MXFP473.0 t/s, 347 t/s prompt
- Qwen3-Next 80B · 3B active · 4-bit49.1 t/s
- Qwen3.5 35B · 3B active · 4-bit109.7 t/s, 1,770 t/s prompt
MLX on M4 Pro 48GB2 reports
- Qwen3.6 27B · 4-bit18.3 t/s(speculative)
- Qwen3.8 27B20.3 t/s(speculative)
MLX on M5 32GB2 reports
- Qwen3 30B · 3B active · 4-bit57.0 t/s, 1,063 t/s prompt
- Gemma 43,029 t/s prompt
MLX on M1 Max 32GB1 report
- Qwen3.8 27B · 4bit15.8 t/s, 82 t/s prompt
MLX on M2 Pro 32GB1 report
- Qwen3.6 35B · 3B active · 4-bit group-6461.7 t/s, 277 t/s prompt
MLX on M4 Pro 64GB1 report
- Gemma 4 · MLX affine mixed 3.40 bpw78.3 t/s, 1,236 t/s prompt
MLX on M5 Max 64GB1 report
- Qwen3.6 27B · 4-bit63.0 t/s(speculative)
MLX on M5 Pro 24GB1 report
- DeepSeek-Coder-V2-Lite 16B · 2.4B active · 4bit125.8 t/s, 364 t/s prompt
MLX on M6 32GB1 report
- Qwen3.5 4B · affine Q4, group size 6416.2 t/s, 439 t/s prompt
MLX, GPU not identified12 reports
- gpt-oss 20B · MXFP4119.0 t/s
- Muse Spark Open 816B · 42B active · NVFP436.7 t/s(4 cards)
- DeepSeek V4 Flash 284B · 13B active · mxfp4542 t/s prompt(2 cards)
- Llama 3.1 8B · 4bit23.1 t/s
- Qwen3.8 27B · 4bit33.4 t/s(speculative)
MLX is Apple's open-source array framework, designed around the unified memory of Apple Silicon. Its mlx-lm package loads, runs and serves LLMs on a Mac.
Community conversions in MLX's own 4-bit and 8-bit formats cover most popular models.
MLX is built for Apple hardware first. Fewer models are converted for it than for GGUF, so llama.cpp's Metal backend is the usual alternative on a Mac.
Compare GPUs running MLX
Every card is compared on one model, Qwen3 14B at 4-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.
| GPU | VRAM | Reports | Median speed, Qwen3 14B 4-bit |
|---|---|---|---|
| M5 Max 48GBapple | 48GB | 27 | Gen 62.6 t/sPP 1738.6 t/s · n=1 |
| M3 Ultra 512GBapple | 512GB | 15 | no Qwen3 14B 4-bit run reported |
| M2 Max 64GBapple | 64GB | 14 | no Qwen3 14B 4-bit run reported |
| M5 Max 128GBapple | 128GB | 7 | no Qwen3 14B 4-bit run reported |
| M5 Ultra 256GBapple | 256GB | 7 | no Qwen3 14B 4-bit run reported |
| M4 16GBapple | 16GB | 4 | Gen 12.7 t/sPP 116.0 t/s · n=1 |
| M3 Ultra 96GBapple | 96GB | 3 | no Qwen3 14B 4-bit run reported |
| M4 Pro 48GBapple | 48GB | 2 | no Qwen3 14B 4-bit run reported |
MLX against other engines
Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.
| Model and GPU | MLX | Other engine |
|---|---|---|
| Gemma 3 12B 4-bitM5 Max 48GB | 66.9 t/sPrompt processing (prefill) 1834.9 tokens/s (1)one generation report, context not stated | llama.cpp61.5 t/sPrompt processing (prefill) 1721.5 tokens/s (1)one generation report, context not stated |
| Gemma 3 4B 4-bitM5 Max 48GB | 176.2 t/sPrompt processing (prefill) 5550.3 tokens/s (1)one generation report, context not stated | llama.cpp150.5 t/sPrompt processing (prefill) 5760.7 tokens/s (1)one generation report, context not stated |
| Qwen 2.5 Coder 14B 4-bitM5 Max 48GB | 60.8 t/sPrompt processing (prefill) 1719.3 tokens/s (1)one generation report, context not stated | llama.cpp56.0 t/sPrompt processing (prefill) 1546.0 tokens/s (1)one generation report, context not stated |
| Qwen 2.5 Coder 32B 4-bitM5 Max 48GB | 28.4 t/sPrompt processing (prefill) 734.4 tokens/s (1)one generation report, context not stated | llama.cpp26.3 t/sPrompt processing (prefill) 685.0 tokens/s (1)one generation report, context not stated |
| Qwen 2.5 Coder 7B 4-bitM5 Max 48GB | 121.8 t/sPrompt processing (prefill) 3362.9 tokens/s (1)one generation report, context not stated | llama.cpp106.0 t/sPrompt processing (prefill) 2901.3 tokens/s (1)one generation report, context not stated |
| Qwen3 0.6B 4-bitM5 Max 48GB | 570.6 t/sPrompt processing (prefill) 22693.8 tokens/s (1)one generation report, context not stated | llama.cpp457.1 t/sPrompt processing (prefill) 23288.5 tokens/s (1)one generation report, context not stated |
| Qwen3 1.7B 4-bitM5 Max 48GB | 366.0 t/sPrompt processing (prefill) 12362.2 tokens/s (1)one generation report, context not stated | llama.cpp288.6 t/sPrompt processing (prefill) 11493.0 tokens/s (1)one generation report, context not stated |
| Qwen3 14B 4-bitM5 Max 48GB | 62.6 t/sPrompt processing (prefill) 1738.6 tokens/s (1)one generation report, context not stated | llama.cpp57.4 t/sPrompt processing (prefill) 1584.3 tokens/s (1)one generation report, context not stated |
| Qwen3 32B 4-bitM5 Max 48GB | 28.4 t/sPrompt processing (prefill) 730.9 tokens/s (1)one generation report, context not stated | llama.cpp26.4 t/sPrompt processing (prefill) 667.0 tokens/s (1)one generation report, context not stated |
| Qwen3 4B 4-bitM5 Max 48GB | 186.0 t/sPrompt processing (prefill) 5505.7 tokens/s (1)one generation report, context not stated | llama.cpp159.7 t/sPrompt processing (prefill) 4972.7 tokens/s (1)one generation report, context not stated |
| Qwen3 8B 4-bitM5 Max 48GB | 110.6 t/sPrompt processing (prefill) 3150.6 tokens/s (1)one generation report, context not stated | llama.cpp97.7 t/sPrompt processing (prefill) 2744.7 tokens/s (1)one generation report, context not stated |
| Qwen3.6 35B · 3B active 4-bitM5 Max 48GB | 151.1 t/sPrompt processing (prefill) 3333.4 tokens/s (1)one generation report, context not stated | llama.cpp124.3 t/sPrompt processing (prefill) 3342.7 tokens/s (1)one generation report, context not stated |
Top models on MLX
Frequently asked
Is MLX faster than other engines?
llamaperf has 12 matched pairs of MLX and other-engine runs on the same card, model size and bit class, but none is comparable enough to say which engine is faster: the sides have too few reports or were run at different context lengths. A pair counts toward a direction only when each side has at least 3 generation reports with a stated context, and the two sides' contexts overlap. The measured figures and their conditions are in the table on this page.
Can I run MLX on Windows or Linux?
MLX is built for Apple Silicon. It also has a CUDA backend for Linux, but the community's MLX model conversions and tools are made for Macs.
What quants does MLX use?
MLX has its own quantization format. Community 4-bit conversions are the most common, and 8-bit versions are widely available for higher quality.