oMLX
An inference engine for running open-weight LLMs locally.
23 community reports
This engine doesn't yet have an editorial profile on llamaperf. The community reports below show how it's been used in practice across different hardware.
Top GPUs running oMLX
| GPU | VRAM | Reports | Median t/s, Qwen3.8 27B 4-bit |
|---|---|---|---|
| M5 Ultra 256GBapple | 256GB | 7 | 50.0 |
| M4 Max 128GBapple | 128GB | 6 | no plain run of this model |
| M3 Ultra 512GBapple | 512GB | 4 | no plain run of this model |
| M4 Pro 48GBapple | 48GB | 2 | no plain run of this model |
| M1 Ultra 64GBapple | 64GB | 1 | no plain run of this model |
| M2 Max 96GBapple | 96GB | 1 | no plain run of this model |
| M3 Max 128GBapple | 128GB | 1 | no plain run of this model |
| M3 Max 48GBapple | 48GB | 1 | no plain run of this model |
oMLX against other engines
Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.
No matched pair yet. No card has plain runs of one model size at one bit class on oMLX and on another engine, so llamaperf can't say how it compares on speed. Add a run.
oMLX results by GPU
Every card people have run oMLX on, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup.
oMLX on M5 Ultra 256GB6 reports
- GLM-5.3 320B · 18B active · oQ476.7 t/s, 2,270 t/s prompt(speculative)
- GLM-5.3 320B · 18B active · oQ4e68.8 t/s(speculative)
- DeepSeek V4.1 Flash 552B · 16B active · Q3-G128-LSQ73.4 t/s, 1,010 t/s prompt
- Qwen3.8 125B · 6B active · oQ8e29.0 t/s
- Qwen3.8 125B · 6B active · oQ4e108.0 t/s, 2,887 t/s prompt(speculative)
oMLX on M4 Max 128GB4 reports
- Qwen3.8 125B · 6B active · oQ4e57.4 t/s, 586 t/s prompt
- Qwen3.8 27B · oQ4e53.3 t/s, 274 t/s prompt(speculative)
- Qwen3.8 125B · 6B active · oQ5e53.9 t/s, 679 t/s prompt(speculative)
- Qwen3.6 35B · 3B active72.5 t/s, 1,472 t/s prompt
oMLX on M3 Ultra 512GB2 reports
- DeepSeek V4.1 Flash 552B · 16B active · oQ4e19.7 t/s, 439 t/s prompt(part in system RAM)
- DeepSeek V4 Flash 284B · 13B active · 8bit affine20.0 t/s, 533 t/s prompt
oMLX on M4 Pro 48GB2 reports
- Qwen3.8 27B · oQ4e24.8 t/s(speculative)
- GLM-5.3 · JANG1.8 t/s(speculative)
oMLX on M2 Max 96GB1 report
- Qwen3.8 125B · 6B active · oQ4e18.3 t/s, 236 t/s prompt(speculative)
oMLX on M3 Max 128GB1 report
- Qwen3.5 35B · 3B active · 4-bit90.8 t/s
oMLX on M3 Max 48GB1 report
- Qwen3.8 125B · 6B active · T538.0 t/s(part in system RAM)
oMLX on M3 Max 96GB1 report
- Qwen3.6 27B · 8bit7.4 t/s, 121 t/s prompt
oMLX on M3 Ultra 256GB1 report
- Minimax M3 428B · 23B active · 4bit17.3 t/s(2 cards, 2 requests at once)
oMLX on M5 Max 64GB1 report
- Qwen3.8 27B · oQ8e34.6 t/s, 400 t/s prompt(speculative)
oMLX on M5 Pro 64GB1 report
- Qwen3.8 27B · 4bit23.0 t/s, 368 t/s prompt(speculative)
oMLX, GPU not identified2 reports
- Qwen3.8 125B · 6B active40.0 t/s
- Qwen3.8 125B · 6B active · oQ329.1 t/s(speculative)
Top models on oMLX
Frequently asked
Is oMLX faster than other engines?
llamaperf has no matched comparison for oMLX yet: no card has plain runs of the same model size at the same bit class on oMLX and on another engine. Speed claims about engines need that pairing, so this page doesn't make one.