oMLX benchmarks
An inference engine for running open-weight LLMs locally.
57 community reports
oMLX results by GPU
Hardware with reported oMLX results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
oMLX on M5 Ultra 256GB22 reports
- Qwen3.8 · oQ6e5.5 t/s(speculative)
- DeepSeek V4.1 Flash 552B · 16B active44.4 t/s(2 cards)
- Tencent-HY4 · oQh8-BF1626.0 t/s
- Qwen3.8 125B · 6B active · NVFP4-QAD100.0 t/s(22 requests at once, speculative)
- GLM-5.3 320B · 18B active · 4-bit74.0 t/s(speculative)
oMLX on M3 Ultra 512GB6 reports
- Qwen3.8 27B · oQ4e352 t/s prompt
- DeepSeek V4.1 Flash 552B · 16B active3,123 t/s prompt
- DeepSeek V4 Flash 284B · 13B active · MXFP414.9 t/s, 445 t/s prompt(speculative)
- Qwen3.8 125B · 6B active · 4bit25.0 t/s
- DeepSeek V4.1 Flash 552B · 16B active · oQ4e19.7 t/s, 439 t/s prompt(part in system RAM)
oMLX on M5 Max 128GB5 reports
- Qwen3.8 27B · oQ5e50.3 t/s, 1,084 t/s prompt(speculative)
- Qwen3.6 35B · 3B active · oQ5e-mtp154.0 t/s(speculative)
- Qwen3.6 35B · 3B active · oQ5e198.6 t/s, 3,658 t/s prompt(speculative)
- Qwen3.8 27B · oQ4e8.4 t/s(speculative)
- DeepSeek V4 Flash 284B · 13B active · 2.44bit-mixed36.1 t/s, 498 t/s prompt
oMLX on M4 Max 128GB4 reports
- Qwen3.8 125B · 6B active · oQ4e57.4 t/s, 586 t/s prompt
- Qwen3.8 27B · oQ4e53.3 t/s, 274 t/s prompt(speculative)
- Qwen3.8 125B · 6B active · oQ5e53.9 t/s, 679 t/s prompt(speculative)
- Qwen3.6 35B · 3B active72.5 t/s, 1,472 t/s prompt
oMLX on M4 Pro 48GB4 reports
- Inkling-2 · oQ4e23.9 t/s, 185 t/s prompt(speculative)
- Qwen3.6 35B · 3B active · oQ490.0 t/s(speculative)
- Qwen3.8 27B · oQ4e24.8 t/s(speculative)
- GLM-5.3 · JANG1.8 t/s(speculative)
oMLX on M5 Max 36GB3 reports
- Qwen3.6 35B · 3B active · 4-bit145.8 t/s
- Qwen3.5 9B · 4-bit78.0 t/s
- Qwen3.8 27B · 4-bit25.3 t/s
oMLX on M3 Max 48GB2 reports
- Qwen3.8 27B · oQ4e-mtp100.0 t/s(speculative)
- Qwen3.8 125B · 6B active · T538.0 t/s(part in system RAM)
oMLX on M3 Ultra 256GB2 reports
- Ling-3.0 124B · 5.1B active · oQ5 affine mixed precision, group 6473.1 t/s
- Minimax M3 428B · 23B active · 4bit17.3 t/s(2 cards, 2 requests at once)
oMLX on M4 Max 64GB2 reports
- Model not named · MLX affine 4-bit, group 6450.9 t/s, 512 t/s prompt
- Model not named · MLX affine 4-bit g64; gate modules 8-bit103.9 t/s, 1,807 t/s prompt
oMLX on M5 Ultra 512GB2 reports
- DeepSeek V4.1 Flash 552B · 16B active · oQ4e54.0 t/s, 1,100 t/s prompt(speculative)
- GLM-5.3 753B · 40B active · 4bit25.7 t/s
oMLX on M1 Ultra 64GB1 report
- Model not named · oQ6e mixed affine + FP16111.8 t/s(speculative)
oMLX on M2 Max 32GB1 report
- Ornith1.5 35B · 3B active · 4-bit108.0 t/s
oMLX on M2 Max 96GB1 report
- Qwen3.8 125B · 6B active · oQ4e18.3 t/s, 236 t/s prompt(speculative)
oMLX on M2 Ultra 128GB1 report
- Qwen3.8 27B · oQ4e23.0 t/s, 304 t/s prompt
oMLX on M3 Max 128GB1 report
- Qwen3.5 35B · 3B active · 4-bit90.8 t/s
oMLX on M3 Max 64GB1 report
- Qwen3.8 125B · 6B active · oQ4e15.8 t/s, 140 t/s prompt(speculative, part in system RAM)
oMLX on M3 Max 96GB1 report
- Qwen3.6 27B · 8bit7.4 t/s, 121 t/s prompt
oMLX on M4 Pro 24GB1 report
- Qwen3.8 35B · 3B active · IQ3_M45.0 t/s
oMLX on M5 Max 48GB1 report
- Qwen3.8 27B · 4-bit93.8 t/s, 1,207 t/s prompt(speculative)
oMLX on M5 Max 64GB1 report
- Qwen3.8 27B · oQ8e34.6 t/s, 400 t/s prompt(speculative)
oMLX on M5 Pro 64GB1 report
- Qwen3.8 27B · 4bit23.0 t/s, 368 t/s prompt(speculative)
oMLX on M5 Ultra 96GB1 report
- Qwen3.8 125B · 6B active · oQ3.5e90.0 t/s, 1,260 t/s prompt(speculative)
oMLX, GPU not identified6 reports
- Qwen3.8 27B · oQ8e-mtp50.0 t/s(speculative)
- Qwen3.8 27B · oQ8e171 t/s prompt
- Qwen3.8 27B · Q675.0 t/s
- Qwen3.8 27B28.0 t/s
- Qwen3.8 125B · 6B active40.0 t/s
This engine doesn't yet have an editorial profile on llamaperf. The community reports below show how it's been used in practice across different hardware.
Compare GPUs running oMLX
Every card is compared on one model, Qwen3.8 27B at 4-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.
| GPU | VRAM | Reports | Median speed, Qwen3.8 27B 4-bit |
|---|---|---|---|
| M5 Ultra 256GBapple | 256GB | 24 | Gen 50.0 t/sPP 1800.0 t/s · n=1 |
| M3 Ultra 512GBapple | 512GB | 8 | no Qwen3.8 27B 4-bit run reportedPP 352.0 t/s · n=1 |
| M4 Max 128GBapple | 128GB | 6 | no Qwen3.8 27B 4-bit run reported |
| M5 Max 128GBapple | 128GB | 6 | no Qwen3.8 27B 4-bit run reported |
| M4 Pro 48GBapple | 48GB | 4 | no Qwen3.8 27B 4-bit run reported |
| M1 Ultra 64GBapple | 64GB | 3 | no Qwen3.8 27B 4-bit run reported |
| M3 Ultra 96GBapple | 96GB | 3 | no Qwen3.8 27B 4-bit run reported |
| M5 Max 36GBapple | 36GB | 3 | no Qwen3.8 27B 4-bit run reported |
oMLX against other engines
Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.
| Model and GPU | oMLX | Other engine |
|---|---|---|
| Qwen3.8 125B · 6B active 4-bitM3 Ultra 512GB | 25.0 t/sone generation report, context not stated | llama.cpp31.1 t/sPrompt processing (prefill) 558.8 tokens/s (1)one generation report, 250K context |
| Qwen3.8 27B 4-bitM3 Ultra 512GB | -Prompt processing (prefill) 352.0 tokens/s (1)no generation reports, 64K context | MLX31.8 t/sPrompt processing (prefill) 391.2 tokens/s (1)one generation report, context not stated |
Top models on oMLX
Frequently asked
Is oMLX faster than other engines?
llamaperf has 1 matched pair of oMLX and other-engine runs on the same card, model size and bit class, but none is comparable enough to say which engine is faster: the sides have too few reports or were run at different context lengths. A pair counts toward a direction only when each side has at least 3 generation reports with a stated context, and the two sides' contexts overlap. The measured figures and their conditions are in the table on this page.