llamaperf

oMLX benchmarks

An inference engine for running open-weight LLMs locally.

57 community reports

oMLX results by GPU

Hardware with reported oMLX results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

oMLX on M4 Max 128GB4 reports

Filter oMLX reports on the M4 Max 128GB

oMLX on M4 Pro 48GB4 reports

Filter oMLX reports on the M4 Pro 48GB

oMLX on M3 Max 64GB1 report

Filter oMLX reports on the M3 Max 64GB

oMLX, GPU not identified6 reports

This engine doesn't yet have an editorial profile on llamaperf. The community reports below show how it's been used in practice across different hardware.

Compare GPUs running oMLX

Every card is compared on one model, Qwen3.8 27B at 4-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.

GPUVRAMReportsMedian speed, Qwen3.8 27B 4-bit
M5 Ultra 256GBapple256GB24Gen 50.0 t/sPP 1800.0 t/s · n=1
M3 Ultra 512GBapple512GB8no Qwen3.8 27B 4-bit run reportedPP 352.0 t/s · n=1
M4 Max 128GBapple128GB6no Qwen3.8 27B 4-bit run reported
M5 Max 128GBapple128GB6no Qwen3.8 27B 4-bit run reported
M4 Pro 48GBapple48GB4no Qwen3.8 27B 4-bit run reported
M1 Ultra 64GBapple64GB3no Qwen3.8 27B 4-bit run reported
M3 Ultra 96GBapple96GB3no Qwen3.8 27B 4-bit run reported
M5 Max 36GBapple36GB3no Qwen3.8 27B 4-bit run reported

oMLX against other engines

Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.

Model and GPUoMLXOther engine
Qwen3.8 125B · 6B active 4-bitM3 Ultra 512GB25.0 t/sone generation report, context not statedllama.cpp31.1 t/sPrompt processing (prefill) 558.8 tokens/s (1)one generation report, 250K context
Qwen3.8 27B 4-bitM3 Ultra 512GB-Prompt processing (prefill) 352.0 tokens/s (1)no generation reports, 64K contextMLX31.8 t/sPrompt processing (prefill) 391.2 tokens/s (1)one generation report, context not stated

Top models on oMLX

Frequently asked

Is oMLX faster than other engines?

llamaperf has 1 matched pair of oMLX and other-engine runs on the same card, model size and bit class, but none is comparable enough to say which engine is faster: the sides have too few reports or were run at different context lengths. A pair counts toward a direction only when each side has at least 3 generation reports with a stated context, and the two sides' contexts overlap. The measured figures and their conditions are in the table on this page.