mlx-vlm benchmarks
An inference engine for running open-weight LLMs locally.
12 community reports
mlx-vlm results by GPU
Hardware with reported mlx-vlm results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
mlx-vlm on M3 Ultra 512GB5 reports
- Qwen3.5 397B · 17B active · 6-bit29.2 t/s
- Gemma 4 31B · 4bit MLX21.4 t/s, 235 t/s prompt
- Qwen3.5 397B · 17B active · 4bit MLX33.0 t/s, 386 t/s prompt
- Gemma 4 26B · 4B active · 4bit MLX67.6 t/s, 1,645 t/s prompt
- Qwen3.8 · BF16 unquantized13.2 t/s
mlx-vlm on M5 Max 128GB2 reports
- Gemma 4 5.1B · 4bit205.0 t/s
- Qwen3.8 27B · 4bit47.0 t/s, 854 t/s prompt(speculative)
mlx-vlm on M5 Ultra 256GB2 reports
- Model not named · mlx-4bit20.0 t/s
- Mimo 2.6 309B · 15B active49.1 t/s, 1,238 t/s prompt
mlx-vlm on M5 Ultra 512GB1 report
- GLM-5.3 320B · 18B active · 4bit71.1 t/s, 736 t/s prompt(speculative)
mlx-vlm, GPU not identified1 report
- Qwen3.8 · 4bit28.0 t/s
This engine doesn't yet have an editorial profile on llamaperf. The community reports below show how it's been used in practice across different hardware.
Compare GPUs running mlx-vlm
Every card is compared on one model, LFM3 2B at 4-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.
| GPU | VRAM | Reports | Median speed, LFM3 2B 4-bit |
|---|---|---|---|
| M3 Ultra 512GBapple | 512GB | 5 | no LFM3 2B 4-bit run reported |
| M5 Max 128GBapple | 128GB | 2 | no LFM3 2B 4-bit run reported |
| M5 Ultra 256GBapple | 256GB | 2 | no LFM3 2B 4-bit run reported |
| M1 8GBapple | 8GB | 1 | Gen 1.0 t/s |
| M5 Ultra 512GBapple | 512GB | 1 | no LFM3 2B 4-bit run reported |
| M6 24GBapple | 24GB | 1 | no LFM3 2B 4-bit run reported |
mlx-vlm against other engines
Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.
No matched pair yet. No card has plain runs of one model size at one bit class on mlx-vlm and on another engine, so llamaperf can't say how it compares on speed. Add a run.
Top models on mlx-vlm
Frequently asked
Is mlx-vlm faster than other engines?
llamaperf has no matched comparison for mlx-vlm yet: no card has plain runs of the same model size at the same bit class on mlx-vlm and on another engine. Speed claims about engines need that pairing, so this page doesn't make one.