Planning a local LLM setup on a Mac with unified memory

Chip and memory, together

"M4 Pro" covers a whole family of machines. A 24 GB Mac mini and a 48 GB MacBook Pro with that chip will give you very different answers, and some memory tiers come with a cut-down GPU on a narrower memory bus. Record the full chip name, the GPU core count if Apple lists one, the memory size, your macOS version and your runtime. Someone comparing with you needs every one of those.

The Mac directory sorts community reports by chip and memory size. Open the ones that match your machine and check the model file, quantization and engine. Where a report leaves a setting out, count it as unknown. Filling the gap with the setting you'd have picked makes the report say something its author didn't.

How far a close match gets you

A report from a bigger or smaller configuration still tells you a model loads and roughly how it felt to that person, so it's a good lead. It won't predict your speed. Bandwidth changes between tiers, and the context size behind the report may be nothing like yours.

macOS keeps a share for itself

Your Mac's memory is shared, and macOS won't give all of it to the GPU. Metal publishes a limit for each device, and llama.cpp, MLX and Ollama all stop at it. Apple defines it like this:

"An approximation of how much memory, in bytes, this GPU device can allocate without affecting its runtime performance." (Apple Metal documentation: recommendedMaxWorkingSetSize)

In practice that's about two thirds of memory on Macs up to 32 GB and three quarters from 36 GB up, and the logs people post bear it out. On a 36 GB M3 MacBook Pro, Ollama printed a limit of 28991.03 MB, which is exactly 27 GiB, three quarters of the machine (ollama issue 2370). By the same rule a 16 GB Mac gives a model roughly 10.7 GB before macOS says no.

Size the model against that limit

The calculator grades a Mac against this limit, with the weights, the context cache and the runtime allowance on separate lines. Its Raised option grades against your memory minus 8 GB and prints the sudo sysctl iogpu.wired_limit_mb=... command that sets it. Whether you raise it is up to you, since whatever the model gains comes out of what macOS and your other apps have.

Choose a context that fits your work. If you mostly ask short questions, you don't need a plan built around a 100-page document. If long documents are the reason you're doing this, test with one from the first day. The long prompt guide explains why the wait before the first word grows with input length.

A clean run first

Close everything you can and give each candidate the same prompts with the same output limit. Note whether each task passed and how long it took, to the first word and to the end. Every setting you try later gets compared against this baseline.

Then a normal working day

Now open your editor, your browser with its usual pile of tabs and anything else you normally keep running. Repeat your most important task while you watch the memory pressure graph in Activity Monitor. Apple says what the graph is made of:

"Memory pressure is determined by the amount of free memory, swap rate, wired memory, and file cached memory." (Apple: View memory usage in Activity Monitor)

When it goes yellow or red and swap climbs, the model and your apps are competing for memory, and either the model or your apps will slow down. Keep these timings in their own group. Average them into the clean baseline and you lose the difference this run was meant to find.

Keep every run, too, because one quick test on an idle machine says little about the rest of your day. And note whether you care about interactive chat or batch jobs you leave running, since you'll put up with very different waits for each.

Try a smaller option next to the big one

Run a lighter model beside the largest one that fits. When both pass your checks, the quicker one usually makes the better default, and the big one gets the jobs the small one keeps failing. Write those jobs down. Two setups for two kinds of work is a perfectly good place to end up.

Buy for a workload you can name

Before paying for more memory, name the task your current Mac can't do. Then find reports from the machine you're considering, ideally with the same model and quantization. More memory lets a bigger model load, and that's all it does for speed: on a Mac, generation follows memory bandwidth, which comes with the chip tier. The hardware guide covers building a shortlist before you spend anything.

If you submit your results, put the chip, memory size and runtime beside your raw timings. Leave out the memory size and nobody can tell which tier your numbers came from, and on some chips the tiers don't even share a memory bus width.