Is it possible to use a 32 GB M2 Pro MacBook for local LLMs?

Some of my work is under NDA or involves personal data covered by GDPR. For that kind of work, pasting code into a cloud AI assistant is often not an option, whatever the productivity gain. So I wanted to know: can the laptop I already own run a local model that's actually usable for coding? Not a demo, but something I can use for practical tasks with a few large files in context.

The machine is an Apple M2 Pro with 32 GB of unified memory. In 2026 it costs nearly half its original price, it's comfortable for coding work, and the possibility of a privacy-oriented setup on it sounds appealing. I wanted some clarity in numbers, so I benchmarked five models on it in September 2026.

The outcome

Yes, a 32 GB M2 Pro can run a usable local coding model, and the one to pick is gpt-oss-20b. It generates 40 tokens per second on an empty context, still 35 at 16k tokens, and 19 at 128k. At 12.8 GB it's also the smallest model I tested, and the only one that fits its full 128k context in memory.

Getting there took some unlearning. The model that looks fastest in published numbers falls behind as soon as the context fills up. Dense models in the 24B to 27B range are too slow on this hardware whatever you tune. And the two standard speed tricks both made things slower.

Setup

I measured two speeds at every depth, because they're limited by different things. Prefill is how fast the model reads your prompt, and it's limited by compute. Decode is how fast it writes the answer, and it's limited by memory bandwidth. Decode is the speed you feel while a response streams in.

Results

Decode speed in tokens per second, at increasing context depth:

Prefill speed in tokens per second, for a 512-token prompt on top of the given depth:

GB is the model file size. The fastest result in each column is in bold. "n/a" means I didn't run that depth, because the model was already too slow to be worth it.

Empty-context numbers mislead

There are many mentions online of quite impressive performance, but they're almost always measured on an empty context, and coding eats up the context pretty fast. By the time a model has read a couple of source files and the conversation so far, you're at 16k tokens or more, and an agent can reach 64k in no time.

The ranking changes as the context fills:

• Qwen3-Coder is 40% faster than gpt-oss on an empty context. At 16k it's about 15% slower, and at 64k it's half as fast.
• GLM-4.7-Flash matches gpt-oss at depth 0, then keeps only 35% of that speed by 16k.
• gpt-oss keeps 86% of its speed at 16k, and still has about half of it at 128k.

If you're choosing a local model for coding, benchmark it at the context depth you'll actually use.

Why dense models are out

Generating a token means reading the model's active weights from memory once. On this machine, that makes memory bandwidth the limit, not compute. A rough ceiling is bandwidth divided by the bytes read per token.

A dense model reads all of its weights for every token. A 15 GB dense 27B model is therefore capped at around 13 tokens per second on 200 GB/s, and I measured 9. That's usable, but slow enough to break your concentration.

Mixture-of-Experts (MoE) models only read the few experts that are active for each token, around 3 billion parameters. That's why the MoE models ran 4 to 5 times faster than the dense ones at similar file sizes. On this bandwidth, only MoE models with a small active-parameter count reach a speed that feels practical.

Why some models slow down faster than others

As the context grows, every new token also has to read the KV cache (the stored attention state for everything already in context). The bigger the cache per token, the steeper the slowdown.

gpt-oss uses a 128-token sliding window on half of its layers. Those layers stop growing, so its KV cache stays around 3 GB even at 128k, and its speed declines slowly. Qwen3-Coder keeps full attention on all 48 layers, roughly 96 KB of cache per token, which is about 6 GB at 64k. You can see that in how quickly its speed drops.

GLM-4.7-Flash dropped fastest of all. It uses DeepSeek-style attention, and my best guess is that llama.cpp's Metal kernels for that attention type are less optimised than the others. I haven't verified that, so treat it as a hypothesis.

Memory sets the context limit

The model weights and the KV cache both have to fit in the roughly 25 GB the GPU can use. Qwen3-Coder fits at 64k (17.3 GB of weights plus about 6 GB of cache). At 128k it would need around 12 GB of cache and start swapping. gpt-oss-20b is the only model I tested that fits its full 128k context.

Prefill also matters more than you'd expect at long context. Reading 64k tokens takes about 4 minutes on gpt-oss and about 11 on Qwen3-Coder, and filling all 128k on gpt-oss takes around 12 minutes. In a chat or agent session the cache is reused, so you pay that once per session, not on every message. Still, it's worth knowing before you point a local model at a large repository and wonder why nothing is happening.

The usual speed tricks made it slower

Two common tuning tips are to enable flash attention with a quantised KV cache, and to use speculative decoding. I tested both on the dense Qwen3.8-27B, measuring decode speed in tokens per second:

Speculative decoding had 49% of its drafted tokens accepted, which wasn't enough to pay for drafting them. On a machine limited by memory bandwidth, both tricks cost more memory traffic than they saved. Most tuning advice is written for NVIDIA GPUs, and it doesn't automatically carry over to Apple Silicon.

Ollama isn't faster, and it can mislead you

Ollama and llama.cpp use the same Metal kernels, and the same gpt-oss-20b weights decoded within 2 to 3% of each other on both. Speed isn't the reason to choose one over the other. The problems I ran into with Ollama were elsewhere:

• A year-old gpt-oss download stopped loading after an Ollama update.
• Ollama stores models in an older GGUF format that current llama.cpp won't load, so the files aren't portable between the two.
• Ollama silently cuts prompts down to its default context window. A test you think is running at 32k can actually be running at 4k, with no error.

The last one is the one that matters for benchmarking. I ran llama.cpp directly to avoid all three.

Reasoning models feel slower than their numbers

gpt-oss thinks before it answers, and those reasoning tokens count as output. A few hundred tokens of thinking at 35 tokens per second adds ten seconds or more before the first word of the actual answer. Qwen3-Coder-Instruct answers immediately. Setting gpt-oss to low reasoning effort narrows the gap.

Conclusion

A 32 GB M2 Pro is enough for private, offline coding assistance, as long as you pick the model for long sessions rather than for its headline number. This is what I'd use:

Three things I'd tell anyone setting this up: measure at the context depth you'll really use, keep the stock settings until you've measured that a change helps, and run llama.cpp directly so you know what context you're actually getting.

One important caveat: this is a speed comparison only. I haven't compared the quality of the code yet. A fast model that writes worse code isn't a win, so the next step is to give both front-runners the same real task and judge the results. I'll write that up separately.

If you're working out whether a local model can handle work you can't send to the cloud, I'm happy to compare notes. Get in touch.