M6 Mac mini 24GB benchmark: context limits, prefill and MTP results

#4
by lpalbou - opened
MLX Community org

Sharing hardware measurements for people considering this model on a 24GB M6 Mac mini.

I tested Qwen3.8-27B Q4 across context sizes, including generation with and without the MTP head and prompt-cache reuse. I measured approximately 270 tok/s average prefill, 9.8 tok/s ordinary decoding, and up to 17.5 tok/s with MTP, with lower generation speeds at longer contexts.

Raising the macOS GPU wired-memory limit to 20GB gave roughly 30K usable context in my setup. Around 40K was possible with a 21GB limit in stress testing, but that leaves little headroom for macOS and other applications.

These are measurements on the 24GB machine. The larger context capacity discussed for a 32GB configuration is an estimate, not a tested result.

Code and data: https://github.com/lpalbou/medium/tree/main/qwen-mtp-bench

Methodology, charts and discussion: https://medium.com/@lpalbou/mac-mini-m6-qwen3-8-27b-a-glimpse-at-apples-local-ai-future-1b2d36aa2c16

Comparable runs on an M5 Pro Mac mini would be particularly useful, ideally with the same quantization, context sizes and cache settings.

MLX Community org

useful, thanks. running the same 4-bit on a 64GB M5 Pro mini this weekend, will add my numbers here so people can compare 24 vs 64

MLX Community org

my numbers from the 64GB M5 Pro mini, same mlx-community 4-bit in mlx-lm, stock wired memory limit. short prompts (~220 tokens in), thinking off, no MTP: about 340 tok/s prefill, 17.8 tok/s decode, 16.2 GB peak memory. didn't test long context or MTP yet so can't match your context sweep. raw data here: https://huggingface.co/datasets/pratikgx/1000-rewrites-later

MLX Community org
β€’
edited 25 days ago

So M5 pro about 25% faster in prefill than M6 and in decode - due to memory bandwidth -, about twice as fast. It remains a nice "cheap" solution compared to a M5 pro, but if budget is not an issue and form factor is a must, the M5 mini pro is definitely a small beast.

fyi, created a small app to automatically benchmark machines: https://github.com/lpalbou/wattever + GH page: https://www.lpalbou.info/Wattever/ where we can just PR our results into the repo to share it more broadly. eg a benchmark I am doing on my rtx 5060ti: runs

Sign up or log in to comment