Why More RAM Won't Make Your Local AI Faster (Memory Bandwidth Explained)

Why can a Mac with more memory run the same AI slower? One number decides how fast AI types on any Mac: memory bandwidth. Here's what it is, how to find yours, and how to predict a model's speed before you download it. WHAT YOU'LL LEARN • Every word an AI writes means reading the whole model from memory once. So speed depends on how fast your Mac can read its memory, not how much memory it has. • The rule: top speed (tokens/s) ≈ memory bandwidth (GB/s) ÷ model size (GB). On real Macs running a big model, speed lands at about 80–95% of that limit. • Same model, same question: an M2 Pro with 16 GB (200 GB/s) writes about 39 tokens a second; an M3 Pro with 36 GB (150 GB/s) writes about 31. • More memory on the same chip adds parking, not lanes: every M5 Pro has 307 GB/s, whether it has 24 GB or 64 GB. • "Up to 4× faster AI" is mostly about reading your question. An M5 Pro reads about 3.7× faster than an M4 Pro, but writes only about 1.3× faster. • Worked example: an M5 (153 GB/s) with a 7.2 GB model gives a ceiling of about 21 tokens a second; the community benchmark measured 18.4. Which AI model fits your Mac (companion video):    • Best Local AI Model for Your Mac's RAM: 8,...   Set up Ollama on your Mac:    • How to Run a Local LLM on Your Mac with Ol...   How the numbers were made: speeds for other Macs come from the llama.cpp community benchmark (Llama 2 7B, the same test on every chip). Bandwidths are Apple's published specs. Estimates are labelled as estimates on screen. ── CHAPTERS ── 0:00 Why can more RAM mean slower AI? 0:34 RAM vs speed: the car park picture 1:19 What is memory bandwidth? 2:09 Does bandwidth really predict AI speed? 3:02 Why a bigger Mac can still be slower 3:56 Why new Macs say 4× faster AI 4:56 Quiz: which Mac writes fastest? 5:38 Predict any model's speed on your Mac 6:16 The race, explained ── SOURCES ── llama.cpp: Apple Silicon M-series benchmarks (GitHub discussion #4167) https://github.com/ggml-org/llama.cpp... Apple MLX: Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU ("generating subsequent tokens is bounded by memory bandwidth") https://github.com/ml-explore/mlx/dis... Databricks: LLM Inference Performance Engineering: Best Practices (Oct 2023) https://www.databricks.com/blog/llm-i... Apple: MacBook Pro tech specs (M5, M5 Pro, M5 Max) https://www.apple.com/macbook-pro/specs/ Apple: MacBook Air tech specs (M5) https://www.apple.com/macbook-air/specs/ Apple: Mac mini tech specs (M6, M5 Pro) https://www.apple.com/mac-mini/specs/ Apple: Mac Studio (M5 Max / M5 Ultra) tech specs https://support.apple.com/en-us/128107 Apple: MacBook Pro (14-inch, M3 Pro or M3 Max, Nov 2023) tech specs https://support.apple.com/en-us/117736 Apple Newsroom: Introducing M1 Pro and M1 Max (Oct 2021) https://www.apple.com/newsroom/2021/1... Apple Newsroom: Apple unveils a more powerful Mac mini featuring the all-new M6 and M5 Pro (Aug 2026) https://www.apple.com/newsroom/2026/0... Apple: MacBook Air (M1, 2020) tech specs (no bandwidth listed, hence the table in the video) https://support.apple.com/en-us/111883 ── FURTHER READING ── Horace He: Making Deep Learning Go Brrrr From First Principles (memory-bound vs compute-bound, explained) https://horace.io/brrr_intro.html Databricks: LLM Inference Performance Engineering (why writing is limited by memory, and why long chats slow down) https://www.databricks.com/blog/llm-i... Every week I explain one idea behind the AI on your own Mac. Subscribe so you don't miss the next one. #memorybandwidth #localllm #applesilicon #ollama #llamacpp #macbookpro #macbookair #m5 #localai #llm #aiexplained #machinelearning #ringzero