Why the Neural Engine Is the Wrong Target for Your LLM (and MLX Isn't)

An iPad Pro has an M-series chip, fast unified memory and a capable GPU. Can it be a real worker in your own inference pipeline, taking model layers streamed from your server? Only through a native Swift rewrite on MLX. The Neural Engine is the wrong target: int4 decode at its compute rate would need about 3.75 TB/s of weight bandwidth, against roughly 153 GB/s available, and it needs weights baked in at conversion time. What's covered: • Why iOS cannot run a Python inference worker (no glibc, no fork or exec, no JIT, every executable page signed) • Why every code change means an Xcode rebuild, and why the protocol should be written down • Why MLX fits: affine int4 with group sizes 32 to 128, runtime-loaded weights, a matmul that matches the weight layout • Why the Neural Engine fails twice: bandwidth-starved, and no streamed weights • Per-app memory: about 4 to 5 GB on an 8 GB device, 9 to 10 GB on a 12 GB one • iPhone vs iPad Pro bandwidth, and why a reported 30 tok/s for a 14B looks optimistic • Foreground-only, throttling and sideloading limits • Prototyping one layer in MLX on a Mac, three checks on the device, and a decision list ⏱️ Chapters 00:00 LLMs on an iPad 01:47 Why Python Can't Run 03:25 Why MLX Fits 04:30 Not the Neural Engine 05:43 Memory and Bandwidth 07:27 Running It for Real 08:39 Before You Build 📖 The full guide (iOS restriction table, memory and bandwidth tables, the worker design, the Mac prototype script, the decision list): → LLMs on an iPhone or iPad: MLX Yes, Neural Engine No: https://www.bigiron.cc/guides/llms-on... 🛠️ Gear (affiliate — same price to you, supports the channel): • Tablet stand for a dedicated, always-on device → https://www.amazon.com/s?k=adjustable... • USB-C charger to keep it on power → https://www.amazon.com/s?k=usb+c+char... • 24 GB-class GPU, usually the easier upgrade → https://www.amazon.com/s?k=24gb+graph... As an Amazon Associate we earn from qualifying purchases. Affiliate links — same price to you, supports the channel. 👉 Subscribe for self-hosted AI + homelab, with new videos every day. #LocalAI #LLM #MLX #iPad #iPhone #AppleSilicon #NeuralEngine #SelfHosted #Homelab #InfiniteModel