Neil Movva - 1MW datacenters, commoditizing compute, and why 95% of inference could be asynchronous

Neil Movva, co-founder and CEO of Sail Research, is building infrastructure for AI agents that run for hours or days. We discuss why fast inference is expensive, where 5-10× cost savings on latency-insensitive inference could come from, and why economies of scale might favor small, distributed data centers. Timestamps: 00:00 Intro: Async inference eats the world 00:58 The AI inference market 03:34 How inference works 08:09 Chips and MoE 15:48 Proactive agents and deep research 19:24 Modular data centers 25:55 Reliability on dropped workloads 31:16 Why slower inference costs less 35:03 Systems engineering 40:06 Can compute become a commodity? 43:00 Post-training and reinforcement learning 47:41 Separating prefill from decode (PD disaggregation) 51:34 Building and financing data centers 1:00:37 Compute contracts and alternative chips 1:04:42 Sail's vision of asynchronous inference Substrate publishes researched writing and conversations on frontier technology. Subscribe for interviews and transcripts: Website: https://substratemag.com X: https://x.com/substratemag Substack: https://read.substratemag.com