Why Human Control Isn't Enough in Military AI with Heidy Khlaaf

In this episode of Responsible Bytes, Dr. Zena Assaad speaks with Heidy Khlaaf, Chief AI Scientist at the AI Now Institute, about why military AI systems are far less reliable than their marketing suggests, especially in safety-critical and combat settings. They unpack the gap between “accuracy” as a narrow model metric and real-world performance, explaining how opaque, brittle systems, automation bias, outdated data, and poor interoperability can lead to serious mistakes while obscuring accountability. The conversation also explores the power dynamics behind the push for AI in warfare, including what Heidy characterises as a “safety theatre,” version-control concerns, and how companies and governments normalise speed over caution. Overall, the episode is a sharp critique of the idea that AI can safely replace human judgment in high-stakes environments, and a call for more skepticism, technical rigor, and accountability in how these systems are deployed. Chapters: 00:00 Introduction to Responsible Bytes and Heidy Khlaaf’s background 01:19 Why meaningful human control is a weak answer to unsafe AI 04:56 Human oversight cannot fix fundamentally inaccurate systems 06:14 What “accuracy” means in defense versus machine learning 07:39 Real-world accuracy collapse in military deployment 10:39 How interfaces create the illusion of intelligence 11:27 “Magic” language, data scale, and the illusion of AGI 13:37 Human feedback labor behind model fine-tuning 14:28 AI cannot know what it does not know 16:19 Why fault tolerance is hard with probabilistic systems 18:05 The Minab school bombing and bad data pipelines 19:30 Sensors, corruption, and missing interoperability 22:22 Why “just feed it more data” is not a real solution 24:13 Speed as the real military incentive behind AI 26:04 How low accuracy can obscure accountability for civilian harm 27:27 “Carpet bombing” by another name through AI-enabled targeting 30:14 Why “AI solutions” often do not solve a real capability gap 33:09 Why large language models are worse than purpose-built models 36:50 Anthropic, the US Department of Defense, and the public red lines dispute 37:47 Why the dispute is better understood as safety theater 41:42 The lack of iteration and version control in model deployment 43:15 How model providers can change behavior in real time 45:57 Concentration of power and unclear model alignment 47:40 The power shift revealed by the Anthropic and DoD standoff 49:02 Why private companies should not be monitoring state operations 51:17 The self-serving feedback loop between vendors and defense buyers 53:12 Companies grading their own homework on validation and evaluation 54:39 Heidy’s responsible byte: don’t take AI company claims at face value More about our guest: 📸Instagram:   / hak90   📘LinkedIn:   / heidy-khlaaf   🔵BlueSky: https://www.heidyk.com/ 🌐Website: https://www.heidyk.com/ More about our host: 📸Instagram:   / zena_assaad   📘LinkedIn:   / dr-zena-assaad   🔵BlueSky: https://bsky.app/profile/zenaassaad.b... 🌐Website: https://www.zenaassaad.com/ Watch or listen to more episodes: 🍎 https://podcasts.apple.com/au/podcast... 🔊 https://open.spotify.com/show/2x2yNvU... Or find us anywhere else you get your podcasts! Follow us online: 📸Instagram:   / responsiblebytespodcast   📘LinkedIn:   / responsible-bytes-podcast   🌐Website: https://www.zenaassaad.com/responsibl...