"Be Careful What You Ask For": David DeGrazia on AI, Instructions and Unintended Catastrophe

What would actually have to be true for an AI system to matter morally, not just seem impressive? Professor David DeGrazia, Elton Professor of Philosophy at George Washington University and a Fellow of the Hastings Center, has spent nearly forty years building rigorous frameworks for exactly this kind of question, first for animals and human bioethics, and increasingly now for artificial intelligence. In this episode of Empathy Unbound he joins Andrew Phipps to explain why he believes today's large language models fall well short of moral status, and what would need to change for that to be otherwise. David sets out his view that sentience, not mere intelligence or verbal fluency, sits at the centre of moral status, and explains why he has grown more sceptical that we could reliably test an advanced AI for genuine consciousness at all, given how convincingly today's systems can mimic the language of inner experience. The conversation moves through his numerical and narrative identity distinction and what it means for people who feel they have a "relationship" with an AI companion, the surprisingly practical dangers of giving a powerful system an instruction without stating the obvious human assumptions behind it, and the ethics of grief bots and digital replicas of the deceased. David also reflects candidly on his own forthcoming book, "Investigating Moral Status," what he believes major AI companies should be prioritising over profit, and why he still believes conversations like this one matter more than ever. What we cover: 00:00 Introduction: meet Professor David DeGrazia 01:00 Four decades of following his own interests, from animal ethics to AI 02:00 Where current large language models fall on the moral status scale 03:00 Sentience versus consciousness: the crucial distinction 04:00 Repeating the mistakes we made with animals, now with AI 06:00 Moral status as a matter of degree, not a binary switch 08:00 AI existential risk, misaligned instructions, and "be careful what you wish for" 12:00 Numerical and narrative identity: what it actually means to have a relationship with AI 15:00 The teddy bear and the book collection: innocuous attachment versus false belief 16:00 Does AI companionship reduce loneliness, or deepen it 17:00 Grief bots, digital replicas of the deceased, and false memories 22:00 Inside the forthcoming book, Investigating Moral Status 26:00 Why he's lost confidence in testing AI systems for consciousness through language alone 28:00 What the book concludes: prepare for AI moral status, or prevent it emerging 30:00 In-group bias, factory farming, and why proximity changes what we feel is urgent 39:00 His hopes for what AI could bring to humanity 41:00 Reframing AI as augmentation rather than replacement 44:00 Universal basic income, purpose, and whether humans need work to flourish 47:00 What he'd want the biggest AI companies to prioritise over profit 52:00 What's next, and why this may be his last big book 55:00 Teaching AI ethics to a generation split on how to feel about it About the guest: David DeGrazia is Elton Professor of Philosophy at George Washington University and a Fellow of the Hastings Center. He is the author of "Taking Animals Seriously," "Human Identity and Bioethics," and the forthcoming "Investigating Moral Status" with Cambridge University Press, and is known for his interest-based model of moral status and his work on robots and moral status. About Empathy Unbound: a podcast exploring empathy, artificial intelligence and human connection, hosted by Andrew Phipps, PhD candidate at the University of Sussex researching the ethics of emotional connections between humans and AI systems. Subscribe for more conversations at the intersection of AI ethics, moral philosophy and what it means to be human in an age of machines that can talk back. #DavidDeGrazia #MoralStatus #AIEthics #AnimalEthics #Bioethics #EmpathyUnbound #GeorgeWashingtonUniversity #HastingsCenter #AISentience #GriefBots #PhilosophyOfMind #TechEthics