Why Most Voice Agents Fail Before They Ever Launch

Voice AI is moving from simple chatbots to systems that can actually make calls, book appointments, understand different languages, and take action. In this episode of The AI Playbook, Ravi Harsh sits down with Raj Jethwa, Voice AI Strategist at Speechmatics, to understand what it really takes to build reliable voice AI systems. They discuss how voice agents are being used in healthcare, legal, and other industries, why speech accuracy and real-world data matter, and what businesses often get wrong when adopting voice AI. The conversation also explores voice cloning, deepfakes, AI scams, multilingual conversations, job displacement, AI regulations, and how people can identify whether they're speaking to a real person or an AI agent. Timestamps: 0:00 - The voice-cloning accent problem 2:15 - Intro: Raj, 35 years in tech, voice AI strategist 2:31 - What he actually does at Speechmatics 6:16 - Helping companies build voice agents, not just selling tech 8:45 - Why speech accuracy is the real foundation 9:26 - The complexity of voice AI in healthcare 13:36 - His 35-year journey: data, cloud, big data, to AI 17:19 - Comparing the dot-com bubble to the AI bubble 21:27 - Why data matters, even for small businesses 26:02 - What people get wrong about voice AI 27:04 - Case study: an NHS trust needing Bengali support 29:10 - "The people are the project" 30:00 - Addressing fear of job loss with empathy 33:14 - The accent and multilingual training challenge 36:26 - Code-switching between languages in real conversations 37:28 - Can companies legally train on your YouTube voice? 38:52 - How to detect a cloned voice call from family 41:30 - How businesses should identify AI callers (EU AI Act) 43:35 - When an AI agent gets asked the wrong questions 45:46 - Steps businesses should take before deploying voice AI 48:35 - The "2.4 hours returned to a doctor" claim explained 50:04 - Will 2026 be the year voice AI proves reliable? 54:01 - Which industries will resist AI the longest (plumbers, electricians) 55:30 - Does the UK need its own AI Act? 58:14 - Final advice: test in real-world conditions In this episode • How voice AI actually works • What makes a voice agent reliable • Voice AI in healthcare and appointment booking • Why data quality matters • Handling accents and multiple languages • Voice cloning and deepfake risks • How to recognize AI voice scams • Why AI agents need strong guardrails • Will voice AI replace jobs? • The future of voice AI across industries • What businesses should consider before building a voice agent Connect with Raj LinkedIn:   / rajkjethwa   🎙️ AI Playbook with Ravi is available across all major podcast, newsletter, and social platforms. Listen, subscribe, and follow on your preferred platform: 🎧 Apple Podcastshttps://podcasts.apple.com/us/podcast... 🎧 Spotifyhttps://open.spotify.com/show/033VeYR... 📰 Substackhttps://substack.com/@aiplaybookwithravi 💼 LinkedIn Newsletter  / ai-playbook-with-ravi-7492329752616284160   📸 Instagram  / ai_playbook_with_ravi   🤝 Connect with Ravi Harsh on LinkedIn  / raviharsh   Subscribe wherever you listen or read—and join us as we explore how leaders, founders, and innovators are using AI in the real world. #AIPlaybookWithRavi