Ep. 5 | Building Reliable AI: Testing, Governance, and Agents, with Andrew Freed

In this episode, John sits down with Andrew Freed, Distinguished Engineer and Inventor at IBM, to talk about what it really takes to build AI systems that work in production. They explore why testing becomes even more important as AI systems grow more autonomous, how teams should define what “good” looks like, and where guardrails, governance, and continuous monitoring fit into the development process. The conversation also covers the challenges of voice AI, using LLMs as judges, choosing the right evaluation metrics, improving AI systems after launch, and why a smaller set of high-quality test cases can be more valuable than thousands of poorly chosen ones. A practical discussion on building, evaluating, and improving reliable AI in the agentic era.