5 metrics that prove your AI agent works with Conviva's Keith Zubchevich

Your AI agent hit its numbers. That doesn't mean it's working. Keith Zubchevich, President and CEO of Conviva, breaks down why adoption rates and satisfaction scores miss what's actually happening inside a conversation. Conviva spent 20 years measuring quality of experience for the world's largest streaming platforms, and Keith walks through what that work reveals about agents: the outcome only tells you if the transaction closed, not whether the experience that got someone there was any good. Key takeaways: Turns to intent: how many back-and-forths it takes before the agent understands what you actually want. One or two is fine. Seven means you're paying for the agent's confusion. Clarification ratio: how often a customer has to repeat or restate their request. Manual search between turns: when a user quietly leaves the conversation to search the site themselves instead of trusting the agent to find it. Scope-narrowing rate: when a customer settles for less than they first asked for. Looks like a win. Usually means the agent couldn't handle the full request. Post-conversation correction rate: what the customer does after the agent marks the job done, like returning a product or fixing a record themselves. Also covered: why bad agent experience is one of the biggest drivers of token cost, and why testing an agent pre-launch can't substitute for watching it run at full population scale in production.