GenAIOps 102 - Or why your Agents keep breaking - Julien Look @ Look Beyond Solutions

Julien addressed the challenge of maintaining production AI agents and preventing silent quality regressions when modifying prompts or agent architectures. He broke evaluation down into offline sandbox testing using LLM judges during development, and online monitoring using telemetry and user feedback to catch live edge cases. Emphasizing that GenAIOps requires cross-functional alignment across engineers and product teams, he cautioned against fully automated feedback loops without human oversight. Hosted by FGS Global https://fgsglobal.com/ Recording by Lukas Rieder from hibase   / lukasrieder