S02 E04 - Running Multi-agent AI on Kubernetes: Lessons from Imagine Learning

Running AI agents on Kubernetes is a distributed systems problem. Long-running inference requests, stateful connections, pod eviction mid-inference. The failure modes are real, and most teams are figuring this out as they go. Blake Romano, Staff Software Engineer at Imagine Learning, has been building and running multi-agent AI systems on Kubernetes for over a year. He walks through how Imagine Learning's internal developer platform chatbot evolved from a single-day AWS Bedrock prototype into a multi-agent system with an orchestrator routing to specialized sub-agents (Argo CD, documentation, ticketing) running as Kubernetes deployments. He covers why the team moved away from RAG toward real-time MCP tool calls, how GitOps makes LLMs dramatically more useful for platform teams, and why the organizational context you encode matters far more than which SDK or framework you use. He also gets into the harder problems: measuring AI quality with DORA metrics, scaling code review when engineers are shipping 10,000 lines a day, and what happens when non-engineers start vibe coding their way into your production platform. Read the blog post at: http://www.buoyant.io/ai-kubernetes-e... TAKEAWAYS ✓ How to architect a multi-agent system on Kubernetes with orchestrator, sub-agents, and the routing logic between them ✓ Why GitOps and infrastructure-as-code are prerequisites for getting real value out of AI on the platform side ✓ How to use MCP tool calls instead of RAG for real-time access to docs, Argo CD, and ticketing systems ✓ Why organizational context (system prompts, skills, MCP connections) is more durable than any specific harness or SDK ✓ How to apply DORA metrics to measure whether your AI platform tooling is actually working ✓ What platform guardrails need to handle now that non-engineers are generating and shipping code