The Future of Data Engineering in the Age of AI | Erfan Hesami

I sat down with Erfan Hesami, founder of Pipeline to Insights, to talk about where data engineering is headed as AI and agents become part of everyday engineering work. We dig into the changing role of the data engineer, the growing overlap between data and AI engineering, and why strong fundamentals may actually become more important as AI makes writing code cheaper and easier We also talk about AI agents inside modern data platforms, governance and security, AI-generated code and technical debt, controlling AI costs, the importance of human judgment, and why communication and business skills are becoming increasingly valuable for engineers. Erfan also shares the story behind Pipeline to Insights, how writing helped accelerate his own learning, and how consistency helped him grow the publication from its first post in October 2024 into a successful data and AI community. If you’re a data engineer, analytics engineer, AI engineer, or someone trying to break into data, this conversation offers a grounded look at what skills are likely to matter as AI changes how we build software and data systems.