Building highly reliable data pipelines at Datadog | Datadog

Get the slides: https://www.datacouncil.ai/talks/buil... Download slides of this talk: https://www.dataengconf.com/speaker/b... ABOUT THE TALK: Every day at Datadog we run massive data pipelines to power core functionality. In this talk, we'll cover best practices we use at Datadog to ensure that we reliably deliver this functionality while processing trillions of points/day in the face of exponential data growth, hardware failures, corrupt data, and even human error. ABOUT THE SPEAKER: Quentin is a Data Engineering Team Lead at Datadog. Over the past 3 years, he has worked with Luigi, Spark, and Go to build and scale many highly reliable data pipelines. These pipelines power numerous internal and customer facing features - including enabling Datadog's users to perform fast real time queries against PBs of historical time series data. ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups. FOLLOW DATA COUNCIL: Twitter:   / datacouncilai   LinkedIn:   / datacouncil-ai   Facebook:   / datacouncilai   Eventbrite: https://www.eventbrite.com/o/data-cou... - 🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️ Meet the world's top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions. → https://aicouncil.com/sf-2026 ⚡ FIND US: X: https://x.com/AICouncilConf LinkedIn:   / aicouncilconf   Website: https://aicouncil.com/