Apache Kafka @ Wikimedia foundation - Andrew Otto

Presented by Andrew Otto. Full post with slides here: https://www.hakkalabs.co/articles/apa... Andrew Otto, Systems Engineer at Wikimedia Foundation, talks about the analytics cluster at Wikimedia that allows them to support ~20 billion page views a month (Kafka, Hadoop, Hive, etc). Andrew shares how and why they chose to go with Kafka (scalable log transport) and how they've implemented Kafka with four brokers, a custom-built producer and kafkatee and Camus as their consumers. ABOUT DATA COUNCIL: Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups. FOLLOW DATA COUNCIL: Twitter:   / datacouncilai   LinkedIn:   / datacouncil-ai   Facebook:   / datacouncilai   Eventbrite: https://www.eventbrite.com/o/data-cou... - 🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️ Meet the world's top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions. → https://aicouncil.com/sf-2026 ⚡ FIND US: X: https://x.com/AICouncilConf LinkedIn:   / aicouncilconf   Website: https://aicouncil.com/