A/B testing and growth analytics at AirBnb, metrics store-Nick Handel-the data scientist show#037

Nick Handel was a senior data scientist leading the launch of the data side of this Airbnb Trips and later built a team that designed aribnb’s end-to-end machine learning platform, bighead. Currently, he is the cofounder and CEO of transform, he first centralized 'metrics store' that empowers data analysts to deliver insights. He was recognized as 30 under 30 by Forbes in 2018. If you like the show subscribe to the channel and give us a 5-star review. Subscribe to Daliana's newsletter on www.dalianaliu.com/ for more on data science. Daliana's LinkedIn:   / dalianaliu   Daliana's Twitter:   / dalianaliu   Highlights: 00:00 intro and career journey 00:10:58 common mistakes in A/B testing: overlook slicing data by different dimensions 00:14:41 common mistakes in A/B testing: Novelty Effect 00:15:37 common mistakes in A/B testing: Cannibalization 00:16:24 common mistakes in A/B testing: blend users using the same cut-off 00:17:25 one-third A/B tests lead to launches 00:25:48 how to do deep dives for A/B tests 00:27:32 surprising A/B testing results (1) 00:29:18 facts vs opinions 00:33:55 A/B testing best practices 00:38:10 surprising A/B testing results (2) 00:40:39 common mistakes in A/B testing: overlap experiments by different teams 00:46:25 use data to influence people 00:55:01 how he built a new data schema for Airbnb Trips 01:00:43 when collect data, build as many metrics and events as possible 01:15:53 do things that don't scale 01:38:53 trend of data science tools 01:52:51 how to give feedback