Can AI Tell What Causes What? | Nick Huntington-Klein

Nick Huntington-Klein (Seattle University, The Effect) on why "correlation isn't causation" isn't enough, and why AI does a B+ job at causal inference. Nick, an economist and author of the free causal inference textbook The Effect, explains how to get from data to a real causal claim: why you need to know where data came from, what "controlling for" a variable does, and when it makes the answer worse. We also cover what happened when 146 research teams analyzed the same data, media mix models versus lift tests, why ad effects change when you scale a budget, whether AI can recall what causes what, and whether ChatGPT hurt high school test scores. CHAPTERS 0:00 Cold open and intro 0:43 Why "correlation isn't causation" isn't enough 2:45 What it means for X to cause Y 6:50 Why you need to know where your data came from 8:45 Causal diagrams 12:36 What "controlling for" does, and when it backfires 20:50 Difference-in-differences, regression discontinuity, and instrumental variables 25:23 Storytelling after an A/B test 29:10 146 teams, same data, different answers 32:25 Will AI help or hurt data analysis? 37:05 Media mix models vs. lift tests 44:18 What happens when you double the budget 48:12 Do promotions create demand or pull it forward? 49:40 When old results stop applying 51:48 Can AI recall what causes what? 1:00:10 Did ChatGPT hurt high school test scores? 1:04:42 Will AI invent new ways to find causal effects? 1:06:47 Rapid fire 1:10:24 "Is the best I can do good enough?" LINKS The Effect (free online): https://theeffectbook.net/ Nick's website: https://www.nickchk.com/ The Sources of Researcher Variation in Economics (the 146-teams study): https://www.nber.org/papers/w33729 Do LLMs Act as Repositories of Causal Knowledge?: https://arxiv.org/abs/2412.10635 Little Impact of ChatGPT Availability on High School Test Scores: https://arxiv.org/abs/2605.08812 Second Order on Substack: https://waismomand.substack.com/ Me on X: https://x.com/momans Note: Nick's ChatGPT and test scores study is a working paper that has not yet been peer reviewed. The AI causal-knowledge study tested models available in late 2024. Views expressed are entirely my own and those of my guests, and do not represent the positions of Google or any affiliated entity. Nothing discussed should be taken as an official statement about any company's products. #CausalInference #Econometrics #MarketingMeasurement