Where AGI timelines go wrong | Toby Ord, Oxford University

Both Silicon Valley and the public can’t get enough of ‘AGI timelines.’ But Toby Ord, senior researcher at Oxford’s AI Governance Initiative and author of The Precipice, believes we consistently make big mistakes when thinking about them. He lays out the 14 ways he most often sees people go wrong: 1. Assuming AI research is just hill-climbing 2. Imagining AI research is just programming 3. Forecasting “could” instead of “will” 4. Believing the current benchmark is the last one 5. Extrapolating trends with no clear finish line 6. Assuming inputs keep scaling at the same rate 7. Conflating intelligence with capability 8. Consuming point estimates and discarding the error bars 9. Dismissing dissenting experts 10. Forecasting very different things while using the same words 11. Assuming capabilities arrive together 12. Treating “we don’t know” as permission to carry on as usual 13. Choosing a plan that minimises regret rather than maximises impact 14. Trusting surface model impressiveness In this extended conversation with Rob Wiblin, Toby also explains why he thinks: • AI self-improvement is uniquely dangerous in four ways, but also might not even work • A ban on superintelligence is possible • A US-China treaty on superintelligence is also possible • ‘Broad timelines’ are what we should act on • Transformative AI is likely a decade away • We should just ban unmonitorable chain-of-thought today This episode was recorded on July 2, 2026. Links to learn more and full transcript: https://80k.info/to26 Want to get up to speed on AI? We’ve got a crash course of 10 of our podcast episodes designed to help you get to grips with transformative AI — particularly if you’re new to the topic — and what you can do to help shape its trajectory: https://80000hours.org/AIPod Chapters: • Toby Ord is back — for the 5th time! (00:00:00) • AI self-improvement might not matter (00:00:14) • 4 ways AI self-improvement is dangerous (00:12:39) • A US-China treaty on superintelligence is possible (00:20:47) • Could we ban superintelligence? (00:37:07) • We should just ban unmonitorable chain of thought (00:57:46) • Why Toby thinks AGI is a decade away (01:09:28) • Even superintelligence needs work experience (01:17:50) • Is AI coming for mathematicians? (01:32:22) • The case for broad timelines (01:45:01) • How should broad timelines change what we do? (02:22:24) • Are current models all they’re cracked up to be? (02:31:03) • Coordinating careers for different timelines (02:43:36) Our production team includes: • Video editors: Josh Alward, Dominic Armstrong, Ollie Bignell, Andrés Escobar, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour • Producers: Elizabeth Cox and Nick Stockton • Coordination and support: Katy Moore and Lou Moran • Camera operator: Jeremy Chevillotte • Music: CORBIT