We have 3 years to solve alignment before superintelligence

When should governments slow the race toward superintelligence? According to Geoffrey Irving, the careful answer is sometime in the past. The useful answer is now. Geoffrey — formerly a safety researcher at OpenAI and Google DeepMind and chief scientist at the UK AI Security Institute — expects full-blown superintelligence in roughly two to three years. Want to work with Geoffrey to help align superintelligence? Resolution is hiring! https://80k.info/work-at-resolution The leading AI companies all have broadly similar plans for keeping superintelligence under control: • Train models to have good character • Use increasingly capable AIs to supervise other AIs • Monitor them closely for signs of deception or scheming Geoffrey thinks that combination could work. The alarming part is that nobody has a strong argument that it will. He expects a crucial “phase shift” as models move beyond human intelligence: • Below that threshold, humans can usually tell whether a model’s work is good and correct its mistakes. • Above it, the models themselves will increasingly determine the feedback used to train their successors. In this episode, Geoffrey and new host Tom Reed explore what might go wrong with the companies’ plans; why Geoffrey’s new nonprofit, Resolution, is pursuing a portfolio of neglected research bets; and whether governments should slow AI development while we work out which methods can actually be trusted. This episode was recorded on June 29, 2026. Full transcript, video, and links to learn more: https://80k.info/gi Chapters: 00:00:00 Cold open 00:00:32 Meet Tom Reed — our newest host! 00:00:59 Who’s Geoffrey Irving? 00:01:38 What misaligned superintelligence will look like 00:12:30 Why are AI companies more optimistic about alignment than Geoffrey? 00:28:05 Why Geoffrey expects superintelligence in 2–3 years 00:31:30 When and how to slow down frontier AI development 00:39:22 Safety researchers can have more impact in governments than companies 00:46:55 How Geoffrey’s new organisation plans to tackle alignment 00:50:29 Post-ASI science: nanotech, solving ageing, and uploaded minds 01:03:30 Why we should expect superintelligence to accelerate scientific progress 01:11:03 Can good character training carry over to superintelligence? 01:16:44 What the field of AI alignment still doesn’t know 01:24:36 Lessons from politics on how to combat power seeking 01:29:22 Solving Pentago and working at Pixar 01:32:40 Geoffrey’s best prediction 01:37:38 Geoffrey’s best bets on which alignment techniques will work 01:43:34 Work with Geoffrey at Resolution 01:54:17 The dangerous asymmetry between capabilities and alignment Our production team includes: • Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour • Producers: Elizabeth Cox and Nick Stockton • Coordination and support: Katy Moore and Lou Moran • Camera operator: Jeremy Chevillotte Music: CORBIT