The Self-Driving Car Problem Nobody Can Solve | The Trolley Problem

June 2016. A website goes live at MIT: a cartoon self-driving car with failed brakes, people crossing ahead, and you have to pick who it hits. It was called the Moral Machine, and by the time the results came out in Nature in 2018 it had collected about 40 million decisions from millions of people in 233 countries and territories. We climb into it one level at a time. Where the puzzle came from: Philippa Foot's 1967 runaway tram, the surgeon case with the same math, and Judith Jarvis Thomson's footbridge. What 40 million clicks said: humans over animals, more lives over fewer, the young over the old, a ranking with a baby in a stroller at the top and the cat in last place, and three clusters of countries that did not agree on how much to save the young. Who gets to write the rules: Germany's 2017 ethics commission, which says a car must not sort people by age, gender or health. Where you come in: people want everyone else's car to sacrifice its passenger, but they want to ride in the one that protects them. And the part the game left out: in Florida in May 2016, a Tesla on Autopilot never saw a white trailer against a bright sky. There was no trolley choice at all. Sources Awad et al. (2018), "The Moral Machine experiment", Nature 563, 59-64 (about 40 million decisions, 233 countries and territories, ten languages, the three strongest preferences, the character ranking, the German rules 7 and 9 and the clash with sparing the young, no doubt about who the characters were, self-selected players): https://cdn.vanderbilt.edu/vu-my/wp-c... MIT News (2018): Western, Eastern and Southern clusters; Iyad Rahwan on interest beyond their wildest expectations: https://news.mit.edu/2018/how-autonom... Paper record: https://www.media.mit.edu/publication... Philippa Foot (1967), Oxford Review; Thomson's name and footbridge case: https://ethics.org.au/ethics-explaine... and https://britannica.com/topic/trolley-... German Ethics Commission on Automated and Connected Driving, June 2017, 20 guidelines: https://www.hunton.com/privacy-and-cy... Bonnefon, Shariff & Rahwan (2016), "The social dilemma of autonomous vehicles", Science: https://arxiv.org/abs/1510.03346v2 APS summary: https://www.psychologicalscience.org/... The May 7, 2016 crash near Williston, Florida, and Tesla's statement: https://www.pbs.org/newshour/nation/d... and https://www.repairerdrivennews.com/20... NTSB investigation (highway photo, slide 14): https://commons.wikimedia.org/wiki/Fi... The dilemma cards are our own recreation of the idea, not the real site. Track maps, the road diagram and the camera view are teaching drawings, not real data. The paper pages and the German rules are our short summaries. Painted scenes are illustrations; no real person is painted. Photos (Wikimedia Commons) MIT Great Dome: Daderot, public domain Iyad Rahwan: Joi Ito, CC BY 2.0 15 Walton Street, Oxford (blue plaque to Philippa Foot): A J Paxton, CC BY-SA 2.0 Federal Ministry of Transport, Berlin: Daderot, CC0 2015 Tesla Model S: Jeremy from Sydney, CC BY 2.0 US-27A near Williston, Florida: NTSB, public domain Logo: Tesla, used to name the company discussed. Chapters 0:00 June 2016: a website at MIT 0:32 The Moral Machine 0:57 You'd have played it too 1:15 Level 1: where the puzzle came from 1:40 The surgeon: same math, different answer 1:58 Doing harm vs letting harm happen 2:23 The footbridge 2:35 Level 2: what 40 million clicks said 2:49 Who got saved most (and the cat) 3:09 Three clusters of countries 3:34 Level 3: Germany writes the rules 4:06 The people vs the rules 4:30 Level 4: would you buy that car? 4:59 A social dilemma 5:18 You were a judge 5:45 Level 5: the part the game left out 6:18 No trolley choice 6:39 What the paper itself says