Poll: Who should a self driving car protect in an unavoidable crash

[RicFlair_What?]

TL;DR

• Self driving cars have revived the trolley problem, a thought experiment introduced by philosopher Philippa Foot in 1967
• The Moral Machine experiment, published in Nature in 2018, gathered about 40 million decisions from people in 233 countries and territories about how autonomous cars should behave in crashes
• A 2016 study in Science found that people approve of cars that minimise total casualties but would prefer to buy one that protects its own passengers
• Germany's 2017 Ethics Commission on automated driving said cars must not choose between people based on personal features such as age or gender
• In practice, engineers focus on avoiding crashes altogether, but the poll asks what you think is right when a choice cannot be avoided

The dilemma

Imagine a self driving car whose brakes fail on a busy street. It can stay on course and hit several pedestrians, or swerve and crash into a wall, seriously injuring its own passenger. There is no time for a human to take over. What should the car be programmed to do

Questions like this used to belong only to philosophy seminars. In 1967 Philippa Foot described a runaway tram heading towards five workers, with the option of diverting it towards one. Judith Jarvis Thomson later named it the trolley problem and explored many variations. Self driving cars have turned this classic puzzle into a practical design question

What people around the world said

In 2018 Edmond Awad and colleagues at MIT published the Moral Machine experiment in Nature. An online game asked people to choose between outcomes in unavoidable crashes, such as sparing passengers or pedestrians, the young or the old, more lives or fewer. The project gathered about 40 million decisions from people in 233 countries and territories. It is one of the largest studies of moral preferences ever conducted

Some preferences were widely shared, such as sparing more lives rather than fewer and sparing humans over animals. Others varied between regions and cultures. The researchers stressed that their results describe public attitudes, not what is morally correct. They hoped the findings would inform, rather than dictate, public discussion

Critics of the experiment have raised fair points as well. Participants chose from simplified scenarios with perfect information, which real cars never have. Online volunteers are not a representative sample of each country's population. The authors acknowledged these limits, and the data is best treated as a starting point for discussion

The social dilemma

A 2016 study in Science by Jean-Francois Bonnefon, Azim Shariff and Iyad Rahwan found a striking tension. Most participants agreed that cars should minimise total casualties, even if that meant sacrificing their passenger. Yet many said they would prefer to buy a car that protected its own occupants. The authors called this a social dilemma, because what people want for society differs from what they want for themselves

This tension matters for safety overall. If people refuse to buy cars that might sacrifice them, fewer people may adopt self driving technology. If that technology turns out to be much safer than human drivers, slow adoption could cost more lives than any rare dilemma. Ethics and public trust are deeply linked here

What rule makers have said

In 2017 Germany's Ethics Commission on Automated and Connected Driving published twenty guidelines. One states that in unavoidable accidents, any distinction based on personal features such as age, gender or physical condition is strictly prohibited. The commission also said that the protection of human life must take priority over damage to property or animals. These principles are among the clearest official statements on the question

Many engineers point out that real crashes rarely look like trolley problems. Cars usually have uncertain information and very little time, and the best strategy is almost always to brake hard and avoid collisions in the first place. Some argue that trolley style dilemmas distract from more important safety questions. Others reply that rare cases still deserve clear principles, because people want to know how machines will behave

Public trust is shaped by real incidents as much as by thought experiments. Crashes involving automated vehicles receive intense media coverage, even when human drivers cause far more accidents overall. People tend to judge machines more harshly than humans for the same mistake. That reaction is worth remembering when we answer polls like this one

Questions to discuss

• Should a car always protect its passengers, always minimise total harm or something else
• Would you buy a car that might sacrifice you to save others
• Should these rules be set by manufacturers, governments or the public
• Is the trolley problem useful for real world engineering, or a distraction

Closing thoughts

No car will ever face a neatly labelled trolley problem, yet the questions behind it are real. They ask what we value, whom we trust to decide and how we balance self interest against the common good. Vote, share your reasoning and please be kind to those who choose differently. Thoughtful disagreement is what makes these polls worthwhile


Further reading

• Philippa Foot, The Problem of Abortion and the Doctrine of the Double Effect, Oxford Review, 1967
• Edmond Awad and colleagues, The Moral Machine experiment, Nature, 2018
• Jean-Francois Bonnefon, Azim Shariff and Iyad Rahwan, The social dilemma of autonomous vehicles, Science, 2016
• Federal Ministry of Transport and Digital Infrastructure, Ethics Commission: Automated and Connected Driving, June 2017

[Brad92]

Swerving into a wall to save five people seems right in theory. In practice, I am not sure I would buy that car. The social dilemma describes me perfectly. It is uncomfortable to admit

Perhaps the rule should be set by law rather than by buyers. Then nobody gets to opt out. That would solve the dilemma neatly

Of course, governments may not want to touch it either. It is a hard question for anyone to own. Nobody wants to be blamed