Autonomous Vehicles Ethics &
Total Page:16
File Type:pdf, Size:1020Kb
RYAN JENKINS AUTONOMOUS VEHICLES ETHICS & LAW Toward an Overlapping Consensus SEPTEMBER 2016 About the Authors About New America Ryan Jenkins studies the moral New America is committed to renewing American politics, dimensions of technologies with the prosperity, and purpose in the Digital Age. We generate big potential to profoundly impact human ideas, bridge the gap between technology and policy, and life. He is an assistant professor of curate broad public conversation. We combine the best of philosophy and a senior fellow at the a policy research institute, technology laboratory, public Ethics & Emerging Sciences Group at California forum, media platform, and a venture capital fund for Polytechnic State University, San Luis Obispo. His interests ideas. We are a distinctive community of thinkers, writers, include driverless cars, algorithms, autonomous weapons, researchers, technologists, and community activists who and military ethics more broadly. His work has appeared believe deeply in the possibility of American renewal. publicly in Forbes, Slate and elsewhere, and he is currently co-editing two books on military ethics and robot ethics, Find out more at newamerica.org/our-story. both for Oxford University Press. At Cal Poly, Jenkins teaches courses in ethics, political philosophy, and the philosophy of technology, among others. He earned his About the Digital Industries Initiative B.A. in philosophy from Florida State University, Phi Beta Kappa, and his Ph.D. in philosophy from the University of The Digital Industries Initiative of New America brings Colorado Boulder. together leading experts and policymakers from the private sector, government, universities, and other nonprofit institutions and the media, to analyze Acknowledgments and debate the future of America’s major economic sectors in the Digital Age. Each month the invitation- A special thanks is due to Colin McCormick and New only Digital Industries Roundtable, hosted at New America fellow Levi Tillemann for comments on an America’s headquarters in Washington, D.C., features a earlier draft. discussion of the challenges of innovation in a different American industry. In addition, the Digital Industries Initiative publishes groundbreaking reports, hosts public Cover image: DimiTVP/Wikimedia. events, and undertakes other activities at the dynamic intersection of technology, economics, and public policy. Contents Abstract 2 Introduction 3 Crash Optimization 4 Overlapping Consensus: Ethics as an Engineering Problem 6 Proposals for Crash Optimization 10 Adjustable Ethics Settings 14 Insuring Autonomous Vehicles 16 Handing off to the Driver 17 Abuse 19 Far-term Issues 21 Next Steps 22 Works Cited 24 Notes 26 ABSTRACT There is a clear presumptive case for the adoption This report was inspired by the Autonomous of autonomous vehicles (AV). It is widely believed Vehicles & Ethics Workshop held at Stanford they will be safer than human-driven vehicles, University in Palo Alto, California in September of better able to detect and avoid hazards and 2015. The workshop was a closed, invitation-only collisions with other drivers and pedestrians. meeting of about 30 participants. Participants However, it would be unreasonable to expect included academics, including ethicists, AV to be perfect. And unlike programming for psychologists, roboticists and mechanical much software and hardware, the conditions AV engineers; insurance lawyers and legal experts; can be expected to face on the road are an “open and representatives from the automotive industry set”: we cannot exhaustively test every scenario, and Silicon Valley. The conference was organized since we cannot predict every possible scenario. by Patrick Lin (California Polytechnic State In light of this, we must think carefully about University), Selina Pan (Stanford), and Chris Gerdes what requirements manufacturers should have (Stanford), and was supported by funding from the to demonstrate before AV are allowed on the US National Science Foundation, under award no. roads. This paper surveys the practical state of 1522240. The meeting was conducted under The the art, technical limitations of AV, the problem of Chatham House Rule, whereby participants are driver handoff, and the possibility of abuse with free to use the information received, but neither AV, such as other drivers playing “chicken” with the identity nor the affiliation of the speaker(s) may AV. It considers AV from the legal, ethical, and be revealed without their expressed consent. This manufacturing perspectives before arguing for report includes input and observations from the an “overlapping consensus”: AV that behave in workshop’s participants. Its interpretations of those ways that are morally justified, legally defensible, remarks, substantive claims and recommendations, and technically possible. The paper closes by however, solely reflect the syntheses of its author, applying this lens to some possible ways that and do not necessarily reflect the views of the AV could behave in the event of a crash, offering workshop’s participants, organizers, or supporting tentative endorsements of some of these, and organizations. A special thanks is due to Colin recommending a closer collaboration between McCormick and New America fellow Levi Tillemann industry and the academy. for comments on an earlier draft. 2 DIGITAL INDUSTRIES INITIATIVE INTRODUCTION There is a clear presumptive case for the adoption Motors, 2016). And unlike programming for much of autonomous vehicles (AV). It is widely believed software and hardware, the set of conditions AV they will be safer than vehicles driven by humans. can be expected to face on the road is an “open For example, AV will not become sleepy, distracted, set”: manufacturers cannot exhaustively test every or angry behind the wheel, and will be better scenario, since they cannot predict every possible able to detect and avoid hazards and collisions scenario. Manufacturers will be unable to ensure with other drivers and pedestrians. Because car that AV are totally prepared to drive on their own, in accidents kill around 30,000–35,000 people per all conditions and situations. year in the United States alone1, and because around 94% of crashes are due to driver error2, the In light of this, stakeholders must think carefully case for AV from increased safety and lives saved is about what requirements should be met before extremely compelling. AV are allowed on the roads. What kind of discrimination capabilities should AV have before Even mildly optimistic predictions concerning AV it’s permissible to deploy them? Is it merely show that they could provide significant benefits enough that AV be superior to human drivers? in terms of social costs of death or injury, as well as How should AV be programmed to behave in the increased convenience and productivity time for the event of a crash, and is it permissible for them individual consumer. to change the outcome of a crash by redirecting harm? Or should we be worried about people who However, it would be unreasonable to expect AV are killed or injured by AV when they would not to be perfect. Software and hardware undergo have been otherwise? These and other issues are continuous development, and their failures are explored below, synthesizing the perspectives of sometimes catastrophic. Since there is nothing philosophers, lawyers, and manufacturers, in search intrinsically different about the software and of an overlapping consensus on the development hardware to be used in AV, the same possibility and deployment of autonomous vehicles. for catastrophic failure exists—witness the failure of Tesla’s autopilot system in May, 2016 (Tesla DIGITAL INDUSTRIES INITIATIVE Autonomous Vehicles Ethics & Law: Toward an Overlapping Consensus 3 CRASH OPTIMIZATION Sensing the World implemented, behavior is continually reinforced or modified by subtle human nudges (Fehrenbacher, Autonomous vehicles use a variety of technologies 2015), and can be fine-tuned with hard-coded to detect the outside world, make sense of it, and inputs, for example, about the distance to keep make decisions based on that data. Manufacturers between cars, the position to maintain in a lane, like Google and Tesla take different approaches how long to wait after a stoplight turns green, and to which sensors they use, how data from those so on. With this suite of technologies, AV have sensors is synthesized, and how AV make decisions already been shown to be able to reliably navigate based on the data. A typical suite of technologies roads in some situations. for sensing the world includes cameras, ultrasonic sensors (SONAR), RADAR, or LIDAR (light detection and ranging). For example, Tesla prefers to use a “No-win” Scenarios combination of cameras and RADAR over Google’s LIDAR because LIDAR instruments are orders of Autonomous vehicles presumptively bring magnitude more expensive, even though they significant benefits: their increased computational offer a higher resolution representation of an AV’s power and reaction time allow them to avoid some surroundings.3 Moreover, cameras are vulnerable to accidents that a human driver could not. A clear some of the same failings as the human eye: they primary goal for AV should be to avoid all collisions. have difficulty seeing in direct sunlight, low-light If AV could successfully avoid most or all accidents conditions, or inclement weather. that are caused by driver error, this could be expected to eliminate tens of thousands of deaths AV take in this information and