Measuring Automated Vehicle Safety Forging a Framework Laura Fraade-Blanar, Marjory S. Blumenthal, James M. Anderson, Nidhi Kalra C O R P O R A T I O N For more information on this publication, visit www.rand.org/t/RR2662 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-1-9774-0164-9 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2018 RAND Corporation R® is a registered trademark. Cover: Courtesy of Andrey Suslov/iStock (Getty Images). Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface The safety of automated vehicles (AVs) is intrinsic to their success both in the marketplace and as the kind of transformative innovation that their proponents anticipate. In the summer of 2017, the Uber Advanced Technologies Group approached the RAND Corporation to request help in crafting a framework for measuring AV safety that could aid in public discussion of the issues. This project builds on previous RAND research into AV trends and related public policy. Whereas prior work has addressed issues broadly and analytically, this project has looked more closely at what companies that are key to the evolution of AVs have been doing to foster and evaluate the safety of those vehicles. The report is intended for a broad public audience. In this report, we develop a framework for measuring safety in AVs that could be used broadly by companies, policymakers, and the public. We considered how to define safety for AVs, how to measure safety for AVs, and how to communicate what is learned or understood about AVs. Given AVs’ limited total on-road mileage compared with conventional vehicles, we consider options for proxy measurements—i.e., factors that might be correlated with safety. We also explore how safety measurements could be made in simulation and on closed courses. The closely held nature of AV data limits the details of what is made public or shared between companies and with the government. The report focuses on identifying key concepts and illuminating the kinds of measurements that might be made and communicated. The research reported here was conducted in two programs. The RAND Science, Technology, and Policy program focuses primarily on the role of scientific development and technological innovation in human behavior, global and regional decisionmaking as it relates to science and technology, and the concurrent effects that science and technology have on policy analysis and policy choices. The program covers such topics as space exploration, information and telecommunication technologies, and nano- and biotechnologies. The RAND Justice Policy Program spans both criminal and civil justice system issues with such topics as public safety, effective policing, police–community relations, drug policy and enforcement, corrections policy, use of technology in law enforcement, tort reform, catastrophe and mass-injury compensation, court resourcing, and insurance regulation. Research in both programs is supported by government agencies, foundations, and the private sector. RAND Justice, Infrastructure, and Environment (JIE) conducts research and analysis in civil and criminal justice, infrastructure development and financing, environmental policy, transportation planning and technology, immigration and border protection, public and occupational safety, energy policy, science and innovation policy, space, telecommunications, and trends and implications of artificial intelligence and other computational technologies. Questions or comments about this report should be sent to the project leader, Marjory S. Blumenthal ([email protected]). For more information about RAND Science, iii Technology, and Policy, see www.rand.org/jie/stp or contact the director at [email protected]. For more information about RAND Justice Policy, see www.rand.org/jie/justice-policy or contact the director at [email protected]. iv Contents Preface ........................................................................................................................................... iii Boxes, Figures, and Tables ........................................................................................................... vii Summary ...................................................................................................................................... viii Acknowledgments ....................................................................................................................... xiii Abbreviations ............................................................................................................................... xiv 1. Introduction ................................................................................................................................. 1 Context for Contrasting AV and Conventional Automotive Safety ......................................................... 1 Measurement Framework Focuses on System and Ecosystem Levels ..................................................... 2 Scope, Approach, and Limitations of This Report .................................................................................... 4 2. Safety ........................................................................................................................................... 6 Foundational Safety Concepts ................................................................................................................... 6 Safety Engineering Meets AVs ................................................................................................................. 9 Federal Motor Vehicle Safety Standards ................................................................................................ 11 Special Aspects of Level 4 AVs .............................................................................................................. 12 3. Measuring Automated Vehicle Safety ....................................................................................... 14 Frame 1. Settings ..................................................................................................................................... 14 Artificial Setting .................................................................................................................................. 15 Public Road Setting With and Without a Safety Driver ..................................................................... 17 A Discussion of the Setting Frame ...................................................................................................... 18 Frame 2. Stages ....................................................................................................................................... 21 Development ....................................................................................................................................... 22 Demonstration ..................................................................................................................................... 23 Deployment ......................................................................................................................................... 27 A Discussion of the Stage Frame ........................................................................................................ 28 Frame 3. Measures .................................................................................................................................. 28 Measure Category 1: Standards, Processes, Procedures, and Design ................................................. 29 Measure Category 2: Leading Measures ............................................................................................. 29 Measure Category 3: Lagging Measure .............................................................................................. 34 Measure Characteristics ...................................................................................................................... 34 A Discussion of the Measure Frame ................................................................................................... 35 From Frames to Framework .................................................................................................................... 36 Measures to Metrics ................................................................................................................................ 40 Numerators and Denominators ..........................................................................................................