Framework for Driver-Automation Integrated Systems
Total Page:16
File Type:pdf, Size:1020Kb
OPERATIONAL DESIGN DOMAIN (ODD) FRAMEWORK FOR DRIVER-AUTOMATION INTEGRATED SYSTEMS By HONGSEOK CHO S.M. AERONAUTICS AND ASTRONAUTICS MASSACHUSETTS INSTITUTE OF TECHNOLOGY, 2012 SUBMITTED TO THE DEPARTMENT OF AERONAUTICS AND ASTRONAUTICS IN PARTIAL FULLFILLMENT OF THE REQUREMENTS FOR THE DEGREE OF ENGINEER IN AERONAUTICS AND ASTRONAUTICS (EAA) AT THE MASSACHUSETTS INSTITUTE OF TECNOLOGY SEPTEMBER 2020 ©2020 Massachusetts Institute of Technology. All rights reserved. Signature of Author ……………………………………………………………………………....... Department of Aeronautics and Astronautics Aug. 18, 2020 Certified by ……………………………………………………………………………................... R. John Hansman. Jr. Professor of Aeronautics and Astronautics Thesis Supervisor Certified by ……………………………………………………………………………................... Julie A. Shah Associate Professor of Department of Aeronautics and Astronautics Co-Thesis Supervisor Accepted by …………………………………………………………………………….................. Zoltan Spakovszky Professor, Aeronautics and Astronautics Chair, Graduate Program Committee [Page Intentionally Left Blank] 2 Operational Design Domain (ODD) Framework for Driver-Automation Integrated Systems By HongSeok Cho Submitted to the Department of Aeronautics and Astronautics on August 18, 2020 in Partial Fulfillment of the Requirements for the Degree of Engineer in Aeronautics and Astronautics (EAA) Abstract Current driving automation systems have technical limitations which restricts their capability. Because of these limitations, the Society of Automotive Engineers (SAE) proposed the concept of the Operational Design Domain (ODD), which defines conditions under which a given driving automation system is designed to function. However, there is not yet a clear standard and a systematic process to evaluate an automation system to determine its ODD. In addition, inappropriate use of the automation outside the ODD may result in accidents. This thesis had following research objectives: 1) to develop a framework and methodology to define the ODD with a principled basis to determine the ODD of driving automation systems, and 2) to understand how the human operators make use decisions in order to support adequate management of the ODD. A risk-based framework was developed, leveraging on the traditional risk theory, to formally provide a threshold to define the ODD in terms of risk related to an automation system’s failure to perform its intended function. Based on this framework, the thesis developed a methodology to determine the ODD by identifying the key dimensions of the conditional hyperspace and the boundary that demarcates the sets of conditions with an acceptable level of risk. The method identifies the relevant conditions that becomes the dimensions of the conditional hyperspace in which the ODD boundary is determined. The methodology was applied to a simple forward collision avoidance system in order to demonstrate the use of the proposed framework. Once an ODD is determined for a driving automation system, it must be adequately managed so that the use of the automation outside the ODD is avoided. The key role of ODD management is to observe available information and assess whether the observed conditions are inside or outside the ODD. This role may be performed automatically by the system (for SAE Level 3 or 4 systems) or performed by the human operator (for SAE Level 1 or 2 systems). An analysis of recent accidents involving driving automation system’s failure and the literature on human cognition process were used to investigate how human operators perform ODD management. A model was developed and used to identify potential failure modes of human ODD management. These include: 1) inability to observe relevant states indicating an ODD violation, 2) failure to observe relevant states indicating an ODD violation and 3) ignorance of the ODD states resulting in an ODD violation. These failure categories were explored to identify potential causes including: lack of observability of the ODD conditions, drivers over-trust in automation, lack of understanding of the automation or ODD, and inaccurate projection of the future conditions. Based on this, a set of recommendations for improving the driver-automation integrated systems that would support to improve the ODD management are suggested. Thesis Supervisor: R. John Hansman. Jr. Title: Professor of Aeronautics and Astronautics Co-Thesis Supervisor: Julie A. Shah Title: Associate Professor of Department of Aeronautics and Astronautics 3 [Page Intentionally Left Blank] 4 Acknowledgements I owe much thanks to my advisors. Prof Hansman, thank you for the amount of time you spent providing guidance, support and feedback through this process. You were always available and were extremely patient with my struggles. Prof. Shah, thank you for the encouragement you provided during our meetings. I would like to thank Dr. Tijerina, Dr. Talamonti and Dr. Kurokawa from Ford for your guidance and valuable feedback through the entire time. I also want to thank Dr. Reimer and Prof. Stirling. Your encouragements and ideas made it possible for me put together the academic plans for me to return to MIT. I would like to thank the members of ICAT who provided me with mentorship, feedback, and encouragements throughout the process. To my family. Minji, thank you for your prayer, patience and understanding throughout this difficult and stressful time. Thank you for being a faithful and prudent wife and a loving mom for Ariel and Elliot. I could not have endured this time without your support. Ariel and Elliot, thank you for being my precious daughter and my adorable son. I thank my God, my Lord and my shepherd who guides me along the right paths for His name's sake. This research would not have been possible without funding from the Ford Motor Company. 5 [Page Intentionally Left Blank] 6 Outline 1. Introduction and Objectives …………………………………………………………......... 11 1.1. Research Motivation and Objectives ……………………………………………… 11 1.2. Thesis Overview …………………………………………………………………... 15 2. Risk-based ODD Determination Methodology …………………………………………... 17 2.1. Past Studies on the ODD Definition ………………………………………………. 17 2.2. ODD Conditional Hyperspace …………………………………………………….. 19 2.2.1. Hyperspace Dimension Determination ………………………………….. 20 2.2.2. Risk-based Threshold Determination ........................…………………… 21 2.3. Risk-Based ODD Determination Process …………………………………………. 22 2.3.1. Preliminary Hazard Identification ………………………………………... 23 2.3.2. Modeling of the Relevant Events ………………………………………… 24 2.3.3. Hazard Evaluation (Conditional Risk Assessment) ……………………… 26 2.3.4. ODD Determination ……………………………………………………… 27 2.4. Example Application of the Methodology ………………………………………… 28 2.4.1. System description ………………………………………………………. 28 2.4.2. Preliminary Hazard Identification ……………………………………..… 29 2.4.3. Modeling of the Relevant Events ………………………………………... 31 2.4.4. Hazard Evaluation (Conditional Risk Assessment) ……………………... 34 3. ODD Management …………………………………………………………………………. 39 3.1. ODD Management Overview ……………………………………………………... 39 3.1.1. Automated ODD Management ………………………………………….. 41 3.1.2. Shared ODD Management ………………………………………………. 42 7 3.1.3 Human ODD Management ………………………………………………. 43 3.2. Historical Accident Analysis …………………………………………………….... 45 3.2.1. Analysis Overview ……………………………………………………….. 46 3.2.2. Review of the Involved Driving Automation Systems …………………… 47 3.2.3. Analysis Result …………………………………..………………………. 49 4.2.4. Analysis Summary …………………………………..…………………… 53 3.3. Human ODD Management …………………………………..…………………….. 54 3.4. Human ODD Management Failures ………………………………………………. 56 3.4.1. Not Observable ODD Violation: Failure of Perception ………………… 57 3.4.2. Ignored ODD Violation: Failure of Comprehension ……………………... 59 3.4.3. Not Observed ODD Violation: Failure of Projection …………………… 61 4. Recommendation to Support Improved ODD Management ……………………………. 65 5. Summary and Conclusion …………………………………..……………………………... 73 Reference …………………………………..…………………………………..………....…… 77 8 Tables and Figures List of Tables Table 1-1: SAE Levels of Driving Automation ………………………………………………... 12 Table 2-1: Identified Relevant Conditions ……………………………………………………... 30 Table 2-2: Assumed Occurrence Rate of Object Presence for Different Road Types …………. 32 Table 2-3: Assumed Detection Distance for Different Obstacle Types, Weather and Light Conditions………………………………………………………………………………………. 32 Table 3-1: ODD Management Responsibility and the SAE Levels of Driving Automation …... 41 Table 3-2: Identified Accidents Related to Driving Automation System ……………………… 46 Table 3-3: The Assumed ODD and the ODD Violation Conditions of the Tesla Autopilot …... 48 Table 3-4: Summary of the Results ……………………………………………………………. 49 Table 3-5: Record of the Hands on the Steering Wheel ……………………………………….. 52 Table 3-6: Possible Failures of the Human ODD Management in the Accidents ……………... 53 List of Figures Figure 1-1. ODD Management Feedback Model ……………………………………………... 14 Figure 2-1. Layer Model Representing Driving Scene (Schuldt, 2017) ………………………. 18 Figure 2-2: Illustration of the Conditional Hyperspace and the ODD Boundary ………………19 Figure 2-3: A Risk Matrix with the Aviation Certification Standard for Small Aircraft (ASTM 3230) …………………………………………………………………………………………... 22 Figure 2-4: Overall Risk-Based ODD Determination Methodology …………………………. 23 Figure 2-5: Illustration of a Causal-Chain and Relevant Condition for a Hazard …………….. 24 Figure 2-6: Conditional Risk Framework (Illustration) ………………………………………. 26 Figure 2-7: ODD Determination Process (Illustration)