Safety Monitoring for Autonomous Systems : Interactive Elicitation of Safety Rules Lola Masson
Total Page:16
File Type:pdf, Size:1020Kb
Safety monitoring for autonomous systems : interactive elicitation of safety rules Lola Masson To cite this version: Lola Masson. Safety monitoring for autonomous systems : interactive elicitation of safety rules. Per- formance [cs.PF]. Université Paul Sabatier - Toulouse III, 2019. English. NNT : 2019TOU30220. tel-02098246v2 HAL Id: tel-02098246 https://tel.archives-ouvertes.fr/tel-02098246v2 Submitted on 11 Sep 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THTHESEESE`` En vue de l’obtention du DOCTORAT DE L’UNIVERSITEF´ ED´ ERALE´ TOULOUSE MIDI-PYREN´ EES´ D´elivr´e par : l’Universit´eToulouse 3 Paul Sabatier (UT3 Paul Sabatier) Pr´esent´ee et soutenue le 21/02/2019 par : Lola MASSON SAFETY MONITORING FOR AUTONOMOUS SYSTEMS: INTERACTIVE ELICITATION OF SAFETY RULES JURY Simon COLLART-DUTILLEUL RD, IFSTTAR, France President of the Jury Elena TROUBITSYNA Ass. Prof., KTH, Sweden Reviewer Charles PECHEUR Prof., UCL, Belgium Reviewer Jean-Paul BLANQUART Eng., Airbus D&S, France Member of the Jury Hel´ ene` WAESELYNCK RD, LAAS-CNRS, France Supervisor Jer´ emie´ GUIOCHET MCF HDR, LAAS-CNRS, France Supervisor Ecole´ doctorale et sp´ecialit´e : EDSYS : Informatique 4200018 Unit´e de Recherche : Laboratoire d’analyse et d’architecture des syst`emes Acknowledgments This work has been done at the Laboratory for Analysis and Architecture of Systems, of the French National Center for Scientific Research (LAAS-CNRS). I wish to thank Jean Arlat and Liviu Nicu, the two successive directors of the laboratory, for welcoming me. During the three years of my PhD, I worked within the Dependable Computing and Fault Tolerance team (TSF), which is lead by Mohamed Kaˆaniche. I thank him for the excellent work environment he has built, allowing everyone to work however it is best for them. Our work focuses on autonomous systems, and the company Sterela provided us with a very interesting case study, the robot 4mob. I thank Augustin Desfosses and Marc Laval, from Sterela, who welcomed me in their offices and worked with us on this robot. In 2017, I was lucky enough to spend three months at KTH, Sweden, in the Mechatronics division. I thank Martin T¨orngren,who welcomed me in the lab, and with whom I appreciated meeting while walking in the forest around the campus. I also want to thank Sofia Cassel and Lars Svensson, with whom I had very enjoyable work sessions, as well as all the other PhD students and post-docs who made my stay a great both work and personal experience. I thank Elena Troubitsyna, from KTH, Sweden, and Charles Pecheur, from UCLouvain, Belgium, for reviewing this thesis. I also thank to Simon Collart-Dutilleul, from IFSTTAR, France, for presiding on my thesis jury. Of course, I don’t forget J´er´emieGuiochet and H´el`eneWaeselynck, from LAAS- CNRS, who supervised this work and accompanied me through the good times as well as the hard ones. Thanks also to Jean-Paul Blanquart from Airbus Defense and Space and to Kalou Cabrera, post-doc at LAAS at the time, for participating in the meetings and for their precious help and advice. The work days of the last three years would have been different if I were not among the PhD students and post-docs of the TSF team. Thanks to those I shared an office with, providing much-needed support on the hard days and sharing laughs on the good ones. Thank to all for the never-ending card games. Special thanks to Jonathan and Christophe for the memorable nights debating, and for their open-mindness. To all, it was a pleasure hanging out and sharing ideas and thoughts with you. Thanks to the one who is by my side every day, for his support in every circumstance. Thanks to my crew who has always been there, to my family and to all those who make me laugh and grow. Contents Introduction 13 1 Ensuring the Dependability of Autonomous Systems: A Focus on Mon- itoring 17 1.1 Concepts and Techniques for Safe Autonomous Systems . 18 1.1.1 Fault Prevention . 18 1.1.2 Fault Removal . 20 1.1.3 Fault Forecasting . 22 1.1.4 Fault Tolerance . 24 1.2 Monitoring Techniques . 26 1.2.1 Runtime Verification . 27 1.2.2 Reactive Monitors for Autonomous Systems . 29 1.3 SMOF: Concepts and Tooling . 32 1.3.1 SMOF Process Overview . 32 1.3.2 SMOF Concepts and Baseline . 33 1.3.3 Safety and Permissiveness Properties . 34 1.3.4 SMOF Tooling . 34 1.4 Conclusion . 35 2 Feedback from the Application of SMOF on Case Studies 37 2.1 Sterela Case Study . 38 2.1.1 System Overview . 38 2.1.2 Hazop-UML Analysis . 39 2.1.3 Modeling and Synthesis of Strategies for SI1, SI2 and SI3 ..... 44 2.1.4 Modeling and Synthesis of Strategies for SI4 ............. 51 2.1.5 Rules Consistency . 59 2.2 Feedback from Kuka Case Study . 61 2.2.1 System Overview . 61 2.2.2 SII: Arm Extension with Moving Platform . 61 2.2.3 SIII: Tilted Gripped Box . 64 2.3 Lessons Learned . 65 2.3.1 Encountered Problems . 65 2.3.2 Implemented Solutions . 68 2.4 Conclusion . 69 3 Identified Problems and Overview of the Contributions 71 3.1 Modeling of the Identified Problems . 72 3.1.1 Invariant and Strategy Models . 72 3.1.2 Properties . 73 6 Contents 3.1.3 Identified Problems . 73 3.2 Manual Solutions to the Identified Problems . 74 3.2.1 Change the Observations . 75 3.2.2 Change the Interventions . 75 3.2.3 Change the Safety Requirements . 76 3.2.4 Change the Permissiveness Requirements . 76 3.3 High Level View of the Contributions . 77 3.3.1 Diagnosis . 77 3.3.2 Tuning the Permissiveness . 78 3.3.3 Suggestion of Interventions . 80 3.4 Extension of SMOF Process and Modularity . 81 3.4.1 Typical Process . 81 3.4.2 Flexibility . 83 3.5 Conclusion . 83 4 Diagnosing the Permissiveness Deficiencies of a Strategy 85 4.1 Preliminaries - Readability of the Strategies . 86 4.1.1 Initial Display of the Strategies . 86 4.1.2 Simplifications with z3 . 87 4.2 Concepts and Definitions . 88 4.2.1 Strategies and Permissiveness Properties . 88 4.2.2 Problem: Safe & No perm. 89 4.2.3 Permissiveness Deficiencies of a Strategy . 91 4.3 Diagnosis Algorithm and Implementation . 93 4.3.1 Algorithm . 94 4.3.2 Simplification of the Diagnosis Results . 97 4.3.3 Implementation . 100 4.4 Application to Examples . 101 4.4.1 SII: The Arm Must Not Be Extended When The Platform Moves At A Speed Higher Than speedmax.................. 101 4.4.2 SI4: The Robot Must Not Collide With An Obstacle. 102 4.5 Conclusion . 104 5 Tuning the Permissiveness 105 5.1 Defining Custom Permissiveness Properties . 106 5.1.1 From Generic to Custom Permissiveness . 106 5.1.2 A Formal Model for the Functionalities . 107 5.1.3 Binding Together Invariants and Permissiveness . 108 5.2 Restricting Functionalities . 110 5.2.1 Diagnosing the Permissiveness Deficiencies with the Custom Prop- erties . 110 5.2.2 Weakening the Permissiveness Property . 111 5.2.3 Automatic Generation and Restriction of Permissiveness Properties112 5.3 Application on Examples . 113 Contents 7 5.3.1 SII: The Arm Must Not Be Extended When The Platform Moves At A Speed Higher Than speedmax.................. 113 5.3.2 SIII: A Gripped Box Must Not Be Tilted More Than α0...... 116 5.3.3 SI3: The Robot Must Not Enter A Prohibited Zone. 117 5.4 Conclusion . 118 6 Suggestion of Candidate Safety Interventions 121 6.1 Preconditions and Effects of Interventions . 121 6.2 Identifying Candidate Interventions . 123 6.2.1 Magical Interventions . 123 6.2.2 Generalize the Interventions Effects . 125 6.2.3 Interactive Review of the Interventions . 126 6.3 Algorithm . 128 6.4 Application to an Example . 133 6.5 Conclusion . 136 Conclusion 137 Contributions . 137 Limitations . 138 Perspectives . 139 List of Figures 1.1 Concept of supervisory control synthesis . 20 1.2 Monitors in a hierarchical architecture . 24 1.3 Typical process of runtime verification (adapted from [Falcone et al., 2013]) 27 1.4 SMOF process . 32 1.5 System state space from the perspective of the monitor . 33 2.1 Sterela robot measuring runway lamps . 39 2.2 HAZOP-UML overview . 39 2.3 UML Sequence Diagram for a light measurement mission . 41 2.4 Behavior for the invariant SI1.......................... 46 2.5 SI3 the robot must not enter a prohibited zone . 49 2.6 Expected strategy for SI3 ........................... 50 2.7 Disposition of the lasers on the robot . 52 2.8 Visual representation of the classes of (x,y) coordinates . 53 2.9 Obstacle occupation zones around the robot . 56 2.10 Declaration of the zones variables and their dependencies . 58 2.11 Merging partitions into a global variable . 60 2.12 Manipulator robot from Kuka . 61 2.13 Strategy for the invariant SII with the braking intervention only . 62 2.14 Strategy for the invariant SII with the two interventions . 63 2.15 Positions of the gripper and corresponding discrete values . 64 3.1 Identified problems, proposed solutions .