Analysing Safety-Critical Systems and Security Protocols with Abstract State Machines Thesis submitted in accordance with the requirements of the University of Liverpool for the degree of Doctor in Philosophy by Farah Al-Shareefi September 2019 Dedication To my beloved family, especially my husband and sons. i List of Abbreviations ASM Abstract State Machines ASM-STPA-SA Abstract State Machines-System Theoretic Process Analysis- Safe Design AsmetaL ASMETA Language AsmetaS ASMETA Simulator AsmetaSMV ASMETA SMV AsmetaV ASMETA Validator DoS REPL Denial of Service REPLay INTRL INTeRLeaving IPCS Insulin Pump Control System MITM Man In The Middle REFL REFLection SASL Simple Authentication and Security Layer Simple REPL Simple REPLay STPA System Theoretic Process Analysis TDC Train Door Controller iii Abstract The research presented in this thesis is directed at the analysis of critical systems and protocols and the improvement of their safety and security aspects respectively, by com- bining formal and informal analysis methods. More specifically, we focus at combining Abstract State Machines (ASMs) as a formal method, with System Theoretic Process Analysis (STPA) as a safety analysis technique, with the aims of developing safer systems and more secure protocols. The ASM method was chosen due to both its generality in specifying any system at a convenient level of abstraction, and its specification which is supported by different formal analysis activities. While the reason for choosing STPA was its capability of eliciting safety requirements originated from inadequate control actions which affect the whole system functions. The first contribution of this thesis is a methodology to analyse safety-critical systems by capturing both the formal representation of ASM and the safety requirements generated by the STPA. This has the advantages of verifying the STPA requirements in a formal way and giving insights to improve the ASM specification, depending on these requirements. We illustrated this methodology by applying it to a train door controller and an insulin pump control system case studies, showing what safety issues it highlighted. The second contribution of the work presented in this thesis is a systematic methodology for analysing security protocols. This methodology was intended to provide a link between the formal simulation of external attack scenarios and protocol under analysis specified by the ASM method, and the analysis outcomes of a proposed technique called FATI method. The FATI (Flaws and Attack Types Identification) method is an inspired form of STPA that applies queries on each protocol action to determine the possible protocol flaws and their expected attack types. The identified attack types help to select the attack scenario specifications whose simulations are likely to produce attacks. Our methodology also minimized the number of the simulated protocol runs by reducing the number of intruder's messages through considering the receiver's expectations about the message type format and the content. Furthermore, we showed how to analyse protocols in the presence of an algebraic property for the commutative encryption. Our methodology for analysing security protocols was applied to several protocols. Moreover, within the security protocols area, we clarified the ambiguous requirements for simple authentication and security layer example depending on the ASM method. Acknowledgements My great full thanks to God, the Most Merciful and the Most Gracious, for giving me the strength and patience to overcome the challenges in my work and life while conducting my Ph.D study. I would like to express my deepest gratitude to both supervisors Dr. Alexei Lisitsa and Dr. Clare Dixon for their ongoing support, patience, constructive criticism and encourage- ment throughout my Ph.D. journey. Their continuous guidance and valuable comments have made the completion of my Ph.D. possible and enjoyable. It has been a great privilege to have worked with you both. I would also like to extend my deepest gratitude to the Higher Committee for Education Development in Iraq (HCED) for their generous financial support that has given me the opportunity to complete my Ph.D. study. I am also thankful to my advisors Prof. Sven Schewe and Prof. Boris Konev for their advice and helpful comments. Thanks also to all staff members and colleagues in the Department of Computer Science at the University of Liverpool who have been helpful whenever necessary. A special thanks go to my family: father, mother, father-in-law, mother-in-law, sisters, brother, sister-in-law, and brothers-in-law, for their prayers, support, and encouragement. Last but not least, my warmest thanks go to my sons and my husband who have given me constant love during the completion of the thesis. I apologize for every moment I missed to care about you all or to share your ups and downs. My husband, you are fabulous. I would never be able to get where I am without you and your help. Please accept my deepest thanks. vii Contents Dedication i List of Abbreviations iii Abstract v Acknowledgements vii Contents xii List of Figures xiv List of Tables xvi 1 Introduction 1 1.1 Safety-Critical Systems . .1 1.2 Security Protocols . .3 1.3 Research Questions and Contributions . .6 1.4 Publications . .8 1.5 Thesis Outline . .9 2 Abstract State Machines Method 11 2.1 Introduction . 11 2.2 Abstract State Machines Theory . 11 2.3 Basic Abstract State Machines . 12 2.3.1 Simple Example . 13 2.3.2 Terminology Used in Abstract State Machines . 14 2.3.3 Ground Model and Stepwise Refinement . 21 2.4 The Abstract State Machines Tools . 23 2.4.1 The Dynamic Algebra Specification Language . 23 2.4.2 The ASM Workbench . 24 2.4.3 The ASM Gofer . 25 ix 2.4.4 Extensible Abstract State Machines . 26 2.4.5 Abstract State Machine Language . 27 2.4.6 The Timed ASM Language and Toolset . 28 2.4.7 Simulator for Real-Time ASMs . 30 2.4.8 Core Abstract State Machine . 31 2.4.9 The ASM mETAmodelling Framework . 32 2.5 Why Abstract State Machines . 43 2.6 Conclusion . 45 3 Safety and Security Analysis Techniques 46 3.1 Introduction . 46 3.2 Software System Development Process . 47 3.3 Safety Analysis Techniques . 48 3.3.1 Fault Tree Analysis . 49 3.3.2 HAZards and OPerability Analysis . 52 3.3.3 System Theoretic Process Analysis . 54 3.4 Evaluation of Safety Analysis Techniques . 60 3.5 Security Analysis Techniques . 63 3.5.1 Attack Tree . 63 3.5.2 Vulnerability Identification and Analysis . 64 3.5.3 Requirements Analysis and Elicitation . 66 3.5.4 System Theoretic Process Analysis for Security . 67 3.6 Evaluation of Security Analysis Techniques . 70 3.7 Conclusion . 73 4 Security Protocols Background 74 4.1 Introduction . 74 4.2 Cryptographic Primitives . 75 4.3 Algebraic Properties of Cryptographic Primitives . 77 4.4 Protocol Notation . 78 4.5 Protocol Attacks and Attack Scenarios . 79 4.6 Protocol Examples . 82 4.6.1 Needham-Schroder Public Key Protocol . 83 4.6.2 Needham-Schroder Symmetric Key Protocol . 83 4.6.3 Denning-Sacco Protocol . 84 4.6.4 Three-Pass (TP) Protocol . 85 4.6.5 Andrew Secure Remote Procedure Call Protocol . 86 4.6.6 Kehne Langendorfer Schoenwalder Protocol . 87 4.7 Protocol Analysis Methods and Tools . 89 4.7.1 BAN logic . 91 4.7.2 GNY logic . 96 4.7.3 Casper/FDR . 97 4.7.4 SPIN . 98 4.7.5 NuSMV . 99 4.7.6 ProVerif . 100 4.7.7 Tamarin . 102 4.7.8 Simulation Based Attack Scenarios . 104 4.8 Conclusion . 104 5 Safe Design Development Methodology for Safety-Critical Systems 107 5.1 Introduction . 107 5.2 The Proposed Methodology for Developing Safe Design . 108 5.3 Case Study: The Train Door Controller System . 112 5.3.1 Modelling the system Behavior via AsmetaL and Simulating the Re- sultant Model . 113 5.3.2 Validating the AsmetaL Model . 114 5.3.3 Eliciting safety requirements for the system via STPA . 116 5.3.4 Formalizing the Elicited STPA Safety Requirements . 118 5.3.5 Verifying the Formulated Safety Requirements . 119 5.3.6 Evaluation of the TDC Case Study . 120 5.4 Case Study: The Insulin Pump Control System . 120 5.4.1 The IPCS Case Study Description . 121 5.4.2 Methodology Applied to IPCS Case Study . 123 5.5 Evaluation . 142 5.6 Related Work . 144 5.7 Conclusion . 145 6 Security Protocols Analysis Methodology 147 6.1 Introduction . 147 6.2 Flaws and Attack Types Identification Method . 149 6.2.1 Application of the FATI Method to the AS RPC Protocol . 152 6.2.2 Application of the FATI Method to the NSPK Protocol . 153 6.3 The proposed Methodology for Analysing Security Protocols . 154 6.3.1 The Protocol Aspect . 157 6.3.2 The Intruder Aspect . 163 6.3.3 The Attack Scenarios Aspect . 166 6.3.4 The Invariant Security Properties Aspect . 173 6.4 Results and Discussion . 174 6.4.1 Manual Analysis Phase . 175 6.4.2 Automatic Analysis Phase . 175 6.5 Related Work . 177 6.6 Conclusion . 178 7 Clarifying Ambiguous Requirements: SASL Case Study 179 7.1 Introduction . 179 7.2 Simple Authentication and Security Layer . 180 7.3 The Formal SASL Specification . 182 7.3.1 The Mechanism Negotiation Phase . 182 7.3.2 The Authentication Negotiation Phase . 186 7.3.3 The Security Layer Negotiation Phase . ..