Model-based Evaluation: from Dependability Theory to Security by Saad Saleh Alaboodi A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philosophy in Electrical and Computer Engineering Waterloo, Ontario, Canada, 2013 © Saad Saleh Alaboodi 2013 Author’s Declaration I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. ii Abstract How to quantify security is a classic question in the security community that until today has had no plausible answer. Unfortunately, current security evaluation models are often either quantitative but too specific (i.e., applicability is limited), or comprehensive (i.e., system-level) but qualitative. The importance of quantifying security cannot be overstated, but doing so is difficult and complex, for many reason: the “physics” of the amount of security is ambiguous; the operational state is defined by two confronting parties; protecting and breaking systems is a cross-disciplinary mechanism; security is achieved by comparable security strength and breakable by the weakest link; and the human factor is unavoidable, among others. Thus, security engineers face great challenges in defending the principles of information security and privacy. This thesis addresses model-based system-level security quantification and argues that properly addressing the quantification problem of security first requires a paradigm shift in security modeling, addressing the problem at the abstraction level of what defines a computing system and failure model, before any system-level analysis can be established. Consequently, we present a candidate computing systems abstraction and failure model, then propose two failure-centric model-based quantification approaches, each including a bounding system model, performance measures, and evaluation techniques. The first approach addresses the problem considering the set of controls. To bound and build the logical network of a security system, we extend our original work on the Information Security Maturity Model (ISMM) with Reliability Block Diagrams (RBDs), state vectors, and structure functions from reliability engineering. We then present two different groups of evaluation methods. The first mainly addresses binary systems, by extending minimal path sets, minimal cut sets, and reliability analysis based on both random events and random variables. The second group addresses multi-state security systems with multiple performance measures, by extending Multi-state Systems (MSSs) representation and the Universal Generating Function (UGF) method. The second approach addresses the quantification problem when the two sets of a computing system, i.e., assets and controls, are considered. We adopt a graph-theoretic approach using Bayesian Networks (BNs) to build an asset-control graph as the candidate bounding system model, then demonstrate its application in a novel risk assessment method with various diagnosis and prediction inferences. This work, however, is multidisciplinary, involving foundations from many fields, including security engineering; maturity models; dependability theory, particularly reliability engineering; graph theory, particularly BNs; and probability and stochastic models. iii Acknowledgements All praise be to Allah, the Creator and Sustainer of this universe. To begin, I would like to express my sincere thanks to my supervisor, Prof. Gordon Agnew. The freedom he gave me as a graduate student to take ownership of my own research interest along with the valuable guidance, support, and motivation was vital to achieving this degree. It made it a unique and outstanding experience, and I am really appreciative. I would also like to thank my thesis committee: the external examiner, Prof. Dawn Jutla, and internal examiners, Prof. Alfred Menezes, Prof. Sagar Naik, and Prof. Catherine Gebotys, for devoting valuable time to reviewing and commenting on this work. I would also like to thank all my dear friends and colleagues who have contributed to making my study experience exceptional, beyond expectation, and very enjoyable. A special thank you is inevitably due to my friend Dr. Abdulaziz Alkhoraidly for the insightful discussions and engaging companionship during his time studying here. I am also thankful to Ms. Mary McPherson of Graduate Writing Services for the exceptional applied writing skills. All of my achievements are owed to my family’s sacrifices, and so is my deepest and greatest gratitude. My father, may Allah have mercy on him, has always been in my memory at every step I have taken; my mother, may Allah grant her long and happy life, remains my lifelong pillar and inspiration; I owe my parents the whole world; my wife, my partner in this sojourn, has been an invaluable source of much-needed support and encouragement; my children continue to be the truest joy in my world; and my siblings have consistently stood behind me and boosted my moral throughout the course of this journey. I am sincerely grateful and indebted to each one of you as I could not have gotten where I am today without you all. Finally, I am also grateful to the Ministry of Higher Education in Saudi Arabia and King Saud University (KSU) for granting me the graduate program scholarship to achieve this work. iv Dedication In honor and memory of my father, Saleh, may Allah have mercy on him; to my mother, Munirah; to my wife, Najla; to my children, Faisal and Shahinaz; and to my brothers, Mohammed, Abdulaziz, Abdullah, and Bader, and my sisters, Fawziah, Norah, Shekhah, Ghadah, Bedoor, Arwa, and Sumayah. v Table of Contents Author’s Declaration .............................................................................................................................. ii Abstract ................................................................................................................................................. iii Acknowledgements ............................................................................................................................... iv Dedication .............................................................................................................................................. v Table of Contents .................................................................................................................................. vi List of Figures ....................................................................................................................................... ix List of Tables ....................................................................................................................................... xii Chapter 1 Introduction ........................................................................................................................... 1 1.1 Motivation .................................................................................................................................... 1 1.2 Problem Description .................................................................................................................... 2 1.3 Summary of Contributions ........................................................................................................... 4 1.4 Thesis Structure ........................................................................................................................... 7 Chapter 2 Background ........................................................................................................................... 9 2.1 Various Studies in Security Modelling ........................................................................................ 9 2.1.1 Qualitative Versus Quantitative Models ............................................................................... 9 2.1.2 Inductive versus Deductive Analysis .................................................................................. 15 2.1.3 Underlying Mechanisms versus Impact of Failure ............................................................. 16 2.1.4 Inconsistency in Security Modeling .................................................................................... 17 2.1.5 Dataset Unavailability ......................................................................................................... 21 2.1.6 What Went Wrong .............................................................................................................. 24 2.2 Reliability Engineering .............................................................................................................. 26 2.2.1 Failure Model ...................................................................................................................... 27 2.2.2 Reliability Block Diagrams (RBDs) and Structure Functions ............................................ 30 2.2.3 Minimal Path and Minimal Cut Sets ................................................................................... 34 2.2.4 Reliability Model ................................................................................................................ 36 2.3 Multi-state System (MSS) Model and Universal Generating Function ..................................... 45 2.3.1 Multi-state System (MSS) Model ....................................................................................... 45 2.3.2 Universal Generating