Automating the Upgrade of IaaS Cloud Systems Mina Nabi A Thesis In the Department of Electrical & Computer Engineering Presented in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy (Electrical & Computer Engineering) at Concordia University Montreal, Quebec, Canada July 2019 © Mina Nabi, 2019 CONCORDIA UNIVERSITY SCHOOL OF GRADUATE STUDIES This is to certify that the thesis prepared By: Mina Nabi Entitled: Automating the Upgrade of IaaS Cloud Systems and submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Electrical and Computer Engineering) complies with the regulations of the University and meets the accepted standards with respect to originality and quality. Signed by the final examining committee: Chair Dr. Luis Amador External Examiner Dr. Michel Dagnenais External to Program Dr. Olga Ormandijieva Examiner Dr. Juergen Rilling Examiner Dr. Wahab Hamou-Lhadj Thesis Co-Supervisor Dr. Ferhat Khendek Thesis Co-Supervisor Dr. Maria Toeroe Approved by Dr. Rastko R. Selmic, Graduate Program Director September 4, 2019 Dr. Amir Asif, Dean Gina Cody School of Engineering & Computer Science ABSTRACT Automating the Upgrade of IaaS Cloud Systems Mina Nabi, Ph.D. Concordia University, 2019 The different resources providing an Infrastructure as a Service (IaaS) cloud service may need to be upgraded several times throughout their life-cycle for different reasons, for instance to fix discovered bugs, to add new features, or to fix a security threat. An IaaS cloud provider is committed to each tenant by a service level agreement (SLA) which indicates the terms of commitment, e.g. the level of availability, that have to be respected even during upgrades. However, the service delivered by the IaaS cloud provider may be affected during the upgrade. Subsequently, this may violate the SLA, which in turn will impact other services relying on the IaaS. Our goal in this thesis is to devise an approach and a framework for automating the up- grade of IaaS cloud systems with minimal impact on the services and with respect to the SLAs. The upgrade of IaaS cloud systems under availability constraints inherits all the challenges of the upgrade of traditional clustered systems and faces other cloud specific challenges. Similar challenges as in clustered systems include the potential dependencies between resources, po- tential incompatibilities along dependencies during the upgrade, potential system configuration inconsistencies due to the upgrade failures and the minimization of the amount of used re- sources to complete the upgrade. Dependencies of the application layer on the IaaS layer is an added challenge that must be handled properly. In addition, the dynamic nature of the cloud iii environment poses a new challenge. A cloud system evolves, even during the upgrade, accord- ing to the workload changes by scaling in/out. This mechanism (referred to as autoscaling) may interfere with the upgrade process in different ways. In this thesis, we define an upgrade management framework for the upgrade of IaaS cloud systems under SLA constraints. This framework addresses all the aforementioned challenges in an integrated manner. The proposed framework automatically upgrades an IaaS cloud sys- tem from a current configuration to a desired one, according to the upgrade requests specified by the administrator. It consists of two distinct components, one to coordinate the upgrade, and the other one to execute the necessary upgrade actions on the infrastructure resources. For the coordination of the upgrade process, we propose a new approach to automatically identify and schedule the appropriate upgrade methods and actions for implementing the upgrade requests in an iterative manner taking into account the vendors’ descriptions of the infrastructure com- ponents, the SLAs with the tenants, and the status of the system. This approach is also capable of handling new upgrade requests even during ongoing upgrades, which makes it suitable for continuous delivery. In case of failures, the proposed approach automatically issues localized retry and undo recovery operations as appropriate for the failed upgrade actions to preserve the consistency of the system configuration. In this thesis, to demonstrate the feasibility of the proposed upgrade management framework we present a proof of concept (PoC) for the upgrade IaaS compute, and its application in an OpenStack cluster. In this PoC, we target the new challenge of upgrade of the IaaS cloud (i.e. unexpected interference between the autoscaling and the upgrade processes) compared to the clustered systems. In addition, the prototype of the proposed upgrade approach for coordinating the upgrade of all kinds of IaaS resources has been implemented and discussed in this thesis. We also provide an informal validation and a rigorous analysis of the main properties of our iv approach. In addition, we conduct experiments to evaluate our approach with respect to SLA constraints of availability and elasticity. The results show that our approach avoids the outage at the application level and reduces SLA violations during the upgrade, compared to the tradi- tional upgrade method used by cloud providers. v Acknowledgments I would like to express my sincere thanks and gratitude to both Prof. Dr. Ferhat Khendek and Dr. Maria Toeroe, specifically for their supervision, encouragement, and continuous support. This thesis would not have been possible without their tireless guidance, knowledge, and mentorship along the way. This work has been conducted within the NSERC/Ericsson Industrial Research Chair in Model- Based Software Management, which is supported by Natural Sciences and Engineering Research Council of Canada (NSERC), Ericsson and Concordia University. My sincere thank goes to my Ph.D. committee members for their advice, comments, and their efforts in the final evaluative process. I express my gratitude to my colleagues at MAGIC (past and present) for their support and friendship, which helped me during the challenging path throughout my Ph.D. program. My heartfelt gratitude to my family, especially my beloved parents (Ahad Nabi and Farideh Abdi), for their support, patience, and endless love throughout my life. My deepest thanks to my husband, Daniel Mastine, for his love, support, patience and his help during the past few years. I cannot thank them each enough for all the sacrifices they made throughout my journey. vi Table of Contents List of Figures ........................................................................................................................ x List of Tables ....................................................................................................................... xiv List of Flowcharts................................................................................................................. xv List of Acronyms .................................................................................................................. xv 1 Introduction ..................................................................................................................... 1 1.1 Thesis Motivation ........................................................................................................ 1 1.2 Contribution of the Thesis ........................................................................................... 3 1.3 Thesis Organization..................................................................................................... 6 2 Background and Related Work ....................................................................................... 8 2.1 Background ................................................................................................................. 8 2.1.1 Cloud Computing .................................................................................................... 8 2.1.2 Scaling ................................................................................................................... 10 2.1.3 OpenStack .............................................................................................................. 11 2.1.4 Availability ............................................................................................................ 11 2.2 Related Work on Upgrade ......................................................................................... 15 3 Infrastructure Resource Information Models and Dependencies.................................. 22 3.1 IaaS Resource Information Domain Models ............................................................. 23 3.2 Possible Changes in the IaaS layer ............................................................................ 28 3.3 IaaS Dependency Characterization ........................................................................... 30 3.4 Summary ................................................................................................................... 36 vii 4 Overview of the Framework for IaaS Cloud Upgrade and Principles .......................... 37 4.1 Definitions ................................................................................................................. 38 4.1.1 Infrastructure Components .................................................................................... 38 4.1.2 Actions, Operations and Units ............................................................................... 38 4.1.3 Upgrade Request...................................................................................................