Configuration and Orchestration Techniques for Federated Cloud Resources Denis Weerasiri A thesis in fulfilment of the requirements for the degree of Discovery and Adaptation of DoctorProcess of Philosophy Views THE UNIVERSITY OF NEW SOUTH WALES SYDNEY AUSTRALIA · School of Computer Science and Engineering A dissertation submitted in fulfillment Faculty of Engineering of the requirements for the degree of Doctor of Philosophy Supervisor: Prof. Boualem Benatallah in Computer Science and Engineering February 2016 Hamid Reza Motahari Nezhad Supervisor: Prof. Boualem Benatallah 12 February 2008 PLEASE TYPE THE UNIVERSITY OF NEW SOUTH WALES Thesis/Dissertation Sheet Surname or Family name: WEERASIRI First name: WICKRAMA ARACHCHILLAGE DENIS Other name/s: DHANANJAYA Abbreviation for degree as given in the University calendar: PhD School: School of Computer Science and Engineering Faculty: Faculty of Engineering Title: Configuration and Orchestration Techniques for Federated Cloud Resources Abstract 350 words maximum: (PLEASE TYPE) Cloud resources are central to the operation of modern software-driven organizations. Due to the exponential growth of the number and diversity of cloud resources, organizations are inspired to build and deploy services, processes and applications by intermixing a set of best-of-breed cloud resources. It is estimated that nearly half of all large enterprises will have such intermixed (or we call federated) cloud resource deployments by the end of 2017. Federated cloud resources, a special category of composite resources, which draws component resources from one or more public clouds and one or more private clouds. In this dissertation, we investigate the problems of configuration and orchestration of federated cloud resources. Addressing this problem is challenging, as component resources of federated cloud resources are distributed across multiple heterogeneous, autonomous and evolving cloud providers. Moreover, cloud-based applications may possess dynamic resource requirements during different phases of their life-cycle. Consequently, designing interoperable, portable and effective cloud resource configuration and orchestration techniques that cope with both heterogeneous and dynamic environments remains a deeply challenging problem. To address these challenges, we first propose a taxonomy framework for cloud resource consumers to improve the awareness of the fundamental building blocks within the domain of cloud resource orchestration. Our taxonomy framework allows consumers to efficiently explore, understand, compare, contrast and thereby be able to wisely and rationally evaluate cloud resource orchestration techniques based on consumers' requirements. We then present model-driven and process-driven techniques to describe, reuse and orchestrate elementary and federated cloud resource configurations. In conjunction with, we also propose a pluggable architecture to translate these high-level models into resource descriptions and management rules which can be interpreted by external configuration and orchestration tools such as Juju and Docker. We next propose a rule-based configuration and orchestration knowledge recommender service which empowers incremental acquisition, curation, and recommendation of knowledge based on users' contexts. Finally we introduce a language for effective comprehension and visualization of cloud resource orchestration concerns. This language allows to visually represent, monitor and control cloud resource configurations. All aforementioned proposals have been implemented tools and experimentally validated based on real-world user scenarios and user studies. Declaration relating to disposition of project thesis/dissertation I hereby grant to the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or in part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all property rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstracts International (this is applicable to doctoral theses only). '2.."' 1.b --o 2.. ~a ............ ......... .. .... ...... .. .1 . ...... · · ······ · ······~Signature ·-··· ~ - --~ ·- ···· Date The University recognises that there may be exceptional circumstances requiring restrictions on copying or conditions on use. Requests for restriction for a period of up to 2 years must be made in writing. Requests for a longer period of restriction may be considered in exceptional circumstances and require the approval of the Dean of Graduate Research. FOR OFFICE USE ONLY Date of completion of requirements for Award: THIS SHEET IS TO BE GLUED TO THE INSIDE FRONT COVER OF THE THESIS ORIGINALITY STATEMENT I hereby declare that this submission is my own work and to the best of my knowl­ edge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of t his thesis is the product of my own work, except to the extent that assistance from others in the projects design and conception or in style, presentation and linguistic expression is acknowledged. S i gner~... ,. ' 2 '0 l 0 2 - 0 Date ....... ......8, .·..-.. ........ ~ ii COPYRIGHT STATEMENT . I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after knovvn, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain t he right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only) . I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright ma­ terial; where permission has not been granted I have applied/ will apply for a partial restriction of the digital copy of my thesis or dissertation. Sign ed ~ ..... , Date ..::Lo........................ / .b ~ D L - o .4 AUTHENTICITY STATEMENT I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format. Signed ..~ . ..2_o /i - 0). - 0Cj Date ...........'? ........ ..... .. iii ACKNOWLEDGEMENTS If I have seen further it is by standing on the shoulders of Giants - Sir Isaac Newton First and foremost, I would like to express my sincere appreciation and deep gratitude to my supervisor, Scientia Professor Boualem Benatallah, for his excep- tional support, encouragement and guidance during the last three years and five months. Boualem taught me how to do high-quality research and helped me think creatively. His truly incredible academic excellence and beautiful mind have made him as a constant oasis of ideas and passions in science, which has inspired and en- riched my growth as a student, a researcher and a scientist. Moreover, I thank him for providing me with the opportunity to work with a talented team of researchers. I gratefully dedicate this dissertation to my father Alfred, my mother Chandra, my wife Nipuni and other members of my family, for their love, patience, and un- derstanding. They allowed me to spend most of the time on this thesis. They are my source of strength and without their countless support this dissertation would have never been started. I gratefully thank my co-authors, Dr. Moshe Chai Barukh and Dr. Amin Be- heshti for their enjoyable collaborations. I would like to thank Dr. Moshe especially for his help in reviewing and editing our publications. My sincere thanks go to everyone in the Service-Oriented Computing (SOC) group at UNSW, especially Professor Fethi Rabhi (my co-supervisor), Srikumar Venugopal, Mortada Al-Banna, John Sun, George Ajam, Mohammed Allahbaksh and Helen Paik for their friendship, support and helpful comments. In addition, I would like to thank the Ph.D. review panels and the anonymous reviewers who provided suggestions and helpful feedback on my publications. Working at the School of Computer Science and Engineering at the University of New South Wales (UNSW) has been a great pleasure and a wonderful privilege. I iv acknowledge Smart Services CRC, University of New South Wales, and the Faculty of Engineering at UNSW for providing scholarships to pursue doctoral studies. In addition, I would like to thank administrative and technical staff members of the school of computer science and engineering at UNSW who have been kind enough to advise and help in their respective roles. Denis Weerasiri Sydney, Australia February 2016 v To my family for their love, patience, and understanding vi ABSTRACT