Elasticity in IaaS Cloud, preserving performance SLAs A Thesis Submitted for the Degree of Master of Science (Engineering) in the Faculty of Engineering by Mohit Dhingra Supercomputer Education and Research Centre INDIAN INSTITUTE OF SCIENCE BANGALORE { 560 012, INDIA FEBRUARY 2014 Purity, patience, and perseverance are the three essentials to success, and above all, love. --- Swami Vivekananda Whether You Think You Can Or Can't, You're Right. --- Henry Ford 1 Acknowledgments I acknowledge my sincere gratitude to my supervisors Dr. J. Lakshmi and Prof. S. K. Nandy for their help to accomplish and realize this work. Without their insights and discussions, this work would not have taken a meaningful shape. I thank Dr. Lakshmi for bearing with my naive style of writing and teaching me how to express my thoughts clearly. This was especially helpful while writing papers and thesis. She has also taught me the art of being patient which I found was almost missing in me and is an indispensable part of research. I thank Prof. Nandy for his ever-encouraging discussions and providing me the opportunity to work in CAD Lab. His experienced reviews really helped during paper submissions. I am forever indebted and thankful to both of them for their efforts. I am thankful to Prof. K. Gopinath and Prof. Chiranjib Bhattacharyya for their discussions and insightful comments during the course project in \Computer Systems and Machine Learning" which later turned out to be the backbone of my thesis. The project meetings with them were really enlightening. I would also like to mention Prof. Matthew Jacob for his amazing lectures and instigating the spirit of asking questions and more importantly, finding the answers on my own. I also thank Prof. Govindrajan, Chair- man, Supercomputer Education and Research Centre for providing a unique environment that stands apart from rest of institutes. Another important mention here is for the administrators of SERC for providing me with the data that I could use in my project. I thank my friends Anurag, Ankit, Aakriti, Akash, Vasudevan, Priyanka, Siva and Nitisha for their constant support, encouragement, love, and making the workplace a real fun. All fun activities involving long informal discussions, cycling trips, eating out etc. rightly complemented the work and were equally important. Anurag and Ankit need a i special mention here as I have had the best times with them during my stay at IISc. Finally, I would like to thank my parents to support me to pursue my Masters as it was difficult to quit job and coming back to academia. I thank them for showing me the right path all the times. I thank Almighty to provide me the great opportunity to be at Indian Institute of Science. The environment is just right to do good work and people are so passionate about research. I can not think of utilizing my previous two years better by being at anywhere else. ii Abstract Infrastructure-as-a-Service (IaaS), one of the service models of cloud computing, provides resources in the form of Virtual Machines (VMs). Many applications hosted on the IaaS cloud have time varying workloads. These kind of applications benefit from the on-demand provisioning characteristic of cloud platforms. Applications with time varying workloads demand time varying resources in IaaS, which requires elastic resource provisioning in IaaS, such that their performance is intact. In current IaaS cloud systems, VMs are static in nature as their configurations do not change once they are instantiated. Therefore, fluctuation in resource demand is handled in two ways: allocating more VMs to the ap- plication (horizontal scaling) or migrating the application to another VM with a different configuration (vertical scaling). This forces the customers to characterize their workloads at a coarse grained level which potentially leads to under-utilized VM resources or un- der performing application. Furthermore, the current IaaS architecture does not provide performance guarantees to applications, because of two major factors: 1) Performance metrics of the application are not used for resource allocation mechanisms by the IaaS, 2) Current resource allocation mechanisms do not consider virtualization overheads, can significantly impact the application's performance, especially for I/O workloads. In this work, we develop an Elastic Resource Framework for IaaS, which provides flexible resource provisioning mechanism and at the same time preserves performance of applications specified by the Service Level Agreement (SLA). For identification of work- loads which needs elastic resource allocation, variability has been defined as a metric and is associated with the definition of elasticity of a resource allocation system. We iii introduce new components Forecasting Engine based on a Cost Model and Resource man- ager in OpenNebula IaaS cloud, which compute an optimal resource requirement for the next scheduling cycle based on prediction. Scheduler takes this as an input and enables fine grained resource allocation by dynamically adjusting the size of the VM. Since the prediction may not always be entirely correct, there might be under-allocation or over- allocation of resources based on forecast errors. The design of the cost model accounts for both over-allocation of resources and SLA violations caused by under-allocation of resources. Also, proper resource allocation requires consideration of the virtualization overhead, which is not captured by current monitoring frameworks. We modify exist- ing monitoring frameworks to monitor virtualization overhead and provide fine-grained monitoring information in the Virtual Machine Monitor (VMM) as well as VMs. In our approach, the performance of the application is preserved by 1) binding the application level performance SLAs to resource allocation, and 2) accounting for virtualization over- head while allocating resources. The proposed framework is implemented using the forecasting strategies like Seasonal AutoRegressive and Moving Average model (Seasonal ARIMA), and Gaussian Process model. However, this framework is generic enough to use any other forecasting strategy as well. It is applied to the real workloads, namely web server and mail server workloads, obtained through Supercomputer Education and Research Centre, Indian Institute of Science. The results show that significant reduction in the resource requirements can be obtained while preserving the performance of application by restricting the SLA violations. We further show that more intelligent scaling decisions can be taken using the monitoring information derived by the modification in monitoring framework. iv Publications 1. Mohit Dhingra, J. Lakshmi, S. K. Nandy, Chiranjib Bhattacharya, K. Gopinath, \Elastic Resources Framework for IaaS, preserving performance SLAs", in Proceed- ings of 6th International Conference on Cloud Computing, June 2013, Santa Clara, California, US. 2. Mohit Dhingra, J. Lakshmi, S. K. Nandy, \Resource usage monitoring in clouds", in Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Com- puting, GRID 12, Sepetember 2012, Beijing, China. 3. Ankit Anand, Mohit Dhingra, J. Lakshmi, S. K. Nandy, \Resource usage monitoring for kvm based virtual machines", in Proceedings of 18th annual International Con- ference on Advanced Computing and Communications (ADCOM 2012), Bangalore, India, December 2012. Prediction is very difficult, especially if it's about the future. --Niels Bohr v Contents Abstract iii 1 Introduction to Compute Clouds 1 1.1 Overview of Compute Cloud . 2 1.1.1 A brief history and evolution . 2 1.1.2 What is cloud computing? . 4 1.1.3 Cloud Architecture . 5 1.1.4 Characteristics of Clouds . 5 1.1.5 Cloud Deployment Models . 6 1.1.6 Service models of Cloud - SaaS, PaaS and IaaS . 8 1.2 Cloud enabling Technologies . 11 1.2.1 Service-oriented Architectures . 11 1.2.2 Grid Computing . 12 1.2.3 Utility computing . 12 1.2.4 Autonomic Computing . 12 1.2.5 Virtualization . 13 1.3 Quality of Service in Clouds . 14 1.4 IaaS Architecture . 16 1.5 Issues with current IaaS architecture . 20 1.6 Objective of the thesis . 21 1.7 Organization of the Thesis . 22 1.8 Summary . 24 vi 2 Elasticity in Clouds 25 2.1 Elasticity as a metric . 26 2.1.1 Variability of Workload . 27 2.1.2 Elasticity of System . 30 2.2 Elasticity in current IaaS cloud systems . 33 2.3 Dynamic Provisioning Proposals . 37 2.3.1 Reactive Scheduler based techniques . 37 2.3.2 Forecasting based techniques . 41 2.4 Main Contribution of this work . 44 2.5 Summary . 46 3 Provisioning for Elasticity in IaaS Architecture with Performance SLAs 47 3.1 Modified IaaS Architecture . 48 3.2 Forecasting Engine based on Cost Model and Resource Manager . 50 3.2.1 Metrics used . 50 3.2.2 Elastic Resources Framework . 52 3.2.3 Changes envisaged in other components . 55 3.3 Modifications in Monitoring Engine . 56 3.3.1 Virtualization overhead: A case study . 57 3.3.2 Analysis . 59 3.3.3 Monitoring virtualization overhead . 61 3.3.4 Proposed Monitoring Framework . 63 3.3.5 Segregation of Hypervisor Usage per-VM . 65 3.4 Summary . 67 4 Forecasting Engine based on Seasonal ARIMA Model 68 4.1 Introduction to Time Series . 69 4.1.1 ARMA Model . 73 4.1.2 Order selection in AR, MA and ARMA Model . 77 4.1.3 ARIMA and Seasonal ARIMA Model for Nonstationary Series . 80 vii 4.2 Modeling Cloud workloads . 82 4.2.1 HTTP Logs . 83 4.2.2 Mail Logs . 90 4.3 Summary . 94 5 Forecasting Engine based on Gaussian Processes 95 5.1 Introduction to Gaussian Processes . 96 5.1.1 Obtaining Optimal Hyperparameters . 101 5.2 Modeling Cloud workloads using Gaussian Processes . 103 5.2.1 Selection of Kernel Function . 103 5.2.2 Prediction Results . 104 5.3 Summary . 108 6 Evaluation of Elastic Framework 109 6.1 System Performance using prediction . 110 6.1.1 Response time with limited resources .