ElasticMark: An Elasticity Benchmarking Framework for Cloud Platforms Sadeka Islam Dissertation submitted in fulfilment of the requirements for the degree of Doctor of Philosophy School of Computer Science and Engineering Faculty of Engineering March 2016 PLEASE TYPE ~SITY THE UNI OF NEw SOU'tl�WA L 8 Th.. I .,Di Hertatlon Sheet Surname or Fam,t name· Islam First name Sad a 0th.. rn m "' s PhD Abbrev,alJon for degree as g, nm the Uno ty cate nd• r: SchoolCSE FacultyEng1neenng IV ochma Trtle ElasbcMa .An Elasba Be g Framewooc Cloud Platforms '°' Abstract 350 words maximum: (PLEASE TYPE) Elasticity Is the unique cost-effective proposition of the cloud. It promises rapid adjustment of resources In response to varying wor1doads so that the application can meet its QoS objectives with minimal operational expenses. It is a crucial attribute that all commercial cloud providers frequently claim to possessin their offerings. However existing literature has not yet provided any meaningful and systematic guidanceto evaluate the elasticity of the cloud platform from the consumer's viewpoint The lack of an elasticity benchmarking framework makes it difficult for the consumer to diagnose and avoid elasticity issues, validate various claims of elasticity and compare the desirability of competing elastic cloud platfonns. As such, this thesis proposes a novel consumer-centric elasticity benchmarking framework ElasticMark that reflects the elasticity of the cloud platfonn as a single figure of merit. It takes the consumer's perspective on running the benchmark by incorporating her application and workload profiles and then encapsulating the consumer's business objectives into the elasticity metric based on observations accessible via the cloud APls. The core framework is comprised of a penalty based elasticity measurerpentmodel, a standard workload suite with time-varying workload patterns and a set of guidelines for instantiating an executable benchmark. Toe measurement model derives the elasticity metricbased on financial penalty rates resulting from over-provisioning (unutilized resources) and under-provisioning (inadequate resources) when the cloud-hosted application is exposed to a suite of fluctuating workloads. ElastlcMant also includes a novel workload model in order to assist the consumer generate representative prototypes of her application-specific fine-scale bursty worlcloads, thus facilitates custom ela�ticity bench�arking. Fu,:thermore, this framework recommends a set of rigorous techniques to ensure repeatable and valid benchmarking results m presence of the performance unpredictability of the cloud environment The framework has been validatedagainst a widely-used commercial cloud service to make sure that the elasti_ci� metric is� good reflection of the low-level adaptability characteristics and observed phenomena. It �as also �een pr�ven effe<:t1ve m comparing and contrasting the elasticity of multiple cloud pl�orms as well as pinpointing anomalies in their adaptive behaviors. Declaration relating to disposition of project thesis/dissertation · · to make available my thesis or dissertationin whole of New South Wales or its agen the n· ht t a hive and 1 hereby grant to the University � provisions of the Copyright Act 1968. I retain forms of media, now or ere a � no ;n subject to the or in part in the University libranes tn all � t all or part of this thesis or the nght to use 1n ure wo rl<s (such as articles or books) all property rights, such as patent nghts. I also retain dissertation ion Abstracts International (this is applicable to . the w350 ord abstract of my thesis in Dissertat 1 also authorise University Microfilms to use doctoral theses only) ······· .... ........... .................... Witness Signature Signature ······ ··· ..... Date copying or conditions on use Requests for e ptional c rcums n re utrin restrictions on onal recognises that there may be xce i : C::: t 1 r f of restriction may be cons idered In excepti The University l.i eq s s o a onger period must be ma de tn wn ng ction for a pen'od f up to 2 years restri ° of the Dean of Graduate Research a�d r 1 e eth approval circumstances � _ - for Award· Date of completion of requirements FOR OFFICE USE ONLY ------ COVER OF THE THESIS GLUED TO THE INSIDE FRONT THIS SHEET IS TO BE i ORIGINALITY STATEMENT I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledge- ment is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project’s design and conception or in style, presentation and linguistic expression is acknowledged. Signed ........................... Sadeka Islam Mar 2016 ii COPYRIGHT STATEMENT I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only). I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation. Sadeka Islam Mar 2016 AUTHENTICITY STATEMENT I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format. Sadeka Islam Mar 2016 iii ACKNOWLEDGEMENTS When life is sweet, say thank you and celebrate. And when life is bitter, say thank you and grow. Shauna Niequist The path to a doctoral degree is full of many ups and downs. Thanks to the Almighty Lord for granting me the ability to adapt well to those fluctuations! More specifically, I am thankful to Him for letting me work on a fabulous topic - elasticity; the more time I spent with it, the more I became enamored with its beauty. This is the area where my mind loves to surf, this is the place where my heart finds its route to contentment! My passion towards research did not grow suddenly out of thin air; as I look back and connect the dots, I see a group of brilliant scholars as my primary source of inspiration. It is their passionate zeal for research that motivates my young research mind, it is their amazing influence that encourages me to strive hard for quality research. I owe them a huge debt of gratitude. Among them, first comes the name of my supervisor, Dr. Anna Liu; she is an excellent mentor with a good combination of academic skills and business insights. Her inspiring words significantly boosted up my confidence. Her thoughtful guidance helped me reach completion within the assigned timeframe. I also feel blessed to have worked with Professor Alan Fekete; his creative thinking and visionary views have a profound impact on my overall research. He also guided me to the fascinating world of benchmarking. Working with my co-supervisor, Dr. Srikumar Venugopal, enhanced my research rigor. He also guided me through the exciting sphere of control-theoretic elasticity mechanisms. My joint supervisor, Dr. Hiroshi Wada, provided me invaluable advice for my reading course and earlier work, which quickly set me up and running. I am also indebted to Dr. Sherif Sakr for his sound guidance on my literature review work. Professor Len Bass’s insightful feedback proved to be instrumental in shaping up my dissertation; remembering that with sincere gratitude. I also worked with Dr. Jacky Keung and Dr. Kevin Lee in my earlier research projects; I appreciate their help and advice. iv I am profoundly grateful to my annual review panel - Professor Ross Jeffery, Professor Fethi Rabhi, Dr. Helen Paik and Dr. Adnene Guabtni, for their thoughtful comments and improvement suggestions. Ross also helped me define my research scope using GQM; his amazing mentorship is truly appreciated with respect. Among others, I’d like to thank UNSW learning center and Ms. Pam Mort for arrang- ing the thesis writing workshop. My sincere gratitude goes to Mr. Colin Taylor for his sound advice regarding administrative matters. I am also grateful to Coursera for offering high quality online courses, which not only quenched my thirst for knowledge but also influenced my thinking pattern. During this PhD, I submitted my papers to a number of conferences and journals and received insightful suggestions from anonymous reviewers and shepherds. I appreciate their invaluable feedbacks; undoubtedly, those enhanced the quality of my work. My sincere gratitude also goes to all researchers and practitioners who provided their constructive feedbacks on my research in those conferences. I acknowledge the generous research grants from Amazon Web Services (AWS) and RightScale, which helped me carry out a series of comprehensive experiments. I received several travel grants too, which allowed me to present my research at numerous conferences; thank you all for supporting a poor PhD student.