ENHANCING STORAGE PERFORMANCE IN VIRTUALIZED ENVIRONMENTS A Thesis Presented to The Academic Faculty by Sankaran Sivathanu In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Conputer Science Georgia Institute of Technology August 2011 ENHANCING STORAGE PERFORMANCE IN VIRTUALIZED ENVIRONMENTS Approved by: Dr. Ling Liu, Committee Chair Dr. Umakishore Ramachandran School of Computer Science School of Computer Science Georgia Institute of Technology Georgia Institute of Technology Dr.Calton Pu Dr. Lakshmish Ramaswamy School of Computer Science Department of Computer Science Georgia Institute of Technology The University of Georgia Dr. Shamkant Navathe Date Approved: 04 May 2011 School of Computer Science Georgia Institute of Technology To my parents, and brothers. iii ACKNOWLEDGEMENTS I first of all thank my advisor Dr. Ling Liu for her great support and guidance throughout my 4-year stint at Georgia Tech. Ling played a great role in shaping up my research skills and provided excellent feedback and encouragement to pursue my ideas in the right direc- tion. I am very thankful to my advisor for having let me work on my own preferred field of research right from the first year of my PhD and I really appreciate the confidence she had in letting me do that. Apart from technical help, Ling has been a wonderful mentor to me in general aspects of career and life. I have always admired her unrelenting commitment to work and she’ll forever be one of my role models for my future career. I also like to thank my thesis committee members Dr. Calton Pu, Dr. Shamkant Na- vathe, Dr. Umakishore Ramachandran and Dr. Lakshmish Ramaswamy for their valuable feedback about my research. My committee members have been a great encouragement to me in completing my research and have also acted as a good motivation for me to excel in the chosen field of research. Next, I thank my two elder brothers Dr. Muthian Sivathanu and Dr. Gopalan Sivathanu, who inspired me to pursue my PhD. They have been a wonderful source of feedback for almost all my research projects so far and I really appreciate the time and effort spent by them in providing me honest feedback on my work and also for the general insights they provided me in the field of computer systems which played a major role in initially gaining interest in this field for pursuing research. Internships have been wonderful times for me all these years and I thank my intern mentors and managers for the fun environment I had there and for giving me the opportunity to work on great projects. I specifically like to thank Dr. Kaladhar Voruganti and Shankar Pasupathy from Netapp, Dr. Cristian Ungureanu and Dr. Eric Kruus from NEC Labs, and iv Dr. Bing Tsai, Dr. Jinpyo Kim from VMware for their excellent contribution in making my internships a fruitful experience overall. My colleagues at the DiSL lab have been very friendly throughout my stay at Georgia Tech and I’ve learned quite a bit from the interactions I had with them over these years. Regular group meeting which showcased the projects my colleagues have been involved in, were really useful to me. I have enjoyed both technical and informal conversations I have had with my neighbor at the lab, Shicong Meng over these years. I cherish the interactions I had with Balaji, Bhuvan, Gong, Ting, Peter, Yuzhe and others in my lab. I am also fortunate to have found great friends during my PhD at Georgia Tech. I specially thank Arun Ponnuswamy, Arvind Subramanian, Bharanidharan Anand, Balaji Palanisamy, Dilip Patharachalam, Sundaresan Venkatasubramanian, Venkatraman Sayee Ramakrishnan and many others for the excellent fun times we had. Above all, I’d like to express my profound gratitude to my father and mother who had been ideal parents to me, providing me with abundance of love, support and encouragement in all my endeavors. I thank them for all the love and affection they have provided me and more importantly the wonderful life lessons and the great thoughts they have inculcated in me. I dedicate this thesis to my wonderful parents and my brothers. v TABLE OF CONTENTS DEDICATION ..................................... iii ACKNOWLEDGEMENTS .............................. iv LIST OF TABLES .................................. x LIST OF FIGURES .................................. xi SUMMARY ...................................... xiii I INTRODUCTION ................................ 1 1.1 Motivation.................................. 3 1.2 Pro-active Storage Framework . 5 1.3 Applications ................................. 5 1.4 Outline.................................... 7 II PRO-ACTIVE STORAGE SYSTEMS ..................... 8 2.1 Overview................................... 8 2.2 Design.................................... 9 2.3 Implementation ............................... 11 2.4 PotentialApplications . 13 2.4.1 Enhancing Storage Performance . 13 2.4.2 Enhancing Data Reliability . 14 2.4.3 More Energy-Efficient Storage . 15 2.5 Extending Pro-active Storage Systems Concept . ...... 15 2.6 RelatedWork ................................ 16 III APPLICATIONS OF PRO-ACTIVE STORAGE SYSTEMS ......... 19 3.1 Opportunistic Data Flushing . 19 3.1.1 Motivation.............................. 19 3.1.2 Why Pro-active Disks? . 21 3.1.3 Design................................ 22 3.1.4 Implementation . 23 vi 3.1.5 Evaluation.............................. 24 3.2 Intelligent Track-Prefetching . 27 3.2.1 Background ............................. 28 3.2.2 Motivation.............................. 28 3.2.3 Design................................ 30 3.2.4 Implementation . 33 3.2.5 Evaluation.............................. 34 3.3 RelatedWork ................................ 38 IV STORAGE CHALLENGES IN VIRTUAL ENVIRONMENTS ....... 40 4.1 Background ................................. 41 4.2 Measurements ................................ 42 4.2.1 ExperimentalSetup . 43 4.2.2 Physical Machines vs. Virtual Machines . 43 4.2.3 Virtual Disk Provisioning and Placement . 48 4.2.4 Inter-VM Workload Interference . 52 4.3 RelatedWork ................................ 58 V PRO-ACTIVE STORAGE IN VIRTUAL ENVIRONMENTS ........ 59 5.1 Background ................................. 59 5.2 VM-specific Opportunities . 61 5.3 Design.................................... 61 5.4 Implementation ............................... 63 5.5 Application: Opportunistic Data Flushing . ....... 64 5.5.1 Scheduling VMs to flush data . 65 5.5.2 Evaluation.............................. 65 5.6 Application: Idle Read After Write . 68 5.6.1 Background ............................. 68 5.6.2 Motivation.............................. 69 5.6.3 Opportunities for Pro-active Storage Systems . ..... 70 vii 5.6.4 Implementation . 71 5.6.5 Evaluation.............................. 72 VI EVALUATION METHODOLOGY ....................... 74 6.1 Introduction ................................. 74 6.2 Background ................................. 76 6.2.1 RealApplications . 76 6.2.2 ApplicationTraces . 77 6.2.3 Synthetic Benchmarks . 77 6.3 I/OTraceCollection............................. 78 6.4 I/OTraceReplay............................... 79 6.4.1 Challenges.............................. 80 6.4.2 Design Considerations . 82 6.5 Architecture ................................. 84 6.6 Load-BasedReplayofI/OTraces . 87 6.6.1 Motivation.............................. 87 6.6.2 Design................................ 88 6.7 Evaluation .................................. 90 6.8 RelatedWork ................................ 97 VII EXPLOITING PRO-ACTIVE STORAGE FOR ENERGY EFFICIENCY: AN ANALYSIS .................................... 99 7.1 Motivation.................................. 99 7.2 Background .................................102 7.3 Modeling Power-Performance Trade-offs in Storage Systems . .104 7.3.1 Needforanewmodel. .104 7.3.2 ModelOverview ..........................105 7.3.3 Energy-Oblivious RAID Systems . 109 7.3.4 Energy-Aware RAID Systems: PARAID . 110 7.4 Improving PARAID: E-PARAID . 116 7.4.1 Motivation..............................116 viii 7.4.2 Design & Implementation . 117 7.4.3 Modeling E-PARAID . 118 7.4.4 Evaluation..............................119 7.5 ModelValidation ..............................122 7.5.1 Simulation environment . 122 7.5.2 Comparison with the model . 124 7.6 Discussion ..................................127 VIIICONCLUSION AND FUTURE WORK .................... 129 8.1 FutureWork .................................130 8.1.1 Exploiting More Opportunities in Storage Systems . 131 8.1.2 Featuring Pro-activeness in other domains . 131 8.2 Summary...................................131 REFERENCES ..................................... 133 VITA .......................................... 141 ix LIST OF TABLES 1 Response Time of 50 block reads at discrete flushing intervals ....... 25 2 IOzone Benchmark showing throughput in KBps . 35 3 Effect of Dynamic Frequency Scaling on I/O Performance . ...... 53 4 ModelParameters...............................106 x LIST OF FIGURES 1 ReadwithWrite................................ 20 2 Idletimes ................................... 21 3 Response time of 50 block reads with background write . ..... 26 4 Linuxkernelcompilation ........................... 27 5 Trackbufferhits(Readsalone) . 31 6 Track buffer hits(Reads with background sequential write) ......... 32 7 Response time for reading 5000 blocks . 35 8 Per-blockresponsetimeforOLTP . 37 9 Latency : Physical machine vs. Virtual machine . ..... 44 10 Cache Hit Rate - Physical Machine vs. Virtual Machine . ..... 44 11 Bandwidth: Physical Machine