ON PERFORMANCE OPTIMIZATION AND SYSTEM DESIGN OF FLASH MEMORY BASED SOLID STATE DRIVES IN THE STORAGE HIERARCHY DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Feng Chen, B.E., M.E., M.S. Graduate Program in Computer Science and Engineering The Ohio State University 2010 Dissertation Committee: Xiaodong Zhang, Adviser Ten-Hwang Lai Russell M. Pitzer Feng Qin © Copyright by Feng Chen 2010 ABSTRACT As an emerging storage technology, Flash Memory based Solid State Drive (SSD) has shown a high potential to fundamentally change the existing Hard Disk Drive (HDD) based storage systems. Unlike conventional magnetic disks, SSD is built on semiconductor chips and has no mechanical components (e.g. rotating disk platters). This architectural differ- ence brings many attractive technical features, such as high random data access perfor- mance and low power consumption. Most importantly, these unique features could poten- tially address the long-existing technical limitations of conventional magnetic disks. Due to this reason, SSD has been called a ‘pivotal technology’ that may completely revolutionize current computer storage systems. On the other hand, SSD also poses several critical challenges to application and system designers. First, due to divergent internal structures, SSD is fundamentally different from rotating media, although they share the same logical and physical interfaces. To date we still lack an insightful understanding about the performance characteristics of SSDs, both the positive and negative sides, and their implications to application and system designers. In this dissertation, we present a thorough experimental study on the unique features of SSDs. Second, although SSDs have shown a great performance potential, especially for handling small and random data accesses, SSDs are much more expensive than conven- tional hard disks. Even considering the decreasing price trend, the price gap between SSDs ii and HDDs will not disappear in the near future and it significantly prevents the wide adop- tion of SSDs in practice, especially in cost-sensitive commercial systems. In this disserta- tion we present the design and implementation of a hybrid storage system, called Hystor, which integrates both SSDs and HDDs to provide a cost-efficient solution for commercial applications with minimal change to other system components and applications. Third, a unique merit of SSD is the internal parallelism, which is the key to effectively exploiting the performance potential of SSDs. Unfortunately, the existing literature mostly focuses on addressing the technical limitations of SSDs (e.g. random write issues) and rarely dis- cusses the internal parallelism. In this dissertation, we present our experimental studies on this unique opportunity provided by SSDs. Our study shows that exploiting internal parallelism can bring great performance benefits, but we must also pay attention to some unexpected dynamics. Our case studies in database systems, a typical data-intensive ap- plication, indicate that internal parallelism can significantly improve performance of real- world applications, and many existing HDD-based application optimizations need to be revisited. iii To my family. iv ACKNOWLEDGMENTS The writing of this dissertation is a joyful journey back to many important moments in my life. When I was about to finish this dissertation, I recalled the days and nights I shared with many great people. I would like to take this opportunity to express my sincere gratitude to them for their love, support, and patience over the years. First and foremost, I would like to sincerely give my greatest thank to my advisor, Professor Xiaodong Zhang. As his graduate student, I have been working with him for several years in two universities. During the days we worked together, he has shown me how to become a great researcher. I am always amazed by his never-ending drive, his strong and persistent dedication to his career, and his scientific attitude on every research detail. As a researcher, he always aims high in his career goal and works extremely hard and energetically all the time. His emphasis on pursuing the research impact on updating critical computing technology in the real world has greatly influenced my academic studies. As an advisor, he is always encouraging and unconditionally supportive. Over the years, he guided me throughout my Ph.D. study and offered tremendous help and support in every aspect, from how to identify critical research problems to how to prepare convincing and articulating technical presentations. I think the most important thing I have learned from him is his attitude on working and living. Without his support and help, I could never have reached this point. v I would also like to thank my collaborators at Intel Labs, Dr. David A. Koufaty and Dr. Michael Mesnier. They greatly inspired and helped me in many aspects and provided critical resources and much constructive advice. It was a wonderful experience working with them. I am also very grateful to Prof. Song Jiang at Wayne State University. We worked together on several research projects, which provided me much valuable research experience in the early days of my Ph.D. study. I am also very thankful to my lab-mates and friends, Dr. Lei Guo and Dr. Shansi Ren at Microsoft, Dr. Rubao Lee at Chinese Academy of Sciences, Shuang Liang at EMC DataDomain, Xiaoning Ding, Enhua Tan, Tian Luo, Yin Huai, and Kaibo Wang at Ohio State University. We have shared many joyful moments together and had many interesting and inspiring discussions. Their friendship, help, and support make the whole group like a family. My special gratitude goes to Xiaoning Ding, Dr. Rubao Lee, and Tian Luo. We have closely worked together on several projects, and during the course of my Ph.D. study they have provided tremendous help and incredible contributions. I would also like to acknowledge Prof. William L. Bynum at the College of William and Mary for his help on proof-reading our papers and his great patience in improving the writing of our papers. I am also grateful to many anonymous reviewers in the community for their constructive comments and helpful suggestions to improve the papers’ quality. I would like to thank the committee members of my Ph.D. dissertation: Prof. Feng Qin, Prof. Ten-Hwang Lai, and Prof. Russell M. Pitzer for serving on my committee and for providing much helpful advice. Finally, I would express my wholehearted thanks to my parents, Li Feng and Guosu Chen, my parents-in-law, Hong Zhang and Jiaqiang Peng, and other family members for their unconditional support and endless love. My enormous debt of gratitude can hardly be vi repaid to my wife, Pinpin Peng, for her continuously strong support, patience, and bringing our little baby to this world. I am so grateful to have our daughter, Emaline. Her coming has brought so much joy, laughter, and fun to our family. vii VITA 2000 .......................................B.E. in Computer Science and Engi- neering, Zhejiang University, Hangzhou, China 2003 .......................................M.E. in Computer Science and Tech- nology, Zhejiang University, Hangzhou, China 2009 .......................................M.S.inComputer Science and Engineer- ing, The Ohio State University, Colum- bus, OH 2003-2005 ..................................Ph.D. student in Computer Science, The College of William and Mary, Williams- burg, VA 2006-present ................................Ph.D. student in Computer Science and Engineering, The Ohio State University, Columbus, OH PUBLICATIONS Research Publications Feng Chen and Xiaodong Zhang. “PS-BC: Power-saving Considerations in Design of Buffer Caches Serving Heterogeneous Storage Devices”. In Proceedings of 2010 Inter- national Symposium on Low Power Electronics and Design (ISLPED’10), Austin, Texas, August 18-20, 2010. Rubao Lee, Xiaoning Ding, Feng Chen, Qingda Lu, and Xiaodong Zhang. “MCC-DB: Minimizing Cache Conflicts in Multi-core Processors for Databases”. In Proceedings of the 35th International Conference on Very Large Data Bases (VLDB’09), Lyon, France, August 24-28, 2009. viii Feng Chen, David A. Koufaty, and Xiaodong Zhang. “Understanding Intrinsic Character- istics and System Implications of Flash Memory based Solid State Drives”. In Proceedings of 2009 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Sys- tems (SIGMETRICS/Performance’09), Seattle, WA, June 15-19, 2009. Feng Chen and Xiaodong Zhang. “Caching for Bursts (C-Burst): Let Hard Disks Sleep Well and Work Energetically”. In Proceedings of 2008 International Symposium on Low Power Electronics and Design (ISLPED’08), Bangalore, India, August 11-13, 2008. Feng Chen, Song Jiang, Weisong Shi, Weikuan Yu. “FlexFetch: A History-Aware Scheme for I/O Energy Saving in Mobile Computing”. In Proceedings of 2007 International Con- ference on Parallel Processing (ICPP’07), Xi’an, China, September 10-14, 2007. Xiaoning Ding, Song Jiang, Feng Chen. “A Buffer Cache Management Scheme Exploiting Both Temporal and Spatial Locality”. In ACM Transactions on Storage (ACM TOS), 3(2): May 2007. Xiaoning Ding, Song Jiang, Feng Chen, Kei Davis, and Xiaodong Zhang. “DiskSeen: Exploiting Disk Layout and Access History to Enhance I/O Prefetch”. In Proceedings of 2007 USENIX Annual Technical Conference (USENIX’07), Santa Clara, CA, June 17-22, 2007. Feng Chen, Song Jiang, Xiaodong Zhang. “SmartSaver: Turning Flash Drive into a Disk Energy Saver for Mobile Computers”. In Proceedings of 2006 International Symposium on Low Power Electronics and Design (ISLPED’06), Tegernsee, Germany, October 4-6, 2006. Song Jiang, Xiaoning Ding, Feng Chen, Enhua Tan, and Xiaodong Zhang. “DULO: An Effective Buffer Cache Management Scheme to Exploit Both Temporal and Spatial Local- ities”. In Proceedings of the 4th USENIX Conference on Files and Storage Technologies (FAST’05), San Francisco, CA, December 14-16, 2005, pp.101-114. Song Jiang, Feng Chen, and Xiaodong Zhang. “CLOCK-Pro: An Effective Improvement of the CLOCK Replacement”. In Proceedings of 2005 USENIX Annual Technical Confer- ence (USENIX’05), Anaheim, CA, April 10-15, 2005, pp.323-336.