Using MEMS-based storage devices in computer systems STEVEN W. SCHLOSSER May 2004 cmu-pdl-04-104 Dept. of Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213 A dissertation submitted to the graduate school in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering. Thesis committee Gregory R. Ganger, Chair L. Richard Carley James C. Hoe Charles C. Morehouse, Hewlett-Packard Laboratories c 2004 Steven W. Schlosser i Keywords: MEMS-based storage, storage systems performance, database sys- tems, disk arrays To my family: my parents, Phil and Kathy; my sister, Jennifer; and my wife, Rachel. Abstract MEMS-based storage is an interesting new technology that promises to bring fast, non-volatile, mass data storage to computer systems. MEMS-based storage de- vices (MEMStores) themselves consist of several thousand read/write tips, anal- ogous to the read/write heads of a disk drive, which read and write data in a recording medium. This medium is coated on a moving rectangular surface that is positioned by a set of MEMS actuators. Access times are expected to be less than a millisecond with energy consumption 10{100× less than a low-power disk drive, while streaming bandwidth and volumetric density are expected to be around that of disk drives. This dissertation explores the use of MEMStores in computer systems, with a focus on whether systems can use existing abstractions and interfaces to incor- porate MEMStores effectively, or if they will have to change the way they access storage to benefit from MEMStores. If systems can use MEMStores in the same way that they use disk drives, it will be more likely that MEMStores will be adopted when they do become available. Since real MEMStores do not yet exist, I present a detailed software model that allows their use to be explored under a variety of workloads. To answer the question of whether a new type of device requires changes to systems, I present a methodology that includes two objective tests for determining whether the benefit from a device is due to a specific difference in how that device accesses data or is just due to the fact that that device is faster, smaller, or uses less energy than iv current devices. I present a range of potential uses of MEMStores in computer systems, examining each under a number of user workloads, using the two objective tests to evaluate their efficacy. Using the evidence presented and the two objective tests, I show that systems can incorporate MEMStores easily and employ the same standard abstractions and interfaces used with disk systems. At a high level, the intuition is that MEMStores are mechanical storage devices, just like disk drives, only faster, smaller, and re- quiring less energy to operate. Accessing data requires an initial seek time that is distance-dependent, and, once access has begun, sequential access is the most efficient. This intuition is described in more detail, and the result is shown to hold for the range of uses presented. Acknowledgements Not one word of this dissertation could have been written without a great deal of help and support from many people. Being a graduate student in the Parallel Data Laboratory has been the most rewarding experience of my life, and I am grateful for having had the opportunity to work with such a fantastic group. The most important thanks go to my collaborator and friend, John Linwood Griffin. We started working on this project in 1999, built the model and simulator together, and produced many of the results presented here. I will always be grateful for his help, insight, and friendship. I also owe special thanks to Jiri Schindler, Minglong Shao, and Natassa Ailamaki, who have helped me a great deal in the last two years to apply some the insights of this dissertation to disk drives and database systems. I have been fortunate to have the support of the members of my thesis com- mittee, who have been crucial to developing this work. Greg Ganger, my advisor, has taught me more than I ever thought I could know about computer systems, storage devices, and navigating the world of research. In addition to taking on the responsibilities of at least four full-time jobs simultaneously, he is always available to his students, which is truly amazing. Rick Carley provided the initial impetus for this work by starting the MEMS-based storage research project in the MEMS Laboratory at Carnegie Mellon. He had the great foresight of involving computer systems researchers in the effort, which led to this work. James Hoe was invalu- able in filling the role of the outsider, making sure that I was covering all the right vi bases. I've also been fortunate to have the support and input of Chuck Morehouse, who leads the probe storage effort at Hewlett-Packard Laboratories. I also want to thank David Nagle, who not only helped start this research, but also made it possible for me to attend graduate school. The students and staff of the Parallel Data Laboratory have been very helpful and supportive over the years. Craig Soules, Brandon Salmon, Eno Thereska, Mike Abd-El-Malek, Garth Goodson, Jay Wylie, Andy Klosterman, Shuheng Zhou, John Strunk, Ted Wong, David Petrou, and John Bucy, have all helped over the years in ways too numerous to mention. Linda Whipkey and Karen Lindenfelser keep our office and lab running smoothly and I can't thank them enough for their help. Thanks also to Joan Digney who endures our constant lateness when preparing posters, but still finds it in her heart to help a student proofread his dissertation. Contents 1 Introduction 1 1.1 Thesis statement . 3 1.2 Overview . 3 1.2.1 Roles and policies . 5 1.2.2 Objective tests . 6 1.3 Contributions . 6 1.4 Organization . 7 2 Background and related work 8 2.1 Basic device description . 8 2.1.1 Carnegie Mellon University . 9 2.1.2 IBM Millipede . 11 2.1.3 Hewlett-Packard Labs Atomic-Resolution Storage (ARS) . 12 2.2 Low-level data layout . 12 2.3 Media access characteristics . 16 2.4 Logical data layout . 17 2.5 Comparison to disks . 19 2.6 Other alternative technologies . 24 2.6.1 Battery-backed DRAM . 24 2.6.2 Miniature disk drives . 24 2.6.3 FLASH . 25 Contents viii 2.6.4 MRAM . 26 2.6.5 Ovonic Unified Memory . 27 2.6.6 FERAM . 27 2.7 Related work . 27 2.7.1 Devices . 27 2.7.2 Parameter sensitivity . 28 2.7.3 Roles . 29 2.7.4 Policies . 30 3 Performance modeling of MEMStores 32 3.1 Piecewise-linear seek model . 32 3.2 Baseline device parameters . 37 3.3 Basic seek performance . 38 3.4 Spring-mass-damper seek model . 40 3.5 DiskSim . 42 3.6 Parameter sensitivity . 42 3.7 Summary . 46 4 Storage abstractions 47 4.1 Disks and standard abstractions . 48 4.1.1 Holes in the abstraction boundary . 49 4.2 MEMStores and standard abstractions . 50 4.2.1 Access method . 51 4.2.2 Unwritten contract . 52 4.3 Summary . 54 5 Roles of MEMStores in systems 55 5.1 Devices for comparison . 55 5.1.1 G2 MEMStore . 55 5.1.2 IBM Microdrive . 55 Contents ix 5.1.3 Seagate Cheetah 36ES . 56 5.1.4 Quantum Atlas 10K . 56 5.1.5 Ub¨ erdisk . 56 5.2 Simple disk replacement . 58 5.2.1 Synthetic workloads . 58 5.2.2 Trace replay . 59 5.3 MEMStores as caches for disks . 60 5.4 Disk array augmentation . 61 5.5 Summary . 65 6 Policies for accessing MEMStores 66 6.1 Request scheduling . 66 6.1.1 Evaluating scheduling algorithms . 67 6.1.2 Existing disk-based algorithms . 68 6.1.3 SPTF and settling time . 71 6.1.4 MEMStore-specific algorithms . 72 6.1.5 Eliminating settling constraints . 75 6.2 Data layout . 76 6.2.1 Small, skewed accesses . 76 6.2.2 Large, sequential transfers . 77 6.2.3 Bipartite layout . 78 6.3 Exploiting tip-subset parallelism . 80 6.3.1 Background . 81 6.3.2 Exposing tip-subset parallelism . 85 6.3.3 Expressing parallel requests . 87 6.3.4 Application interface . 88 6.3.5 Experimental setup . 89 6.3.6 Accessing blocks for free . 90 6.3.7 Efficient 2D table access . 92 Contents x 6.3.8 Summary . 101 6.4 Energy conservation . 102 6.5 Summary . 105 7 Conclusions and future work 106 7.1 Future work . 108 7.1.1 Reliability and fault tolerance . 108 7.1.2 Other roles and policies . 109 7.1.3 New features of MEMStores . 110 7.1.4 Integration of MEMStores and computation . 112 Bibliography 113 Figures 2.1 Components of a MEMS-based storage device. 9 2.2 The movable media sled. 10 2.3 Data organization on MEMS-based storage devices. 13 2.4 Cylinders, tracks, sectors, and logical blocks. 14 2.5 Mapping LBNs to optimize sequential access. 18 3.1 Piecewise-constant approximation of acceleration and velocity dur- ing a Y-dimension seek. 33 3.2 Seek time profile from corner of media. 39 3.3 Seek time profile from center of media. 39 3.4 Seek time profile of G2 MEMStore from corner of media for Hong's model. 41 3.5 Seek time profile of G2 MEMStore from center of media for Hong's model. 41 3.6 Sensitivity of MEMS-based storage device performance to the access velocity. 43 3.7 Delta in seek times from <-1000,1000> given a spring factor of 75% (compared to 0%) using a G2 MEMStore.