;login JUNE 2013 VOL. 38, NO. 3 : File Systems & Linux File System Evolution via Patch Analysis Lanyue Lu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, and Shan Lu &Interview with Ted Ts’o Rik Farrow & Ganeti: Cluster Virtualization Manager Guido Trotter and Tom Limoncelli & Future Interfaces for Non- Volatile Memory Andy Rudoff Columns Practical Perl Tools: Constants David Blank-Edelman IPython and Notebook David Beazley Sampling and Monitoring Dave Josephsen For Good Measure: Calculating Risk Dan Geer and Dan Conway /dev/random: Clusters and Understanding *nix File Permissions Robert Ferrell Book Reviews Mark Lamourine and Trey Darley Conference Reports FAST ’13: 11th USENIX Conference on File and Storage Technologies UPCOMING EVENTS FOR A COMPLETE LIST OF USENIX AND USENIX CO-SPONSORED EVENTS, SEE WWW.USENIX.ORG/CONFERENCES 2013 USENIX Federated Conferences Week Workshops Co-located with USENIX Security ’13 June 24–28, 2013, San Jose, CA, USA www.usenix.org/conference/fcw13 EVT/WOTE ’13: 2013 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections USENIX ATC ’13: 2013 USENIX Annual Technical August 12–13, 2013 Conference www.usenix.org/conference/evtwote13 June 26–28, 2013 USENIX Journal of Election Technology www.usenix.org/conference/atc13 and Systems (JETS) Published in conjunction with EVT/WOTE ICAC ’13: 10th International Conference on www.usenix.org/jets Autonomic Computing Submissions for Volume 1, Issue 2, due: August 6, 2013 June 26–28, 2013 www.usenix.org/conference/icac13 CSET ’13: 6th Workshop on Cyber Security Experimentation and Test HotPar ’13: 5th USENIX Workshop on Hot Topics in August 12, 2013 Parallelism www.usenix.org/conference/cset13 June 24–25, 2013 www.usenix.org/conference/hotpar13 HealthTech ’13: 2013 USENIX Workshop on Health Information Technologies UCMS ’13: 2013 USENIX Configuration Safety, Security, Privacy, and Interoperability of Health Management Summit Information Technologies June 24, 2013 August 12, 2013 www.usenix.org/conference/ucms13 www.usenix.org/conference/healthtech13 8th International Workshop on Feedback Computing LEET ’13: 6th USENIX Workshop on Large-Scale June 25, 2013 Exploits and Emergent Threats www.usenix.org/conference/feedbackcomputing13 August 12, 2013 www.usenix.org/conference/leet13 ESOS ’13: 2013 Workshop on Embedded Self-Organizing Systems FOCI ’13: 3rd USENIX Workshop on Free and Open June 25, 2013 Communications on the Internet www.usenix.org/conference/esos13 August 13, 2013 www.usenix.org/conference/foci13 HotCloud ’13: 5th USENIX Workshop on Hot Topics in Cloud Computing HotSec ’13: 2013 USENIX Summit on Hot Topics June 25–26, 2013 in Security www.usenix.org/conference/hotcloud13 August 13, 2013 www.usenix.org/conference/hotsec13 WiAC ’13: 2013 USENIX Women in Advanced Computing Summit WOOT ’13: 7th USENIX Workshop on June 26–27, 2013 Offensive Technologies www.usenix.org/conference/wiac13 August 13, 2013 www.usenix.org/conference/woot13 HotStorage ’13: 5th USENIX Workshop on Hot Topics in Storage and File Systems LISA ’13: 27th Large Installation System June 27–28, 2013 Administration Conference www.usenix.org/conference/hotstorage13 November 3–8, 2013, Washington, D.C., USA www.usenix.org/conference/lisa13 HotSWUp ’13: 5th Workshop on Hot Topics in Software Upgrades SESA ’13: 2013 USENIX Summit for Educators in June 28, 2013 System Administration www.usenix.org/conference/hotswup13 CO-LOCATED WITH LISA ’13 USENIX Security ’13: 22nd USENIX Security November 5, 2013, Washington, D.C., USA www.usenix.org/conference/sesa13 Symposium Submissions due: July 30, 2013 August 14–16, 2013, Washington, D.C., USA www.usenix.org/conference/usenixsecurity13 FAST ’14: 12th USENIX Conference on File and Storage Technologies February 17–20, 2014, Santa Clara, CA, USA www.usenix.org/conference/fast14 Submissions due: September 26, 2013 JUNE 2013 VOL. 38, NO. 3 EDITOR Rik Farrow [email protected] EDITORIAL MANAGING EDITOR Rikki Endsley 2 Musings Rik Farrow [email protected] COPY EDITOR OPINION Steve Gilmartin 6 On Teaching Style and Maintainability Geoff Kuenning [email protected] PRODUCTION Arnold Gatilao FILE SYSTEMS Casey Henderson 10 A Study of Linux File System Evolution Michele Nelson Lanyue Lu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, TYPESETTER and Shan Lu Star Type [email protected] 18 Ted Ts’o on Linux File Systems: An Interview USENIX ASSOCIATION Rik Farrow 2560 Ninth Street, Suite 215, Berkeley, California 94710 22 Shingled Magnetic Recording: Areal Density Increase Requires New Phone: (510) 528-8649 Data Management FAX: (510) 548-5738 Tim Feldman and Garth Gibson www.usenix.org ;login: is the official magazine of the USENIX CLUSTERS Association. ;login: (ISSN 1044-6397) 31 Ganeti: Cluster Virtualization Manager is published bi-monthly by the USENIX Guido Trotter and Tom Limoncelli Association, 2560 Ninth Street, Suite 215, Berkeley, CA 94710. 37 PRObE: A Thousand-Node Experimental Cluster for Computer $90 of each member’s annual dues is for a Systems Research subscription to ;login:. Subscriptions for non- Garth Gibson, Gary Grider, Andree Jacobson, and Wyatt Lloyd members are $90 per year. Periodicals postage paid at Berkeley, CA, and additional offices. HARDWARE POSTMASTER: Send address changes to ;login:, USENIX Association, 2560 Ninth Street, 40 Programming Models for Emerging Non-Volatile Suite 215, Berkeley, CA 94710. Memory Technologies ©2013 USENIX Association Andy Rudoff USENIX is a registered trademark of the USENIX Association. Many of the designations used by manufacturers and sellers to COLUMNS distinguish their products are claimed 46 Practical Perl Tools David Blank-Edelman as trademarks. USENIX acknowledges all trademarks herein. Where those designations 50 A PyCon Notebook David Beazley appear in this publication and USENIX is aware of a trademark claim, the designations have 55 iVoyeur Dave Josephsen been printed in caps or initial caps. 58 For Good Measure Dan Geer and Dan Conway 61 /dev/random Robert G. Ferrell 64 Notes BOOKS 66 Book Reviews Mark Lamourine and Trey Darley CONFERENCE REPORTS 69 11th USENIX Conference on File and Storage Technologies (FAST ’13) Musings RIK FARROWEDITORIAL Rik is the editor of ;login:. he trouble with the future is that it never arrives. For me, there is only [email protected] right now, with an imagined future some constant distance away in Tthe, uh, future. But things do change. I love attending FAST, because some of my own roots are on the vendor side, and FAST provides one of the best blends of academic and vendor research of any USENIX conference. FAST ’13 was no exception, and when Rick Wheeler suggested I attend a particular BoF (one I might have skipped), I discovered something interesting. A Game Changer? Andy Rudoff, now with Intel, once a Sun engineer, was already well into his presenta- tion when I wandered in. Andy was enthusiastic, that was certain. But would Non-Volatile Memory (NVM) really be the game changer that he was hinting at? Not that Andy was really hinting at anything. His presentation, and his article in this issue, was about adding two new interfaces for NVM. We already have two interfaces for NVM, in the now familiar form of SSDs: a block and a file interface. What Andy was explaining relies on there being a form of NVM that is addressable at the byte level, instead of the block level, as with current Flash devices. This suggestion got my attention. During HotOS 2011, Katelin Bailey presented a position paper about systems built with NVM [1]. Imagine a system with gigabytes of byte-address- able persistent memory. If you hibernate such a system, it can sleep using no power, but wake up immediately with all its memory intact. You don’t need virtual memory, because you don’t need to use disk for backing limited DRAM. But there are also problems with this picture. For example, now we can safely assume that rebooting a system clears memory (well, almost [2]), but if DRAM is replaced with NVM, that assumption is no longer true; whatever got us into trouble before is still there. Andy suggests two new interfaces to byte-addressable NVM: PM (persistent memory) Volume Mode and PM File Mode. Although these might sound similar to the current ways we have for accessing Flash, they are different in that they assume the CPU can perform load/ store instructions at the byte level, which is very different from the block-oriented Flash we’ve been learning about and using for years. In an interview [3], Intel’s Chief Technology Officer, Justin Rattner, said that new NVM will have low latency, in the tens of nanoseconds. Just compare that to typical disk latency, which is measured in milliseconds. Well, let’s see: nanosecond = 10-9 vs. millisecond = 10-3 That would be quite a difference. I tried to pry more from Andy about just how much NVM we might expect to have, but he couldn’t tell me—or if he could, I couldn’t tell you. You know, the usual NDA conundrum. But my suspicion is that NVM could be a real game changer, if the various technologies for creating this memory actually prove capable of being used in cheap, reliable, fast, and dense devices. 2 JUNE 2013 VOL. 38 NO. 3 www.usenix.org EDITORIAL Musings Shingles, Not on a Roof believe) write-once; if SMR drives could provide better perfor- Not that I am expecting terabytes (or tebibytes [4]) of affordable mance, if 64 MB bands were used instead of using ext4, I think PM anytime soon. Hard disk drives (HDDs) have managed to that Google just might be interested. grow in capacity at a rate of about 40% a year. But this growth is Many years ago, a friend who worked on IBM mainframes in a going to hit a wall soon, as the ability to write sectors to ever nar- bank let me look at some of their documentation.