1 The Theory and Craft of Digital Preservation Manuscript Submitted to Johns Hopkins University Press By: Trevor Owens June, 2017 2 Table of Contents Acknowledgements 3 1. Beyond Digital Hype and Digital Anxiety 5 2. Artifact, Information, or Folklore: Preservation’s Divergent Lineages 11 3. Understanding Digital Objects 26 4. Challenges & Opportunities of Digital Preservation 39 5. The Craft of Digital Preservation 50 6. Preservation Intent & Collection Development 56 7. Managing Copies and Formats 70 8. Arranging and Describing Digital Objects 85 9. Enabling Multimodal Access and Use 104 10. Conclusions: Tools for Looking Forward 122 Bibliography 131 3 Acknowledgements I spent a year working on this book, but it represents the culmination of about a decade of trying to make my own sense of digital preservation. As such, I have a lot of people to acknowledge. The strengths of this book come from the international digital preservation community I’ve been welcomed into. Its’ weaknesses are my own. I first learned about digital preservation in my time at the Roy Rosenzweig Center for History and New Media. Before he passed away, Roy made an extensive and lasting impression those of us lucky enough to work for him. My constant hope is that the compassion, dedication, and pragmatism Roy brought into every day of his work at the Center comes through in my own. My understanding and appreciation for issues in digital history and digital preservation were sparked by four years of discussion and collaboration with colleges there; Dan Cohen, Josh Greenberg, Sean Takats, Tom Scheinfeldt, Sharon Leon, Sheila Brennan, Dave Lester, Jeremy Boggs, Jim Safley, Kari Kraus, Connie Moon Sehat, Miles Kelly, Mindy Lawrence, Jon Lesser, Kris Kelly, Ken Albers, Faolan Cheslack-Postava, John Flatness, Dan Stillman, and Christopher Hamner. It was in Dan’s digital history graduate seminar that I first really delved into the issues around digital preservation in any formal way. When Sharon couldn’t make it to a Library of Congress NDIIPP meeting I went instead and ended getting introduced to many of my future colleagues. Josh generously offered substantive edits on my undergraduate thesis, was always happy to chat about science and technology studies, and ultimately was a my reference applied to work for the National Digital Information Infrastructure and Preservation Program (NDIIPP). When I joined the NDIIPP team in 2010 I had no idea how lucky I was. I was able to join a team of some of the smartest folks in the world working on digital preservation. The best part was that they were surprisingly down to earth. On my first day, D an Chudnov (who’s Emperor’s New Repository article remains one of best pieces of writing on digital preservation) t old me “You can love the Library of Congress but it won’t ever love you back.” I t’s true. Massive institutions like the LC can feel like cold bureaucracies. However, those c hallenges can c reate bonds and friendships that last. T he best treasures of the Library of Congress are found in it’s staff. My colleagues in NDIIPP, and across the institution, are the people who taught me both w hat digital preservation is and how it actually happens. To that end, I owe a considerable t hanks to; Martha Anderson, Abbey Potter, Michelle Gallinger, Jane Mandelbaum, Dan Chudnov, Butch Lazorcheck, Erin Engle, Leslie Johnston, Jefferson Bailey, Abbie Grotke, E d Summers, Nicholas Taylor, Chris Thatcher, Niki Saylor, Chris Adams, Thomas Padilla, Emily Reynolds, G loria Gonzalez, Jimi Jones, Bertram Lyons, Kate Murray, Caroline Arms, Carl Fleischhauer, Caitlin Hammer, Andrew Cassidy-Amstutz, Kathleen O'Neill, Meg McAleer, Moryma Aydelott, Beth Dulabahn, Kate Zwaard, David Brunton, and many others. While working for NDIIPP I was able to serve as the first co-chair of the National Digital Stewardship Alliance’s digital preservation working group. The monthly conference calls and discussions of what was and wasn’t working factored heavily into shaping ideas behind this book. While everyone involved in NDSA related meetings and events has had an impact on my thinking I would like to specifically mention; Micah Altmen, Andrea Goethals, Andy Johnson, 4 Lori Emmerson, Jason Eppink, Amanda Brennan, Euan Cochran, Megan Phillips, Karen Carani, Dragan Espenschied, Cal Lee, David Rosenthal, Vicky Reich, Katherine Skinner, Nick Krabbenhoeft, Robin Ruggaber, Cory Snavely, Peter Krogh, Micheal Edson, Mike Giarlo, David Pearson, Bethany Nowviskie, Robert Horton, Chris Lacinak, Kara Van Malssen, Abby Smith Rumsey, Howard Besser, Don Watters, Doug Reside, Ian Bogost, Henry Lowood, Jason Scott, Richard Pierce-Moses, Mark Matienzo, Maureen Callahan, Jaime Schumacher, Stacey Erdman, Seb Chan, and Aaron Straup Cope. I would also like to thank people who took the time to comment on the public drafts of this book I shared on my blog; Bertram Lyons, Alan Liu, Chris Adams, Karl-Rainer Blumenthal, Porter Olsen, Matthew Kirschenbaum, Jon Ippolito, Thomas Padilla, Jessica Tieman, Glynn Edwards, Andrew Jackson, Euan Cochrane, Shira Peltzman, Clarissa Ceglio, Annie Johnson, and Steven Lubar. The book is significantly stronger as a result of their input. Across all of these places and spaces I’ve been lucky to have a set of mentors who have taught me so much about how cultural institutions work; Roy Rosenzweig, Dan Cohen, Josh Greenberg, Martha Anderson, Carl Fleischhauer, Kim Sheridan, Richard Staley, and Maura Marx. Lastly, and most importantly, I would like to thank and acknowledge Marjee Chmiel. My wife. My constant companion. You read the, often, incoherent first drafts of this book and helped me make them less so. You did that just as you did with my cover letters and resumes for each of the jobs I’ve mentioned. You brought me to D.C. from Wisconsin. You are so present in my life and in my work that it is impossible to articulate specific things to attribute to you. I do know that I really have no idea who I would be or what I would be doing outside of the life we live together. We did our dissertations together. Each taking the turn powering through writing and research while the other would make dinner, wash the dishes, and walk the dogs. Your accomplishments inspire me. Your insights refine my thinking. Your thoughtfulness and compassion inspire me to work to be a better person. 5 Beyond Digital Hype and Digital Anxiety "Life is short and craft is long." – Hippocrates (5th Century BCE) “A problem has been detected and Windows has been shut down to prevent damage to your computer" – BsoD (Late 20th Century) “The Dream thrives on generalization, on limiting the number of possible questions, on privileging immediate answers.” – Ta-Nehisi Coates (2016) A summit on digital preservation at the Library of Congress in the early 2000s brought together leaders from industry and the cultural heritage sector to work through the mounting challenges in ensuring long- term access to digital information. One participant from a technology company proposed something like “Why don’t we just hoover it all up and shoot it into space.” The “it” in this case being any and all historically significant digital content. Many participants laughed, but it wasn’t intended as a joke. Many individuals have, and continue to seek similar (although generally not quite as literal) “moon-shots,” singular technical solutions to the problem of enduring access to digital information. More than a decade later, we find ourselves amid the same stories and imagined solutions we have heard for the last (at least) 20 years. For the public, there is a belief (and worry) that if something is on the Internet, it will be around forever. At the same time, warnings of an impending “digital dark age,” where records of the recent past become completely lost or inaccessible, frequently appear in the popular press. It is as if digital information will last forever but also, somehow, disappear dramatically all at once. The hype cycles of digital technology, combined with a basic lack of understanding about digital media leave people ill-equipped to sort through the hype and anxiety. Yet, I’ve found that when I tell people that I work on digital preservation and explain what I mean by that term and the work, most people respond something along the lines of “Gosh! I never even thought about that!” For many executives, policy makers and administrators new to digital preservation it seems like the world needs someone to design some super system that can “solve” the problem of digital preservation. The wisdom of the cohort of digital preservation practitioners in libraries, archives, museums, and other cultural memory institutions who have been doing this work for half a century suggests this is an illusory dream. To them, the recurring idea of the singular technological super system that “solves the problem” is distraction not worth chasing. It’s also something that diverts resources from those doing the work. Working to ensure long-term access to digital information is not a problem for a singular tool to solve. It is a complex field with a significant set of ethical dimensions. It is a vocation. It is only possible through the application of resources from our cultural institutions. This book is intended as a point of entry into the theory and craft of digital preservation as it has emerged in practice. The purpose of this book is to offer a path for getting beyond the hyperbole and the anxiety of “the digital” and establish a baseline of practice. To do this, one needs to first unpack what we mean by preservation. It is then critical to establish a basic knowledge of the nature of digital media and digital information. With these in hand, anyone can make significant and practical advances toward mitigating the most pressing risks of digital loss.