Chapter X7

Special Topics

Electronic Journals At the same time, many publishers were finding that the ability to promise long-term preservation and access for The long-term preservation of electronic journal content their content would constitute a significant competitive is a topic of great concern to academic and research librar- advantage. In 2000 the Andrew W. Mellon Foundation ies, many of which have been reluctant to invest in all- funded seven major research libraries to plan and pilot electronic subscriptions without a believable guarantee the development of e-journal and address busi- that the content will be responsibly archived. This is a mat- ness models for sustainability. One major conclusion from ter of no small consequence: ARL statistics for 2003–2004 these studies was that the costs of archiving could not show member libraries spent $301,699,645, or roughly 31 be assumed by individual libraries on behalf of the wider percent of their total materials expenditures, to license library community.1 Mellon subsequently focused its www.techsource.ala.org Technology 2008 Library Reports February/March electronic materials, almost exclusively journals. funding on two projects with quite different approaches, The preservation of electronic scholarly journals LOCKSS development at Stanford, and the JSTOR presents special organizational and technical challenges. Electronic-Archiving Initiative (E-). LOCKSS is Organizationally, the normal model for delivery of elec- discussed in Chapter 6. The JSTOR E-Archive project ulti- tronic journals, where a for-profit publisher hosts the con- mately evolved into Portico. tent in a central system and sells subscription access to The Portico archive is a centralized preserva- it, means that libraries, the parties with the most stake tion repository of scholarly electronic journals. Unlike in preserving the content for future generations, neither LOCKSS, which harvests Web content and so stores own nor physically control it. Preservation requires the access versions by definition, Portico technology is based negotiation of agreements between the parties defining on acquiring the publishers’ source files and archiving rights and responsibilities on both sides, including condi- both original and normalized versions. Normalization tions for access and limitations on access. occurs at the time of ingest, and involves converting from Technically, the source for online journals can be the publishers’ proprietary SGML and/or XML formats anything from images of printed pages to highly marked- to the National Library of Medicine’s Journal Archiving up text in XML or SGML. Different journals, even from and Interchange DTD, a de facto standard.2 Portico will the same publisher, can have different markup formats. also perform forward migration as file formats threaten Receiving preservation systems must control and relate to become obsolete. Because its preservation strategy is content at article, issue, volume, and title levels. Quality based on normalization and migration, Portico promises control is a big issue, as issues often fail to meet the pub- to preserve intellectual content only, not necessarily the lisher’s own documentation standards. Many items of con- original look and feel of the journals. However, much tent, from masthead information to advertisements, are attention is paid to the integrity of the scholarly record, now dynamically provided by the online delivery system and Portico works directly with individual publishers to and may not be part of the journal source files at all. ensure the appropriate content is archived. Early studies suggested that publishers would not be Portico is supported by a combination of grant fund- trusted by the library community to provide long-term ing, publisher fees, and library subscription fees. In return preservation and access capability for their own content. for an annual support payment based on the total materi- 33 34 Library Technology Reports www.techsource.ala.org February/March 2008 is paramount. demonstrable the reliability,authenticity, and accuracy of electronic records maintaining managers, records and For them. produced which activities the ment record. value The of evidentiary records is docu- they that the of content the as preserve to important as is nection con- this demonstrating relationships of documentation explicit or implicit the the key,and is and activity organizational record the between connection The business. ated or received by an organization in the course of doing To archivists, a record is a bit of recorded creinformation - and practice. forefrontbeen at theory the of preservation have archivists internationally but far, so report this in emphasized been not has records digital of Preservation Records andArchives Readings worldwide. initiatives other and e-Depot, KoninklijkeBibliotheek’s the project, kopal PANDORAAustralia’sare includedin archive, Germany’s preserve their national journal output. Electronic journals www.portico.org Portico Web site source. available publisherorother from the trigger certain events occur, such as when when the publications are no longer journals archived to access wide campus- obtains institution the library, the of expenditure als •  •  •  •  to means in investing are libraries Globally,national abst.html. A review and comparison of journal journal of archiving programs. comparison and review A abst.html. www.clir.org/pubs/abstract/pub138 2006, 138, and Bounds: A Survey of the Landscape, CLIR pub Kenney,R. Anne al., et journals. related toarchiving electronic issues difficult the of exposition clear A abst.html. http://www.clir.org/pubs/abstract/pub106 2002, Archiving,Media Digital in Issues Preservation: for Strategy National a Building in Periodicals,” Digital “Preserving Flecker, Dale .org/news/PapersFromPortico.1.Overview.pdf. Portico’s of Overview Approach,” An “Archiving Fenton, Journals: Eileen Electronic and Kirchhoff Amy www .portico.org/news/Archiving2006-Owens.pdf Journals,” Electronic of Preservation and Ingest the Workflow“Automated for Owens, Evan www.portico 2006, Papers, Portico E-Journal ArchivingE-Journal Metes CLIR pub 106, pub CLIR a sre hi itne proe s records.” as purpose intended their serve can they which in those from substantially differ that forms in stored arerecords “Electronic use. for render to ware - soft requires which record, digital the and bitstream tal Archives developed an open-source tool called Xena that that Xena called tool open-source an developed Archives The normalization. on based archive preservation digital (VEO). Object VERSEncapsulated called the object, single a in content record the with metadata the encapsulates and provided, be must which metadata for standard a specifies VERS maintained, are reliability and authenticity, context, that ensureTo record. original the of format the on depending formats, preservation term long- several of one to accession of time the at records The VERS preservation approach is based on normalizing be used to to reliably mechanisms export records from and agencies to the PROV. support, must systems keeping record- agency local that functions the for specifications developed has VERS Office Victoria, throughout agencies state Record Public the by Victoria (PROV). Because the PROV 1995 receives records from in Strategy begun Records (VERS) Electronic Victorian the is initiative will beabletoimplement. concrete plans archivesthat action limited resourceswith earlier research (2007–2012) to is translate intended into InterPARES activities. scientific,ernmental, and 3 artistic recordsgovto producedby and - authenticity, to addition in accuracy and reliability include to focus the expanded cre- records of ators, and one for creators preservers. InterPARES 2 (2002–2006) the for intended one records, digital of authenticity the maintaining and assessing for requirements of sets two included record-keepingDeliverables systems. and legal and administrative recordsin digital of authenticity the maintaining for ods - InterPARES ever addressed(1999–2001) meth 1 undertaken. initiative preservation digital influential most and longest the arguably is It countries. twenty-one in pants researchtional, multidisciplinary project involving- partici - interna an is Systems) Electronic in Records Authentic transformations. these through record the of authenticity the ensure together which procedures and metadata establishing and erties, prop- these preserve which transformations developing defining significant of properties various forms of records, on instead concentrated have They intact. data original keeping at aimed strategies preservation and record the storedof form havethe archivists abandon quickto been bitstream itself, which must be preserved. For this reason, the not performance, the is it that notes and formance,” ArchivesNational of Australia, calls rendering this a “per- interpreted are the bitstreams of Wilson, Andrew process. rendering their some through as only used be can Archivists Archivists emphasize differencethe - digi the between The National Archives of Australia has also adopted a keeping record digital mature most the Perhaps Permanent on Research (International InterPARES 3 Records Library Technology Reports www.techsource.ala.org February/March 2008 35 Within this general technical framework there is Archive crawlers start from “seed” URLs used to founded corporation nonprofit a Archive, Internet The much much settled on a common set of tools and approaches. Web pages are downloaded using a web crawler (spider) similar to those used by Internet search engines. By far the most popular tool for archiving is Heritrix, an open- Capture Archive. Internetthe by developed crawler source results are written in records in the ARC or WARC - for developed format simple a is acronym) mats.an (not ARC for the Internet Archive, while WARC (Web ARC), is an updated, expanded version still under development. NutchWAX, The as such tool a with indexed is content archived Nutch engine search Web open-source the of extension an cat- are archives Some files. ARC with work to customized provided be can access End-user not. are some and aloged by a number of tools including theWayback Machine Internet or Archive’s WERA (Web ARchive tool thatnewer supports Access), full-text searching. a much variation in the scope and focus of archiving and in how the addresses to archive are obtained. Most is either site-centric, topic-centric, or domain- centric. Site-centric archiving is most by an commonly organization to preserve done its own Web site(s). Site- but comprehensive, focused narrowly centric are archives because the URLs to pursue are known in advance and access is granted to any restricted content. Internal links necessaryto as deep levels many verticallyas followed are capture the entire site, but external links may be not be on a focus particularpursued. Topic-centric archives sub- ject (e.g., medical Web sites or Chinese studies) or event (e.g., an election or Hurricane Katrina), while centric domain- archives focus on a particular location gen- country, state, territory. In a or to specific and commonly are broader be to tend archives domain-centric and topic- eral, but less deep than site-centric archives. A new model for preserving is emerging for virtual archiving - environ Web Withoutments, Oil. or World such as Second Life to parsed then are which pages, Web of set initial an fetch obtain additional links. Seeds can be supplied manually, generally by subject experts, or automatically. A number of projects center around the development of tools for managing the crawl, such as providing seeds and setting harvest parameters. Tools are also needed for analyzing results in order to determine appropriate future param- eters frequency. such as crawl by Brewster Kahle, has been critical other both for tools open-source providing contentin and Web in archiving archiving efforts. The Internet Archive also hosted offers service the Archive It, which allows catalog,archive, and collectionssearch subscribers of their own. The to International the of member charter a was Archive Internet Internet Preservation Consortium (IIPC) along with the Finland, Denmark, Canada, Australia, of nationallibraries France, Iceland, Italy, Norway, and Sweden, the British

“The Long-Term Electronic Records: Preservation Findings of the InterPARES of Project” [n.d.], Authentic www.interpares.org/book/index 1. .cfm. Report of InterPARES “A Framework of Principles for the Development Strategiesof Policies, the Standards and for Long- Term Preservation of Digital Records,” version 1, Aug. 1, 2007, www.interpares.org/ip2/display_file .cfm?doc=ip2(pub)policy_framework_document .pdf. One of several important deliverables from 2. InterPARES Despite, or perhaps because of, theseof, thebecause Despite,challenges, perhaps or •  •  In the United States, the National Archives and The idea of preserving the Web has taken on almost myth- almost on taken has Web the preserving of idea The ical proportions because of the vast size of the Web and the notorious ephemerality of Web content. Even apart from spatial and temporal concerns, Web archiving - pres Withblogs, challenges. thornytechnical of number entsa RSS feeds, and easy authoring tools provided as “Web 2.0” technologies, Web sites are updated with ing frequency - increas and share with databases the difficulty of inter - The capture. for frequency appropriate selectingthe connectedness of the Web means every harvest involves a decision about how broadly and how deeply to pursue links. sites Many Web or parts of sites are excluded from or IP authorization password by requiring crawling or by generated Dynamically files. robots.txt as such convention Web pages returned in response to queries are a techni - through returned pages are harvesters,as for problem cal links. client-side JavaScript community that specializes in Web archiving has pretty Web Harvesting Web Readings ERA Web site ERA Web www.archives.gov/era encapsulates the original file in XML and creates a second, second, a creates and XML in file original the encapsulates normalized in an open format (see Chapter 4). version Administration (NARA)Records - differ taking is slightly a in building theent approach Archives Electronic Records (ERA), a system to capture, preserve, and provide access to digital federal records. In late 2003 NARA issued an RFP and requirements document, following which two finalistswere selected to participate in a one-year design competition. In September 2006 Lockheed-Martin contract a development six-year awarded worth was $308 mil- on the site. ERA can be followed Web lion. Progress 36 Library Technology Reports www.techsource.ala.org February/March 2008 reliable, and of high quality. The digital of the authentic, is data that ensuring for and standards, dural proce- and technical appropriate accordancewith in data or creating for responsible be must creators used forbase is being created research.and actively Data - data the while databases practices curation data good of with begins preservation long-term the that standing in academic there is environments, an under- Particularly and science of government. and business conduct mention to not science, social the to critical are Databases Databases Readings www.iwaw.net International Web Archiving Workshop freely available Web. onthe tations presenArchivingand - Workshop,papers makesall which Web International annual the follow to is developments pointofview. ervation pres- a from problematic particularly archivesWebmake volume of material and the diversity of source huge file formats potentially The archives. Web for methods vation preser- into research for need acknowledged an is there and preservation, active its to than content Web turing Web. publishedonthe materials capturing and selecting for tools developing are projects Both Urbana–Champaign. at Illinois of University the by led Depository, Echo the the and by Library, Digital California led project, Web-At-Risk the including awards, storage,managing andproviding access. crawls, managing crawling, for applications open-source of set a vetted or developed has and archiving, Web for standards and techniques, tools, common of use agesthe encour- IIPC The Congress. of Library the and Library, •  •  One of the best ways to keep current in Web archiving cap- to paid been has attention more far now, to Up grant NDIIPP of focus a been has archiving Web Preservation Consortium. Preservation Web archiving and of coordinates Internet the International guru the is author The required). www.springerlink (login 2006), .com/content/u723352353416271/fulltext.pdf (Springer, Methods Julien Web Consortium http://netpreserve.org. site, Preservation Internet International aaé, ed., Masanés, e Aciig Ise and Issues Archiving: Web preservation of these databases a matter of major impor- major of matter a databases these of preservation stewardship and the making data, digital of siveamounts spent on research generate and relythat projects on mas- lowing issues: fol- the listed Scotland Edinburgh, in held Preservation on Workshop International 2007 preservation. A answers. database than questions more are there for time this at Nonetheless, techniques practice best professionals, underdebate. isstill information generic to opposed as specialists, subject be must which to extent The linkages. and tation enhancingand anno- documentation, maintaining access, and discovery for metadata creating for responsible Curators are preservation. and reuse for center curation a to over passed be should research primary for used and isespecially important. lection, col- data in used instruments and techniques, methods, the of documentation includes case this in which data, In the sciences in particular, millions of dollars are dollars of millions particular, in sciences the In •  •  •  •  •  •  •  •  •  •  •  •  •  s h crtra faeok s mrig s are so emerging, is framework curatorial the As collected actively being longer no data Optimally, adapted for databases? strategies, be archivists by developed policies procedural and preservation the can extent what To for preservation? ofdatabases selection for the What can be learned from traditional archivalervation? appraisal What are the legal encumbrances on database pres- andinwhatformat? database, a with together preserved is documentation What preservation? redundantuted, model ofdatabase Can we move from a model centralized to - a distrib ofdata? bytes - tera containing databases archived of hundreds to provided be access online multi-user can How versions? database frame- preservation works, efficient while retaining the to ability query different have we can How to evolve? continues it while data preserve we can How and structure? semantics data original the preserve we can How base management environment? - data specific a from data the separate we do How cost)? (atacceptable longterm usable inthe and readable databases archived keep we do How life cycle? ervation’s pres - database the differentin stages arethe What that should bepreserved? database a of features salient the are What 4 Library Technology Reports www.techsource.ala.org February/March 2008 37 Individuals 7 6 “The Archiving and Preservation of Born-Digital Art Born-Digital of Preservation and Archiving “The www.erpanet.org/events/2004/(2003), Workshop” glasgowart/briefingpaper.pdf. Agood overview - pre as a shortpared briefing and bibliography. paper Alain Depocas, Jon Ippolito, Media Variable The and Change: through “Permanence Caitlin Jones, Approach,” Guggenheim , .variablemedia.net/pdf/Permanence.pdf. 2003, www Richard Reinhart, “A System of Formal Notation for Scoring Works of Digital and Variable Media Art” (2007), formalnotation.pdf. www.bampfa.berkeley.edu/about/ Librarians and archivists have a professional as well A major challenge in the preservation of new media •  •  •  For artists and curators, the preservation of new media media new of preservation the curators, and artists For accumulate personal content using computers in homes theirand at work, digital cameras and video recorders, photos music, downloaded store They phones. camera and of their children, tax returns, schoolwork, and lots and lots of e-mail. as personal stake in the preservation of personal digital collections. Today’s personal collections are the source material for tomorrow’s historians and biographers, and heritage cultural by acquired and/or desired be will many some institutions. Moreover, believe that the in interest thepublic’s general longevity of their own digital materials can be leveraged to achieve a awareness of and support greater for institutional digital - preser vation initiatives. Personal Collections Nearly everyone in the developed world has some inter- est in the longevity of their personal digital collections. According to statistics by the Nationmaster.com, United States had 762 personal in computers per 1000 2004 people (compared with 740 television sets). art is simply establishing what preservation means in a the properties significant installationcontext where an of each beholder. for differ may Readings has proposed an XML-based language called Music Art Notation System (MANS) for recording the information obtained. This XML “score” can enable an artwork to be technologies.performed again using different the Coddington, James As urgency. some of matter a is art “If Art,Modernsaid, of theMuseum conservatorfor chief generationsfuture to are understand the art of our time, they need to have real examples presented in authentic manner to understand what and we our artists talk - were ing about. And that is very difficult.” 5 (Sept. (Sept. 2005), www.nsf.gov/ National Science Board, “Long-Lived Digital Data Collections: Enabling Research in and the 21st Century” Education pubs/2005/nsb0540/nsb0540.pdf. A strategy for the National Foundation. Science Another preservation strategy applicable to new •  New Media Art New media art is defined in Wikipedia as“a genre that encompasses artworks created with new media technolo- gies, including computer graphics, computer animation, the Internet, interactive biotechnologies.” Many of technologies, the objects created robotics, by these technologies are usually addressed with and the preservation for (as emulation or videos) for migration(as strategiesof executables). The success of these strategies- in reproduc - untried,how largely artworkis an of performance the ing “Seeing EmulationDouble: Theoryin Practice”and ever. is a noteworthy exception. That 2004 exhibition at the Guggenheim Museum showed original and emulated - ver sym- A side. by side installationsart media of set a of sions posium held in conjunction with the exhibition explored attemptedthe of success the and emulation of process the recreations. current in recreation total or reinterpretation, is art media media. Regardless of preservation strategy, establishing preferably key, properties artwork the of significant the is with the advice and consent of the artist. Jon Ippolito of Network Media Variable the and Museum Guggenheim the inter- an Questionnaire, Media Variable the developed has artist’ssig- an of document activeconception help to tool nificant properties and identify appropriatestrategies for preserving the work. Building on this, Richard Rinehart, from the Berkeley Art Museum/Pacific Film Archive, Reading tance to scientific programs and funding agencies. Both the Both agencies. funding and programs scientific to tance U.K. and e-Science Programme the U.S. National Science datacuration for mechanisms consider (NSF) Foundation to be a part of the necessary cyber-infrastructure for the 21st century. The U.K. Core e-Science Programme funds activitiesthe Digitalitskey of six Curationone as Centre (see Chapter 5). The NSF issued in June 2006 a call for proposals for “Sustainable Digital Data Preservation and Access Network Partners (DataNet)” with initial funding - exem few establisha to aims project The million. $100 of typeorganizationnew thatof integrateswholly a of plars library and archival sciences, cyber-infrastructure, com- puter science, information science, and domain science expertise digitalreliable to “provide preservation, access, integration, and analysis capabilities for science and/or engineering a decades-long timeline.” data over 38 Library Technology Reports www.techsource.ala.org February/March 2008 www.bl.uk/digital-lives/index.html Digital LivesWeb site provide for the longevity of their own digital files. people ordinary how of studies field in-depth first the of one was Archives” Personal for Model Service Towarda above. the of light in Library British the workflows curatorial of the assess finally, and objects, personal of preservation for promisingtechnologies into look repositories, by tion acquisi- their and collections digital personal impact that issues ethical and legal investigate individuals, of - tudes atti and behavior the about more ascertain to attempt will study The repositories. research and collections tal - of personal digi Lives intersection is studyingDigital the from Library, run British the by Spearheaded 2009. through to 2007 project research Lives Digital the funded Council Research Humanities and Arts (U.K.) the 2007 In data. personal of creators individual for guidelines of set useful a includes also which workbook, online an of form the in curators for guidelines best-practice duced pro- Paradigm 2007. through ran which Media), in Digital Accessible Archives (Personal project Paradigm the in collections personal digital of preservation the gated andanalogone: the artifactual arenathan more difficult tral challenges of personal archiving that make the digital occasional the ex-husband). In listthey addition, evenfour cen- (and relatives and friends from support IT hoc ad of malware pervasiveness als, such as and the reliance on individu- for archiving complicate that factors ronmental envi- many identified the the authors findings,interesting “The Long Term Fate of Our Digital Belongings: Belongings: Digital Our of Fate Term Long “The - investi Manchester and Oxford of Universities The •  •  •  •  for long-term access. for long-term facilities support not does metaphor desktop The formats.” anddigital systems an incomplete understanding of heterogeneous file direct consequence of benign coupled neglect with a ways many “in are practices curation Personal offline media. and online of types different many and machines multiple on scattered be to tend assets Personal futurejudge worth. their to harder it making belongings, physical than rate are much a at accumulated faster materials Digital 8 Among Notes Readings 8. 7. 6. 5. 4. 3. 2. 1. u Dgtl eogns Twr a evc Mdl forarchiving2006-marshall.pdf (accessed Nov. 17,2007). Model of Service www.csdl.tamu.edu/~marshall/ a Fate 2006, Archives,” Personal Toward Term Belongings: Long Digital “The Our al., et. Marshall Catherine .php (accessedNov. 17,2007). www.nationmaster.com/index Web NationMaster.com site, (accessed Nov. 28,2007). .com/2006/03/29/arts/artsspecial/29digital.html Norm,” Terry Schwadron, “Preserving Work That Falls Outside the 17, 2007). Nov. www.nsf.gov/funding/pgm_summ.jsp?pims (accessed _id=503141&org=OCI&from=home site, Cyberinfrastructure of Web Office NSF (DataNet), Partners http://homepages.inf.ed.ac.uk/hmueller/ and Access Network Preservation Data Digital Sustainable site, (accessedNov.presdb07/index.html 17,2007). Web 2007 Preservation Database on Workshop International 2007). 17, Nov. (accessed www.interpares.org/book/index.cfm 6, Project, Electronic InterPARES the of Authentic Findings Records: of Preservation Long-Term The dtd.nlm.nih.gov (accessedNov. 17,2007). NLM Archiving and Interchange Tag Suite Web site, http:// 2007). Nov.(accessed 17, www.diglib.org/preserve/ejp.htm 2003, Foundation, Mellon W.Andrew the by Funded Research ed., Contera, Linda •  •  1 o 6 Jn 20) www.dlib.org/ dlib/june05/beagrie/06beagrie.html. 2005), (June 6 no. 11 Magazine Bottom? the Collections,” and at Libraries Digital Personal Room of “Plenty Beagrie, Neil Papers, 2007,www.paradigm.ac.uk/workbook. Project, Paradigm www.nytimes 2006, 29, March Times, York New rhvn Eetoi Journals: Electronic Archiving okok n iia Private Digital on Workbook Appendix D-Lib