A DIY Approach to Digital Preservation
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Jamie Shiers, CERN
Investing in Curation A Shared Path to Sustainability [email protected] Data Preservation in HEP (DPHEP) Outline 1. Pick 2 of the messages from the roadmap & comment – I could comment on all – but not in 10’ 1. What (+ve) impact has 4C already had on us? – Avoiding overlap with the above 1. A point for discussion – shared responsiblity / action THE MESSAGES The 4C Roadmap Messages 1. Identify the value of digital assets and make choices 2. Demand and choose more efficient systems 3. Develop scalable services and infrastructure 4. Design digital curation as a sustainable service 5. Make funding dependent on costing digital assets across the whole lifecycle 6. Be collaborative and transparent to drive down costs IMPACT OF 4C ON DPHEP International Collaboration for Data Preservation and Long Term Analysis in High Energy Physics “LHC Cost Model” (simplified) Start with 10PB, then +50PB/year, then +50% every 3y (or +15% / year) 10EB 1EB 6 Case B) increasing archive growth Total cost: ~$59.9M (~$2M / year) 7 1. Identify the value of digital assets and make choices • Today, significant volumes of HEP data are thrown away “at birth” – i.e. via very strict filters (aka triggers) B4 writing to storage To 1st approximation ALL remaining data needs to be kept for a few decades • “Value” can be measured in a number of ways: – Scientific publications / results; – Educational / cultural impact; – “Spin-offs” – e.g. superconductivity, ICT, vacuum technology. Why build an LHC? BEFORE! 1 – Long Tail of Papers 2 – New Theore cal Insights 3 4 3 – “Discovery” to “Precision” Volume: 100PB + ~50PB/year (+400PB/year from 2020) 11 Zimmermann( Alain Blondel TLEP design study r-ECFA 2013-07-20 5 Balance sheet – Tevatron@FNAL • 20 year investment in Tevatron ~ $4B • Students $4B • Magnets and MRI $5-10B } ~ $50B total • Computing $40B Very rough calculation – but confirms our gut feeling that investment in fundamental science pays off I think there is an opportunity for someone to repeat this exercise more rigorously cf. -
Module 8 Wiki Guide
Best Practices for Biomedical Research Data Management Harvard Medical School, The Francis A. Countway Library of Medicine Module 8 Wiki Guide Learning Objectives and Outcomes: 1. Emphasize characteristics of long-term data curation and preservation that build on and extend active data management ● It is the purview of permanent archiving and preservation to take over stewardship and ensure that the data do not become technologically obsolete and no longer permanently accessible. ○ The selection of a repository to ensure that certain technical processes are performed routinely and reliably to maintain data integrity ○ Determining the costs and steps necessary to address preservation issues such as technological obsolescence inhibiting data access ○ Consistent, citable access to data and associated contextual records ○ Ensuring that protected data stays protected through repository-governed access control ● Data Management ○ Refers to the handling, manipulation, and retention of data generated within the context of the scientific process ○ Use of this term has become more common as funding agencies require researchers to develop and implement structured plans as part of grant-funded project activities ● Digital Stewardship ○ Contributions to the longevity and usefulness of digital content by its caretakers that may occur within, but often outside of, a formal digital preservation program ○ Encompasses all activities related to the care and management of digital objects over time, and addresses all phases of the digital object lifecycle 2. Distinguish between preservation and curation ● Digital Curation ○ The combination of data curation and digital preservation ○ There tends to be a relatively strong orientation toward authenticity, trustworthiness, and long-term preservation ○ Maintaining and adding value to a trusted body of digital information for future and current use. -
Digital Data Archive and Preservation in the Cloud – What to Do and What Not to Do 2 © 2013 Storage Networking Industry Association
DigitalPRESENTATION Data Archive TITLE and GOES PreservationHERE in the Cloud - What To Do and What Not To Do Chad Thibodeau / Cleversafe Sebastian Zangaro / HP SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced in their entirety without modification The SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney. The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK. Digital Data Archive and Preservation in the Cloud – What to do and What Not to Do 2 © 2013 Storage Networking Industry Association. All Rights Reserved. 2 Abstract Cloud Archive Challenges and Best Practices This session will appeal to Storage Vendors, Datacenter Managers, Developers, and those seeking a basic understanding of how best to implement a Cloud Storage Digital Data Archive and Cloud Storage Digital Data Preservation service. -
Comparing Tape and Cloud Storage for Long-Term Data Preservation 1
Comparing Tape and Cloud Storage for Long-Term Data Preservation November 2018 Table of Contents Abstract ......................................................................................................................................................... 2 Introduction .................................................................................................................................................. 2 Perceptions of Leading Edge Technology ..................................................................................................... 2 Reliability ....................................................................................................................................................... 3 Security ......................................................................................................................................................... 4 Speed ............................................................................................................................................................ 5 Storing Data .............................................................................................................................................. 5 Retrieving Data ......................................................................................................................................... 6 Data Growth .................................................................................................................................................. 6 Redundancy ............................................................................................................................................. -
Archives First: Digital Preservation Further Investigations Into Digital
Archives First: digital preservation Further investigations into digital preservation for local authorities Viv Cothey * 2020 * Gloucestershire County Council ii Not caring about Archives because you have nothing to archive is no different from saying you don’t care about freedom of speech because you have nothing to say. Or that you don’t care about freedom of the press because you don’t like to read. (after Snowden, 2019, p 208) Disclaimer The views and opinions expressed in this report do not necessarily represent those of the institutions to which the author is affiliated. iii iv Executive summary This report is about an investigation into digital preservation by (English) local authorities which was commissioned by the Archives First consortium of eleven local authority record offices or similar memory organisations (Archives). The investigation is partly funded by The National Archives. Archival institutions are uniquely able to serve the public by providing current and future generations with access to authentic unique original records. In the case of local authority Archives these records will include documents related to significant decision making processes and events that bear on individuals and their communities. Archival practice, especially relating to provenance and purposeful preservation, is instrumental in supporting continuing public trust and essential to all of us being able to hold authority to account. The report explains how Archival practice differs from library practice where provenance and purposeful preservation are absent. The current investigation follows an earlier Archives First project in 2016-2017 that investigated local authority digital preservation preparedness. The 2016-2017 investigation revealed that local authority line of business systems in respect of children services, did not support the statutory requirement to retain digital records over the long- term (at least 100 years). -
Response by the Polar Data Community to the OGC Request for Information on Arctic Spatial Data
Response to the Open Geospatial Consortium Request for Information on Arctic Spatial Data by the Polar Data Community March 24, 2016 Overview The polar data community welcomes the recent Request for Information (RFI) on Arctic spatial data interoperability and infrastructure issued by the Open Geospatial Consortium (OGC). The OGC’s interest in this topic is timely since, as we will discuss below: . The polar regions are of increasing interest to the whole world as a result of their linkage to global climate systems, opportunities for economic development, geo-political strategic importance, and their environmental importance as homes to Indigenous populations and other residents and sensitive ecosystems. Polar data are required by the scientific community and residents to support research on topics such as climate, atmosphere, land, oceans, ecosystems, ice and snow, permafrost, and social systems; and by the operations community to support impact assessments, engineering design, safe navigation and operations, risk management, emergency response, weather forecasting, and climate change adaptation. These activities contribute to environmental protection, heritage preservation, economic development, safety of life and property, and national sovereignty. The polar data community is well organized and is pursuing activities to improve data management for all of the diverse members of the polar community. Polar data infrastructure is evolving from a system where data are discovered in data catalogues and downloaded to the local machines of users, to a system of distributed data made interoperable using standards and providing users with storage and computational capacity close to large repositories of data. Indigenous Peoples of the Arctic and their representative organizations are increasingly active in using information and communications technologies (ICT) to access data and share their information and knowledge. -
Backups, Archives & Data Preservation
Protecting Your Data: Backups, Archives & Data Preservation DataONE Community Engagement & Outreach Working Group Lesson Topics Key Digital Preservation Concepts Backups: Things to Consider Data Preservation Recommended Practices Learning Objectives After completing this lesson, the participant will be able to: Define the differences between backups and archiving data Identify significant issues related to data backups Identify why backup plans are important and how they can fit into larger backup procedures Discuss what data preservation covers List several recommended practices The DataONE Data Life Cycle Data Protection, Backups, Archiving, Preservation Differences at a Glance Data Protection Includes topics such as: backups, archives, & preservation; also includes physical security, encryption, and others not addressed here More information about these topics can be found in the “References” section Data Protection, Backups, Archiving, Preservation (continued) Terms “backups” and “archives” are often used interchangeably, but do have different meanings Backups: copies of the original file are made before the original is overwritten Archives: preservation of the file Data Preservation Includes archiving in addition to processes such as data rescue, data reformatting, data conversion, metadata A Closer Look: Backups vs. Archiving Backups Used to take periodic snapshots of data in case the current version is destroyed or lost Backups are copies of files stored for short or near-long-term Often performed on a somewhat frequent schedule -
INLS 752: Digital Preservation and Access
Summer, 2015 Monday-Friday, 8:00-11:30 INLS 752: Digital Preservation and Access The Instructors. Dr. Helen R. Tibbo : 962-8063(w); 929-6248(h) Office: 201 Manning Hall FAX #: (919) 962-8071 : Tibbo at email dot unc dot edu Class Listserv: INLS752- [email protected] Office Hours. I will be available until 12:30 most days. I will probably not be in the office in the afternoons during this intensive course session. Feel free to call me at home in the evening before 9:00 PM. Course Timeline. First Class: Wednesday, May 13, 2015 Last Class: Thursday, May 28, 2015 Final Project Due: Friday, May 29, 2015 Brief Course Description. This course focuses on integrating state-of-the-art information technologies, particularly those related to the digital curation lifecycle, digital repositories, and long-term digital preservation, into the daily operations of archives, records centers, museums, special collections libraries, visual resource collections, historical societies, and other information centers. Issues, topics, and technologies covered will include the promise & challenge of long-term digital preservation and curation; creating durable digital objects, approaches to preservation; development of institutional repositories; image processing; selecting materials for digitization and managing digitization projects; resource allocation and costing, risk management, digitization and metadata; rights management and other legal and ethical issues; digital asset management; standards; file formats; quality control; funding for developing and sustaining digitization projects and programs; and trusted repositories. Goals and Objectives. By the end of the course, the student should be able to: Identify the key events in the history of digital preservation and access. -
Digital Preservation Handbook
Digital Preservation Handbook Digital Preservation Briefing Illustrations by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark Who is it for? Senior administrators (DigCurV Executive Lens), operational managers (DigCurV Manager Lens) and staff (DigCurV Practitioner Lens) within repositories, funding agencies, creators and publishers, anyone requiring an introduction to the subject. Assumed level of knowledge Novice. Purpose To provide a strategic overview and senior management briefing, outlining the broad issues and the rationale for funding to be allocated to the tasks involved in preserving digital resources. To provide a synthesis of current thinking on digital preservation issues. To distinguish between the major categories of issues. To help clarify how various issues will impact on decisions at various stages of the life-cycle of digital materials. To provide a focus for further debate and discussion within organisations and with external audiences. Gold sponsor Silver sponsors Bronze sponsors Reusing this information You may re-use this material in English (not including logos) with required acknowledgements free of charge in any format or medium. See How to use the Handbook for full details of licences and acknowledgements for re-use. For permission for translation into other languages email: [email protected] Please use this form of citation for the Handbook: Digital Preservation Handbook, 2nd Edition, http://handbook.dpconline.org/, Digital Preservation Coalition © 2015. 2 Contents Why Digital Preservation Matters -
A New Digital Dark Age? Collaborative Web Tools, Social Media and Long-Term Preservation Stuart Jeffrey Version of Record First Published: 05 Dec 2012
This article was downloaded by: [University of York] On: 10 December 2012, At: 04:01 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK World Archaeology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/rwar20 A new Digital Dark Age? Collaborative web tools, social media and long-term preservation Stuart Jeffrey Version of record first published: 05 Dec 2012. To cite this article: Stuart Jeffrey (2012): A new Digital Dark Age? Collaborative web tools, social media and long-term preservation, World Archaeology, 44:4, 553-570 To link to this article: http://dx.doi.org/10.1080/00438243.2012.737579 PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. A new Digital Dark Age? Collaborative web tools, social media and long-term preservation Stuart Jeffrey Abstract This paper examines the impact of exciting new approaches to open data sharing, collaborative web tools and social media on the sustainability of archaeological data. -
The DCC Curation Lifecycle Model Higgins, Sarah
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Aberystwyth Research Portal Aberystwyth University The DCC Curation Lifecycle Model Higgins, Sarah Published in: International Journal of Digital Curation DOI: 10.2218/ijdc.v3i1.48 Publication date: 2008 Citation for published version (APA): Higgins, S. (2008). The DCC Curation Lifecycle Model. International Journal of Digital Curation, 3(1), 134-140. https://doi.org/10.2218/ijdc.v3i1.48 General rights Copyright and moral rights for the publications made accessible in the Aberystwyth Research Portal (the Institutional Repository) are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the Aberystwyth Research Portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the Aberystwyth Research Portal Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. tel: +44 1970 62 2400 email: [email protected] Download date: 03. Oct. 2019 134 The DCC Curation Lifecycle Model The International Journal of Digital Curation Issue 1, Volume 3 | 2008 The DCC Curation Lifecycle Model Sarah Higgins, Standards Advisor, Digital Curation Centre, University of Edinburgh June 2008 Summary Lifecycle management of digital materials is necessary to ensure their continuity. -
Web Archiving and You Web Archiving and Us
Web Archiving and You Web Archiving and Us Amy Wickner University of Maryland Libraries Code4Lib 2018 Slides & Resources: https://osf.io/ex6ny/ Hello, thank you for this opportunity to talk about web archives and archiving. This talk is about what stakes the code4lib community might have in documenting particular experiences of the live web. In addition to slides, I’m leading with a list of material, tools, and trainings I read and relied on in putting this talk together. Despite the limited scope of the talk, I hope you’ll each find something of personal relevance to pursue further. “ the process of collecting portions of the World Wide Web, preserving the collections in an archival format, and then serving the archives for access and use International Internet Preservation Coalition To begin, here’s how the International Internet Preservation Consortium or IIPC defines web archiving. Let’s break this down a little. “Collecting portions” means not collecting everything: there’s generally a process of selection. “Archival format” implies that long-term preservation and stewardship are the goals of collecting material from the web. And “serving the archives for access and use” implies a stewarding entity conceptually separate from the bodies of creators and users of archives. It also implies that there is no web archiving without access and use. As we go along, we’ll see examples that both reinforce and trouble these assumptions. A point of clarity about wording: when I say for example “critique,” “question,” or “trouble” as a verb, I mean inquiry rather than judgement or condemnation. we are collectors So, preambles mostly over.