Safeguarding the Nation's Digital Memory
Total Page:16
File Type:pdf, Size:1020Kb
Mathematics for safeguarding the nation's digital memory Martine J. Safeguarding the nation's digital memory Barons, CMath Archives Martine J. Barons, CMath The digital preservation problem IDSS Director Applied Statistics & Risk Unit The National Archives University of Warwick Project Soft Elicitation [email protected] Data SEJ The Tool ECMI21 13th April 2021 Lessons Lockdown Digital Preservation Awards c Martine J. Barons 2021 Overview Mathematics for safeguarding 1 the nation's Archives digital The digital preservation problem memory Martine J. Barons, 2 IDSS CMath Archives 3 The National Archives Project The digital preservation Soft Elicitation problem Data IDSS The National SEJ Archives Project The Tool Soft Elicitation Data SEJ 4 Lessons The Tool Lockdown Lessons Lockdown Digital Preservation Awards Digital Preservation Awards c Martine J. Barons 2021 What are archives? Mathematics for Collections of information known as records safeguarding the nation's Records digital memory letters Martine J. Barons, reports CMath minutes Archives The digital registers preservation problem maps IDSS The National photographs and films Archives Project digital files Soft Elicitation Data SEJ sound recordings The Tool Lessons Records in an archive are primary sources. Archives provide Lockdown Digital Preservation first-hand information or evidence relating to historical events Awards or figures. c Martine J. Barons 2021 Archives are managed by a variety of types of institutions and the materials they collect differ. Mathematics for Archives safeguarding the nation's Government or national archives: materials related to all digital memory levels of government Martine J. Barons, Corporate archives: manage and preserve business records. CMath College and university archives:preserve materials related Archives to the institution. The digital preservation problem Historical societies: preserve materials related to a specific IDSS region, event, or industry. The National Archives Museum archives: diverse collections typically consisting of Project Soft Elicitation artwork or artifacts Data SEJ Religious archives: collect and preserve materials related The Tool Lessons to a faith, denomination or place of worship Lockdown Digital Special collections: a collection of items that are either Preservation Awards irreplaceable or rare, usually in a library. c Martine J. Barons 2021 Archives are managed by a variety of types of institutions and the materials they collect differ. The National Archives Mathematics for TNA safeguarding the nation's Official archive of the UK Government digital memory Preserve the key records created by around 250 Martine J. Barons, government departments e.g. Multiple email threads on a CMath decision around public safety which will have rich Archives embedded metadata, multiple attachments and image The digital preservation footers; AI algorithms used to decide who has the right to problem IDSS remain in the country The National At last count the digital archive held 4759.5TB of records Archives Project Records are text, videos, sound recordings, databases, 3d Soft Elicitation Data models, images etc. SEJ The Tool Have to preserve those records forever Lessons Lockdown Socially important records e.g. London Olympics Digital Preservation Awards Legally important records e.g.Hillsborough 15 April 1989 c Martine J. Barons 2021 Digital Preservation Mathematics The act of preserving digital records is termed digital for safeguarding preservation. There is a wide variety of risks to digital files.The the nation's digital risks are connected and influence one another. memory Martine J. Digital preservation is a relatively new concern, especially Barons, CMath for archivists, many of whom are still working primarily Archives with analogue materials. The digital preservation It takes time to get to a point where professionals know problem IDSS what data they need to start monitoring in order to better The National make decisions. Archives Project Sharing data on risks means admitting something has Soft Elicitation Data gone wrong, which understandably might be something SEJ The Tool many organisations and individuals will be cautious to do. Lessons Lockdown Obsolescence and limited lifespan: Archivists like to say Digital Preservation that digital records last for ever - or 5 years, whichever Awards c Martine J. comes first. Barons 2021 Digital Preservation challenges Mathematics Need to preserve the original bitstream and be able to render it for safeguarding ‘sufficiently’ for use. the nation's digital memory Challenges Martine J. Software - wordstar, bespoke software - emulators Barons, CMath Storage medium - Facebook, Myspace, Google (reduced Archives volume), cloud, account access The digital preservation problem Storage hardware - floppy disc, 8-track, VHS / Betamax, IDSS phone The National Archives Storage life - laptop, flash drive, hard drive, SD card, Project Soft Elicitation magnetic tape, malware corruption Data SEJ Copying errors The Tool Lessons Storage compression - photographs on Facebook or Google Lockdown Digital Preservation Natural disaster or accident - Fire, flood, earthquake - Awards location c Martine J. Barons 2021 Integrating Decision Support systems Mathematics for A formal & defensible statistical methodology to draw together safeguarding the nation's inferences when: digital memory Users are decision Centres Martine J. Barons, Centres motivated to act as a single coherent unit for a CMath common goal Archives Consensus about utility structure to scrutinise efficacy of The digital preservation problem candidate policies IDSS Consensus about an overarching description of dynamics The National Archives driving the process. Project Soft Elicitation Consensus about who is expert about what, to identify Data SEJ appropriate expert panels The Tool Lessons Expert judgements from disparate panels of experts Lockdown Digital Each component panel informed by complex models & Preservation Awards huge data sets c Martine J. Barons 2021 Integrating Decision Support systems Mathematics for safeguarding the nation's digital memory A single, comprehensive probabilistic model is inappropriate Martine J. Barons, infeasibly large CMath no shared structural assumptions so no centre can `own' Archives The digital the full joint distribution preservation problem dynamic revisions lead to fast obsolescence IDSS The National Archives Full technical details in Project Coherent Frameworks for Statistical Inference serving Integrating Decision Soft Elicitation Data Support Systems Jim Q. Smith, Martine J. Barons & Manuele Leonelli, SEJ submitted & on arXiv: 1507.07394 The Tool Lessons Lockdown Digital Preservation Awards c Martine J. Barons 2021 Theorem Mathematics for safeguarding the nation's digital memory Theorem: Martine J. Barons, Suppose an IDSS for a CK class (U, D, S) is adequate where CMath U and D are arbitrary and S includes the consensus that the Archives IDSS is delegable, separately informed, cutting and commonly The digital preservation problem separated at time t. Then it will also be sound and distributed IDSS at time t. Furthermore it is common knowledge that the SB's The National beliefs about each panel's parameter vector are the same as Archives qi Project those of the corresponding expert panel Gi , i 2 [m], for all Soft Elicitation Data d 2 D and at any time t ≥ 0. SEJ The Tool Lessons Lockdown Digital Preservation Awards c Martine J. Barons 2021 Examples of sound and distributive frameworks Mathematics for Staged trees safeguarding the nation's Bayesian Networks digital memory Chain event graphs Martine J. Decomposable graphs Barons, CMath Multiregression dynamic models Archives The digital preservation problem IDSS The National Archives Project Soft Elicitation Data SEJ The Tool Lessons Lockdown Digital Preservation Figure: Manuele Leonelli & James Q. Smith(2015) Bayesian Decision Support for complex systems with Awards many distributed experts Ann Op Res c Martine J. Barons 2021 The National Archives Project Mathematics for safeguarding the nation's digital memory Martine J. Soft Elicitation Barons, CMath Model building Archives Model quantification - Data The digital preservation problem Model quantification - SEJ IDSS Evaluation / feedback The National Archives Software engineering Project Soft Elicitation Launch & adoption Data SEJ The Tool Lessons Lockdown Digital Preservation Awards c Martine J. Barons 2021 Soft Elicitation Mathematics An essential starting point with any problem is to interact with for safeguarding problem-owners, their advisers, experts and close stakeholders the nation's digital to understand their perspectives, views, values, uncertainties, memory worldview Martine J. Barons, CMath Structure the problem Archives What are the processes, inputs, outputs, actors, The digital preservation perceptions of cause and effect? problem IDSS How do they interact? The National What are the specific objectives, uncertainties, challenges Archives Project to address? Soft Elicitation Data SEJ How might these be modelled? The Tool What relevant data and expertise are available? Lessons Lockdown Digital How should outputs be presented? Preservation Awards Iterate c Martine J. Barons 2021 Soft Elicitation with TNA Mathematics for safeguarding Problem identification the nation's digital Digital preservation is a mission to send messages to the memory future Martine J. Barons, CMath Those messages to be faithfully transmitted, to retain their meaning and to be useful for generations to come. Archives