4/5/2018
Technical Support
Outsourcing Audiovisual If you experience any Digitization technical difficulties
Session 2 during the class, call 800-999-8558 ext 3971 Annie Peterson, LYRASIS Preservation Services Librarian
lyrasis.org
File naming
• Provide schema for vendor • Create names that will allow useful sorting • Keep names as short as possible Copyright Guidelines from Video At Risk: • Include only alphanumeric characters Strategies for Preserving Commercial Video • Avoid spaces, abbreviations, and most symbols except _ and • Do not use & % ( ) * etc. Collections in Libraries • Format dates to enhance sorting • Year_month_day http://archive.library.nyu.edu/preservation/VAR • 2016_02_23 RFP.pdf • Use file name for version control • Document file naming rules • Unique • Consistent throughout project, across institution • http://bentley.umich.edu/giving/donate-your-archives/digital- recordkeeping-best-practices-for-directory-and-file-naming/ lyrasis.org lyrasis.org
1 4/5/2018
File directory Directory structure example
• Organize files into folders Peterson oral history project • Follow file naming tips for folders • Peterson-condition-report.pdf • Peterson-metadata.xls • No special characters • Peterson-metadata.xml • No spaces • Peterson-checksums.md5 • Easily sorted Peterson_0001 • Document file directory structure Access • Consistency across projects, institution • 0001.mp3 • 0001.mp3.md5 • Communicate file directory structure to vendor at beginning Preservation of project • 0001.wav • 0001.wav.md5 Images
lyrasis.org lyrasis.org
Recap Recap
• Plan your project • Obtain quote(s) • Assess internal capabilities • Choose a vendor • Assign responsibilities • Prepare items for shipping • Establish workflows • Ship items • Assess and inventory collections • Format identification • Condition • Define technical specifications • Metadata • File naming • ….more to be discussed in session 2
lyrasis.org lyrasis.org
2 4/5/2018
Agenda Agenda
• Session 1 • Session 2 • Project planning • Capture guidelines • Selection and • Film to film preservation prioritization • Quality control • Choosing and working • Delivery and storage with a vendor media • Preparing and assessing • Digital preservation a quote • Access • Digitization preparation • Funding • Metadata
lyrasis.org lyrasis.org
Capture
Internal guidelines guide vendor specifications • Follow best practices for preservation • Discuss needs with vendor • Balance with internal capabilities – staffing, funding, storage • Access platforms • Source material • Consistency across projects, institution
Capture
lyrasis.org lyrasis.org
3 4/5/2018
Capture Capture
• Equipment should be thoroughly, regularly cleaned • Playback equipment should be calibrated as necessary
Wilson Library, UNC Chapel Hill
lyrasis.org lyrasis.org
Capture Capture
Azimuth • Measure of the angle between the tape heads and the physical tape itself • Ideal is 90°, but depends on the recording equipment • Playback head must be aligned at the same angle as the original recording itself • Improper alignment leads to loss of higher frequencies • Vendor should adjust azimuth for transfer • Ask for information
http://www.indigitization.ca/indigitization-toolkit/audio-digitization/audio- “Azimuth Adjustment for Magnet Audio Recordings,” Audrey Young and Peter Oleksik cassette-tape/digitization-guide-audio-cassette/ https://www.avpreserve.com/uncategorized/azimuth-adjustment-for-magnetic-audio-recordings/ lyrasis.org lyrasis.org
4 4/5/2018
Capture Capture
• Vectorscope • Waveform monitor Parallel • Calibrate to known reference Multiple • Bars/tone on tape • No bars or tone – adjust Semi-monitored to broadcast legal limits • https://library.nyu.edu/pr eservation/VARRFP.pdf Full monitoring Image Stanford Media Preservation Lab Single Ingest
lyrasis.org lyrasis.org
Capture Capture guidelines: audio
Minimum Digitization Capture Recommendations, Association for Library • No corrections on Collections and Technical Services Preservation and Reformatting Section preservation masters • Add metadata during Min resolution Min bit depth capture? Audio 96 kHz 24 • Head to tail capture?
Stanford Media Lab
lyrasis.org lyrasis.org
5 4/5/2018
Capture guidelines: audio Capture guidelines: audio
• Sample at 2 x highest Sampling frequency • Human voices between 50 – 20 kHz • Minimum sample rate = 44.1 kHz • Wider range for musical performances • Minimum sample rate = 96 kHz
lyrasis.org lyrasis.org
Capture guidelines: audio Capture guidelines: audio
Audio file size calculator • Bit depth = number of bits of information in each sample https://www.lyrasis.org/LYRASIS%20Digital/Documents/Digital%20Toolbox/audiofile sizeprojection.xls • CD = 16 bits per sample • Best practice for digitization = 24 bits • Higher sample rate = wider dynamic range, higher signal to noise ratio
lyrasis.org lyrasis.org
6 4/5/2018
File formats Deliverables
• File format • Preservation considerations • Largest file • Uncompressed, or least compressed • Open • Mezzanine • Non-proprietary • Not always necessary • Widely adopted Broadcast • Large file used to edit or create derivatives from • Metadata support • Access • Works with your access Wave • Streaming or other access platform • Most compressed • Matches with your in- (BWF) • Not for long-term preservation house technical capabilities • Established best practices lyrasis.org lyrasis.org
Deliverable package example Deliverables
• Preservation master • BagIt / Bagger • Broadcast wav • https://sourceforge.net/pr • Encoded with PCM ojects/loc- • 96/24 xferutils/files/loc-bagger/ • Access file • Exactly • MP3 • Encoded • https://www.avpreserve.c • 44.1/16 om/tools/exactly/ • Images • Package according to • Condition report a specification for safe • Metadata transfer and • Embedded management • XML file
lyrasis.org lyrasis.org
7 4/5/2018
Deliverables package Capture guidelines: video
• BagIt specification, Bagger: Minimum Digitization Capture Recommendations, Association for Library Collections and Technical Services Preservation and Reformatting Section https://github.com/LibraryOfCongress/bagger Min Min bit Notes Resolution depth
Analog NTSC Video 720 x 486 8 Use 10-bit whenever possible
Digital Video Source Tape, Native Native Native where possible to access bits Digital Video Source Tape, Decompresse 8 or 10 where not possible to d 720 x 486 access bits
Digital Video File Native Subject to file format obsolescence evaluation. If deemed obsolete, decompress to 10-bit native raster (horizontal x vertical pixel count) Video Optical Disc Native Native lyrasis.org lyrasis.org
Capture guidelines: video Capture guidelines: video
• Best practices range • Recommended delivery specification, analog source, NTSC / • Depends on source material 525 line • Method: playback into encoder, wrap output as file • Wide range of video formats • Wrapper: .mov (QuickTime) or .avi • “Refining Conversion Contract Specifications: Determining • Video: Suitable Digital Video Formats for Medium-term Storage” • Video compression: uncompressed • http://www.ala.org/alcts/sites/ala.org.alcts/files/content/resources/prese • Frame size: standard definition: 720x486 rv/images/Appendix%20V.pdf • Frame size: high definition: native • Aspect ratio: Native: 4:3 for SD; 16:9 for HD • Bit depth: 10-bit • Color Space: YcbCr • Chroma subsampling: 4:2:2 • Interlaced/Progressive: Native • Frame rate: Native, 30 or 29.97
lyrasis.org lyrasis.org
8 4/5/2018
Capture guidelines: video Capture guidelines: video
• Recommended delivery specification, analog source, NTSC / • Recommended delivery specification, digital source, 525 line (cont’d) NTSC/525 line • Time code: Native if present; midnight start and Non Drop Frame • BeacamSX (NDF) if synthetic • XDCAM • Audio: • DV- (all) • Audio channels: same as original • Method: transfer data to file wrapper without transcoding • Audio compression: uncompressed, PCM • Audio sample rate: 48kHz • Wrapper: native-as-associated with the underlying encoded • Audio resolution: 24-bit essence (such as .dv, .imx, .mpeg, .mp4, etc.) or .mov (QuickTime) or .avi. Otherwise wrap in Quicktime.
lyrasis.org lyrasis.org
Capture guidelines: video Capture guidelines: video
• Recommended delivery specification, digital source, • Recommended delivery specification, digital source, NTSC/525 line (continued) NTSC/525 line (continued) • Video • Audio • Video compression: native • Audio channels: same as originals • Frame size: native • Audio compression: native, typically uncompressed, PCM • Aspect ratio: Native: 4:3 for SD; 16:9 for HD • Audio sample rate: native, typically 48 kHz • Bit depth: Native, 8-bit or 10-bit • Audio resolution: native, typically 24 or 16 bit • Color Space: Native, YCbCr • Chroma subsampling: Native, typically, 4:2:2, 4:1:1, or 4:2:0 • Interlaced/progressive: native • Frame rate: native • Time code: Native when present; midnight start & NDF if synthetic
lyrasis.org lyrasis.org
9 4/5/2018
Capture guidelines: video Compression
• High definition vs. standard definition • Uncompressed • SD: 640 x 480 pixels / samples • Large files • HD: 1280 x 720 or 1920 x 1080 pixels / samples • WAV, Waveform Audio File Format • AIFF • Video: Native – capture HD formats in HD, SD in SD • JPEG2000 • Film: HD • Uncompressed 10-bit • Lossless compression • FLAC, Free Lossless Audio Codec • ALAC, Apple Lossless Audio Codec • Lossy compression • MP3 • AAC • WMA • DV • H.263 • MPEG4 lyrasis.org lyrasis.org
Video Compression Codecs
http://clockwiseproductions.com/wp-content/uploads/2016/05/From-www.videomarker.com_.png
lyrasis.org lyrasis.org
10 4/5/2018
Codecs and wrappers Codec selection
1. Adoption 2. Disclosure 3. Transparency 4. External dependencies 5. Documentation and metadata 6. Pre-planning 7. Maintenance 8. Obsolescence monitoring 9. Maintaining the original 10. Avoid unnecessary transcoding or re-encoding
A Primer on Codecs for Moving Image and Sound Archives & 10 Recommendations A Primer on Codecs for Moving Image and Sound Archives & 10 Recommendations for Codec Selection, for Codec Selection, Chris Lacinak, AVPreserve Chris Lacinak, AVPreserve http://www.avpreserve.com/wp-content/uploads/2010/04/AVPS_Codec_Primer.pdf http://www.avpreserve.com/wp-content/uploads/2010/04/AVPS_Codec_Primer.pdf lyrasis.org lyrasis.org
Video Codecs File formats: video
• WMV (Windows Media Viewer) – proprietary • Preservation master file formats • DivX – proprietary • Uncompressed 8-bit or 10-bit Quicktime MOV • Uncompressed 8-bit or 10-bit AVI • Sorenson (used in Quicktime) – proprietary • Lossless JPEG 2000 • Xvid – open source • Native file format (born digital) • FFV1 • Intermediary file formats • MediaConch • DC25, DC50, DVCPro50 • https://mediaarea.net/MediaConch/about.html • Apple Pro Res 422 • Access file formats • MPEG2 • H.264 MPEG4
lyrasis.org lyrasis.org
11 4/5/2018
Motion JPEG 2000 File formats: audio
• Lossless video compression • Audio • Embedded metadata capabilities • Preservation master: • Large file size Broadcast wav • Past recommendations towards it with MXF wrapper • Access copy: mp3 • Continue to monitor!
Image from the Preservation Self Assessment Program
lyrasis.org lyrasis.org
Film to film preservation
• Physical copy for long- term preservation • Access challenge • More expensive than just digitization • Materials • Chemicals • Wet gate transfer demonstration Film to film preservation • https://www.youtube.com/wat ch?v=2AXq54r6YGM
lyrasis.org lyrasis.org
12 4/5/2018
Film to film preservation
• NFPF Preservation Grants • New film preservation elements • Two public access copies, one of which must be a film print • Closed captioning for sound films destined for online or television exhibition • http://www.filmpreservation.org/nfpf-grants/basic-preservation-grants
Quality control
lyrasis.org lyrasis.org
Quality control Quality control
• What are vendor’s procedures for quality control? • Checksums • Review digitized content? • Vendor should generate • QC metadata? • MD5 or SHA 256 • Calibration? • Verify checksums after data transfer • Monitoring transfers? • Fixity: https://www.avpreserve.com/tools/fixity/ • What are vendor’s procedures for corrections? • BagIt specification, Bagger: • Timelines for re-transferring materials? https://github.com/LibraryOfCongress/bagger • When are originals returned? • When are files deleted?
lyrasis.org lyrasis.org
13 4/5/2018
Quality control Quality control
• Dedicate staff • Establish a quality control station responsibility • Control light around viewing stations • Compare originals and digitized version? • Incorporate in workflow • Playback equipment required from beginning • Expertise with handling and playback • Headphones • Use tools to automate • Over the ear • Professional quality
lyrasis.org lyrasis.org
Quality control Quality control
Checklist for assessing materials Checklist for assessing materials (continued) • Checksum verification • Compare vendor condition report and digitized materials • File format verification • Capture is complete and consistent • Content and metadata match • Accurate configuration (mono / stereo) • Metadata is complete and accurate • No digital artifacts introduced in the process • File naming schema was followed • AV Artifact Atlas http://avaa.bavc.org/artifactatlas/index.php/A/V_Artifact_Atlas • File directory structure accurate • All files expected are present
lyrasis.org lyrasis.org
14 4/5/2018
Quality Control Quality control
• MDQC • Download https://www.avpreserve.c om/tools/mdqc/ • Automate quality control of metadata
lyrasis.org lyrasis.org
Quality control
Delivery and storage media
lyrasis.org lyrasis.org
15 4/5/2018
Delivery and storage media Delivery and storage media
CD/DVD • Easy to use and share • Poor long-term storage option • Mechanical failure • Chemical deterioration • Limited storage space • Obsolete?
http://www.cdfreaks.com/news/6450-CD-Recordable-discs-unreadable-in-less-than-two-years.html Photo by Brian Teutsch / CC BY 2.0 lyrasis.org lyrasis.org
Delivery and storage media Delivery and storage media
Flash drive External hard drive • Easy to use and share • Rotating disc or solid • Not designed for long- state term storage • Large storage capacity • Easily lost or mishandled • Good transport • Can be cheaply made method, not appropriate for long- term storage
Photo by AmsterdamPrinting.com/ CC BY 2.0 Photo by walknboston / CC BY 2.0 lyrasis.org lyrasis.org
16 4/5/2018
Delivery and storage media Delivery and storage media
Tape Servers • LTO tape (linear tape- • Locally maintained open) servers • RAID • High storage capacity • Backups and • Relatively stable maintenance required • Migration required • Good long-term storage • Not backwards choice compatible
Photo by Victorgrigas/ CC BY 2.0 • Expertise required LTO-2 cartridge
lyrasis.org lyrasis.org 66
Delivery and storage media Discussion
Cloud storage • Ease of use • Fine print • Fees How will you store digitized • Cloud storage vendor profiles, AVPreserve audiovisual materials at your • https://www.avpreserve.co institution? m/avpsresources/papers- and-presentations/
Photo by Kris / CC BY 2.0 lyrasis.org lyrasis.org
17 4/5/2018
Digital preservation
Digital preservation combines policies, strategies and actions to ensure access to reformatted and born digital content regardless of the challenges of media failure and technological change. The goal of digital preservation is the accurate rendering of Digital preservation authenticated content over time. http://www.ala.org/alcts/resources/preserv/defdigpres0408
lyrasis.org lyrasis.org
Digital preservation Digital preservation
Challenges with audiovisual materials • File size • Long-term storage • Sustainable budget for storage • Migration and maintenance time • Rights management
Essential Optimal • Reasonable given a limited set of • Reasonable to expect given ample resource and expertise set of resources and expertise • Non-negotiable • Ensure the most reliable long-term preservation
https://educopia.org/sites/educopia.org/files/publications/Guidelines_for_Digital_Ne wspaper_Preservation_Readiness_0.pdf
lyrasis.org lyrasis.org
18 4/5/2018
Digitize with preservation in mind Digitize with preservation in mind
• High quality preservation masters • Best possible storage media for preservation masters • Description • Backups from the start • Embed metadata • At least 3 copies, at least 1 different geographic location • Preservation metadata • Incorporate QC measures into long-term management • Technical • Checksum verification • Structural • Verifying newly ingested materials • Descriptive • Communicate costs and need for ongoing administrative • Organization support • File naming • File directory
lyrasis.org lyrasis.org
Digital preservation Digital preservation
Plan for management Establish policies and procedures over time • Digital collection development policy • Collaborate with IT • Digital preservation policy • Schedule maintenance, • Disaster plan perform checks on materials • Authenticity • Bit rot • Migrate from obsolete formats and technology • Ingest procedures for new collections Image
lyrasis.org lyrasis.org
19 4/5/2018
Digital preservation
• Systems and tools • Automation of functions: • Format checks • Checksum verification • Rights management • DuraSpace • MetaArchive • ArchiveMatica • Preservica Access
Photo by Lachlan Donald / CC BY 2.0 lyrasis.org lyrasis.org
Access Access
• Description Louisiana Digital Media Archive http://ladigitalmedia.org/video_v2 • Digital asset management system • Collection management system • Local or hosted solutions • Promotion and outreach
https://archive.org/details/middfilms_clip_a8ff.1950__scie nces
lyrasis.org lyrasis.org
20 4/5/2018
Access Access
Moving Image Preservation of Puget Sound https://archive.org/details/mipops Southern Folklife Collection, Wilson Special Collections Library, UNC Chapel Hill http://www2.lib.unc.edu/mss/inv/s/Seeger,Mike.html
lyrasis.org lyrasis.org
Access Discussion
Southern Folklife Collection, Wilson Special Collections Library, UNC Chapel Hill http://dc.lib.unc.edu/cdm/singleitem/collection/20009/id/2710/rec/1 How will you provide access to audiovisual materials at your institution?
How will you promote your digital project?
lyrasis.org lyrasis.org
21 4/5/2018
Funding
• In-house staff • Processing • Project management • Quality control • Outsourcing costs • Capture • Description • Access • Digital Asset Management System Funding for digitization • Digital preservation • In-house staff • Training • Vendor solutions • Storage lyrasis.org lyrasis.org
Funding Funding
• Internal funding • Grammy Foundation grants • “support efforts that advance the archiving and preservation of the • Organization or music and recorded sound heritage of North America” individual that created • Applications in the fall, grants awarded in the spring the materials • Archiving and Preservation Projects • Preservation implementation: $20,000 maximum • Donors • Assistance, assessment, consultation: $5,000 maximum • National or local • https://www.grammy.org/grammy-foundation/grants foundations • Federal grants
Image from flickr user khrawlings / cc by 2.0 lyrasis.org lyrasis.org
22 4/5/2018
Funding Funding
• National Film Preservation Foundation Preservation Grants • National Endowment for the Humanities, Humanities • Awarded to nonprofit and public institutions for laboratory work to Collections and Reference Resources preserve culturally and historically significant film materials • Foundations grant • $1,000 - $18,000 in cash and/or laboratory services • “identifying and prioritizing humanities materials for digitization, • Registration in January, application February, grant period starts in developing project-specific selection criteria, evaluating technical June requirements for digital preservation and access, reformatting test-bed • Require film preservation masters items, and/or exploring third-party service arrangements” • http://www.filmpreservation.org/nfpf-grants/basic-preservation-grants • $40,000 for up to two years • July deadline for projects beginning Spring of following year • http://www.neh.gov/files/grants/humanities-collections-reference- resources-july-21-2015-edit-2.pdf
lyrasis.org lyrasis.org
Funding Recap
• Council on Library and Information Resources, Digitizing • Establish internal policies to guide vendor specifications Hidden Collections • Capture specifications • “digitize and provide access to non-digital collections of rare or unique • Deliverables content in cultural heritage institutions” • Metadata • Up to $50,000 • Create quality control procedures, learn about vendor • Spring deadline, dependent on CLIR Mellon funding procedures • http://www.clir.org/hiddencollections/applicants/applicants.html • Select appropriate media for delivery and storage • Plan for long-term preservation of digitized content • Consider access plans throughout project • Funding
lyrasis.org lyrasis.org
23 4/5/2018
Thank You! Phone 678 235 2923 Email [email protected]
lyrdigpres lyrasis
Annie Peterson LYRASIS Preservation Services Librarian
Course Evaluation http://www.lyrasis.org/classes-and-events/Pages/Training-Options.aspx
24