A Short Guide to Choosing a Digital Format for Video Archiving Masters | S
Total Page:16
File Type:pdf, Size:1020Kb
A short guide to choosing a digital format for video archiving masters | S... hps://www.scart.be/?q=en/content/short‐guide‐choosing‐digital‐forma... EN FR NL Contact Home Articles Interviews Case studies Guidelines Projects Blog A short guide to choosing a digital format for video archiving masters Author: EmanuelEmanuel LorrainLorrain (PACKED(PACKED vzw)vzw) Publication date: MarchMarch 20142014 Hundred of thousands of hours of audio-visualaudio-visual materialmaterial areare stillstill being held by Flemish cultural heritage institutions and broadcastbroadcast archives on alalreadyready – or soon to become – obsolete carriers. From the end of 2013 the Flemish Institute for Archiving (VIAA)1 willwill operateoperate asas aa serviceservice provider who will organise the digitisation and storagestorage of audio-visual contents for owners and caretakers. The digital files prodproduceduced will ultimatelyultimately replace old tape-based formats and become thethe newnew archivingarchiving mastersmasters2.. File-based video formats have brought a number ofof newnew termsterms (wrapper,(wrapper, codec,codec, compression,compression, etc.) and facets to video preservation that have toto be learned byby collecticollection caretakers. Confusion regarding technologies can cause heritage institutionsions toto bebe reluctantreluctant aboutabout entrustingentrusting their collections and devoting their reresourcessources to large-scalelarge-scale digitisatidigitisationon projects. In such a context choosing the destination format and specifications is always a very complex phase due to the lack of a real consensus in the archival world as toto whichwhich formatsformats and specificationsspecifications should be used forfor thethe long-termlong-term preservationpreservation ofof video.video. ThisThis dedecision is, however, a critical step that will have consequences on the future use and accessibility of the digitised content. In the framework ofof thethe preparationpreparation forfor thethe VIAA digitisation projects, PACKED vzw has conducted some research, lookinglooking at common prpracticesactices in broadcast organisations and audio- visual archives in order to see whwhatat would be the best solution for the digitisationdigitisation of audio-visual collections in Flanders' cultural heritage institutionsinstitutions.. TheThe followingfollowing documentdocument givesgives anan overviewoverview of the different elements that should be taken into account when choosingchoosing thethe destinationdestination formatformat and related specifications, listinlistingg the various options available. 1. Video formats 1.1 Codecs and containers Video files are composed of different data streams encapsulated inin aa containercontainer oror wrapper.wrapper. VideoVideo andand audioaudio signalssignals are encoded using codecs. A codec is a piece of hardware or softwaretware neededneeded toto encodeencode aa datadata streamstream oror signalsignal forfor transmission,transmission, storagestorage oror encryptionencryption andand toto decodedecode itit forfor plplayback or other purposes such as editing. Codec is a 'portemanteau''portemanteau' termterm constructedconstructed fromfrom thethe wordswords coding/decoding.coding/decoding. TheThe termterm 'codec''codec' isis commonlycommonly usedused toto referrefer directly to a coding or compression format. Video and audio essences (the bit streams) can be encoded with different codecs, with or without compression. Some examples of codecs for video are: H264, MPEG2, JPEG2000, IV41, Cinepak and Sorenson. InIn orderorder toto createcreate aa videovideo filefile readablereadable byby computercomputer software,software, thethe encodedencoded videovideo andand audioaudio streamsstreams areare wrappedwrapped togethertogether inin aa containercontainer withwith aa numbernumber ofof otherother datadata streamsstreams sucsuch as descriptive metadata and subtitles. The number, type and variety of data streams that a container can hold are specific to the container format used. Some examples of containers for video are: AVI, MOV, MP4, WMV and MXF. 1.2 Uncompressed video, lossless and lossy compression As mentioned, audio and video can be encoded with or without compression. In an uncompressed video file, the entire information of the digitised source is captured and encoded without any compression. Uncompressed video leadsleads toto veryvery bigbig filesfiles requiringrequiring importantimportant storagestorage capacitycapacity whwhen great amount of content needs to be digitised. In order to generate smaller file sizes and bit rates, video compression is used to re-encode the original content differently. Compression codecs can be lossless or lossy. When using a lossless codec, a bit-identical copy of the data can be achieved (as in an uncompressed file). When using a lossylossy codec,codec, thethe entiretyentirety ofof thethe datadata isis notnot 1 of 9 8/23/2018, 4:09 PM A short guide to choosing a digital format for video archiving masters | S... hps://www.scart.be/?q=en/content/short‐guide‐choosing‐digital‐forma... maintained. Video compression can be brought about using different methods and algorithms (wavelet, motion compensation, discrete cosine transform or DCT). Compression methodsthods areare usuallyusually divideddivided intointo threethree mainmain categories: lossylossy compression;compression; visually lossless compression; mathematically lossless compression. With lossy compression a number of bits are removed in order to reducereduce thethe sizesize ofof thethe videovideo file.file. MostMost ofof thethe time,time, thisthis isis donedone byby reducingreducing thethe amountamount ofof colourcolour information.information. ThisThis processprocess impliesimplies thatthat aa partpart ofof thethe image,image, andand detailsdetails of its chrominance (the chroma sub-sampling and the colour bit depth) and also sometimes its luminance are lost permanently. MPEG-2/D10, Apple ProRes, DVCPro and H264 are examples of codecs performing lossy compression. The majority of digital cameras capture video natively with lossy compression codecs; lossy compressed formats are always used for production and for access (a.o. web, TV and DVD). Manufacturers sometimes label technically 'lossy' compression schemes as 'visually lossless', because the difference between the compressed video and the original is supposed to be imperceptible to the (common) human eye. Despite its name, 'visually lossless' is a compression method in which a part of the data is permanently discarded. For this reason, 'visually lossless' is also sometimes more accurately defined as 'near-lossless compression'. In the remainder of this document, thethe termterm 'loss'lossy' will also be used to refer to 'visually lossless' compression. 'Mathematically'Mathematically lossless'lossless' compressioncompression isis alsoalso aa methodmethod usedused toto reducereduce thethe sizesize ofof aa file,file, butbut herehere thethe encodedencoded datadata remainremain exactlyexactly thethe samesame onceonce itit isis decoded.decoded. InIn 'real'real losslesslossless ccompression', no information is lost. The file size is reducedreduced byby representingrepresenting exactlyexactly thethe samesame informationinformation moremore conciconcisely, using for instance statistical redundancy. Lossless compression codecs can't achieve the same compression ratiosratios asas lossylossy (and(and 'visually'visually lossless')lossless') codecscodecs but they result in smaller files than uncompressed video while retainingretaining thethe entireentire information.information. InIn thethe remainderremainder ofof thisthis documentdocument 'lossless''lossless' willwill bebe usedused toto referrefer toto 'mathematical'mathematicallyly losslesslossless compression'.compression'. 1.3 Compression ratios Data compression ratio is the ratio between the uncompressed size of a file and its compressed size. Different compression algorithms and methods produce different compression ratios. The examples below show the differences in storage space required when lossy, lossless and uncompressed video codecs are used: uncompressed (e.g., v210) 10-bit -> approx. 100GB per hour of video;ideo; losslesslossless compressioncompression (FFV1(FFV1 andand JPEGJPEG 2000)2000) 10-bit10-bit ->-> approx.approx. 45-45-50 GB per hour of video; lossylossy compression;compression; MPEG 2 (50 Mbps) -> approx. 25 GB per hour of video; DV (DV25) -> approx. 12 GB per hour of video; MPEG 2 (DVD quality) -> approx. 3.6 GB per hour of video. 2. Choosing a format for long-term preservation 2.1 Differences between broadcast and heritage archives Broadcast and cultural heritage sectors often have different views on how audio-visual material should be preserved. While both are entitled to preserve and make audio-visualisual heritageheritage accessible,accessible, theythey dealdeal withwith differentdifferent typestypes andand quantitiesquantities ofof audio-visualaudio-visual material.material. ThisThis leadsleads toto didifferentfferent requirements,requirements, viewsviews andand approachesapproaches onon whatwhat preservation means and on how to do it. Within the context of VIAA,IAA, thethe materialmaterial toto bebe digitiseddigitised comescomes fromfrom aa widewide rangerange ofof mutuallymutually differentdifferent institutions,institutions, withwith approximatelyapproximately seseventy per cent originating from the broadcast sector (public,(public, commercialcommercial andand locallocal televisions)televisions)