The Digital Dilemma-Issues in Archiving

THE DIGITAL DILEMMA

STRATEGIC ISSUES IN ARCHIVING AND ACCESSING DIGITAL MOTION PICTURE MATERIALS

THE SCIENCE AND TECHNOLOGY COUNCIL OF THE ACADEMY OF MOTION PICTURE ARTS AND SCIENCES

THE DIGITAL DILEMMA STRATEGIC ISSUES IN ARCHIVING AND ACCESSING DIGITAL MOTION PICTURE MATERIALS Table of Contents

S FOREWORD

T 1 EXECUTIVE SUMMARY 1

N 2 ARCHIVING 3

E 2.1 History and Characteristics of Hollywood Film Archiving ...... 5

T 2.2 Current Hollywood Film Archiving ...... 6 2.3 Hollywood Archives vs. Libraries...... 7 N 3 THE TRANSITION TO DIGITAL 8 O 3.1 Audio Converts First ...... 8 C 3.2 Visual Effects and Animation...... 10 3.3 Postproduction...... 10 F 3.4 Exhibition ...... 12

O 3.5 Acquisition...... 12 3.6 The Impact of Digital Technology on Motion Picture Archiving...... 13 3.7 Television...... 19 E

L 4 CURRENT PRACTICE • Other Industries 20

B 4.1 Corporate America...... 21 4.1.1 Sarbanes-Oxley Act Requirements ...... 21 A 4.1.2 Oil Exploration...... 21

T 4.2 Government and Public Archives in the U.S...... 22 4.2.1 Library of Congress...... 22 4.2.2 National Archives and Records Administration...... 23 4.2.3 Department of Defense ...... 23 4.3 Medical...... 24 4.4 Earth Science...... 26 4.5 Supercomputing ...... 28 4.6 Summary ...... 29 4.6.1 Consensus View ...... 29 4.6.2 Unresolved Issues...... 30

5 ARCHIVING IN THE CHANGING ENVIRONMENT 31 5.1 Digital Storage Technology...... 32 5.2 Risks and Threats to Digital Data...... 35 5.3 Digital Preservation Strategies...... 38

6 DIGITAL MOTION PICTURE ARCHIVING ECONOMICS 40 6.1 Digital Storage Economics ...... 40 6.2 Digital Motion Picture Storage Economics ...... 42 6.3 What this Means for the Motion Picture Industry ...... 45 6.3.1 Economics of Archiving are Changing...... 45 6.3.2 Save Everything ...... 46 6.3.3 Don’t Save Everything...... 46 6.3.4 Who Decides, How, and When? ...... 46 Table n m a O © n o s

e d t 2 9 8 7 11 10 c r

0 a a c a

h r 0 d s

a s u e 7 n t q b

a INDUSTRY A t APPENDIX CONSENSUS u s t a ACKNOWLEDGMENTS t u c a REFERENCES b i a t e c of i u l d y t i t t t

e Table Table A.2 Table Table Table Table Table Table A.1 8.3 8.2 8.1 e e y o m

a f

f o t y r i h t e r

Contents n o

e a r e f

e d

c s M g o v s A-5 A-3 A-8 A-7 A-6 A-4 A-2 A-1 i i n

o s c f t o t e t e e i r

o n o

r Case Generic Finally… Long To a n e t – f NEEDS –

– – – – – p – – h a P

a n e t Case i Estimated Estimated Estimated from Storage from Storage Delivered Delivered c Delivered r r Start r

a a t t e u i d t i c n t r Term e Study u o e

m a l r

a A n r r a e a a e

r Element ...... r

p t y n k Study s u ...... Data Film . AND s o

r a , Categories Categories t

p Initiatives n

a Data g o n d u Sound Film s Data Annual Annual Annual d

a e S

Capture r

t c a a Capture h i e n n e OPPORTUNITIES n Data d t

Trees Tables O e c Capture

n e e Capture s o d s c Elements...... n

, a (

- a Cost Cost Cost r A ...... i

n n s for for . d M f t Production r a

...... Production i A n t . P ...... u g . M . Picture Picture e A e of of of Picture t m . . t Picture S P e e ) .

Sound Element Element A t . n

h S . t S e . e

c A e o o x n and and n p p Elements y d y ...... r

Elements

l r e ...... Element p e i s g g r s h i a Storage Storage n Sound Sound l t y l t e

i i d n n d i g f

s p o , c

r r F l o m a e p i b a m e ...... Storage Elements Elements r t r u i s t o – –

a y a n r , l Data Film

y l o c

w o f 2

n A 0 a t r 0 . a M r a 8 i Capture n ...... n Capture .

P e

t “ d . i O A e

h s . s S , e c

. i r a

e r T c i , n ” l h u

e “ ...... i d s A

i ...... a

n n c c g o a c

d t u t

h e l r e a e m g c

w a y y l

, a

a c r d w r o a v m 6 49 5 70 68 69 66 74 58 65 59 56 62 62 63 a 5 5 7 n i c r t p 4 4 d i 1 1 1 e l e , e , ”

s t a

a e o n n n f d

d e

i s

s t s h

T A B L E O F C O N T E N T S F O R E W O R D T H E D I G I T A L D I L E M theaters sound effects, benefits D very to make consequences motion digital with on p – impact appropriate marked government preservation senior convening Archival Herein make surprisingly danger to, the or well) production seem the and industry M r A from I e

2 preserve, hybrid G s the a 0 standard most The This To However, As e Foreword its 0 much r I company’s respect 7 studio not v T about decisions

technologists editing i assets, an issues. long-term exists n date mastering of studio A lies pictures, acceleration and for Science g has Committee project L evangelical to

systems organization the m

a within the T once the have planning, encouraged there asset. how have e and “summit” to other of not in E that d is

mentioned digital the and executives This C i Digital long-term c moving that Hollywood. motion the cinematic a H and originated yet other it the clear to l thoroughly and implications in changes have , to the N platforms,

mixing; leaves m first report, preserve from produce had in O an about those Academy of revolution. Technology i l Academy’s or final consequences i L organizations. Dilemma. that historically t been the the images effort picture a O in by a and r even above. the the y storage G have assets chance of a

and use the defining the two color Academy’s a would Y

no n series their The unfortunate explored technology theaters the the

Hollywood to d in I is as of new and vendors

S in industry. tended years g definitive

stay

is some appropriately e grading; But digital analog A a mission. Council of digital long-term o to cinematic the of concerned for clearly L whole. g recorded world the and r for R step on a Academy ago

because and continually That the p E the Science cases to h of A issues the era the photochemical systems access i department

c D back when studios financial As arrive going studies in first of begins subsequently digital question a

Even l Y summit is art

by curl preservation holdings, sound d sound evidenced digital

with the P a not that time and and concerned t studies. to of a R Phil a piecemeal, to was

of some full O its and technologies comparing – business increasing digital motion approaching the

and the be D consider Technology a capture, the of by bring motion led life Feiner, n not understanding heads U those d an what and art of universities, motion

cultural C digital in c to surveyed systems as o motion I ongoing of, being the primarily together l rather N pictures, image the decisions l a about e and to charged happens pictures G chair c sound the long-term and digital current t artists

e those wave – realization S picture d – accompanied consequences. so Council, I that digital

the than capture, convenient d G pictures experts how it’s of concern, e rapidly archivists N effects, t the this distribution with who – a with charged that industry of the sometimes to costs have I here. i l F the an much e industry the (if U.S. revolution d I a study C Digital

have in companies the effort i digital proposed the that visual and business n all long A that of and potential f and the N o of access with and cultural digital goes r may by T falls m other been the the been not them

field faces yet a to t i o of a n Foreword D

While there have been several well-researched and informative papers on the R problems associated with digitizing existing media archives and on digital data preservation in general – and we liberally reference some of those works here – none O have examined the topic from the unique perspective of the Hollywood studios, a W perspective that developed over a 100-year period. From this perspective, it’s clear that a totally committed, binding switch to E

digital has one major drawback: the absence of guaranteed, long-term access to R created moving image and sound content. O “The Digital Dilemma” is designed to bring industry executives up to date on major technological changes that are affecting and will continue to affect how F content owners create and manage their digital motion picture materials. The replacement of analog film systems with digital technology has a significant impact on costs, operations, staffing and long-term access. But the motion picture industry is by no means the only one wrestling with these issues. As this report demonstrates, the federal government, the medical profession, astronomers and other scientists, the military and other entities are all struggling with remarkably similar issues. Through our research, we endeavored to learn what is happening now, what problems they have encountered, what they foresee, and what plans, if any, they are making to accommodate the changes that come with digital storage technologies, as well as the unintended consequences of those changes. It is a study that offers more questions than answers. But the questions are enormous ones and they need to be addressed very soon by the motion picture industry as a whole, starting with those in the key corporate decision-making positions. We offer this report as a call to action to generate fruitful collaborations and workable long-term solutions.

MILT SHEFTER, Lead, Digital Motion Picture Archival Project ANDY MALTZ, Director, Academy Science and Technology Council

A NOTE ABOUT SOURCES Many senior and staff-level employees of the major Hollywood studios, laboratories and archive facilities spoke openly and candidly about what is going on in their organizations and what they see happening around the industry. They also provided us with their personal views of the issues of preservation of and access to digital motion pictures. We chose to encourage the beginning of a productive industry-wide conversation by providing a safe environment to express the unfettered views and facts as seen by the “boots on the ground,” and in support of that openness we chose to leave this information unattributed. -Ed.

THE SCIENCE AND TECHNOLOGY COUNCIL E X E C U T I V E S U M M A R Y 1 /

T H E D I G 1 I T A Executive L D I s Economic energy digital storage preservation between I interest corporate p m D g a a b a n “ p t I c u a s t r c o data L i h h t e n N E u n n c B a s e f u r r n o o o M s c

V a t s o i e e n 2. 1 t r d d

n t n k e t n i

r e o M e n . s s r n w i v e h T , t a D Experience Current T I F s n - s e r s o e g r A

b a significant No E r s e e t

g n u m e storage r t d n

n o

o a

g l h o l 2 h g u t v s h v i e a a r o

y s t t 0 h r w r e

n , c e e e e e e

s w materials e l l i

E – r 0

costs n

”

m a s site, y a i n o a d – s s

m r l enterprise r

H a to 7 n n

o e n a g

t b t

y e

an n t g i

i t o o M

v e

x r r e y s

a e o r f h a d

l o t businesses

a t i s g e n e e n o n a i p s t r . the u models h n r c r e e t

m l

s e s r c

e n O r t i l a

l l l

a archive

p e

r a

circulating y

g y o o is m practices

y k e s t add T conditions d t s m n l v h

p s d g w t

, i

T w t e d n a - Summary

l e e

n t e d a e r , h e a a motion a short-term. i e n r t a o i h a

g c d I o w o d t n in c n i a n n h n p d

e n and r n i t d O s - a

c t i has o l

a d l d e

g r t i significantly d t c s

i d t o u t e e “ e i i g r e d i m g c

N o the s h - o s

l s s

r comparing o i l a u s i r a

w o e s s

yet o o m b s a

n e t n a i m o l and n s . u e m

m s “ , a n u

v ongoing f s

long-term u t

t h

P t s and e f

c m h

s e l o s , i u e e t g

i D in

i e s above e

s I a s t a l h t

s d n o i t t a u picture d o

r i has n w i , C s e o

s t e h r its s f r r e

g f a a a t i t l f with i

a v e t e

a n

e e s other a

e o t v t t r

T f . s d supercomputing e c r

o v f n l

o n a r a a y , i a library. s

o p a i i n e r o

l i a U duplicated

p t m , d

e l m a g v t y - a f p e

m w e l e u a r e g e p

t r u e e i t x R d sectors d e r n f i o a h t d e t h

a long-term h s r long-term r l u p a i a capital

d a e a w ” to a b

d t e long-term s l E e t m

e i industry: c b a sectors n r m l l d h b

e i t i r n s objectives. r l

o i t t p t o v

o 1 l d v e e “ o o t

y e t

a i

o I e g Based plus s p the e

t n e o

h c r

e m l r

t N t t t

i a o

” a p i

v c b

r e

e d

o h e c s

n b t n The

s a s o

o h i a i r s s a r o e D h . n s e v s n a n r e the n

underscores n i b t f f n t e n r a g o r total e v w

g e on o g r d c

l T investment t a o r o t r e l e t f a i r holdings e

c e such i , s r amortized h s

u y l y t

e n f u i d l

l t a . e x d s h o m S

d s l a i o archive e , n

e e c o w a e l

g access

p ” monthly s n strategy

i e l T a v m

g t g t a s t storage g r r s F s t

e i h t e u e . t s i c

a y y a cost o R d

o i

a r

o e have c e t

r a

n t e o t b p

By r e f f o a

as i n s e t n a e W m r w Y

n i i

– m l a p e

l n l

t e t i i c d c

, t h d m m

m a w cost o o h n

a t

m medical, a t e e i cost h r o s

y i T

o current a of capability. r t r h t e

i d l e s o g 1 on

o holds u i e h

w s t l e a

spotlighted l i

e p b a v e H

d r

r i r d r 0 r the s of s w l costs c c

m d n c

t a k a e l u of o n ownership i t a i u q h and , or e 0 o e a o c

t l i s l E film i

g f s t

demand. e

q a o e e a d t u i 40 l

w t - w o l n s

s m

o s r g h R o r y

a s a s u i f p i

i l fact

s solution f t i h n t - n r b

s s a l e w

s a archive o

b cents i e E master-level h a e t w c e e i t e

n a v a s g of l v

operational h t n h s r r d practice y c u a s e

o d t r t a i earth l a t d i m , p , I o t e t a t o

e e

t t

t e

l e i i e b g . t S o

c n i e that r h o

film r l n r o

c per o s l a o r t d a i i r

n e e r

a c r

- a i h t n

, n t A element T A

e m b d

n d c i s m g

a a h s t s v p l g a r two t i p

1,000 a l

i t o

a m l

h o e e o e s of e g r a o t i

e c library a M An u a science, o n n t

d e e m 4 d a

ongoing e n that t f versus v h i f l g – a o a s s e t d s i

t l

. 0 a e , t e z l a e

t and A n e a digital d

u r i

t p m r i o n

c n foot n major i a r manufacture. l d m l a i d archive t n r y y J o a m t v

a r g f l o l t n

d u O

e i l

l o

o t e f l content e l a t i o

f c t does d

i a e a o t r h s expense. i t

film p d e

t t f o

definition, t R n a s i , a l r a i ( is t t s e e w h o a i l n digital

u s

o s Y l m h n t t a a o a i c . $

t government,

f , e a

c g n

a e l D c

reel r

h d C

materials.

a n h 1 labor

findings p n t m a m

o q n e

H l a r e

n i t

e , a p o n M temporary not r I

e w e o w p u 0 h r that x d a l n o o e a in F e i w s g o d f n t 5 i i e

r c h t i p n l s

e d c v i ) u F m in w s

preservation n o

a t 9

n e w

a t i o a . D l s a materials c u a r E

s n g g

y u

require e t

c r and l a n m n i o e b p

t r

e y stores

v a t R

r i r p r o e u d a s s m n l g i e e . l a e k i i y t o i

, digital n a

E r s

s l i

r b t

f t e of b l

t d

n e r k a s , a o F y i i i N o title a t i

a e l

d o o n i n l - s f t m i l o n s - i n h

l n C y t d t f m f e conditions i r . i o e E s i , - 1 r s 1 Executive Summary Y and the annual cost of preserving a 4K digital master is $12,514,2 an 11-fold difference. R

The annual preservation costs for a complete set of digital motion picture source materials A also are substantially higher than those for film, and all digital asset storage requires significant and perpetual spending to maintain accessibility. M Advice from the above sectors includes not allowing the equipment manufacturers M and system designers to continue to foster technology obsolescence as they did in the television industry and are now doing in the information technology realm. Instead, U the stakeholders must be the driving force. S There is an urgent, historically justified opportunity for content owners and archivists to manage the transition from current to future practices. This is best accomplished while E film preservation can still be done in parallel, and essential digital assets that are not suit- V

able for film preservation are small in number and relatively young. Furthermore, the task I of preserving digital assets is too large for isolated or piecemeal efforts. The primary challenge for proponents of digital systems is to meet or exceed the T benefits of the current film system. These benefits include worldwide standards; guaran- U

teed long-term access (100-year minimum) with no loss in quality; the ability to create C duplicate masters to fulfill future (and unknown) distribution needs and opportunities; E picture and sound quality that meets or exceeds that of original camera negative and production sound recording; independence from shifting technological platforms; X interoperability; and immunity from escalating financial investment. E The risk management decisions about what digital materials to keep, migrate, or otherwise manage must consider the broad set of issues inherent with digital storage technologies. The passage of time will inevitably determine the cultural value of assets, but economics will force an ongoing assessment of the future financial value of assets each time a major data migration is considered. The risk management decisions cannot be postponed until the data migration deadline arrives. Digital archiving is an enterprise-wide consideration that requires support at the highest level to be successful. This report is a call to action for the motion picture industry to understand the issues, clearly define the problem, and create discussion among all the major stakeholders to produce standards and technological alternatives that will guarantee long-term access of digitally created motion picture content. To this end, the Academy has initiated several collaborative projects, which include: • research on related digital preservation issues and potential solutions • development of digital file formats for acquisition, mastering and archival applications • development of a digital preservation case study system • facilitating productive dialogue among the stakeholders

The digital dilemma arrived with the digital era. It demands concerted, committed industry action.

2 Based on an annual cost of $500 per terabyte of fully managed storage of 3 copies of an 8.3 terabyte 4K digital master. THE SCIENCE AND TECHNOLOGY COUNCIL / 2 A R C H I V I N G 3 /

T H E D I 2 G I How T without errors, without preservation The A Archiving L D I L E Much M goal M A DI 1 TRILLION 2 3 TRILLION 4 TR 5 TR 6 TR 7 TRILLION

2 A TRILLION 0 GRA access 0 Data, 7 ILL ILL ILL end. M ISNO

is ION ION ION T O S 2002 C ALE 5.5 Berkeley School preservation contents and community retrieve its of leather, world, history records tions A the exabytes telephone, m – R a Book film holdings g C total trillion n protect Enter An H e in t APP I preserved i of of until c and V canvas, assets archive every

of volume of Equivalent [Lyman a I Information various R N n digital radio, books information O the LIBR d assets with other to G without the X

o stored society

. 21M H community, p archive B wood, is AR t present O A i flowing in up-to-date television information: and c – gauges not considered objects OK Y OFC S a ILLION l of the

A in – S INTH errors, in Varian just

new stone, L t Management must it. physical h every recorded O O age, SH a and through deemed NGR t N

Archiving data O a and but, E collection G access r catalogs, WN AST be ceramic, all e 1], valuable specifications. ES

era

H according s DIGI e the reliably in S archives media in e I the on have S n telephone, important without 2002, T

general, T Internet. o 5 AL D O world and storage HE r , purposes indexes

4 metal, enough of h R invested of 9 e E authentic, 7 Y A a consisted Q ,55 old to Systems stored each

r T I U d A generated end. N archiving

8, IV a

media p by GEN radio, silk, content.

13 and 2003 AL The r H to i era vary m 8, in that U in ENT NU ERA keep 880 a photographic at M other r the – of UCB accurate four i comprised TV study from l A the y papyrus, TED IN200 society “things” 5

is N

t long-term “for An h MBE exabytes and

physical meant tools University r S report o domain by O u effective T (1 B the R O and g C the at O 1MI h researchers I needed parchment,

that less E F B that O f to future.” 2 estimated o plates, T Internet complete. , OK ISE – media: u preservation O Y systematically to L r than the LI OK

exist . archive of time. e

ON domain, l K e to S California, c equivalent QU sheets E t

one-third B paper, r search in Ideally, from Y in o YTE

that IV paper, From n

I the integrates The i N 2002. A c S

LENT and S and c of ) the the T physical h film, and goal collect a the pre- I n T of of rolls from 5 n U e l is - s : 2 Archiving continued

Digital media are adding to the explosion of data in the world. This data explosion also comes with threat of data extinction. G N I “In the excitement about the solutions digitization offers, the right questions about costs are often not asked, especially about long-term V I costs for keeping the digital files alive. This enthusiastic attitude is risky,

“for the conversion process to create the digital files may well be quite H expensive to start with, and these investments may turn out to be C wasted if planning for the future is ignored and no structural funding for maintenance is secured. R A Without such long-term planning, digitization projects can come to behave like black holes in the sky. Scanned information, which in the analog world could be accessed simply by the use of our eyes, is suddenly

stored in an environment where it is only retrievable through the use of technology, which constitutes a constant cost factor. The more informa-

tion is converted, the more the costs for accessing it go up. The digital black hole has got its firm grip on the project. It will go on swallowing either money or information: the funding must“ be continued or the input will have been wasted. If funding starts to fade, the information may still be retrieved but after a while it will no longer be accessible due to corrupted files, or obsolete file formats or technology. Then the digital information is lost forever in the black hole.”3

The problems of “data extinction” are only growing as more and more aspects of human activity move to the digital domain. Consumers have to move (migrate) their downloaded digital music to new media players when their old players get too full, sometimes requiring re-registration of their Digital Rights Management (DRM) authorizations to insure they do not lose access to any favorite songs. Authors must find current applications that are interoperable with their old word processing software in order to read manuscripts originally written with software that has since become obsolete. Digital photographs recorded on old floppy disks cannot be accessed on modern computers, which no longer have floppy disk drives. The only way to play old video games is to keep the old game system hardware running, which often requires scouring flea markets for old circuit boards that can be cannibalized for obsolete parts. Modern digital data – the media on which it is stored, the hardware needed to play it and the applications that use it – are all changing at a rapid pace. In the face of these challenges, preserving digital data and assuring its accessibility over the long term requires a systematic process that is generally described as “digital archiving.”

3 From “The Digital Black Hole” by Jonas Palm, Director, Head of Department of Preservation, Riksarkivet/National Archives, Stockholm, Sweden.

THE SCIENCE AND TECHNOLOGY COUNCIL / 4 A R C H I V I N G 5 /

T H E D I 2.1 G I T A is “library” The by possessed assets valuable the L D I History L one E a M most M studio. A

2 0 of 0 7

and Characteristics considered earliest the of commercial Everything recordings, also is [Motion number year including of not Hollywood the ket without Making served The that p House Sciences, operate MANY locations of OCN for t t k t a for color video discovered Preservation have well houses trash; libraries a h h d e u k e e to e o r archiving Hollywood film e n p p p inclusion broadcasters, s demand. past factors s by

The While The But T With including i t can o understood. to “library” , t “save a n survived, u

i TV s for s h etc. other o g from d e so

and stewards until e n separations keep movies

i as CINEMA a

o new 100 o incurring on of

appreciate in f modern explicit Picture

o in V UCLA, f broadcasters they s i in h the

the l

“public” f everything,” Since m m that they it l i D films important

Hollywood, is others. all e flammable g cinema h generations the order flops. feature-length the current the a Plan]. h a

years, s revenue in o has a saved r is number k

arrival

m can all cinema the n r i and c were i n c set n o original e could a one holding 1960s h e collection

released Association d it and commercial g n not

the the motion future i As h e to

undue v still s be ARCHIVES t m

Hollywood and is less n Hollywood’s u e in e archives, o The archives from

lost of

t m climate

o protect 1 of a b e nearly always more Library original p from exploited – n value generate r 9 to archives u of e photographs a result, nitrate made t than the starting e the r 9 s black-and-white c a private saw production, either

creation y DVD, i because community 0 k i p beam n new in the new n

picture

s i r e D m . s complete. most motion i old profitable interpositive of s

c 2006 over impossible

are s 20% their a been control assets such V i e T of

the biggest motivations of

n fewer camera m n productions r stock costs D o commercially g e cinema is assets t profitable to to film with it America

l maintained

Congress, Hollywood AROUND s d a m p valuable history long-term time color d of the

business t film i has the survive was a w as consumers create i e f o by c v f pictures l i in than d e “private” taken k

e were t archives requirements those b in hit the r assets h l sheer e a is

negative for y case, e become e geographic Just l 607, g and

adapting masters n

movies archives c “ e to a ( n addition a t e of d d movies (IP) TV 3]. various new i e m half x very studios. destroyed a assets from e new

w of on-set, tell t does m t s at the can counting that size r e archiving current “in e p an

preservation using a

by p

THE e the e i in

or D have the of t in d h v of archives media v in profitable e industry revenues (OCN) in Museum 11% a i turn e e be trust.” V

of and the a a must apart the possessed l culturally. n l n the the and their u the , o Hollywood versions to D advance

Academy

o very the separation. e all f w m film the bought emerged for WORLD i m ” s r Hollywood 1920s e 1950s, preserving

for by final old o s to increase feature f b r e MPAA-rated products everything t the Film r has r archives

be from n

homes. y o body p e long-term

comprehensive a owned “vinegar” Another

archives. film, o

fire m s e the practice a p media by

n r of added d approach of notated V r been movie, -

which by d V o u [US, and i H

n movie Modern converting or f i of many content n motion Many H as of n i films

the all t S i studio over a a CONTINUE g t a

S by

Archiving Motion r merely even

for valuable

The b assets sold uneven. t various m format c studio.

s to studio a LC, the l t h c distinguishing is to finished e

p shots a at a e film in But corporations scripts i

p public made e v future n n 2005’s studios w the saved titles films, when s save e i introduction u e different more for original n

pictures i hot, owners archives s NFPB, Art, s i t f

to g n , h a a looking thrown

preservation archive

and

c will archiving to n b

Picture

combinations old h t large t Assets corporate

t be all l d h in before h Many i u o and from humid archives. markets. s the Eastman it movie,

e the e attractive 549 r t o t happily

i o o preserved, early reliable movies processed be 1 p n w can r themselves,

TO audio Natl. 9 e g was y j sums. are total i physical more. new u r d 8

. in selected back every s are films the c

s Arts f e 0 of , 1950 feature o o t be

s s i film ware- o among s policy p the r but f not

of t

u y a the assets pre- r Film mar- worst s t to done n job e t h ) were over and - a , d e d 2.2 Current Hollywood Film Archiving

The terms “preservation masters” and “archival masters” G N

describe the 35mm original camera negative (OCN), interpositive I

(IP) and yellow-cyan-magenta (YCM) separations on black-and- V white film stock stored in environmentally secure film vaults. I H C SINCE THE BEGINNING OF THE “HOME should be archived. But generally speaking, the issues video era” around 1980, most studios have come to of analog archiving are well understood by many R

recognize the potential long-term value of their film people. After more than one hundred years of technical A libraries and some have embarked on ambitious “asset innovation and market forces, the photochemical media protection” programs. Paramount Pictures is a case in formats have settled down to just a few remaining point. From 1987 to 1993, Paramount reportedly spent choices, manufactured by only a few remaining vendors over $35 million inspecting its negatives, audio tracks in widely accepted standard formats. Today, there is and black-and-white separations; doing film repairs; and broad consensus about how to manufacture and preserve printing new preservation materials. In 1990 it opened a 35mm film archival masters. new $11 million archives building, with environmentally One of the largest film vaults in Hollywood, as an controlled vaults for preprint and color materials. example, holds about 425,000 film elements of various Paramount stores some of the master elements in an kinds. This particular vault holds film elements from underground facility in Pennsylvania and tracks millions motion pictures produced in 1912. Like all the of items worldwide through a custom-designed comput- Hollywood film vaults built (or renovated) in the last erized inventory and tracking system. By investing in fifteen years, this facility is designed for preservation the physical care of its collections, the studio expects storage with cold temperatures, low humidity and fire to extend the shelf life and revenue potential of film suppression systems. elements as well as to expedite retrieval. A similar Archive services are basically the same for whatever archive construction and asset-protection project was film element is placed in the vault, whether it is a undertaken by Warner Bros. television program, theatrical release, or documentary, Industry storage practices vary, of course. Other no matter if it is cut or uncut. Every piece of film studios have their own film vaults on their premises, and coming into the vault is inspected before storage. store other film material at commercial vaults. In addi- Inspectors manually inspect the film, look at the tion, most large studios routinely keep preservation masters information on the leader and compare it to the of films they produce as well as additional materials – container labeling. In addition to inspecting the film such as foreign-language soundtracks or edited television element for physical and photographic integrity, a staff and airline versions – as required for ancillary markets. restoration management director may have a laboratory For each title, a studio may keep many different make a viewing print to ensure that there is nothing preprint and sound elements. The depth of preserva- wrong with the master negative, and that it is intact tion protection depends on the scope and duration of from first frame to last before committing it to the studio’s commercial rights and the film’s expected archival preservation. value over time. As part of the intake process for every film For most people in Hollywood today, the terms element, the archive staff manually logs basic asset “preservation masters” and “archival masters” describe management information such as element title, reel the 35mm OCN, IP used in the print manufacturing information, element type (OCN, IP, etc.), version process, and YCM separations on black-and-white film description (editor’s cut, etc.), program type (theatrical, stock stored in environmentally secure film vaults. The TV, cartoon, documentary) and aspect ratio. Also, one OCN is the most fragile and is only accessed to make or more unique barcode identifiers are assigned for new IPs or restoration elements when needed. The IP, inventory management. For a new release today, by usually extensively duplicated, is typically used to make the time the original lab materials reach the vault, a new printing masters, and the separations are used when High Definition (HD) master has already been made, all else fails. and this is used to supply various downstream electronic Each studio has its own list of what specifically distribution platforms.

THE SCIENCE AND TECHNOLOGY COUNCIL / 6 A R C H I V I N G 7 /

T H E D I 2.3 DETE underway: Warner petitive. distribution formats, come centages interviewed and High non-theatrical as prints, have distribution almost typically primary demands and s commercial elements manufacture and/or tion preservation, even bought global expense. policy particular j this print necessary a FILM G u t t I o T u s derivative A t r r

replicas Digital e L e s means cable are In The While d libraries D o expanded billions underground master

Definition I through m L i media to Hollywood ARCHIVES but n 100% E parallel internegative (Digital and very e M Bros.,

commercial

there of

traditional t (including At protect M and These s – h and scene. h period A ATLAS the also an e a o

for for sold distribution

libraries libraries 2 least actively versions elements d r of new containing studio’s 0 studios companies t maximize film-based i entire 0 satellite will release, non-theatrical have s major to e in 7 archives example, t Digital this End-To-End), dollars. l are

e their video valuable i of two They m of b film release recent be u vaults (through e carried digital report analog IN sound t three movie n window film managed operated to operate i of studios’ revenue the o growing major t film can such system archives s n for Cinema function

digital DISTANT, programs revenue a

release who requirements. years l so prints if r copies today i assets e to b be sale believe distribution collections

tracks) no on system must r as n far, Archives structured a studios, Ascent five e short-term digital r found for have, archives operators. e potential Standard i by are to other to the platforms pressure e to respectively. d intended and prints, the s are Packages produced e that potential as

each include be d years. satisfy a these television in that r needed financial

that a e t over accessed commercial o other locally.

Media r markets COLD-TEMPER- good-quality only Sony a digital kind e can

c r t title for have r e interpositive c as

same separates library to i for The p Definition e customer e film the

for to s a v not be Several (DCP) higher s millions finished at during to of i move e e Pictures produced, r a d Group) d stay stayed

broadcasters books

formats used years, only long-term o ; Sometimes assets truly great studios

in insurance given meet

distribu- only f vs. r s r digital

projects o e r new m q e com- the per- people as p u when to the e film of film l e or t title a and and and i well n Libraries c for m t e protected l e

y a s assets be encrypted, pixel content, as c T bution archives on people libraries consolidate transfers infrastructure. to bandwidth power, speed. processing, formats accessed are assets library perhaps t digital o c d a u a t r and the encrypted i i e a c r o f i s t o h p e g

n t e l u compared a n e

co-locate

i stored e d l

distribution a a n

retrieved t i s library

l c

a other Titles T The D s d

l d t a t m

e i a l count, , d o r i b

i would h

s i n m g g

i f preservation

g e

r o services, responsible is e digital n But u g o e i s a d i has v

employing t and for u o themselves. e t l g through l which f r from c storage a

y e l a being

n un-encrypted y t by r e t o

l t DRM

l as i o a b in

might e h

i a ready o n mean m m often t n n a many

continuing i r u asset historically the e n e lower t t v r a archives. as typically c t part to h

- t d the e c i e

o t e s d o g p storage d h e e archiving

r t

redundant

h m i e This s a

a t i is carried i and i the the n archive, r a c i u p h a v t g and o e metadata. to r t library t distribution a preservation

using the e that c t also e a of most v u o c w a redundant n t h s t

bit-depth, i for h e e r

r d n u

different , digital other archives e t c go a n d e media o

i the i r h s could l h v u r s administrative s

l archival u f e t e

f t require e capacity contain

n i it d preservation c m e i r

a v It a

on over l o likely purpose-built increases c a i a

b required

inside r n e a b w distribution d may service, to o library n r e c n - e l y also c u

r for h i t m

c h l a g

demand. h r g b y media t a processing i l reduce processing i a s i i o v e

i p e l u t s o to n and t v e

a c a s user e r r functions s archive i n become and e

formatted s s l would reformatting o interface. l library e t u n e t from embedded

the

s and s h the

m f o a q e l t l

s t o and y i o s r a e f u but h e b slow for s

d distribution p interfaces p i c e e compressed. m r a o

e library, c the a “ t o

digital r r r

n k digital d a and s r digital o b c o m libraries a

r systems distribution

the n l y not e Or i h bring p t process. t t c o might services. y s

s e h hardware d o more and/or e i o R&D. t

r number u f facilities

v c a r s e a s n

i r shared n t i u e s n t n those m s a at they digital b

e .

when

t i n a t t d l i d watermarking f e u e domain. i o s r s but i networking e

k r processing

lower e together

c t e when

e n

b i a i efficient – e l be d h d t n a o v for for s c y l h i i s

might s expensive y t

e n r

s one v working r i library a e Titles e always r s y by

Archival t l e

pre- they of a t n f y n and e . distribu- r l p for p

s l u the distri- i i d

, d

. b t b a o ” t N l

they u data e e l h

s r

u for s

d B t

n d a

i p e e t p h t the r are ,

d u y i w i o in y a t o e x a t o r r y n e

i n y t i s l f , t 3 The Transition to Digital

IT IS IMPORTANT TO UNDERSTAND THAT THE MOTION PICTURE L industry has been adopting digital technologies in a piecemeal fashion over the last

The recent 25 years. The following sections present a brief history of the digital conversion. A The recent introduction of digital technologies into the final links in the production T introduction and distribution chain is, in fact, a “tipping point” that fundamentally changes the I industry’s economics and practices. of digital The digital transition affected different aspects of the production process at G technologies different times, although the fully digital production still results in a “film-out,” or I creation of a film negative from the final digital master. This, along with the YCM black-and-white separations made from the final digital master, is the only finished D into the final film asset that is currently being saved using a well-recognized technology with

links in the understood and accepted long-term preservation and access characteristics. O production T 3.1 Audio Converts First and N

distribution O MODERN AUDIO RECORDING, POSTPRODUCTION AND DISTRIBUTION chain is, all use fully digital workflows yielding digital audio files best saved on digital storage I

media. In fact, analog audio tape is rapidly disappearing. There are few remaining T in fact, a manufacturers of analog audio tape or the professional recorder/player devices that can I handle such tape. This is compelling the sound departments of the major Hollywood S “tipping studios and elsewhere to transition to digital archiving for lack of a better alternative. N point” that Digital Audio in Acquisition and Postproduction A fundamentally The introduction of digital audio recorders and processing equipment in the early R changes the 1980s was the start of the motion picture industry’s conversion from analog electronics and film technology. The Nagra series of analog audio tape recorders manufactured T industry’s by the Swiss company Kudelski, S.A., long the de facto standard for motion picture production sound recording, began to be replaced by the Digital Audio Tape (DAT) E economics format, which was subsequently replaced by field recorders that use hard drives and

recordable optical storage devices. By the late 1980s, the supporting analog mixing H consoles and tape recorders used downstream for sound editing, effects, and mixing

and practices. T began to be replaced by Digital Audio Workstations (DAW), although the final soundtrack continues to be output in analog form to film stock coated with a magnetic layer (“fullcoat mag”), and then ultimately as an analog optical track on film prints. Digital Audio in Exhibition Although it was announced in late 1990, it wasn’t until 1992 that Dolby Laboratories first introduced the SR/D format, known today as Dolby Digital, with the release of BATMAN RETURNS. The development that made this format possible was the AC3 audio data compression algorithm for 5.1 audio channels, with the “.1” signifying a limited-frequency bandwidth subwoofer channel. Real estate on film is precious, and so Dolby opted to record the “bit map,” or images representing the actual digital bits, between the sprocket holes. It should be noted that the optical analog sound track was retained as a backup measure, which still remains on film prints today.4 More digital formats then appeared in the cinema marketplace. Digital Theater Systems (DTS) introduced the DTS digital 5.1 format in 1993 with the release of

4 Optical Radiation Corporation was the first producer of a commercial theatrical digital audio reproduction system, used first on DICK TRACY in 1990, but the lack of an optical backup track, coupled with the system’s complexity, prevented the system from being adopted by the major studios.

THE SCIENCE AND TECHNOLOGY COUNCIL / 8 T H E T R A N S I T I O N T O D I G I T A L 9 /

T H E D I 3.1 G I T A s a t t m d a D L o h o n u e D i a I

e Audio g u L y d g r E n

M e i n

r i i t o M o n i a c d a f A f

a o e d

l t 2

a 0 t b a s v a 0 h t 7 t i p e t

l e s i a i r e o t

Converts l i y n l n .

formats the formats, petitors Film a exclusive optical film, this Todd-AO SDDS elements tape, and Symposium, presentation 15 such I with reason degradation increased “brick formats Studio Archiving J I the format n N URASSIC film s t T years, i film analog capability. t preservation It Digital In HE u scheduled that producers and and t print is wall” First that e is sound 1993 and track L

in for do print o in important P [Taylor a physical and INE audio as approximately DVD-R f 70mm ARK

the optical 7.1 S not terms some multi-channel records audio Digital with with manifests failure t at for NBC/Universal Sony a OF therefore and itself. n . marketplace. format,

the have migration d

enjoy As “noise” continued DTS backup F a system the and studios both of magnetic area r IRE preservation track preservation optical introduced d with Association mode. only to s

sonic adequate Audio For

printed a Regal]. and the places in note n of of resurrecting half which d as Dolby that [Karagosian]. half an the the have

the the choice T to disks capabilities, L

sound a That that e AST analog format, will

backup. c the There recordable inability foreseeable uses of digital film. Studios other h long-term backed the can n methods departments Digital, of to a A each not o digital is, and for CTION l collection a Moving o SDDS access

generally synchronization g In digital are the while combination although audio [The y cinema. innovation of , discussed to up

practice, distribution D limitations additional the audio DVDs. the are future, H recover V survival and their digital analog X formats, data D Image ERO SDDS existing of getting have Lab]. be

t not preserve bits e . disks digital c or filtered it the

there

h audio any However, Unlike that audio of characteristics, n Archivists’ long all is time on full-range and o data capability, SDDS development track more online more l digital can o feature of come CD-ROMs will audio g its known format y advantages tape out, code the is

Dolby be h motion on sophisticated. and recorded a continue relies hard s according formats sound expected along

digital degradation data films d May effects the for e more and that with g Digital r drives, more on primarily film, a of to picture with d are 2007 in at to the audio digital a directly channels the occupies to its the recordable common t a all. i to each released to o also than proprietary be economics digital n multiple and LTO3 dual stereo last Digital

the

In manifests c sound It tape a h audio of due preserving a variety DTS, on one a is more r National releases an of a the delivery to for using c data the to CDs t Asset the e format. com- tape release r this their i than of s of as t i c of s 3.2 Visual Effects and Animation

JURASSIC PARK WAS NOT ONLY A WATERSHED EVENT FOR MOTION L Digital Asset picture sound; it is also widely regarded as the first major motion picture to use photore- alistic digitally created characters in a central role.5 The movie’s dinosaurs were originally A

Management planned to be shot with traditional stop-frame animation techniques using miniature T models, but the initial tests of the digital dinosaurs were so promising that a commitment I systems was made to go completely digital. The final product was impressive, and the popular

folklore is that audiences “could not tell the digital dinosaurs from the real ones.” G require 1995’s TOY STORY was the first feature film with completely computer-rendered I 3D characters, and in the years since, 2D animation and visual effects have been almost ongoing completely created using digital tools. D investment in The adoption of purely digital tools for visual effects and animation created a need

for effective digital data management tools for production activities, also known as O Digital Asset Management systems (DAMs). DAMs, in most cases, effectively enable

infrastructure, T the backup and production-related access of the digital character models. This is not hardware, without its costs, as they require significant investment in Information Technology (IT)

infrastructure, ongoing hardware and software upgrades, and highly trained personnel. N software, and But the combination of digital visual effects and DAM has proven effective in making

some of the most commercially successful movies of the past few years. O highly trained I personnel. 3.3 Postproduction T I S Picture Editing N The transition from cut-and-splice film editing to electronic nonlinear editing began in A Interchange the mid-1980s with the introduction of computerized videotape- and videodisc-based editing systems. Film-originated television programs were the first to adopt these R systems because they did not require the conforming of the film negative to produce of images the final edited master. Television program masters were assembled from master T between videotapes using electronic “instructions” generated by the nonlinear editing systems. The development of “negative cut lists,” coupled with the instantaneous access of digital E video stored on computer hard disks in the early 1990s, made electronic nonlinear facilities, a H editing practical for the editing of feature-length motion pictures. requirement Today, almost every theatrical motion picture is edited on a digital nonlinear editing T system, and consumer versions of this professional tool have found their way into tens of in today’s millions of homes. For better or worse, the rise of personal video-sharing websites such as YouTube would not have happened without the development of professional digital world of video editing tools. It should be noted that in the three cases discussed to this point, the transition to multi-facility complete adoption of each of the digital technologies took no more than ten years from collaboration, initial commercial introduction. is problematic, Mastering The final step in motion picture production is in the midst of its conversion from film given the lack to digital technology. Generally referred to as the Digital Intermediate process (although of digital Digital Mastering is a more appropriate term), the final color balancing and visual styling of the final master film record more often than not are done using digital tools file format such as interactive color correction systems rather than adjusting film stock exposure and developing controls. The Kodak Cineon system, introduced in 1992, demonstrated standards. that analog film images could be converted to digital bits, processed and enhanced, and then re-recorded back to film with powerful results. This concept is used for both visual

5 Earlier motion pictures such as 1985’s YOUNG SHERLOCK HOLMES and 1989’s THE ABYSS integrated computer-generated characters, but in relatively small supporting roles. THE SCIENCE AND TECHNOLOGY COUNCIL / 10 T H E T R A N S I T I O N T O D I G I T A L 1 1 /

T H E D Visual t H s t b e I w c b b ( i s 3.3 I n h o 7 6 c o a n G f o

e e e

i T f

o v f “ d 2 l a I i t n l e l l e T IA v 4 o F g h i o i l 1 e l n n a Precision c e h n c A l 6 e e K h r r O G w e y t h v

e A a 0 d m L G g c ” A Color Co D RAM ISN s c a a e r

) h i m i D i c R s t p c n i s v d , amut a a i w g r I i

i a n h s v e x Postproduction s L t t e d t e MA xel s h n h e s unt E i a t i e e e Attributes f u m o h e o h l o e i M e i m g l a s t n e v r y d g n r

a a s h e t M i y f s - e n r h r t i u v t O a i

. a q t t T

a d A m h a o F s e T t r e i n a t n u h t

o f e r h n f b f 2 i e s d e t s

e . a o n O 0

o e s o y

i T e o a r v l f e e m n 0 d e T n o r i c

S r r e r p r r t d 7 o r e C e a r e m y r v

a e o A t t e o d h u e i “ h a c v c n g x j LE n w l e t h e s e a b i e e a e s d c 4 o s s

r l a l o r h v l i

t m s 1 e i y d f y

n K 1 9 n i o

e u i a q f g i a b s l 20H X1080V r of i g n f s p h n h ” e i u

0 s u d e o t o d e c i c l t “ H e e a t 0 e n t s l a w n o h r i t w t n l o , o o D l Image w y p e

i c s ( p f e p e n n c a m d TV e o r a i D r , t e o H o e x

e g e y i e w r a e s m e

p f c l n m c t d l i o m r a h D t o - i i u h

s s a c p i g r n

e c f w a l i o t

i b a i T a i a h m l o t d v l e u t m r g

s a c d a n r e i n o V a h i t s e i r - l t t r e

Formats n l ” e l a c q I

b r e h a i a ) d r s o i s e e t d f e s a n c j i DI f n u t i g s u d

o n s d e u t e g s c i t i q h o e a t u continued r r t , h s l e i t a e t m m d u i l n , n y

l u c r l

i n h y

i t s m n h i e m n a t g

r GIT p , e g e y a 9 19 d t e

a a , e o y q o e d f t r m a i e 20H X1080V [ r m - r u s t n t e n d 20 X1080 c e q i a h t i i o n S i A q h v s o e e a c a d i i u n a l a a c a d n c u m L CINE n i g

n t l n a r l h p t e r e i t r i i p g e e l i w n s h t r r s e a c i a e t u p s i t e 6 t t r m i e

i n a v u y i m t z c s a h r s o t s n r r m g s e t e e u s h i w

o a u u 3 r MA s r e e a

i i e t e c u ] e 5 m

n r s h . s a . r e c

i - m e e a

d t a s M s s l

t a m g o s e e . o e s DI s l

r y f

t f e i 2K s o - l

m i r n m . f

a 2048 r t m GIT

i n choices motion effectively creative digital camera many Mastering and is master Furthermore, l i a e n

t r i m e

o A not H X1080V t g n o L CINE a u

o therefore Another l t d a n i r a c

cases t defined

u , y h camera – s negative i historically ’ picture’s given s control e s

MA p

data

t w r o shifts o process d j o bears e a DI c r y unintended t l .

what the the 4K

tape, d c by

is A a

n enabled o

significant (or 4 4 used

overall b f little any K final lack

e GIT is m

096

i made f goes optical m o digital u u that * App a A n standard H X2160V g l in d of t resemblance L CINEM e physical i

a - h “look” by into r t production). f o a digital

the a side by h x s ima c

disk, camera t 4 decisions i digital p l 0 : the i t / final the 9 e t / effect

y A 6 pix w or

downstream form p c w el file magnetic cinematographer. i o x archive w c

industry e l

ount digital 35 mastering . l l original o s a to

s format of i b c n ˜ regarding a of of 35mmf o

4 t

r the h s r The the 096H X2 .

e a o

the FILM

t r h is g i master M o o / original hard r Digital agreement, c not i n data M standards. z level o ilm nega digital o , from u tools

n n i t 160 c s a i

defined. the l drive l / p

if d in of a V* tiv r i d r o on-set a e v e b c a t n - i o c 7 – n e d

a . h n t d m

l .

3.4 Exhibition

THE CONVERSION FROM FILM PROJECTION TO DIGITAL CINEMA L projection technology8 is underway, and so much has been written and is still being At what written about this topic that the subject will not be covered here except to point out A point film that it is unclear at what time in the future film prints will become obsolete. Of the T

approximately 37,000 commercial theaters in the United States, 3,595 are Digital I prints are Cinema-enabled, and conversions are occurring at the rate of approximately 200 screens per month [DCinema Today; Overfelt]. The conversion rate is expected to accelerate G no longer when Digital Cinema Implementation Partners, a consortium representing over 14,000 I U.S. screens, and Technicolor Digital Cinema begin their deployments. Assuming a doubling of the current conversion rate to 400 screens per month beginning in 2008, D

economically there would still be over 8,000 film-only screens remaining in the U.S. in 2013. The

justifiable is international conversion rate is expected to be slower than the domestic rate, given the O unique business and governmental issues. With over 70,000 screens outside the U.S., unknown. there are likely to remain a substantial number of film-only screens for some time. At T what point film prints are no longer economically justifiable is unknown. N

3.5 Acquisition O I T DIGITAL MOTION PICTURE CAMERAS THAT IN SOME RESPECTS MEET I or exceed the perceived image quality of 35mm film negative are now in regular

There is, of S commercial use. The digital output of these cameras is recorded either to HDCAM SR digital videotape, a magnetic hard drive – based digital recorder or solid-state “flash” course, the N memory devices. According to the motion picture camera manufacturers interviewed matter of for this report, approximately 20 to 30 major motion pictures per year are now shot A using these cameras. Reported advantages of these cameras over film include immediate how (or playback of recorded scenes in certain circumstances, increased color saturation in low R

light-level situations, and longer recording duration between media reloading. T whether) to Reported disadvantages include reduced spatial resolution and exposure latitude

relative to 35mm film and postproduction workflow challenges caused by the large E preserve the amounts of digital data produced. Additional trade-offs must also be considered when enormous choosing the capture medium for these cameras: HDCAM SR digital videotape or H digital data recorders. For example, HDCAM SR uses mild image compression T amount of and digital data recorders do not; digital data recorders allow for deferring certain image processing choices until later in the postproduction process; and higher spatial digital data resolutions and greater bit-depths are possible with digital data recorders. This new capture technology has had some interesting effects on production produced practices. For example, the relatively low cost of digital videotape as compared to film negative has resulted in letting the camera roll for much longer periods of time than when with film, enabling directors and actors to spend more time achieving a desired performance [Kirsner]. There is some concern that the greater amounts of source material shooting generated with this production style will result in added overall costs when postproduc- digitally. tion and archiving costs are factored in. It is also reported that some directors will do “circle take” selection on-set, deleting the digital files containing takes they know they will not use [Hurwitz]. The concern voiced about this approach is that it increases the risk of accidentally erasing a good take or eliminating potentially useful alternate takes. There is, of course, the matter of how (or whether) to preserve the enormous amount of digital data produced when shooting digitally. This subject will be delved

8 “Digital Cinema” is defined as the theatrical projection standards currently being developed by the Society of Motion Picture and Television Engineers’ DC28 Technology Committee.

THE SCIENCE AND TECHNOLOGY COUNCIL / 12 T H E T R A N S I T I O N T O D I G I T A L 1 3 /

T H E D 3.5 I G I T assets. to accessibility long-term assure reasonably can exists or hardware media, Today, A L D I Acquisition L digital software E M M A

2 that 0 0 no 7

continued pre-mix/pre-dub They foreign-language of manner. comprising tion o a a p e c o T 3.6 will print tool tographers 35mm original By depth, generates such location analyzed into again elements is and long-term software. suitably shorter, components, much systems accessible the n n x i u f i H n c

c a d t D digital t e e l p any

E u u goal huge m m p be obsolescence Archiving A Generically, What u in i and in r s

as g o film, A i may e t until e c a longer i v s t

r c

t number Motion The D film diminished their Section e

a e measure, d g a a o is f it source long-lived n s

of shooting. i l l

i access, r for d V are s backlog n

magnetic o

image s When t 60 d to M e But that would t i accessibility e is also g w E it using a t i r a they p structured n

s as i l N i a n creative

this b users . e unknown going to digital n g is than v

and s e n u

t g a today, the e of S t the e include d material. t Impact unknown l 6,

of o more d

u t i r frames with f need O e it o u

i r digital this report

the dialog audio be e n digital a n will t of o the n though d . backup over F primary o the

comes period c

g media m as 2 into hard

Digital i

e preservation t o

T simple a films 0 D o p

toolbox, new

is r the no o than Picture 0 to a formats t r be

r o c lifetime I h data t a n e to 5 at h digital a s G that h result files, a d produced e p archiving tracks, “temporary t

i assets be drive media, e “

p p essential. i long v r I intention large to c t digital at this digital embedded a f during it e waiting T a e r copies, i r i 2,000 t acquisition o s o capture

p l e files migrated to e A t e m make is n this d cinema, e a h r and n L of r t s a u of and e

in time period recorder t camera

i satisfy

worth l is i of number

i v c f a m and

n C r r f t t e in hardware

the o i time

assets. cameras i which a i system d e

l I other a which o terabytes individual v m p etc.) m is well a N up to Digital i that

new e n a g r c n medium of is s text

E i o digital Archiving the adoption S

s a format t w in digital be mentioning the r c some

l of a r M whether digital t whether d e t originals, long-term e c a o l using over h or a shooting

or s digital

h n challenge of

the a r a A r ingested s systematic d files time. a A such is k i f s o e requirement t v i seem T system o s the f s

l a o tapes or i e dynamic that l t y r

n of O years o storage” t m a h d information 5,000

solid-state objects s that g storage w for t a masters, containing e

uncompressed , G software

”

assets U

c equivalent data

of c s w “assets.” digital

c k the The h R , to a f the n i

n h o or digital Technology theatrical e n n a with A i from preservation, the of can into

r o i n v for

e here h c agree

n P HDCAM w e m economic d (depending l disks h digital information e o r o key similarly need commonly H approach where

media, l

a s n a

t new e a i the be r l cameras multiple h

t Y film d r a g o t by exists i now. y that e “flash” e

audio n g - , n

o digital

problem that

t indexed r e subtitles g d is to W g s of e f studios. e to storing d t r

digital e u r motion s ingestion, u t o still

m encoded. r they e I s h consider i one 13,000 d

w hardware D t a be digital n

p that e they r

i r long-lived SR viability i

p original

r o y c n E that i s will memory e of

r archive. n h digital s on has accessible t g include s m S e ’ of u will it e t e

s P tools. videotapes and e p can in the i r d r digital in u e

are n s r v R At u contains pictures. pixel i r the ultimately anticipates

s v its recording

e o v [ to s t n the E i

various R not ’

h a when

e a storage, information s u r reasonably If simply searched one A t w and

o sound stem e of benefits. 436,000 i m l motion t b o D c

s o

s on recorder sequences the t design u hardware count, e All o

n “objects” b be h

film t n

s d n is

e h A e studio, i software t o e r n s archiving

a h D files, f remain

looked i period very f p Cinema- digital e t languages. from d a o

a another tracks,

l i s n preserva- O l systems r e t o supplant negative t s , failures m h s r d

pictures in . y of bit-

DVDs. e

As m i t s long v assure d y t T

some there s a such a o

and on i o I s r g l were f at with t . e O i i

e i e t 1 o t a N – y ] n l , 3.6 The Impact of Digital Technology on Motion Picture Archiving continued

The discussions with studio personnel were wide-ranging that the studio will not adequately invest in future- and in depth on the challenges they face in this changing proof archiving and access, and will thereby risk the L

environment. In general, there is no clear picture of long-term survival of expensive corporate assets that also A how to deal with not only the production and intermediate have important cultural value. digital elements, but also the proliferation of different At a third studio the most immediate problem is T finished versions of a movie (e.g., foreign language again how to handle digital camera origination materials I

versions, edited for cultural sensitivities in other mar- stored on hard drives. Some digital cameras are using G

kets, etc.). There is also much concern about the trend digital videotape, but many are recording straight to I to create digital masters at 2K (only slightly better image hard drive or flash memory storage. Without physical quality than HDTV), which contains significantly less capture media such as tape or optical disk, there is no D visual information than the film masters created today, or easy way to keep the entire digital negative. Studio even those created 40 years ago. The fear is that projec- archivists do not know if they have archived everything O tion and display technology will continue to improve, but because no physical media exist – there is no equivalent the archived source material will produce nothing better of OCN in these cases. They are also encountering this T than what can be seen on today’s display technologies. problem with audio recordings, and comment that it is On the topic of storage media, LTO3 data tape is unlikely that every digital videotape or digital audio tape being used at one studio as an archival medium because in storage today can be transferred to data tape in the N

they believe there is no better choice, and they recognize future because of cost. Management of metadata – the O

that this commits them to migrating the data to a newer “data about the data” that allows for efficient indexing, I format sooner or later. They also recognize that there search and retrieval – is critical for archivists but is not a has been no planning for that eventuality. The biggest high priority for manufacturers or users. T challenge is that they are worried about making the At still another studio, senior technologists worry I

wrong choice, given that their long-term objective is that there is no formal corporate archiving strategy S one hundred years of content life with guaranteed across the whole company. The technology group can access. They also believe YCM separations are the best lead (intellectually) and formulate “recommended prac- N protection and insurance available today, because they tices” that the business units may adopt or not, as they A provide a safety net to allow the use of a digital storage wish. Strategy decisions for archiving are largely made format with a much shorter lifespan such as LTO3, and at a business unit level. The business unit that is tasked R

to find a better solution within 7 to 10 years, assuming with storing assets has to pay the storage costs. Archiving T

LTO3 lasts that long. They emphasized that digital issues are currently handled in a decentralized manner, archiving of the finished program is the number one but people are starting to realize that if the different priority. Archiving so-called “floor content” (trims and business units were to compare notes and start to share E

outs) that are saved today as part of the film archiving resources even a little bit, the enterprise as a whole H system is a secondary concern for this studio. could be more efficient about how to tackle the digital At another studio, digital acquisition of principal archiving challenges. Given the complexity of internal T photography for “A titles” is seen as a looming chal- accounting, operational and business responsibilities, lenge. This studio already saves key components, the best approach, they think, would be to centralize including the digital output negative and separations, knowledge about digital archiving but decentralize but they do not have a system to save the original control and budgeting for specific archiving facilities digital camera data. Ultimately, they want a method for and the assets they hold. Stand-alone “silos” of digital long-term digital storage that works as well as film does. archives remain, and basic problems related to internal They are confident that it is feasible to keep digital archiving are unresolved. Therefore, compatibility with elements protected and accessible for 5 and possibly up external archives is still a low priority. to 10 years, but long-term digital archiving is an One executive argues that digital archiving is strategic unsolved problem. They are hoping for help from the to the future global media business of the studio, and storage industry in terms of new archival-quality media that this work needs to be done fairly close to home and other non-film archival methodologies that can be because the production processes and the archiving applied to digital sound and to digitally captured processes are getting interwoven. There is an assump- motion pictures. Eventually, if they are no longer able tion that everything produced in the future is going to to output to film, then of course everything will need get re-purposed, sliced and diced in many different ways to go to digital storage. There is concern about the for different markets over many years. “It’s not like the economics of digital archiving, but the bigger fear is film vault of old where you could ship stuff off to

THE SCIENCE AND TECHNOLOGY COUNCIL / 14 1.0 0.8 0.6 0.4 0.2 0 .0

T H E T R A N S I T I O N T O D I G I T A L 1 5 /

T H E D d c e t s a e efforts, be challenges dynamically you.” few underground 3.6 On Making I o o G n x a r i c o

I p a T d t a h organizational,

b l h A

l R u o a i times Black-and-White o One Another A

l L

a v E i l

e n g

t D t e

C t

L i

The

n I O e A m i The n L a process Motion n i d

E e S R n n g a

E D senior o / s 5 g t

t a R M a

s m h E for

e t s 0

assets v R h

A l year a

t n y Digital i e now.

t e y s d studio 2

t , mines o

r e w digital 0

a e a

a 0 re-engineering d Impact o s d v o a On Bl Ma r technologist 7 t and a

a n e n s u

a in t i

a e t requiring d l n a t l a s h

l i d

h b b executive y the o used i e

tell n archiving a , l o kin

, YCM a

e ack

s s Picture s h t u a

t Polyester t o t u

e d a archive l h s e n

them n d i e w for -and n o o

n i g aDigi t o o u e

o i a s n c

long-term o g u e Separation c c anticipates k storage, l h r of a l c a explained i . d

and - n r t to e n t in Whit

e s all e o

H l h s c i u

f i ship k m h

a R b

s Film e Digital t e self-discipline. t need e E n o

i o “studio a

l b

C o t o i Archiving g c 4 e d ORDING F o t e and l o l n K ORGR y a o l stuff s

P tal Y educational M i m e y h

g Stock e YELL t t oly t that . that to a v A o o y

C i p v c

GENT

then Y T u e e

a e s Archival A be h e x

back v culture” u t O h t a i N a e s e in t EA h the n W i l c r r o t s l ORM CM Se treated a g d A y

u e

m TER call h t w

t addition

t r Technology

p h d i i t biggest to a

o s v F o h r i t A u e n i ilm w e e v a T l

d e : will e t s

. h s t r

u Master e St - continued par o ck that i o H is explained filled But b themselves. shoots storage everything archive footage t n o atio n e t

e both

g w e c

o r f o a today On e n any i n n e n with a l n t g l is s n A i

a facility, all

e from

n o the a a strategy c DUPL sent financial r g N

r on t that h

there e E of un-inventoried e on t h e d G x

o other a The r i t A

OPT v the v digital IC t e a to chiv T paper h a y r archival r A IVE l n while

c r

are r TE o a h IC p a for e scripts hope u l i r s hand, different AL . obligation v e p

g e

still s o h cameras, I goes al Ma e

archival OR n n n

anything r t v

e s is h storage h a i w DIGIT a b and e i t a that s

i i into lot o H

u l very “stuff.” v i n s t library i D y e e

AL R the preservation

and of w his s everything o t C c the o of f , to senior t t

cardboard i t E shooting m p v e company h C M major a i r be “all t O i e r than o s i cultural t

e MBINE

t a S m e s saved C d

R c studio paper” a HE a a t g

k t the t motion t a a a C e h will i

p s logs. K PRINT be n a e

cartons from wants e

n i s obligation. videotapes s t s t d t

executive

able

eventually i u archive a a

m s d l d

But a

o i p pictures r photo o t r e i to to a ’ s i r s just b g k a i a s t n , i s v a e e l s . 3.6 The Impact of Digital Technology on Motion Picture Archiving continued

survive even the potential risks of global economic natural conservatism of studio management L bad times or investor-driven corporate budget cuts will tend to extend the use of film for archiving from “on high.” He emphasized the critical of motion pictures. Interestingly, the cataloging A

requirement that truly archival assets be able to and indexing systems for film archives, especially T survive even if someday there is no money for the the crucial metadata databases needed to imple- I next data migration. For him, “archival” means ment any enterprise-wide DAM system, have

“store and ignore,” with the belief that survival for already gone fully digital in many cases, although G

20+ years without worry, and playability 50 to there is no commonality of implementation I 100 years from now without major additional among the studios. investment and in the face of “benign neglect” are D essential ingredients of any sustainable archiving Cinema motion picture archiving must strategy. The problem, he believes, is that no encompass digital archiving; “born digital” O modern corporate institution will or can commit to

assets have no film elements to preserve T “forever” funding. His studio today also holds data tapes, but does not consider them archival. Like all the other media industries that have

They would like to be able to archive data – treat adopted digital technologies before it, the cinema N it as an archival asset – if and when there are good industry is starting to generate an increasing solutions to the problems of data migration and percentage of important media assets that have O similar digital preservation strategies. The only no analog version – that is, they are not created on I thing that meets this studio’s definition of film in the first place. These assets are “born T “archival” now is 35mm film, so “archival preser- digital.” The growing use of digital cameras for I vation” at this studio depends on film prints and principal motion picture photography on “A title” positive/negative elements, plus YCM black-and- studio movies means that instead of original camera S white separations on film that go into deep negative at the end of a day’s shooting, there are N (underground) storage. His company’s film IP boxes of HDCAM SR videotapes or terabytes of typically goes bad every 6 to 7 years due to repeated camera-generated data files on disk-packs and data A use. If the OCN is in good shape when the IP tapes. The move away from shooting film is also goes bad, the IP can be remade from the OCN. associated with a reported trend to higher shooting R

If not, it can be reconstructed from the black-and- ratios, yielding more boxes of videotape or more T white separations. If digital files need to be terabytes of data, depending on the production. reconstructed in the future, the black-and-white The output of the Digital Mastering process is no E separations can be re-scanned using the higher- longer a cut negative in many cases, but rather ter- quality, faster digital scanners of the future. He abytes of uncompressed digital frames on magnetic H would like to find a digital alternative to film data tape. And with the spreading deployment of T archiving, but has not seen it yet. And until then, Digital Cinema to theaters in the coming years, the only thing the studio can truly depend upon is the use of release prints overall is likely to decline film for archival preservation. in favor of Digital Cinema Packages (DCP) for Hollywood will probably continue to archive digital distribution to theatres via hard disk, fiber new motion pictures on film as long as film stock or satellite. and film processing remain available and economical. Based on interviews, it appears that within The simplicity and dependability of film’s “direct the major studios there is no clear strategy yet for view” access compared to the software-based dealing with these new digital assets. Born-digital “interpreted view” of digital content continues assets are being generated in growing amounts. to be attractive to many in Hollywood. The Without a clear plan or direction, managers at economics of film archiving are well understood production companies, post houses and the stu- compared to those of digital archiving, and dios themselves generally seem to be taking the film archiving requires little new investment. safest short-term approach, which is to continue Furthermore, old motion pictures already in the the conventional practice of saving everything for film archives are expected to survive intact for the possible future use, and keeping the assets in their next 50 to 100 years, assuming the temperature original format – that is, putting digital camera and humidity in the film vaults are maintained originals on HDCAM SR tapes, magnetic hard under proper film preservation conditions. If for drives and LTO data tapes on a shelf in a cool, no other reasons, institutional inertia and the dry place until further notice. Some studios are

THE SCIENCE AND TECHNOLOGY COUNCIL / 16 T H E T R A N S I T I O N T O D I G I T A L 1 7 /

T H E D studios tional “born productive recording. percentage smaller, media entertainment consistently targeting originally formats demand ture material archived Naturally, show cinema, of the change h t m p b f t p m f n t o l t m o E to New long-term It o picture recording 3.6 I o o h h r h o G n w f r i o u i v e n c r t a a u a e e r o I is w

e n f T s this i m n t

n n i t film g t s d i e i n A u r m n changing d h l t

n - y u s One

s a m u movies L a

m r o t

s e e t

e s f e h u t e length: f e D c

types u w x

n reasonable a t film digital” v s o - a s elements s r s a t e studio’s I c u c

c The d b L s

- in e m h s to n with Motion r i r y a h for f l E t more t preservation d l

n a i k for created u e o d

l n today M u w h [ t o small studio

s i w t l e G h for “digital s r

l n archival o p d r LTO e e s f M strategy. u these

i t a i i

archiving

e will

output e i d a l An

n r v o

i t to Internet a n A a n l r l d e nor short m i n archived. e t h l

new t l n g r l

of e x r e s “ 22 s l 2

form l the . g

b i o e t x b y varied, e

o e e i d 0 s and b 135 c from a

x e increasing r r t t by business c d

w portable 0 y a v l be T Impact e a u t e o

that content

l data u executive television 7 are n at r

p t minutes l o

p u m a s a r t t h

and

i o World b t t archiving d i e f r h y c o e videos s t master” e i i studios. n i film-based

e rationales of e cannot i n e n b v a

minutes the

] factor s r

e t t o

a s e l . d p

l h e the c h d r

g more d e

distribution, a

w n n are s s a s practices. tape e

u r e e a

commercial e and l s e prudent t S

d c

t e e n

i Picture

s t n n u w r As o o desktop H l u e c o l s e

i l t e d c

primary i

f i

o v now

h media m n created a t n g t Wide

a d g y a v and D i a h m and c e g o of i o s n percentage are e customers

more e cartridges i explained types t t e be o a files programs. e t e o f g e

n s r i C s o m

e h But f m questions.

v r per s . i s n , o i e w f content. r c

t p

e e A r will u

is e

a a r f o e animation. t o preserved s m i or of

o F 44 “ i

” r l , t r r M interim o not n l t f t

Web,

f r

players e t u of

u c l

about level h

a t h i f l title, This

a a l s e numerous. a of

i by t l y r

r i l c even “content” r

e d a

t m r e t l h b minutes a e largely S s

c o n growing e - elements c

the h media l

r b o c . r h n

e l Digital R c

digital n h

e i

e suited m e a k m e the l i v i h i o w d i t n

r n r l and v that i is i get v Archiving c i v o i and is n and i n

e of

e g i n

e completed e e suitable feature-length i g There i b Eighty n i d r t d - g n tactic, t d v g are s neither s c u h l e t presenting studio y h g i

through

i e

o e e e a h v a e a n a r more e

c be

assets

putting

m o

b s he

, e e n e a

m i television- h content o e studio’s New channels,

p t per a f n t a n a not n a o volume d w u f a r a a s being r l produced t

l u driven e d f n i s p

o d A r t e

expects

z a o e t is e h e i

m d c g e l d i but percent i e c r z r t e r episode.

o o o

e for will of e e decreasing

s c yet e u a o e for i will m c growing e t s

digital . s n m g u h d l r p c a w h

Technology tradi- ” i

a r i u i t e s r motion them their p n e T a v n

i not s i s s c e film n c being f a y s fea- e n o the

a e e be by of c l

o h d e t be e , t l t s r d o the s

, r t h e o

a t b

u e a u i s b e m a r u s n e n - u . t d t t continued Preservation determine computer assets tions marketing within order video accompanied a T market medium Long-term further decreases, m As laboratories pressure n d d w H d s in consumer to the welcome “always commit stocks and demand manufacturing make innovators All specialty and dated entertainment t c i i i e o o i g a g a t o l g the u IDC c d l t n i i demand l a

are t t e k enormous l a a e u a a t Agfa Agfa, The The The y

s l r y i f l l w f about l . v o

to i significant created games to y ’ a

late l c d and s e

d o the big m c

depressing i H

i i and market n l t o e their f s n on get for a just u black-and-white theatrical consumer manufacturing i . trends t such

e d continue o

l r c on t r although is models, prices r film m

m g 1990s, the h and ’ i r w companies demographics film i I and s b e n e shrinks how n sales e and

their greater . forever.” a e a - u by also on D

viability three

s v

s m by for firms p t

c c A popularity e a game’s e film i l i peaked a a games a improvements, o o d g market r a research, chemicals, l the m film r l commitment , l i suppliers n s increase

r the e the new

a volumes

t the t e k film t t

a l but h e at s h h “ f l customer scenery e remaining l release to r

e o e is film d

to for t e games e Agfa customer e a l C

l simultaneous

s studio , a c s is u l f game a

risk e o already

investments y o o supply interactivity i marketing. t had with s c the w products. Nor

n at h w - n r e not m any leaders. of c a

e i e

market o e base f i l n o m produces

worldwide and n a i l 80 l of targeted m e l

stock e for f d of of r entertainment film m because g division declined l and

m their a r k such i even would - proud i consumable y

should a k

n d digital e t demand

I to

e t film-processing a e h high-quality, r t o t a n for shrinking i film quality awareness r vendors, e

l to e n e t g f y new t i l given software r e

t 90 r g e e

as for m s is r But i ” managements e c as

an a

n c a a e high-quality

n While c u

s a t r c e c histories also This o by their in e manufacturers, i release d “sunset” and e t million only l o d e the a archive v cameras. of

e sound

movie n l option to s

p m i i e e manufacture , none .

o n v a of

the film a s preservation control r w both p t f e the

t m i s about e

( collapsing n o digital r r Kodak, i e “play H D raises

l shareholders s and o print overall

e technology, t programs Kodak, and l a i p s

s sinking d n o film n o C p a

of studio R&D a applications. are u is a n d l l d prints reliable u theatrical the t in e l as I n c market. a

t y

t ) s has o

companion e increasingly t p higher d t chemicals.

value.” According suffers,

i r d new c f o film this technical r likely characters,

business a 35mm o

n o o i i will o digital f Fujifilm c v n n n

s o n i w a consoli Fujifilm c million or

t n t s in film d i s h a a e

due n of l o e f o film case. m that - a r ques- a and film i m w year g not r m b f l v sales

t to m r e e i a i a a e l r u r i to

d a - t d t m - y e o 3.6 The Impact of Digital Technology on Motion Picture Archiving continued

by 2005, and is continuing to drop 20 to 30% per migrate them to another generation of film stock, year [Hogan]. This weakens yet another pillar of or to migrate them to a future digital format with L

the film business that has historically offered valu- its to-be-determined preservation methodology. A able economy-of-scale advantages to the largest film manufacturers. Some old film assets will be selectively T I Several people interviewed for this report added to digital archives

acknowledged that the demise of film is a long- G The wholesale migration of major film archives to term eventuality, but do not expect film to I disappear in the next decade. However, studios digital storage is such a large and expensive under- planning their long-term archiving strategies must taking that no studio appears to be considering D recognize the risk that analog film archiving of this currently, at least for archival preservation in new titles may become more expensive and/or the strict sense. It seems likely that the studios O stop being a viable option altogether in the future. will start to consider digitally scanning some older

Is it prudent to build long-term preservation content to protect irreplaceable film elements T infrastructure based on a medium that, if not when they become dangerously fragile or deterio- totally obsolete, may only survive in niche markets – rated. Some old film assets will also be digitized like motion picture archiving? When YCM for commercial exploitation on a deal-by-deal N

separations reach the end of their archival life, basis, because once converted from analog film to O digital files the content can be more easily manip- archivists entrusted with these valuable corporate I assets must consider whether it will be better to ulated and re-purposed to generate new revenues. T I S N A R T

E H T

THE SCIENCE AND TECHNOLOGY COUNCIL / 18 T H E T R A N S I T I O N T O D I G I T A L 1 9 /

T H E D 3.7 encouraged and distribution sions d for film tape News 35mm all more daily on sheer difficulty real preserving included tion studios, networks television [US, According H h t I o m p t A tape their l few today, archives to I a G e i n f i i a o r L l I c s

i a

s e T g n t s c U-matic f t t the

T preserving v A h operational, t t S a e premium u a e h t

weeks assets i overseas The Network L s c o e

o o h r programs H is recorders to r s a t LC, archives c i production t d a D or e volume f i than p , r , Gathering n o o

e I O e news most

Television being y d L p m film. r i t e n all-digital n d n E i c even i policies.

v e of

c a M U n

g o

oldest o d both b n e a preserving have a t NFPB, programs e i e M of s f r h u c k before G t l these e to on c local t

r a t A videotape, d s e

systematic T m

t i they local channels,

e l t H p s 16mm o

i i v value markets

m replaced 2 r programs. r in o a e a a they from news r o i 0 what y

keeping

h were t film

a n for n r o been 1 l a 0 a t ’ i 1997 T c television e s i g e

d j 7 1997, 6 i programs news h d o

o b s v

d (ENG) r H

f needs p

broadcast. m the Television/Video], archives news , n i In e r e i their i

t v e c s

produced to s t s a v

I H

and film m r i divisions, s they w i p introduced held film n h S i n destroyed a s i all n fact, o a i v p report e a d videotape

o g

for their c by between R e s f n e consumer p e leading g

l t had n historically preservation. i

l a i n content c u of l their p prime-time

b E y videotape t t

m to n economic A t

judged

r t Many was cameras e w by r American

i HDTV p P f i e d archives the v n o i s a

their using r s r was r o

O l e n

corporate videotape, a implemented i c

o w i a o n , o g a by b d the

h c d R d every programs n d t phased o g r c c even growth m e h u i a

r a to T e due v explicit: u i s 1950 t e c the r s local in s n

reporters is is

t h i major i

r was t t t general-purpose s u o n e packaged I

s i e e

w gaps acquisition and o t s S i 1 d recycled. not r

l g assets. the h are complete s

f t e i day. s n in h to

Library i w 6 s s TV

v programming a A o

r u

i h

valuable t t i out n stations a l h historically,

and All owners. y s e N m

e of a 1997, a deliberate n e compact and on saved a o i 1980s. e s in

v studios l o because r t

d n s

they

e programs, c f o I e h n d o non-broadcast They

r m

the N after

a the

c asset-preserva- and t e

b o 16mm and/or 1975. h The l . a p h h o n o from m e V media of r a n e

n e y o

s But o more were s archives y E f represent n

major s i k public

simply even t g n for

u i

editors, i s Congress S

Electronic c 35mm focused r and reason r c l m i g a m T o video- g a e of m t e

video a deci m

n n t the

i Even p and I m the w h g few having sales, p i G v t than l the that m f r has e

i e h

o i a a o

A c e a r a t i t - i a n t n n i i of t - c e - for n a o n d t quality g d n g i t q a . t - v e u a d i n s l operational, advantage has less videotape to past Information b i t c c p h a v a t t a i f m c in formats obsolete of their better “format come quality f formats. then, with commercial improvements s s y a o i a a o l n o o n b c i e a a

a t c d p p r e

s l n n c l d c s d

i t a many the i m many e t e i n e e e

m o k c k u

v s i

o d

repeatedly

o durable l l t t R u f Since T M Over Videotape m a t e d e e r 5 a l s h a h e e e its t o

own e g l r m s t improvements. n there r & l a f down c w d o y e r s s devices s e

archives.

e i to a o p o l r c e s , i v

t has d e r i

o d h t e t ’ D o proprietary o f to e n e v wars”

t a r h o devices, h

incompatible s in n t e a c

i s b n 10

e Scanning m i y e e o o l

t . a

r the t o recording s v advantage. shifting

r e of h

s d e c d o , w r n a new

t o o e

m o gone - have e n t the e

B p medium

videotape

b just f i i

v n economic t d g a v l years, n v d Technology t y e o l v

l technical e i r m u l i a c r a b i o v e years, o d o e i

that

d t

a e s for h chosen r g t a s

since o r e n l u o recorder t

d e f e e

a a field.

g t y formats s o i i i

e l s t r i d s t n o

t s e o as s e i been v

up l h h a s c requiring , s a f l n u e

l l v

g e s y h d i

e c market l e i o c o a f

r e from

i a b

users d

still f

v o steadily. o e

o t c o a a v methods, p t p f

t l i e 2'' c e h

competing l e o steadily

e n a m (which n e i AMPEX n m e r f e l

u o d t r than d c s e d

a a

r r a a c t more l t n p to This o e t b

a a recorder p v p a e formats i advances e and/or l

g o n h Quadruplex a

o o f f s

s n i i

a t r a i to e r vendors t n e v 16mm

n n n t u p have o e i d t a i o o n “work o a adopt , h o

p a i l

(IT) f f e t m t l t share, a d

m p m i e p u i r n a u y e - film), d h r 3 access b e r migration a e t t t a . e has e n r e s m s

e is o 0 than

l i s

o a

while s e

c ,

w r

e f

e d V

r [ o t

t t signal u . t I

o sequentially c

s s r introduced y K

e h m recognized a r quality i m 3 t infrastructure e - u

u H o

d a n r t e that r h p d o T in tape left a vendors d 0 film - h i in c n

c a e c a a churning e that c i t f e i r p have S the h

i t r c

t s h t s s r

60 g the n e y n

t u i s

h a t u k o

1956, m e c o e h l e size progress”

e t e z c a a g r d o a a

e m o o video

a n l run a s e e encoding n tape H

r a c a e n v t e formats acquisition r

r m different

broadcast m l

i offer

n value H c t , n D

e f s e e d a n a from g f

improvements. engineered m D a

o r ,

d o c o m n “ u e e f and i g

s i

D V r n i u c B v i d

r n

m d f

C only have d n n s e r e H

H a i

the format. t i D

r o

C v t t i archives l n the u u to c

A m w m a e y n e adopted h out i t g u near-term o D t d r m r a d A . of h r

cost old M g e u o

n k a e e l a k e e ,

be e according l

M C VRX-1000 to for r n r m .

e t a d W f d y on

t i o fought first

k w i u and the n videotape o t p u w S new

A n t

f industry t r e t

tape g i a i a f f take a R h S l g u e m have o s M l n i to a o s o the p , s l r

R c e

” r

o c

much t v l r w many i e Since assets f o a n

e i

, i c full t image

d , i o t

m 1 n . S to

u r r and a

s b

a - r o t d t ] l u e R r m o e h r i l , n i i l y t

- e e f n n i i e - , a s n t g g d

t 4 Current Practice • Other Industries

ONE OF THE MOST OFTEN-ASKED QUESTIONS WHEN DISCUSSING S Hollywood’s digital archival issues is, “What are other industries doing?” Many The scale of industries have already adopted strategies for the preservation of their digital data E collections, and while Hollywood’s digital assets are large in number and size, they are I

Hollywood’s not unique in these respects. Potentially, Hollywood does not need to invent digital R archiving from scratch, so the risks of trying new approaches can be kept relatively low archiving T by studying others. requirements There are several areas of modern society that have large collections of media S assets of various types with archival requirements similar to the needs of the cinema is not so industry. While the digital motion picture assets that the Hollywood studios want to U

protect are exceptionally large on a per-title basis, the scale of Hollywood’s archiving D different requirements is not so different from that of institutions in other domains which also require long-term preservation of very large volumes of valuable pictures, sounds, text N from that of and other types of data in support of their missions. I institutions Advanced visualization technologies have always been supported by three “pillar”

industries that drive the state of the art: entertainment (cinema and publishing), R defense/intelligence, and science/medicine/education. Historically, they have all used in other E analog imaging techniques developed for their particular needs, with little dialog or domains technical cross-fertilization between them. However, with the widespread adoption of H high-quality digital imaging, all three pillars are moving away from film to common which also digital platforms applied to their different purposes. All three have a growing need for T digital archiving of still and moving images. All are facing similar challenges in terms O

require of infrastructure, workflows and requirements for long-term preservation. The defense/intelligence and science/medicine/education communities are already operating long-term large digital image archives and can provide valuable reference for Hollywood studios • initiating their own digital archiving programs. All of these large-scale, long-term preservation digital archives have refined their system designs over the years to accommodate data E ingest, search and retrieval, as well as relatively efficient and reliable data migration, of very large C file format updating, auditing, quality control and (when used) discard/transfer I volumes of processes needed to insure the accessibility and integrity of their digital assets. They also anticipate long time horizons: 50 to 100 years, or even “permanently” in some T cases; that is, for the life of their particular enterprise, assuming adequate funding. valuable C Other media archives are already transitioning from analog to digital. Many large pictures, public libraries and archives with extensive collections comprising many media types A are preserving their most recently acquired content digitally because more and more sounds, text modern media are originally produced digitally, “born digital” assets that are delivered R

to the library on some kind of digital storage media or directly as data files via a P and other computer network. Users have generally appreciated the faster, easier accessibility of the digital collections offered by the libraries. In response, librarians and archivists are types of data T digitizing their most important (most popular) analog assets to make them more in support accessible, too. Other assets are being converted from analog to digital when the N analog media are deteriorating, putting survival of the content at risk in the absence E of their of a reliable digital preservation strategy. Similar trends can be seen in several commercial media industries as well. R

missions. R U C

THE SCIENCE AND TECHNOLOGY COUNCIL / 20 C U R R E N T P R A C T I C E • O T H E R I N D U S T R I E S 2 1 /

T H E D Section operating system production. data gigabyte practices preservation. volume the tory seven-year data action. m t the corporate century, the Sarbanes-Oxley duration archiving most involved IN 4.1.1 trails, “Write i m p migration, specialized available longer years, motion niques T N 4.1 I n e G r e c

a e e I o r T R h duration record The a n t m p A TERMS i archives, life corporate n e n t o A This e L requirement than replaces q

s o n r significant t S D u

Sarbanes-Oxley rigorous comparable l h l further t Corporate a t period I y and o i costs, L

h Once, cycle r e l r picture 6. of

The E g

d a e only and M R in today reviewed of d y t contrasts m t and e is data perspectives. most e M

. e e corporate mandatory s discussed g

digital t e legal f s i

long-term A h archival y generated n e g

S e OF archived

so of This e old n terabyte from

2 a n t [ O requires accounting of Many

s Hollywood contrasting Read 0 R r archiving s e archive

for access c 0 can i f archives X d implementation a v o Act o h 7 difference compliance. time THE

e data r s

single

for c of a e

preservation C has to seven for D o d both f n to significant reasonably f r o a Many” of t preservation. e i of p data later t h existing “protecting u g c control, a the range, than data o that archival i this significant a archival t preservation preservation n by period

t 2002 TECHNICAL s

r l the p a c This

generation

a be initial years, m 3 l r i t

l e scandals a Hollywood that e ; P in ’ parameter archiving

s report s o Act can

is America between N any studies preserved e r

single d (WORM) s R e r t which a this (SOX), s always v a topic l film e e starts t y data t a c requires a r support period. a and capital be l strategies

o t v digital

. t Requirements i

a m a r emphasizes term o R report. the r impact a were t digital n

discarded c m i archives. n at e For on o authentication is h

of of of s is at s s e n corporate is rolling e preservation. i a t n is passed v the unalterability” covered

a r for c goal substantially difference S digital Hollywood marginally certain a d digital corporate investment a the r the technology t

storage, DIFFICULTIES sized example, this l c y the without t i a There

o e s h s t

t such motion turn on g 50 i n y Nonetheless, e

o 5 i time m s intended a that e n storage t 9 level l over

s e as s

s after in overall That to the

d m storage assets, d f a 6 types in of o a as is record n e a 9 s r t the audit 100 digital of s

d using a

] use c i a IT detail result the a as of data . this , m picture

r the within

t is and statu- w trans- i tech L h b p less new of o e h the e a of of l n e d i c - g - in of h - between archiving

nature the rations enables data and 4.1.2 goal archival access versions with reliance), of solutions heavy they data on generate masters), about geological mon be preserve rithms size any geological valuable have it d I N a was t

standard identified a hundreds I motion

of N immediate IT Finally, According The s for storage began are data something with y T and 25 learned to Oil s oil of improve t with E infrastructure, significant e this practices experiencing of the than hundreds R oil m valuable and which uncompressed motion ad migration, corporate deposits the data data V undesirable Exploration

old to picture m file I is industry requirements original the industry. IT hoc E using of a a the be motion that W n hardware to set must value formats typical the vendors magnetic over a in limit tight S archiving archived g and picture raw data.

the e W can to economies of the common: “old” r industry,

captured a time, I be and – has be data problems future integration n T people data captured intended picture as reliance oil be survey d that H

(which heavily Therefore, that

4K extracted. well and a data. another

production. a data 200 A and

across more sets, d and formats procedures, need

value a digital L freedom are given t interviewed, the software A as a of of industry: data: terabytes motion tapes.

on R that and data. enforces s processed that than the t not the scale to corporate G derived o of characteristic being vendor-specific r E

the motion are a standardized maintain The new is is

relatively business g sound Gulf O easily

a e of That and normally the

to picture I a specialized decade s no L a e (the the choice, single-vendor processing oil long-term

r data guarantee typical C v of before incentive raw close experience achievable i America, familiar: O picture is, c deposits location e Mexico size systems M working

common is c captured industries geological ago, o in P stored collabo- a more m raw it of A lack com- N p has and algo- a to Y a can and n can ’ in y S , 4.2 Government and Public Archives in the U.S.

THE LIBRARY OF CONGRESS’ NATIONAL DIGITAL INFORMATION S Infrastructure and Preservation Program (NDIIPP) initiative and the National Archive The task and Records Administration’s Electronic Records Archive (ERA) program have both E of saving put emphasis on collaborative problem solving, drawing on the opinions of experts I

from many fields and providing forums for valuable exchange of information that R serves the purposes of these institutions and contributes to general understanding of digital the challenges and possible solutions to large-scale institutional preservation of digital T assets is assets. The Academy participates in both of these federal government initiatives, as a S member of NARA’s Advisory Committee on the Electronic Records Archive (ACERA) too large for and a partner in the Library’s Preserving Creative America program under NDIIPP. U solitary 4.2.1 Library of Congress D ACCORDING TO ONE VETERAN AT A LARGE STORAGE MEDIA N efforts. I manufacturer, in the late 1980s the U.S. Library of Congress said it wanted a 200-year archival life for its digital assets. The vendor’s engineers went to work on

accelerated aging tests to try to meet the Library’s goals. However, after several years R it became clear that no digital storage scheme available then (or now, or in the E foreseeable future) can be sustained for 200 years. The Library realized that digital

media are so ephemeral and digital technology changes so rapidly that long-term H digital preservation was going to need a new approach. In December 2000, Congress appropriated $100 million for the NDIIPP collabo- T

rative project in recognition of the importance of preserving digital content for future O

generations. Led by the Library of Congress, NDIIPP has generated digital archiving guidelines that are useful for any organization that is formulating its own strategy to •

collect, archive and preserve growing amounts of digital content for current and future generations, especially materials that are created only in digital formats. NDIIPP set five initial goals for the Library of Congress and, by extension, for any organization E

faced with digital archiving challenges [US, LC, Digital Preservation]: C I 1. Identify and collect at-risk “born digital” content that is created only in digital form, before it is lost, misplaced, goes obsolete or becomes corrupted. T

2. Build and support a network of partners working together to preserve C

digital content. The task of saving digital assets is too large for solitary efforts. A

3. Develop and use technical tools and services for digital archiving. R

4. Encourage development of strategic policies to support efficient and P reliable preservation of digital information. Document the rules, and educate the staff; technology is only part of the problem. T 5. Show why digital preservation is important for everyone in the enterprise. Saving information, especially the right information, has to become everyone’s task. N

The Library of Congress has an additional motivating factor for the development E

of digital preservation technologies and practices: its National Audio-Visual R Conservation Center (NAVCC), located in Culpeper, Virginia, will house the entire collection of the Library’s Motion Picture, Broadcast and Recorded Sound Division. R

The collection contains increasing amounts of digital materials, and the NAVCC’s U digital storage system is expected to ingest over 8 petabytes (equivalent to about 1,040 uncompressed 4K digital motion picture masters) per year when fully operational C [US, LC, Natl. Audio 15]. Furthermore, as the repository for mandatory copyright deposits, the NAVCC system must consider the copyright term of 120 years or longer. It is clear that the digital preservation concerns of the Library of Congress are quite similar to those of the Hollywood studios.

THE SCIENCE AND TECHNOLOGY COUNCIL / 22 C U R R E N T P R A C T I C E • O T H E R I N D U S T R I E S 2 3 /

T H E D 4.2 (ERA) therefore complex of capable preserve will business that N f 1 A a d m 1 u r a o A T 4.2.2 realized within deliver ment statutory right covery the tion, any with the U.S. aligning the developed commercially “lifecycle” priority, attack funded and management with selves, I o e G n s f

9 n a n d H

f t r f e A I t d o T t e c o i data m m

n a h c A r l R e

E f

federal U.S. next-generation

a type t i r have e professionals. NARA As N f 3 v

L going f a i a

s e i government,

computer NARA’s A c n on i of n a e % t D c U records, s l l f e A .

r the n i c e ’ so i i l I Administration National and program

certain Government g n f assets s L e a s e the objects. e s . research

R i

access, E they t o

S c l o

H e t s . NARA any “permanently” will h defining government’s M r s

problems of w

u t m requirements to d A established ERA v . f h i a

M g critical o r N e to management handling

t e p forward N t i ’ government l 21st a ( y delivery n i r s s h w A r record, d archiving u o o accept, l t n

A e i A computing

e l use could c o t i to 2 support p f n e n specialists,

e m n h

viable, i essential u R 0 o with v r T time d r e c c

established g y now a 0 scientists, r (

e r n a

h e A o the I n with century N to r t s 7

i projects, NARA r n n ) electronic O g a

h c the h preservation

t n u o t n i A

a Archives and m

a n m

requiring N meet Through a a p more r m e d not t created rules r r limits. catalog, R to Archivist s national a v c

thousands and n e e d o

d A the mainstream l h e e t c the A b r requirements o l e-commerce, responsibilities s

e o i

n y o i a L t anyone t g e in u b v r IT ) archiving to e

do t r or e

in m recognized e g o c e of e c s d began, archivists, platform, i c t their A for n Electronic that s

e r s s and h overall h a f the the o engineers, s g . preserve e

n R practices l

a , any n r b a t or

n c e records.

a k

using records

e The h h B o b n o o C o t search, t c network e the new s workshops, e e long-term a information o n

o e r t l y

p ERA’s process of compel think more H d b o

i h t visionary e and r l

t v e with n o problems h d donor; a a N e g

of h

e l and NARA a I n

t c

t e i e life y the ERA

r i direction a e V n s e l A mission. m t c s n

kind t any technologies

a d d formats d o i i g e t e E R from f s b n all d r

industry

s a e-government o that s in

that Records official b give to of a o Records s an p S l

United They A e i information r strategy

e r i u a v y f n t r l

d n of perhaps are m

i type record ’ r y official team o A a

o s terms f e and s i

align everything c e t of the e

i interest o c

t d h planners a N a l planning,

e goals: Public e

a o e partnerships

NARA a these r any d access t s

e r as s symposia n c mandated

i b ( i t digital D

t e i

p t infrastructure, o n t s h

W want and

v i f of l h c Republic. of provide

As e e m e r n e government c the e e

t at o e e l c d States better experts

R h e r

e of entity Archive o has

creators f r s a r 1 IT f s e

govern- applica- a c o e d e e r leading r E t are such, t e preserve NARA r 9 r t

trillions e l r o being t h r to s . a and first c the r d r C v )

, 6 itself

to m l e e o h

in archive i

realized i

been o 0 A a u c n O n W g n i very and them 1 and and

n o n v s t o use g

t and i . in 8 dis u R the e f r c d h l legal v by a

y e s d 0 Archives r a e e r D . w

to e l s

0 r - l to I n s S - I . -

harvesting, and NARA system. four system million 2009. Bush’s accepting consensus issues NARA/ERA N THE 4.2.3 a topics from h l t a w t m ( g i w many DoD historical, these that single strongly formats, consumers and expressed serves Office need this dous was, a t a n h h D i u r o a o r h i digital c t a o e e d s v t g o n o h i t e D h h t e e i c i t i D

o i o r system o , metadata, the The While Also h o n a are v other a “Don’t a n h v to overlap n o basic u n n technologies i t ) e d and ACADEMY’S of large i m organizations, as e

i s such . D a s t s p of s d Department t r O administration NARA at i d believes

u l a email e

implement DoD,

A part e c a o s ’ Although believe library i especially d long-term a in s head ERA s n i o s t n c interviewed the a u g that l A much the intelligence t organizational n r t in t m d h y e federal i f u issues of one c a d are t and digital- e p f as s r

e d h a r e d p t a a let o g v of a e is the l e between v the i r Director i g r d r electronic digital r e messages v i n m

of metadata a f time, e a project e t e expects as in n f critical e t n in n m i s might r h the r a n this than that those s n that c d s i interest, o h d t c ’ e e the a field. c i of g A well a e collaborating e m e e u t n i e a agencies m n m d y n on i s e m the with s s s m , digital Intelligence C t in o s

i position D t digital i i happen and . g i the . U.S. u r v it media Intelligence even T a PARTICIPATION a n a

E n a o y n f were e

m n the their t as consider technologies m is g t communities the i ’ when to T s to c system of R e e

digital of is s s e a t and p e l r d p t b h r respect and u r h g records A use phased, r i r automated l is e o t e both a e not and the e y i process a National d e e e d motion c b r l v as Defense p future. m o n

l archiving i r i s o representatives to currently storage e i a r u n own. a s o e m to a n o attachments

v r p in e of t r g digital he t f s m to large c l a c i system’s e continued n economically a c e t u technical t n develop archive, h is y n c d a a b c develop on t m m Hollywood.” h standardized e h g p k a t the i with t L t dig i to leaves Community, v s e p i

of still a l e e e i with v s e f c i e d

Community. l t and D s a e i t o e l o As the t picture r t s , b as y

a e metadata. i s e to w e r

President Intelligence, l r ADL v deeply and n r l

t l object f e

s e e e ingest o b o n are i h t l and e in large p g working the p t c a o it a t a i n more i a ingest part r i o the f long-term s n a office e and n there n r a r n t t h t a standardized r mmendation s is, r development, reasons, o g

a i g g practices populated d t o e h i h n P n t v m DoD - e i into help A system r i e i a t i industry r o f initial o ON s producers n for d e d from registries. v of r o into e s e and test s n D feasible o r g

complex e u e u n v b is m e over t s file in m s j

e s e t George

h e L f p this a e e a the o r various to They s with build c

m a e t a v r s b r THE and p the v t

a metadata January w o r the r f i ( They l y the the who a t tremen

e r D formats t e e more goal develop m i success. e e m e s n w e A used r 800 a ERA office r t e s g w g t e i with D o base will e e n e c a those for e i m f

r h n r n l file d e f and v L s o W. e t of g n i a ’ s ) of in s s s - s , - e 4.3 Medical

THE HEALTH INSURANCE PORTABILITY AND ACCOUNTABILITY ACT S of 1996 (HIPAA) requires hospitals, clinics, medical equipment rental companies, Digital physicians’ networks, dentists, drug stores, medical insurance companies, medical E medical billing companies and nursing homes to preserve and protect the privacy of electronic I

medical records, including diagnostic medical images. HIPAA requirements are, in R practice, designed more to protect privacy and promote operational efficiency than images are T comprehensive archiving. The motivation for archiving medical data comes from its

“born benefits for research and education. According to a report by the National Science S Foundation’s Science Board: archival”... U D all of a “It is exceedingly rare that fundamentally new approaches to research

and education arise. Information technology has ushered in such a N patient’s I fundamental change. Digital data collections are at the heart of this identifying “change. They enable analysis at unprecedented levels of accuracy and R sophistication and provide novel insights through innovative“ information

data is integration. Through their very size and complexity, such digital E collections provide new phenomena for study. At the same time, such collected collections are a powerful force for inclusion, removing barriers to H before the participation at all ages and levels of education.” T diagnostic O And according to the Mayo Clinic in Rochester, Minnesota, “cine [filming] for image is radiography has served the medical industry’s needs well since the early 1950s. For • captured, the first time, it allowed recording of motion studies of the cardiac structures on film. The cine technique has been standardized over the years, both the camera and the E display. Cine filming techniques, however, have not advanced, except for new film and the C products with faster emulsions and better-quality films. Video imaging for cardiology metadata is has made rapid advancements.…With the advent of interventional procedures in the I

cardiac catheterization laboratory, the need to assess images immediately cannot be T forever fulfilled by cine filming because of the requirement for the processing of the film with its inherent delays” [Holmes, Wondrow, and Gray 1]. C

associated According to the Cleveland Clinic, one of the largest hospitals in the U.S. that A maintains a substantial digital image archive supporting both its clinical and research with a activities, much of the medical field began its conversion from film imaging to digital R

imaging a few years after the Holmes paper was published in 1990. Currently, all of P captured the Cleveland Clinic’s medical imaging (including radiology) is done digitally, and its plan is to keep all digital images forever for historical trend analysis and research. image. T The current film holdings are not digitized because it is not cost-effective, and the

old films are stored in a cool and dry warehouse. N The Cleveland Clinic’s archive currently stores 1 petabyte of digital data, composed of objects such as chest images that are about 20 megabytes per image, and E

motion clips that are 500 gigabytes per patient. Image data compression is not used R because life-or-death medical decisions are made based on this data. The archive is growing at a rate of 3 terabytes per week, which will double its size in the next year. R

Image pixel counts have increased and more images per patient are being made, so U this growth trend is expected to continue. Their storage strategy is to store the most recent data on an array of magnetic C hard drives, and there is currently 100 terabytes of such online storage. The biggest problem with this system is that the disk lifecycle is only three years – every disk drive must be replaced after that interval. Hardware is not the only cost; all of the

THE SCIENCE AND TECHNOLOGY COUNCIL / 24 C U R R E N T P R A C T I C E • O T H E R I N D U S T R I E S 2 5 /

T H E D 4.3 collaborated and between the their file introduced positive. format and 12 per every tape certain getting as data I G I T the blocks A formats vendors. 100,000 the the The L library, must technology, D 90 archive I Medical standardization, L point, longer E National two vendors,

M days. medical When away M be in A that with are on

operations.

2 copied When it grows, 0 the and

0 The from the linked is 7 digital were

industry and Electrical every the 1970s, automatically longer. development reliability the when the the American designed it continued by medical tape but was users

time primary A all a has

the Once Manufacturers secondary fiber-optic it a vendors automatically wanted successful was requirement required some hardware to imaging College transferred the of archive lock not the experience had data to archive particularly their connection. for Digital of strategy move is devices proprietary at ages evaluated replaced, Association copying Radiology is to the users zero is data a to with Imaging Clinic, located were data for a into errors is and file account, address, That are done and so p implemented with dard, and difficulties after picture with is r o captured, that p “born other Communications It r DICOM, a image i is, in or e is captured t interoperability industry’s a proprietary all date DICOM r interesting etc. y archival” reasons,

with e of capture, and x – t of differently e a n is reportedly, patient’s birth, metadata image. s the i collected o practice [Digital archiving n with image to s metadata and

h attending in note a is

v This respect by identifying the e Medicine still management.

of formats. b Imaging before is each that e of associated e generating that contrasts not n is

medical a to forever physician, digital d manufacturer, the the achieved. d essential image e 5]. data d diagnostic

standard b with

downstream medical y images associated The metadata

– m format billing name, a

metadata. the n For problem u is f is a and motion images image c this stan- still t u r e r s , 4.4 Earth Science

THE CENTER FOR EARTH RESOURCES OBSERVATION AND SCIENCE S (EROS) is a data management, systems development, and research field center for the EROS stores U.S. Geological Survey’s (USGS) Geography Discipline. Organizationally, the USGS E film today is a bureau of the U.S. Department of the Interior. The archive contains aerial I

photography and satellite remote sensing data of the Earth’s land surface. The EROS R mission is to preserve this data “permanently” and make it easily accessible and readily

in archives T available for study. Residing in the USGS’ EROS Data Center near Sioux Falls, South for the same Dakota – one of the largest computer complexes within the Interior Department – is the S National Satellite Land Remote Sensing Data Archive (NSLRSDA), a comprehensive, reason permanent, and impartial record of the planet’s land surface derived from 40+ years of U satellite remote sensing. Aside from the larger question of change at the global scale, D Hollywood NSLRSDA permits scientists to study water, energy, and mineral resource problems

over time; to help protect environmental quality; and to contribute to prudent, orderly N saves film: management and development of our nation’s natural resources. I Over the past three decades the U.S. government has invested money to acquire film can be and distribute data worldwide from the Landsat series of satellites – more than reasonably 630 terabytes of which are held at the EROS Data Center. The archive also includes R more than 28 terabytes of data from the Advanced Very High Resolution Radiometer E (AVHRR) carried aboard National Oceanic Atmospheric Administration's polar preserved for H orbiting weather satellites, and more than 880,000 declassified intelligence satellite

100 years or photographs. T The primary objective of NSLRSDA is to preserve entrusted data records more. “permanently” and to distribute this data on demand to a worldwide community of O scientific users. As a result, the EROS Data Center has become a world leader not

only in techniques of archiving remotely sensed data, but also in getting the data to • end-users quickly, in forms they can use, at costs they can bear. According to the

archivist at EROS, every advance in online distribution, in storage media, in applica- E tions research, or in cost-saving delivery technologies means more people can use the data. As demand increases, user expectations about delivery times and efficiency rise. C EROS archives also contain approximately 4 million satellite images of global I

scale and 8 million aerial images of the U.S. Images held are stored both on film and T digital media, but almost all new images are digital. In 2004, the archive included 80,000 pieces of film, and by early 2007, the film archive had grown to 110,000 pieces C

as other agencies send their collections to EROS for archival preservation and scientific A access. Film assets are preserved in climate-controlled film vaults that have been inspected by the National Archives, which estimated a 100+ year shelf life for these film assets. R

In 2004, EROS archives held roughly 2 petabytes of digital image data “nearline P and online” in robotic data tape library systems and magnetic hard drive arrays. It took 30 years for the archive to reach this size. In 2004, the archive was growing at the rate of 2 terabytes per day, and EROS forecast a doubling of the digital archive in just four T

years. As of this writing, EROS archives hold more than 3 petabytes, on track to match N the 2004 forecast, and the archives continue to grow at the rate of 2 terabytes per day. EROS stores film today in archives for the same reason Hollywood saves film: E

film can be reasonably preserved for 100 years or more. The question is: at the end R of 100 years’ life of film in archives, does one make a new film copy or make a digital copy? Currently, EROS does not keep high-resolution scanned data from original R

film images; it scans on demand from the film archives, with full-resolution scans at U approximately 7904 x 8512 pixels, approximately 800 megabytes per frame, at a cost of $20 to $30 per frame. C EROS provided some interesting observations from its experience building and operating a large digital archive: the operating cost is proportionate to the number of times the data is read, and the risk of losing data is proportionate to the number of

THE SCIENCE AND TECHNOLOGY COUNCIL / 26 C U R R E N T P R A C T I C E • O T H E R I N D U S T R I E S 2 7 /

T H E D 4.4 moving and place intervals time early as of collection They emphasized Canadian for ness, had archives They tant growing and do disks opinions times I G I T a losing not A both possible but power investments EROS L not to data requiring appear every D prefer related data I have L Earth execute E toward one that to cost. of before M a to migrations – rather M tape collection, consumption become 3 also these A other that at to suitable supplier. attractive strategy

chose a 2 to

0 Furthermore, a story invest an 0 is 1992. the and has 5 7 cost this in than

organizations accessed. digital years, immediate Science a robotic SUN dependent experience were about long-term technology to of was were in and

potentially

from The Since millions ensure the and archive of a T10000 performed they very two lesson

library supplier magnetic EROS migration a EROS based 1992, and cost long-term on with – different, believe reliability expensive, of operators called one experience continued not unplanned a tape systems perspective on expressed dollars. migration said single went data at European hard to lessons costs migration CREO 7- technology) it be migration. large characteristics, access. out to technology. took that was (they drives

forgotten. They of the 10-year similar migration of learned has but digital a which impor- optical a and “cost” busi- are

is works Their long taken they a and one archive, but tance. desirable be learned third priority during after EROS other preservation it versions Their is • • • disrupted is inevitable Since In library First Third Second including archive efficiency-enhancing goal planning

is Today, the migrations. as has such of to getting copy the a copy is opinion the or build been result copy functions/systems for to – that events on that – EROS data 100% for to – keep near-line/online, extended easier, placed magnetic physically – offsite EROS of establish disaster distribution offline of in of the the read-after-write has the the faster September on data 9/11 best-quality will only people periods. archives. “basement protection. technologies, hard a a offsite 100% full storage and have attacks will a library drives few interviewed offsite less on

three 11, merge. Therefore, redundant terabytes within data, a that data expensive, tapes”

and 2001, But robotic system archives: migration air and verification long-term EROS driving a at travel offsite, it tape offsite higher fewest – EROS, is even a also more at can dis- 4.5 Supercomputing S E

The primary causes of digital archive loss are I

human error and magnetic disk hardware failures. R T S

THE SAN DIEGO SUPERCOMPUTER CENTER migration and a program of ongoing data verification U (SDSC) operates three supercomputers for the national to ensure data preservation. But there is a level of research community and provides supercomputing uncertainty to data verification because the act of D services for a range of extreme computing needs such reading data for verification increases the probability of as astrophysics visualization, bioinformatics and other error, but not reading data for verification can allow N science and engineering disciplines. The SDSC latent errors to accumulate unnoticed. This topic was I also operates a large hybrid storage system with 2.5 covered in detail at the 2006 Eurosys Conference petabytes of online magnetic disk storage, and 5 [Baker, et al. 3]. R petabytes of near-line data tape storage, with a capacity SDSC’s biggest concern is not with technological E of 25 petabytes for the robotic tape storage system. For obsolescence – they think the biggest risk to data perspective, 25 petabytes is enough to store over 3,000 preservation is gaps in funding for system maintenance H uncompressed 4K digital motion picture masters, or and data migration. To address this concern, SDSC T approximately 5 to 6 years of MPAA-rated motion expects to begin allocating storage costs to system

pictures. SDSC’s storage volume is doubling every 14 users, instead of just billing for computing power as is O months, and they expect it to grow to 10 petabytes by currently the practice.

the end of 2008. Computer networking is a factor in the SDSC •

According to those interviewed at the SDSC, archiving picture. In a collaboration among SDSC, data migration, or the copying of data to new storage the National Center for Atmospheric Research located devices and media from those becoming obsolete, is a in Boulder, Colorado, and the University of Maryland, E

fact of life for them, “like painting the Golden Gate these three organizations have agreed to share archive C

Bridge,” in that one must continually repeat the process resources that will only be accessed if one party I to avoid decay. SDSC’s migration period is five years, loses its primary copy. Despite some opinions that and they say one of the benefits of migration is that the archives should not connect to networks, those inter- T

newer storage technology is faster and denser than the viewed at SDSC recommend that operators of digital C old technology being replaced. archives learn how to use networking because they feel One of the factors determining the migration it is faster and more efficient for debugging and A

period is concern about “bit error rate” (BER), which is problem solving. R a fundamental measure of a storage medium’s reliability SDSC is also collaborating with the Library of and integrity of the recorded data. BER generally Congress on a pilot “third party repository” project P increases with the age of the storage media, and there- designed to address certain risk scenarios, e.g., stream- fore the oldest data tapes in SDSC’s collection are only lining the number of file formats and file systems to T 6 to 7 years old. However, they say the BER is nearly support. In general, they think very-long-term data irrelevant to long-term preservation and that the pri- preservation and “cold storage” digital archiving are N mary causes of digital archive loss are human error and unsolved challenges suitable for future research. E magnetic disk hardware failures. SDSC relies on data R R U C

THE SCIENCE AND TECHNOLOGY COUNCIL / 28 C U R R E N T P R A C T I C E • O T H E R I N D U S T R I E S 2 9 /

T H E D 4.6 I G I formats.” proprietary create vendors the letting we mistakes the “Don't T A L D I Summary L made E different same M M A

2 all make 0 0 7

by p T 4 of have while the and THIS c w c u a i h i . c n s l 6 long-term t l e s

There • • • • • • • • • • • • O •

u motion e r m . b long-term 1 e d r Currently Economics The order system, It technologies, Biodiversity, long-term An (and is Unproven There The The The Multiple Good each are a e f

e a to

i SURVEY

i p substantial k is s i “store i n n

n a e o extensible g summarize

the

very d number m cost total stakeholders, e n t a t is time they h n u i

to project s c u is p e picture s e both u e limiting c t

e r digital so f r continue definite l h a r and n copies difficult u of a y f l cost preservation

there r e

should

or t s a management m a the t

u force c u n labor, h OF g t major r consensus o of or

o exotic amount a r s ignore” e management c is r e t file . t industry’s of

e h

initial : e e these data V file

spreading

is economic m of important; o

OTHER T d i economy an ownership i i s to be e e c system h using e no

and not important t data t e n formats i w a h technology s

preservation. ongoing change t i o

chosen issues a

l digital choice long-term i of

o f t s i the

secondarily needs

s n n and w e u archival migration firsthand

v costs conversations h e t S e should h INDUSTRIES , a technological e vendors, factors r

of as e a t c y i.e., disagreement and is

vendors s t is alternative is carefully) o t p assessment revealed i

scale digital o well n o i essential a t much critical.

n e r

s do should a t

file preservation a

i system p 6 be v n o experience. in p . e as f t is not 3 the in

e l should systems a t i chosen . r digital data e more after h done the n v digital s a o d i

be cost to not e about t Make s set in

to o w e w software risk advice

of among keep e dependent should i t WITH h all even n than h analog d drive of be a of archives t to future e

t e archiving used f across

migrations

o digital for r common m sure t used electricity, v keep o r labor

i one o offered

just e t d requirements t h those 50 w be i i LARGE should after i film o s to value e s c n

d several down year maintained media a r to archiving,

costs

e on r negotiate p f d p r systems archiving i interviewed 100 issues o , by o c its

a t m o r of of u be not t r single

those license down

DATA a costs o e assets h years using different g

minimized u o i r technology, that n t e w s and e and d i

up d d u v who vendor if

e a s a t expires also

t l STORAGE h the front t standards u on r it new h a y a t e , b by is

i n key goal l arise m s e o worth-

in d now o a

o i n t s issues i n

- o in a e n s s e t 4.6 Summary continued

4.6.2 Unresolved Issues S

In discussions with people outside the motion picture industry, a wide range of opinions E

surfaced on a number of issues: I

• Is data migration most effectively done in-house or on an outsource basis? R

• Is data compression an important technical consideration? (Although T

those interviewed all supported “no compression” unless an asset is S “born compressed”) U • What data should be saved, and what should not be saved? D • What level of geographical separation should be achieved; i.e., how far apart

is far enough? N I • Should a primary and backup archive system be connected, or not be connected, by a network? R • Are standardized file formats necessary? E • What is the best digital preservation strategy? H

The issues of standardization and policy-setting merit further discussion. T

Standards and Policy O

With respect to standardizing file formats, some recommend converting everything to a normalized or universal archive file format upon ingest, usually based on a widely • accepted standard of the day. But there are many contra-indicating examples of standards not staying “standard” or going obsolete. Some recommend archiving in the original E file format submitted to the archive. This avoids having to normalize formats upon C ingest. This approach requires the archive to be capable of holding many different file I formats, including, potentially, original data creation (acquisition) formats, intermediate data processing (postproduction) formats and final data delivery (distribution) formats. T Returning to the motion picture industry, one of the major Hollywood studios C stated quite clearly that it is not concerned about picking an eternal universal archiving format, nor is it worried about compatibility between different archives. When they A need to, they say, they will convert, or “transcode,” formats. They say it is better if converting can be avoided, but it is not an insurmountable obstacle when necessary. R

It “just costs money and time.” This studio recognizes that it does a lot of format P transcoding today, and they assume this transcoding requirement continues as a “fact of life,” but probably will become even easier and faster in the digital future. T Some of the people interviewed for this report believe the stakeholders – the owners and the archivists – should try to influence any standardization process to their advan- N tage because they will be paying for the long-term preservation costs and thus should E have a large say in what is created. They said, “Don’t make the same mistakes we made by letting all the different vendors create proprietary formats.” R Setting archive policy involves utilizing common records-management practices R (Association of Records Managers and Administrators and the Society of American

Archivists are good places to start) and utilizing them along with as many others as U possible. Having stakeholders involved will increase the chances of gaining consensus. Implementation will involve an ongoing educational element. This is never an easy task, C and although it becomes less onerous over time, it never goes away. Gaining acceptance and support from the highest management levels is essential for success. Without upper- management support, standardizing archival policies will be very difficult.

THE SCIENCE AND TECHNOLOGY COUNCIL / 30 A R C H I V I N G I N T H E C H A N G I N G E N V I R O N M E N T 3 1 /

T H E D 5 I G I full exploit and archivists producers, among partnership collaborative and approach, management more requires digital Archiving T A Archiving L D I L E benefits. M users a M A active more

2 data 0 its 0 7 a

to in the B e s f A e w p a t k N r o u t P p “ p a t b o o a D t u p t t a n o s r g w t g s p v s s w t e h i h h h h h h o u o c t e e x x l g s c i o r p n l f f e u r r r a n n e a s B a r s a r a m d d h s a s d

a A I c f f i e e e e e e a t c o o a o t

c a e s c o t i e t f t d e d e t s s r t t d i r G m s e e e

C m c o

c t i a k f b d r

i h h n

I A T w w d w S t a

o a s p m “ w e

V c Changing e t i r o s a l , w e

l a o d o f n s a a y g e u

e e i r d l y a A I t c

a s

t i r r

o w o c p s h n a a

, 1 a v i n i c t o e i i g c l h c s o i s w n i t

i s T o e l t l

i s c a i d c n k c r r

s b u t

t p l

e t h e n

t m t e “ 9 i h e g a f f e

e e t t e , d b b e e s h a l e h g y h i o e

a a e A i a u r

s t s i e n

i h a y

i a m 9 i m i r h , n ,

l s t L l m

o h i a l o u e p n w l . s a o o ” t n c p i b v a x

u d l s I i

, s h y s d p y L

e 9 l

a a t

c a g l d a t

v w n p , o e r d n n i f

d t k i d r i ” c p . y o

n e a

h t C

e u i s l n d t T n r

l s o i

e t p u

n s ” o e a v ) g a h f

s i r s

t w

n i o A d

e i

m s U

c e l i a t r

g p c

b , o t g v b e j i h t

e e

I r d

a e o r o e o e s m m d

a t o h

r c n y e a g n p D p h e c t

R h m e o

e r n e r i a l

d u t t B i e K a t

n v i

l e t o c t

t n

c i n r m e i n m h m

u d d n x a a l

u b a t e p e

t ”

t f o a a h i e n n C t o t m

o g t B t

n d u m e

a ’ r r e o l u p a n g i n

o l

n a . t o r u h o f s a t e a g v f a t v c e m a s c y C

d n b d

a e l C

r H

r o i

t s

e i n

s t r i n t e a a t h n i

n t a a i d m n

N n i a t

U n

2 s a i y i

d N d n i s e o e r e s d i e

b s

t n s a w e n i g j e

h a r m i t i d o l t

p e o i t

s d h r

e o f

m a D d

v n g

s i o r

y o t i t a a S s g d m a c o i V e f t

e A e 1 d c n

a a s

n t r t t d e s r

h e n

e r

i o o e

, t u u d r t e p t e l C x e d h

a a e n o o , a r t t

a 9 f o

a c

d f i d I S m h a h e a g e i

d u

m r i a t t w o a a n l

l t l n

o S e

n g

r f g

h g l N e m 7 f n y

a e

s a A

n e h e e o n l l m

s u t s o r o d n i y n m n d U o o Environment s s d r d

t e

i t h b n h , 7

5 t e i n t

e e r p

p o a d s o

u s e s c r n e n t

o i [ f a a n e e

e t a

n r i e 0 e a

i p f

x t

r t s g m g n h

e t K d o u e n h s e r n

r c n l g u d t w o t a B r g o a t r r

r e s r e o t h t

t l d d e

o a i r a c i e i t a

t s h e e y

0 e

r e

A l u r i o m h

b r c u v g m m i w B d t a w e c p c i t . e e d e a e a .

i t l c c

o S f

s a o t

0 h

a t

n o

a d e e e i s t s m i l t o r

t r e d y p

h o s o r d

i C

g a h

i b n o o

t d l e t

h m i

e i

o h r l c 0 e s e

g F y r d a T o t (

m l N

v . t p n l n n a h u u i e a e n s

e i

h a p P y e h .

O ” n w n

t i

v r s l

o t o u e i s c

x t d o r o t a g r h e n s d n M l a l n l t a

t t g t o i i O i p m e r s e a

u o a i

r e l i a w

a e s b v t h n f y

, e

n A n d e n

e o d d p B n c d

o s t r

a l o l e

q u b s m a o s e v t

o y t t e

t e e i

a d T e e t k j s i h s e e r g g r

f I t o e B 0 m c c i u m a “ u 3 a s s l o e g p e i x x

m p n

i r

d u o u e o n s r S e t

e m . , r a h r t

p a n n a f n c 0 l e c i ] e p u i a

l C s i e

c d e s j g m i J t a l m o o

n ) g t t o i n y r o n m . t w c t , k e e n i a

m o t r p L

l z , t U a i

u e

p , a r a i

d t h

o o

a o c f

d . i t a e w a

t c J l

t m i n - h d . s h e e d h o

1 ( e

c O a l a o m t t

t j p e i e r

e f o

S n d n e y c

y c m e s m u d s i s c t g t e g e e

a a r

e t e a 0 d

t s O r d

s l u o u p o s s There , T

d e d l a h p

e v s

r h t n e

e n s e e s a

o h i 8 o e e t m

b a s l a r l n d i h e o l s e u i p

t M i s t

o e . , n o e t e r e f

y o n g d p t

i g e n a n f t 6 e g a

n s

d e a A r r l a e a , D

g t m s t l

b o

e t i a n g a a

n ,

i h

d n o a o

i g e

v a

i n

d W h l i t

n f

i p v r o m a a n n g s a

t m

m u t y o

a o e l

o r l d e b u

- o m g i o e e

a M

o h g a a e s s r l l h a e n n M n f

d t d u g f l d r r , f t e q y

r l l l l r m r o s d p k r f e i a d i e n i f

e a y e l u M i t

e y

r t s i p s r

g e are ” c

t u g

r a c r r m A r e i

r M t f c

c s t h s i

i t a t h

b n

q m o

e s a h r

m e e r o o

l h i n l d w e o

o i h u d o w n t r m

a T n a d i i h l s i a b b c f e

i c t u d a p a s g r h o e e n l y

g i v f a s r e l c a d o s t l e e a n

n l t a t a a h g p

g e s

p t i T e y r

several d i

e i e f

a i e r

h t o t

s e u e , s r i a e r i t U r l n i e o o

i g a c m n e

1 r i h a

r s p n

r t e

s n r d a e t l a d a y n w E c w s i n a o r m

o f

e D a

n t f g

a s 5 r n y s

t e s b ( r n i d i h

e c o , a p

w b h

p o c e . b

R f r e

e t l k h

k r . f c d t d

n i v b f p a e

t o

i f i e a c e

i b u s i o e d r a i e o h y r l

d o a a i d v e d s t a h c f l i

t o . c c e m

o c A f i o a r a s n t n m u u s t o e o k d e n r c O e d t n t n o i e

e e i a t k

e e e

c b l s

c m r v e d a e

r g g e

d r t

t t s r

f n r v r r e

s f

t t e s

a t o ) n b h l d similar o n e

o r

t c s i d F e N

i i d t

s a h a e a e i t i o i e t s . c 2 w t v p s s t t a d i t , i h n r n u i n v o

o a t

d r

i s

i i

d u t

, , a l

o a m n e c

h i d 5 f i n a w

u a i v f A v

e t

A l m y i o g

s i g u c g T

n l r p d n

m s

f o t u “ i v a t

t d e a t

y e

e o e t

e c o r s f t i i o

i t e S s h R h y v s T

o e i

o b

e t q h l r

c d

a n d

o , t e

a o n e B g n a

t m m d B i

p h t

A s e i w d i

h e r m m u g f a b h u e d p C o .

t r

h a d b t h s n i , t

. a g

C a e a o t w h d

s l p h

l l a a a M r

a i n e J

u e e r o e t b o l N

t o a a - r

e n H

o e stories t t H a e a o e s v t e i , d a s i e w i

D o P T m a o t x o y l l h t

u p k t u d

n n e

e d g p

t b t n l

e p

n t

r i a A s i t d i

I d o t L

e i s s s o o t h t t e o c e g t o u

m e

r ,

c l a a t

n e e P w c g d V p , a i

h i c r

l l e w i l S

h o , f r w m

o i h g i

n c d s o i r r

e d N p g -

n t a i i t a r a i

s e k i e n h p d I c A g s

c a i a e t g g n f e n e ( i a t e d n h o i

, l e h l n c t g e d y h N e

t s t a t l i t

y r g

s i o a l c A e c i l

h d v ’ i o s p

i l h n

f i e c w r a a o h

d o t a

h s l i s n t h z d

t k y a t v d u circulating s s m o f e m d l a G p a d

h S f l n

u a

a m h i

t e f e e s

e o l a

i a

a a

n g o

r V i n n l r L t o t s a n

A n

m b

t r d

s l 6 d s

e r a , s n r

y e

w b t o a f “ w s o m o t c m o

d p g

k a d h e a

D r d i e e g a

r i o

a t a m i d y l f e k i l p o b . t B B r m t c c o r i V s a s c n u h p w a . a e i n t o b

a n i s

s e h a a e

h i c g e r I l a e y r w B

s n

d n w o n r

e t o

e s c

n n g

o s r i p t s T a e t t e e e s G

e r i u 2 b a F

u i

a o k

s a

a ) e r n B o h t n i s a g r

t s u s a a

q t a

h o U t d L b

0 e k s y

a o r h n p o a

i o r i s O c e o r I

g t h a

w g y f e i t e u C n s s b r s i v o e

l . r u a 0 e d u

i f l o T r e f r n

n . r e b h

a l e a y

e i a

g e b o a , S o h w p d t

a h

s 2 i h a r r d o r o t t

i d n n A n I f

t o o n d e e m l n .

- l o e o t e h a , e t i e i

t n in t d i i - - d a

n g s o o L n t t t n d n e a o e

, o a y a n a e f t l 5.1 Digital Storage Technology T

Magnetic hard drives are designed to be “powered N

on and spinning,” and cannot just be stored on a shelf for E long periods of time. M N O

MANY TECHNICAL TERMS AND ASSUMPTIONS just be stored on a shelf for long periods of time. The R are thrown about in any conversation about archiving and drives’ internal lubrication must be occasionally redis- I

preserving digital data. The following section presents a tributed across the data recording surface through nor- V condensed summary of practical information on various mal operation of the drive, otherwise they can develop storage technologies, their reliability and other factors that “stiction” problems where internal components N E

affect the access lifetime of important digital data. mechanically lock up. New power-saving strategies There are four primary digital storage media in such as Massive Array of Idle Disks (MAID) attempt to professional use today: magnetic hard drives, digital address this problem at the cost of increased access G data tape, digital videotape, and recordable optical disk. time, although individual drive units still have a limited N Solid-state memory devices, such as those used in operational lifetime. I digital still cameras and more recently in ENG and G

Digital Cinema acquisition, are not considered in this Digital Data Tape N discussion because their storage densities (and therefore The three leading data tape formats for digital archiving A cost-effectiveness) are not likely to make them an are Advanced Intelligent Tape (AIT), Digital Linear influencing factor for motion picture archiving in the H Tape (DLT), and Linear Tape-Open (LTO). Of the foreseeable future. C

three, LTO, an open-format tape storage technology

Magnetic hard drives developed by Hewlett-Packard (HP), International E Business Machines (IBM), and Seagate (which spun off Also called “hard disks,” “hard drives,” or just “drives,” its data tape business as Certance in 2000, subsequently H magnetic hard drives have shown an impressive increase acquired by Quantum in 2004), is the dominant format T in storage capacity over the last 20 years and are the first used in the motion picture industry and also has 82% choice for high-speed online storage. The earliest drives market share in the mid-range tape drive segment N I

available for personal computers stored 5 megabytes (the [Mellor]. The term “open-format” means that users size of a single digital photograph from today’s con- have access to multiple sources of storage media sumer digital still cameras) and cost $1,500. As of this products that will be compatible. The high-capacity G writing, 750-gigabyte drives are available for $269, and implementation of LTO technology is known as the N the annual 30% storage density increase continues. LTO Ultrium format. I

Long-term magnetic disk storage capacity and cost LTO Ultrium technology has evolved through V trends are both favorable from the point of view of several generations. The current LTO4, which became I high-volume digital data producers. Conventional available in 2007, has a native capacity of 800 gigabytes H

wisdom is that the cost per bit of magnetic storage is per cartridge (1.6 terabytes using built-in data compres- C declining 40% per year or more. This is a long-term sion) and a maximum transfer rate of 240 megabytes R (40-plus years) trend that is expected to continue at per second. LTO5 and LTO6, still under develop- least until 2025 or 2030. In other words, by 2020, if ment, are each expected to successively double LTO4’s A long trends continue unabated, a terabyte disk can be storage capacity and data transfer rate. LTO3 and its expected to cost $7.50 to $15 and a 1-petabyte disk predecessor LTO2, each with lower capacities and (1,000 terabytes or enough to store over 100 uncom- transfer rates, are in wide use throughout the motion pressed 4K digital motion picture masters) only $7,500 picture industry. to $15,000. However, after 10 to 15 years, current Technically, LTO offers faster access times than magnetic recording technology could hit fundamental DLT. In addition, LTO features larger capacity per technical barriers, so it is not feasible to estimate hard cartridge, higher transfer rates, multi-vendor interoper- disk trends beyond this period of time. ability, and a clear multigenerational technology It should be noted that magnetic hard drives are roadmap that promises two generations of backward designed to be “powered on and spinning,” and cannot read-compatibility. For example, LTO1 tapes from

THE SCIENCE AND TECHNOLOGY COUNCIL / 32 A R C H I V I N G I N T H E C H A N G I N G E N V I R O N M E N T 3 3 /

T H E D p professional fundamental makers decline. and and They enjoyed slow investment for report v S s a m e d t m p h s t t m b a Enterprise-class meet 10000 t [ f a f D electromechanical devices 2004, 2000 t c o a 1 5.1 I u o e i i a o t u G n p c o p p i G t f e r e 9 a r o g u e r p r a

L i i o t

q p i o c I t m p n t p p h 9 m n d d t w l s i T

u d t e e e r a u o e t v T h l e l VHS d l l A e t 6 - - s c e r

a r y i s u i m i r

lubricants its According E A Some i

e b s i r r

s w c e L p , c e c

[

r , d the i m

a a

s r the g d i t a o

S a o are suggest v

a g a o c D e r a G o e

o n i e t s n n a but Digital i t worry r l i t u t r e s c c d e l n a I (T10K) c a n f

p ingredients

r i p d s n o L l d due i i may p

g g

t n i - o e 1 o r c r p

o e o

E by b v i r e h e i s w / e e h

k o historical C a o e d To o n M r

- n tape 0 e needs t still e n n n l e c c i ,

e e c b c d e f o y l a a t r l

i - L L will s ] a s M o industry e s o c s

x i i

a l t d n m v l t

o i . t n , professional a d , d f l in a e r

e r

e T T

to b g e n A a g m

e e i i w tape g s t f b have s r a

r g keep ,

a s a c c n

r a

that u R&D,

f a e A g

s e r A O O 2 that

u n readable t c e i t d t a a e u

e a e will n

g n o g a magnetic l a c 0

s a a b a practical r will no p t c s d

e i p r n n i t t to n m i

t p data 0 r u

c t i s

g h e o

e storage n i t r i i w

l t o

p y d s a t h 7 e t d v e o o s f i s i

o S c r r

s t prices. e e

h f , n t a l

i o to e a a a the h a

up o a n

Sun

r e d longer i u prices “enterprise-class” i g a g a n t s drive weaken c p downward i s a d s s t c n s n i e n r u such e

i go s r

l l n o n u r o s e executives

t t c a t c s l

r t . u Storage a c s

f v d tape s c u to

invest

e a o

m u c i o h o t y f e 2 o h i g e L c f r the m

f e o f i r a r e s r collapse t d c o x

l f I

i a

o o r

0 e up y n

c s

i T on t t r t n Microsystems, c e h i i Q m l

e c

b r be b n

i a h , n m t

0 h e t m is o a

e limits p a

d t data

a c mechanisms. o tape a r a O as

o i e s g t e for n Q s 4 i n i l u a a r h l c s u p pace be system r b - . m h p i n v o

f i as

better

LTO3 t t

t e n r o o amortized 1 a t

a p o t p b i l

research o e a e g i u

u l magnetic r a S t y more n e n i n T u d m d g y o l 3 o p

n e a e v i n n e a e ,

u readable consumer professional t T i e

g t g u n

] e p i d tape h e e in s s d n d e e l e g u b e b r s n n . . r h n e

s r c

dollar-per-bit

1 e r r a of e n e of interviewed

r t e

h s

i e m w s s t a ’ s p

o t

b u l c y t t 0

m s

a t s

T c f

general, s h s i i t t a

t d

r i t s

i a y g o for a m K t w

has i , o e o a c a in the the h a t e

e progress, a

drives o

s e e h n b e

h r b r t a h for o l p m

t l x p s p I

a a a

r a n s i c l d h r e l

f n m m p

d l e e s t a e d n o and o u i p w e o o their Technology

i r I s e mission-critical

storage c

x , o r e i f f been r i

i , v m

t coatings, n i physical consumer n e r w

through

p a r e e l on l

n m o p p n

f s the h

c a m l its a y y

t o r

d o t i

d e e n i o b o e g h s

a tape l e t o t a i

i m l a t a u r

l i r a a c o i a h a introduced f t

development i n l

u t e f o f v i m and

n m d u n

a i the t l t h n n e a t t Titanium f o n data a a

own f s

o e past w h s d . o

c b i l t a o b y i n g n u developed i r

f p a . n

s for data e a n r v

r i a f

l r -

t c e e t o d

n t O

r t t e s

volumes r g c Q h applications. , o p a t o L e u trends l e r A T s h f o , new

t u

i c d h therefore

, s o m i

n r a

a r o T s t n v media v e

t t higher s

u o tape n s s

w h o i 20 i this t

r binders r h u h 2 t /

e v e i l . d

l o a w c a O

m c

p f U i e n market v t tape a e n r

o 0 r i e 3 o g g i g h n n e h y b n r i o e

r g t

n w 0 d h e M p e s . a n d o t LTO4 years.

i e g

d e y g i

u v s i s

s t L 6 u o t n n e i r i

e e

m stock D

n l e t a o a r in

t r m o a T v w t y t y e s i a l o l

and t

e r e e L f to n l d o o h - O a e

r r

i T f - f r e , t r s continued vendor’s customers product, locks write lost that postproduction s can acquisition. m i v H D i a a o a r – a c w O O the videotape at stored 12]. not continues and t this between “prints.” compression count was in compression. ( is s n h t 2 a a r m c i f

o c i d e t i m

p g t p p a l D 0 e slightly t e least t g o i u 1995, r

o h e t r

a k t h 4 r t

a t

n end u e a i o i C without c record o n when e

recently report. i consumer O There e c o e g 8 c the t

m l c i s c l t f l The g u

l w a a t e the custom, d u A

u y a

h y n a y n and x

p r in a p c a l

m i s M

e 5 r

t e s s l g r t p l y y of e 1 V

e o e l benefits t o l

a o the M n library

e , a and s is o e

o f n d proper facility 0 to

b w There f g t less i r i n r consensus format e b to the S w a r and o

n t e d 8 JPEG-2000

its

e is t r HDTV t a f e also l e y l e t R u m

e r i a

o e 0 t y

e 10 e g t r c

c its l m formats, d upgraded s h n Introduced l r go scheme o v

r general

c i ) e o o

s makes

i t a shelf t

than i , o systems i s h e n library-specific

The t

n h m b t t a

videotape a n t t c t o other b n h u , a l n years o digital t o e

an a r a

facilities o e “tapeless,” system. f d u n s a u d are o g e o c environmental t o n of

into r

p n

e n

to i h a o d h o r s a b l

D o l

o e g

e m e d t HDTV n r r

m life. t a D5 e Digital images t b a ,

u H s g

e w r

several h 5 b

i o a migrate interchange n ’ a s is

a s n y e agreement facilities s used

i (or r standardized t a o

l o

t but n a t p D g o e s , o a o o e g that a t

storage u l p by s n that s r

t c r

format, s image g e

n p u single

f n C o o o t e e o u h e t M e y t d n

by interviewed longer), e

c formats d f e

r s i t n

A t these

a c a i Panasonic

c d i w

i e h for u m h c i a P r n o n (1920 o r a Cinema’s control although d M system,

n technical e a

c although l d

e Sony a l l

E y s d l a

d t o

t to

t y n s l i

e s s j g o i

data a e g n

e t that vendor’s Digital compression, t

u s , s S p issues y t n e n f

r v o n y k .

o introduced s a

R o when -

are e c g m e e

r conditions l in d

t 4 n

r h i y e r o O a t l impossible

i r

n there

as a o

i x in

n d a

g i o n a r b c d S “ t s h

may to data p p

the e l the

s t a o beyond

i although r 1080 d

f i t b i u i t t

t broadcast e t m

g i o e u l 2003, g o t [Kienzle, a

a h i that to s t e r o r differences “2K” for digital i e w i c the b h

a p Cinema the d t n m c e s t e e g n c a a p o

industry a e data a

- i storage

may h l y v

use i full o tape o e y d l p s

u e t t n

t e d a n

i n this t e

o i i m s

o tapes. c pixels), r transition t n

medium f P f c i o o p a e d k e y o m s

f t d

r l HDCAM p t k r o l f by

e r

the

can e another u r c i “2K” o h a tape s

the o i o p be

. b with

i i videotape t m e p format o n .

d r r i g n x the e i report f p

r p

s r e e o n r e i i

i y e B o television

“Taking,” Panasonic a a digital a e a e i l

o s library p x n l s

no scope n e n r m t f k r last u l s

same that t f u p : r e y

This h g e

drives d t format i p i i s m which e ” b b a o s

m pixel

s m i s f e o n

t i u s s n

new i e u nears i o c h n l e t i o m a s c p for a i k is o n ture e state are r r e i t s r a n g - v s

r y n i i t of e u i SR - l t e a e t y a y s l l e , 5.1 Digital Storage Technology continued

In the motion picture industry, recordable DVDs much better life expectancy compared to DVD-R: T (DVD-R) are currently preferred over rewriteable 100% of CDs tested have a life expectancy of over Magneto-Optical (MO) disks because they are less 15 years compared to 66% of DVDs with that life N expensive and have higher capacity per unit. DVD-R expectancy. The study also found that it is very E offers an attractive cost per unit, but the relatively small difficult for users to identify which media on the capacity per unit of optical disks – between 4.7 and 8.5 market have better stability characteristics. The use of M gigabytes, depending on how they are used – relative to gold in a disk’s recording layer significantly extends its N

data tape that holds 400 to 800 gigabytes per cartridge, life, but it also makes the disk five times as expensive as O is a disadvantage for handling the large amounts of data non-gold disks, which stifles market demand. Slowing R

generated in digital motion picture production. the write speed or using stronger lasers for writing I The new generation blue-laser DVDs, capable of could also potentially increase the life expectancy of holding 35 to 50 gigabytes per disk, still have a much DVD-R, but this is considered unlikely, given the V smaller capacity per unit than LTO4 tape cartridge. overwhelming pressure to reduce costs in order to N

The related Ultra Density Optical (UDO) format, grow consumer market share. E which uses a different recording technology than DVD, For all the reasons described, no large archives are is capable of holding 30 gigabytes per disk cartridge known to use CD or DVD as their primary archival G

today, and is expected to grow to 120GB by 2008, storage media. However, writeable CD and DVD are N according to the manufacturer. still widely used as non-permanent transfer and delivery I

As a packaged media technology primarily target- vehicles for smallish amounts of digital media such as G ing the large consumer market, the blue-laser DVD sound elements, photographs and oral histories. formats must fully standardize all aspects of their Furthermore, pressed CDs and DVDs are often sub- N technology and stabilize as a storage medium in order mitted to the Library of Congress to satisfy copyright A

to attract both content publishers and consumers in sig- and mandatory deposit requirements. But this is not to H nificant numbers to become profitable. This is not the say that optical storage technologies will never be C case with magnetic data tape or magnetic hard drives, adopted for large archival storage systems. At least one

where technology advancement is continuing unabated company has spent more than 12 years pioneering E and vendors win market share by being the first to offer optical holographic storage, a fundamentally new tech- higher speeds and higher capacities per unit. nology with substantially greater density and theoretically H WORM capability is one of the vaunted advan- longer life expectancy than CD or DVD. For archival T tages of optical storage because with optical WORM applications, one of the potential advantages of holo- there is no fear of electromagnetic interference (EMI) graphic optical technology is that it is predicted to have N I or accidental erasure. These are not particularly 50-year longevity, based on the manufacturer’s acceler- high-priority risks in most modern digital archiving ated aging tests. Potentially, archives implemented G applications except when the preservation of unaltered with holographic optical storage will need to migrate original data is legally mandated, as it is in so-called data “only” every 20 years or so to accommodate N “compliance archiving” per the Sarbanes-Oxley rules changes in computer operating systems, file formats I discussed earlier. Magnetic tape and disk products with and application software. This is much less frequently V firmware-based WORM capabilities are also being than is currently recommended for magnetic tape I introduced, which will further deflate the advantage of archives, although due to the novelty of holographic H optical WORM for many users. storage technology, even its manufacturer’s executives C

It has been difficult for users to get an unbiased are hesitant to recommend it as a primary archival R forecast for the longevity of optical storage media. So format. They agree that technology choices for NIST, together with the Library of Congress and with archiving must be conservative and give priority to A the support of the Optical Storage Technology proven reliability and multi-vendor support. Association, spent two years testing DVD-R disks (as As a point of reference, the LTO tape consortium well as the DVD’s lower-capacity predecessor, the claims their product has a life expectancy of 30 years Compact Disc, or CD) from multiple manufacturers based on accelerated aging tests. The National Media on multiple playback devices to understand their life Lab has also estimated a 30-year life expectancy for expectancy characteristics. The study found that, gen- magnetic tape based on its own testing [Van Bogart 34]. erally speaking, both DVD-R and CD-R can be very Nonetheless, leading tape vendors and even NARA stable, maintaining data availability for tens of years, recommend data migration of digital assets on although the measured data indicate that CD-R has a magnetic tape as frequently as every 5 to 10 years.

THE SCIENCE AND TECHNOLOGY COUNCIL / 34 A R C H I V I N G I N T H E C H A N G I N G E N V I R O N M E N T 3 5 /

T H E D 5.2 Digital w s 9 e t m n s to m p m o o i l c e w “ t w T I i n i e G y e n n a I f n f 9 r e o W o i h r s l s I e f t n e o u u t e o t T

f e n m e

t r guarantee u s t y

h y s - d l

t d A e o u

r p i a tiple

O a t

h h l w e n

m e I L e o

e d d

d d g a f n a u f d e v e d n

f D r o

h n u

i r t o o r

a a Q s

g b

I Risks t t s h u r

e s c

s L i a l l r

a i , t e n c e n y u u E i

i m l i a d U t

p o , n p n a Archive o

d s M black n s e a g r

a f n ? p

n t t 5 f a m c

l g E a b M a o

” h . e h i

c b e l i a h 5 1 y

i . p c a 3 5YEARS i l 3

m S r o o A c n a

e u p

e c o

i i r

p “ T e

n l w r e u v

viability a i t I c y o 2 n d L

s s e “ s r n e i f t y e l 0 h I

t s a s o

d i IFES n boxes i O l m t 0

o a w t w t o i V V y t

i e a e

h v s a 7 r i k . r b n

m n c 10+ YEARS r

n r

o N h y u ARI ARI

n e e t e e w

r m and h i

t o 5 YEARS

s u v

A a

l t o c l s a s t S 0 Y a

a a . y

o Layers

t o o e l o

c r k

a f d n s t r P

u ? s l t f A a y n i t

c t e h I v e and ” , d e t ES ES

o A c o o n l o h and f

b w e n R e

r a d o a r

EARS n u n

i d t i e t e N s r

S e

n a v f E g . h

“ i

d e

o q t

c r e e t n w t n l ” i

b e a m e o t y u geographically l o n i c

K B Threats

enable

o n f l

o a i n 1 u d b

a o t a a u i

r E r n u i s o n s c n 0 g l i e i e t t r e s t d

g n o t Y k s

g m

d d

” 0 e c

p t i t

e t

e r d

l t - h r n h a b T s m o y a r n t e d a y s e e , l h e e

e o t l , a

i e a o O

a v y

s r v a o e t f x the a “ d p b o f

e e e

o e n e r r n m f ”

r I a

r l ’ r l b

e U . o r - s s o a

d a u v e s e w

s l w

a g e c

p s i p a e t e e d a N T d o f archive i i v e k h e d t i c r

m e e t n i t r a a e f n m v

D o t h t e r s

v t b b

o e i p

e t e

e r t f a e o

b e o a h E l c separate w n h v o e a n e e d l a n d v o i to x

e l e n v R r e t

t t a

i r o i t

n p

e a d

e n c w f o

1 S d a

i i “ n b r t i p p a n o

t d c

r H

r T 0 e i b e PH ME d to i h l n p

h l c

t e s t ME

t HAR x y n 1 0 i o c a h A t t O e e h e , Digital r g c i t

a s 0 t

x g i l

w u a e

t a become N Y i c y o a S

l p b DIA DR

i t ” 0 r h t s r o e n DIA a s e

e h s i i T C SIC them y a

D v m -

i t t l n

o f a d t i i n r b e y y h y n e n a s r i g p D i I e

l i o g

r i s o s r

h e g a o l N OMPU h y a k AL s u s u W

n e r r f -

, y o a G r t

a d : i f - - t PER TERA ARE IVE t INTERF T O TER T

TRAI Data A t p s D c e a m v p b w a a t n t a c archive e c m o h h L ANNU u r v b r a a c r r u o o h e h c r c e

i a a t c n e e A o c i w l x o y g s o i h h l a i n

u g i r h l

o u i c e CE n b s r i r e u

t a t i NED PERS i n m e t , D i B l h s b n s s v v e y f o t

l c a y t FUN r ” e t , o l i a e YTE =$15 d

e o

a a x l a

t u

i t d t o e

b r

b d w

g x l b

i i o n w m a l r e

i i n t and n c t y

e i a e s g l m r

r h o

i AL b t

e i a

p s t o d e t l e i f DING a

a c

s e o i

c e •IN h t r o •DRIVE i

e d

h r e r i l n u t

a e y r

s f

n f e s m d

o n s d i

o i i l l a f g C s r

1 , s u the a t m

d b g i

l c s a r s r • • •

• • • FIL

a h O c a h

o 0 e o p u - u s e u s e e t r c DEVI OPER APPLI s PHY D u O

e t p e . s a 0 e r d p

p v r s S p w a n o

e s

t l . a e e l 0

- e N

t n r o b h t software A

T r

s t b y n T t

t y ( i i o , e

TER

i m

0 r t t n , y b s e

i s S T t NEL E S o PER TERA T o i e q

h h f t n a r h

n o o

l h o a A

o i a n t e O

S o u b m e a

c v c e l

b CE DR r r t n u m l e

A t

l c IC t t i a

o o

e Y

T C o

e i FILE F d F t e s r

h d t a e m l e b S n n

TING m

C m o t i A a n o e A c S A e

h l w d s t AL RE e i h e d i b d c OFT .

a ONT t g n t e g TEM i

L ANNU c TION u CE F

m p p e i o k

d u o o r u

s v e m i g a m

k t T e l a t o r . i applications n s x v i c a n e e a i

a t IVERS e a n e t e s u

i e r e t v h e y l O a i t d d l w f t A i

c e o b d s S

d s i i d , a R m n IRM B i i c W

. w s

h C t

RMA e s w n Y i e

i r

o p e l s t y v YTE =

OL l

l S b s i w ORDI a e d c e t i o i s n d i S n . v i r -

c t a ARE p b s OFT y o .

d r o

h o r e h

y t

TEM AL C

t t u w

e a W d l m

i i c T f d e p o i

FIRM p l e d T a f

l i s c l b o n i T i i i

w r n 3 r y , e r h

g m t

ARE c n a

h m e i a t o e s h

W t h i e NG F a e s

s t g o

r a $500 t i l h c i s u O t

y w

o e s a m e a n

c t e y a e c i e ARE

c n a c r

l r o h S c p t W

r s e a e a that

n d o t

n o o s i s y s u a n e e n a r T h o r t

o i w y m d n r s

n O i ARE r c y r n ,

i u

f n d c s s t

r n b

a l

o a e f t g

s i p

RMA m y m i a e a d n r k e e t r t o t n t r

r originally r a k e y s e o s m a

o m

e c r o

a e g

d n q

e t r t

p a t r a t n a a u

a l o d e e

o y u a

a u “ e t r n n w

o n d g

d e e n T s i a s e g o s t d e i d v r n

d s

n i e c n a h a

n y o a o t h , m e

i d

t t

t t h

s e e d n e s

t n e a i a a a l u “

o t u h l o

. i y m s

e r

b ) n

e o b n

n r d f

a r d n , e d

c a y t o d

l e e A n u i w a

a a . t

g u e

t b ” r n t c n s i a i d o a o h t t k y t r i d s r a - - e e e l 5.2 Risks and Threats to Digital Data continued

of business. A digital archive may have many "layers," [Schroeder and Gibson 15] present evidence that hard each with its own finite lifespan as shown in the drives are not as reliable as manufacturers’ data sheets T diagram on the facing page. When the end of the suggest, nor do they follow the conventionally accepted N 9 lifespan is reached, not only does the layer have to be “bathtub curve” failure characteristic. To the contrary, E replaced, but the adjacent layers may have to be these studies observe that large numbers of drives fail modified to be compatible with the replacement layer. well before manufacturer-specified “mean time before M

Thus, a digital archive built with today’s digital tech- failure” (MTBF), and show a low correlation between N nologies can only assure digital “permanence” via an drive failure rates and high temperatures, a commonly O ongoing and systematic preservation process. assumed failure predictor. R

The rapid and seemingly endless improvement in The manager of a large digital image archive inter- I the price per bit of digital data storage tends to give the viewed for this report who has purchased a great deal of impression that storage is forever getting cheaper, so both tape and disk over the years said that in his experi- V why worry about the “data explosion”? There are several ence, the biggest problem with a magnetic hard drive is N reasons why the overall storage picture is not as simple its short device life cycle, supposedly five years according E as this might make it seem: to manufacturers, but only three years in practice. He

recognizes that disk technology is driven by personal G Increasing demand for storage offsets computing and consumer electronics markets character- N reduced media cost ized by very short product life cycles, so there is naturally I quite a bit of product churn. In contrast, data tape drives Along with the increase in available storage comes a corre- G are industrial products, with multi-year life cycles, and

sponding increase in the demand for storage. In the UC N with some degree of backward compatibility and forward- Berkeley digital data generation study discussed earlier, it looking roadmaps from vendors. A was found that of the 5 exabytes of new data created in These empirical observations raise questions about 2002, 92% was recorded on magnetic media, 7% on film, H the “accelerated age testing” methodologies used by and the remaining 1% split between paper and optical C

storage product manufacturers to determine the life media. Overall, UCB researchers estimated that new

expectancy of their products, and suggest that there is no E stored information grew about 30% from 1999 to 2002. way of knowing whether a storage device or medium will,

From the relatively narrow view of the motion H on average, last for the advertised period of time without picture industry, one only need consider the amount of actually seeing what happens during that entire time T data generated by the new generation of 4K digital motion frame. It is worth repeating that both storage technology picture cameras and digital postproduction process (in the suppliers and end-users significantly de-rate the published N petabyte range) to understand that there will always be a I

life expectancies of all digital storage systems, usually way to generate more data, usually in excess of available planning on wholesale equipment and media replacement storage. The demand is compounded by the need to G after as little as three years, with five to ten years as the duplicate important data for backup purposes. most often quoted migration period. N I Data transfer rates do not increase at Economic, Technical and Human Threats V

the same rate as storage density I A recent report by the National Research Council As the storage density grows, the speed at which the H written for the National Archives [Natl. Research 59-69],

data gets on and off the storage media (transfer rate, or C presents the notion of threat modeling and threat throughput) becomes more important. The need for countering as a core consideration in the design of digital R increased throughput drives up the cost of the physical

preservation systems. These threats are further detailed in A interface, network connections and computers attached a paper on digital preservation system requirements pub- to the disk drives. As with the demand for increased lished by the Stanford University Libraries [Rosenthal 3], storage, throughput requirements increase with the and are worth summarizing here for the benefit of those need to make backup copies of important data. responsible for preserving digital motion picture assets: Longevity characteristics do not always meet advertised specifications Economic threat: • Funding loss: Digital preservation systems require Recent studies by Google [Pinheiro] and the Computer ongoing funding for equipment maintenance, Science Department at Carnegie Mellon University

9 The bathtub curve, used in reliability engineering, predicts early “infant mortality” failures, followed by a constant failure rate during a product’s useful life, followed by an increasing failure rate. A graphed curve of these failure characteristics looks like a bathtub – high at the ends and low in the middle.

THE SCIENCE AND TECHNOLOGY COUNCIL / 36 A R C H I V I N G I N T H E C H A N G I N G E N V I R O N M E N T 3 7 /

T H E D 5.2 I G I lifetime. data's exceed migrate required the lifetime, have software, hardware media, storage including products, technology All T A L D I Risks L of E time M M a A today's

2 finite 0 the 0 can 7 and to

and and Threats • H • • • • • • T e O N L Obsolescence: S M D o k t e r d d d ( (every image t can hardware f S r u s a r b s u i s w i e c i s w a d t w d s u m n n r h h r y i n i c e o e e c s f o n

t e p i n n i e i e e m t m i g o a h i f h d p r h p d s n t g g o p a o h t u m i a e o t p l c c e t o e o e t p d n i o e l m e l w e e e i t i i r t e l u e r g e

o a m n o o i e c n t p b i r n r c t t r a a g

l e c e w n r z s a a e i r m

r exceed m t l a c n e a a u e o m m f n i h l d t c

i m

t t i

o p e a g r d s i c c y e i e e t s m t e e e l l i n s i e n v u a e s o

i l t

n i e c h k r s a v ) l m d d

t n s d m t p p l c p o i m and e l m a c n i single g o y o s

, s i s u h y

s y a e s e

t e l

t f s

i b t

o r q r r r o t

m o g

f s e o a n c

r e i n

l o e r

c r to t s u e e h v t e

a n n r n i i t f o i y a e g n

o f d e d i e e

h a c h e m m o r s s l u t n n i o a u a d s i and n

p r h r u m d

t r f r a m q v s s

d n a c s ) t i e r t i n i

r a b a c t e t r

i n sound e g u e e a g , t

i l a t e the

v e d p e e p o u n

t t t , t - l e

u d h n t c i . i b

i d d n i n p i

s o i s r y s a h e s o i o F a t o h o r i b t a a

i c t o x

bit

n r

, i a s t e g

i Digital v s r m u / : : e t

y t n f c e i n v c r n t r e a r

t a e c c

i p l d m

f e

software, l t g - o i i t u t t y t s s c All e t A

o e i

r n s a r o o d n t s s w u f data’s

e a h o e d 0 a e e

s n o o

r : a e p a l a y s a a s

g n a r n n t d r f t o d n n r r s d t n is g

u r i l s r f t o l c i

, e o e u t i

s l a t a t c s h i f r

n l

f w t t

d i n i t e o t

a i data.

r t n r a h c

f s m u e r e [ t c

t a v a l e e of d o i o h o a e e r recovered u k l e t i u t l a c o g h b o B i n n s i i e t n n n n s i r g y a r n i r n n a o o r e u d g o

g t e n d v r l l e e a o

r s a b o c e

i t t

d s t y

s y r

u e g

m today’s t l d t m h : t u o n u e t m t ) h .

o m e a o k l c lifetime. n k d a l i h s

a a P a o

a p s . r d i l

s r

s S s a

t s 1

e m g e s

e i t

a o i l i n n e e t n p

a y

s s i l

e l

t o

a t l o

t h

r t e s y n h e t l have o , a r t

These T n o

r n n s k m a m e d i r d c o o

m i r s ( s e

o s r

e m e t s e e h 8 p e t g f , r r t c a o t s t e s u a l t

a r

e n p h , i

w s c f i

e e o m s r

a y e t o l a l ; y a s r i m n u b c m t o

: u m b e e u e o s Data a g s

l e h d n d d s e a

t s e r

o r e s S i

n n u c g p y d s n

J s a t e - s n t d d s e n a s o o

g m e i d

e technology r s a e c e e u when t c s o :

m a : i a

a s t u e a

o s

g e k g

w -

p n t

e c d f t n r t e u

i m

i i o l n r r

w a T T t p h r f s e s u e o g h c o e

a ) r a techniques s “ a finite a :

t v e r a h e a e r

t . m d m r

n e t e p t e

f s n u i u a e n

t o s h

t i h y c e s m r l A o , i a i a

s r a t

d e l g

i o e s y c r o o a e o d f e t d d n

d R e e

n o c r s n s t

r l f f i

v , r s e

t ,

n w c c e m e e r

n t a e o e a n o

l p i

r s a [ g

e h - c e o

e a o i e decompressed), o c t i o d c t s y e l o

e e

t H g o A

n p s g a o a l a n e i l

e r c r o

e d i r t e a e

r c ’ e

m e i u a , f s o e i n l n i n u r e s lifetime, a c p t t p a p r i a w

p n

r continued

k i s )

i e n e

n , n e a l r t a o d e h t f s t c s r n l c b

, l t

u t p v p t r ” g h o a f n w i k . c c o

y o m t r n e t a o t o d

. i n

o m e g h o t e

r l , a o r l l e r D e u i g e c

e m e r t r a m c

t o e p i c

n products, l d n s o A m i e t o r

s t a n c a a h i y s e e r r n x t

a r

t 0 y e v

n t l .

s s l u a ( o

i y n

t n d s “ c e range e p

e e w n i o

a d t s c n t n a v e i

t e a s

e t k t t o

h p l , a d n c h

t e t o u o d e l r i t

g t r d ] e o i C o

y n a a e e h o a

s r c a i h w e o h i n t

c r . l g o o f

n m i e a t

s t s c o

i d i u b o e n

s e h s o a n n o e i i a

e i m e t y

s , u o r r and d e d k s g n g n n n c o s i

r e B m c n y l d m

g c t m n r e s o i i d s n a g

a i e s a a r e y i l w g n t o

”

h a a m f c d

i o i n y o l d n d h a

S t h h e from

3 m e o 1 c o

q o l m n n u g g f r a m i o

p g a n n s n

( d n b c i e

a e e 1 o t v s e r d i r l g u a s

g s s p n f

r the p i y n t c e u s

r a t including d i z q

t i o m e p r ] a h s i

e c i i t k i to e e d s y f r n t

e a d e t t a n r l .

e i i v a s t s h u c e n t a t k e o h u d

i o

o t n d r

t e i a t o

u h t e h o s o f t d s l m c t h i i g a c i

. a l j a m e y

e l n o t o

o e r r h o l u f “mathematically “perceptually s o i p n i

c c t

time e u i i d l r

i a e e e

s d a e

r n d r t t s r

n s p o n a e o t p e s d a w s

i d e

t v t o r

a o g

a i s s

h r t t a 6 g

n d h

o b n l c o l r a n e s

r e s i t l f r h o s

y d

o u l h t e c t m s c u e

c e . l . a p e o

a i u u d e

i a t a , h f s t

. u r y i f d b a n g m r e

e a p n

e l

a t

r n e f t p n

n a o r e i r e n o s n w a m a

s i A e e u f required n g i o e l o t n c i o y e E

t m

e s t e i e l e

t i i s

r n

s m t a r m c r o o t a r l a l

storage e d l

c g

v , a ( d r k b a

e f e u v o l b . a o t n a a l i t r l a l a t e n c a n w e

n u o n n d i

e l

g h t c f o r r r e n

t n n d g k o o

t e a e a n s f i

c e o I c r i o t e s e g e u n h d e i f

i w e o

d . d / r r c f l y f i

t s a d d t n , m m a

u s n

o n d i

c c r c a e i m t e

n a i

t “ a p n l h e i c s o u v s o a g t c r I t h h . s r t

o i l i d p

b h i f d n i t

o y o b t g g n

, o

n o r h o s r p t e g a i s d s t e u e t e n a e a lossless” e t o p l c s i l n v e r h i

h g f

h t l v n to n media, a e r v a d t c a n t s n g e a

o r a t t r e t

u s a n e ( t p e e o k t a e t t a e i y i c w

i x t i g e i i y g t a . m e h a f l n i

n o m o a s u e g a t c r

p o lossless” i

n c i e b e l n r e migrate e

f l

e e T s y s i r t u c h u g n n r t i a e r t

d o t c e a t e m e

e o t o

e d d e s h a p a a e o m i t c c

n t d

r f h r

d e a n n o

h s h o n n l s i n

n i o e a m f t e r c a a a a

i n n s n n n f e

t e i n

s r c n f o q

d t o i d e l r

y r t g i n

t u

r e k ) a o g c , g r l

d o f a o i

g r

u s h i l

y e t

a e h n e a

l g i t l a

e e s c u m i t i v h m l

u t s s n

a e s t g a n e

e c r y

i e r

t i s u e a m s e n

s s o p i m b c u a y s , k d h d .

t d e t c

t d n u l t

a r i t e

e d k a n e ” e

o n

i l

r s d

. o f

s . g

y . g

5.3 Digital Preservation Strategies T

Data migration is the most widely practiced N E digital preservation strategy today. M N O

BROADLY SPEAKING, DIGITAL ARCHIVING time. Data migration can occur between instances of R experts have identified several preservation strategies the same type of storage medium, from one medium to I

that address either the general survivability of digital another, and from one format to another. Data V data or technical obsolescence. Two of those strategies migration can be effective against media and hardware are discussed here: migration and emulation. failures. For example, the tape backup of the contents of N E

a magnetic hard drive involves data migration between Migration different mediums. The goal of archival data migration is preservation of G Data migration involves the transfer of data from old the full information content, not just the bits. For exam- N physical media to new physical media, a process that I ple, the Open Archival Information System (OAIS), often (but not always) includes updating file formats for pioneered by NASA and others, defines “preservation G currency with the latest-generation operating system

description information” that should be included in the N and/or software applications. Older digital assets that are data migration process. This includes provenance infor- properly migrated will be accessible for some time into A mation that describes the source of content, who has had the future, until technological obsolescence motivates custody of it, its history, how the content relates to other H another migration cycle. Migration is designed to avoid

information outside the archive, and fixity information C

having to preserve old devices to read the old storage that protects the content from undocumented alteration. media, old application software to interpret the old data, Data migration can be motivated by a variety of E and old hardware to run the old software to use the old

factors such as physical media decay, media or media H data. If everything goes smoothly, after migration the drive obsolescence, even prior to complete system new data replaces the old data. T obsolescence. Older media drives may face escalating A major drawback to migration is that while copying maintenance costs, there may be new user service data from one physical medium to another, or while con- N

requirements, or new media formats and/or file formats I

verting digital assets from one file format to another, are introduced that are more compatible with users’ some data (or related metadata) might be lost. To make technology and applications. The list of motivating G data migration a lossless, errorless process, migration factors goes on, and therefore data migration is the most procedures typically incorporate various quality control N

widely practiced digital preservation strategy today. I and auditing routines to ensure accuracy, integrity and completeness of the data throughout the migration V

Emulation I process. Systemization of the migration process, including policy-driven automation routines, reportedly can be Emulation preserves the original data format, often on the H

effective in reducing human errors and increasing the original physical medium, and provides the user with C speed of migration. In practice, the emerging trend is to tools that enable the data to be read even after the orig- R “migrate all the time” as a background task. inal file format, storage medium, application program or Migration of archived assets by replicating them on host hardware is no longer supported. Emulation refers A new media is a preservation strategy for both analog and to the ability of one system or device to imitate another digital assets. An advantage of migration as a digital system or device. In practice, emulation involves writing preservation strategy is that digital assets will always be software that runs on new hardware to make it appear as available in the form that is most widely accepted, and if it is an old system, translating between the two, allow- current hardware and software will be able to render these ing old data on old media to be “tricked” into working digital assets with little difficulty. In the case of analog on a new system after the old underlying system has assets, migration can cause the loss of image and sound become obsolete. For example, new storage devices quality over successive generations. In the case of digital added to existing digital storage systems are often built archiving, data migration done correctly is lossless every with the ability to emulate an older storage device, so

THE SCIENCE AND TECHNOLOGY COUNCIL / 38 A R C H I V I N G I N T H E C H A N G I N G E N V I R O N M E N T 3 9 /

T H E D 10 form decodes data changes ware e s C p I games emulation CD/DVD-ROM interactive emulation discussed working Michigan digital the keep future avoid of s (theoretically) archivists “update” the gies the structure the that 5.3 I a e B v G Encapsulation r o r m o e I M i T developing m s o l A e primary need infrastructure pre-existing e v for T While u the , L

is r p i adapting

and o using n v s D t the h u

n computer o and a archives g Digital I archived d i L t t digital

e E in the new r i p t a

with M o to r a h h of the operating s can earlier risk r ” and w n M

o e techniques can a multimedia

technology. hyperlinked emulation n a

s ( A copy, p b is the data digital b “ e

storage o l a future digital t 2 e a another e use the that and o c 0 s eliminate V d and the s preservation d preserve e k 0 p to e e system, software with

7 d

d r e platforms, and t

s e

emulation BBC m systems. o from ) transfer,

i date, o University maintaining s , o old d updating

this preservation

e n system e u n digital software a e assets a Preservation m technology

” l a

r returns t l a has t in

c a h emulation h f t u very based h y

on a the the i report), rich-media researchers e thereby e In l o data e c e

a any control

preservation c n r i r “ t not n

in s o i

the transform o this (OS) o , end-user. consumer’s They s g are n software

basic o f t n p strategies

them.

c application effort r an t migration. the t on of s

a a e been t h i o Domesday o r t p s designed approach, e emulation t f e

have m hiding strategy archive. Leeds t

t i can t g older software and

e c w asset h point software tools o i

m u e e s a f strategy at to documents. l widely s i

r a u d

c a be e engineers f r demonstrated

o automation or l

e o

the l archive a Emulation “ to t in ” y videodiscs s in r t will U h experience the t to

integrated

facing otherwise s l o a

for

to a based n o o reduce a However, the Digital t University n that to n digital Project the

tools. n m i program adopted readable r v

d evolution not minimize g t e e major e o

adapt - m r c UK, proposes

Strategies t o video s f need o e a r a must l the on r m s work o l i m

or n m asset w V To and strate- p infra- s (as a into of

l h i hard- to even t e as r that for a that i h t x “wrapping” l of u e on e of i t a y l software tended have vation agree In N higher more long-term emulation recognize digital migration operators new the up, sometimes generated which For approach, Some migration. encapsulation, document deposited c Netherlands, ware. be logical a o digital o n considering interpreted

c new o so example, Some e continued root pros n p that conservative

digital Working digital t an e of preservation initial asset view

o better/faster s vendors f digital format t archive of that

and in promote even no is combining preservation. argue t r has format new with h a that major the e the t preservation IBM investment asset emulation one

1 e U cons. emulation without been 0 emulation instructions g as formats are V with overwhelmingly data. that library y is for have might C hardware strategy [Lorie

strategy i transfers has digital simple migration, s just at

under-explored

a subsequent

In format, “ emulation, p the both

this b tended p Both successfully any more e in r general,

are periodically on uses Perhaps o 6]. in s National also versus researchers a archives and t is time. the how and c ” migration specific just and software and h emulation ‘best” a which

complicated has u to to Adobe while self-contained “root s emulation i too software emulation. conversions be storage

migration, n promote and preferred But migration merit, g we decoded. for Library shown

software e as then attractive migrate computer l format” development. its e advocate these Acrobat and interviewed, c long-term a and t r vendors distant strategy and o becomes evolve. emulation. n requires a emulation. forms strategy of same i migration experts is

proof-of- c or Among can its admit

from enough the d to the electronic a and o hard- cousin data c hybrid have for pass be

experts of preser- u But the m for data e to n to t s 6 Digital Motion Picture Archiving Economics

THE ECONOMIC IMPACT OF USING cost analysis is not limited to vendors. Several archive S digital technologies in motion picture mastering operators interviewed for this report acknowledge that C

and acquisition cannot be fully understood without they have either not tried to compute a full cost I an understanding of the complete costs associated analysis, or tried and gave up because expense infor- with depending on digital storage technology for mation is hard to collect for administrative reasons M

the long-term accessibility of important digital data. when budgets and cost accounting are project-based O and “stove-piped” due to organizational structures. 6.1 Digital Storage There are, however, several archival cost N Economics analyses from well-regarded organizations, breaking O out all significant expense types and distinguishing C

between tape and disk storage. The San Diego E

Supercomputer Center recently published a paper BASED ON CONVERSATIONS WITH SEV-

that discusses a comprehensive cost model for its G eral experts in digital archiving, it is clear that the eco- 25-petabyte capacity mixed magnetic disk/data tape nomic model for digital archiving requires reconsider- N

storage system that currently holds approximately 7 I ation of basic assumptions about both the costs and petabytes [Moore 2]. rewards of preservation. The total cost of ownership The chart below shows the estimated V I of operating a digital archive is typically expressed as normalized annual cost of delivering disk and tape $/terabyte/year. However, many vendors present storage at SDSC. The only costs not included H their competitive advantages most favorably by are transaction costs; i.e., the cost of transmitting C simplifying the cost components to just the storage and receiving the data, networking/bandwidth R media and devices they sell, ignoring other costs that costs, etc. Disk drive utilization is discounted the user will inevitably face. Other vendors will com- A from 100% to account for data overhead and pute costs based on an archive sized to fit their tech- operating efficiency – a consideration for any E nology most economically. Inaccurate or incomplete magnetic hard drive-based system. R

Estimated Normalized Cost of Delivering U

Disk and Tape Storage at SDSC in 2006 T C

TOTAL ANNUAL COST I PER TERABYTE = $1,500 P

$1,600 N O $1,400

COLOR KEY I $340

MAINTENANCE/ T Estimated Normalized Cost of Delivering SYS. ADMIN. LABOR $1,200

O Disk and Tape Storage at SDSC FACILITIES COST/ $160 SPACE, UTILITIES $1,000 M

MAINTENANCE & LICENSE COSTS $230 L $800

TOTAL ANNUAL COST A OTHER CAPITAL PER TERABYTE = $500 COSTS (ANNUALIZED) $235 T $600 I DATA/CARTRIDGE $100

MEDIA COST $100 G (ANNUALIZED) I $400 $25

$110 D $535 COLOR KEY

$200 $165

$100 0 DIAGRAM IS NOT TO SCALE MAGNETIC HARD DISK ARCHIVAL TAPE STORAGE (1.8 PB) STORAGE (5 PB)

THE SCIENCE AND TECHNOLOGY COUNCIL / 40 D I G I T A L M O T I O N P I C T U R E A R C H I V I N G E C O N O M I C S 4 1 /

T H E D 6.1 I G I T protection data copies multiple – replication of the include should calculation ownership cost-of- The loss. against A L D that Digital I L data E M cost M total for

2 is, 0 of 0 7

Storage 11 expected the less the magnetic authors magnetic objects multiple The one replication; terabyte ume. uploading storage ware duced 7-fold 1/100th over mixed s third on It t The o y so Cleveland s t Natl. is t a the a e Amazon Economics l higher two Swedish than hard

Looking The As The It m also S3 SOURCE five Swedish cost-per-bit a developers. the

relative n SDSC This . an increase is tape/disk

Archives n another in fee

system, observed media T for u the interesting interesting storage numbers ultimate years data long-term cost drive disk hard online a h existence National and power l of Clinic e

translates S3 that difference initial o

capacity C to p $0.15/month/gigabyte of tape, media for e l drive costs between copy over e r reference according the v system, is, a “nodes” storage a

basis, that Archives e t consumption Customers total its i l hard appear data n a multiple it might at ARCHIVE increase future, to n the to g is (200 and and Representative (25 costs 200 of d

to (4 may data c any Unknown forecast

note note o the cost-of-ownership C upload, PB disk-based $0.13 PB SDSC the 40 $1,843/terabyte/year service s numbers and TB be continued 7 one data 1 t l and point, to Capacity) i terabyte-capacity s Capacity) to n PB one PB be are

Capacity) somewhat tape annual TB the f i SDSC that that o in c copies scale the Amazon.com, can tape r

SIZE physical that tape r

falling and time. m e in annual the SDSC were per called plus media p Amazon.com, storage although Swedish a o upload 2009-2010. in g copy data r expense storage system. $0.18/gigabyte tape cost n less t of terabyte calculated s Annual e inverse between

plus by t i Simple data STORAGE i t study locations, due costs costs c s tape Mixed Mixed of at

system) Hard media half c hard hard h data o National to a a calculation SDSC data electrical will s Tape r

for for 1 data Total t transaction d replicates ongoing in 1 will of are proportion s states every

[Palm drive drive drive $133 d 2005 Storage tape/ tape/ a for to

a protection t the likely magnetic But r costs tape

i falling is $ large tape v TYPE remain the with Cost would data e 1 over storage and that

for large , 15 a Archives’ even 5 and power. 7]. is n S3 are 0 diminish. storage should d the individual data Service, of at 0 months, data storage one-fifth by

the / $11,000/terabyte/year, fee d service to $184 t online best be only Ownership $500 media least SDSC if hard e a against r t half $/TB/YR the a tape a magnetic of cost access, $2,000/terabyte/year. $11,344

b $1,500 $1,843 t for system y include projected a numbers 36% two cost-per-bit $.10/gigabyte per – services, disk t p or every size e and and retailer,

e storage $1,500 differential / the One

“cold data y S3, s loss. terabyte e y copies depending of a storage and s pay a cost r t were 36 (less disk e

targeted the of “crossover” f the objects m o

storage,” plus For improvements. annual system r 20% TRANSACTION recently months, s a the

of , calculated a of cost

than storage media monthly r

$133 per 1 and e Unknown Unknown Unknown hard example, between data s $102 study’s - for p p $/TB of across on of e at e access. cost is c a t – the the data a t given intro- costs soft- data disks $184 i b in one- vol- v in per y e 2007, t l e y . 6.2 Digital Motion Picture Storage Economics

IT IS IMPORTANT TO DETERMINE THE COST final delivery. A successful delivery is usually tied to S of archiving suitable elements for preserving and creating the final payment, so producers are keen to understand motion picture content well into the future. These the exact delivery requirements from the studio as early C I elements are generally considered to be the master in the process as possible. In the end, a production materials from which all downstream distribution company will deliver a mountain of film, paper, mag- M materials are spawned, with an expected access time- netic hard drives, DVDs, data tape and videotapes O frame of at least 100 years. This section of the report according to the schedule. applies what was learned about the total costs of digital Executives from participating studios were N

storage to the digital elements produced in typical extremely helpful in validating the accuracy of the O

motion picture productions. results. One studio opened several cartons of aggregat- C To develop an understanding of the actual motion ed materials to provide an average count of like media E picture deliverables requiring long-term storage, two for the analysis. This average count is conservative, and case studies were undertaken based on actual recent it is used in the digital capture case study to estimate G motion picture productions. One case study was a the average number of HDCAM SR camera original motion picture captured on film and digitally finished videotapes that are stored in a single carton. N I to distribution. The second case study was a motion The case study information is presented in a series picture that was captured digitally, or “born digital,” of tables in the Appendix that summarize the number V and also finished digitally through to distribution. The and type of elements, and their estimated annual stor- I scope of this analysis was limited to picture and sound age costs. Because of varying practices between studios H elements created during production and postproduc- and production workflows, several assumptions were C

tion that led to worldwide theatrical exhibition. necessary regarding the calculation of the number of R The film-capture production was chosen from elements and the “byte count” of the digital elements. A

among a number of average-length features (90 to 120 As with any case study, the results represent a snapshot

minutes) with an “average” budget (>$60 million) of time, and while production practices continue to E and few if any visual effects. The digital-capture evolve, the data presented are still considered valid at production also met the same content criteria and was this time, although the summary data presented in this R photographed using a modern digital motion picture section incorporates further assumptions (described U camera generating digital frames at 1920 x 1080 pixel later in this section) to reflect current industry trends. T

count. The camera output was recorded to HDCAM In the summary cost analysis, two key definitions C

SR videotape (this case study preceded digital capture are used: I

directly to magnetic hard disk), and the digital “mas- P

• “Archival” is defined as storage of the master ters” were also created at a pixel count of 1920 x 1080 elements from which all downstream distribution and stored on magnetic hard drives and LTO data tape. N materials can be created over a 100-year timeframe. The studios participating in the case studies provided O

complete inventory reports for each feature. • “Working Library” storage is a broad term I Each studio uses proprietary software systems to for elements that are generally kept on hand for T track archive and library assets. Since the two case distribution purposes. studies contained inventory data from different studios, O The participating studios save their elements in there was a need for a common reference of generated either archival storage conditions or working library M materials. This was accomplished by creating a generic conditions, depending on their preservation and near- hierarchy of materials for both picture and sound, L term access policies. which was then populated with elements described in The only picture element that continues to achieve A the separate inventory data supplied by the participat-

broad consensus as the indisputable archival master T

ing studios. These picture and sound hierarchy charts I picture element for a major motion picture is the YCM are in the Appendix. The charts are color-coded to

separation master on black-and-white polyester film G indicate to which storage category (archival or working stock. The current cost of creating a complete set of I library) each element is assigned. The source informa-

archival separation masters is estimated to be between D tion for the hierarchy charts is an amalgamated (albeit $65,000 and $85,000, depending on service options. typical) delivery schedule used by studios in third-party With respect to base storage costs, physical element production agreements. The delivery schedule is a legal storage cost information was obtained from several approach to describing the known “common sense” companies engaged in that business, given the difficulty results of the production process and specifying how in determining accurate on-the-studio-lot storage costs. the studio expects to receive these items at the point of

THE SCIENCE AND TECHNOLOGY COUNCIL / 42 6.2 Digital Motion Picture Storage Economics continued

The cost figures for data storage came from the San Diego Supercomputer Center S study discussed earlier in this section, which describes the lowest observed cost for a C The cost fully managed digital storage system with both online magnetic hard drive storage and I near-line data tape storage. It is important to restate that this baseline cost represents

M of storing only a single fully managed copy of the data.

O 4K digital The baseline storage costs used for this study are: N masters was • $4.80 per physical item per year for archival storage O • $1.80 per physical item per year for working library C found to be

E • $500 per terabyte per year for near-line data tape storage enormously G higher – Initial inspection and access costs are not included in the baseline film storage

N costs, nor are access or ingest costs included in the baseline digital storage costs because I 1,100% reliable information for the latter is not available. Nonetheless, these costs should be

V taken into account when considering the type and quantity of assets being stored.

I higher – than The table on the facing page summarizes the annual storage costs, exclusive of ingest, H the cost of inspection and access costs, for five common scenarios: C 1. An “all film” production that generates no digital assets R storing film

A 2. A film-captured, digitally finished production at 4K

masters. 3. A digitally captured, digitally finished production using HDCAM SR E videotape as the capture medium at 1920 x 1080 R 4. A digitally captured, digitally finished production using an uncompressed U digital data capture system at 2K T

C 5. A digitally captured, digitally finished production using an

I uncompressed digital data capture system at 4K P

The film-capture/digital finish production and 4K-captured productions produce 12 N 4K masters, and the 2K-captured productions produce 2K masters. Three copies of the digital master are assumed, given the recommended practice of data replication, O

I and this is consistent with the practice of archiving between two and five film masters, although three is most typical: a YCM, a finished negative, and an interpositive. The T cost of producing the three film masters ($80,000 amortized over 100 years) is added

O to the annual storage cost of film. The finished master is assumed to be 120 minutes in duration for all scenarios; a shooting ratio of 25:1 is assumed as an industry average M

to calculate the amount of source material, and two copies of all digital source material

L is assumed to reflect current industry practice and insurance requirements. Using current preservation methodology, the cost of storing 4K digital masters A was found to be enormously higher – 1,100% higher – than the cost of storing film T masters. The overall costs increase further with the use of magnetic hard drive data I capture systems at 2K, and further still with 4K digital capture systems. G Although 1920 x 1080 capture using HDCAM SR videotape appears to be a cost- I effective alternative to 4K, it is worth repeating that this cost reduction comes with a D corresponding reduction in certain performance characteristics relative to film. These

12 For this report’s calculations, a 4K frame is composed of 4096 x 2160 pixels, 48 bits per pixel; a 2K frame is composed of 2048 x 1080 pixels, 30 bits per pixel; and a 1920 x 1080 frame is composed of 1920 x 1080 pixels, 30 bits per pixel.

43 / THE DIGITAL DILEMMA 2007 6.2 Digital Motion Picture Storage Economics continued

performance characteristics, and their effect on per- Again, there are longer-term costs to consider S ceived image quality, are beyond the scope of this beyond those associated with the initial creation of a C

report, although there is significant and ongoing debate digital motion picture. I about these tradeoffs. Furthermore, the decision to With an understanding of the new storage cost

migrate HDCAM SR source material would likely be realities of digital motion picture data, the questions M 13

made in approximately 10 years. If the choice is made that must be asked now include: What materials should O to copy the tapes to some newer videotape format be stored for commercial exploitation on some new (assuming such a videotape format is developed in the distribution technology or platform as yet unseen? N

future), the cost to do so is estimated as follows: What bonus materials will be needed? What about a O

potential “director’s cut” or newly edited version? Is it C • Total number of original production videotapes

sufficient to protect only the finished master? Should E (from case study): 5,347 “mild” or “mathematically lossless” data compression

• Cost of new tape stock: $100 per tape be considered to reduce digital storage requirements by G one-half or more? Is the image quality of the archived

• Cost of copying to new videotape: $400 per tape N master sufficient for future display technologies? The I • Total cost of migration: 5,347 x ($100 + $400) = new economics of digital motion pictures require a $2,673,50014 careful look at the complete asset picture. V I

Annual Storage Costs of Motion Picture Materials H C

Annual Storage Costs of Motion Picture Materials R A

$220,000 E $210,000 $208,569 R $200,000 U T

$50,000 C I

$40,000 P

$32,589 $30,000 N $20,000 $12,514 $12,514 O I $10,000 $1,955 $1,830 T $1,000 $1,059

$486 $486 O 0 $1955 $180 M

DIAGRAM IS NOT TO SCALE ARCHIVAL MASTER SOURCE MATERIAL L

COLOR KEY A T ALL FILM I

FILM CAPTURE, 4K MASTER G I DIGITAL CAPTURE TO HDCAM SR TAPE,

1920 X 1080 MASTER D

13 The perceived longer “shelf life” of videotape as compared to data tape is attributed DIGITAL CAPTURE TO 2K DATA, 2K MASTER to the (generally) longer useful life of videotape formats, lack of dependence on computer hardware, operating systems and application software, and the use of DIGITAL CAPTURE TO 4K DATA, 4K MASTER error concealment techniques to counter increasing bit error rates over time. 14 Tape stock and copy costs based on current high volume dubbing rates for HDCAM SR tapes.

THE SCIENCE AND TECHNOLOGY COUNCIL / 44 D I G I T A L M O T I O N P I C T U R E A R C H I V I N G E C O N O M I C S 4 5 /

T H E D There needed and struction one-time incurs necessary payment will. never for everything” growing. step definitions of responsible other where. a increase the ongoing and Swedish at t and increases process investment and accession of additional accessed expenses accession structures TRADITIONAL 6.3.1 6.3 I a n G p I o the property born-digital T e authenticity t A costs ) h smaller storage sustained utilization The T of L

increase while The e h But countries, D purged. r San Economics r are most a a I What

the

L to which d For s “ E as National and costs

c economic until i M of the records

costs b costs costs costs, fact t the o The also have catalog, i M Diego costs. e a way. o (surveyed s to for e ownership, data A hardware because ongoing example, t of ongoing n percentage n

importance utilized, 2 o a archiving of

funding. of

support 0 may such typically l f the assessor’s t for the This to and assets of maintaining

0 high

h followed a and d no

7 life.

management e storage r These is

Supercomputer o On acquire c the

Archives, ANALOG the b This i copy expenses h n a include model lower doubt) time i as i costs new of g to initial g v expenditures cost costs has plot g the in asset,

prices when i building b n e the describe

property Archiving archives s variable u of g of and To every t archiving technology a

as s by lower records of

other i money the costs that i e lines) s for property n ongoing costs the x

the delivery digital percentage the but e w Means date, organizational p there as up accurate, low decline preserve s e i collection, s ARCHIVING traditional d county n services, society ” total digital of tax associated initial front, hand, e

analog to s requires changes. expenses i and are e l storage transfers, n point y –

storage Center, may c

hold for policy migration. assessor’s H a a data always digital and ownership c cost continuously t over trend of are taxes e o c the up-to-date archives g in in digital accepts delivery e labor l be asset l o p labor to y the r archive t maintenance higher w of the America one-time storage y e time, (disks film

must collection. Changing to for substantial with EROS, Records old d , o paid has, down preservation property continuity

ownership. b o collection, management and a needs access/con- office form d u archiving s and COST

records

archives t scale and This j a

and the be always access u every of levels a and n the power a s s while records d t

power (and and “save of s

to con- is are e e tax up the l e s n be e of - technology Motion total specific a lished skills funding conservation media longevity order vert driven on o of jobs, assets. emulation and ditures objects. emulation dures the preservation and t archives central Hollywood’s enhance cial appreciate archives to f archives. archives or profit Historically, benefit require o o p range r

e ( the

more the the f i r digital m media u a exercise, they As Both or t which t required services, cost to i m is u “public organizational staff, n good

durability data periodic restore These r role to explained will investment of g media e

e primarily are have around and citizens. make the So d

Picture

require e of g Digital traditional support i “white-glove” x over archive required rather products a e p t in migration. be n bandwidth leads digital not of film e potential activities. trend most organizational e techniques, e data media l of developed processes n y r works” by the film types. them realized society a ) s time frequency t

e going the e i a s and

archives than o conversion to traditional elsewhere l

In ownership. determined quality i business. using archives a the n new m archives “today” assets n for s assets in world peaks useable/saleable. , d Hollywood, and i labor and and

proper n for c

for regular as to physical down c a the

o in long-term Industry requirements business can But a t n as current e p conservation a and be can when religious digital v

managerial control, commercial is the i and t of and e continue t whole. future. h valuable have a y will in continuity. be in

primarily the e cheap, l But i archivists use can

technology

n yield secondarily indefinite i o procedures by n the this will and g done valleys l conservation they v

d more require been t model technology archiving of e private

expand h digital private c s preservation

belief data e t or o Many report, Digital profitable play m they media-specific

corporate s c to as are t u competence e for scholarly s

power operated modern determined in exploitation techniques To n

l are o operate t periodic migration to

t recurring u an “tomorrow.” retrieved, Specialized f migration , that are digital obsolescence- the the

potential cinema r cinema by of f a a reiterate, sustain i well-estab- longevity generally archiving l l l increasingly l

not “ and data d of sustained workload system, some p commer-

e assets a approach a p of purposes, as as r media t cinema batch c e going r proce- h i n audit for digital public expen- m non- and/or digital and by in i d access of or v the i o i that of n will n n g g y , ” 6.3 What This Means for the Motion Picture Industry continued

at larger digital archives has been to automate data cases, at film labs and postproduction houses at migration so it can occur as an ongoing back- no cost as a courtesy to their studio clients. S ground task in order to smooth out the workload This model has well served the studios when the C I and the budgetary expenses over time. This elements are completely film-based, but the increases the importance of organizational conti- situation changes dramatically when digital M nuity and sustained funding. elements are involved. O At the same time as the cost structures of archiving are changing with the transition from 6.3.3 Don’t Save Everything N

analog to digital, it is clear that access to archives O THE “SAVE EVERYTHING” POLICY, (even analog/film) has become increasingly valu- whether motivated by concerns about future sales C able over the past few decades. Digital archives opportunities or adopted because it is the path of E are potentially more accessible than analog/film, least resistance, must confront the practical reality which implies that digital archives will become described by several studio executives, which is G potentially more valuable than analog archives.

that studios are producing so many bits and pieces N Digital archives offer the benefits, compared to of digital content that they cannot afford to save I analog archives, of faster search and asset retrieval,

everything forever, but instead must learn what to V easier local and remote access via networks, lower discard. This is especially true for feature-length I cost of replication and distribution, and easier and motion pictures because, as we have seen, long- faster format conversion, including the ability to H term digital archiving incurs ongoing preservation extensively “slice and dice” old digital assets into C costs that are significantly higher than archiving

new content that can be commercially exploited R film – on an annual basis, $8.83 per running for new markets via new distribution channels. 15 minute to archive a film master versus $104.28 A

6.3.2 Save Everything per running minute to archive a 4K digital master. It is useful to look again at what is being E

CURRENT PRACTICE IN HOLLYWOOD IS done in television. ESPN – arguably the largest R to “save everything” on film in the film archives or cablecaster of sporting events – is awash each U warehouse storage facilities. This ensures future weekend in data tapes. ESPN covers professional, users will have maximum flexibility to pick and college and local sports in most categories, and by T chose what they want, when they want it. As an Monday morning, unless the weekend’s captured C archiving policy, “save everything” is fast, compre- material is cleaned out, there may be no physical I P

hensive and simple to understand. And it is safe, room to ingest the next week’s national and because no one has to take responsibility for international feeds. ESPN’s guiding rule is to deciding what not to save. This practice has been save only assets that cannot be reasonably recon- N extended to include most every element not on structed. In their case, the sheer volume of data O film, which includes paper documentation, digital forces decisions on what to save, and the same I data tapes, videotapes, optical disks and hard drives. sheer volume of data makes it impossible to save T

At one studio, a senior technologist recog- everything. The decision as to what to discard O nized the potential for future value of all assets, and what to save has been described as “triage on but said the decisive reason to save everything is the fly.” In the motion picture industry, the M that it is just too troublesome to sort out the practice of deleting digital non-“circle takes” on desirable materials from the undesirable. The eas- set has been reported, but until the cost of L iest thing to do is throw it all in the vault. And, digital archiving is considered, there has been A

in the view of this technologist, ideally the digital no compelling reason to do this or some other T

“vault” should be online magnetic disk storage culling process on a regular basis. I

because this will make the digital assets more G accessible for both re-purposing and data mining, 6.3.4 Who Decides, How, and When? I

including new techniques to extract contextual D THE VALUE PROPOSITION OF “SAVE and descriptive metadata automatically. everything” is changing as the media business goes Historically, motion picture elements were digital because for the first time in history it is and continue to be stored in several locations: on becoming feasible to create digital distribution the lot in the studio archives, at independent film libraries and digital archives capable of exploiting archives and storage warehouses, and in many

15 Assumes 3 copies of digital data and a YCM separation/negative/interpositive set for film. THE SCIENCE AND TECHNOLOGY COUNCIL / 46 D I G I T A L M O T I O N P I C T U R E A R C H I V I N G E C O N O M I C S 4 7 /

T H E D b this owners. necessarily saving marketing economic the a t c b s h A c

REVENUE ($) 6.3 C h h I h h t Content y e a G 0 n T

a a s a a v

I t ont d T C t t r IME (YEARS h n n e s marketing

A e R THE e report,

T e s e n g h

L l r a ELEA

l t o i e s h r t D l l h n o e everything w i e l i N I What e

s m a g Motion L r s n A i

ent L

E ON- s t

s 1 L A This s

M v

TRIC s e

a a r a aspects a l SE o o

n concept, M i a r l practical n o l v n e and l Lifetime g e d r A u THE d f s a g s g u

m l

2 i e

AL v r theory e s t b n theory 0 T

r o d e e l 0 o i o e therefore o A a l t 2

n if s 7 n r u n t i )

i c

TRIC

of a o for h f l d m

e k

o l This e o u a t

x s b its r h for r f t e

g this c time u e

f came

e future a h e W of p o l c s AL RELEASE Picture o

e e . application a r r t u

d a r today’s e o

i n

D 3 “The y n d s r T r theory s d

e e

s deserves i i c w d t u

g , t h T o d

h up

i c i e o a Means A MIGRA m t and l u e f t

l s a

s L e e

l r a Long

f in c d h

o 4 t e i g m b motion t h r i e l present n i a g a o s unknown

v a several e z g t t s i t e t

t d i i t some

h and

a v n l a o T a i y

r e l Tail.” a e r r t

t Industry T a l e

e m o m i y o i t 5 c ION

o i attendant v u

n e a f picture n an a

r e e discussion. d k interviews l o e c

a w r i e d l x v for k

t o a

p t

e e i interesting uses n While

u w c h s d l a l t o u t p a e r r

y i w e 6 r

i d

i n

n I a b r

a e content l n e h

are

u e n m m the n 2 t i d t cost d g e 0 t i the a g a r h o

0 n for n r not e n e k 4 s d e s e continued t t 7

of t . o r

8 lead argue assets n t d e c b l c o t p d b p i h h x o o f t e u a r e i o THE g

t e e t o m n c c c s s d l e i

a l k m t i e t t v l n i i n a p u i o

a a n g global a n i l d e o l l g s w i n g l e L for

that e u t s e 9 s u t e m , i ONG T c a t s

a e d e e . c p o r n l t

e e

o n o a

b o m o a m new d m A

s s r u a

f digital a t i t p

e media f l e a

e u d

t d o t n r h d

a n i e AIL n h i o a n f i 10 c e y i r g t a

s distribution a e e f

t w g a s d i s i

t .

x i t i t l e

i v h e c a ,

p a a

n $ t

v e l i a l B e

media b

t e i t

i companies x e g 100 l w

v h i a t c l

u i r p e i i e e d s n i t i

v a t s r

e s r e l t u

i e l t d l e y h n d t t t

r n p t p h

i y

d a e s

s a ( p

l o p e e e

n e

b i t l and I h i

a c

p c i i s s u n t p L t c k a a l a

p o l i t a t e e t channels. s o e c

o e F i l e o x r

t n t n o

r e i r f ILM LI

i h t digital o f o n o o o to o n t g , e

e f u r d

f t r e n n

e T

o n t t , o d . c o a expand v h

r s i a o w s a

e e

f t t

e l 100 a T i

l d d m n

i o FETIM l

s l n l e a

m o e t e

u h n distribution t s s e b m o l s w h

e e g r i

o f e r t Long v

e d e o s e o a c d a o L e o

r r e g r a

n c f r e

o their c r

r r e u

p y E u r d t y i i e t n

n r a v

s e h s

a -

i a g e l t a e s r p

s n e i Tail d o

“ u

i d s

o u o T d r s h m i a m b

business e e t f d r

f t i a

s i q r a p r t i e o

r e e i t o o s c w u l a o r d q s n theorists ”

will c m n s r

i m s

i i u a

e t i r e a t m

t t h e l s

e w h

o y a a s d a n a e

) h s t

o k

d t t s e r e o i e n a t t 6.3 What This Means for the Motion Picture Industry continued

by first selling a few products in large volumes to mass try, the comparable techniques might include re-editing audiences, and then selling small volumes of many old content, or extracting certain types of scenes or S products to thousands of niche markets, reaching new dialog, or re-sizing and re-compressing for new C I customers who are willing to dig deeply into a studio’s distribution channels that did not exist when the media media catalog. assets were originally created. For example, one studio M The key supply-side factor that determines whether executive described how his company had generated O a sales distribution has a Long Tail is, of course, the several million dollars in new revenue by extracting cost of inventory storage and distribution. Where certain phrases from old television shows and selling N

inventory storage and distribution costs are insignifi- these sound bites to consumers as downloadable O cant, it becomes economically viable to sell relatively ringtones for cellular telephones. The Long Tail C unpopular products; however, when storage and distri- implies that studios can maximize their profits by E bution costs are high, only the most popular products adopting a digital archiving strategy of “save everything can be sold. forever,” since everything will have value to someone G Netflix is often referenced as a Long Tail business someday. However, some feel that although the success story. A traditional movie rental store has limited Long Tail is very long, it is also very thin, and therefore N I shelf space, for which it pays facilities overhead; to max- the cost, today, of saving everything, may preclude imize its profits, it must stock only the most popular implementing this strategy. V movies to ensure that no shelf space is wasted. But I Netflix stocks its movies in centralized warehouses, Where to save – islands of archiving H so its storage costs per unit are far lower and its distri- C The number of groups in a major studio capable of bution costs are the same for a popular or unpopular R movie. Netflix is therefore able to build a viable business making their own digital media is growing. The creative talent and production facilities are becoming A stocking a far wider range of movies than a traditional decentralized. Some or all of the content made by

movie rental store. Those economics of storage and E distribution then enable the advantageous use of the these teams may need to be preserved in digital Long Tail. Reportedly, Netflix finds that in aggregate archives. At the same time, the potential distribution R over time, “unpopular” movies are rented more than channels served by the studio are proliferating, driving U popular ones. And the people watching these sorts of creation of more digital formats. One studio reported T that without a “grand plan” for archiving, they are movies are typically prepared to accept a delay of a few C days between requesting a title and watching it. seeing spontaneous emergence of independent “islands” I of digital archives in different business units and func- The Long Tail theory of digital media marketing is P consistent with new “data mining” practices emerging tional groups. These islands have been developed as stand-alone solutions, and often hold redundant in other fields such as oil/gas exploration, medical N imaging, Earth observation science, and even commer- inventory without consistent naming conventions (metadata). There is frequently no inter-operability, O cial credit card services. Data mining essentially I involves analysis of old data (from the digital archives) even for interchange within the enterprise itself. T using new algorithms running on more powerful com- According to some, digital archiving operations puters than were available when the data was originally can add value to assets in terms of potential content O generated in order to extract new value from the old repurposing far more quickly than analog archives. M data. Data mining is being successfully used to find The question of future accessibility is the basic question new oil and gas deposits, perform epidemiological behind all those decisions. Right now, access over L studies of medical trends, track climate changes over extended periods (100 years or longer) is not guaran- A time, and enable credit card default analysis by region, teed in the world of cinema except for YCM separation by time, or by individual. For the entertainment indus- masters on film. T I G I D

THE SCIENCE AND TECHNOLOGY COUNCIL / 48 I N D U S T R Y N E E D S A N D O P P O R T U N I T I E S 4 9 /

T H E D 7 I G replaces. system preservation the capable least should system archival A I T Industry A L digital D I L E film M M as A

2 be 0 it 0 as 7

at Needs Immunity Guaranteed film As picture solutions l a n p Council improved for obvious, access state, long-term tion sition tion needs. and and based, of T and Picture must Cinema than needs: Ability s inaccessible. reductions subject characteristic film Simply access system e u n r e v H b a s d e relevant s c a o and the

I r

t p in digital 1. To Herein Armed a S i p production and and preservation is starting i n sufficient g r c be R in t to e t to to e e i hybrid

no other e Film m s universally entire

performance A As to, materials put, to E that e o t r and our or able sees h a e but r implementation grounding digital the a f P that v s digital

e replacement l longer safety stated digital t a create

O a in in

facts data Opportunities handheld from s e c o with t lies

that n archival view, end, i

c R w sound access point, industries content v no they o to Although of the history staffing e v and e solve n T i p i quality r t r . the l h do t , motion handling the and o reason

a T of i the is

earlier, sound available. stated extended o n system p words n infrastructure duplicate what W simply archival the “born-digital” u requested p g m H what the and a opportunity t as

for i film quality e so

t n informed perspective e U masters,

r of stored e portable digital or h n g e most c many to S technology g that

a same, t follows with characteristics h to film without at n access our a a v picture of recordings: a funding n F they it archive l e r e generate articulate

practices of d o A e

abandon c primary least t one replaces d

l o periods R h the o t fundamental in industry. masters may which s archival appropriate similarly n system o problems by e g

, for when o H have

i media to t a opinions the e f are i studio h motion b

t s are every materials. 100 regard o for A e of i

is i both motion important slowly o would a l

t S i m any n h n available the t archive that goal occurring with S the a y meets clearly in these e d s properly of the o

, r years: to system T

solid players. I t

m that studio large

r o M i executive, t the existing Although most distribution h as e n neglect

needs

for its picture e f s of fulfill d e motion d about degrade not film, u P t technologies t r e picture the industry needs. o u h of n e L following f

or should information this r defined, contents s

o i today’s Some d and Y i meets a The t n today, s d corporate basic r should a cause t the and P

e i created new i exceeds s m

both n met e and

o Any

a future the s o d and work R n and e industry’s , n long-term

picture

e single n

E if periods q there a d elements, v w film traditional of t n needs digital u be quite S

e that the needs a creation available within c funding o the replacement d master, l e unrealized E or l ultimately be o o remain financial these

u s respects:

n is N and guaranteed m t p t l (and i h and that archive d e content and e o may exceeds characteristic T Science at to is e m e

base, d n industry successfully a d n of E

regarding of technologies

l u what s b least stored, , m t provide requirements D needs the

practices n h data y cultural p o an of digital whether and unknown) not benign shortfalls accessible o i a

A solutions: f e c o

t s the r a m

t t original we h hardship: motion L archive p distribution t enabling h as

to film they and e a a storage o r be preservation e archival

A p l with o t the o for r w following have to capable spoke i disappear s p

R u o sufficient storage , i a an e

the n neglect. assets. n r G Technology y t break that r want archive for

by d o at of

l s p E y p were

regard

u equivalent even for a picture seem r been h

i archiving s least camera e c r and c a distribution film C a 4K may sensible t with t m r t technology guarantee r v i Another e u digital O as c i digital when e a technologies, from e a r u

to t channels. s i L e sections if a Digital

l preservation e of or n

to the 100 informa-

technology a That

to a L d appear of p was c

r it result n a industry

E h r

more become f create. n n o d negative the i o film- were C the and l e archival d

selec-

i m motion years. o of b or e c T that u is, d p u e c I the

tran- s t s and t in O o i , if i i o - o f N n n . 7 Industry Needs and Opportunities continued

archival masters support the generation of distribution

2. Standardized nomenclature S masters with little or no quality loss. The current use of 2K and HDTV mastering pipelines, and 2K digital cam- The multi-studio case studies undertaken as part of E eras for theatrical motion pictures, as well as insufficient this report uncovered a problem that has taken more I than 100 years to develop: each studio has a different

attention to image quality during the mastering process, T together are generating archival elements that are of naming and identification system for the physical I noticeably lower quality than films created more than 40 and digital objects they create in the manufacture of years ago. theatrical motion pictures. These differences developed N The deployment of 4K Digital Cinema projection for perfectly logical reasons: each studio’s inventory systems and the introduction of 4K consumer displays16 management system developed organically, along with U its internal business systems, so there is no uniformity are clear indications that future display systems will T make greater picture quality demands on the archival across studios. Unfortunately, it is impossible to effec- master. At a minimum, image quality metrics regard- tively leverage any digital solution with the inefficien- R cies this situation creates. Our attempt in the case ing spatial resolution, color gamut and dynamic range O as defined by SMPTE Digital Cinema standards should studies to simply and accurately quantify the amount be the baseline quality standards, as well as the corre- of film and digital materials generated during motion P picture production was hampered by the wide variation sponding standards for audio. P of inventory management practices. Further refinement

No dependence on shifting technology platforms: of the industry’s needs in this area will be that much O

Film stocks have changed over the years, generally with more difficult without uniform naming practices. increasing quality and stability characteristics (with one Individual studios will also benefit from such stan- or two notable exceptions). This technology evolution dardization efforts. As part of a recent internal review, D has not compromised the accessibility of the film archival

one studio identified nine different ways its various N master, and therefore a replacement archival technology business units referenced a single 60-year-old property 17 should not subject the archival master to such a risk. [Solomon]. Rationalizing object names not only A improves access capability, it also enables strategies

that reduce the number of duplicated items taking up S valuable digital storage space. One studio executive

claimed that “de-duplication” of his studio’s libraries D

reduced the overall inventory by 30 to 50%. E E N

Y R T S U D N I

16 Sharp Electronics 4K LCD shown at CEATEC 2006 and CES 2007. 17 The discontinuation of older print stocks has, however, required compensating color timing and/or correction of older films, since the color characteristics of new print stocks have changed.

THE SCIENCE AND TECHNOLOGY COUNCIL / 50 C O N S E N S U S 5 1 /

T H E D 8 I G I T of to equivalent c longevity process format master archival digital There A Consensus h L D a I film. that L E r M a M c A

t 2 is 0 e or 0 7 r with

no i s t i c s That g T i z i by questions decisions presented While d a g p a s A 8.1 what our the A today’s with n d e a b r o r i e s L h p e c e a t f yesterday’s digital picture responsibilities Identify quantity film looked practice fundamentally Accept o l a

l i

i n t i e T s a digital u o l v a v n u problem t A 2. 1. Given • •

s r industry, t

e n e e a n H

a , r i longevity s i l

being actions What What n

r c t e

t e a t n n t e i To h a i O s there h c t is h f preservation Enable Create n o d

l assets

e a e d o

( e g e n

d s d d distribution cost-prohibitive

u e e f a m

at regarding

and

assets. h that production f

t in e a i v d n m G g the in the e b h g l i of r a e i said, d h i c s should should i y from l e s l H v t are a Start t t i t of this

t to

motion e relatively o a e s

while there a e i a q e data understand i p t r conclusion

n l r stakeholders must r f v

s characteristics r l e u T , y f i i w digital

c e

take the film r small o c n t a h t

r t H

there s s h a paper, i We different h r l g s with

y this , o d p i l d i h

a e

and E n w w e e be be a is s t y n a r o digital work

c t t

e b

h going o enterprise t a e e u l P i n f picture

have separation

general o channels l f n and a r groups e t new i i d o done done e archiving i e R are n s respect i n n c t simple d c ongoing asked

o r strategy

i c h h e a their a g o a v I

e n a , r e v l M

t - i

continues with s

that o c r

that i n r t u a some postproduction. i s s than o o asset heard t perspective. e

h s forward. r a t r i l in A u c

r n f over right g m t r y r i a r

m i a v u y a production R s i “ agreement m e within influence equivalent d when n l a

a n

c the asset h to a l there preserving (

current d Y a – l t y t g s a i t 3

i preserving e n creation u e z common preservation t

m g

t and 5 d u television, r I r from the a d the d d e r h now? a a N m t e enterprise

n l a

e w t i n y t s e u s o a and e considering masters s T o on

m t x

g is i s t n a g m each n e The long shine creation,

t p

e i E n develop y t r h i n q

h m no e several a . m digital f

d N o developing

u g

is i f c n e c The inventory

f o

among to a o t are S m u

T o p

u (including t e r i l not digital n consensus u elements studio h digital term? o s m m i r

f d

film,

t t a o a o a s r m O that a y

Internet,

r e c p a n c c f made. n a and light , “save c e o expense. t g e F e o ” u - g exerted storage d u h studios -

s r e

m preservation n

r t s m a

r i s T e e t D a v s e the of e the n ; and a o r r e motion i

that l archival s a i s H define i i d d b l n m t

p y on z

management i d g i everything” rational e film,

u r suitable g e o

I t

to a i e e r e a huge This o s people d S t

corporate t n view r

- n as v

a r

a r t mobile, e

the

p p technologies, understand t - g until that n e t t h

l D be

h p

s u r o

u l t d such, e C w o u o e i h their o r O e the

p picture c d p e u

a i i considered: is important l f t master number answers n t t c s e n i C u c u o s the h h ) l e d e c

well long-term interviewed m t and s too c

r s i a U f emphasis m i a e e

e t n a s r t o digital s r every s d

- interests, i and e r

s m e

v r a v growth p structure)

t t ( i practice e

m e d h e y n late m u d access P after l process a E

i materials a e format g e r e s s

o o a the o t N

e i t x

t g two so of c i e r f t d e r t

n p assumption

r e s k given d b T i i r to i i a

related o b

o d t s a g

o u choices on t d the

b issues n u s g u i h I is solutions. a r

n of e t u of ask basic u e o

t S n s roles r a a d used i p i with r s t e and s w n q

l on - – o or r ) e

i i must T

alternate d important digital w , y u the

c n m n

s n

c t has is i the t t

O e e e

g e h

a process i t u a n protecting issues a

b s l h c i questions: p e h and corporate l t r

s with c t presented e regard h u

a D e huge n e

a a i h w e n fractured t r n s s s b l be

i i t

) E and f e e i o f l a n m o

i s motion q e l e i t l l F r . l l e r m

s o d u

m g o for y o s I . s e a s t , N f . i s to n o , E i n - 8.1 To Start continued

structure, and expectations for future fulfillment do not always match up with the S realities of current practice. Digital preservation and access must be defined for the

Accept and enterprise as a whole; e.g.: U ∑ understand • What is a long-term asset? S

• What is considered perishable? N that • What elements justify the cost of digital preservation? E • What are the methodologies under which these decisions are made?

preserving S

digital Enable collaboration among the stakeholders to develop a strategy for digital N preservation. The following questions raised by using digital technologies cannot motion be answered by any single department or division: O picture • What is the value of the content? C • Who determines the value of the content? materials is • What content will be archived? fundamentally • Who determines what content will be archived? • How will the content be archived? different than • Who determines how the content will be archived? preserving 3. This is an industry problem, and to solve it, the industry film. must work together The founders of the motion picture industry knew early on that their business was about selling movies. The mechanisms needed to create their product were simply means to that end, and they generally went to great lengths to reduce the costs of production. Collaboration on solving technical problems dates back to 1916 with the creation of the Society of Motion Picture Engineers (now the Society of Motion Picture and Television Engineers), and 13 years later, at this Academy through its Producers-Technicians Joint Committee. The standardization of 35mm motion picture film, camera aperture and theatrical sound equalization, among other things, was the result of collaborative efforts by companies that otherwise competed with each other. The issues of archiving and accessing digital materials are of the same nature: no studio or filmmaker will make any money from the technological solutions that enable the long-term preservation of and access to motion picture content. However, unless and until the issue of long-term access is solved, future revenue streams – and possibly the art form itself – are highly endangered. The motion picture industry must not necessarily accept solutions that fall short of what has been used successfully for 100 years. The technological solutions are likely to come from outside the industry, but it is vitally important that the industry speak with a common voice on its unique needs. There is also an opportunity to collaborate with other industries that share common aspects of long-term digital preservation and access, particularly with respect to influencing storage vendors and system solution providers to develop products that more closely match our requirements.

4. For the short term, actively protect important digital assets There is no denying the reality that over a billion dollars18 has been spent generating digital motion picture assets. Creating YCM separation masters on black-and-white polyester film stock protects the final theatrical product, but there may be tremendous value remaining in the multiple digital masters generated from a motion picture, and quite possibly in the original digital camera files and tapes. There may be both business

18 Based on the number of digital masters and digitally captured movies to date. THE SCIENCE AND TECHNOLOGY COUNCIL / 52 C O N S E N S U S 5 3 /

T H E D 8.1 specifications. SMPTE standards standards, be dards are image technical, Lower Open-source Standards: done two have It Design D p t o i d r o f l m e t a c f a c l t p t prietary changing interesting (Lots iRODs as data source I o n e u u G o h h h e x r o b s p p e i r t - n

e I i c n

i t e i e e c e t n l T is e e t a p e the g a s u a e t e applied g

l s i d A , r m s

d important u t u r d r o a t

r e

r t a c f W L years.

i i worth e a i s m i l i storage e r a r s r i n l t o n finite n p p o g t e a t to D g v e i e Of t i , i development i To g i m Storage v r n

f g t l and u i h f a I i

a h o n o software i t n u m o L

t t . i e

s g t t n distributed a h M a E n s i w a n the single-vendor v mitigate m r

r l e t i i l c M s

is c y f l u i e o e c m

Copies e for s a d

a i vendor

a t s c o it s

a h r z M

i o n l e s i

the n o r u w to sound

o t e w u s documenting usable e opportunities repeating

e a However, s q a r o s e A t

m If i n m

t s needs e s m t Start a t m n h s problems. i

human u e i s

i risk s s l e 2 e e a b s h t n low o a

today’s g a e l t o s i e 0 standards

p c software: p o p work d

e Resource industry o l v v n a t n e t n n g

c 0 e e d l

p o o f t a d being d

r r o h t i o 7 o h s

n a t i l t

i g Keep

f d

l t s

the

strategies e u d a f c a o i h l n

l e encoding a h a a lifetimes, d c g c o i y u t n from e n for r a m c d risk n g r a n o u e h a e b d o r storage community, y t y h g t p i i a

d items t t t i

l e

g t w , ’ t , z i e t

t i t n y impact that o p y s f

a s needs.

o o i o a h u a i there , t o M t digital l i developed ’ t e c a o f l n products. e

t continued Stuff t o t s f n a e

r n

r h r e

i n

a should

l a g f Base a o d l n

g o t of t s o e Broker, n d exist, a c There s i e l i s s

h l e

u threats: Digital l a i

c n i g h for U n g

h s i a d

w m modern g n a t m o g i n for p n e - s y n r o d s

n o i and system, t t n i w technical t i o b a are i p r t t

specifications,

in , r o a p t u e l s z s d Safe™) t

s i a

o r l l a u l of a e i minimizing

n r software archiving, t In l i i a t e d

o s i l n

a u g o a m . l o the r m b o g n

use t t s m g e

r w many n s c o e i g r i s organize s t a is r some e d a t . n technology o business o s y t i e f

t b y , a the t a

e r n the i a o s

and t g n h

n ’ c Cinema i u specifically d g y v r y j d i a t a s a o n international g d h e e t them. e o n r s e s p

e ,

technology w c i

n c

d a d

e t

a m “ large n

a and l m o e c g c s economic, w e i y t p m absence t s s n

next-generation g l t o e n o n r s m e program g cases u t w

things many r r i i

a p e t e o o w l g m d i a c i l o t a technologies w n o o n d r d i l t . a t n r r i e t s

W d l a i s obsolescence t e

the much e

l i e m e h i l h

l to d d i d m h m b l a

goals, o u s

body l o

i d g l

i b a the

File e

a s c

distribution a i y f p l c e n y c a , p a t v a i p as s and t e n e t

create r t z e o h r o

r w

churn: t e e i a e of

c d LOCKSS s c d t e e m e w a that n e t m

s of e s

c . o for o i e a t s h f

c short t t

m h impact r t a o d e m formats, i

s r

f o p a o a offer them products e as e h a e o of l o g W c a o e n r and metadata e k l relevant

i e i

m n

stan r o e v n , u u y e m g n e b v a o large t b e t r t t a can

n u l x l o r h n e open-

s i t m m t the a o d t t h a a g a o i u l s a r e , d i v h s o o b b t n as c i e g e t o l such t i i t pro- - t i d a

c e p t c s ” a h e can a o s n g i e s o l of e be e r g e l e n o

l - r s - t d s e r . . s o o

. empowered stored files, is tion (or easily management and many of organizations total In-house Outsourced a f t storage system, initial reduce affect a h n c e the d t unintended r • • • • • • • • • Appropriately • • • • o / not o

e in For r and and software with depending e f personnel initial replacement software initial risk data “rental” data increase electrical cost which s c r a x doubles practice total t redundant o u investments and c t the the o r system, managed, e i total u l r both c i r i this mitigation software t t n n of respect s Systems: i ingest ingest o e o a absence g cost the cost and hardware, n s in u l c decision. ownership

n Systems: cost a r drives i policies, cost o we scenarios, in c d e costs or n of and turn power s multiple associated i e t costs, total n t d per replacement of on w r s costs

g and and w . eliminating triples copies spoke r F sizing of o to d as e costs for ownership. applications hardware to r the of a h bit, When reduces k i storage of u g ownership l the the e cost

access access a i n r e construct. and including storage sensible t as every operating t c s When b service a service h is c t although with l data the of the of another e a s case threats e u digital r of cost. t data s t i m cooling e building o l hardware, s motion d

costs r the amount storage c o bit c ownership i a costs costs have copies. n media may o o unnecessary r maintenance g outsourcing e –

s g information s provider e duplication. provider’s

Larger , t t a That includes: of s t amount ongoing this s s asset described factor system ,

h e d up. they

f r be) t the o costs e data v a picture i r r g system a i costs x e

can of

c i d a e That

requires t e digital operating “De-duplication” by i a common collection systems s require s s r l that data :

is includes: t e s failure a r of d and vary training i n t an level b saved. o redundant d i a elements, u is, earlier r f data will lifecycle contracts

f a will t i managed digital organiza- Many e n storage e g application r there widely, s d another e e larger tend u of s

n also s problem r to impact y

systems t y grows a This c s service s n t t o be e c e are to m s e m t s 8.2 Long Term Initiatives

THOSE INTERVIEWED FOR THIS REPORT AGREED THAT ACTIONS S can be taken to produce better solutions for long-term digital preservation and access The motion than we have today. The Science and Technology Council’s goal is to move these U picture notions from just being written about to being acted upon. S industry 1. Collaborations N We stated earlier that the motion picture industry should organize itself to speak with a E should common voice on matters of digital archival technology and solutions, thus enabling it S to effectively join forces with other industries that have similar needs with respect to organize digital preservation and access. This is not a problem that can be solved without great N leverage – there needs to be a consortium of end-users, i.e., customers, who can O itself to economically scale their demands to make it attractive for vendors to agree to open speak with standards. We point to the audiocassette, CD, 35mm film and for a while, the DVD, C as examples of this. Many, many companies were successful in manufacturing, a common distributing and selling these standardized formats. They did not need proprietary “secret sauce” to be successful in creating and servicing their markets. voice on There are a number of examples of cross-industry collaboration, the most notable of which, for our purposes, is the National Digital Information Infrastructure matters of Preservation Program (NDIIPP), created by the Library of Congress (discussed earlier in this report). The Library acknowledged that the scope of this problem is simply too digital large for any organization, even the United States government, to tackle on its own. The NDIIPP program currently funds more than 16 external partners working on digital archival preservation research and collections, and the Library is engaged in numerous digital technology preservation-related partnerships with notable institutions including the National Archives and Records Administration, the National Science Foundation, and the and solutions. Digital Library Federation, as well as digital preservation initiatives abroad. In August 2007 the Academy and the Library of Congress announced the Academy’s participation in NDIIPP’s Preserving Creative America project, a joint effort to address the issues of digital preservation as they relate to theatrical motion pictures. Participation in this program will bring increased visibility to the motion picture industry’s needs, and it is hoped that we will also discover new ideas that will lead to better solutions for the industry. Topic areas of this joint effort include:

• a report on the Digital Dilemma from the perspective of the independent filmmaker and smaller, public film archives • development of a digital preservation case study system to investigate various digital motion picture archival strategies • development of requirements and specifications for digital file formats that support long-term digital preservation • education and research activities related to digital motion picture preservation

This is just one example of the opportunities available to leverage the efforts of several organizations and industries toward a common goal.

2. Standards Development While we have heard conflicting advice from other industries on the value of standards with respect to digital preservation, it is clear that the motion picture industry has benefited, and indeed would not exist, without worldwide standards for the interchange of motion picture content. International standards have the added

THE SCIENCE AND TECHNOLOGY COUNCIL / 54 C O N S E N S U S 5 5 /

T H E D filenames, als. of tial but e was I munity: but by these exhibition. oping in publish (ISO) and Motion leadership community successfully the which benefit 8.2 I n x G h i I archiving use t T equipment i i user A constant

b the by a taken the Similar We Image L i t standards t i D standards are o today, v provides a I Long of e L r process most International Picture E group. believe s the s consortium M

– the c being from metadata, role M o

file

a implemented It A effort of n Hollywood and churn s

and of s 2

two w is o manufacturers 0 those formats, in 0 are r was a another that e and the automatically for t 7 interesting access l i Term built-in

l both their u

must accredited a m based of s Digital driven motion Television standards

standards of and . f

Organization r new

o organizations development. M of one their be m important studios metadata on u mechanism and

digital t c technology. applied by h Cinema h and segment picture Initiatives e specifications

standards associated and

i a adopted e n reviewed are takes q Engineers p committed via important u u motion technology i t most p to user

registries the for o m industry’s distribution an of n are

the for e The

n when t bodies Standardization Digital the active h group “wrappers,” every likely t actively

e picture dealing m ad (SMPTE)

written s Society to user and a p n all hoc e the providers, five note u c to and standards that – i Cinema f f are a influen- com- i continued the and be materi- c devel- c world user with t a years, u not of t that of i r o e n r s s , digital together standardized. ized. camera is equivalent so-called Cinema dards recommendations postproduction, cycle universally digital ment limited sustained around The formats, file format definition have there formats, Compared Based subject to

of on Digital efforts formats formats will acquisition usefulness one even be digital distribution metadata are what Digital for of on of preserved accepted require or needs still wrappers, are agreement a archival the film

the Technical they Digital several for are motion to building distribution influencing Source cut experience unless scanner motion and a formats Digital still image a are focused standard in formats. detailed negative, purposes. digital Cinema image subsequent in pictures and a on Master innovation there digital consensus their formats Cinema picture output and what how effort the and to formats and evaluation for

But is is infancy. archive metadata assets date (DSM), archive. evolution they a not industry-wide all archiving. sound formats are film, SMPTE to DSM that the around phases and in standardized, that build are also – format master television, motion might production,

There i.e., of encoding market is. registries. to can not are of alternative consensus in DC28

Digital

be Digital The the various not the be digital for one standard- used. is agree- picture forces digital no DCI the life the day stan- nor

8.3 Finally...

IN THE CENTURY SINCE CINEMA WAS FIRST INVENTED, MANY S different perforation schemes, emulsions and sound track formats evolved and can

This is an be found in film archives around the world. Yet today, more than 100 years after its U introduction, 35mm film is the shining example of a standardized and sustainable S issue that format that is widely adopted, globally interoperable, stable and well understood. requires The bottom line is that any system proposing to replace photochemical film N technology must meet or exceed film’s capabilities. While it is true that end-users E top-down benefit from new features and cost efficiencies that generally come with new products and technologies, the economic benefits of technological obsolescence accrue primarily S to the hardware manufacturers and software system developers. In exploring this examination, N digital dilemma, it becomes clear that if we allow the historical phenomenon of tech- enlightened nological obsolescence to repeat itself, we are tied either to continuously increasing O costs – or worse – the failure to save important assets. This is an issue that requires decision- top-down examination, enlightened decision-making and intra- and inter-industry C cooperation for the benefit of today’s content creators and tomorrow’s audiences. making, The Academy was founded to, among other things, represent the viewpoint of and intra- the actual creators of motion pictures and facilitate technological progress among the creative leadership of the motion picture industry. It is therefore the proper role of and inter- the Academy to spotlight this issue by bringing together the resources that produced this report, and to lead in the actions necessary to solve this dilemma. In addition to industry initiating the activities discussed earlier in this report, in the coming months the Academy will bring together studio decision-makers and technology resources, as well cooperation as other experts, to further define the requirements and issues in the archiving of and access to digital motion picture materials. These efforts are a start, but what is also for the needed is commitment by the primary stakeholders, and objective overview of the manufacturers and system designers, to produce cooperation, standards, and guaran- benefit of teed long-term access to created content. today's Only then will we have solved this Digital Dilemma for the benefit of all the players. content The place to start is here. The time to start is now. creators and tomorrow's audiences.

THE SCIENCE AND TECHNOLOGY COUNCIL / 56 57 / THE DIGITAL DILEMMA 2007 9 APPENDIX • Case Study Data

The following sections contain summary information from the two X

case studies discussed in this report. The subject productions were I captured either on film or HDCAM SR videotape and mastered at D 1920 x 1080 pixel count with 10-bit precision per color component. N This results in lower total byte counts than 2K/10 bit and 4K/16 bit E encoding. However, the number and type of elements identified in the case studies are believed to be representative of those generated by P both 2K and 4K productions. P A The Element Trees were derived from inventory data provided by the participating studios. The succeeding Case Study Data Tables contain the actual inventory data (with certain noted assumptions), using the identifying terms from the Element Trees.

THE SCIENCE AND TECHNOLOGY COUNCIL / 58 9 APPENDIX • Case Study Data continued

A.1 Generic Element Trees • Picture Element Hierarchy: Film Capture

0.0 0.2 0.4 0.6 0.8 1.0

Master Files Editing Original Camera Master Files for Textless System Files Negative DNegative

Screen Tests, B Roll, YCM 35mm 35mm Deleted Scenes Separation Digital Cinema DNegative DNegative Masters Distribution Master (dcdm) (Archival) Textless

35mm Telecine Dailies DNegative 35mm Production 35mm Interpositive Composite Answer Print

35mm 35mm 35mm Production Composite Conformed Interpositive Answer Print Foreign M&E Titles

35mm Production Distribution Master Internegative

35mm Check Print

SYMBOL KEY COLOR KEY

ARCHIVAL STORAGE

35MM DIGITAL VIDEOTAPE WORKING LIBRARY FILE FILM PERISHABLE ELEMENT

59 / THE DIGITAL DILEMMA 2007 9 APPENDIX • Case Study Data continued

A.1 Generic Element Trees • Picture Element Hierarchy: Data Capture

Original 0.0 0.2 Cloned 0.4 0.6 0.8 1.0 Production Footage Production Footage

Master Files Master Files Master Editing for Final Edited Files System Files for Textless Picture, Out-Takes Digital Negative and Trims

35mm Distribution Master Digital Cinema YCM DNegative 35mm Distribution Separation (Archival) DNegative Master (DCDM) Masters Textless

35mm DNegative 35mm Production 35mm Interpositive 35mm Conformed Production 35mm Foreign M&E Interpositive Composite Titles Answer Print

35mm Production Internegative

Combined Screen Tests, B Roll Continuity/Master Deleted Scenes English Subtitle Spotting List 35mm Check Print

SYMBOL KEY COLOR KEY

ARCHIVAL STORAGE

35MM DIGITAL VIDEOTAPE WORKING LIBRARY FILE FILM

PERISHABLE ELEMENT

THE SCIENCE AND TECHNOLOGY COUNCIL / 60 9 APPENDIX • Case Study Data continued

A.1 Generic Element Trees • Sound Element Hierarchy: Film or Data Capture

0.0 0.2 0.4 0.6 0.8 1.0

LTO2 Copies of All Materials

Original Production Orchestra/ Sound/Field Pre-Dubs Scoring Sessions Sound Recordings

DVD-R Copies of All Materials

Foreign Dialogue Dialogue Stem Music Stem Effects Stem Stems Discrete/Stereo Discrete/Stereo Discrete/Stereo x 11 Languages 6-Track, 5.1 6-Track, 5.1 6-Track, 5.1

M&E Stem Foreign Language Discrete/Stereo Print Master Domestic 6-Track, 5.1 5.1 Domestic Stereo Dolby SR/SRD/ Print Master LT/RT SDDS/DTS OSTN 5.1

Uncompressed 5.1 M&E Foreign Language Fully Filled Efx Stem LT/RT Print Master (M&E Minus Music) Dolby Digital 5.1 Digital Cinema LT/RT Lt/Rt Version 5.1

SYMBOL KEY COLOR KEY

ARCHIVAL STORAGE DIGITAL 35MM FILE FILM WORKING LIBRARY

61 / THE DIGITAL DILEMMA 2007 9 APPENDIX • Case Study Data continued

A.2 Case Study Data Tables X

Table A-1 lists the delivered picture elements identified in the film capture case study plus the medium I of storage, the number of items per element and the estimated file size if digital storage is used. D N NUMBER FILE SIZE STORAGE MEDIUM ELEMENT E OF ITEMS in TERABYTES P 35mm Film 35mm Digitally 178 Cans or Cartons NA

Created Negatives P 35mm Answer Print 35mm Production IP A 35mm Production IN 35mm Check Print 35mm Textless DNegative 35mm Textless IP 35mm Textless Answer Print 35mm Foreign Language Main and Ends Negative 35mm YCM Separation Masters 35mm Original Camera Negative 35mm Trims and Outs

LTO2 Data Tape 1920x1080 Master Files 1920x1080 Master Files for 15 LTO2 Data Tapes 3 TB Textless DNegative

DVD-R Optical Disk Editing System Files 1 Disk .005 TB

HDCAM SR Videotape Telecine Dailies 486 Tapes 173 TB1

D5 Videotape Distribution Master 9 Tapes .202 TB1

Table A-1 – Delivered Film Capture Picture Elements

1 Calculated.

THE SCIENCE AND TECHNOLOGY COUNCIL / 62 A P P E N D I X 6 3 /

T H E D 2 1 9 Table A.2 magnetic medium I C I t G e a I m DVCAM D5 HDCAM Magnetic 35mm l T c STORAGE APPENDIX A u s

l L n a Videotape o t D e t A-2 I Case

d L y . E e Film M of t

hard d M Videotape SR e lists Hard storage, A l i v

MEDIUM 2 e Videotape Study 0 r e drives 0 the d 7

Drives t

a delivered r the c h Table or Data i v e average . data • Distribution Screen Cloned Original Editing Outtakes 1920x1080 Textless 1920x1080 1920x1080 Language 35mm Masters 35mm Answer 35mm 35mm DNegative 35mm 35mm 35mm 35mm 35mm Created 35mm Editing Case A-2 Tables picture tapes. number – Conformed YCM Textless Textless Textless Check Production Production Answer Digitally Tests/B-Neg System Files Delivered Production Print Production Negatives ELEMENT DNegative and elements Mains continued Study Master Master Master Master Separation of Print Trims Print Project IP items and IN IP Foreign Footage/ Data Files Files Files Footage identified Ends Files per Data for for Capture element in 0 129 728 5,347 42 the Tapes continued Hard Picture and Cans Tapes data Tapes O NUMBER 1 F the Drives or ITEMS capture Cartons Elements estimated case file study size 24 NA 3,257 10.7 in plus TERABYTES FILE TB if TB 2 stored TB the SIZE 2 on 9 APPENDIX • Case Study Data continued

A.2 Case Study Data Tables continued X I Table A-3 lists the delivered sound elements for both film and data capture, the medium of storage, the average number of items per element, and the estimated file size if stored on magnetic hard D disks or data tape. The delivered sound element types for both case studies were identical, although as expected, the quantities differed between productions because of variances in production and N postproduction practices. E P

NUMBER NUMBER FILE SIZE P STORAGE ELEMENT OF ITEMS OF ITEMS in

MEDIUM A (Data Capture) (Film Capture) TERABYTES

35mm Film Optical Soundtrack 34 Cans 27 Cans NA Negative (OSTN) or Cartons or Cartons

DVD-R 6-Track Dolby Digital 71 DVD-R 371 DVD-R .83 TB Optical Disk Digital Cinema Version (Data Capture) LT/RT Musefx .42 TB 5.1 Efx Stems (Film Capture) Foreign Language Dialogue Stems Foreign Language Print Master (5.1) Foreign Language Print Master (LT/RT)

LTO2 Data Dolby LT/RT Print Master 13 LTO2 01 .004 TB Tape Dolby 5.1 Print Master Data Tapes (Data Capture) 6-Track Dolby Digital

Magnetic Music, Dialogue and 43 Magnetic 3 Magnetic 2.6 TB Hard Drive Effects Stems Hard Drives Hard Drives (Data Capture) Domestic LT/RT .73 TB Domestic 5.1 Print Master (Film Capture) 5.1 Musefx Orchestra/Scoring Sessions

Table A-3 – Delivered Sound Elements 1 Items not yet delivered to archive.

THE SCIENCE AND TECHNOLOGY COUNCIL / 64 A P P E N D I X 6 5 /

T H E D 9 is stream As in Table A.2 studio and I G Working Mixed Archival Working I a T the stated APPENDIX A broad STORAGE sound L D data A-4 I because Case distribution L E Archival/ M previously, M term Library Library elements lists capture A

2 Study 0 CATGEGORY of 0 the for 7

differing elements elements, production. materials from “archival” Data Sound • Table the practices. Case Tables that can the film is Elements A-4 number are be defined capture Spotting Continuity/English DVD-R 35mm and 35mm 35mm 35mm 35mm 35mm D5 for LTO DVD-R HDCAM B-Roll, HDCAM 35mm Negative 35mm 35mm (from 35mm 35mm for LTO 35mm created

– generally continued Items Study Textless Digital – Storage Ends – – Distribution PICTURE Digital 1920x1080 1920x1080 Deleted Digital YCM Foreign Production Check Production Digital Composite Trims Answer Production Original as of – – from production. List SR SR Combined Editing in over storage items Negative Digital kept bold – – Separation Negative) and Categories Print Negative Negative Dailies Screen a Language a Scenes Print, ELEMENT Camera Data Film and System 100-year Master on Outs exist Master Master of IP, Interpositive Answer Internegative Subtitle Negative the hand Textless Tests, the Textless

Capture This Textless only Masters Files Mains master storage Files Files Print continued for timeframe, for breakdown in Picture distribution the Production elements category film Master Foreign Master Foreign Foreign 5.1 Musefx 6-track, Musefx Dolby Domestic D Dolby Music Effects Dialogue Orchestra/Scoring Pre-Dubs Original Sound Copies LTO NA o and l Fully and b may y capture SOUND

or S from Digital Stereo assignment Stems Stems R “working Materials LT/RT 5.1 purposes. Language Language Language LT/RT Stem of 5.1 DVD-R Production Filled / vary Stems S Print all R D which production LT/RT LT/RT Working Discrete ELEMENT / Efx depending S Master D Stem D Print Print Dialogue library” Sessions for S all Sound / D Library down- picture T S and

on O storage Stems S T the not N 9 APPENDIX • Case Study Data continued

A.2 Case Study Data Tables continued X

Table A-5 lists the elements, the number of items and the storage category assignment for picture I

and sound elements from the data capture production. Again, this may vary depending on the studio D because of varying practices. Items in bold exist only in the data capture production. N E STORAGE CATGEGORY PICTURE ELEMENT SOUND ELEMENT P

Archival 35mm YCM Separation Masters LTO or DVD-R P Hard Drives – 1920x1080 Copies of all Working Library Master Files for Digital Negative Sound Materials A 35mm Digital Negative 35mm Composite Answer Print (from Digital Negative) 35mm DNegative Textless

Hard Drives – 1920x1080 Mixed Archival/Working Master Files for Final Edited NA Library Picture, Outtakes and Trims HDCAM SR – Screen Tests, B-Roll, Deleted Scenes

Working Library HDCAM SR - Original Original Production Sound Production Footage Pre-Dubs HDCAM SR– Cloned Production Footage Orchestra/Scoring Sessions DVD-R – Editing System Files Dialogue Stems LTO – Master Files for Effects Stems Textless Digital Negative Music Stems HDCAM SR – Distribution Dolby Stereo LT/RT Master Dolby SR/SRD/SDDS/DTS OSTN 35mm Production Interpositive Domestic Print Master 35mm Production Internegative Dolby Digital LT/RT 35mm Check Print Musefx Stem Discrete 6-track, 5.1 35mm Production IP, Textless Musefx LT/RT 35mm Foreign Language 5.1 Fully Filled Efx Stem Mains and Ends Foreign Language Dialogue Stems Foreign Language Print Master 5.1 Foreign Language Print Master LT/RT

Table A-5 – Storage Categories for Picture and Sound Elements from a Data Capture Production

THE SCIENCE AND TECHNOLOGY COUNCIL / 66 A P P E N D I X 6 7 /

T H E D of not ingest Initial • • • • As Baseline A.2 9 I G $1,500 $500 $1.80 $4.80 I assets T stated APPENDIX A available. L D costs inspection Case I per L per per E being M per previously, M Storage terabyte included physical physical A terabyte

Study 2

Nonetheless, 0 stored. 0 7 and

per item item the in Costs Data per access year the baseline • year per per baseline these Table costs Case for year year for near-line storage costs are online for for s digital continued not working archival Study should costs magnetic included data storage storage used be tape library considered Data hard costs in storage for the this drive because baseline (single study continued when storage reliable are: film copy) considering (single storage information copy) costs, the type nor for and the are quantity access latter is or 9 APPENDIX • Case Study Data continued

A.2 Case Study Data Tables continued X

Table A-6 lists the estimated annual cost of storing the delivered picture elements from the film I

capture production. This includes digital elements that are created during postproduction and stored D on LTO2 data tape, HDCAM SR (or equivalent) videotape, and DVD-R optical disk. Items in bold are stored in archival conditions and are so noted. N E Today, the practice in Hollywood is to store digital media as physical items in either archival or working library conditions. Given the special handling requirements of digital data, and the P associated costs, the following table calculates the estimated cost of storing the digital elements P separately as data, on data tape, in a fully managed environment consistent with the archival intent. A

ANNUAL STORAGE ANNUAL FULLY STORAGE COST OF DELIVERED MANAGED MEDIUM ELEMENT ITEMS STORAGE COST IF STORED ON DATA TAPE

35mm Film 35mm Digital Negative $1,5061 NA 35mm Answer Print (Archival) 35mm Production IP $290 (Working Library) 35mm Production IN 35mm Check Print 35mm Textless Digital Negative 35mm Textless IP 35mm Textless Answer Print 35mm Foreign Mains and Ends Negative 35mm YCM Separation Masters 35mm Original Camera Negative 35mm Trims and Outs

LTO2 Data Tape 1920x1080 Master Files for $72 (Archival) $1,465 (Archival) Digital Negative 1920x1080 Master Files for Textless Digital Negative

DVD-R Editing Files $2 (Working Library) $2 (Archival or Optical Disk Working Library)

HDCAM SR Telecine Dailies $2,333 (Archival) $86,498 (Archival) Videotape

D5 Videotape Distribution Master $16 (Working Library) $96 (Working Library)

Table A-6 – Estimated Annual Cost of Element Storage – Film Capture

1 Includes amortized cost of YCM separation master manufacture, which is $800 per year.

THE SCIENCE AND TECHNOLOGY COUNCIL / 68 A P P E N D I X 6 9 /

T H E D represent The Table A.2 in postproduction, capture 1 9 I I n G DVCAM Videotape HDCAM Hard Magnetic 35mm c I bold l T u APPENDIX A d estimated L e STORAGE s D MEDIUM

A-7 Case Drives a I L m production. are Film E o M the r SR t M lists i stored z A Table e

d use Study 2

0 c cost o the 0 s and 7 t of

o in f A-7 estimated of

Y uncompressed

stored C This archival Data M storing S C Original Trims Master Complete Master Complete for 1920x1080 Masters 35mm Print 35mm 35mm 35mm 35mm 35mm 35mm 35mm 35mm Editing

- s c e l o • Estimated r p includes e n a Digital e r e on a Tables Case n d t annual i

conditions a

o YCM Textless Textless Answer Digital Textless Check Production Production T System P Files, Files n magnetic Production e single r

m o s ELEMENT ELEMENT 1920x1080 1920x1080 t d Negative a s s u Master digital / both t Separation for c e B Outtakes Print r t Negative cost Annual

- i

m Project Print copy o N continued IP Answer Negative Study Textless n a e n

born-digital IN IP F g hard and u of o

f data Footage Files a o c of t t storing u a Files are and r Cost g born-digital drive e recorders e ,

w / so h i c Data h of noted. or the

i s elements

$ Element HDCAM 8 COST $124 $1,102 (Working $100 (Working $1,170 $64 (Working delivered ANNUAL 0 now 0

p materials e (Archival) r continued

OF y 1 in (Archival) e ITEMS and a Storage r use. . DELIVERED Library) Library) Library) SR STORAGE picture digital on (or data equivalent) – elements elements Data tape (Working $11,245 (Working $1,629,128 $5,127 NA Capture STORAGE ANNUAL is IF from DATA MANAGED created also STORED videotape. (Archival) Library) Library) calculated the TAPE FULLY COST during data ON

Items to 9 APPENDIX • Case Study Data continued

A.2 Case Study Data Tables continued X I As stated earlier, the trend in the audio domain, where all delivered elements are born digital, is to

copy all master audio files to DVD-R and LTO3 and geographically separate these materials for D protection. The master files remain in the working library on magnetic hard drives for instant access. This approach is believed to be the most comprehensive attempt to create an archival process N

around motion picture sound elements, provided that the process includes a data integrity check and E migration plan that outlives economic and labor factors. P

Table A-8 lists the estimated annual cost of storing the delivered sound elements from the case study P productions. The estimated annual cost of storing the sound elements on magnetic hard drives is also included to reflect current practice at certain studios. A

ANNUAL ANNUAL ANNUAL FULLY MANAGED FULLY MANAGED STORAGE ELEMENTSTORAGE COST STORAGE COST STORAGE COST MEDIUM ELEMENT OF DELIVERED IF STORED ON IF STORED ON ITEMS DATA TAPE HARD DRIVE 35mm Film OSTN $61 NA NA (Data Capture: Working Library) $49 (Film Capture: Working Library

DVD-R Multi-Channel $144 $414 $1,242 Optical Disk Master Stems (Data Capture: (Data Capture: (Data Capture: 5.1 Domestic Archival) Archival) Archival) Printmaster $668 $212 $635 Domestic LT/RT (Film Capture: (Film Capture: (Film Capture: Working Library) Working Library) Working Library) 6-Track Dolby Digital Digital Cinema Version 5.1 Musefx LT/RT Musefx Production Sound Pre-Dubs 5.1 Efx Stem Foreign Dialogue Stems Foreign Language Print Master (5.1) Foreign Language Print Master (LT/RT) Copies for Geographic Separation

LTO2 Data Dolby LT/RT, 5.1, $4 (Data Capture: $2 (Data Capture: $5 (Data Capture: Tape 6 Track Print Masters Working Library) Working Library) Working Library)

Magnetic Music, Dialogue and $79 $1,222 $3,667 Hard Drives Effects Stems (Data Capture: (Data Capture: (Data Capture: Domestic LT/RT, Working Library) Working Library) Working Library) 5.1 Print Master $5 $366 $1,099 Musefx (Film Capture: (Film Capture: (Film Capture: Working Library) Working Library) Working Library) Orchestra/ Scoring Sessions

Table A-8 – Estimated Annual Cost of Sound Element Storage

THE SCIENCE AND TECHNOLOGY COUNCIL / 70 R E F E R E N C E S 7 1 /

T H E D Kushner, Kirsner, Kienzle, Kienzle, Karagosian, Hurwitz, Lorie, Holmes Hogan, Herman, Global Galloway, Digital DCinema Baker, Anderson, 10 I G I http://www.kb.nl/hrd/dd/dd_onderzoek/reports/4-uvc.pdf. Dec. Preservation http://www.wired.com/wired/archive/15.01/nasa.html?pg=1&topic=nasa&topic_set=. Wired http://www.hollywoodreporter.com/hr/search/article_display.jsp?vnu_content_id=1002688109. The TV TV http://www.mkpe.com/publications/digital_cinema/misc/multichannel.php. MPKE http://www.videography.com/articles/article_14970.shtml. Videography http://www.ncbi.nlm.nih.gov/sites/entrez?db=PubMed. Catheterization http://www.photonics.com/content/spectra/2006/March/business/68479.aspx. http://www.gstinc.com/white/report-2004.pdf. GST http://www.hollywoodreporter.com/hr/content_display/news/e3if727c623f03c782b8ad564866c828796. The http://medical.nema.org/dicom/2007/. National http://www.dcinematoday.com/dcdb/DCinemas.aspx?app=1&cnty=49&dci=1. h New Proceedings http://www.wired.com/wired/archive/12.10/tail.html. T ttp://www.lockss.org/locksswiki/files/3/30/Eurosys2006.pdf. A L Raymond. Technology Technology References D Mary, Hollywood Hollywood Storage Imaging Research I 2002: York, Hank. L Jr., Scott. Claudia. Claudia. E Matt. Consulting Robert. 15.01 David. M Today. Stephen. Chris. M Electrical David Michael. A et NY,

2 “Film

1-47. Jan.

“Digital of Technologies 0 Study “Crime and al. 15

“One

and “The

Telephone “Breaking “Taking 7 221-234. “The the “Digital R., Reporter Reporter 11 11 Mar. Cardiovascular

“A 2007:

Communications

“Studios Fades LLC July July Amsterdam, Merrill Manufacturers Engineering

2006 Report “Multichannel Fresh UVC: Giant Long Scenes Cinematography: 2007. Sept. 2007: 2007:

the Installations.” 1-3. Out.”

13 10 22 Eurosys News A. Look

Tail.” Screwup interview A Series Are Inc. Tapeless June 2003.

July

Jan. and 19 CondeNet Wondrow Method

12. 1

Photonics

Hunting & Group July Netherlands. at “GST’s . Diagnosis

2007. Wired 2007 Compression . 2006. 12. Number Conference, the . Film Association

Disc 4 in for 2007 with Archive Sept. for Medicine. Reliability

2004: Sept.

Mankind.” Annual

Today.” The and Inc. 12.10

Nielsen the at Nielsen Preserving Laurin Spectra 20.1 4. 11: 2007 Next

Joel Big 2007. 15

Oct. Challenge.” International

1-15. 18 The April Schemes: PS State July 1990: Pixel.”

Business Herr.

E. of Business “Part Aug. Big Mar. 3.1-2007:

2004. Gray.

Migration

Long-Term Lake WHM 2007 Digital 18-21, of Property.”

1-4.

2007 1

1: 2006. Tape Feb. Forest,

Introduction “Isn’t Media, CondeNet The

Media,

Cinema 2006: PubMed Business Documents: 1-21.

2007. Report to

File-Based Laurin Digital it

Tapeless CA. Inc. Time

Inc. Rosslyn, Leuven, Consulting,

Abstract. Oakdale,

Inc. Machines for 7

Publishing

16 Storage.” and 19 Sept. to 2004.”

July

Proof July Workflow 17 News Abandon Belgium. Overview.” VA. Aug. 2007 2007 MN, 2007

22 LLC. of Corporation

Production.” Co., 2007 Concept.” Aug. Aug. USA.

Cine 21 for ACM Inc.

Sept. 2007 David 2007

Film?” 1 Press, Oct. and

2007. IBM/KB

Fincher’s 2007. Koninklijke

Long-Term

Zodiac.” Bibliotheek.

10 References continued

Lyman, Peter and Hal R. Varian. “How Much Information? 2003.” School of Information Management and Systems 30 Oct. 2003: 1-112. University of California, Berkeley. 3 Aug. 2007 S

http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/printable_report.pdf. E

Mellor, Chris. “Super-tape Market Edges Toward Monopoly.” Techworld.com. C 30 Jun. 2005. International Data Group. 17 Sept. 2007 N http://www.techworld.com/storage/news/index.cfm?newsid=3951. E Moore, Richard L. et al. “Disk and Tape Storage Cost Models.” San Diego Supercomputer Center, University of California, San Diego, CA: La Jolla, 2007. R E Motion Picture Association of America. “2006 U.S. Theatrical Market Statistics: Worldwide Market Research & Analysis.” F mpaa.org 2006: 1-24. 24 Aug. 2007

http://www.mpaa.org/2006-US-Theatrical-Market-Statistics-Report.pdf. E

National Research Council of the National Academies. Building an Electronic Records Archive at the National Archives R and Records Administration: Recommendations for a Long-Term Strategy. Ed. Robert F. Sproull and Jon Eisenberg. Washington, D.C.: The National Academies Press, 2005.

Overfelt, Maggie. “Saving the Cineplex.” CNNMoney.com 19 July 2007. DCinema Today. 22 Aug. 2007 http://www.dcinematoday.com/dc/extURLs.aspx?ID=290.

Palm, Jonas. “The Digital Black Hole.” Swedish National Archives 2006: 1-14. Stockholm, Sweden. 19 July 2007 http://www.tape-online.net/docs/Palm_Black_Hole.pdf.

Pinheiro, Eduardo, Wolf-Dietrich Weber and Luiz André Barroso. “Failure Trends in a Large Disk Drive Population.” Paper presented at the 5th USENIX Conference on File and Storage Technologies – Paper, February 12-13, 2007: 1-13 San Jose, California. 8 Aug. 2007 http://209.85.163.132/papers/disk_failures.pdf.

Rosenthal, David S., et al. “Requirements for Digital Preservation Systems.” D-Lib Magazine 11, (11) 2005. 19 June 2007 http://www.dlib.org/dlib/november05/rosenthal/11rosenthal.html.

Scherzer, Barbara. “Cookson Puts Focus on Classics.” Variety 15 Apr. 2007. Reed Business Information. 20 Aug. 2007 http://www.variety.com/index.asp?layout=print_story&articleid=VR1117963134&categoryid=2222.

Schroeder, Bianca and Garth A. Gibson. “Disk Failures in the Real World: What does an MTTF of 1,000,000 Hours Mean to You?” Paper presented at the 5th USENIX Conference on File and Storage Technologies – Paper, February 12-13, 2007: San Jose, California. 26 Mar. 2007 http://www.usenix.org/events/fast07/tech/schroeder/schroeder_html/index.html.

Science Applications International Corporation. “Architecture and Technology Program: Offline Archive Media Trade Study.” United States Geological Survey. 2006: 1-34. USA. Aug. 2007 http://wgiss.ceos.org/archive/archive.pdf/FY06%20Media%20Trade%20Study.pdf.

Solomon, Madi. Presentation. Digital Asset Symposium. Association of Moving Image Archivists. Pickford Center for Motion Picture Study, Linwood Dunn Theater, Hollywood, CA, 11 May 2007.

Taylor, Jeff and Tom Regal. Presentation. Digital Asset Symposium. Association of Moving Image Archivists. Pickford Center for Motion Picture Study, Linwood Dunn Theater, Hollywood, CA, 11 May 2007.

THE SCIENCE AND TECHNOLOGY COUNCIL / 72 R E F E R E N C E S 7 3 /

T H E D United United U Storage Lots Linear i Film Consultative Advanced Bibliography The Van United United 10 I Rule G n http://www.sdsc.edu/srb/index.php/Main_Page. http://www.lockss.org/lockss/Home. http://www.lto-technology.com/technology/uroad.php?section=0&subsec=uroad. Corporation, http://irods.sdsc.edu/index.php/Main_Page. T http://public.ccsds.org/publications/archive/650x0b1.pdf. CCSDS http://www.adlnet.gov. http://www.thexlab.com/faqs/opticalmedialongevity.html. http://www.imation.com/government/nml/pdfs/AP_NMLdoc_magtape_S_H.pdf. Commission http://nsf.gov/pubs/2005/nsb0540/start.html. Education http://www.loc.gov/film/tvstudy.html. Volume http://www.loc.gov/film/implemen.html. I and http://www.digitalpreservation.gov/library. I i T mplementation t h A e X Bogart, of Restoration e L d Oriented

References Lab.

D Tape-Open F Storage S States. States. States. States. Copies I i Resource t L r a E s M t t 650.-B-1. Distributed 1:

P M “Optical s John r

in A Committee . Report. o

j 2 L Keep Infrastructure National Library Library Library e the on Data and 0 i c b 0 & t W. r 7 Broker. ’ a Preservation s Technology.

Strategy. Conservation 21st r

Quantum. F Blue y Stuff Media Systems. C.

Oct. n o Learning. of of of a f

Century

l “Magnetic Science C

for Book, R Congress. Congress. Congress.

Safe. o June e 1997. n Longevity.” p Space

g o 22 RFP r continued r

t Aug. 19

Issue

2007. s , and Sept. Ultrium Jan.

NSB-05-40 Board.

s C 2007. Strategies. 28 . LOC-NAVCC-06-01. Sept. o

Data Tape D n

Access; 2007. Aug. National National National 2004. c i 1. 2007. g l

u San i

s 2007

Jan. 2007. Office a National Roadmap. Systems. i Storage o l 2007

n P

Diego

s San r 22

2002:

St. e 2005:

Stanford 2004. – s Film Film Audio-Visual

G e of 18 Aug. Paul, Diego r u v Science and the i Supercomputer a Sept.

d Preservation Preservation “Reference

e 1-148. 2007.

i 1-92. European o l 2007 i MN: Under n n Handling: University. Supercomputer e .

2007 Washington,

s W

– Foundation 3

Washington,

National h Hewlett-Packard, R Conservation Aug. a Secretary e t c

Model Film t o h Board. Board. m

A e 2007

Center. m

27 L Guide Heritage i e Media b n July r D.C.: for Long-lived d a of Center.

r a Television/Video National D.C. y t an Defense

i for Center 2007 i o

s 27

Laboratory, n

Library D Open s on Libraries

International . July 8 o

i 27 n Aug. r the u g Film Data – s . for 2007 Archival

July s Culpeper,

of T e Threshold 2007 l h s Personnel Preservation Congress, e and Collections : 2007

June

L G i b a Preservation Archives.” Business r b Information a r 1995: r i Virginia: e y l Of

l o e and 2006. f

C The C Enabling

l 1-48. Plan: o a Machines Readiness. e n s

g Digital Washington, Computer , r Study:

System R e An

s o 17 s y . a Research

Aug.

l 2

Era, F 9

i l

(OAIS).” m

A 28 Server 2007 u

A g Aug. DC: r .

c 2 and h 0 i v 0

e 2007

7 .

11 Acknowledgments

This report would not have been possible without the support of the members of the Science and S Technology Council’s Digital Motion Picture Archival Project Committee, the consultants who performed the research and interviews, and the many experts from the motion picture industry and T other sectors who generously provided information and insight on the Digital Dilemma. N

This report, in its various draft forms, was reviewed by several of the individuals listed below to assist the E Council with delivering a final version that meets Academy standards for overall quality and accuracy. The reviewers’ comments and identities will remain confidential to protect the peer review process. M

The reviewers have not seen the final version of this report, nor do they necessarily support the report’s G conclusions and recommendations. The Academy of Motion Picture Arts and Sciences is wholly responsible for the contents of the final report. D E Science and Technology Council The Council would also like to Tim Smith thank all of the individuals, Greg Thagard L Ray Feeney, Co-chair studios and other organizations Ken Thibodeau

Bill Taylor, Co-chair W that participated in the creation Rick Utley Andy Maltz, Director of this report: Rudy Valdez Chris Wood O Digital Motion Picture Archival

David Arctur Bob Yacenda N Project Committee Mary Baker

Phil Feiner, Chair Art Beckman Amazon.com, Inc. K Milt Shefter, Report Project Lead Peter Boer Ascent Media Group C Steven Anastasi* Ken Boettcher The Cleveland Clinic Foundation

Craig Barron Richard Buchanan Eastman Kodak A Elizabeth Cohen* Laura Campbell Hess Corporation Grover Crisp Dave Cavena HP Labs Clay Davis* Robert Cecil Imation Corporation Carl Fleischhauer Chris Cookson InPhase Technologies Rob Hummel* Brian Davis Laser Pacific Media Corporation Brad Hunt* Jeff Davis Library of Congress Tim Kittleson John Faundeen Microsoft Corporation Phil Lelyveld Michael Friend National Archives and Records Edrolfo Leones Edward Grogan Administration Gregory Lukow Jim Hannafin National Security Agency Gary Morse Wade Hannibal NBC/Universal Studios Michael O’Brien Margaret Hedstrom Office of the Director of Michael Pogorzelski Robert Herman National Intelligence Eddie Richmond Scott Kelly Open Geospatial Consortium Garrett Smith* Glenn Kennel Ovation Data Services, Inc. Madi Solomon Mark Kimball Paramount Pictures Charles Swartz* (in memoriam) Bob Lambert Pro-tek Media Jim Lindner Preservation Services Laurin Herr, Consultant & Lead Interviewer Stephen Long Quantum Von Johnson, Case Studies Consultant Howard Lukk San Diego Supercomputing *Report Project Committee Member Tad Marburg Center Brian McKay Sony Pictures Entertainment Richard Moore Sun Microsystems, Inc. Liz Murphy 20th Century Fox Film Bob O’Neil Corporation Mark Ostlund United States Geological Survey, Peter Otto Center for Earth Resource Rick Picton Observation and Science Jerry Pierce University of Michigan Brian Saunders The Walt Disney Company Greg Servos Warner Bros.

THE SCIENCE AND TECHNOLOGY COUNCIL / 74 ABOUT THE SCIENCE AND TECHNOLOGY COUNCIL

The mission of the Science and Technology Council of the Academy of Motion Picture Arts and Sciences is:

• To advance the science of motion pictures and foster cooperation for technological progress in support of the art • To sponsor publications and foster educational activities that facilitate understanding of historical and new developments both within the industry and for the wider public audience • To preserve the history of the science and technology of motion pictures • To provide a forum and common meeting ground for the exchange of information and to promote cooperation among divergent technological interests, with the objective of increasing the quality of the theatrical motion picture experience

For more information on the Science and Technology Council, visit http://www.oscars.org/council.

THE DIGITAL DILEMMA: STRATEGIC ISSUES IN ARCHIVING AND ACCESSING DIGITAL MOTION PICTURE MATERIALS