SET OF METHODOLOGIES FOR ARCHIVE FILM DIGITIZATION AND RESTORATION WITH EXAMPLES OF THEIR APPLICATION IN ORWO REGION Karel Fliegel a, Stanislav Vítek a, Petr Páta a, Miloslav Novák b, Jiří Myslík b, Josef Pecák b, and Marek Jícha b a Czech Technical University in Prague, and b Film and TV School of Academy of Performing Arts in Prague, Czech Republic

ABSTRACT PERCEIVED DIFFERENCES IN COLOR AND LIGHT TONALITY APPEARANCE I Set of verified methodologies for archive films’ restoration and digitization. I Assisting tool for measuring and monitoring the already digitized image sources. I Suitable especially for nonstandard laboratory or creative techniques in so-called ORWO region. I Displayed using a reference digital projector with split screen (DFRRP vs. DRA). I Umbrella of the techniques formed by Digitally Restored Authorizate (DRA) methodology. I Can be also used to assess the differences in respect to the analog film (RRP vs. DFRRP). I Aiming the appearance of the digitized film as close as possible to the original author’s concept. I Procedure based on CIEDE2000 color difference formula as a guidance to assess the impact of I Tools for objective assessment of perceived differences in the outcomes of the color grading process. perceived differences in color and light tonality in the projected images. I Techniques for evaluation of appearance match among various versions of the digitized film. FIGURE 4

INTRODUCTION AND PROBLEM DESCRIPTION Film projector The motivation comes from the need to establish methodologies and proper technical procedures to Image achieve high-quality digitization and restoration of the films produced in the very particular ORWO region. files DCI DCI (XYZ) server projector Projection screen I ORWO region comes from ORiginal WOlfen movie stock manufacturer, Wolfen near Leipzig (former GDR). (XYZ) (DCI-P3) XYZ I Comprises countries and their film archives in Central and Eastern Europe (e.g. Czech Republic, Simulation Calibration RGB D-SLR Slovakia, Poland, Hungary, Romania, Bulgaria, former Yugoslavia and Baltic countries, etc.). (DCI-P3) (RGB→XYZ) (RAW RGB) XYZ XYZ I Specific laboratory and creative techniques in film production mainly throughout the years 1953 to 1998. (3) 2D imaging colorimeter (2)

XYZ XYZ Spectro- I Combination of Eastmancolor OCN (Original Camera Negative) and low-cost variable quality ORWO XYZ→Lab photometer positive film stock, see Figure 1. Lab (1) I Release prints (RP) for distribution printed from OCN and not from the intermediate (IM) films. Some feature films in Academy format (aspect ratio 1:1.37) shot on Agfacolor OCN, see Figure 2. I General principle of the three techniques for derivation of colorimetric screen data in projected cinematographic images. Creative techniques were modified, e.g. altered lighting setup, facial makeup, colors of costumes and I (1) CIE XYZ sample measurement using spectrophotometer. (2) Measurement of 2D distribution of CIE XYZ tristimulus props, to achieve the required light tonality artistic concept tuned to ORWO RP. values using 2D imaging colorimeter. (3) CIE XYZ tristimulus values calculation using image data files.

FIGURE 1 Example of specific color characteristics of ORWO and FIGURE 5

Kodak Eastmancolor release print (RP) before and after 24 grading. 22 Example of color differences film prints analysis using ∆E00 20 18 CIEDE2000 color difference map. 16 Frame of “The Firemen’s Ball”, an important piece of 14 00 12 DE The overall value ∆E¯ 00 as an objective measure of color Czechoslovak cinematography produced in 1967, nominated for 10 the Best Foreign Language Film at the 41st Academy Awards. In 8 difference CIEDE2000 can be transformed into categories 6 this case ORWO prints were used for Central and Eastern 4 that indicate the subjective perception of the difference. 2 Europe cinema distribution and Eastmancolor prints were 0 24 Example for the test frames from Figure 1. used for theatrical distribution in USA and western Europe in 22 (a) ORWO scan vs. (b) Kodak scan vs. a slightly different censored versions and formats (Novák, 2014). 20 18 ORWO graded Kodak graded The distinct look has to be respected in digitization and 16 14 ¯ ¯ 00 ∆E00 = 15.9 ∆E00 = 9.0 12 restoration in order not to alter the author’s original concept. DE 10 (c) ORWO scan vs. (d) ORWO graded vs. 8 (a) RP ORWO scan (b) RP ORWO graded 6 Kodak scan Kodak graded 4 ¯ ¯ (c) RP Kodak scan (d) RP Kodak graded 2 ∆E00 = 12.5 ∆E00 = 8.2 0

24 Example for the test frames from Figure 2. 22 20 18 Agfacolor PC7 vs. Agfacolor PC7 vs. Orwocolor PC7 vs. 16 14 FIGURE 2 00 12 Orwocolor PC7 Agfa-Gevaert CP10 Agfa-Gevaert CP10 DE 10 Example comparing Agfa and ORWO color prints made in the 8 (a) ∆E¯ = 9.2 (b) ∆E¯ = 13.5 (c) ∆E¯ = 8.5 6 00 00 00 4 years 1956, 1976 and 2001 from the same OCN Agfacolor B, 2 0 with distinct look of Agfa and ORWO.

Two frames from the Czechoslovak film “Silver Wind” (1954). CIEDE2000 color difference calculated (CIE xyY measured with VFX-Consulting SpecTD spectrophotometer & densitometer) ANALYSIS OF IMAGE FILES BASED ON THEIR STATISTICAL PROPERTIES for the skin tones of the frame in the upper row: I Simple conformity assessment of color structure in levels of primary image colors R, G, B based on the

∆E00(56−76) = 16.3, ∆E00(56−01) = 14.0, ∆E00(76−01) = 4.1. analysis of histograms of image files in different digital formats. I Komogorov-Smirnov distance as a guideline for the rapid determination of compliance or (a) Agfacolor (b) Orwocolor (c) Agfa-Gevaert non-compliance in the luminance structure of the image files containing the same content. PC7 (1956) PC7 (1976) CP10 (2001) FIGURE 6 DIGITALLY RESTORED AUTHORIZATE (DRA) 10-4 (a) (e) 4.5 1 Example of Kolmogorov-Smirnov distance calculation. image x c (r ) The methodology of Digitally Restored Authorizate (DRA) defines a procedure and set of tools to achieve x k image y 0.9 c (r ) 4 y k Kolgomorov-Smirnov distance the audio and visual appearance of the digitized film as close as possible to the original author’s concept 0.8 maximum Histograms for two versions of the test frame (example for R 3.5 0.7 color channel), Cumulative distribution functions and (Fliegel, Krasula, et al., 2014; Fliegel, Vítek, et al., 2016; Jícha and Šofr, 2016). The steps and the simplified 3 0.6 Kolmogorov-Smirnov distance calculation. diagram describing the main idea of the DRA methodology can be seen in Figure 3. 2.5

0.5 CDF {-] CDF

PDF (-) PDF 2 0.4 I Degree of similarity of two image files may be determined 1.5 I Master restorer assigned, and group of experts formed. 0.3 as a correlation of probability functions (PDF, histogram) 1 I Selection of the film Reference Release Print (RRP), with the appearance closest to the original. 0.2 of R, G, and B color channel. I Key scenes critical for the film appearance selected. 0.5 0.1 I It is preferable to choose a statistical description using the 0 0 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4 cumulative distribution function CDF. I Key scenes samples scanned from the RRP in suitable resolution and proper quality. pixel intensity (-) 104 pixel intensity (-) 104 I CDF usually takes the form of a smooth monotonically 10-4 (b) (f) Accurate Digital Facsimile of RRP (DFRRP) created with the appearance equal to RRP. 1.8 I 1 increasing function. image x c (r ) x k image y 1.6 0.9 c (r ) I OCN is scanned in suitable resolution resulting in Digital Source Master (DSM). y k Robust tool for the determination of degree of match Kolgomorov-Smirnov distance I 0.8 I Color grading of the OCN in selected key scenes led by the master restorer and supported by the 1.4 maximum between the distribution functions is 0.7 1.2 Kolmogorov-Smirnov distance (Brunelli and Mich, 2001; group of experts performed, removing all unwanted color and light tonality drifts resulting in 0.6 1 Rubner, Tomasi, and Guibas, 2000). Educated Guess of Answer Print (EGAP). 0.5

PDF (-) PDF 0.8

0.4(-) CDF I DRA of the whole movie is created based on the EGAP by the digital colorist followed by the fine 0.6 Example of two versions of the test frame, before and after 0.3

tuning and approval from the master restorer and the expert group. The DRA is not a new version of 0.4 grading, taken from film “Capricious Summer” (1967). 0.2

the original work, but it is its original digital source. 0.2 0.1

0 0 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4 4 pixel intensity (-) 10 pixel intensity (-) 104

-4 (g) FIGURE 3 10 (c) FIGURE3.5 7 1 c (r ) image x x k image y 0.9 c (r ) y k 3 Kolgomorov-Smirnov distance 0.8 maximum Example of results obtained using DRA methodology. 2.5 0.7

0.6 2 I “Silver Wind” (1954), DFRRP from RP Agfacolor PC7 (left)

0.5 CDF (-) CDF

PDF (-) PDF and DRA from OCN Agfacolor B (right). RRP DFRRP DCDM IAP 1.5 0.4 I “Radúz and Mahulena” (1970), DFRRP from RP Orwocolor 0.3 1 PC7 (left) and DRA from OCN Eastmancolor 5251 (right). 0.2 0.5 OCN EGAP DRA MAP 0.1

0 0 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4 4 Simplified diagram of DRA methodology. Task comparison of film and digital restorer and the relationship between pixel intensity (-) 104 pixel intensity (-) 10 professional film and digital restorer of cinematographic works. (Jícha and Šofr, 2016) I Original sources: Original Camera Negative (OCN), Digitally Restored Authorizate (DRA), Master Archive Package (MAP). CONCLUSIONS I Copies: Reference Release Print (RRP), Digital Facsimile of Reference Release Print (DFRRP). I Tools supporting the work of digital film restorer, group of experts and digital colorist. I Digital dissemination masters: Digital Cinema Distribution Master (DCDM), Intermediate Access Package (IAP). I Crucial step of restoration based on estimation of DRA using Educated Guess of Answer Print (EGAP). I Intended for objective assessment of perceived differences between various outcomes of the restoration process. SUPPORTING TECHNICAL METHODOLOGIES I The techniques help to achieve high-quality digitization results, maintaining the film author’s concept. Supporting technical methodologies and techniques are needed in order to provide quantification of I Three sources for the technical-historical research have high importance, i.e. film stock identification and perceived differences among various outcomes of the color grading process. measurements, archival documents interpretation, and oral research.

I Assessment of perceived differences in color and light tonality appearance based on CIEDE2000 REFERENCES (Sharma, Wu, and Dalal, 2005) color difference formula applied to various types of evaluated images: (1) Brunelli, R. and R. Mich (2001). “Histograms analysis for image retrieval”. In: Pattern Recognition 34.8, 1625–1637. measurements in digital image files, (2) spectrophotometric analysis from the projection screen, and Fliegel, K., L. Krasula, et al. (2014). “System for objective assessment of image differences in digital cinema”. In: Proc. SPIE 9217. (3) colorimetric measurements based on imaging colorimeter (Fliegel, Vítek, et al., 2016), see Figure 4. Fliegel, K., S. Vítek, et al. (2016). “Evaluation of color grading impact in restoration process of archive films”. In: Proc. SPIE 9971. I Assessment of differences between image files based on statistical properties of histograms. Jícha, M. and J. Šofr (2016). “Digitální restaurování památek filmového umění. Metoda DRA”. In: Zprávy památkové péče 76.1. (in Czech), 76–90. Novák, M. (2014). “Rekonstrukce paměti (české) kinematografie v čase její digitalizace”. In: Film a kultúrna pamäť, 55–57. ACKNOWLEDGEMENTS Rubner, Y., C. Tomasi, and L. J. Guibas (2000). “The earth mover’s distance as a metric for image retrieval”. In: International journal This work was supported by the project NAKI DF13P01OVV006 “Methodics of digitizing of the national film of computer vision 40.2, 99–121. fund” of the Ministry of Culture of the Czech Republic (http://www.research-dra.com/). Sharma, G., W. C. Wu, and E. N. Dalal (2005). “The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations”. In: Color Research & Application 30.1, 21–30.