F. Cao, F. Guichard, H. Hornung, R. Teissières, An objective protocol for comparing the noise performance of silver halide film and digital sensor, Digital Photography VIII, Electronic Imaging 2012. Copyright 2012 Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited. http://dx.doi.org/10.1117/12.910113 An objective protocol for comparing the noise performance of silver halide film and digital sensor Frédéric Cao, Frédéric Guichard, Hervé Hornung, Régis Tessière DxO Labs, 3 Rue Nationale, 92100 Boulogne, France ABSTRACT Digital sensors have obviously invaded the photography mass market. However, some photographers with very high expectancy still use silver halide film. Are they only nostalgic reluctant to technology or is there more than meets the eye? The answer is not so easy if we remark that, at the end of the golden age, films were actually scanned before development. Nowadays film users have adopted digital technology and scan their film to take advantage from digital processing afterwards. Therefore, it is legitimate to evaluate silver halide film “with a digital eye”, with the assumption that processing can be applied as for a digital camera. The article will describe in details the operations we need to consider the film as a RAW digital sensor. In particular, we have to account for the film characteristic curve, the autocorrelation of the noise (related to film grain) and the sampling of the digital sensor (related to Bayer filter array). We also describe the protocol that was set, from shooting to scanning. We then present and interpret the results of sensor response, signal to noise ratio and dynamic range. Keywords: Digital photography, analog versus digital, image quality evaluation, signal to noise ratio, dynamic range, characteristic curve. 1. INTRODUCTION The comparison of the performance of films and digital sensors is an exciting topic, although passionate enough so that truth and facts are not always cleared from speculations. Film industry has designed methods to evaluate the performance of film from an objective point of view1,2,3,4,5 (making measurements in labs) and correlate these metrics with subjective image quality6. Unfortunately, many of these protocols are not relevant for digital images. Instead of trying to apply film evaluation protocols to digital sensors, we chose the reverse approach: since films have gone digital through scanning, we can apply (up to a few modifications) the same protocols as for sensors. This allows a fair comparison with more than 150 DSLR camera sensors tested on a website called www.dxomark.com. We first describe the image quality attributes that are, in our opinion, the most relevant for comparison between film and digital sensors. We shortly describe how they are characterized for films and sensors and explain why the results are not directly comparable. To make the comparison possible, we choose to apply the digital procedure to a digitized (i.e. scanned) film. We describe in details the operations we need to consider the film as a RAW digital sensor. We then present and interpret the results in terms of sensor response, signal to noise ratio and dynamic range. We finally conclude and give some perspectives of other comparisons. 2. EVALUATING IMAGE QUALITY 2.1 Image quality attributes When comparing image quality of films and digital cameras, several topics are usually emphasized: resolution, graininess, tone and color rendering or dynamic range. In this paper, we will only focus on the comparison of graininess and dynamic range. Graininess is the random variation of the photograph of a perfectly homogeneous zone. It can give a special look to picture, but it objectively destroys some part of the signal. Therefore, if the camera is thought of as an instrument to accurately capture a scene, graininess adds some noise to the image, that is, unpredictable variations that can be mistakenly interpreted as a variation of the scene, or can hide some subtle variations of the scene. Dynamic range is the ability of the camera to capture both shadows and highlights on a same picture. Any good photographer knows that backlit scenes should be avoided because the contrast is usually not pleasing to the eye. However, some scenes still contain very luminous and very dark parts in the same time. When the picture is shot, if the shadows are too dark or the highlights are burnt, there is no way to recover the lost information. Digital sensors are traditionally known to have a low dynamic range. Even though very recent cameras show outstanding progress, demanding photographers still complain about the necessary trade-off they sometimes have to make between shadow and highlights. Dynamic range also relates to exposure latitude and this also has a direct impact on image quality. 2.2 Film and sensor characteristics Before going in details on graininess and dynamic range, a preliminary step is necessary to understand the type of response of film and sensors to light. The response of film to light is characterized by the film density at all levels of exposure. The density describes the opacity of the film, and is the logarithm in base 10 of the film absorption (once developed). A high density corresponds to a large opacity, and therefore a very dark film. For a negative film, density is an increasing function of the film exposure (the film becomes darker when exposed to light). For a positive film, density is a decreasing function of the film exposure (the film becomes brighter). In both cases, the characteristic curve has a sigmoid shape. The response varies very slowly for very low light; it becomes larger for mid light; for high light, the film again responds more slowly to a given increase of exposure. A very important fact is that even though the response of the film varies very slowly in highlights, it still does vary, meaning that the film still records some information 3.5 Blue 3 2.5 Green 2 Red 1.5 Optical Density Optical 1 0.5 0 -4 -3 -2 -1 0 1 2 3 Log Exposure (lux.s) Figure 1. Characteristic curves of a film (in this case Kodak Portra 160NC). Note the very slow start in low exposure, the regular slope in the mid-tones, and very slow convergence to a maximal density in the highlights. Remark the scales on both axes are logarithmic. The response of a film is characterized by the curves of the three photosensitive layers (red, green and blue). The sensitivities of the different channels are tuned for a given type illuminant (indoor or outdoor). While the sensitivities of each layer are fixed, the ratio of the sensitivities are tuned for a given type of illuminant (indoor or daylight). For color negative films, the ratios can be adjusted so that a single film can give pleasing results in situations with mixed or multiple illuminants, for example, tungsten and daylight. If we now compare the response of a film with a RAW digital sensor for commercial cameras, we can observe some important differences. First, the response of a digital sensor is essentially linear. (Some sensors for very specific applications do not have a linear response, but they are not used for digital photography.) A given increase in exposure always yields the same increase of response. The second very noticeable difference is the behavior in highlights. At a given exposure, the sensor response suddenly stops increasing, no matter how high the exposure. The saturation is related to the full well capacity of the sensor. Beyond the saturation point, all the information of the scene is lost, except the fact that the exposure exceeds the sensor capacity. Also note that before saturation, a digital sensor can be easily used as an exposure measurement device since the response is linear with respect to exposure. In the shadows, the sensor usually has a positive response to a null input signal (pedestal). 120 100 80 60 40 Sensorresponse (%) 20 0 0.00 0.20 0.40 0.60 0.80 1.00 1.20 Exposure (lux.s) Figure 2. Characteristic curve of a digital sensor. The sensor usually has a non null response even when not exposed (pedestal); then the response increases linearly with the exposure before a sudden saturation corresponding to the pixel full well capacity. 2.3 Different measurements for films and sensors Any measurement is corrupted by noise and this will be the main topic of interest in this article. Photon shot noise is on the luminous signal itself, and therefore independent from the sensing device, film or digital sensor. Moreover, the structure of a film is very locally irregular, since the sensitive surface is made of big aggregates of silver halide crystals, whose size and shape are not perfectly constant. Larger grains are sensitive to lower exposure whereas smaller grains need more light to react. On a digital sensor, photo response of pixels may differ of about 1%. Sensor technologists would say that the photo response non uniformity (PRNU) is much larger on films than on sensors. The granularity of a film can be measured in different ways. A commonly used protocol leads to the RMS (root mean square) granularity8 which is the standard deviation of the density of the film integrated through a hole of 48µm diameter. Some studies in the film industry, at Eastman Kodak in particular, showed that some other metric could be derived, with a very good correlation with subjective image quality, as the Print Grain Index6 (PGI). However, this measurement is really adapted to film grains that are pretty large (typically 15µm).