EUROGRAPHICS 2002 STAR – State of The Art Report Tone Reproduction and Physically Based Spectral Rendering Kate Devlin1 Alan Chalmers1 Alexander Wilkie2 Werner Purgathofer2 1 – Department of Computer Science University of Bristol 2 – Institute of Computer Graphics and Algorithms Vienna University of Technology Abstract The ultimate aim of realistic graphics is the creation of images that provoke the same responses that a viewer would have to a real scene. This STAR addresses two related key problem areas in this effort which are located at opposite ends of the rendering pipeline, namely the data structures used to describe light during the actual rendering process, and the issue of displaying such radiant intensities in a meaningful way. The interest in the first of these subproblems stems from the fact that it is common industry practice to use RGB colour values to describe light intensity and surface reflectancy. While viable in the context of methods that do not strive to achieve true realism, this approach has to be replaced by more physically accurate techniques if a prediction of nature is intended. The second subproblem is that while research into ways of rendering images provides us with better and faster methods, we do not necessarily see their full effect due to limitations of the display hardware. The low dynamic range of a standard computer monitor requires some form of mapping to produce images that are perceptually accurate. Tone reproduction operators attempt to replicate the effect of real-world luminance intensities. This STAR report will review the work to date on spectral rendering and tone reproduction techniques. It will include an investigation into the need for spectral imagery synthesis methods and accurate tone reproduction, and a discussion of major approaches to physically correct rendering and key tone mapping algorithms. The future of both spectral rendering and tone reproduction techniques will be considered, together with the implications of advances in display hardware. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Viewing Algorithms I.3.7 [Computer Graphics]: Color, shading, shadowing, and texture 1. Introduction ends of the rendering pipeline have been neglected by com- parison: the question which entities are used in rendering The ultimate aim of realistic graphics is the creation of im- programs to describe light intensity during the calculations ages that provoke the same response and sensation as a performed by the rendering algorithms, and the mapping of viewer would have to a real scene, i.e. the images are phys- the luminances computed by these algorithms to the display ically or perceptually accurate when compared to reality. device of choice. Weaknesses of a system in both areas can This requires significant effort to achieve, and one of the key make any improvements in the underlying rendering algo- properties of this problem is that the overall performance of rithm totally pointless. Consequently, good care has to be a photorealistic rendering system is only as good as its worst taken when designing a system for image synthesis to strike component. a good balance between the capabilities of the various stages In the field of computer graphics, the actual image syn- in the rendering pipeline. thesis algorithms – from scanline techniques to global illu- mination methods – are constantly being reviewed and im- In this paper we review the state of the art on these two proved, but two equally important research areas at opposite topics in an interleaving manner. We first present the basic c The Eurographics Association 2002. Devlin, Chalmers, Wilkie, Purgathofer / Tone Reproduction and Physically Based Spectral Rendering problems in both areas in sections 2 and 3, and then discuss edge of the Human Visual System (HVS) is still limited, but previous work on tone mapping in section 4 and for spectral its ability to perceive such a wide dynamic range in the real- rendering in section 5. world requires some form of reproduction to produce per- ceptually accurate images on display devices. Changes in the perception of colour and of apparent contrast also come into 2. Tone mapping play when mapping values to a display device. The develop- While research into ways of creating images provides us ment of new psychophysically-based visual models seeks to with better and faster methods, we usually do not see the address these factors. To date, methods of tone mapping tend full effect of these techniques due to display limitations. For to concentrate on singular aspects for singular purposes. This accurate image analysis and comparison with reality, the dis- approach is understandable given the deficit in HVS knowl- play image must bear as close a resemblance to the original edge, but is inefficient as the HVS responds as a whole, image as possible. In situations where predictive imaging rather than as isolated functions. New psychophysical re- is required, tone reproduction is of great importance to en- search is needed to address the workings of the HVS in their sure that the conclusions drawn from a simulation are correct totality. (Figure 1). 2.2. Tone mapping: art, television and photography 2.1. The need for accurate tone reproduction Tone mapping was developed for use in television and pho- Tone reproduction is necessary for two main reasons: the tography, but its origins can be seen in the field of art where first is to ensure that the wide range of light in a real world artists make use of a limited palette to depict high contrast scene is conveyed on a display with limited capabilities; and scenes. It takes advantage of the fact that the HVS has a the second is to produce an image which provokes the same greater sensitivity to relative rather than absolute luminance responses as someone would have when viewing the scene in levels 26. Initial work on accurate tone reproduction was in- the real world. Physical accuracy alone of a rendered image sufficient for high dynamic range scenes. Either the average does not ensure that the scene in question will have a realis- real-world luminance was mapped to the display average, or tic visual appearance when it is displayed. This is due to the the maximum non-light source luminance was mapped to the shortcomings of standard display devices, which can only maximum displayable value. However, the process failed to reproduce a range of luminance of about 100:1 candelas per preserve visibility in high dynamic range scenes as the very 2 square metre (cd m ), as opposed to human vision which bright and very dimmed values were clamped to fall within ranges from 100 000 000:1, from bright sunlight down to the display range. Also, all images were mapped irrespective starlight, and an observer’s adaptation to their surroundings of absolute value, resulting in the loss of an overall impres- also needs to be taken into account. It is this high dynamic sion of brightness 38. range (HDR) of human vision that needs to be scaled in some way to fit a low dynamic range display device. The extensive use of tone reproduction in photography and television today is explained in Hunt’s “The Reproduc- In dark scenes our visual acuity — the ability to resolve tion of Colour in Photography, Print and Film” 26 and Poyn- spatial detail — is low and colours cannot be distinguished. ton’s “A Technical Introduction to Digital Video” 55, which This is due to the two different types of photoreceptor in the give comprehensive explanations on the subject. This re- eye: rods and cones. It is the rods that provide us with achro- search area is outside the scope of this paper, and it is sug- matic vision at these scotopic levels, functioning within a gested that readers with an interest in this area refer initially 6 2 range of 10 ¡ to 10 cd m . Visual adaptation from light to to these works. dark is known as dark adaptation, and can last for tens of minutes; for example, the length of time it takes the eye to adapt at night when the light is switched off. Conversely, 2.3. Gamma correction light adaptation, from dark to light, can take only seconds, Gamma is a mathematical curve that represents the bright- such as leaving a dimly lit room and stepping into bright sun- ness and contrast of an image. Brightness is a subjective light. The cones are active at these photopic levels of illumi- 8 2 measurement, formally defined as “the attribute of a visual nation, covering a range of 0.01 to 10 cd m . The overlap sensation according to which an area appears to emit more (the mesopic levels), when both rods and cones are func- 54 2 or less light" . It describes the non-linear tonal response of tioning, lies between 0.01 to 10 cd m . The range normally the display device and compensates for the non-linearities. used by the majority of electronic display devices (cathode 2 For CRTs, the use of RGB values to express colour is actu- ray tubes, or CRTs) spans from 1 to 100 cd m . More de- ally specifying the voltage that will be applied to each elec- tailed information on visual responses with regard to tone tron gun. The luminance generated is not linearly related to reproduction can be found in the papers by Ferwerda et al., this voltage. In actuality, luminance produced on the display ¢ ¢ Pattanaik et al. and Tumblin 13 ¢ 48 50 81. device is approximately proportional to the applied voltage Despite a wealth of psychophysical research, our knowl- raised to a power of 2.5, although the actual value of the c The Eurographics Association 2002. Devlin, Chalmers, Wilkie, Purgathofer / Tone Reproduction and Physically Based Spectral Rendering ¡ ¢£ Display ¤ ¤ ¤ Tone Reproduction Display with Observer Operator Limited Capabilities ¥¦ Scene Perceptual Match ¡ £ ¢ ¤ Observer Real-World Figure 1: Ideal tone reproduction process exponent varies 54.