Volume 0 (1981), Number 0 pp. 1–27 COMPUTER GRAPHICS forum The State of the Art in Interactive Global Illumination Tobias Ritschel1;2 Carsten Dachsbacher3 Thorsten Grosch4 Jan Kautz5 Télécom ParisTech / CNRS (LTCI)1 and Intel Visual Computing Institute2 Karlsruhe Institute of Technology3 University of Magdeburg4 University College London5 !"#$%&'( ) *+,#-. ) !"#$%&'() 6/-**7) /$0+() %&8&'5-"*) 3,4*5'*) 1',2&%$"0) Figure 1: Photography of a scene with global illumination: Multiple diffuse and specular bounces, caustics and scattering. Abstract The interaction of light and matter in the world surrounding us is of striking complexity and beauty. Since the very beginning of computer graphics, adequate modeling of these processes and efficient computation is an intensively studied research topic and still not a solved problem. The inherent complexity stems from the underlying physical processes as well as the global nature of the interactions that let light travel within a scene. This article reviews the state of the art in interactive global illumination computation, that is, methods that generate an image of a virtual scene in less than one second with an as exact as possible, or plausible, solution to the light transport. Additionally, the theoretical background and attempts to classify the broad field of methods are described. The strengths and weaknesses of different approaches, when applied to the different visual phenomena, arising from light interaction are compared and discussed. Finally, the article concludes by highlighting design patterns for interactive global illumination and a list of open problems. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Color, shading, shadowing, and texture / Radiosity / Raytracing 1. Introduction the underlying principles of light-matter interaction are well understood, the efficient computation is still a challenging Light in the real world interacts with surrounding media and problem. Many processes are so computationally expensive surfaces creating stunning visual phenomena such as color that they require the development of simplified models, tai- bleeding, reflections, crepuscular rays, and caustics. Need- lored algorithms and data structures. In this article we will less to say that the reproduction of the real world has been give a survey and classification focusing on interactive meth- the ultimate goal of computer graphics ever since. Although ods for computing light transport in virtual scenes (global c 2011 The Author(s) Journal compilation c 2011 The Eurographics Association and Blackwell Publishing Ltd. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. T. Ritschel, C. Dachsbacher, T. Grosch, J. Kautz / The State of the Art in Interactive Global Illumination illumination). These methods resort to work in related fields because all virtual modifications seamlessly integrate into of computer graphics: first, measuring or modeling of re- the real world [Deb98]. If the current lighting conditions flectance of surfaces or scattering of participating media it- are captured, this information can be used to illuminate the self is an active area of research beyond the scope of this virtual changes, e. g., virtual objects appear with consistent il- report. Second, global illumination often builds on methods lumination and mutual shadows [GCHH03] between real and that compute visibility between a point and a surface, or two virtual objects can be displayed using differential rendering. surfaces, e. g., when computing shadows. Again, we refer to Typical applications are virtual furniture, virtual prototypes an excellent survey [HLHS03] and a recent book [ESAW11]. and feature films. Worth mentioning in this context are also a survey on am- We will start by introducing the most important theoretical bient occlusion [MFS08] (which can be considered as an concepts, such as the rendering equation, reflectance, visibil- approximation to global illumination), a survey of specular ity, etc., as a basis for the subsequent discussion of different effects oin the GPU [SKUP∗09] and courses both on “Global approaches, models, and simplifications (Sec.2). The ap- Illumination Across Industries” [KFC∗10] and scattering in proaches include among others ray tracing, finite element participating media [GWWD09]. A book about global illu- methods, bi-directional methods such as photon mapping mination in general is “Advanced Global Illumination” by and instant radiosity (Sec.3). Afterwards we will discuss the Dutré and colleagues [DBBS06] as well as PBRT by Pharr strengths and weaknesses of the different approaches when and Humphreys [PH04]. being applied to certain phenomena, e. g., diffuse or glossy Although we focus on the state of the art in interactive global interreflections, caustics, volumetric scattering (Sec.4). The illumination this article will reveal a huge body of work in this article highlights some design patterns and concepts that have field that we faithfully attempted to collect, review, classify, proven to be useful when computing GI, e. g., precompu- and compare. In this selection we concentrate on methods tation, using surfels to decoupled from the input geometry, that produce plausible global illumination (GI) solutions in multi-resolution and screen-space techniques (Sec.5). The less than one second (on contemporary hardware). Although design patterns in Sec.5 are the building blocks of the ap- other definitions of “interactive speed” exist, we consider this proaches in Sec.3 and at the same time the approaches in threshold adequate for computing solutions to this inherent Sec.3 consist of components which are strategies in Sec.5, complex problem. Interactive GI is usually based on certain e. g., “Splatting” can be used for GPU “Photon mapper” and simplifications and making tradeoffs in the continuum be- “Instant radiosity” at the same time. Therefore, Sec.5 and tween computationally expensive high-quality approaches Sec.3 constitute two views on the same state of the art which and simple models. are not fully orthogonal. Finally, we conclude and show a list of open problems (Sec.6). The rendering equation introduced by Kajiya [Kaj86] and the operator notation by Arvo et al. [ATS94] are two impor- 2. Theory tant concepts that allowed to compare and evaluate GI ap- proaches in a formalized way. The insight that light transport In this section we will introduce the basic theory of GI. Light can be described as an integral equation, however, is signifi- transport between surfaces with no surrounding media (i. e., cantly older, e. g., see Yamauti [Yam26] and Buckley [Buc27], in vacuum) is described by the rendering equation (Sec. 2.1), and interreflections have been studied and computed by Hig- which “links” light sources, surface reflectance (BRDFs), and bie [Hig34] and Moon [Moo40] to name at least a few of the visibility. The transport of light in the presence of participat- pioneers in this field. ing media is more involved and described by the volumetric rendering equation (Sec. 2.2). Traditionally, one important application of GI is architectural visualization. In recent years, the ever growing (and converg- ing) markets of interactive media, such as computer games 2.1. Rendering equation and feature films, have started to shift from ad-hoc modelling The rendering equation (RE) [Kaj86] (Fig.2) states that the of visual effects to more physically-plausible lighting compu- outgoing radiance Lo at a surface location x in direction w is tation. GI also has applications in computer vision [Lan11]; the sum of emitted radiance Le and reflected radiance Lr: while Forsyth and Zisserman [FZ91] consider GI an artifact that complicates pattern recognition, Nayar et al. [NIK90] Lo(x;w) = Le(x;w) + Lr(x;w): (1) exploit the information stemming from interreflections to The reflected radiance is computed as: support shape acquisition. Color constancy, i. e., discounting Z + for the color of the illuminant to recover reflectance, can Lr(x;w) = Li(x;wi) fr(x;wi ! w)hN(x);wii dwi; + also benefit from GI computation [FDH91]. Removing il- W (2) lumination has applications, among others, when scanning books [WUM97] or compensating indirect illumination in where W+ is the upper hemisphere oriented around the sur- ∗ virtual reality projector systems [BGZ 06]. Augmented Re- face normal N(x) at x, fr the bi-directional reflectance func- ality applications profit from interactive global illumination, tion (BRDF) and hi+ a dot product that is clamped to zero. c 2011 The Author(s) Journal compilation c 2011 The Eurographics Association and Blackwell Publishing Ltd. T. Ritschel, C. Dachsbacher, T. Grosch, J. Kautz / The State of the Art in Interactive Global Illumination To determine the incident radiance, Li, the ray casting oper- Other commonly used models in computer graphics are: (1) spot lights, which can be considered as point lights with a f (x) directionally focused emission; (2) directional lights, assum- r ing parallel light rays; and (3) environment maps which store Lo (x,ω) x Li (x,ωi ) incident radiance for every direction, however, assuming that Le (x,ω) Li(x;wi) is independent of x, i. e., Li(x;wi) = Lenv(wi). Figure 2: The rendering equation Reflectance The Bi-directional Reflectance Distribution Function (BRDF) is a 4D function that defines how light is reflected at a surface. The function returns the ratio of re- ator is used to determine from which other surface location flected radiance exiting along wo to the irradiance incident this radiance is emitted and reflected. It can be seen that the on the surface from direction w . Physically plausible BRDFs rendering equation is a Fredholm integral equation of the i have to be both symmetric fr(wi ! wo) = fr(wo ! wi) and second kind. The goal of global illumination algorithms is R + energy conserving + fr(w ! wo)hN;w i dw < 1. A spe- to compute L (x; ) for a given scene, materials and lighting W i i i o w cial case is the Lambertian BRDF which is independent of L .