Eurographics Symposium on Rendering 2016 Volume 35 (2016), Number 4 E. Eisemann and E. Fiume (Guest Editors) Efficient Ray Tracing Through Aspheric Lenses and Imperfect Bokeh Synthesis Hyuntae Joo1, Soonhyeon Kwon1, Sangmin Lee1, Elmar Eisemann2 and Sungkil Lee†1 1Sungkyunkwan University, South Korea 2Delft University of Technology, Netherlands Figure 1: Examples of bokeh textures generated with our ray-tracing solution. Abstract We present an efficient ray-tracing technique to render bokeh effects produced by parametric aspheric lenses. Contrary to conventional spherical lenses, aspheric lenses do generally not permit a simple closed-form solution of ray-surface intersections. We propose a numerical root-finding approach, which uses tight proxy surfaces to ensure a good initialization and convergence behavior. Additionally, we simulate mechanical imperfections resulting from the lens fabrication via a texture-based approach. Fractional Fourier transform and spectral dispersion add additional realism to the synthesized bokeh effect. Our approach is well-suited for execution on graphics processing units (GPUs) and we demonstrate complex defocus-blur and lens-flare effects. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image Generation—Display Algorithms 1. Introduction no general closed-form solution for calculating ray-surface intersec- tions, as exists for spherical lenses [KMH95, SDHL11, HESL11]. Optical systems are typically imperfect and not all rays will follow Instead, we present an efficient numerical scheme to trace rays the expected optical path toward the sensor, which leads to optical through these lens elements. Additionally, aspheric lenses often con- aberrations [Smi00]. Classical systems make use of many spherical tain small imperfections due to the fabrication process (e.g., mechan- lens elements to reduce this effect. A recent alternative is an aspheric ical grinding [LB07]), influencing out-of-focus blur. In particular, it lens, whose profile does not correspond to a part of a sphere [KT80]. can impact the intensity profile of bokeh, the aesthetic appearance of Their use typically implies fewer optical elements for a similar out-of-focus highlights, resulting in “onion-ring bokeh” [ST14]. We reduction of aberrations, which, consequently, results in less weight. simulate this characteristic aspect by using a normal map derived This property made them a popular choice (e.g., smartphones lenses, from a virtual grinding process. pancake lenses, and eyeglasses). Despite their wide-spread use in practice, aspheric lenses have hardly been investigated in computer The contributions of this paper can be summarized as: graphics. • an aspheric lens model, including imperfections; • In this paper, we model and simulate aspheric lenses. Because an efficient numerical aspheric-lens intersection method; their surface profile is usually nonlinear and non-spherical, there is • a rendering solution for optical systems. Our paper is organized as follows. We discuss previous work (Sec.2) and introduce our aspheric lens model (Sec.3), including † [email protected] (corresponding author) imperfections (Sec.4). We then cover our rendering algorithm and © 2016 The Author(s) Computer Graphics Forum © 2016 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd. H. Joo & S. Kwon & S. Lee & E. Eisemann & S. Lee / Efficient Ray Tracing Through Aspheric Lenses and Imperfect Bokeh Synthesis intersection test for aspheric lenses (Sec.5) before presenting results sa sa (Sec.6) and concluding (Sec.7). (a) spherical lens (b) aspheric lens 2. Related Work This section reviews previous work on simulating optical systems Figure 2: A spherical lens (a) can cause spherical aberration, while and ray-tracing techniques for parametric surfaces. even a single aspheric lens (b) may focus well. 2.1. Optical Simulation but unreliable for complex surfaces. These problems manifest them- Early approaches for optical simulations have relied on simple op- selves even more clearly when using limited precision floating-point tical models [Coo86, Che87], but simulating a full optical system numbers, as is often the case for GPU approaches. In contrast, our has been of growing interest. Many approaches opted for additional solution relies on a bracketing method supported by tight proxy realism in terms of geometric distortion, defocus blur, and spectral geometry to perform the intersection test, which even works for dispersion. They usually build upon formal data sheets specifying lower-precision floating point values. the optical system in terms of its optical elements. The seminal work of Kolb et al. obtained geometric accuracy for distortion and field of view [KMH95]. Steinert et al. included additional aberrations and 3. Aspheric-Lens Model spectral diversities using Monte-Carlo rendering [SDHL11]. Hullin We here introduce our parametric aspheric-lens model. An aspheric et al. added reflections and anti-reflective coatings, which play a role lens adds nonlinear deviations to its base curvature c of a spherical for lens flares [HESL11]. They attained physically faithful image surface (Figure2), leading to a better convergence of peripheral rays quality but were restricted to spherical lenses in order to simplify on the focus plane. The common parametric form of an aspheric the ray-surface intersection tests. In this paper, we add support for surface with its sagittal profile z on the optical axis (along the z-axis), aspheric lenses. is expressed in terms of the radial coordinate r from the optical axis Optical imperfections can be classified into intrinsic and extrin- as: 2 sic ones. The former ones include aberrations and flares [HESL11, cr 2i z(r) := p + ∑A2ir ; (1) LE13], the latter ones include issues due to mechanical manufactur- 1 + 1 − (1 + k)c2r2 ing (e.g., grinding and polishing) and real-world artifacts (e.g., dirt or axis misalignment). where k is a conic constant [FTG00] and A2i are coefficients for even-polynomial sections; the coefficients of particular systems can One of the pronounced intrinsic imperfections are (spherical) be found in patents or lens design books [Smi05]. aberrations manifesting in bokeh and can be simulated via ray trac- ing [LES10, WZH∗10, WZHX13]. For higher performance, early The design of an aspheric lens goes through an iterative opti- models captured the appearance phenomenologically [BW02]. Later mization procedure to minimize aberrations. The conic constant approaches relied on efficient filtering; polygonal filtering [MRD12], k is often a consequence of the lens’ base shape from which the separable filtering [MH14], low-rank linear filters [McG15], or look- aspheric lens is derived, hence, the focus lies on optimizing A2i. up table [GKK15]. The design requires high expertise and care in optics. We refer the reader to the textbooks for mathematical properties and practical Extrinsic imperfections are not easily incorporated into ideal para- usages [FTG00, Smi05]. metric models [KMH95,WZH ∗10,SDHL11,HESL11,WZHX13]. One attempt was to use artificial noise for diffraction at the aper- ture [RIF∗09,HESL11], whereas our method is driven by simulating 4. Imperfections the lens-fabrication process. In the real world, the fabrication of large spherical/aspheric lenses involves mechanical grinding and polishing [LB07] (Figure3). The process has usually several phases; for instance, spherical grinding, 2.2. Ray Tracing of Parametric Surfaces precision grinding, and polishing are applied in sequence [FTG00]. While many modern ray tracers focus on polygonal mod- Similarly, in precision glass molding, a mold is carved, which will els [PBMH02,FS05], early approaches often included several para- then exhibit the imperfections that are passed on to the lens, when it metric surface types, which can be classified in four categories: sub- is produced with the mold. The processes are not entirely standard- division [Whi80], algebraic methods [Kaj82,Bli82, Han83,Man94], ized and although the various phases have been improved over the Bézier clipping [NSK90, EHS05], and numerical techniques [Tot85, years, the result is not perfect yet. JB86, AGM06]. The most visible artifacts resulting from the fabrication process In optics, much work addressed intersections with spherical sur- are due to a misalignment between grinding and optical axis, grind- faces but few approaches covered aspheric lenses; typically, tessel- ing asymmetries resulting from the swinging of the tools, and limited lation [MNN∗10] and Delaunay triangles [OSRM09]) or Newton- grinding precision. A few nanometers of deviation can lead to a visi- Raphson methods [AS52,For66] were applied. Tessellation is costly ble impact on bokeh. We model the resulting imperfections indirectly for sufficient precision, while the Newton-Raphson method is fast by simulating the grinding process itself. Following the fabrication © 2016 The Author(s) Computer Graphics Forum © 2016 The Eurographics Association and John Wiley & Sons Ltd. H. Joo & S. Kwon & S. Lee & E. Eisemann & S. Lee / Efficient Ray Tracing Through Aspheric Lenses and Imperfect Bokeh Synthesis (a) grinding (b) polishing Offline preprocessing Online rendering load an optical system for each RGB channel swinging swinging for each ray for each surface element s if s is an aperture apply aperture stop at intersection else if s is an aspheric lens find analytical intersection with proxies for each aspheric lens test numerical intersection - find proxy surfaces refract ray (with