Sony Pictures Imageworks Arnold CHRISTOPHER KULLA, Sony Pictures Imageworks ALEJANDRO CONTY, Sony Pictures Imageworks CLIFFORD STEIN, Sony Pictures Imageworks LARRY GRITZ, Sony Pictures Imageworks Fig. 1. Sony Imageworks has been using path tracing in production for over a decade: (a) Monster House (©2006 Columbia Pictures Industries, Inc. All rights reserved); (b) Men in Black III (©2012 Columbia Pictures Industries, Inc. All Rights Reserved.) (c) Smurfs: The Lost Village (©2017 Columbia Pictures Industries, Inc. and Sony Pictures Animation Inc. All rights reserved.) Sony Imageworks’ implementation of the Arnold renderer is a fork of the and robustness of path tracing indicated to the studio there was commercial product of the same name, which has evolved independently potential to revisit the basic architecture of a production renderer since around 2009. This paper focuses on the design choices that are unique which had not evolved much since the seminal Reyes paper [Cook to this version and have tailored the renderer to the specic requirements of et al. 1987]. lm rendering at our studio. We detail our approach to subdivision surface After an initial period of co-development with Solid Angle, we tessellation, hair rendering, sampling and variance reduction techniques, decided to pursue the evolution of the Arnold renderer indepen- as well as a description of our open source texturing and shading language components. We also discuss some ideas we once implemented but have dently from the commercially available product. This motivation since discarded to highlight the evolution of the software over the years. is twofold. The rst is simply pragmatic: software development in service of lm production must be responsive to tight deadlines CCS Concepts: • Computing methodologies → Ray tracing; (less the lm release date than internal deadlines determined by General Terms: Graphics, Systems, Rendering the production schedule). As an example, when we implemented Additional Key Words and Phrases: Ray-Tracing, Rendering, Monte-Carlo, heterogeneous volume rendering we did so knowing that it would Path-Tracing be used in a particular sequence of a lm (see Figure 1), and had to rene the implementation as artists were making use of our early ACM Reference format: prototype. Such rapid iteration is particularly problematic when Christopher Kulla, Alejandro Conty, Cliord Stein, and Larry Gritz. 2017. Sony Pictures Imageworks Arnold. ACM Trans. Graph. 9, 4, Article 39 (March 2017), these decisions impact API and ABI compatibility which commercial 18 pages. software must be beholden to on much longer time frames. The DOI: 0000001.0000001 second motivation to fork from the commercial project is more strategic: by tailoring the renderer solely towards a single user base, 1 INTRODUCTION we can make assumptions that may not be valid for all domains or all workows. By contrast, a commercial product must retain 1.1 History some amount of exibility to dierent ways of working and cannot Sony Imageworks rst experimented with path tracing during the optimize with only a single use case in mind. production of the lm Monster House. The creative vision for the While our renderer and its commercial sibling share a common lm called for mimicking the look of stop motion miniatures, a ancestry, the simple fact that development priorities changed the look which had already been successfully explored in early shorts timeline around various code refactors have rendered the codebases rendered by the Arnold renderer [Jensen et al. 2001]. Despite some structurally incompatible despite many shared concepts. limitations that were worked around for the lm, the simplicity This paper describes the current state of our system, particularly focusing on areas where we have diverged from the system described Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed in the companion paper [Georgiev et al. 2018]. We will also catalog for prot or commercial advantage and that copies bear this notice and the full citation some of the lessons learned by giving examples of features we on the rst page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). removed from our system. © 2017 Copyright held by the owner/author(s). 0730-0301/2017/3-ART39 $15.00 DOI: 0000001.0000001 ACM Transactions on Graphics, Vol. 9, No. 4, Article 39. Publication date: March 2017. 39:2 • Christopher Kulla, Alejandro Conty, Cliord Stein, and Larry Gritz 1.2 Design Principles 2.1 Subdivision Surfaces The most important guiding principle in the design of our system Displaced subdivision surfaces are probably the most common geo- has been simplicity. For the artist, this means removing as many metric primitive at Sony Imageworks. Therefore, we needed a fast unnecessary choices as possible, to let them focus on the creative and highly parallel subdivision engine to minimize time-to-rst choices they must make. For us as software developers, this has pixel as well as a compact storage representation of highly subdi- meant intentionally minimizing our feature set to the bare necessi- vided surfaces to reduce memory consumption. We opted to focus ties, aggressively removing unused experimental code when it falls on tessellation as opposed to native ray tracing of patches for the out of favor, and minimizing code duplication. simple reason that our geometric models are usually quite dense Perhaps as a direct consequence of this goal, we have always which makes storing higher-order patches more expensive than focused on ray tracing as the only building block in our system. simply storing triangles. Moreover, methods based on higher-order Because the natural expression of geometric optics leads one to patches typically fail in the presence of displacements requiring think in terms of rays of light bouncing around the scene, we felt extra geometric representations and going against our principle all approximate representations of such eects (shadow maps, re- of code simplicity and orthogonality. Our system also focuses on ection maps, etc.) were simply removing the artist from what they up-front tessellation which greatly simplies the implementation really wanted to express. compared to deferred or on-demand systems that try to interleave tessellation and rendering. We further motivate this choice in Sec- 1.3 System Overview tion 7.4. We divide the architecture of our system into three main areas to A number of criteria dictated the approach we take to subdivi- highlight how we meet the challenge of modern feature lm pro- sion. Firstly, the subdivision engine must be able to handle arbitrary duction rendering: geometry, shading and integration (see Figure 2). n-gons. Even though quads are the primary modeling primitive at the facility, other polygons do creep in to the system for a variety Geometry. The rst and most important design choice in our ren- of reasons. Secondly, we chose to ignore support for weight driven derer has been to focus on in-core ray tracing of massive scenes. The creases to maximize interoperability between software packages. natural expression of path tracing leads to incoherent ray queries We anticipate that as the rules of the OpenSubdiv library become that can visit any part of the scene in random order. Observing more widely supported, we may need to revisit this decision, but that the amount of memory available on a modern workstation can ignoring this requirement signicantly simplies both the imple- comfortably store several hundred million polygons in RAM, we mentation and the data transport across the production pipeline. explicitly craft our data structures to represent as much unique de- While creases can make models slightly more lightweight when used tail as possible in as few bytes as possible. We also heavily leverage optimally, our modelers are quite adept at explicitly adding tight instancing as its implementation is well suited to ray tracing but bevels to mimic the inuence of creases without needing special also closely matches how we author large environments by re-using rules in the subdivision code itself. Thirdly, we wanted a patch-based existing models. This design choice also has the benet of keeping subdivision engine where patches could be processed in parallel the geometric subsystem orthogonal to the rest of the system. independent of their neighbors. With these criteria in mind, we chose to evaluate all top-level regular quads with bicubic b-spline Shading. The role of the shading system is to eciently manage patches, and all “irregular” top-level n-gons with n Gregory Patches large quantities of texture data (terabytes of texture data per frame (one patch per sub-quad) with Loop’s method [Loop et al. 2009]. is not uncommon) and eciently execute shading networks that Patches are processed in parallel independently of each other, and describe the basic material properties to the renderer. Unlike older we control the writing of shared vertices by prioritizing based on rendering architectures, shaders are only able to specify the BSDF patch ID number. per shading point rather than be responsible for its evaluation. Subdivision is parallelized rst by treating multiple objects in parallel and then within objects by processing patches in parallel. Integration. Turning shaded points into pixel colors is the role This is further discussed in Section 6.5. of the integrator which implements