Masaryk University Faculty of Informatics Real-time Snow Deformation Master’s Thesis Daniel Hanák Brno, Spring 2021 This is where a copy of the official signed thesis assignment and a copy ofthe Statement of an Author is located in the printed version of the document. Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Daniel Hanák Advisor: Mgr. Jiří Chmelík, Ph.D. i Acknowledgements I would like to express my gratitude to my advisor Jiří Chmelík, assis- tant professor at the Department of Visual Computing, for his guid- ance, helpful advice, and remarks. Furthermore, I would also like to thank my friends and colleagues for their reviews of my work. Lastly, I would like to thank my family for their encouragement and support during my studies. iii Abstract Procedural terrain deformation has remained an open problem in com- puter games. Having dynamic objects such as characters and animals interact with the environment makes the scene more immersive and alive. This thesis presents a scalable real-time snow rendering tech- nique implemented on the GPU using compute shaders and hardware tessellation. The snow can be deformed by any dynamic object and has a sliding window that moves with the player, which allows this technique to be used in open-world games. Furthermore, it supports el- evation along the edges of the deformation and filling generated trails over time. Presented are also performance results of the proposed technique evaluated on multiple GPUs. iv Keywords computer graphics, game development, rendering, shader, compute, tessellation, hlsl, deformation, snow v Contents 1 Introduction 1 2 Related Work 3 2.1 Assassin’s Creed III . .3 2.2 Batman: Arkham Origins . .4 2.3 Rise of the Tomb Raider . .5 2.4 Horizon Zero Dawn: The Frozen Wilds . .6 2.5 The Last of Us Part II . .8 3 The Graphics Rendering Pipeline 11 3.1 Vertex Processing . 11 3.2 The Tessellation Stage . 12 3.3 The Geometry Shader . 13 3.4 Pixel Processing . 13 3.5 The Compute Shader . 14 4 Implementation 17 4.1 Rendering Orthographic Height . 17 4.2 Deformation Heightmap . 17 4.3 Sliding Window . 20 4.4 The Vertex Shader . 20 4.5 The Tessellation Stage . 21 4.6 The Pixel Shader . 22 4.7 Creating Textures . 24 5 Example Scene 27 5.1 Snow Renderer . 27 5.2 Level Design . 28 6 Results 31 6.1 Memory Requirements . 31 6.2 Performance . 31 6.3 Visual Quality . 33 7 Conclusion 35 Bibliography 37 vii A Electronic Attachments 41 viii List of Tables 6.1 Measured GPU performance of the compute shaders. 32 ix List of Figures 2.1 Deformable snow in Rise of the Tomb Raider 5 2.2 Horizon Zero Dawn: The Frozen Wilds environment 7 2.3 The Last of Us Part II snowy forest 8 4.1 Debug view of computed normals. 19 4.2 Undeformed snow material 25 5.1 Chomper rendered using PBR 29 5.2 A shot from the testing environment 30 6.1 Average frame rate measured on multiple GPUs 33 6.2 A shot from the winter forest 34 6.3 The deformable snow material with different parameters 34 xi 1 Introduction Nowadays, many AAA1 video games push the boundaries of 3D com- puter graphics with groundbreaking open-world environments. Game studios employ various sophisticated algorithms to solve problems re- lated to rendering interactive virtual worlds, such as snow simulation, which has always been a challenging task. Having dynamic objects such as characters, animals, and debris from explosions interact with the snow makes the environment more immersive and alive. A straight- forward way how to achieve a snow surface is to store white color in the object’s diffuse texture. Character footsteps are then rendered as decals projected on the snow geometry [1]. Decals are often used in games for effects such as bullet holes and tire marks. They are usually rendered as textures that are orthographically projected through a convex volume. Unfortunately, this common technique lacks scalabil- ity and persistence. The footsteps in snow usually have an upper limit to the number of active decals at the same time. This thesis aims to develop a scalable real-time snow deformation technique that can be used in video games using implementation on modern GPUs to achieve real-time performance. More specifically, the proposed snow deformation technique has to work in a massive open-world game on top of static objects. It also has to work for any dynamic object with skeletal animation, and it has to run fast on the GPU. The following chapter describes various approaches of real-time snow deformation used in video games. Chapter 3 presents the real- time graphics rendering pipeline that is used in computer games to render 2D images on a screen. The in-depth overview and implemen- tation details of the proposed real-time snow deformation technique are shown in Chapter 4. Implementation details of the testing environ- ment created in Unity Engine are presented in Chapter 5. Performance results of the implementation evaluated on multiple GPUs are pre- sented in Chapter 6. Finally, the thesis concludes in Chapter 7 with a summary and discussion of future work. 1. A triple-A video game (AAA) is generally a title developed by a large studio with a higher development budget. 1 2 Related Work This chapter presents several different approaches to real-time snow deformation rendering. The following games with presentations and articles on the subject achieved notable results with their advantages and disadvantages. 2.1 Assassin’s Creed III One of the first deformable snow implementations was proposed by Ubisoft Entertainment in 2012 for the game Assassin’s Creed III [2]. Deformable snow in the winter environment is one of the leading graphics features of the game. The snow affects both visual and game- play aspects of the game because it is deformed by all characters and animals, which allows the player to track them. The game takes place in a vast and varied world, which supports both summer and winter conditions. The snow mesh for the winter environment is generated as a displaced copy of the base ground mesh. When a dynamic object steps into the snow mesh, the colliding triangles are removed from the index buffer. A new set of tessellated and displaced triangles is generated based on the removed triangles and displacement texture. To accomplish this, they use a technique known as render to ver- tex buffer (R2VB) [3], which allows runtime GPU tessellation1. For the tessellation, they use barycentric coordinates of pre-tessellated triangles, with the limitation that all created triangles have the same tessellation factor. The displacement is rendered into a render target as a geometrical approximation of the character’s movement and used as a vertex buffer in a second pass to push down and tessellate triangles. This technique has a significant drawback. The outer vertices of a tessellated polygon may not lie within the adjacent polygon’s edge. These vertices are known as T-junctions [5]. It also happens when two triangles have edges that fall within the same line in space but do not 1. As a Playstation 3 and Xbox 360 title, the game was bounded to Shader Model 3.0 GPUs. Compute shaders and hardware tessellation became available on next- generation consoles [4]. 3 2. Related Work share the same endpoint vertices. These vertices cause visible seams in the snow mesh. 2.2 Batman: Arkham Origins Back in 2013, WB Games Montreal in collaboration with NVIDIA pro- posed a technique for the rendering of deformable snow featured in Batman: Arkham Origins [6]. The game takes place in Gotham City, which is a fictional city set within the United States. Therefore, theen- vironment consists of streets and rooftops. For each rooftop and street, they dynamically allocate and deallocate displacement heightmaps at runtime. These heightmaps are created and released based on player visibility, object size, and occupancy. To generate a heightmap, they render snow-affecting objects from under the surface using an orthogonal frustum (to avoid any perspec- tive distortion) into an offscreen buffer. The frustum height equals the height of the snow. The rendered depth buffer is then filtered and accumulated with a custom post-process chain using ping-pong buffers [7]. It uses two offscreen buffers, first for reading andsecond for writing; after each frame of the depth rendering, the roles of these buffers switch. The depth buffer is filtered with a 4-tap bilinear Poisson disk which adds gradience to the overall result. They also subtract a constant value from the heightmap to replenish the deformed snow during the merging stage. The deformable snow surface has two states: non-deformed snow and completely flattened snow, both with snow materials that simulate the snow state. They blend these two materials in a pixel shader based on the heightmap. Diffuse and specular textures are blended using linear interpolation, but for normal textures, they use reoriented normal mapping [8], which is a quaternion-based technique that blends two normal maps correctly. This snow deformation technique is used with relief mapping [9] on consoles (Playstation 3 and Xbox 360); the PC version is enhanced with GPU tessellation on DirectX 11, where the heightmap is used to displace the geometry of the snow mesh. The technique allows for visually powerful and interactive de- formable snow, which works with dynamic shadows and dynamic 4 2.