Master of Science in Game and Software Engineering May 2020
Run-time Terrain Deformation in Unreal Engine 4
Rikard Magnom
Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for the degree of Master of Science in Game and Software Engineering. The thesis is equivalent to 20 weeks of full time studies.
The authors declare that they are the sole authors of this thesis and that they have not used any sources other than those listed in the bibliography and identified as references. They further declare that they have not submitted this thesis at any other institution to obtain a degree.
Contact Information: Author(s): Rikard Magnom E-mail: [email protected]
University advisor: Adjunct Prof. Stefan Petersson Department of Computer Science
Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57 Abstract
Background. When developing video games, an often desired trait is that of player immersion. Player immersion can be achieved through many different ways. How- ever, a common technique used is to allow the player to affect the environment in some way, through destruction or deformation. Unreal Engine 4 is a commonly used commercial game engine, that lacks features necessary to deform terrain at run-time. Objectives. This thesis explores the viability of run-time performant terrain defor- mation in Unreal Engine version 4.23. A deformation technique of a depth-based Dynamically-Displaced Height Map (DDHM) is selected for implementation and tested with various sizes of landscape terrains. Methods. A small literature study is performed to select a feasible run-time terrain deformation technique. The technique is then implemented in the engine, along with modifications to the engine to provide the necessary functionality. The implementa- tion is then tested on performance and accuracy. Results. The DDHM technique is shown to provide run-time performant terrain de- formation for smaller terrain sizes, consisting of an individual landscape component. The selected technique does not meet the run-time requirements for full, large-scale deformation. Conclusions. Run-time performant terrain deformation is shown to be achievable in Unreal Engine 4.23. While the selected technique fails to achieve satisfying results for larger terrain sizes, results on individual landscape components show feasibility for generic technique implementations.
Keywords: terrain, deformation, real-time, unreal engine
i
Sammanfattning
Bakgrund. Under utvecklingen av datorspel är det ofta en eftertraktad egenskap att kunna övertyga spelaren om realismen i spelvärlden. Detta kan uppnås på flera sätt, men en vanlig teknik är att låta spela påverka omgivningen i spelet på något sätt, genom förstörelse eller deformering. Unreal Engine 4 är en vida använd kom- mersiell spelmotor som saknar funktionalitet för att deformera terräng i realtid. Syfte. Syftet med detta arbete är att utforska genomförbarheten i att tillföra ter- rängdeformering i realtid i Unreal Engine version 4.23. En deformeringsteknik in- volverande en “Dynamically-Displaced Height Map” (DDHM) väljs ut för implemen- tation och testas med olika storlekar på landskapsterräng. Metod. En mindre litteraturstudie genomförs för att välja en rimlig teknik för ter- rängdeformering i körtid. Tekniken implementeras sedan i motorn, i sambard med modifikationer till motorn för att tillföra nödvändig funktionalitet. Implementatio- nen testas sedan på prestanda och precision. Resultat. DDHM-tekniken visas kunna tillföra körtidsduglig terrängdeformering för mindre terrängstorlekar, bestående av individuella landskapskomponenter. Tekniken möter inte realtidskriterierna för full, storskalig terrängdeformation. Slutsatser. Deformering av terräng i körtid visas vara genomförbart i Unreal Engine 4.23. Den utvalda tekniken visar lovande resultat för mindre storlekar av landskap- sterräng, men misslyckas att prestera för större terrängstorlekar.
Nyckelord: terräng, deformering, realtid, unreal engine
iii
Acknowledgments
First and foremost I would like to sincerely thank Stefan Petersson for his superb guidance and valuable feedback during the writing of this thesis, and for helping me stay sane throughout the process. I also express great gratitude towards Logic Artists for collaborating with me in the execution of this thesis, and for giving me free reign to work whenever needed throughout the process. An additional special thank you to my dear friend James Partridge for his assistance in proofreading. Finally, a warm thank you to my mother and my grandparents, for staying supportive of me forever and always.
v
Contents
Abstract i
Sammanfattning iii
Acknowledgments v
1 Introduction 1 1.1 Environment Interaction ...... 1 1.2 Objectives and Research Questions ...... 1 1.3 Terminology ...... 2
2 Related Work 3 2.1 Hardware Tessellation ...... 3 2.2 Unreal Engine Landscape Systems ...... 4 2.3 Displacement mapping ...... 5 2.4 Parallax mapping ...... 5
3 Method 7 3.1 Literature Study ...... 7 3.2 Depth-based DDHM ...... 9 3.3 Visual Representation ...... 11
4 Implementation 13 4.1 Deformation Actor ...... 13 4.1.1 Actor Initialization ...... 13 4.1.2 Class Structure ...... 13 4.2 Engine Source Code Modifications ...... 15 4.3 Engine Limitations ...... 17 4.3.1 Limitations of the USceneCaptureComponent2D ...... 18 4.3.2 Limitations of the Landscape Module ...... 18
5 Results 21 5.1 Experimental Results ...... 21 5.1.1 Testing Environment ...... 21 5.1.2 Full Algorithm Performance ...... 22 5.1.3 Render Target Import Performance ...... 23 5.1.4 Heightmap Update Performance ...... 23 5.1.5 Average Error ...... 24
vii 5.1.6 Conversion Resolution Scalar ...... 24
6 Analysis and Discussion 27 6.1 DDHM ...... 27 6.2 Generic Run-time Deformation ...... 27 6.3 Sustainability ...... 28 6.4 Validity Threats ...... 28
7 Conclusions 29
8 Future Work 31
References 33
A Deformation Actor Source Code 37
viii List of Figures
2.1 Visual example of overlapping landscape components. Note the du- plicated vertices [13]...... 4 2.2 Parallax mapping used on a wall texture in Bethesdas The Elder Scrolls IV: Oblivion [28]...... 5
3.1 Details of the first Google Scholar database search of the small liter- ature study...... 7 3.2 Details of the first BTH Summon database search of the small litera- ture study...... 8 3.3 Details of the second BTH Summon database search of the small lit- erature study...... 9 3.4 Details of the third BTH Summon database search of the small liter- ature study...... 9 3.5 The depth capture camera configuration for generation of a DDHM [1]. 10 3.6 The process of depth information being converted to a heightmap through the DDHM technique. (a) shows the object depth clipping through the terrain and being rendered to the depth capture render target. (b) shows the depth value data being used to repopulate the height map. [1] ...... 11 3.7 Example of one iteration of the terrain deformation. (a) shows the original unmodified terrain. (b) shows the sphere mesh used for this deformation example, rendered for demonstration purposes. (c) shows the resulting deformation in the terrain...... 11 3.8 Example of height levels converging to set stages as a result of colour banding in the depth buffer. This example shows the result on the landscape after 30 iterations...... 12
4.1 One iteration of the implementation performed on a flat landscape, visualized in gray, deforming around a cube shaped mesh, visualized in orange. (a) shows the configuration before the deformation. (b) shows the result of one iteration of the implementation...... 14 4.2 The orthographic projection matrix. The direction parameters left, right, top, bottom represent the size of the orthographic projection from the center. The parameters far and near represent the distance from the camera of the far and near planes respectively...... 18
5.1 Execution time of the algorithms performed on a single landscape component with no collision updating...... 22
ix 5.2 Average error distance per vertex with the DepthCaptureCamera placed with the near plane at various distances from the landscape mesh. . . 24
x List of Tables
4.1 All modified engine source code files...... 17
5.1 The results of the execution time of a full iteration with collision up- dating enabled...... 22 5.2 The results of the execution time of a full iteration with collision up- dating disabled...... 23 5.3 The results of the execution time of the render target import. . . . . 23 5.4 The results of the execution time of Algorithm 3 with collision updat- ing enabled...... 23 5.5 The results of the execution time of Algorithm 3 with collision updat- ing disabled...... 24 5.6 Depth buffer heightmap resolution and resolution conversion scalar at various distances of the DepthCaptureCamera...... 25
xi
List of Algorithms
1 ImportHeightmapFromRenderTarget ...... 15 2 DeformActorInitialize ...... 16 3 UpdateLandscapeHeightData ...... 17
xiii
Chapter 1 Introduction
In video game development, an often desired element is that of player immersion. Player immersion, which involves convincing the player of the realism of the inter- active game world they are experiencing. A strong contributor to player immersion involves being able to directly interact and affect the game environment. This can be done a multitude of ways such as simply being able to have the environment respond to player interaction [21].
1.1 Environment Interaction One common form of environment interaction found in video games is the ability for the player to destroy parts of the environment dynamically. Destructible environ- ments in games have a long history, dating back to early examples such as the 1978 arcade game Space Invaders, which featured cover for the player to hide behind. This cover would then progressively get destroyed upon being hit [3]. This trend continued into the 3D era of video games, with series such as Volition’s Red Faction utilizing interactive and fully destructive environments as a primary selling point. Utilizing their “GeoMod technology” the player could dynamically destroy parts of the cohesive environment. This also included being able to modify the terrain in addition to the placed game objects, introducing further interactability [31]. While plenty of work exists exploring destructible terrain in video games in real- time, from volume-preserving techniques [24] to more specific applications, such as creating vehicle trails [10] and tire tracks [4], not much work exists exploring solutions based in Unreal Engine 4. Existing research is primarily limited to work outside of commercial game engines or in the cases where game engines are used, the Unity game engine by Unity Technologies is usually the engine of choice, which has a different terrain and landscape system compared to Unreal Engine 4.
1.2 Objectives and Research Questions This thesis aims to investigate the feasibility of run-time terrain deformation by an arbitrary mesh given the existing terrain and landscape systems present in Epic Games’ Unreal Engine version 4.23 game engine. A suitable technique for run-time terrain deformation is to be identified through a small literature study. The identified technique will then be implemented in Unreal Engine 4.23 and will be quantitatively evaluated on the execution time of the algorithm, measured in milliseconds. If the
1 2 Chapter 1. Introduction specific method selected contains additional steps that would not apply in the generic case of terrain deformation, this will be evaluated separately. The accuracy of the deformation method will be measured in Unreal Units, the primary unit of measure- ment in 3D-space in the Unreal Engine game engine [14]. This thesis is performed on behalf of Logic Artists, who have requested the im- plementation to be performed in version 4.23 of Unreal Engine. As of the writing of this thesis, no newer version of the engine exists that contains major modifications to any module required for the implementation presented in Chapter 4. The research questions posed in this thesis are:
1. Does a run-time terrain deformation technique for arbitrary mesh deformation exist that can be implemented in Unreal Engine 4.23?
2. How well can we achieve run-time performant terrain deformation in Unreal Engine 4.23? The criteria for concluding whether the implementation of the identified technique is suitable for run-time utilization is that the method needs to be suitable for use in a game environment where the frame rate does not drop below 30 frames per second. This criterion is necessary to not negatively impact the gameplay experience of a player, should this technique be implemented in a computer game [5]. The implementation will be tested on a platform of equal or comparable performance to that of the average consumer configuration as recorded by a hardware survey at the time of writing this thesis [7].
1.3 Terminology This section defines some important terminology used throughout the thesis.
DDHM - Dynamically Displaced Height-Map depth-based - Primarily utilizing the depth-pass of the rendering pipeline UE - Unreal Engine Unreal Unit - Unit of measurement used in Unreal Engine Actor - Base component in a scene in Unreal Engine Landscape Component - A single component of a terrain Landscape in Unreal Engine Run-time - In this thesis, defined in Section 1.2 as being able to execute at 30 frames per second Chapter 2 Related Work
This chapter details various works relevant to the area of landscape and terrain deformation as well as covers the details of rendering techniques that can be used as a basis for run-time mesh deformation.
2.1 Hardware Tessellation One method of achieving high-resolution meshes while keeping memory bandwidth low is to utilize hardware tessellation. Hardware tessellation is a feature introduced initially in the Direct3D 11 graphics API [26]. It allows for the generation of ad- ditional primitives at run-time through the use of patch primitives, giving a higher detail representation of the mesh than a triangle-based approach [22]. Patches are further subdivided into triangles, points, and lines. In the Direct3D 11 API these operations are performed in the following three pipeline stages:
1. Programmable Hull shader stage
2. Fixed-function Tessellator stage
3. Programmable Domain shader stage
The programmable hull shader stage is performed once per patch primitive and outputs a set of between 1 and 32 patch control points along with patch constant data, which can be used by the domain shader stage. Materials in Unreal Engine currently support two different types of tessellation through D3D11: Flat Tessellation and spline-based PN Triangles tessellation [30]. The flat tessellation behaves like regular tessellation, meaning it simply subdivides primitives to add geometric complexity. The spline-based PN Triangles tessellation makes use of smoothing groups defined in the mesh to also smooth out sharp edges, which may be a desired feature when utilizing tessellation for adding detail to an object. For displacement however, the more straight forward flat tessellation would be more suitable as no unintended alterations will be made to the mesh. One approach to the problem involves utilizing the 2D image space of a recti- linear grid of vertices. Aquilio A. et al. present a two-step render pass approach involving the use of a DDHM (Dynamically-Displaced Height Map) in conjunction with a generated offset map to create an incrementally deformed terrain structure [2]. The technique involves isolating the area of the terrain to be deformed through an examination of the bounding boxes of the intersecting objects. This is followed
3 4 Chapter 2. Related Work by positioning a camera perpendicularly to the ground plane directly below the area to be deformed. The first render pass is then executed using this camera, including the intersecting objects as well as the pre-deformed terrain, and finally storing the resulting depth values in a texture on the GPU. In the second render pass, this tex- ture is then sampled as an offset map for rendering the post-deformation terrain and then stored as the DDHM for use in consecutive frames.
Figure 2.1: Visual example of overlapping landscape components. Note the dupli- cated vertices [13].
2.2 Unreal Engine Landscape Systems
Landscapes are a feature introduced in Unreal Engine 4 as a successor to the ter- rain feature of Unreal Engine 3. The landscape editor provides tools to create large, spanning terrains for use in games while also implementing several performance op- timizations. These optimizations include techniques such as dynamic level of detail or DLOD, meaning the resolution of the mesh varies depending on the distance from the viewer, as well as a subdivision of the terrain mesh into smaller, uniformly sized components. Each component is handled as a separate draw call by the engine and can thus be regarded as it’s own individual mesh. Each component can optionally be divided into four landscape sections. These sections are the base unit of LOD calculations in the engine and thus, the level of detail is constant throughout a single landscape section. As each component is independent of each other, the vertices at each component edge overlap with the neighboring components as seen in Figure 2.1. Landscapes in Unreal use a heightmap-based rendering technique. The landscapes are created as a uniform grid of vertices on a simple plane, which then samples a texture containing displacement values in the vertical axis to accurately position each vertex. This is an effective technique as displacements can be stored correctly regardless of the resolution of the mesh. However, it also introduces some limitations such as the inability to accurately model non-vertical displacements such as caves. This is not necessarily a limitation as the concept of caves in a heightmap based terrain could, for example, be circumvented by the use of transparency on the terrain in combination with a separate 3D-mesh representing the cave. 2.3. Displacement mapping 5
Layered materials are another key feature of landscapes. Material layering allows for up to ten separate materials to be painted onto landscape components and blended through the use of blending techniques such as alpha or weight blending. The painted weight-map, or splat-map, textures can be exported and imported individually and may also contain additional information such as heightmap displacement, which can be utilized for height blended terrain textures [12].
2.3 Displacement mapping Displacement mapping is a technique that utilizes texture sampling to displace ver- tices on the GPU, allowing for detailed geometry to be rendered at various levels of details without requiring explicit modeling of each level. As displacement mapping modifies the positions of the vertices, it allows for a more accurate representation of geometric detail compared to using a technique such as normal mapping to simulate surface detail through lighting. Displacement mapping is primarily divided into two separate methods, intensity displacement, and vector displacement. Intensity displacement is the less memory demanding of the two as the displacement map texture is used to store a single intensity value, which is then sampled and used to displace each vertex along its precalculated normal. As intensity displacement only displaces vertices outwards, the result is a more smooth, continuous surface. Vector displacement instead stores displacement vectors in the texture, allowing vertices to be displaced in any direction but at the cost of three times the memory per displacement map texture, as opposed to intensity displacement [29].
Figure 2.2: Parallax mapping used on a wall texture in Bethesdas The Elder Scrolls IV: Oblivion [28].
2.4 Parallax mapping Parallax mapping, also known as “virtual displacement mapping” or “per-pixel dis- placement mapping,” is an alternative technique to traditional displacement map- ping. This technique involves adjusting the texture coordinates of pixels to give the illusion of depth [9] as seen in Figure 2.2. The geometry itself is therefore unaffected since this technique is performed entirely at the pixel shader stage after the geometry has been rasterized. This is a benefit in terms of performance as it is independent of 6 Chapter 2. Related Work the resolution of the initial mesh. However, a primitive application of the technique is unsuitable for large displacements such as terrain deformation and is primarily used for minor detailing on textures [17]. Chapter 3 Method
This chapter details the small literature study conducted to select a suitable terrain deformation method and then presents the theoretical details of the selected method.
3.1 Literature Study
This section covers the small literature study performed with the purpose of selecting a suitable terrain deformation method to be implemented and tested in Unreal Engine 4.23. The databases used for the small literature study are Google Scholar and the Blekinge Institute of Technology database Summon. Search results covering work inapplicable to real time rendering solutions in a game engine were discarded and not considered.
Google Scholar Search 1 Search Term real-time terrain deformation Filters Last 5 years Date searched 2020-03-04 Results 9130 Selected 1 [32]
Figure 3.1: Details of the first Google Scholar database search of the small literature study.
Ping Yu et al. present an interesting work involving implementation of a multi- scale dynamic terrain system suitable for application in real-time terrain rendering of local regions [32]. However, the work focuses primarily on the subdivision of the terrain mesh based on an existing deformation algorithm, which is not a focus of this research and a feature already supplied by Unreal Engine 4.23. This paper therefore does not provide a strong enough basis for the implementation in this thesis.
7 8 Chapter 3. Method
BTH Summon Search 1 Search Term real-time terrain displacement Filters Last 5 years & Subject terms: computer science Date searched 2020-03-02 Results 108 Selected 4 [20, 17, 19, 6]
Figure 3.2: Details of the first BTH Summon database search of the small literature study.
The database search presented in Figure 3.2 yielded four scientific works that were examined further.
The 2019 work of Lee and Shin proposes a method of achieving real-time land- scape visualization using hardware-based adaptive terrain meshes through temporal coherence [20]. While the work covers powerful optimization techniques for real-time terrain rendering, the main focus remains on the continuous rendering of the terrain mesh rather than the method of terrain deformation. Again, this covers areas not targeted by the research questions and is not sufficient as a basis for the implemen- tation.
González et al. cover explorations into combining the methods of displacement mapping and parallax mapping presented in Chapter 2 [17]. The work shows promis- ing results for a combination of these methods for improved visual results and reduced texture stretching and may warrant further exploration in Unreal Engine. However, as with the aforementioned work the target of this lies outside the scope of this thesis and is left for future work.
Lee et al. present an alternative displacement mapping approach with a focus on enhanced detailing [19]. Their work covers vertex relocation as a means of concentrat- ing a fixed amount of vertices in areas with high deformation to achieve high detail levels without the addition of extra triangles, as a tessellation based approach would. As this thesis however involves the landscape systems of Unreal Engine which rely on a rectilinear grid of vertices, this method would require a redesign of the module. For this reason, this method was not considered for implementation.
Further use cases of displacement mapping methods can be seen in the work of Condonnier et al., exploring interactive snow simulations [6]. The work presents an interesting application of the displacement mapping technique. However it focuses more heavily on fluid dynamics and the behaviour of the simulated snow, and does not cover any specific displacement mapping rendering technique that may be suitable for implementation. 3.2. Depth-based DDHM 9
BTH Summon Search 2 Search Term real-time heightmap displacement Filters Last 5 years Date searched 2020-03-04 Results 17 Selected 2 [22, 27]
Figure 3.3: Details of the second BTH Summon database search of the small litera- ture study.
The work of Nießner et al. covers the usage of hardware tessellation in combi- nation with displacement mapping techniques for real-time rendering [22]. While tessellation has the potential to increase the visual quality of the technique, it also heavily impacts performance in Unreal Engine 4.23 [16]. This is detailed further in Section 3.3. Further use case coverage of displacement mapping is provided in the work of Smelik et al., presenting application through the use of dynamic run-time terrain displacement in a military simulation environment [27]. While the work does provide a good basis for emphasis on the importance of such techniques, it does not cover any technicalities behind the techniques discussed, nor does it cover any performance aspects. The work is therefore not deemed suitable as a base for implementation.
BTH Summon Search 3 Search Term real-time dynamic terrain displacement Filters Subject terms: computer science Date searched 2020-03-11 Results 227 Selected 1 [2]
Figure 3.4: Details of the third BTH Summon database search of the small literature study.
Aquilio et al. present a work that covers a real-time simulation of dynamic terrain using a Dynamically-Displaced Height Map (DDHM) [2]. The method presented in their work utilizes simple, core graphics rendering elements that have corresponding components available in most commercial game engines, including Unreal Engine 4.23. The proposed deformation technique is shown to achieve real-time performance as well as present application areas within 3D-games, which covers the criterion for the research question of this thesis. This work is therefore selected as the primary basis of the implementation and thus, the small literature study is concluded.
3.2 Depth-based DDHM The DDHM based terrain displacement method presented by Aquilio et al. utilizes a few core rendering components that are available for use in Unreal Engine 4.23. The method involves using a secondary camera to render a depth-only pass of the terrain 10 Chapter 3. Method from below along with any deforming objects. The depth buffer image generated by the render pass can then be converted to a heightmap as the elevation values will correspond to depth values in the buffer.
Figure 3.5: The depth capture camera configuration for generation of a DDHM [1].
The initial setup of the technique places a secondary camera in the scene directly below the terrain. Once a depth render has been made of the landscape to a render target, the render target values need to be mapped to the values of the original heightmap used to render the terrain. This is done through the use of a conversion scalar, resulting in a linear transformation from the render target depth buffer space to heightmap space. After this has been established, further depth buffer renders of the terrain can be made with the inclusion of deforming objects. Any objects seen passing through the terrain will be rendering to the depth map render target by the camera. Then, once the render target depth buffer is converted into a heightmap through the use of the conversion scalar, the terrain will be deformed in the locations where the objects pass through the terrain. This process is visualised in Figure 3.6. 3.3. Visual Representation 11
(b) (a) Figure 3.6: The process of depth information being converted to a heightmap through the DDHM technique. (a) shows the object depth clipping through the terrain and being rendered to the depth capture render target. (b) shows the depth value data being used to repopulate the height map. [1]
3.3 Visual Representation
(a) (b) (c) Figure 3.7: Example of one iteration of the terrain deformation. (a) shows the original unmodified terrain. (b) shows the sphere mesh used for this deformation example, rendered for demonstration purposes. (c) shows the resulting deformation in the terrain.
Figure 3.7 shows an example of one iteration of deformation performed on a typical landscape in a game level. Note the minor errors in the heightmap in Figure 3.7c as a result of colour banding due to resolution discrepancy in the depth buffer. This is further shown in Figure 3.8 after multiple iterations since each subsequent iteration has the potential to introduce further errors. This resolution inaccuracy can potentially be mitigated through the use of hard- ware tessellation in combination with a higher resolution render target, as covered 12 Chapter 3. Method in Chapter 2. The tessellation options currently available in Unreal Engine 4.23 but introduce a rather large performance overhead [16]. Due to this, the implementation described in the following chapter will not include any usage of hardware tessellation, as it is not vital to the proposed research questions. Potential approaches to include this is discussed further in Chapter 8.
Figure 3.8: Example of height levels converging to set stages as a result of colour banding in the depth buffer. This example shows the result on the landscape after 30 iterations. Chapter 4 Implementation
This chapter describes the implementational details of the method presented in Sec- tion 3.2.
4.1 Deformation Actor The deformation actor is the object in the scene that controls the rendering of the depth-map as well as the updating of the height-map data in the landscape actor. The actor contains several components necessary for one iteration of the deformation algorithm.
4.1.1 Actor Initialization Before the first iteration of the deformation algorithm, the actor needs to be ini- tialized as demonstrated in Algorithm 2. The render target is created using the dimensions given by the BaseLandscape input actor. Memory for the necessary ar- rays is also reserved at this point. The original pre-deformation heightmap is saved as the heightmap may need to be reset in the future. At this point, the upper and lower value bounds of the heightmap are recorded. The DepthCaptureCamera ac- tor is then set up in a position directly below the origin of the landscape, rotated pointing upwards and rotated 180 degrees in the Z-axis as the original heightmap is generated as seen from above. A capture of the scene from the perspective of the DepthCaptureCamera is made to the render target and the upper and lower bounds of the values in the render target are recorded. The conversion scalar between the two ranges is then calculated by dividing the resolution of the original heightmap by the resolution of the resulting render target values.
4.1.2 Class Structure The class structure of the deformation actor contains a few core components. The primary camera capture actor utilized for capturing the depth image for DDHM generation is provided through the USceneCaptureComponent2D class. The actor also contains references to ALandscape and an instance of UTextureRenderTarget2D used to access the heightmap and the depth buffer render target. The actor also contains functions for each of the algorithms presented in this section, as well as functions to call these from an Unreal Engine-blueprint in order to access them at run-time. The full source code for the deformation actor is detailed in Appendix A.
13 14 Chapter 4. Implementation
(a) (b) Figure 4.1: One iteration of the implementation performed on a flat landscape, visualized in gray, deforming around a cube shaped mesh, visualized in orange. (a) shows the configuration before the deformation. (b) shows the result of one iteration of the implementation. 4.2. Engine Source Code Modifications 15
Algorithm 1: Importing the heightmap from the Render Target and con- verting to a readable heightmap. input : BaseLandscape, DepthCaptureCamera, HeightmapLowerBound, RenderTargetLowerBound, ConversionScalar, RenderTargetResource output: DeformedHeightValues /* Read the depth values from the Render Target */ 1 OutputRT Heightmap ← RenderTargetResource.ReadFloat16Pixels(); 2 foreach Height value h in OutputRTHeightmap do /* Convert the range in the Render Target to the range of the Heightmap */ 3 NewHeight = ((h − RenderT argetLowerBound) ∗ ConversionScalar) + HeightmapLowerBound; 4 ImportedHeightV alues.Add(NewHeight); 5 end /* As the values are captured from below, the heightmap is mirrored. We need to flip the values in the array */ 6 width ←RenderTargetResource.GetSizeX(); 7 height ←RenderTargetResource.GetSizeY(); 8 foreach row x in ImportedHeight do 9 foreach column y in ImportedHeight do 10 DeformedHeightValues.Add(ImportedHeightValues[width − 1 − x + (height − 1 − y) × width]); 11 end 12 end
4.2 Engine Source Code Modifications
Along with the implementation, a few modifications to the source code of Unreal Engine 4.23 had to be made to make use of a subset of functions during run-time as well as for testing purposes. These changes are confined to the landscape module and involve the removal of definition blocks containing WITH_EDITOR that limit the compilation of some functions to the editor-only build of the engine. Some necessary individual functions such as GetLandscapeInfo in the Landscape.cpp file also contain run-time checks for the variable GIsEditor. These checks have also been removed in the implementation. These landscape tools are intended by the developer to be used by the engine with the landscape editing functionality in the Unreal Engine editor. However, the code is capable of run-time execution and will execute without error as long as the definition blocks are removed to provide full initialization of the code at run-time [8]. Table 4.1 lists all occurrences of editor checks that were removed in the implementation. While as mentioned, the module is usable at run-time through these modifications, it may lack some optimizations that are otherwise found in code intended for run-time use. It is also worth noting that while no fatal errors were encountering during this 16 Chapter 4. Implementation
Algorithm 2: Initialization of the deformation actor. input : BaseLandscape, DepthCaptureCamera, DepthCameraDistance output: RenderTargetLowerBound, HeightmapLowerBound, ConversionScalar, RenderTargetResource 1 Origin, LandscapeBounds ← get extents of BaseLandscape; 2 create RenderTarget; 3 RenderTargetResource ← RenderT arget.GameT hread_GetRenderT argetResource(); 4 reserve memory for heightmap arrays; 5 BaseHeightmap ← original unmodified heightmap data; 6 foreach vertex v in BaseHeightmap do 7 if v > HeightmapHigherBound then 8 HeightmapHigherBound ← v; 9 else if v < HeightmapLowerBound then 10 HeightmapLowerBound ← v; 11 end /* DepthCaptureCamera setup */ 12 set DepthCaptureCamera projection mode to Orthographic; 13 set RenderTarget as DepthCaptureCamera texture target; 14 set DepthCaptureCamera render mode to ShowOnlyList; /* Position the DepthCaptureCamera */ 15 DepthCaptureCamera.ShowOnlyActors.Add(BaseLandscape); 16 DepthCaptureCamera.Position ← (Origin.x, Origin.y, DepthCameraDistance); 17 DepthCaptureCamera.Rotation ← (0, 90, 180); /* Render the scene to RenderTarget using the DepthCaptureCamera */ 18 DepthCaptureCamera.CaptureScene(); 19 HeightData ← ReadFloat16Pixels(RenderTarget); foreach vertex v in HeightData do 20 if v > RenderT argetHigherBound then 21 RenderT argetHigherBound ← v; 22 else if v < RenderTargetLowerBound then 23 RenderTargetLowerBound ← v; 24 end 25 RenderT argetResolution ← RenderT argetHigherBound ÷ RenderTargetLowerBound; 26 HeightmapResolution ← HeightmapHigherBound ÷ HeightmapLowerBound; 27 ConversionScalar ← HeightmapResolution ÷ RenderT argetResolution; 4.3. Engine Limitations 17
Algorithm 3: Update the landscape heightmap using values generated from Algorithm 1. input: DeformedHeightValues, BaseLandscape /* Create an instance of FHeightmapAccessor */ 1 HeightmapAccessor ← FHeightmapAccessor(BaseLandscape); 2 HeightmapAccessor.SetData(DeformedHeightValues); 3 clean up heightmap arrays;
implementation, run-time usage is not documented by Epic Games and may cause issues when utilized in methods not covered by this thesis. The UpdateHeightfieldRegion function in UpdateHeightfieldRegion function con- tains one line referencing editor only functionality. The height field reference variable RBHeightfieldEd was changed to instead utilize RBHeightfield which is available at run-time.
All occurrences of editor specific definition blocks and checks such as #if WITH_EDITOR and check(GIsEditor); were removed from the following files. Filepath Occurrences
Runtime/Landscape/Classes/Landscape.h 7 Runtime/Landscape/Classes/LandscapeInfo.h 1 Runtime/Landscape/Classes/LandscapeProxy.h 5 Runtime/Landscape/Private/Landscape.cpp 5 Runtime/Landscape/Private/LandscapeCollission.cpp 1 Runtime/Landscape/Private/LandscapeEdit.cpp 4 Runtime/Landscape/Private/LandscapeEditInterface.cpp 1 Runtime/Landscape/Public/LandscapeEdit.h 2
Table 4.1: All modified engine source code files.
The LandscapeEdit.h and LandscapeEditInterface.cpp files were also edited for testing purposes. In LandscapeEdit.h, the SetData function on line 348 was edited to always consider the bUpdateFoliage boolean to be false, as this research focuses on simply modifying the heightmap. Similarly in LandscapeEditInterface.cpp, the SetHeightData() function on line 220 was edited to always consider the InUpdate- Collision boolean to be false for a subset of the test cases. These aforementioned test cases are marked as “No Collision Update” in Section 5.
4.3 Engine Limitations
This section details various limitations of Unreal Engine 4.23 that impacted the efficacy of the method or altered the implementation. 18 Chapter 4. Implementation
4.3.1 Limitations of the USceneCaptureComponent2D The engine class USceneCaptureComponent2D is used to capture and generate the render target depth map that is later converted into a heightmap for the terrain deformation. As the precision of the depth buffer is determined by the distance be- tween the near and the far plane, demonstrated in Equation 4.1, which is derived from the orthographic projection matrix presented in Figure 4.2, it would be benefi- cial to limit these planes to enclose the scene as close-fitting as possible for optimal precision. However, as of Unreal Engine version 4.23, it is currently not possible to set a far plane distance in a USceneCaptureComponent2D manually. This limits the options for depth buffer precision to what the engine internally determines as an optimal far plane given the scene. We can manipulate this by varying the distance of the USceneCaptureComponent2D from the scene; this is covered further in the Results section. A simple solution to this problem would be to allow direct accessing of the view matrix used in the USceneCaptureComponent2D. This would enable the developer to freely change and define the variables used for the view transformation, such as the distance of the near and far planes. A further precision affecting limitation of the USceneCaptureComponent2D is the 16-bit depth of the depth buffer used for the render target. While the internal height map for the landscape is stored in 16-bit format, as mentioned, we do not have explicit control over the depth range due to the inability to access the far plane of the camera. Depth precision is therefore lost once it exceeds the areas covered by the height values in the landscape. This has the potential to be further impacted by floating-point precision inaccuracies in the depth buffer, particularly at distances close to the near plane [25, 23].
4.3.2 Limitations of the Landscape Module
2 0 0 0 right − left 2 0 0 0 top − bottom −2 0 0 0 (far − near) right + left top + bottom far + near − − − 1 right − left top − bottom (far − near) Figure 4.2: The orthographic projection matrix. The direction parameters left, right, top, bottom represent the size of the orthographic projection from the center. The parameters far and near represent the distance from the camera of the far and near planes respectively.
The landscape module API used to access and alter the heightmap of the land- scape mesh through the use of the FHeightmapAccessor interface does not provide explicit access to the device resource used by the landscape. This results in the im- plementation needing to fetch the resource from the device every iteration instead of performing operations directly on device data. This adds significant overhead to the method, and omission of this step would significantly improve the performance 4.3. Engine Limitations 19 because fewer data communications are needed between the CPU and the graphics device. The cost of this operation is covered in Section 5.1.3.
z − near z0 = 2 × − 1 (4.1) far − near
Chapter 5 Results
The experimental results in terms of heightmap accuracy and performance are pre- sented in this chapter. All of the following test cases include recalculation of the normals on the landscape mesh as per the default implementation in the landscape module. A subset of the test cases excludes the updating of the collision mesh. These cases are marked with “No Collision Update”. Each test case consists of a single Unreal Level, containing a single Landscape actor as well as the DeformableLandscape actor described in the previous section. Each function test is timed in C++ through the high resolution time stamps, and each individual test is executed through the editor calling the function in the De- formableLandscape actor. The tests are ran on three different sizes of landscape. Each landscape size con- tains a set number of landscape components, with the smallest size representing a single landscape component consisting of the default size of 255 by 255 vertices. Each further larger landscape represents an square increase in size from the center of the landscape, consisting of 3 by 3 and 5 by 5 landscape components respectively. These sizes were chosen to give an accurate representation of a simple square level layout of various sizes. As all of the operations are performed per mesh and vertex, the actual size and scaling of the components in unreal units is irrelevant, and performance will therefore remain the same regardless.
5.1 Experimental Results
This section covers the testing platform and the quantitative data gathered on the implementation.
5.1.1 Testing Environment All of the data presented was gathered from testing the implementation on a com- puter running Microsoft Windows 10 Pro Version 10.0.18363 Build 18363. The test- ing environment is comprised of a Intel Core i7-4770k at 3.5GHz, a Radeon Fury X with 4GB of video memory using video driver version 19.9.2, and 16GB of RAM at 1600Mhz. All measurements were taken using the FPlatformTime Unreal Engine in- terface, which internally uses the highest resolution timer available on the platform. For the testing platform, this utilizes the QueryHighPerformanceCounter() Win32 API function [11]. All tests were performed in a non-debug, developmental build of
21 22 Chapter 5. Results the engine. The developmental build configuration includes a majority of the opti- mizations applied to a full release build, while still including statistic and profiling tools otherwise not available in the “Shipping” release build configuration. [15]
5.1.2 Full Algorithm Performance
The execution times for one iteration of the full algorithm are presented in Table 5.1 and Table 5.2 for the cases involving collision updating and no collision updating respectively. A further breakdown of the data for full algorithm execution in the case of single component with collision updating disabled is shown in Figure 5.1.
No Collision Update, Single Component, Execution Time 26 24 22 20 18 16 time (ms) 14 12 10 8
Execution 6 4 2 Read RenderTarget Set Heightmap Data Full Function
Figure 5.1: Execution time of the algorithms performed on a single landscape com- ponent with no collision updating.
Execution time of the full function Landscape Size Average Max Min 1017x1017 144.9ms 160.4ms 109.5ms 763x763 80.37ms 116.9ms 58.86ms Single Component(255x255) 22.21ms 25.31ms 8.002ms
Table 5.1: The results of the execution time of a full iteration with collision updating enabled. 5.1. Experimental Results 23
Execution time of the full function - No Collision Landscape Size Average Max Min 1017x1017 141.2ms 200.7ms 132.1ms 763x763 73.10ms 119.2ms 38.32ms Single Component(255x255) 20.48ms 25.62ms 16.44ms
Table 5.2: The results of the execution time of a full iteration with collision updating disabled.
5.1.3 Render Target Import Performance
The execution times for the render target import detailed in Algorithm 1 are pre- sented in Table 5.3. As this involves a graphics device command and not any calls to the landscape API, the results are the same whether collision updating is enabled or not, therefore only one table is presented. It is also worth noting that the land- scape sizes listed corresponds to the sizes of the render targets in accordance with the implementation.
Execution time of the render target import Landscape Size Average Max Min 1017x1017 42.33ms 54.67ms 39.72ms 763x763 28.97ms 33.60ms 5.765ms Single Component(255x255) 16.94ms 19.32ms 5.044ms
Table 5.3: The results of the execution time of the render target import.
5.1.4 Heightmap Update Performance
Collision Update
Execution time of the heightmap update Landscape Size Average Max Min 1017x1017 103.5ms 124.2ms 96.79ms 763x763 42.13ms 61.09ms 38.68ms Single Component(255x255) 7.871ms 11.58ms 7.000ms
Table 5.4: The results of the execution time of Algorithm 3 with collision updating enabled. 24 Chapter 5. Results
Non-collision Update
Execution time of the heightmap update - No Collision Landscape Size Average Max Min 1017x1017 67.18ms 129.07ms 56.78ms 763x763 35.30ms 60.59ms 31.64ms Single Component(255x255) 3.38ms 4.29ms 3.22ms
Table 5.5: The results of the execution time of Algorithm 3 with collision updating disabled.
5.1.5 Average Error Multiple iterations of the method on the landscape mesh can introduce an error due to insufficient resolution in the depth buffer, as shown in Figure 3.8. The data presented in Figure 5.2 shows the progression of the average error per heightmap vertex compared to the original heightmap over the course of multiple iterations. The depth capture camera is placed at various distances in the Z-axis away from the origin of the landscape actor.
600 0 1000 5000 500 10000 20000 40000 400 50000 80000 300
200
100
0
verage Error per Vertex (Unreal Units) 1 2 3 4 5 6 7 8 9 10 A Iterations
Figure 5.2: Average error distance per vertex with the DepthCaptureCamera placed with the near plane at various distances from the landscape mesh.
5.1.6 Conversion Resolution Scalar Due to the resolution discrepancy between the depth buffer in the DepthCapture- Camera in comparison to the original resolution of the heightmap array as described in Section 4.3.1, a conversion scalar is needed to convert depth values to convert these values into usable height values for the DDHM technique. Table 5.6 shows the resolution discrepancy of the depth buffer heightmap at various distances of the DepthCaptureCamera near plane from the landscape actor. 5.1. Experimental Results 25
Resolution Conversion Scalar - Heightmap Resolution 2910 Distance from Near Plane Render Target Resolution Scalar 0 1576 1.846 1000 1050 2.771 5000 537 5.419 10000 281 10.36 20000 140 20.79 40000 70 41.57 50000 70 41.57 80000 32 90.94
Table 5.6: Depth buffer heightmap resolution and resolution conversion scalar at various distances of the DepthCaptureCamera.
Chapter 6 Analysis and Discussion
This chapters covers the results presented in Chapter 5 and provides a basis for conclusions of the stated research questions.
6.1 DDHM The DDHM run-time deformation technique was shown to be fully implementable in the engine in Chapter 4. Regarding performance, the technique achieved run-time performant deformation on single landscape components, displaying an average frame time of 20.48 millisec- onds, enough to fulfill the criterion of 33.33 milliseconds required for a frame rate of 30 frames per second. The performance of the reading of the render target, however, sets a bottleneck for the smaller landscape sizes accounting for roughly 83% of the average frame time in the single component case. This is in contrast to the larger landscape sizes where the heightmap update takes up the majority of the execution time, see Section 5.1.2. The technique suffers from accuracy problems as shown in Figure 5.2. Due to the inability to explicitly set a far plane for the USceneCaptureComponent2D used in the implementation, the technique cannot fully utilize the Z-range of the depth buffer. Further precision issues may be attributed to floating point inaccuracies as the bit depth of the depth buffer is limited to 16-bits, as covered in Section 4.3. This may also be a contributing factor to the seemingly high inaccuracies observed at distance 0 from the near plane. Due to this, more explicit control of the depth capture camera, as well as the depth buffer, is needed to fully utilize this method, and reduce the resulting error. The error can be mitigated by adjusting the distance of the component from the actors involved, however, it cannot be fully omitted.
6.2 Generic Run-time Deformation The execution times for a generic technique-irrelevant heightmap update are pre- sented in Table 5.4 and Table 5.5 respectively. This data shows promising results for dynamic run-time heightmap updates of individual landscape components. The execution times exceeding 33.33ms for the larger, full-landscape updates rule out the feasibility of large scale deformation, however individual components can be con- structed to provide local deformations in areas where the player may be active. As components are handled as individual meshes in the engine, the possibility exists
27 28 Chapter 6. Analysis and Discussion for threading and streaming of the updates, which may provide feasible larger scale deformation.
6.3 Sustainability As of the writing of this thesis, the most recent publicly released version of the Unreal Engine game engine is version 4.25, two revisions ahead of the 4.23 version used in the implementation. These revisions, however, contain no major changes to the landscape module that would affect the implementation in any way. Further developments of the engine and the underlying graphics APIs may im- prove the performance of the implementation. As discussed in the previous sections, an optimization of render target read times will significantly improve performance in the case of the individual landscape component, as the majority of the execution time is spent on the render target read.
6.4 Validity Threats One possibly validity threat could be that the specific version of the engine used contains parameters that affect the results in some way. As stated together with the formation of the research questions, this validity threat has been attempted to be mitigated through the analysis of newer versions of the engine. It was concluded that the landscape modules do not contain any significant changes in the newest version of the engine, however, as the tests have not been ran on this version the possibility cannot be fully dismissed. While the testing platform was shown to be equivalent to an average user platform at the time of this thesis, it also cannot be fully excluded the possibility that this specific platform used has specifications that would in some way directly affect the validity of the results. Chapter 7 Conclusions
This thesis aimed to investigate the viability of achieving run-time terrain deforma- tion in the Unreal Engine 4.23 game engine, as well as the feasibility of implementing said technique at a performant level, defined as being able to maintain a frame rate of 30 frames per second. The following research questions were posed:
1. Does a run-time terrain deformation technique for arbitrary mesh deformation exist that can be implemented in Unreal Engine 4.23?
2. How well can we achieve run-time performant terrain deformation in Unreal Engine 4.23? The identified Dynamically-Displaced Height Map technique presented in Chapter 3 was shown to be fully implementable and feature-complete in Unreal Engine 4.23, answering research question 1. However, the implementation suffers from accuracy problems due to limitations of the game engine resulting in limited precision of the depth buffer. Details regarding the inaccuracy are presented in Section 5.1.5. Research question 2 touches on the practical feasibility of run-time performant terrain deformation in the engine in general. Rum-time performant was defined in Section 1.2 as being able to achieve a steady frame rate of at least 30 frames per second. The evaluated Dynamically-Displaced Height Map technique was shown to achieve run-time performant execution times sustainable for a frame rate of 30 frames per second when executed on single landscape components. In this test case, the execution time of the technique-specific render target import was shown to be the primary limiting factor, with the generic case operations constituting a fraction of the frame time. Run-time performant terrain deformation is therefore concluded to be achievable in Unreal Engine 4.23 for individual landscape components, how- ever caution should be exercised when performing deformations on large, full-scale landscapes.
29
Chapter 8 Future Work
For future work, there are many potential routes to further explore this area. This work covers the exploration of the viability of dynamic run-time terrain in Unreal Engine 4.23 using one identified technique, however, there are likely many more which may be more suitable for application in a game. As the specific implementation is limited by certain factors related to the engine itself, covered partially in Section 4.3, future versions of Unreal Engine may enable for further improvements in some areas. These may include things such as an im- provement in depth map resolution by having access to alter the near plane distance of the depth capture camera. A further way to improve the implementation would be to look at a solution that performs all operations directly on the render target, which would eliminate the overhead of a queued render target read operation. This would significantly improve the performance of the algorithm as the primary execution time cost for minor deformations in the terrain is the reading of the render target, as seen in Chapter 5. As mentioned in Chapter 2, the technique can be expanded upon through the use of tessellation with a higher resolution height map and depth buffer. This provides further detail and higher precision of more finely detailed deformations at the cost of the performance overhead of including the tesselation stage in the pipeline. With the introduction of Turing mesh shaders to the DirectX 12 API, more opportunities for expansion of the technique are possible [18]. As mesh shaders can decompose large meshes into meshlets, the landscape can be handled as a large entity rather than the separate components as in the current implementation, which may provide further performance improvements. The mesh shader pipeline is also vastly different to the more traditional pipeline and will most likely require alterations to the implementation presented in this thesis.
31
References
[1] Anthony S. Aquilio, Jeremy C. Brooks, Zhu Ying, and G. Scott Owen. Real- time gpu-based simulation of dynamic terrain. In Lecture Notes in Computer Science (LNCS), volume 4291, page 894, 2006. [2] Anthony S. Aquilio, Jeremy C. Brooks, Zhu Ying, and G. Scott Owen. Real- time gpu-based simulation of dynamic terrain. In Lecture Notes in Computer Science (LNCS), volume 4291, pages 891–900, 2006.
[3] Brian Ashcraft. How cover shaped gaming’s last decade. Accessed at: https:// kotaku.com/how-cover-shaped-gamings-last-decade-5452654, Jan 2010. [4] X. Chen and Y. Zhu. Shader based polygon stitching and its application in deformable terrain simulation. In 2011 Sixth International Conference on Image and Graphics, pages 885–890, Aug 2011. [5] Mark Claypool, Kajal Claypool, and Feissal Damaa. The effects of frame rate and resolution on users playing first person shooter games. Proceedings of SPIE - The International Society for Optical Engineering, 6071, 01 2006. [6] G. Cordonnier, P. Ecormier, E. Galin, J. Gain, B. Benes, and M. -. Cani. In- teractive generation of time-evolving, snow-covered landscapes with avalanches. Computer Graphics Forum, 37(2):497–509, 2018. [7] Valve Corporation. Steam hardware & software survey: April 2020. Accessed at: https://store.steampowered.com/hwsurvey/ Steam-Hardware-Software-Survey-Welcome-to-Steam, May 2020.
[8] Dark-Veil. 26495-terrain editing in runtime. Accessed at: https: //forums.unrealengine.com/unreal-engine/feedback-for-epic/ 26495-terrain-editing-in-runtime?p=295744#post295744, Apr 2015. [9] EricChadwick, Noors, Cheeseplus, and Caustic. Parallax map. Accessed at: http://wiki.polycount.com/wiki/Parallax_Map, Feb 2017. [10] Alexander Frisk. Real-time vehicle trails in deformable terrain. Master’s thesis, Umeå University, Department of Computing Science, 2017.
[11] Epic Games. Fplatformtime. Accessed at: https://docs.unrealengine.com/ en-US/API/Runtime/Core/Unix/FPlatformTime/index.html.
[12] Epic Games. Landscape materials. Accessed at: https://docs.unrealengine. com/en-US/Engine/Landscape/Materials/index.html.
33 34 References
[13] Epic Games. Landscape technical guide. Accessed at: https://docs. unrealengine.com/en-US/Engine/Landscape/TechnicalGuide/index.html.
[14] Epic Games. Unreal units. Accessed at: https://docs.unrealengine.com/ udk/Three/UnrealUnits.html, 2012.
[15] Epic Games. Build configurations reference. Accessed at: https://docs.unrealengine.com/en-US/Programming/Development/ BuildConfigurations/index.html, May 2020.
[16] Epic Games. Performance guidelines for artists and designers. Ac- cessed at: https://docs.unrealengine.com/en-US/Engine/Performance/ Guidelines/index.html, May 2020.
[17] Cesar González, Mariano Pérez, and Juan M. Orduña. Combining displacement mapping methods on the gpu for real-time terrain visualization. The Journal of Supercomputing, 73(1):402–413, 2017.
[18] Christoph Kubisch. Introduction to turing mesh shaders. Accessed at: https: //devblogs.nvidia.com/introduction-turing-mesh-shaders/, Sep 2017.
[19] Eun-Seok Lee, Jin-Hee Lee, and Byeong-Seok Shin. Vertex relocation: a feature- preserved terrain rendering method for pervasive computing environments. Mul- timedia Tools and Applications, 75(22):14057–14073, 2016.
[20] Eun-Seok Lee and Byeong-Seok Shin. Hardware-based adaptive terrain mesh using temporal coherence for real-time landscape visualization. Sustainability, 11(7):2137, 2019.
[21] Jamie Madigan. The psychology of video game immersion. Accessed at: https://www.psychologytoday.com/us/blog/mind-games/201207/ the-psychology-video-game-immersion, Jul 2012.
[22] M. Nießner, B. Keinert, M. Fisher, M. Stamminger, C. Loop, and H. Schäfer. Real-time rendering techniques with hardware tessellation. Computer Graphics Forum, 35(1):113–137, 2016.
[23] Outerra. Floating point depth buffer. Accessed at: https://outerra. blogspot.com/2009/12/floating-point-depth-buffer.html, Dec 2009.
[24] Jesper Persson. Volume-preserving deformation of terrain in real-time. Master’s thesis, Linköping University, Information Coding, 2019.
[25] Nathan Reed. Depth precision visualized. Accessed at: https://developer. nvidia.com/content/depth-precision-visualized, Jul 2015.
[26] M. Satran, M. Jacobs, and D. Batchelor. Direct3d 11 features - win32 apps. Accessed at: https://docs.microsoft.com/en-us/windows/win32/ direct3d11/direct3d-11-features?redirectedfrom=MSDN. References 35
[27] Ruben Smelik, Freek van Wermeskerken, Robbert Krijnen, and Frido Kuijper. Dynamic synthetic environments: a survey. The Journal of Defense Modeling and Simulation, 16(3):255–271, 2019. [28] Bethesda Softworks. Bethesda.net. Accessed at: http://www.bethsoft.com/.
[29] Kenneth Styrberg. Displacement maps. Accessed at: https://download. blender.org/documentation/htmlI/ch11s03.html, Aug 2005. [30] Alex Vlachos, Jörg Peters, Chas Boyd, and Jason L. Mitchell. Curved pn tri- angles. In Proceedings of the 2001 Symposium on Interactive 3D Graphics, I3D ’01, page 159–166, New York, NY, USA, 2001. Association for Computing Ma- chinery.
[31] Ewan Wilson. A history of environmental destruction. Accessed at: https: //thegamesedge.com/602/a-history-of-environmental-destruction/#. g7DzjgrMBa, Sep 2018. [32] Ping Yu, Tao Xu, Bo Zheng, and Yanfeng Zhang. Research and implementation of multiscale dynamic terrain. In 2018 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), pages 217–222. IEEE, 2018.
Appendix A Deformation Actor Source Code
DeformableTerrainActor.h
#pragma once
#include "RomeMinimal.h" #include "GameFramework/Actor .h" #include "Components/SceneCaptureComponent2D.h" #include "Landscape.h" #include "LandscapeEdit.h" #include "LandscapeProxy.h" #include "LandscapeInfo.h" #include "LandscapeEditorUtils.h" #include "Engine/TextureRenderTarget2D.h" #include "../Foliage/FoliageClasses.h" //#include "InstancedFoliageActor.h" #include "DeformableTerrainActor.generated.h"
UCLASS( ) class ROME_API ADeformableTerrainActor : public AActor { GENERATED_BODY( )
p u b l i c : // Sets default values for this actor’s properties ADeformableTerrainActor ();
UFUNCTION( BlueprintPure , Category = "Deformable Terrain", DisplayName = "Get Deformable Terrain Actor", meta = (WorldContext = "WorldContextObject", UnsafeDuringActorConstruction = "true")) ADeformableTerrainActor ∗ Get ( ) ;
UFUNCTION(BlueprintCallable , CallInEditor) void EditorInit();
UFUNCTION( BlueprintCallable )
37 38 Appendix A. Deformation Actor Source Code
void ImportHeightFromRenderTarget ();
UFUNCTION( BlueprintCallable ) void ResetHeightmapToDefault();
protected : UPROPERTY(VisibleAnywhere , Category = "Base Settings") USceneCaptureComponent2D∗ DepthCaptureCamera ;
UPROPERTY(EditAnywhere , Category = "Base Settings") ALandscape∗ BaseLandscape ;
UPROPERTY(EditAnywhere , Category = "Base Settings") UTextureRenderTarget2D∗ HeightmapRenderTarget ;
// Called when the game starts or when spawned virtual void BeginPlay() override;
virtual void EndPlay( const EEndPlayReason::Type EndPlayReason) override ;
p r i v a t e : TArray
float scaler = 1; int32 RTDepthLowerBound = 0; int32 HeightmapLowerBound = MAX_int32;
void ResetHeightMap(); void Init(); long double CalculateError(TArray
p u b l i c : // Called every frame virtual void Tick(float DeltaTime) override;
}; 39
DeformableTerrainActor.cpp
#include "DeformableTerrainActor.h" #include "Materials/MaterialInstanceDynamic.h" #include "LandscapeStreamingProxy.h" // Only for testing/logging purposes #include
// Sets default values ADeformableTerrainActor :: ADeformableTerrainActor() { PrimaryActorTick.bCanEverTick = false ; DepthCaptureCamera = CreateDefaultSubobject
ADeformableTerrainActor ∗ ADeformableTerrainActor ::Get() { return this; }
void ADeformableTerrainActor :: EditorInit() { I n i t ( ) ; }
void ADeformableTerrainActor :: ImportHeightFromRenderTarget() { uint64 startTime; uint64 endTime; ALandscape∗ Landscape = BaseLandscape; ULandscapeInfo ∗ LandscapeInfo = Landscape−>GetLandscapeInfo (); HeightData .Empty();
startTime = FPlatformTime:: Cycles64(); RenderTargetResource−>ReadFloat16Pixels( OutputRTHeightmap , CubeFace_MAX ) ; for (FFloat16Color Color : OutputRTHeightmap) { uint16 Height = (uint16)((Color.R.Encoded − RTDepthLowerBound) 40 Appendix A. Deformation Actor Source Code
∗ scaler) + HeightmapLowerBound; HeightData.Add(Height ); }
TArray
FHeightmapAccessor
HeightmapAccessor.SetData(MinX, MinY, MaxX, MaxY, FlippedHeightData.GetData()); long double error = CalculateError(FlippedHeightData); // Clean up arrays HeightData .Empty(); FlippedHeightData .Empty(); OutputRTHeightmap.Empty(); }
void ADeformableTerrainActor :: ResetHeightmapToDefault() { ResetHeightMap (); }
// Called when the game starts or when spawned void ADeformableTerrainActor :: BeginPlay() { Super :: BeginPlay();
I n i t ( ) ; }
void ADeformableTerrainActor ::EndPlay( const EEndPlayReason ::Type EndPlayReason) { 41
Super :: EndPlay(EndPlayReason); }
void ADeformableTerrainActor :: ResetHeightMap() { ULandscapeInfo ∗ LandscapeInfo = BaseLandscape−>GetLandscapeInfo (); FHeightmapAccessor
void ADeformableTerrainActor :: Init () { ULandscapeInfo :: RecreateLandscapeInfo(GetWorld() , false ); ULandscapeInfo ∗ LandscapeInfo = BaseLandscape−>GetLandscapeInfo (); LandscapeInfo−>GetLandscapeExtent( MinX, MinY, MaxX, MaxY) ;
RenderTargetResource = HeightmapRenderTarget−> GameThread_GetRenderTargetResource ();
SampleRect = FIntRect(0, 0, FMath::Min(1 + MaxX − MinX, HeightmapRenderTarget−>SizeX ) , FMath : : Min(1 + MaxY − MinY, HeightmapRenderTarget−>SizeY ) ) ;
// Make sure arrays are clean OriginalHeightData .Empty(); HeightData .Empty(); OutputRTHeightmap.Empty();
// Reserve memory for heightmap arrays int32 HeightmapSize = SampleRect.Width() ∗ SampleRect.Height (); OriginalHeightData .Reserve(HeightmapSize); OutputRTHeightmap. Reserve(HeightmapSize ); HeightData.Reserve(HeightmapSize );
FHeightmapAccessor
HeightmapLowerBound = MAX_int32; int32 HeightmapHigherBound = MIN_int32; for (int i = 0; i < HeightmapSize; ++i) { auto a = OriginalHeightData.GetData()[ i ]; if (a > HeightmapHigherBound) { HeightmapHigherBound = a; } if (a < HeightmapLowerBound) { HeightmapLowerBound = a; } }
// Place the camera under the terrain FTransform ModifiedTransform = GetTransform(); FVector Origin , BoundsExtent; BaseLandscape−>GetActorBounds( false , Origin , BoundsExtent); DepthCaptureCamera−>OrthoWidth = BoundsExtent.X ∗ 2 ; SetActorTransform(ModifiedTransform ); RenderTargetResource−>ReadFloat16Pixels( OutputRTHeightmap , CubeFace_MAX ) ; int32 RTHeightmapLowerBound = MAX_int32, RTHeightmapHigherBound = MIN_int32; for (FFloat16Color Color : OutputRTHeightmap) { uint16 Height = (uint16)Color.R.Encoded; HeightData.Add(Height ); if ((uint16)Color.R.Encoded > RTHeightmapHigherBound) { RTHeightmapHigherBound = (uint16)Color .R.Encoded; } if ((uint16)Color.R.Encoded < RTHeightmapLowerBound) { RTHeightmapLowerBound = (uint16)Color .R.Encoded; } }
int32 temp = RTHeightmapHigherBound − RTHeightmapLowerBound ; int32 temp2 = HeightmapHigherBound − HeightmapLowerBound ; RTDepthLowerBound = RTHeightmapLowerBound; scaler = (float)temp2 / (float)temp; } 43
long double ADeformableTerrainActor :: CalculateError( TArray
// Called every frame void ADeformableTerrainActor ::Tick(float DeltaTime) { Super :: Tick(DeltaTime); }
Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden