RAGE: A PROGRAMMABLE SHADER-BASED OPENGL
RENDERER AND SCENE GRAPH
A Project
Presented to the faculty of the Department of Computer Science
California State University, Sacramento
Submitted in partial satisfaction of the requirements for the degree of
MASTER OF SCIENCE
in
Computer Science
by
Raymond L. Rivera Albaladejo
SPRING 2017 © 2017
Raymond L. Rivera Albaladejo
ALL RIGHTS RESERVED
ii RAGE: A PROGRAMMABLE SHADER-BASED OPENGL
RENDERER AND SCENE GRAPH
A Project
by
Raymond L. Rivera Albaladejo
Approved by:
______, Committee Chair V. Scott Gordon, PhD
______, Second Reader Pinar Muyan-Ozcelik, PhD
______Date
iii Student: Raymond L. Rivera Albaladejo
I certify that this student has met the requirements for format contained in the University format manual, and that this project is suitable for shelving in the Library and credit is to be awarded for the project.
______, Graduate Coordinator ______Jinsong Ouyang, PhD Date
Department of Computer Science
iv Abstract
of
RAGE: A PROGRAMMABLE SHADER-BASED OPENGL
RENDERER AND SCENE GRAPH
by
Raymond L. Rivera Albaladejo
Computer applications are generally constrained to working only on specific platforms because they are engineered to rely on their services directly. The complexities of computer games only magnify and propagate this problem into different domains. To avoid limitting computer games to specific platforms, such as Microsoft Windows or GNU/Linux, an intermediate service framework is proposed to decouple the game applications from their host operating systems.
While such frameworks already exist, they are almost exclusively built for the professional software engineer. This often makes their scope, licensing, complexity, and cost, prohibitive factors in the context of a single semester for an undergraduate course. The purpose of this report is to describe a new framework, called RAGE1, that is based on current technology, significantly simpler than professional suites, freely available with a freedom-respecting license, compatible with Microsoft Windows and GNU/Linux, and promotes student access, contributions, and collaboration.
1 RAGE has several meanings such as Raymond's "Awesome" Game Engine and, recursively, RAGE replAces saGE, among others.
v Although instructors have a framework developed in-house, called the Simple Adaptable
Game Engine (SAGE), it relies on deprecated technology, is bound to the Microsoft Windows platform, and is released under a proprietary license, which prevents direct student access to the original source code –limiting their ability to study, modify, and share the code itself.
RAGE, however, supports both Microsoft Windows and GNU/Linux platforms and is itself released under a libre/free-software license, granting everyone the right to access, study, modify, and share the source code, all of which are expected to make a positive impact for interested users –especially for students and their educational experience.
______, Committee Chair V. Scott Gordon, PhD
______Date
vi DEDICATION
To the one and only true God, who always keeps his word.
“Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee whithersoever thou goest.” –Joshua 1:9 (KJV) ✝
I dedicate my master's project to my mother, Elsa N. Albaladejo � � � , who taught me the value of education, wisdom, and knowledge from a very early age and encouraged me to always work my way toward greater goals. I could not have gotten this far without her infinite amount of support –or an interest in games that can be traced all the way back to the 8-bit NES console she got me as a Christmas gift when I was 5 years old, which continues to work to this day � . She had no idea it would have serious and irreversible side-effects, such as game engine development... � , but I am forever grateful.
I also dedicate this project to my sister, Frances J. Rivera � � � , with whom I played many games for countless hours including Super Mario Kart for SNES, Command and Conquer:
Red Alert Retaliation for PS1 with a cross-over cable for our first proper "LAN Party", and her all-time favorite NES title, Bubble Bobble, one of several games that came to be the digital embodiment of the word "rage", with 100 traps levels that will trick you into getting stuck so that
vii you are forced to commit seppuku while it continues to play cute tunes in the background, and that, as its final act of hate, would also refuse to reveal it's "true" ending unless it was completed in two-player mode... � �
While contemplating the prospect of moving to California several years ago, she was one of the persons I spoke to. At one point during that conversation she said "I'd do it". It is, in part, her fault that you are reading this now �
I also dedicate this work to my immediate family and close friends who have supported me over the years, including those who participate in the LAN Parties I have been hosting every year at home for over a decade. Thank you and keep gaming � � � �
viii ACKNOWLEDGEMENTS
I thank my advisor, Dr. Scott Gordon, for his support in this project, and particularly for his feedback on early drafts of this report and associated presentation slides. His help clarifying some of the transformation matrices early on, as well as his time, which we sometimes spent on extended discussions, was greatly appreciated. I am also thankful for his support of my choice of software license for RAGE and its associated documentation.
I thank my secondary reader, Dr. Pinar Muyan-Ozcelik, for reviewing a draft of this report and providing feedback. Her comments and suggestions were helpful in improving its overall clarity, particularly in early, more introductory, chapters.
I thank my sister, Frances J. Rivera, for reading early drafts and checking for typos, formatting, and consistency issues. I also appreciate my friend, Hector I. Nieves, for his time and feedback on early versions of the presentation slides, and particularly for smoothing out the hard shadows in the last slide's image. My GIMP-fu will never be as strong as his Photoshop-fu.
I also thank users Sven Gothel, Xerxes Rånby, and jmaasing for some of the help I received in the forums while coming to grips with some aspects of JOGL and for accepting my tessellation shader feature request and merging my test case contributions into the main tree.
Finally, I thank the StackOverflow, GameDevelopment, Math, and ComputerGraphics sub-communities of the wider StackExchange network for their assistance in some of the questions I had during the course of this project.
ix TABLE OF CONTENTS
Page
Dedication...... vii
Acknowledgements...... ix
List of Tables...... xvi
List of Figures...... xvii
List of Code Listings...... xxi
Chapter
1. INTRODUCTION...... 1
1.1. Games as Regular Applications...... 1
1.2. Operating System Considerations...... 1
1.3. Hardware Considerations...... 2
1.4. The Problem with Variation...... 2
2. GENERAL DESIGN GOALS...... 4
2.1. Loose Coupling Between Components...... 4
2.1.1. Decoupling Game Clients from Scene Management...... 6
2.1.2. Decoupling Scene Structure from its Contents...... 7
2.1.3. Decoupling Scene Management from Rendering...... 8
2.1.4. Decoupling Assets, Loading, and Management...... 9
2.1.5. Uniformity Across Asset Managers and Loaders...... 10
2.1.6. Improving Asset Management Efficiency...... 10
2.1.7. Decoupling Assets and Scene Objects from States...... 11
2.1.8. Replacing Hard-Coded Paths for Configuration Files...... 12
x 2.2. Singleton Objects without the Singleton Design Pattern...... 12
2.2.1. Global Objects are No Better than Global Variables...... 12
2.2.2. Global Data is Prone to Race Conditions...... 13
2.2.3. Dependencies Among Objects Not Reliably Communicated...... 13
2.2.4. Tight Coupling Between Classes is Promoted...... 14
2.3. Rendering with Programmable Shaders...... 15
2.4. Choosing a Free-Software License for Source Code...... 15
3. HIGH-LEVEL FRAMEWORK OVERVIEW...... 17
3.1. General Framework Architecture...... 17
3.2. Game Client Architecture...... 18
3.3. Package Organization Overview...... 21
3.4. Scene Management and Organization...... 23
3.5. Graphics Processing...... 24
3.6. Asset Loading and Management...... 25
3.7. User Input Handling...... 25
4. DETAILED OVERVIEW OF FRAMEWORK COMPONENTS...... 26
4.1. The Engine Class...... 26
4.2. Scene Management and the Scene Package...... 27
4.2.1. Scene Management Architecture...... 27
4.2.2. The Scene Manager...... 28
4.2.3. The Scene Object...... 30
4.2.4. The Scene Node...... 31
4.2.5. Entities and Manual Objects...... 33
xi 4.2.6. Sub-Entities and Manual Object Sections...... 36
4.2.7. Cameras and Viewing Frustums...... 38
4.2.8. Global and Local Scene Illumination...... 40
4.2.9. Distant Backgrounds and Sky Boxes...... 42
4.2.10. Node Controllers in a Scene...... 43
4.3. Asset Management and the Asset Package...... 43
4.3.1. Asset Management Architecture...... 43
4.3.2. Assets...... 45
4.3.3. Asset Managers...... 46
4.3.4. Asset Loaders...... 48
4.3.5. Materials...... 49
4.3.6. Meshes...... 49
4.3.7. Shaders...... 50
4.3.8. Textures...... 51
4.4. Scene Rendering and the Render System Package...... 53
4.4.1. Rendering System Architecture...... 53
4.4.2. The Rendering System...... 54
4.4.3. Rendering Windows...... 55
4.4.4. Viewports...... 56
4.4.5. Renderables...... 57
4.4.6. Rendering Queues...... 59
4.5. Render Systems and Render States...... 60
4.5.1. Render State Architecture...... 60
xii 4.5.2. Render States...... 61
4.5.2.1. Z-Buffer States...... 61
4.5.2.2. Texture States...... 62
4.5.2.3. Front Face States...... 63
4.6. Rendering Systems and GPU Shader Programs...... 64
4.6.1. GPU Shader Program Architecture...... 64
4.6.2. GPU Shader Programs...... 65
4.6.3. Program Stages...... 66
4.6.3.1. Vertex Shaders...... 66
4.6.3.2. Fragment Shaders...... 67
4.6.3.3. Lighting and Shading Models...... 68
4.6.4. Program Inputs Attributes and Uniforms...... 69
4.6.5. Program Context...... 71
5. EXTERNAL FRAMEWORK DEPENDENCIES...... 72
5.1. Vector, Matrix, and Quaternion Math with RML...... 72
5.1.1. RML Package Architecture...... 73
5.1.2. Vectors...... 74
5.1.3. Matrices...... 75
5.1.4. Quaternions...... 77
5.1.5. Angles...... 78
5.2. Writing Test Cases with the TestNG Unit-Testing Framework...... 80
6. FUTURE WORK...... 81
6.1. Multiple Light Sources...... 81
xiii 6.2. User Input Devices and Management...... 82
6.3. Sound Processing...... 82
6.4. Animations...... 82
6.5. Networked Multi-Player...... 83
6.6. Bounding Volumes...... 83
6.7. Collision Detection and Rigid-Body Physics...... 84
6.8. Space Partitioning Scene Managers and Visibility Determination...... 84
6.9. Graphics APIs: Moving from OpenGL to Vulkan...... 84
6.10. Artificial Intelligence and Behavior Trees...... 85
6.11. Continued Maintenance, Optimizations, Testing, and Debugging...... 85
6.12. Adding New Features and Refactoring for Improvement...... 86
6.13. Specific Areas in Need of More Immediate Attention...... 86
6.13.1. Scene Rendering and Shader Improvements...... 86
6.13.2. Scene Management Improvements...... 87
Appendix A. Sample Scenes Rendered with RAGE...... 89
Appendix B. Source Code Repositories...... 99
Bibliography...... 100
xiv LIST OF TABLES
Tables Page
2.1. Brief summary of the SOLID acronym...... 5
4.1. SceneManager methods for other concrete SceneObject types...... 29
4.2. Key convenience methods in the ManualObject interface...... 35
4.3. The different Light.Types supported in RAGE...... 42
xv LIST OF FIGURES
Figures Page
1.1. RAGE decouples clients from the underlying platforms...... 3
2.1. High-level system architecture...... 5
3.1. High-level client/engine architecture and functionality...... 18
3.2. The Game interface...... 18
3.3. Basic client architecture...... 19
3.4. The AbstractGame class...... 20
3.5. The BaseGame class...... 20
3.6. High-level framework package organization and collaborations...... 22
3.7. Scene organization using a hierarchical scene graph...... 24
4.1. The Engine class...... 26
4.2. Interfaces and collaborations of the top-level scene package...... 28
4.3. The SceneManager interface...... 30
4.4. The SceneObject interface...... 31
4.5. The Node and SceneNode interfaces...... 32
4.6. The Entity interface...... 34
4.7. The ManualObject interface...... 35
4.8. The SubEntity interface...... 37
4.9. The ManualObjectSection interface...... 38
4.10. The Camera and Frustum interfaces with other dependencies...... 39
4.11. The AmbientLight interface...... 40
xvi 4.12. The Light interface...... 41
4.13. The SkyBox interface...... 42
4.14. Interfaces and collaborations of the top-level asset package...... 44
4.15. Structural pattern between top-level asset and example concrete implementation.. 45
4.16. The Asset interface...... 46
4.17. The AbstractAsset class...... 46
4.18. The AssetManager interface...... 47
4.19. The generic AbstractAssetManager class and its only protected abstract method..47
4.20. The generic AssetLoader interface...... 48
4.21. The material package, its interfaces, concrete classes, and collaborations...... 49
4.22. The mesh package, its interfaces, concrete classes, and collaborations...... 50
4.23. The shader package, its interfaces, concrete classes, and collaborations...... 51
4.24. The texture package, its interfaces, concrete classes, and collaborations...... 51
4.25. Interfaces and collaborations of the top-level rendersystem package...... 52
4.26. The RenderSystem interface...... 53
4.27. The RenderWindow interface...... 54
4.28. The Viewport interface...... 56
4.29. The Renderable interface...... 58
4.30. The RenderQueue interface...... 59
4.31. Interfaces and collaborations of the rendersystem.states package...... 60
4.32. The RenderState interface...... 60
4.33. The ZBufferState interface...... 61
xvii 4.34. The TextureState interface...... 61
4.35. The FrontFaceState interface...... 62
4.36. Interfaces and collaborations in the rendersystem.shader package...... 63
4.37. The GpuShaderProgram interface, with less relevant details ommitted...... 64
4.38. Vertex shader transformations. Gray boxes specify relative coordinate spaces. Red boxes
specify transforms that move vertices across coordinate spaces...... 65
4.39. Components of the Phong Lighting Model. From left to right they are ambient, diffuse,
specular, and the combined result...... 67
4.40. Comparison between the Gouraud Shading model (left) and the Blinn-Phong Shading
model (right). Note how the specular highlight on the left looks jagged, while the one on
the right looks more realistic and smooth...... 68
5.1. Math library used in RAGE...... 71
5.2. RML structure overview. Classes are single-precision floating-point implementations.
Package-private Matrix and Vector shown...... 72
5.3. The Vector4f class...... 74
5.4. The Matrix4f class...... 75
5.5. The Quaternionf class...... 77
5.6. The Radianf and Degreef classes as implementations of the Angle interface...... 79
6.1. GPU data under GNU/Linux during stress test. Note the Used Dedicated Memory and
GPU Utilization values...... 86
A. Single scene observed from multiple Camera angles and Viewports...... 89
B. Textured Earth/Moon entities and a directional Light...... 91
xviii C. Asteroid Entity inside a textured SkyBox...... 92
D. Spheres lit by a point Light and an emissive Material...... 94
E. Large cube lit by a spot Light and an emissive cone Entity...... 95
F. Performance test with thousands of objects in a single scene...... 96
xix LIST OF CODE LISTINGS
Listings Page
2.1. Increased client and scene type coupling due to direct use of new operator...... 6
2.2. Reduced client and scene type coupling due to SceneManager abstraction...... 7
2.3. MeshManager returns already loaded Meshes to clients...... 11
2.4. Method explicitly declares dependencies on Person and Database interfaces...... 13
2.5. Singleton Design Pattern hides dependencies and promotes tight coupling...... 14
2.6. Singleton Design Pattern causes test case to add test data in production database...... 15
3.1. Key client initialization sequence in startup(Engine) method...... 21
4.1. General pattern followed by AssetManager implementations...... 48
4.2. Vertex shader shows declarations for attributes and uniforms...... 69
4.3. GlslRenderingProgram bindings to vertex shader attributes and uniforms...... 69
4.4. A GpuShaderProgram sets a uniform declared in Listing 4.3...... 70
A. Defining Viewports for a RenderWindow...... 90
B. Creating Cameras and pairing them with Viewports from Listing A...... 90
C. Creating and defining a directional Light for a scene...... 92
D. Creating and enabling a textured SkyBox...... 93
E. Creation of SceneObjects, SceneNodes, and Node.Controllers for stress test.. 97
xx 1
1. INTRODUCTION
All computer applications are very similar in some aspects. A program to calculate taxes is, fundamentally, no different from a program intended to send emails, predict weather patterns, or edit text. All of them take a set of inputs, process them, and produce a set of outputs as a result.
They do this while relying on the abstractions and services provided by the host operating system
(OS) such as desktop managers, windowing systems, file systems, and underlying hardware devices. However, this knowledge also creates a tight coupling between the applications and the host environments under which they must execute.
1.1. Games as Regular Applications
Computer games are just like any other application users are familiar with. A game's level of realism depends on its intended purpose and goal. While they may appear to be very different at times, all of them share common traits and functionality such as organizing the contents of a scene, managing resources (e.g. sounds, images, etc), and drawing objects onto the screen. They must also work within the various software and hardware constraints of their target platforms.
Engineering games is a process of understanding and managing trade-offs.
1.2. Operating System Considerations
Operating systems allow games to create their windows, read and write files, communicate over the network, and so on. While platforms such as Microsoft Windows and GNU/Linux offer similar services, they do so under very different sets of abstractions and philosophies, all of which have direct implications on how games and other applications need to be engineered. For example, Microsoft Windows views hardware devices and files as fundamentally different objects, and provides different methods to interact with each of them, while GNU/Linux views hardware devices and regular files as being the same, and provides the same abstractions for both.
When knowledge about these details is included in a game, a tighter coupling has been made and 2 getting the same game to run on a different platform will be more difficult and require more engineering effort.
1.3. Hardware Considerations
Games also tend to require specialized hardware to work as intended. The component most likely to be mentioned will be the Graphics Processing Unit (GPU). Games need to process and transform potentially large amounts of data into a final rendered image to be projected on the display's 2D surface, and GPUs are ultimately responsible for this task. Different GPU architectures are produced by Independent Hardware Vendors (IHVs), even on newer generations of the same product line. Games need to work regardless of which GPU type, or generation, is installed.
1.4. The Problem with Variation
These differences in abstractions and methods of interaction present portability problems for games. When games use these services directly, they become bound to that platform as a consequence. This coupling constrains the environments under which they can execute, the hardware they can interact with, and the userbase they can reach. This issue can be particularly frustrating not only for the software engineers who want to reach more users, but also to the users that prefer to use unsupported platforms. The number of possible software and hardware combinations present practical issues that would need to be repeatedly solved every time a new game is developed.
Rather than keep solving the same problem in every different game, a better investment of this additional effort is to engineer an intermediate service layer that allows decoupling all the game clients from having direct knowledge of the host OS. In the context of games this is known as a "game engine", although "service layer" or "framework" would probably be more appropriate terms2. 2 These terms will be used interchangeably. 3
The Computer Science department has been using the SAGE3 framework for several years, and has invested a significant amount of engineering time and effort into it, but the technology has shown its age and is now in need of replacement. RAGE is expected to eventually take its place as it becomes more mature. Figure 1.1 shows the relationship between game clients, game engines, the host platforms, and the underlying hardware.
Platform-Agnostic GameGame Clients Clients
GameGame Engine Engine (RAGE) (RAGE) Platform-Specific
WindowsWindows GNU/LinuxGNU/Linux macOSmacOS
GPUGPU Driver Driver
GraphicsGraphics Processing Processing Hardware Hardware Hardware-Specific Figure 1.1: RAGE decouples clients from the underlying platforms.
3 SAGE stands for Simple Adaptable Game Engine 4
2. GENERAL DESIGN GOALS
In order for games to be built without any direct knowledge of the underlying platforms, a domain-specific abstraction, or model, needs to be created. System implementations are then made so that they conform to the given model specification while clients need only rely on the architecture, documented behaviors, and other properties of the model itself.
This approach has been successfully used in a wide variety of engineering contexts. A common example is keeping operating systems decoupled from the hardware they manage.
Operating systems expose a device driver model and it is up to IHVs to make sure their implementations are compliant with the model of their target OS.
Most of the goals described in this chapter are, in part, meant to address issues users had first-hand while using the SAGE framework and/or its graphicsLib3D math library, both of which are currently in use by the Computer Science department at the time of this writing.
2.1. Loose Coupling Between Components
The overall design-guiding principles, which are meant to increase the re-usability and modularity of a system while minimizing coupling, can be described by the SOLID4 acronym, which is briefly summarized in Table 2.1.
Loose coupling is a desirable property in any software system. It allows different components in a system to work by relying on little or no knowledge of the definitions and implementation details of other components in the system, allows components to be extended while reducing the number of reasons for modifying their inner workings, and so on.
When using a loosely coupled design, replacing any given component imposes minimal or no changes to other components of the system. This helps reduce the overall effort required to change or maintain the system, when compared to a tightly coupled one.
4 See https://en.wikipedia.org/wiki/SOLID_(object-oriented_design) for more information. 5
Mnemonic Description SRP Single Responsibility Principle: Classes should have only one responsibility/reason to change. OCP Open-Closed Principle: Components can be extended without requiring internal modification. LSP Liskov Substitution Principle: Replacing objects for sub-types preserves correctness. ISP Interface Segregation Principle: Many client-specific interfaces are better than one general-purpose interface; clients should not depend on methods it does not use. DIP Dependency Inversion Principle: Systems should depend on abstractions, not implementation details. Table 2.1: Brief summary of the SOLID acronym.
Figure 2.1 shows the main abstractions provided by the core sub-systems in RAGE and how they relate to each other, at a very high level.
Figure 2.1: High-level system architecture. 6
2.1.1. Decoupling Game Clients from Scene Management
Game environments can range from a modest indoor setting all the way to an "open world" in
which the player can travel to any arbitrary location, including different continents or even
planets and galaxies5. This variation in possible environments creates different scene management
challenges, some of which might be better handled if different implementations were allowed to
make specific assumptions about the worlds in which game clients intend to take place.
A naïve approach would simply require game clients to directly create the objects they
need to organize, structure, and manage a given scene. Consider Listing 2.1, where a game client
creates a scene for a simple house environment.
SceneNode house = new IndoorSceneNode("House"); SceneNode room = new IndoorSceneNode("Room"); SceneNode bed = new IndoorSceneNode("Bed"); SceneNode kitchen = new IndoorSceneNode("Kitchen"); SceneNode oven = new IndoorSceneNode("Oven");
house.add(room); house.add(kitchen); kitchen.add(oven); room.add(bed); Listing 2.1: Increased client and scene type coupling due to direct use of new operator.
In this example, the game client directly uses an indoor-friendly implementation of a
SceneNode type to build the scene. But suppose that a better implementation is made, called
HouseSceneNode, to specifically work with house environments, as opposed to any generic
"indoor" environment, and the client wants to use it. Listing 2.1 would require five changes. Now
consider Listing 2.2 in light of Listing 2.1.
5 Some of these include No Man's Sky, by HelloGames, and Star Citizen, by Behaviour Interactive. 7
SceneManager mgr = new IndoorSceneManager(); // ... SceneNode house = mgr.createSceneNode("House"); SceneNode room = mgr.createSceneNode("Room"); SceneNode bed = mgr.createSceneNode("Bed"); SceneNode kitchen = mgr.createSceneNode("Kitchen"); SceneNode oven = mgr.createSceneNode("Oven");
house.add(room); house.add(kitchen); kitchen.add(oven); room.add(bed);
Listing 2.2: Reduced client and scene type coupling due to SceneManager abstraction.
Listing 2.2 relies on a SceneManager abstraction and its factory methods to reduce the
coupling between the client and the specific details of the scene itself. In this case, if the client
wants to use the optimized HouseSceneNode implementation, it would only need to change
the SceneManager implementation from an IndoorSceneManager to, presumably, a
HouseSceneManager. In this trivial comparison, updating clients based on the
SceneManager design would require 80% less refactoring effort. A client might not even need
to change at all if the SceneManager abstraction is received as an argument or created by a
factory, which is the approach taken in RAGE.
2.1.2. Decoupling Scene Structure from its Contents
All players expect games to display the visible objects of a scene on screen. However, not all the
content will be visible at the same time (e.g. objects outside the player's field of view). Games
need a way to not only manage the content of a scene, but also its structure. A scene's structure is
already handled with the help of the SceneNode, which allows the scene graph to be built. The
logical, but naïve, next step might be to slightly expand the role of the SceneNode so that it also
represents the very objects that are in the scene itself (e.g. by adding geometry). A consequence of 8
this approach is that it tightly couples the role of managing a scene with the process of rendering
the objects in it, because the SceneNode is now aware of how a scene is structured and how an
object is meant to be drawn. Just adding this responsibility to the SceneNode makes it
impossible to update scene management logic without potentially breaking the object rendering
logic in the process, and vice versa.
In contrast, RAGE addresses this design problem by limiting the responsibility of a
SceneNode to that of a scene's structure and organization while a SceneObject is instead
used to deal with the content. In order to place a SceneObject in a scene, it must be attached
to a SceneNode. The SceneObject then relies on its parent SceneNode for information
about the scene, such as its position and orientation in the world. This also means that knowledge
about the structure of a scene in a SceneNode no longer needs to exist at the the lower levels of
the system. In fact, different scene management and rendering strategies can be arbitrarily
changed without also requiring the components they collaborate with to change.
2.1.3. Decoupling Scene Management from Rendering
In RAGE, managing and rendering a scene are viewed as "high-level" and "low-level" concepts
respectively. The engine's architecture reflects this conceptual distinction by abstracting the
SceneManager and the RenderSystem. The RenderSystem abstracts the low-level 3D
graphics API, such as OpenGL6 or Vulkan7. However, the RenderSystem does not know what
a SceneObject is. Rather, it knows about Renderable objects, which is what the
component parts of a SceneObject are made of.
6 https://www.khronos.org/opengl 7 https://www.khronos.org/vulkan 9
2.1.4. Decoupling Assets, Loading, and Management
Games need a way to acquire and handle external resources. For this engine's design, the concept
of an Asset, an AssetManager, and an AssetLoader were used.
An Asset is defined as any resource outside the engine itself. These include 3D models
and images produced with Digital Content Creation (DCC) tools –e.g. Blender8 and GIMP9. From
the game developer's perspective, an AssetManager is used to retrieve an Asset without
needing to know the details of how it should be loaded, or even whether the Asset had been
loaded already.
Managers rely on file extensions to determine which concrete AssetLoader will be
selected. Every different file format requires a separate AssetLoader implementation to
process the data and transform it into a format that can be used by the framework. For example,
the engine has a built-in RgbaTextureLoader for images in the RGBA color space (e.g. PNG
files), but if images using a different organization are used (e.g. bitmaps use BGR instead of
RGB), then a separate loader, presumably named BmpTextureLoader, responsible for
transforming the data into the currently expected RGBA color space would need to be
implemented.
Loaders in RAGE do not create or instantiate Assets. Rather, an AssetManager will
create the Asset and then forward the instance to its chosen AssetLoader. The loaders
simply read the raw data, transform it as necessary, and then assign it to the Asset instance
waiting to be properly initialized.
A naïve approach might have given loaders additional knowledge about how and when to
instantiate concrete resources, which would violate the Single Responsibility Principle (SRP) and
8 https://www.blender.org/ 9 https://www.gimp.org/ 10
create a tighter coupling between the responsiblities of loading raw data, managing assets, and
knowing about concrete Asset implementations rather than just their abstractions.
2.1.5. Uniformity Across Asset Managers and Loaders
In the previous section, an Asset was defined as an external resource. Based on that definition,
the next logical step was to provide game clients with a consistent set of uniform interfaces for all
managers and loaders. As noted previously, these interfaces are the AssetManager and
AssetLoader respectively. All concrete manager implementations (e.g. MeshManager) must
implement the AssetManager interface.
However, the AssetLoader interface gets extended for specific Asset types (e.g.
MeshLoader), and then implemented by concrete classes to process a specific file format (e.g.
WavefrontMeshLoader). This makes it easier to support multiple format-specific loader
implementations simultaneously and requires no updates on the clients because they are not
expected to interact with loaders directly. The managers and loaders are set up automatically
during the framework's built-in startup sequence, which avoids tightly coupling loaders and
clients. Still, clients have the flexibility to supply their own custom loaders if needed.
2.1.6. Improving Asset Management Efficiency
Efficient use of computing resources is always necessary. Games generally need to work with
more content than other applications –and more quickly. A naïve design would have the
framework loading and creating assets every time the client makes a request, but this method
does not scale well.
In RAGE, AssetManagers keep track of the existing Assets in order to avoid re-
creating content based on the same underlying data. Listing 2.3 shows a snippet from the
GenericSceneManager.createEntity10 method implementation.
10 An Entity is a concrete SceneObject. 11
@Override public Entity createEntity(String name, String path) { // ... Mesh mesh = meshManager.getAsset(Paths.get(path)); Entity entity = new GenericEntity(this, name, mesh); // ... return entity; } Listing 2.3: MeshManager returns already loaded Meshes to clients.
Consider a game with several planets based on the same geometry, stored in the
"planet.obj" file. In the first invocation, the MeshManager will instantiate a new Mesh
and use a WavefrontMeshLoader to initialize it. The SceneManager then takes the Mesh
and puts it inside a new Entity.
However, for subsequent requests based on the same file, the MeshManager will return
the same Mesh instance it had loaded during the first invocation back to the SceneManager,
without its knowledge, so that it gets re-used in each new Entity, also without their knowledge.
The MeshManager will only use a MeshLoader when it determines the Asset had not been
requested before to avoid wasting memory and other system resources.
2.1.7. Decoupling Assets and Scene Objects from States
Engine Assets basically work as data containers, without knowledge about how clients intend to
use them. For example, Meshes contain the geometry used by SceneObjects, but no
translation, rotation, or scaling (TRS) transforms, while Textures contain pixel data, but no
targets, filtering methods, or wrapping modes.
The TRS transforms for SceneObjects are stored in SceneNodes. A single
SceneNode can have many SceneObjects attached at once. The TRS transforms of the
parent SceneNode are then applied to them without modifying underlying Asset data. This 12
means a significant amount of duplicated TRS transforms, and computations, can be reduced or
avoided altogether, reducing memory usage and freeing CPU cycles for other tasks.
On the other hand, Textures need to have certain state data applied to them when they
are submitted to GPU texture units, such as minification and magnification filters. Knowledge of
these states is decoupled from Textures via TextureStates. Similar to the relationship
between SceneObjects and SceneNodes, a TextureState stores this data only once so
that it can then be applied to multiple Textures at once.
2.1.8. Replacing Hard-Coded Paths for Configuration Files
Developers often need to change the content to be used by a game, such as alternating between
debug and production builds. The process also needs to be fast, easy, and flexible. To achieve this,
configuration files are used. A Configuration is the only external resource that is not an
Asset, because it is the object that specifies where to find Assets in the first place (e.g. file
system paths, window icons, etc.); it is also a singleton object.
2.2. Singleton Objects without the Singleton Design Pattern
A singleton is a class that allows only one instance of itself to exist through the application's life-
time and, by implication, makes it globally accessible. While the concept of a singleton object can
be implemented in different ways, the Singleton Design Pattern (SDP) has been used for decades
as a way to provide this guarantee [Patterns, 1995]. However, implementing the pattern does not
come for free; there are several key trade-offs that come with it, and these became the basis for
avoiding the SDP altogether. Rather than using the SDP in RAGE, a more convention-driven
approach was taken instead.
2.2.1. Global Objects are No Better than Global Variables
Introducing global variables also introduces several complications. Global variables make it
easier for mistakes to creep in due to their greater scope and accessibility. At the same time, it is 13
more difficult to track down and resolve such issues because now, instead of being able to narrow
down the problem to a smaller and more contained section, the cause could be in any part of the
whole system.
2.2.2. Global Data is Prone to Race Conditions
Applications rely on multi-threading to allow certain tasks to execute concurrently. But multi-
threaded systems are also more sensitive to state changes and more care must be taken to ensure
predictability. The global nature of data means that, whenever different threads need to use or
update global values, they can end up causing problems for each other –i.e. there are race
conditions.
While explicit synchronization schemes can be used, these also cause unwanted
performance penalties. Global data is certainly not the only way for race conditions to arise, but
avoiding the SDP and globally-scoped objects reduces not only the chances of these being
introduced, but also the scope of their side-effects if some are actually discovered.
2.2.3. Dependencies Among Objects Not Reliably Communicated
Dependencies between components are generally declared by exposed public interfaces and the
parameter lists for some operations. Consider how dependencies are declared Listing 2.4.
public void save(Database db, Person p) { db.insert("name", p.getName()); db.insert("dob", p.getDateOfBirth()); // ... } Listing 2.4: Method explicitly declares dependencies on Person and Database interfaces.
However, the SDP undermines explicit dependency declarations by removing them from
parameter lists and directly hiding them inside methods. Consider a different implementation in 14
Listing 2.5, which relies on the SDP to acquire the database reference. The Database
dependency is hidden and the client is tightly coupled to a MySQL-specific implementation.
public void save(Person p) { Database db = MySqlDatabase.getInstance(); db.insert("name", p.getName()); db.insert("dob", p.getDateOfBirth()); // ... } Listing 2.5: Singleton Design Pattern hides dependencies and promotes tight coupling.
2.2.4. Tight Coupling Between Classes is Promoted
As shown in Listing 2.5, the SDP promotes tight coupling between components. In addition, this
coupling can hinder testing efforts. While Listing 2.4 makes it possible to pass in mock
Database implementations when writing test cases to verify functionality, the SDP in Listing
2.5 does not.
Listing 2.5's design actually makes testing a dangerous proposition, because an
unsuspecting developer might try to write and execute innocent-looking, yet misleading, test
cases and unknowingly end up inserting meanigless data into the real database in a live
production environment.
As Listing 2.6 shows, it is not obvious to a reader that the test case is actually inserting
test data into a real production database, whose dependency is hidden inside the save call from
Listing 2.5. Finding the source of the problem will not be as easy11, especially in large non-trivial
codebases.
11 While someone could quickly realize that the bad insertions match data hard-coded into a test case they are familiar with, the point is that the problem was preventable and could have been avoided altogether. 15
public void testPersonCanBeSaved() { Person p1 = new Person(...); save(p1); } Listing 2.6: Singleton Design Pattern causes test case to add test data in production database.
2.3. Rendering with Programmable Shaders
The modern graphics pipeline is based on programmable shaders, which are programs that execute directly within the GPU at specific stages. Some legacy systems, such as SAGE, still rely on the deprecated fixed-function (i.e. non-programmable) pipeline. Since modern graphics hardware is built and optimized to work within the programmable paradigm, performance in older systems is likely to suffer negatively. In addition, these discrepancies can make it more difficult and impractical to use SAGE as a teaching aid in the context of real-time rendering and computer graphics. This is an important reason for RAGE's architecture and focus on the programmable shader-based pipeline.
2.4. Choosing a Free-Software License for Source Code
Ultimately, all computing devices are controlled, not by the user, but by the software that is running in it. The software decides what the user is allowed or not allowed to do, and it has the power to enforce it. A significant portion of the software people use today is non-free/proprietary.
This means the people who are expected to use the software cannot change the software's behavior, or inspect it to make sure it does only what it claims to be doing, and so on –i.e. they cannot even look at it. In fact, most End-User License Agreements (EULAs) make it illegal for users to even try to do so by using anti-circumvention provisions or just rely on the Digital
Millenium Copyright Act (DMCA) as the means for this end. Many of its effects have been documented [Lohmann, 2010]. 16
This is where free software comes in. The term "free" in this context is in the sense of freedom, not cost, and is analogous to the distinction between free speech and free food. The Free
Software Foundation (FSF) provides the following definition for the term "free software"12:
A program is free software if the program's users have the four essential freedoms:
• The freedom to run the program as you wish, for any purpose (freedom 0). • The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this. • The freedom to redistribute copies so you can help your neighbor (freedom 2). • The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.
A program is free software if it gives users adequately all of these freedoms. Otherwise, it is nonfree.12
While SAGE and graphicsLib3D do not come with any explicit EULAs, they are still protected by Copyright and are provided to students as proprietary/nonfree software. To address this problem, a well-known free-software license, the GPLv313, was chosen for RAGE and the
RAGE Math Library (RML).
This license gives developers in general, and CSc-165 students in particular, several advantages over potential alternatives. For example, student access is not limited to only using
RAGE for their games. Rather, it grants them permission to directly access, learn, modify, share, and contribute to the source code as a protected right –not just a priviledge. There are, of course, additional implications and responsibilities that everyone must understand and abide by. While the following is not legal advice or a substitute for the full text of the license, generally speaking:
You may copy, distribute and modify the software as long as you track changes/dates in source files. Any modifications to ... GPL-licensed code must also be made available under the GPL along with build & install instructions.14
12 https://www.fsf.org/about/what-is-free-software 13 https://www.gnu.org/licenses/quick-guide-gplv3.html 14 https://www.tldrlegal.com/l/gpl-3.0 17
3. HIGH-LEVEL FRAMEWORK OVERVIEW
Game client requirements can be broadly divided into the following categories:
1. game-specific assets and rules
2. platform-specific services and abstractions
Engines are meant to act as service providers by exposing uniform abstractions that hide platforms from all the game clients. Engine features are based on common needs present in most games. This way, game developers only need to familiarize themselves with a single framework and learn how to use it, instead of having to figure out how to build all of these components for every game and learning the details of the underlying target platform(s).
3.1. General Framework Architecture
The general architecture of RAGE can be divided into the following categories:
1. scene management
2. asset management
3. graphics rendering
4. other/external libraries and bindings
While most areas are available to the clients directly, the stacked architecture is meant to imply that clients should, with few exceptions, rely most of the time on higher-level components, such as scene and asset management.
One of these exceptions is the RML, which was designed as an external dependency of the engine and can be used by itself. Figure 3.1 shows the high-level architecture and dependencies in RAGE. 18
Game Clients
Game Client 1 Game Client 2 ... Game Client N
Game Engine (RAGE)
Scene Management Scene Manager Scene Node Node Controller Camera Light Manual Manual Entity Sub Entity Sky Box Object Object Section
Asset Management Mesh Shader Texture Material
Mesh Manager Shader Manager Texture Manager Material Manager
Mesh Loaders Shader Loaders Texture Loaders Material Loaders
Rendering Renderable Render System Viewport Window GPU Render States Render Queue Canvas Shader Program
Libraries/Utilities JOGL Bindings Configuration RML (Math) Buffer Manager Logging
Figure 3.1: High-level client/engine architecture and functionality.
3.2. Game Client Architecture
RAGE exposes a simple Game interface, shown in Figure 3.2, that must be implemented by clients. It also serves as the main point of access to the Engine class itself.
Figure 3.2: The Game interface.
The framework also provides several abstract classes with common operations implemented, so that developers can get started more quickly. Figure 3.3 shows the basic client architecture. 19
Figure 3.3: Basic client architecture.
Clients extend the VariableFrameRateGame class to get started. The
AbstractGame provides a logical breakdown of the common steps needed to set up a game.
Some operations include control over startup and shutdown sequences, setup for windows, viewports, cameras, and scenes, as well as placeholders for client updates. Figure 3.4 shows this in more detail. 20
Figure 3.4: The AbstractGame class.
An effort was made to make these steps intuitive by specifying the more relevant parameters. For example, the setupWindowViewports step only expects the
RenderWindow that will be sub-divided into Viewports. The BaseGame, shown in Figure
3.5, implements most of these steps in addition to the built-in Java EventListener interfaces.
Figure 3.5: The BaseGame class. 21
The key step here is the startup(Engine) method, which uses the Template Method pattern [Patterns, 1995] to define the initialization sequence that applies to clients, while making clients responsible for the actual logic within those steps. Listing 3.1 shows the relevant details, with key sequence-defining steps in bold.
@Override protected void startup(Engine engine) throws IOException { Configuration conf = engine.getConfiguration(); loadConfiguration(conf);
RenderSystemFactory rsf = createRenderSystemFactory(); RenderSystem rs = rsf.createInstance(); // ... GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment(); setupWindow(rs, ge);
setupWindowViewports(rs.getRenderWindow()); setupGpuShaderPrograms(rs, engine.getShaderManager());
SceneManagerFactory smf = createSceneManagerFactory(); SceneManager sm = smf.createInstance(); // ... setupCameras(sm, rs.getRenderWindow()); setupScene(engine, sm); } Listing 3.1: Key client initialization sequence in startup(Engine) method.
3.3. Package Organization Overview
Figure 3.6 shows how RAGE packages are organized and key collaborations.
Game clients can access the Engine by using Game.getEngine(), which in turn provides access to other sub-components. Each sub-system has a core interface that represents it.
The scene package's core interface is the SceneManager, which allows games to create
SceneObjects and other types. Some of the concrete ones include Entity, Camera,
SceneNode, and Light. The SceneNode is the foundation of the hierarchical scene graph, 22 which is the data structure used to organize and structure the contents of a scene. They also allow
TRS transforms to propagate automatically.
Figure 3.6: High-level framework package organization and collaborations. 23
The asset package has the AssetManager interface, which allows games to load, and keep track of, external data (e.g. geometry, images, etc.) for their own use. More concrete built-in types include the MeshManager, MaterialManager, ShaderManager, and
TextureManager. These allow uniform handling of Mesh, Material, Shader, and
Texture Assets respectively. More importantly, if game clients request data that a manager has already loaded before, the manager will return a reference to the pre-existing Asset instead of re-loading and duplicating it in memory.
Although not shown in Figure 3.6, these Assets are loaded using concrete file format- specific implementations of the AssetLoader interface. The AssetManager implementations map known file name extensions (e.g. ".obj") to specific loaders (e.g.
WavefrontMeshLoader) in order to make sure that the correct AssetLoader is chosen at run-time.
Within the rendersystem package, the RenderSystem interface acts as a thin low- level abstraction for the underlying 3D Application Programming Interface (API), allowing geometry and other data to be submitted to the GPU. A game client's direct use of the
RenderSystem is expected to be minimal and generally limited to creating RenderWindows,
GpuShaderPrograms, and RenderStates when necessary. Most of the time, clients will only need the SceneManager, which automatically submits Renderable objects to the
RenderSystem for processing.
3.4. Scene Management and Organization
Every game needs to organize the contents of its virtual world. The SceneManager lets clients create SceneNodes. SceneNodes are designed to be placed in parent-child relationships by 24 the clients. These relationships form the logical structure of the scene. Figure 3.7 shows a sample scene represented as a hierarchical scene graph.
Hierarchical Scene Graph
SceneScene
CamerasCameras WorldWorld
Camera-1Camera-1 ... Camera-NCamera-N SceneScene ObjectsObjects PlayersPlayers
LightsLights EntitiesEntities Player-1Player-1 ... Player-NPlayer-N
Light-1Light-1 ... Light-NLight-N Entity-1Entity-1 ... Entity-NEntity-N Figure 3.7: Scene organization using a hierarchical scene graph.
3.5. Graphics Processing
Engines manage the interaction between the games, the underlying 3D API, and the graphics hardware. This makes it easier for games to render the contents of a scene onto the screen. These
APIs present a thin, or "low-level", abstraction layer of the graphics processing hardware, allowing games to work regardless of which particular GPU is installed on a system.
Some APIs include Direct3D15, OpenGL, and more recently, Vulkan. These APIs are not necessarily supported or available across all platforms. For example, Direct3D is only available in
Microsoft Windows, whereas OpenGL and Vulkan are available in most platforms16. However, the rate at which OpenGL has been able to support new features has been slower by comparison.
Thus, by directly dealing with the APIs and their trade-offs, engines remove a significant amount of complexity from each game client.
15 https://msdn.microsoft.com/en-us/library/windows/desktop/bb153256(v=vs.85).aspx 16 Currently, Apple appears more committed to their proprietary API for iOS/macOS, called Metal, with OpenGL support lagging behind and Vulkan support coming in the form of a separate layer built on top of Metal, called MoltenVK. 25
3.6. Asset Loading and Management
Games typically require a significant amount of external resources, or assets. Some of these include the geometry that defines 3D models, the images to texture them, materials to describe how their surfaces reflect light, and so on. These assets can be produced by a variety of Digital
Content Creation (DCC) tools, such as Blender17, and then exported into a wide range of different file formats18.
The complexity of these formats varies depending on the data they contain, such as whether they support animations or not. Engines enable clients to easily load and manage these external assets efficiently, as long as their internal file formats are supported. RAGE provides the
Asset, AssetManager, and AssetLoader interfaces with concrete implementations for representation, management, and loading of Materials, Shader source code, Textures, and
Meshes.
3.7. User Input Handling
The interactive nature of games means that all clients must handle user inputs. These inputs can be provided by a variety of peripherals including gamepads, keyboards/mice, and joysticks.
Currently, RAGE supports keyboards and mice by relying on the built-in Java event listeners such as KeyListener, MouseListener, MouseMotionListener, and
MouseWheelListener. At this time, it is the client's responsibility to implement their own user input handling schemes, especially if support for devices other than keyboards/mice is desired.
17 https://www.blender.org/ 18 Some of these may include, but are not limited to, Wavefront and Ogre XML. 26
4. DETAILED OVERVIEW OF FRAMEWORK COMPONENTS
This chapter discusses each sub-component of the framework in more detail, including diagrams and other important abstractions that are necessary for interactions across different sub-systems.
4.1. The Engine Class
The Engine class is found in the top-level package of the system, ray.rage, and it is the client's central point of access to framework sub-components. Figure 4.1 shows the Engine diagram.
Figure 4.1: The Engine class.
The Engine allows granular control over its startup and shutdown sequences and also provides direct access to the SceneManager, MeshManager, RenderSystem, and so on. Its design is based on the Façade design pattern [Patterns, 1995], and provides a simpler way for clients to interact with the whole. For example, during Engine.startup(), concrete asset 27
managers and loaders will be initialized automatically, so that by the time the client's
startup(Engine) setup begins, they will be available for use.
The Engine is one of several singleton objects19 in the framework and is always
available to game clients via the Game.getEngine() method.
4.2. Scene Management and the Scene Package
The scene package contains a set of interfaces that define the types relevant to a scene and how
they relate to each other. A generic implementation is included in scene.generic package.
4.2.1. Scene Management Architecture
Some of the key interfaces in this package include the SceneManager, SceneObject and
SceneNode. Figure 4.2 shows these and key components from other packages as needed.
A key observation is how the Entity and SubEntity types are defined. While the
Entity is a SceneObject, the SubEntity is a Renderable object, which defines the
"parts" of a larger "whole", much like individual pixels combine to define a whole image. It is
this specific design decision that helps minimize the amount of coupling between scene
management and rendering –a key design goal.
19 See section 2.2 Singleton Objects without the Singleton Design Pattern in page 12. 28
Figure 4.2: Interfaces and collaborations of the top-level scene package
4.2.2. The Scene Manager
The SceneManager is what all clients mostly rely on to control the contents of the game world.
This includes not only what objects get to exist in it, but also keeping track of all of them on the
client's behalf, submitting them to the graphics pipeline, and their eventual disposal at the end of
the client's execution.
The SceneManager's design is based on the Abstract Factory pattern [Patterns, 1995].
It is responsible for instantiating all the concrete SceneObjects the client needs. The client is
not meant to be creating new SceneObjects directly with the new operator. Rather, if the
client needs to create a new Entity, it must use the 29
SceneManager.createEntity(String, String) factory method instead. Table 4.1 shows the key methods for other types.
Method Names Description createLight(String, Light.Type) Creates a new Light of the specified type. createCamera(String, Creates a new Camera with the specified projection Camera.Frustum.Projection) type (e.g. perspective). createManualObject(String) Creates a new ManualObject with the given name. createSkyBox(String) Creates a new SkyBox with the given name. createSceneNode(String) Creates a new SceneNode with the given name.
Table 4.1: SceneManager methods for other concrete SceneObject types.
Figure 4.3 shows a complete diagram of the SceneManager interface. The general pattern is to offer methods that can create, search, check, iterate, and destroy each concrete type. 30
Figure 4.3: The SceneManager interface.
4.2.3. The Scene Object
All the objects in a scene are based on the SceneObject interface. As the name implies, a
SceneObject represents an object that can be placed in a scene. Most of the time, these will be
based on an Asset, such as a Mesh that defines a sphere, but this is not a requirement. 31
Cameras and Lights are two concrete SceneObjects not based on Assets. Figure 4.4
shows the SceneObject diagram and the interfaces it extends.
Figure 4.4: The SceneObject interface.
SceneObjects do not contain any TRS transformations, which decreases the coupling
between scene content and scene structure. Instead, they must be attached to a parent
SceneNode in order for them to be processed as part of a scene.
4.2.4. The Scene Node
In order to actually position a SceneObject in a scene, it must be connected to a SceneNode.
SceneNodes contain the necessary TRS transforms for this and are meant to be in parent-child
relationships with one another. This is the basis of the hierarchical scene graph data structure and
allows a scene to be updated with minimal effort from both the game client's and engine
developer's perspective.
The SceneNode is actually an extension of a more general Node interface. Figure 4.5
shows the diagram for both Node and SceneNode, including the nested Node.Controller
and Node.Listener interfaces. 32
Figure 4.5: The Node and SceneNode interfaces.
The main difference between Nodes and SceneNodes is that, while Nodes allow other
Nodes to be added as their children as part of an articulated graph, only SceneNodes can have 33
SceneObjects attached to them and are meant to be used in a scene graph.
When a Node needs to be updated, its update() method should be used. When
update() is invoked, the Node first makes sure its local- and world-space TRS transforms are
current. Then, it iterates over all its children to update() them. To update the entire graph, only
the root Node needs to have update() invoked. The TRS transforms will then propagate
through the graph automatically.
If a game client implements the Node.Listener interface and registers itself with the
specific Node of interest, then the Node will invoke
Node.Listener.onNodeUpdated(Node) to notify the client about its completed update at
the end of the process. If at some point a SceneNode cannot trace its ancestry all the way up to
the root SceneNode, then the SceneNode is and its descendants are no longer part of the
scene graph and any SceneObjects attached to them will be skipped during the rendering
process.
4.2.5. Entities and Manual Objects
An Entity is a SceneObject that is based on a single Mesh. To create an Entity, the
client must use the SceneManager.createEntity(String, String) method, which
expects a name and the file system path to the file it will be loaded from. The process from that
point on is handled by the manager and the client receives an Entity that is completely built,
including a Material and a TextureState with a Texture assigned. Figure 4.6 shows the
Entity diagram. 34
Figure 4.6: The Entity interface.
The Entity provides a few convenience methods that are meant to apply something
(e.g. a Material) to all the SubEntity parts of itself, rather than requiring the client to always iterate over each SubEntity explicitly.
A ManualObject is just like an Entity, with a key difference being that it is up to the game client to actually build it. For example, if a game client has to algorithmically generate
(or has hard-coded) geometry to define a sphere (i.e. vertices, texture coordinates, normals, etc.) because it does not have an actual file from which an AssetLoader can get this data, then using a ManualObject is a good idea.
Another key difference is that, while RAGE will automatically set a default Material and Texture for an Entity if they are not defined by the Mesh, ManualObjects will not get this automatic treatment, which could cause undesired visual effects. Therefore, developers are advised to keep be mindful of these important details. Figure 4.7 shows the ManualObject diagram. 35
Figure 4.7: The ManualObject interface.
Just like the Entity, the ManualObject has several convenience methods for clients to more easily apply properties to all the ManualObjectSections that make them up. Table
4.2 shows some key methods specific to the ManualObjects.
Method Names Description setVertexBuffer(FloatBuffer) Sets vertex positions (i.e. geometry), in object- space. setTextureCoordBuffer(FloatBuffer) Sets coordinates for texture sampling, in texture space. setNormalsBuffer(FloatBuffer) Sets normal vectors perpendicular to the surface, in object-space. setIndexBuffer(IntBuffer) Sets vertex buffer index values to re-use geometry. setDataSource(DataSource) Sets whether vertices or indices should be used for rendering.
Table 4.2: Key convenience methods in the ManualObject interface. 36
Depending on what the client intends to use the objects for, some properties may become
optional. For example, vertex normals are required for lighting effects and texture coordinates are
required for texture mapping to work as intended. However, SceneObjects like the Entity
and ManualObject cannot themselves be rendered –only the parts they are made from can. In
this sense, these SceneObjects simply act as "containers".
4.2.6. Sub-Entities and Manual Object Sections
Unlike the Entity and ManualObject, which are SceneObjects, the SubEntity and
the ManualObjectSection are the Renderable objects, which are the independent parts
that make them up and will be covered in more detail in a later section. For now, it is enough to
say that the purpose of a Renderable is to hold the low-level primitive data that is used by a
RenderSystem at a lower-level in the stack.
A SubEntity defines a sub-part of its parent Entity and is based on a single
SubMesh and a single Material. Similarly, the ManualObjectSection is a sub-part of
its parent ManualObject. Neither can be created directly by the client. Instead, the client must
rely on factory methods provided by parent Entity and ManualObject respectively. Figure
4.8 shows the SubEntity diagram. In both cases, there is a one-to-many relationship between
the parent container and the aggregated parts. 37
Figure 4.8: The SubEntity interface.
Sub-Entities are automatically created by their parent entities when they themselves get created by the SceneManager. An Entity will create one SubEntity for each SubMesh of the Mesh the Entity itself is based on. The SceneManager then relies on
SubEntity.Visitors to automatically create RenderStates for each SubEntity.
Unsurprisingly, ManualObjectSections must be explicitly created by the game client by directly invoking the ManualObject.createManualSection(String) factory method. While these might offer a different degree of flexibility (e.g. their content can be defined algorithmically), they can also be more error-prone because they require clients to have a better understanding of how data is organized directly inside the primitive buffers. Figure 4.9 shows the
ManualObjectSection diagram. 38
Figure 4.9: The ManualObjectSection interface.
Unlike the SubEntity, ManualObjectSections have "setter" methods for clients
to specify the data. For performance reasons, clients are always required to provide direct buffers.
The utility BufferManager class can be used to do this quickly and easily.
4.2.7. Cameras and Viewing Frustums
In the same way that movies are filmed through the lens of a camera, a scene is observed from the
point of view of a virtual Camera. Each Camera has a viewing Frustum that defines its
viewing volume, projection type and planes, aspect ratio, and so on. With this information, it is
the Camera that tells the SceneManager that it should render the scene from its vantage point,
via the SceneManager.notifyRenderScene(Camera c, Viewport vp) method.
Cameras do this when their own Camera.renderScene() method is directly
invoked. Cameras notify their Listeners before and after rendering a scene. Game clients can
implement the Camera.Listener interface if they are interested in these events. Figure 4.10
shows the interfaces and relevant enumerated types. 39
Figure 4.10: The Camera and Frustum interfaces with other dependencies.
Note that the Frustum, not the Camera, determines whether an orthographic or perspective projection will be used for the viewing transform. Currently, only the
Projection.PERSPECTIVE has been implemented.
Cameras always render their contents through Viewports. Cameras cannot be used without them and implementations will throw exceptions if attempted. There is a one-to-one relationship between them and it is the client's responsiblity to assign them to each other. This can be done with the Viewport.setCamera(Camera) method. The Viewport automatically invokes the Camera.notifyViewport(Viewport) method. 40
4.2.8. Global and Local Scene Illumination
There are several different ways in which a scene can be lit. The AmbientLight is used to
control global scene illumination and can only vary in intensity. Figure 4.11 shows the
AmbientLight diagram.
Figure 4.11: The AmbientLight interface.
The AmbientLight is currently treated as a singleton20 by the SceneManager.
Because of this, the SceneManager automatically creates an AmbientLight during its
initialization. Clients can only access it with the SceneManager.getAmbientLight()
method. However, Lights must always be explicitly created by the client and offer more
options. Figure 4.12 shows the Light diagram.
20 While done for simplicity, a future improvement could provide one ambient light per camera to allow different parts of a scene to be illuminated with different intensity levels, such as outdoor/indoor cameras in a split-screen game. 41
Figure 4.12: The Light interface
Note that, while Lights are SceneObjects and can be added to a scene by attaching them to SceneNodes, the AmbientLight is a singleton and does not extend the
SceneObject interface –making this impossible. That being said, the AmbientLight does not really need this ability because, by definition, it applies globally to the entire scene.
The Light.Type enumeration defines different types of Lights and this is used to determine which options are taken into account when rendering a scene. Table 4.3 explains their differences and behaviors.
Light Type Description POINT A Light with a position, but no direction. It is emitted equally in all directions from a specific position and gets affected by attenuation and range. DIRECTIONAL A Light with a direction, but no position. It is emitted in a specific 42
Light Type Description direction, but from infinitely far away and is not affected by attenuation or range. SPOT A Light with both position and direction, defined by a cone's shape. This Light.Type is affected by attenuation, range, and falloff; its effect will only be applied to objects inside the cone and within range.
Table 4.3: The different Light.Types supported in RAGE.
4.2.9. Distant Backgrounds and Sky Boxes
Games generally need to give the illusion that they are taking place in an actual world, rather than
plainly showing a floating level in the middle of nowhere. There are several techniques to make
this happen, with sky boxes being one of them. Figure 4.13 shows a diagram of the SkyBox
clients have access to.
Figure 4.13: The SkyBox interface.
A SkyBox, as the name implies, is a box with textured images of a distant background,
such as the sky. SkyBoxes must completely surround the player, so images of mountains, cities,
and other kinds of environments are often combined for full effect. SkyBoxes are not only meant
to be unreachable by players, but to also avoid having having completely empty/black regions of
unoccupied space, which would give the games an unfinished look and break the player's
immersive experience. 43
4.2.10. Node Controllers in a Scene
RAGE has a few built-in Node.Controller implementations for basic behaviors. The
scene.controllers package offers implementations for TranslationController,
RotationController, ScalingController, and OrbitController. The client can
simply instantiate any of them directly and then add them to the SceneManager via its
addController(Node.Controller) method for it to update them automatically.
Otherwise, it is the client's responsibility to update the controllers explicitly.
4.3. Asset Management and the Asset Package
Every game needs a way to access external game-specific resources. RAGE's support for Asset
loading and management is found in the asset package. The specific types of supported
Assets are Materials, Textures, Meshes, and Shaders.
4.3.1. Asset Management Architecture
The asset package is fairly small, with only a few interfaces and abstract classes that
implement basic behavior. Figure 4.14 shows the diagram of the top-level package interfaces and
classes that are publicly accessible. The abstract classes are not meant to be used by clients.
Rather, they are publicly available to allow implementation-specific package classes inside the
framework to extend them. This is necessary due to Java's limitation of only supporting
independent packages –not actual sub-packages or namespaces.
This limitation makes it impossible to access classes or interfaces from different packages
within RAGE, while keeping them hidden from clients who use RAGE. This is a case where C#
language features, such as support for namespaces and the internal21 keyword, among others,
would have been very useful.
21 https://msdn.microsoft.com/en-us/library/7c5ka91b(v=vs.140).aspx 44
Figure 4.14: Interfaces and collaborations of the top-level asset package.
Figure 4.15 shows the general architectural pattern between top-level and implementation packages with a hypothetical concrete package and types. 45
Figure 4.15: Structural pattern between top-level asset and example concrete implementation.
Based on Figure 4.15, future Asset-centric diagrams will focus mostly on the relevant
implementation-specific package contents, rather than their relationships to all external packages,
to avoid repetition.
4.3.2. Assets
The top-level Asset interface used to represent external data is very simple. Figure 4.16 shows
the Asset diagram. 46
Figure 4.16: The Asset interface.
The Asset is mainly meant to serve as a foundation for providing a basic degree of
uniformity and consistency in how external resources are represented and handled by
AssetManagers. The basic functionality of an Asset is implemented by the
AbstractAsset class, which is shown in Figure 4.17.
Figure 4.17: The AbstractAsset class.
4.3.3. Asset Managers
Game clients request all their concrete Assets from concrete AssetManagers based on their
file system path or, if they are manual Assets, their given names. Figure 4.18 shows the
AssetManager diagram. 47
Figure 4.18: The AssetManager interface
The generic nature of the AssetManager interface means that returned assets are always of the specific concrete type requested and do not need to be type-cast by the client. The core functionality is implemented in the AbstractAssetManager class, whose diagram is shown in Figure 4.19.
Figure 4.19: The generic AbstractAssetManager class and its only protected abstract method. 48
The AbstractAssetManager class relies on the Template Method [Patterns, 1995]
design pattern to delegate the actual instantiation of concrete Assets to concrete manager
implementations. The general pattern followed by implementation classes is shown in Listing 4.1.
public final class ConcreteManager extends AbstractAssetManager
@Override protected Concrete createAssetImpl(String name) { return new Concrete(this, name); }
} Listing 4.1: General pattern followed by AssetManager implementations.
4.3.4. Asset Loaders
The AssetLoader interface is also fairly simple, with only one method to implement. Figure
4.20 shows the top-level AssetLoader diagram.
Figure 4.20: The generic AssetLoader interface
The first argument, named dst, is the destination Asset to be initialized by the loader
based on the data read from the filesystem at the specified Path. Alternatively, a method 49
signature such as T loadAsset(Path src), which would be responsible for internally
creating and then returning a new instance, could have been used. However, that was not done
because it would end up adding more responsibilities to loaders (i.e. instantiation, which is the
AssetManager's role) and also increase the coupling between Assets and AssetLoaders,
in violation of the SRP.
4.3.5. Materials
A Material is a concrete Asset that describes whether a SceneObject's surface reflects
and/or emits light in a scene or not, and how much. They are found in the asset.material
package, managed by concrete MaterialManagers, and loaded by concrete implementations
of the MaterialLoader interface, which are found in the asset.material.loaders
package. Figure 4.21 shows a diagram of these.
Figure 4.21: The material package, its interfaces, concrete classes, and collaborations.
4.3.6. Meshes
A Mesh is a concrete Asset that holds the geometric data on which a SceneObject may be
based. A SubMesh defines a smaller portion of a larger Mesh and directly stores the data for the 50
section it represents. Meshes act more as a containers and are divided into one or more
SubMeshes. They are found in the asset.mesh package, managed by concrete
MeshManagers, and loaded by concrete implementations of the MeshLoader interface, which
are found in the asset.mesh.loaders package. Figure 4.22 shows a diagram of this
description. Note that the Meshes and SubMeshes are the only Assets that are divided into a
part-whole relationship.
Figure 4.22: The mesh package, its interfaces, concrete classes, and collaborations.
4.3.7. Shaders
A Shader is a concrete Asset that holds the source code used to build a
GpuShaderProgram. They are found in the asset.shader package, managed by concrete
ShaderManagers, and loaded by concrete implementations of the ShaderLoader interface,
which are found in the asset.shader.loaders package. Figure 4.23 shows a diagram of
this description. 51
Figure 4.23: The shader package, its interfaces, concrete classes, and collaborations.
4.3.8. Textures
A Texture is a concrete Asset that holds pixel image data. They are found in the
asset.texture package, managed by concrete TextureManagers, and loaded by concrete
implementations of the TextureLoader interface, which are found in the
asset.texture.loaders package. Figure 4.24 shows a diagram of this description.
Figure 4.24: The texture package, its interfaces, concrete classes, and collaborations. 52
4.4. Scene Rendering and the Render System Package
The responsibility of processing and drawing the contents of a scene was given to the
rendersystem package. RAGE contains an OpenGL 4-based implementation, which
internally relies on the Java OpenGL Bindings (i.e. JOGL22) library.
4.4.1. Rendering System Architecture
A RenderSystem works by processing a RenderQueue to draw Renderable objects. It is
also responsible for creating the RenderWindow where contents will be drawn. Figure 4.25
shows the package contents, structure, and interactions.
Figure 4.25: Interfaces and collaborations of the top-level rendersystem package.
22 http://jogamp.org/ 53
4.4.2. The Rendering System
The RenderSystem acts as a thin abstraction of the underlying 3D API for the game clients,
such as OpenGL or Vulkan. Implementation classes are expected to have access to a lower-level
binding library that will enable native interaction with the hardware.
RenderSystems are responsible for creating the main RenderWindow and the
surface, or Canvas, and drawing Renderables. It is the SceneManager's role to determine
what, if anything, gets submitted. The RenderSystem is not meant to be "smart" in the sense
that it is expected to process everything in the RenderQueue, rather than trying figure out if a
Renderable is within the viewing Frustum or not23. Figure 4.26 shows the rendersystem
package diagram.
Figure 4.26: The RenderSystem interface.
RenderSystems need to query the hardware for information about the capabilities they
support. These features are encapsulated by the nested RenderSystem.Capabilities
23 See section 6.8 Space Partitioning Scene Managers and Visibility Determination in page 83. 54
interface. While currently minimalistic, it can be updated to account for more hardware-specific
information in the future.
4.4.3. Rendering Windows
The RenderWindow is the main widget game clients present to players. RenderWindows
contain the surface (i.e. Canvas) on which drawing commands requested by a RenderSystem
are performed. Figure 4.27 shows the RenderWindow interface.
Figure 4.27: The RenderWindow interface.
RenderWindows must always have at least one Viewport. A new RenderWindow
always starts out with a default Viewport that covers the whole screen. If the client wants to
have multiple Viewports in the same RenderWindow (e.g. local multiplayer support), then it
is the client's responsibility to explicitly add the Viewports it needs and adjust their relative
dimensions. New Viewports can be added via the RenderWindow.addViewport(float
bot, float left, float width, float height) method. 55
4.4.4. Viewports
A Viewport is a section within a RenderWindow through which a Camera can render the
contents within its viewing volume, or Frustum. While most clients will simply work with a
default Viewport covering the entire RenderWindow, some might want to use multiple
Viewports to support features such as local multiplayer.
When multiple Viewports are used, it is the client's responsibility to define the relative
area they will occupy within a RenderWindow. These dimensions are specified in screen-space
coordinates. These coordinates are in the [0,1] range, with 0 being used for the bottom and
left-most corners and 1 for the top and right-most corners. Therefore, a Viewport that covers
the entire RenderWindow will have [bottom=0, left=0, right=1, top=1]
respectively as its coordinates, whereas another that covers only the top half of the screen has
coordinates [bottom=0.5, left=0, right=1, top=1]. Figure 4.28 shows the
Viewport diagram.
When the RenderSystem detects that its RenderWindow has been resized, it
automatically iterates over the Viewports its RenderWindow knows about and notifies them
that RenderWindow dimensions have changed and they need to update themselves. This allows
Viewports to maintain their proper sizes relative to the actual size of their RenderWindows. 56
Figure 4.28: The Viewport interface.
4.4.5. Renderables
A Renderable is an object that can be drawn onto the screen by a RenderSystem.
Renderables are defined by vertex positions, texture coordinates, normals, and (optionally)
indices.
The vertex positions are the points (i.e. geometry) that define the bounds and shape of the
object relative to an arbitrary origin within the object itself. Because of this, these vertices are
said to be in local- or object-space. The values in a vertex buffer are interpreted as 3-component
vectors with (x,y,z) coordinates specifying a point in 3D space. For example, if not relying on
vertex indices, a simple cube can be specified by 36 vertices24 relative to the cube's origin.
The texture coordinates specify which part of a Texture will be sampled to determine
the color for a pixel at the surface of the given Renderable. These values are in the [0,1]
24 There are 6 faces * 2 triangles per face * 3 vertices per triangle. 57 range and said to be in texture-space, with (0,0) being the bottom-left corner of the Texture and (1,1) being the top-right corner.
A vertex normal is a 3-component vector that is perpendicular to the plane of the
Renderable's surface. These normal vectors are necessary to differentiate between the "front" and "back" of geometric shapes and properly simulate how light is reflected by the
Renderable when it gets lit.
The vertex indices are values that identify the specific vertex positions that should be used when defining a primitive. For example, when using indices, the previous cube can now be defined with only 8 vertices, instead of 36. In this case, the indices are responsible not only for defining which vertex positions will be used to define the triangle primitives, but for re-using them to avoiding having to duplicate vertices in memory.
The Renderable interface is key to decoupling scene management responsibilities from scene rendering ones. The RenderSystem only knows how to draw Renderables.
Packages at higher levels must implement the Renderable interface if they expect to submit content to a RenderSystem for rendering. This is precisely what was done with the
SubEntity and ManualObjectSection. Figure 4.29 shows the Renderable diagram.
There are several enumerated types defined within the Renderable interface. The
Primitive determines which geometric shape, or primitive, will be used to render the object.
The FrontFace determines which vertex ordering convention, or winding, should be used to identify the "front" side, or face, of a Renderable. This allows the graphics pipeline to perform back-face culling and avoid wasting time trying to render invisible surfaces. The DataSource specifies whether the RenderSystem should use a Renderable's vertex buffer for all of its geometry, or whether valid indices have been defined for it. 58
Figure 4.29: The Renderable interface.
4.4.6. Rendering Queues
Renderables cannot be individually submmitted to a RenderSystem. Rather, they must be
submitted in queues. A RenderQueue is a queue that contains Renderables that a
RenderSystem is expected to render. It is the SceneManager's responsibility to determine
which Renderables will be added to the RenderQueue and which ones should be skipped.
Figure 4.30 shows the RenderQueue diagram.
Currently, a RenderQueue is very minimalistic in the set of operations it provides. The
SceneManager implementation also adds Renderable parts of SceneObjects as long as
the SceneObjects are Visible, attached to a SceneNode, and the SceneNode is
connected to the scene graph. 59
Figure 4.30: The RenderQueue interface.
Game clients can also choose to implement the RenderQueue.Listener interface in
order to receive relevant notifications. In some cases, the client can decide whether processing of
the specified RenderQueue should actually continue –or it can choose to cancel it.
4.5. Render Systems and Render States
A RenderState is an object that encapsulates RenderSystem options and settings that need
to be enabled/configured or disabled for a given Renderable to be processed in the desired
manner. Enabled RenderStates are automatically applied by RenderSystems and can be
used to place Textures into GPU texture units, specify how Textures should be filtered, or
determine if depth buffer testing should be performed, what testing functions should be used, and
so on.
4.5.1. Render State Architecture
RenderStates can be found in the rendersystem.states package. Figure 4.31 shows
the diagram. 60
Figure 4.31: Interfaces and collaborations of the rendersystem.states package.
4.5.2. Render States
Figure 4.32 shows the RenderState diagram, which is quite simple. At a minimum, they can
be enabled, disabled, and applied.
Figure 4.32: The RenderState interface.
4.5.2.1. Z-Buffer States
A ZBufferState specifies whether depth buffer testing should be performed and, if so, how
the test should behave when applied. Figure 4.33 shows the diagram for this interface. 61
Figure 4.33: The ZBufferState interface.
4.5.2.2. Texture States
A TextureState specifies which Textures will be used, in what GPU texture units they will be stored, how they will be filtered, and other settings that the RenderSystem needs to be aware of when using Textures. Figure 4.34 shows the detailed diagram.
Figure 4.34: The TextureState interface. 62
4.5.2.3. Front Face States
A FrontFaceState, shown in Figure 4.35, specifies how the "front" and "back" sides, or
"faces", of a surface are identified. When vertices and indices used to define a Mesh are specified, they follow a particular ordering convention, clockwise or counter-clockwise, when projected onto the screen, which may vary depending on which DCC tool is used.
The FrontFaceState.VertexWinding enumeration can be used to communicate this ordering convention to a RenderSystem. Since only once side of a polygon/surface is expected to be visible at any given time, identifying its front face allows the RenderSystem to perform a process called back-face culling.
Back-face culling can be described as a test to determine whether a polygon should be rendered or not based on its orientation. If a client specifies that the front-facing polygons in a
Renderable have a clockwise winding, but the polygons end up with a counter-clockwise winding when projected onto the screen, then the polygon is said to be facing away from the
Camera and is not rendered.
Figure 4.35: The FrontFaceState interface.
This optimization allows the GPU to avoid wasting resources trying to render polygons that not meant to be displayed. This is also the reason why, when a player's Camera mistakenly 63
moves to an unexpected location, such as going through a "solid" 3D model, the player can
actually see through its opaque polygons as if they were not there at all –their back-faces were
never rendered.
4.6. Rendering Systems and GPU Shader Programs
In contrast to RenderSystems, which are only meant to make direct calls to the underlying 3D
APIs, GPU Shader Programs run directly on the GPU. The modern graphics pipeline is
programmable and divided into several stages. GPU Shader Programs allow developers to specify
the logic these stages should follow during the rendering process.
4.6.1. GPU Shader Program Architecture
The GpuShaderProgram and related types are defined in the rendersystem.shader
package. Figure 4.36 shows the package diagram. There are several types relevant to a
GpuShaderProgram, namely the Stage of execution, its Inputs, and its Context, which
will be covered in the following sections.
Figure 4.36: Interfaces and collaborations in the rendersystem.shader package. 64
4.6.2. GPU Shader Programs
A GpuShaderProgram is a user-defined program for direct execution by the graphics
processor. The source code is loaded and stored in a Shader by the ShaderManager, which
is then passed to the GpuShaderProgram. The program is then submitted to the GPU for it to
be built. The build process has a dependency on the RenderSystem because it is implemented
by the underlying 3D API. For this reason, RenderSystems are always responsible for
instantiating GpuShaderPrograms. Figure 4.37 shows the GpuShaderProgram diagram.
Currently, two program Types are available, RENDERING and SKYBOX, to process
Renderables and SkyBoxes respectively.
Figure 4.37: The GpuShaderProgram interface, with less relevant details ommitted. 65
4.6.3. Program Stages
The modern pipeline has several Stages. The shaders in RAGE only implement vertex and
fragment shaders, which are the minimum number of required Stages for the pipeline to work.
Clients need to provide the source code for all relevant Stages of the specific program to be
used.
4.6.3.1. Vertex Shaders
The vertex shader is the first programmable pipeline Stage vertices go through. It runs
roughly once for each vertex in the stream [Khronos, 2015] and outputs one vertex per input
vertex –i.e. there is a one-to-one mapping. Figure 4.38 shows the sequence of transformations
vertices go through when they are processed by the vertex shader.
Coordinate Space Transformations in Vertex Shader
Object-SpaceObject-Space Model-MatrixModel-Matrix World-SpaceWorld-Space
Projection-MatrixProjection-Matrix View-SpaceView-Space View-MatrixView-Matrix
Clip-SpaceClip-Space Figure 4.38: Vertex shader transformations. Gray boxes specify relative coordinate spaces. Red boxes specify transforms that move vertices across coordinate spaces.
Each vertex is defined as a 4-component vector. Vertices in the vertex buffer are stored in,
and sent to, the GPU in object-space. Vertices are in an object-space coordinate system when the
scalar values are specified relative to the origin of the object whose shape they define. This does
not include any of the TRS world transforms. These TRS transforms are concatenated into a 66 single matrix, called the model25 matrix. Multiplying vertices in object-space by the model matrix moves the vertices into world-space.
Being in world-space means that vertex positions are now specified relative to the world's origin, rather than the object's. At this point, the object's vertices have been translated, rotated, and/or scaled in some way. Still, not all of them will necessarily be displayed. Only the content within the Camera's viewing Frustum should show up in the final image.
To move objects into Camera-space, vertices in world-space must have the viewing transform applied to them –i.e. they must be multiplied by the view matrix. This moves the vertices into a coordinate system that is relative to the viewer. After this transformation, objects that are still inside the Camera's view Frustum, in whole or in part, need to be processed and then projected from 3D space onto a 2D plane.
To transform vertices from view space into clip space, they must be multiplied by the projection matrix. While the projection matrix may represent an orthographic or perspective projection, games rely on perspective projections to render their worlds in a way that allows players to identify objects that are closer or farther away. Vertices in clip-space are now ready to exit the vertex shader and continue their journey through the pipeline.
4.6.3.2. Fragment Shaders
The fragment shader is the last programmable Stage of the pipeline. Like the vertex shader, it is required. This stage processes fragments generated by the rasterization process into a set of colors and a depth value. Rasterization is the non-programmable stage of the pipeline preceeding this one.
25 The "model" matrix is sometimes referred to as the "world" matrix. While this author thinks calling it the world matrix is more appropriate, most texts use the term "model", so it is used here to avoid potential confusion. In fact, even the deprecated fixed-function OpenGL pipeline itself used the "model" term, as shown by the GL_MODELVIEW constant for use with glMatrixMode. 67
A fragment is a collection of values produced by the rasterizer [Khronos, 2016]. It can be thought of as a "candidate pixel". It is during this phase that Textures loaded into GPU texture units are sampled based on their texture coordinates and lighting calculations are performed to determine how the scene's illumination contributes to the final color of the pixel. The built-in fragment shader includes these lighting and sampling calculations for Renderables.
4.6.3.3. Lighting and Shading Models
Scene illumination is based on models, which are simplified abstractions of the real world. RAGE uses the Phong Lighting Model (PLM) for illumination. This model specifies ambient, diffuse, and specular (ADS) components for Lights and works by combining them together. Figure
4.3926 shows the isolated ADS components and the result of combining them together.
Figure 4.39: Components of the Phong Lighting Model. From left to right they are ambient, diffuse, specular, and the combined result.
There're several shading techniques that can be implemented within the PLM. One technique is Gouraud Shading (GS), which performs lighting calculations on a per-vertex basis in the vertex shader and then relies on the rasterizer to linearly interpolate the results. While this method is faster, the quality of the results can be negatively affected by a model's polygon count and/or shape.
26 By Brad Smith - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1030364 68
Another technique, used in RAGE, is Blinn-Phong Shading (BPS), which is computed on
a per-fragment basis in the fragment shader. It is slower than GS, but provides more realistic and
better-looking results. The difference between GS and BPS is clearly shown in Figure 4.4027.
Figure 4.40: Comparison between the Gouraud Shading model (left) and the Blinn-Phong Shading model (right). Note how the specular highlight on the left looks jagged, while the one on the right looks more realistic and smooth.
4.6.4. Program Inputs Attributes and Uniforms
Inputs for GpuShaderPrograms can be represented by attributes or uniforms. Attributes are
used for data that varies within a single rendering pass, such as the scalar values being streamed
from the vertex buffer and grouped to be interpreted as 4-component vectors by the vertex shader.
Uniforms are used for data that remains constant during a single rendering pass, such as the
projection matrix that will be used to transform each vertex attribute.
Currently, the built-in GpuShaderProgram implementations are stored inside the
rendersystem.shader.glsl package. Listing 4.2 shows input declarations in the vertex
shader for Renderables, found in assets/shaders/glsl/renderables.vert.
27 Spheres were rendered locally by running the "phonglighting" sample program, published in [Bible6, 2014], and edited to be side-by-side with GIMP. 69
layout (location = 0) in vec3 vertex_position; layout (location = 1) in vec2 vertex_texcoord; layout (location = 2) in vec3 vertex_normal; // ... uniform struct matrix_t { mat4 model_view; mat4 projection; mat4 normal; } matrix; Listing 4.2: Vertex shader shows declarations for attributes and uniforms.
The vertex shader uses attributes for variable data and uniforms for constant data. The model_view matrix is a single matrix that already holds the concatenation of the model and view matrices, computed by the client-side (i.e. RAGE) before submitting it to the server-side
(i.e. the GPU). The normal matrix is the transpose of the inverse of the model_view matrix and is specifically used to transform vertex normals in a way that they remain perpendicular to the surface they are on.
Listing 4.3 shows how the server-side inputs from the previous listing are represented by
RAGE on the client-side28.
private GlslProgramAttributeBufferVec3 positionsBuffer; private GlslProgramAttributeBufferVec2 texcoordsBuffer; private GlslProgramAttributeBufferVec3 normalsBuffer; // ... private GlslProgramUniformMat4 modelViewMatrix; private GlslProgramUniformMat4 projMatrix; private GlslProgramUniformMat4 normalMatrix; Listing 4.3: GlslRenderingProgram bindings to vertex shader attributes and uniforms.
As these code listings show, these implementations are tightly coupled to the actual shader programs they encapsulate. This coupling is necessary because the class is a client-side representation of the server-side shader program running directly in the GPU28.
28 In this context, client refers to RAGE itself and server refers to the graphics hardware –i.e. the GPU. 70
4.6.5. Program Context
The GpuShaderProgram.Context, shown in Figure 4.37, encapsulates the inputs for a
GpuShaderProgram. The Context allows the GpuShaderProgram to receive all the
arguments it must submit to the graphics pipeline from a single fetch(Context ctx)
invocation, which keeps GpuShaderPrograms themselves simpler.
A Context holds the Renderable about to be processed, view and projection
matrices, an AmbientLight, and a Light. These are then sent to the server28 as attributes or
uniforms by the GpuShaderProgram implementation on the client28, as shown in Listing 4.4.
// modelview_matrix ← view_matrix * model_matrix Matrix4 mv = view.mult(renderable.getWorldTransform()); normalMatrix.set(mv.inverse().transpose()); // ...
Listing 4.4: A GpuShaderProgram sets a uniform declared in Listing 4.3. 71
5. EXTERNAL FRAMEWORK DEPENDENCIES
RAGE has some external dependencies. Specifically, these are the RAGE Math Library (RML) and the OpenGL bindings for Java (JOGL).
The RML is not an off-the-shelf dependency, but was actually created for use in this project. However, it is a separate codebase in a separate repository and can be used by itself on non-RAGE related projects. The RML supports several types that are important to computer graphics and games in general, including vectors and matrices.
The JOGL bindings allow the GL4RenderSystem implementation of the
RenderSystem interface to make direct calls to the OpenGL API. Other than noting this fact, discussing the specifics about the JOGL bindings is outside the scope of this report.
5.1. Vector, Matrix, and Quaternion Math with RML
Linear algebra is a very important part of scene management transformations and real-time rendering, both of which are core sub-systems in RAGE. Figure 5.1 shows public interfaces exposed by RML.
RAGE Math Library (RML) Vector2 Matrix2 Quaternion
Vector3 Matrix3 Angle
Vector4 Matrix4 Figure 5.1: Math library used in RAGE.
These public interfaces are built by extending simpler package-private interfaces, such as
FourDimensional and LinearlyInterpolable
29 Enumerating all the package-private interfaces is outside the scope of this report. Note, however, that the operations they define are later listed in implementation class diagrams. 72
5.1.1. RML Package Architecture
Figure 5.2 shows the public interfaces and implementations while ommitting most package-
private contents for readability.
Figure 5.2: RML structure overview. Classes are single-precision floating-point implementations. Package-private Matrix and Vector shown.
With the exception of Matrix and Vector, all the interfaces and classes in the diagram
are public. Ommitted private interfaces represent simpler types (e.g. ThreeDimensional,
FourDimensional, etc.) or independent operations (e.g. Invertible, Scalable, etc.).
Interfaces like Matrix, Vector, and Quaternion are larger extensions of several simpler,
and often generic, interfaces.
Implementation classes are immutable. As such, all methods return a new instance with
the results of the operation while leaving the operands unchanged. Game engines often need to
rely on multi-threading. Immutable objects make it easier to perform computations in separate
threads without having to worry about synchronization or hard-to-trace bugs caused by race
conditions. The main purpose of the interfaces is to allow single- or double-precision floating-
point types to be used. 73
This does come with a trade-off. While avoiding race conditions and related issues is
easier to do, the Java Virtual Machine (JVM) garbage collector has more content to clean up as
references to the original operands get discarded during computations.
5.1.2. Vectors
Vectors in the RML are column vectors. This is implied in the interfaces by their method
signatures. Multiplication between vectors and matrices is one clear instance of this. Those
operations are always of the form v '=Mv rather than v '=vM , where v is the original
column-vector, M the original column-major matrix, and v ' the resulting column-vector.
For example, the operation could be written as Vector4 v = M.mult(v), but trying to write
Vector4 v = v.mult(M) will result in a compilation error. Figure 5.3 shows the
Vector4f implementation class.
The class has several factory methods, which when combined with their immutability,
allow some optimizations that avoid unnecessary instantiation of new objects. For example, every
time Vector4f.createZeroVector() is invoked, a reference to a cached instance is
always returned –i.e. no new instances are created. It also adds flexibility because it allows
greater use of method overloading and clarity of expression in how code is written –i.e. it is "self-
documenting". 74
Figure 5.3: The Vector4f class.
5.1.3. Matrices
Matrices in the RML have column-major orderings, as implied by the column-vectors of the
previous section. Figure 5.4 shows the Matrix4f implementation class.
In addition to the same types of caching optimizations mentioned in the previous section,
the factory methods make instantiation more readable. For example, writing Matrix4 m =
new Matrix4f() does not make it clear what the initial values of the matrix are supposed to
be. Is it an identity matrix or is it a zero matrix? 75
On the other hand, Matrix4 m = Matrix4f.createIdentityMatrix() makes it clear without having to rely on potentially outdated documentation as time goes on. In the off-chance that this method ends up doing something different (e.g. creating a zero-matrix), it becomes clear that the behavior is incorrect, whereas the default constructor cannot communicate this.
Figure 5.4: The Matrix4f class. 76
5.1.4. Quaternions
A Quaternion represents an orientation in space. The axis of rotation is represented by a
Vector3 and amount of rotation by an Angle. The RML implementation is provided by the
Quaternionf class. The class follows the same pattern, based on private constructors and
public factory methods. Figure 5.5 shows its diagram.
Quaternions, which only require 4 components, are more space-efficient than
homogeneous matrices (e.g. Matrix4), which require 16 scalar values. They are also good
options to perform smooth interpolation between values relative to time and avoid limitations of
matrix transforms, such as Gimbal Lock30, where a rotation axis can be "lost" under specific
circumstances. However, they are not necessarily as intuitive to understand and/or use.
Quaternions do not negate the need for matrices. The graphics pipeline does not
support Quaternions, which means they must always be converted to a Matrix
representation before their values are sent to the shaders running in the GPU. This conversion is
already required by the Quaternion.toMatrix3() interface, and implemented by
Quaternionf class.
30 https://en.wikipedia.org/wiki/Gimbal_lock 77
Figure 5.5: The Quaternionf class.
5.1.5. Angles
Unlike most math libraries publicly available, which use primitive types such as float and
double to represent angles in degrees and radians, RML represents these with the Angle
interface. This allows users to more easily write code within the problem domain and minimize
errors.
The error-prone nature of using primitive types, instead of Abstract Data Types (ADTs),
is widely documented. One of several notable errors that resulted in a spectacular (and costly)
failure was the destruction of the Mars Climate Orbiter (MCO) in September 23, 1999 –to the
tune of $327.6 million USD [NASA Sheet, 1999]. A postmortem examination report [NASA
Board, 1999] concluded that the craft and its mission were doomed to failure due to a unit
conversion error. 78
The MCO MIB has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of ... trajectory models. Specifically, thruster performance data in English units instead of metric units was used in the software application code ... The output from the SM_FORCES application code as required by a MSOP Project Software Interface Specification (SIS) was to be in metric units of Newton-seconds (N-s). Instead, the data was reported in English units of pound-seconds (lbf-s). [NASA Board, 1999] (my emphasis)
While it could be argued that following documentation and specifications, both of which were available, would have prevented the error, the problem is that external documents cannot impose their requirements on any piece of software. The same report goes on to note this fact.
The SIS, which was not followed, defines both the format and units of the AMD file generated by ground-based computers. Subsequent processing of the data from AMD file by the navigation software algorithm therefore, underestimated the effect on the spacecraft trajectory by a factor of 4.45, which is the required conversion factor from force in pounds to Newtons. An erroneous trajectory was computed using this incorrect data. [NASA Board, 1999] (my emphasis)
Had the units been represented as their own separate ADTs, such as Newton and
Pound, instead of primitive types that simply hold scalar values that can be (mis)interpreted in many ways, such as double and float, the error would have been caught without even requiring test cases. Simply trying to build the code would have exposed the problem, because explicit unit conversion could have been required by the language itself due to the inherent incompatibility of different types.
While a game engine like RAGE is certainly not a mission-critical piece of software like
NASA's MCO, the same argument still applies. In fact, it can be argued that, in the worst of cases, an unreliable engine will cause game clients to be unreliable, which can ultimately doom a game project and its development studio to failure. This is the main reason for providing the
Angle interface with Degreef and Radianf implementation classes. Figure 5.6 shows their diagram. 79
Figure 5.6: The Radianf and Degreef classes as implementations of the Angle interface.
5.2. Writing Test Cases with the TestNG Unit-Testing Framework
Part of the engineering process is actually testing the software. There are two basic kinds of test cases in both RAGE and RML: automated and interactive.
Automated test cases rely on the TestNG31 framework. It provides many useful features compared to the standard JUnit, such as testing for expected exceptions to be thrown with specific messages, among others. Over 390 test cases in RML are automated and can be executed quickly32.
Interactive test cases are those that users need to visually inspect, such as the result of a rendered image (e.g. lighting), or verify that objects in the scene are behaving properly by moving them around (e.g. transformations). The interactive test cases mostly focus on verifying the functionality within the scene.generic and rendersystem.gl4 packages.
31 http://testng.org 32 The IDE can be set up by using the testing-suite.xml configuration file included in the repository. 80
6. FUTURE WORK
Game engines are large and complicated pieces of software. As such, the scope of this project was limited to several core areas. These included scene management, rendering, and external assets.
This means there is a large number of missing features, some of which will be briefly mentioned.
Ultimately, the goal should be to prefer solutions that are compatible and consistent with the same free-software principles RAGE is built on and to support both Microsoft Windows and
GNU/Linux platforms. Support for Apple-based hardware and software is generally more difficult because they tend to lag behind. For example, both Windows and GNU/Linux support OpenGL
4.5 at the time of this writing, while macOS is no longer supported by OpenGL SuperBible 7 code samples due to their implementation being out of date33 –still on OpenGL 4.3.
Note that, while most features are more likely to be added in RAGE, some others may get added to RML instead. Any ambiguous cases that come up are likely to benefit from further discussion.
6.1. Multiple Light Sources
RAGE is currently limited to a single Light per rendering command. To prevent unexpected issues for developers, the GenericSceneManager currently prevents game clients from creating more than one Light instance. This is an arbitrary, and temporary, limitation to help game clients avoid issues. The SceneManager is capable of handling them.
Adding this feature would require refactoring other components, such as the
RenderSystem, GpuShaderProgram, and the underlying shaders they represent so that they can receive the proper inputs. Currently, shaders can only handle two input shader uniforms: one Light and one AmbientLight.
33 "The source no longer supports Mac because the OpenGL implementation is too out of date to run any of the new samples", http://www.openglsuperbible.com/example-code/ 81
6.2. User Input Devices and Management
Supporting a wider variety of input devices, such as joysticks and gamepads, is a basic expectation of most players. Currently, the built-in Java listeners are being used, but these are not enough to support devices beyond mice and keyboards.
Currently, SAGE relies on the JInput34 library. This author's experience with JInput was inconsistent; it worked as intended within the Microsoft Windows platform, but it was not usable in the GNU/Linux platform. Attempts to detect or read devices, which are owned by the root user, were unsuccessful35. Whether this was an issue with SAGE's use of JInput, or JInput itself, was not clear to this author. However, based on some initial research, JInput may actually be the only practical choice available.
6.3. Sound Processing
The system does not have support for loading or playing music, sounds, or other audio effects at this time. Since sound processing requires hardware support, bindings will be required as well.
There are Java bindings for OpenAL36 (i.e. JOAL22) which can be used in a very similar way to how JOGL is being used for graphics processing.
6.4. Animations
Animations are an important part of every game. Animation data may be defined in different ways, depending on the underlying format, or not exist at all. For example, the Wavefront file format, which is what RAGE currently relies on, does not support animations37.
Other formats, such as OgreXML, support animation38. However, RAGE does not currently have a concrete MeshLoader that can process the OgreXML file format (e.g. 34 https://java.net/projects/jinput 35 Launching game clients as the root user were equally unsuccessful, although this kind of workaround is undesirable. 36 http://www.openal.org/ 37 "Obj files also do not support mesh hierarchies or any kind of animation or deformation, such as vertex skinning or mesh morphing." https://en.wikipedia.org/wiki/Wavefront_.obj_file#Other_geometry_formats 38 This author used a Blender plugin, blender2ogre, in the CSc-165 course of Spring 2014 for a semester project. However, recent searches have produced only dead repository links, so the actual state of this plugin appears to be unknown. 82
OgreXmlMeshLoader). Other details to consider include where the animation data for each
Renderable part of a SceneObject would be stored (e.g. SubMesh), how to "play" the animations, and so on.
Note that the current design separating Meshes from SubMeshes is already consistent with the OgreXML file format, which is simply an XML document defining a Mesh and the
SubMeshes that it is made of, potentially including bone assignments and other details.
Note that the WavefrontMeshLoader is currently implemented as a trivial case with one-Mesh/SubMesh, whereas a, presumably named, OgreXmlMeshLoader will need to process several SubMeshes for a single parent Mesh, and so on.
6.5. Networked Multi-Player
Most games are more fun when played with other people. While RAGE allows game clients to support local multi-player with split-screens, being able to play over the network is a must. It allows players to be more comfortable, makes it more difficult for them to cheat by looking at each other's screens, and provides a more practical way to teach networking concepts to students, among other benefits.
6.6. Bounding Volumes
Bounding volumes are geometrically simple shapes, such as boxes and spheres, that completely surround individual objects in a scene, but are typically not rendered. SceneObjects have unpredictably complex geometric shapes that would make certain conditions (e.g. collision detection, visibility, etc) more time-consuming to perform. Using simpler shapes with less polygons than the objects they surround can significantly speed up these tests.
Bounding volume examples include axis-aligned bounding boxes (AABBs) and bounding spheres, among others39. Enabling less expensive computational methods to quickly check if two
39 https://en.wikipedia.org/wiki/Bounding_volume#Common_types_of_bounding_volume 83 objects are "in contact" with one another, test if they are inside a Camera.Frustum before submitting them to RenderSystems (even if partially), and so on, can make other features like
Collision Detection and Rigid-Body Physics, described in Section 6.7, easier to implement and practical to use.
6.7. Collision Detection and Rigid-Body Physics
RAGE lacks the ability to detect SceneObject collisions and compute how they should react based on relevant properties and forces. There are multiple physics-based libraries, such as
JBullet40 and ODE4J41. However, these are CPU-based libraries. The idea of using GPU hardware-acceleration via OpenCL42, with the JOCL22 bindings, sounds like an interesting area for further investigation.
Adding support for Bounding Volumes, as described in Section 6.6, could help make the process of integrating physics support easier than trying to do it the other way around. It is worth considering.
6.8. Space Partitioning Scene Managers and Visibility Determination
The space in a scene can be partitioned, or divided, into different sections. Different
SceneManagers might be better suited for specific visibility determination algorithms, some of which might focus on indoor or outdoor environments. Some examples, in addition to the current implementation, include Binary Space Partitioning (BSP), Quadtree, Octree, and Portal-based
SceneManagers.
6.9. Graphics APIs: Moving from OpenGL to Vulkan
The Vulkan API was released in 2014. A recent game with Vulkan API support, since August of
2016, is Doom43, by id Software. With Vulkan and references for it already out, it should make
40 http://jbullet.advel.cz/ 41 https://github.com/tzaeschke/ode4j 42 https://www.khronos.org/opencl/ 43 https://en.wikipedia.org/wiki/Doom_(2016_video_game)#Release_and_marketing 84 sense to add a Vulkan-based RenderSystem implementation when the drivers are more mature and bindings for Java are made available.
6.10. Artificial Intelligence and Behavior Trees
The field of artificial intelligence (AI) has become more mainstream in recent years, especially with the popularization of fancy terms like "cloud" and "big data". The widely publicized Google
DeepMind Challenge44 was a match between AlphaGo45 and and Lee Sedol46. Sedol, an 18-time world champion in the game of Go, was defeated 4-1 by AlphaGo.
Games rely on AI for non-playable characters (NPCs) and wildlife, among other things.
Some examples of AI include path finding for navigation, tactical planning for an ally (or adversary), and simpler behaviors like those of animals fleeing when players get too close or attacking them when they "feel threatened", and so on. Behavior trees47 are worth considering in this area.
6.11. Continued Maintenance, Optimizations, Testing, and Debugging
Any non-trivial piece of software is guaranteed to have bugs, even if their side-effects have not
(yet) been observed. As such, adding test cases to verify correct behavior of existing or planned features, correcting possible ambiguities or inconsistencies in documentation, and other details need to be taken into account. It is necessary to keep documentation up-to-date and relevant.
While maintenance is important, performance cannot be ignored. There are always areas for improvement to allow the engine to do more with the resources it currently has available in its platform. Profilers, such as VisualVM48, can help measure actual system behavior in order to identify potential performance bottlenecks, or frequently executed sections, so that efforts can be focused in those areas. Even a few small improvements in a section of code that is executed
44 https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol 45 https://en.wikipedia.org/wiki/AlphaGo 46 https://en.wikipedia.org/wiki/Lee_Sedol 47 https://en.wikipedia.org/wiki/Behavior_tree_(artificial_intelligence,_robotics_and_control) 48 https://visualvm.github.io/ 85
frequently (e.g. 90% of the time) can produce better results than more significant improvements
in a section that is barely executed at all (e.g. 10% of the time).
The importance of test cases cannot be over-emphasised. While RAGE might appear to
behave normally in the systems, hardware, and platforms used for this project, this does not mean
it will behave the same way when any combination of these is changed. Just changing a video
card or updating a graphics driver version can impact the engine's behavior. As the framework's
rate of adoption increases, previously unknown issues will be discovered and will need to be
fixed. This is an on-going and never-ending process.
6.12. Adding New Features and Refactoring for Improvement
While RAGE has several useful features, it also lacks many others. As the system continues to
expand with new features, they also need to be continuously refactored in order to improve its
readability, modularity, and overall design. The system's design is not an arbitrary document –it is
the codebase itself.
6.13. Specific Areas in Need of More Immediate Attention
This section briefly lists a few areas that are worth paying attention to in the near future.
6.13.1. Scene Rendering and Shader Improvements
Renderable processing by RenderSystems can be improved. Currently, every time a
Renderable is about to be submitted to a GpuShaderProgram, the RenderSystem
requests and applies the RenderStates that have been assigned to the Renderable. These
RenderState changes can be expensive and finding ways to minimize them could improve
performance.
One specific example is GPU texture unit management, which is done by the concrete
TextureState implementation –i.e. GL4TextureState. The
EntityPerformanceTest, which is a stress test, appears to indicate a potential memory 86
bandwidth bottleneck, as shown in Figure 6.149. It shows the Used Dedicated Memory at 99%
while GPU Utilization hovers around 64%, sometimes dropping to the low-twenties.
While it is very likely that better handling of GPU texture units is possible and could
remove this apparent bottleneck, it is still an area that would require additional investigation.
Figure 6.1: GPU data under GNU/Linux during stress test. Note the Used Dedicated Memory and GPU Utilization values.
6.13.2. Scene Management Improvements
There is currently a single SceneManager implementation, the GenericSceneManager in
the scene.generic package. Adding support for bounding volumes, planes, and rays would
significantly reduce difficulties in trying to determine which SceneObjects are within a
Camera's viewing Frustum. In turn, this would help avoid submitting Renderables to the
49 All test were done with NVIDIA cards (GTX-770 and GTX-960M). It would be beneficial to test on GPUs from other vendors, such as AMD and Intel. The NVIDIA X Server Settings view is from GNU/Linux. The NVIDIA Control Pannel for Microsoft Windows 7 does not show this information. 87
RenderSystem only for their pixels to be discarded in the depth testing stage after the fragment shader has completed its execution.
This kind of optimization would be a noticeable performance improvement. For example,
SceneObjects that are farther than the Frustum's far plane are, presumably, not visible to the
Camera. Therefore, they should not get submitted to the RenderSystem. Those that are far away, but still visible, could be submitted using a lower Level of Detail (LOD), requiring less geometry to be processed. Such functionality still needs to be implemented. 88
APPENDIX A
Sample Scenes Rendered with RAGE
1. Multiple Viewports Test...... 89
2. Earth-Moon System Test...... 91
3. Asteroid and SkyBox Test...... 92
4. Point Light Illumination Test...... 93
5. Spot Light Illumination Test...... 95
6. Entity Performance Stress Test...... 96 89
1. Multiple Viewports Test
Figure A shows how the same scene can be observed from different points of view with four independent Cameras and Viewports, assigned to each other, in a single RenderWindow.
Figure A: Single scene observed from multiple Camera angles and Viewports.
This scene includes three visible Entity instances, each with a single SubEntity.
Each Entity is based on a single shared sphere Mesh, made up of only one SubMesh. The same SubMesh is assigned to each SubEntity.
There are two TextureStates and two Textures loaded, one for each, with the blue
Texture and TextureState being assigned to two of the Entity instances. The rotations are handled by a RotationController that gets automatically updated by the
SceneManager. Only the default AmbientLight is used to illuminate this scene.
Listing A shows the code responsible for defining the Viewports for the
RenderWindow. 90
@Override protected void setupWindowViewports(RenderWindow rw) { Viewport vp = rw.getViewport(0); vp.setDimensions(.5f, 0f, .5f, .5f); vp.setClearColor(Color.GREEN);
rw.createViewport(.5f, .5f, .5f, .5f); vp = rw.getViewport(1); vp.setClearColor(Color.GRAY.darker());
vp = rw.createViewport(0, 0, .5f, .5f); vp.setClearColor(Color.GRAY.darker().darker());
vp = rw.createViewport(0, .5f, .5f, .5f); vp.setClearColor(Color.GRAY.darker().darker().darker()); } Listing A: Defining Viewports for a RenderWindow.
Listing B shows the basic idea of how Cameras and Viewports are paired together.
@Override protected void setupCameras(SceneManager sm, RenderWindow rw) { Camera.Frustum.Projection proj = Projection.PERSPECTIVE; Camera cam1 = sm.createCamera(CAMERA_TL, proj); cam1.getFrustum().setNearClipDistance(0.1f); rw.getViewport(0).setCamera(cam1);
SceneNode root = sm.getRootSceneNode();
final String nodeName = CAMERA_TL + "Node"; SceneNode camNode1 = root.createChildSceneNode(nodeName); camNode1.setRelativePosition(0, 0, 4); camNode1.attachObject(cam1);
// same pattern for next 3 cameras omitted... } Listing B: Creating Cameras and pairing them with Viewports from Listing A. 91
2. Earth-Moon System Test
Figure B shows the Earth and the Moon, with the Moon's orbit bringing it closer to the Camera at the time the picture was taken. Both entities also spin around their own axes. The movement is managed by a RotationController and an OrbitController. A low-intensity
AmbientLight and a more noticeable directional Light are used to illuminate the scene.
Figure B: Textured Earth/Moon entities and a directional Light.
The code snippet in Listing C shows how to define directional Light and specify the direction light rays should come from. 92
@Override protected void setupScene(Engine eng, SceneManager sm) throws IOException { // ... Light.Type type = Type.DIRECTIONAL; Light light = sm.createLight("SunLight", type); sunLight.setDiffuse(Color.WHITE);
final String nodeName = sunLight.getName() + "Node"; SceneNode sunLightNode = root.createChildSceneNode(nodeName); sunLightNode.attachObject(sunLight); sunLightNode.setRelativePosition(1, 0, 1); // ... } Listing C: Creating and defining a directional Light for a scene.
3. Asteroid and SkyBox Test
Figure C shows a textured asteroid Entity inside a SkyBox using space Textures and a directional Light for illumination. There are some Node.Controllers to asteroid rotation and the Camera's orbit around the asteroid SceneNode.
Figure C: Asteroid Entity inside a textured SkyBox. 93
Listing D Shows the relevant code sample to responsible for creating the SkyBox and assigning properly rotated Textures to each of its sides.
// ... Configuration cfg = engine.getConfiguration(); TextureManager tm = engine.getTextureManager();
tm.setBaseDirectoryPath(cfg.getValue("assets.skyboxes.path")); Texture front = tm.getAssetByPath("front.png"); Texture back = tm.getAssetByPath("back.png"); Texture left = tm.getAssetByPath("left.png"); Texture right = tm.getAssetByPath("right.png"); Texture top = tm.getAssetByPath("top.png"); Texture bottom = tm.getAssetByPath("bottom.png"); tm.setBaseDirectoryPath(cfg.getValue("assets.textures.path"));
AffineTransform xform = new AffineTransform(); xform.translate(0, front.getImage().getHeight()); xform.scale(1d, -1d);
front.transform(xform); back.transform(xform); left.transform(xform); right.transform(xform); top.transform(xform); bottom.transform(xform);
SkyBox box = sm.createSkyBox("SkyBox"); box.setTexture(front, SkyBox.Face.FRONT); box.setTexture(back, SkyBox.Face.BACK); box.setTexture(left, SkyBox.Face.LEFT); box.setTexture(right, SkyBox.Face.RIGHT); box.setTexture(top, SkyBox.Face.TOP); box.setTexture(bottom, SkyBox.Face.BOTTOM);
sceneManager.setActiveSkyBox(box); // ... Listing D: Creating and enabling a textured SkyBox.
4. Point Light Illumination Test
The point Light scene, shown in Figure D, has two Entity instances based on a sphere Mesh, with the smaller of the two appearing to emit Light. This effect is achieved by having both the smaller Entity and the Light attached to the same SceneNode, and setting the 94
SubEntity's Material emissive property set to its maximum intensity value –i.e.
Color.WHITE.
Figure D: Spheres lit by a point Light and an emissive Material.
A careful look at the Light's reflection in the larger sphere subtly shows that pixels farther away from the Light source are slightly dimmer due to attenuation. The code is very similar to that shown in Listing C, with the key differences being that the Light is of
Light.Type.POINT instead of Light.Type.DIRECTIONAL and the positioning of the
SceneNode it is attached to. 95
5. Spot Light Illumination Test
The scene shown in Figure E verifies that spot Lights work as intended. It has a large cube
Entity, attached to a SceneNode with a 45 degree rotation about the Y axis, lit by the
Light, with one of its edges facing the Camera.
Figure E: Large cube lit by a spot Light and an emissive cone Entity.
A spot Light's area of effect is defined by a cone shape and behaves like a flashlight.
Just like in Listing C, the Light here is created as a Light.Type.SPOT and attached to a
SceneNode. The SceneNode's rotation determines the Light's direction. As in Figure D, attenuation effects can be observed in Figure E, particularly on the left and right edges of the
"heart"-shaped surface area created by the Light's illumination cone. 96
6. Entity Performance Stress Test
The scene shown in Figure F is intended to be a simple stress test to observe how much content
RAGE can handle at once in a single scene.
Figure F: Performance test with thousands of objects in a single scene.
The scene contains 1,20150 Entity instances and a Texture-mapped SkyBox. Each
Entity is attached to a SceneNode, all of which are managed by their own
RotationController, OrbitController, or both. AmbientLight intensity is low and a directional Light completes the scene's illumination.
There are three types of asteroid Meshes in use, in addition to the Earth's sphere Mesh, which means that only a total of four Meshes were loaded for this scene. The relevant portions of code responsible for this scene is shown in Listing E.
50 One Earth and 1,200 Asteroids 97
@Override protected void setupScene(Engine engine, SceneManager sm) throws IOException { // ... Entity planet = sm.createEntity("Planet", "earth.obj"); planetNode = root.createChildSceneNode("PlanetNode"); planetNode.attachObject(planet); // ... cameraNode.lookAt(planetNode); RotationController planetCtrl = /* ... */ planetCtrl.addNode(planetNode); sm.addController(planetCtrl);
float orbitalSpeedLevel = 0.075f; OrbitController ringOrbitCtrl = new OrbitController(planetNode); ringOrbitCtrl.setDistanceFromTarget(75); ringOrbitCtrl.setSpeed(orbitalSpeedLevel); sm.addController(ringOrbitCtrl); // ... for (int i = 0; i < MAX_TOTAL_ASTEROIDS; ++i) { // alternate between 3 different asteroid // meshes for some variation and increased // stress on the system int j = (i % 3 + 1); String name = "Asteroid-" + i; String mesh = "asteroid_" + j + ".obj"; Entity rock = sm.createEntity(name, mesh); SceneNode rockNode = /* ... */ rockNode.attachObject(rock); ringOrbitCtrl.addNode(rockNode);
// set different rotation speeds/axes for each asteroid float x = random.nextFloat(); float y = random.nextFloat(); float z = random.nextFloat(); float s = random.nextFloat(); Vector3 axis = Vector3f.createFrom(x, y, z); RotationController ctrl = new RotationController(axis, s); ctrl.addNode(rockNode); sm.addController(ctrl); // ... } // ... } Listing E: Creation of SceneObjects, SceneNodes, and Node.Controllers for stress test. 98
APPENDIX B
Source Code Repositories
The source code for RAGE and the RML are actually two separate Git51 repositories. They can be publicly accessed at https://gitlab.com/ghost-in-the-zsh/rage and https://gitlab.com/ghost-in-the- zsh/rml respectively. Additional information including licensing terms, contribution guidelines, usage, and so on can be found in the associated documentation.
Note that the RML can be used independently and by itself for non-RAGE purposes, but
RAGE requires RML to work. This separation allows more flexibility in its use, especially when
RAGE is not needed (e.g. in CSc-155).
51 https://git-scm.com/ 99
BIBLIOGRAPHY
[Patterns, 1995] Gamma, Erich. Helm, Richard. Johnson, Ralph. Vlissides, John. (1995). Design
Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley.
[Lohmann, 2010] Von Lohmann, Fred. (2010). Unintended Consequences: Twelve Years under
the DMCA. https://www.eff.org/files/eff-unintended-consequences-12-years.pdf
[Khronos, 2015] Khronos Group. (2015). Vertex Shader.
https://www.khronos.org/opengl/wiki/Vertex_Shader
[Khronos, 2016] Khronos Group. (2016). Fragment.
https://www.khronos.org/opengl/wiki/Fragment
[Bible6, 2014] Sellers, Graham. Wright, Richard. Haemel, Nicholas. (2014). OpenGL SuperBible
Sixth Edition Comprehensive Tutorial and Reference. Addison-Wesley.
[NASA Sheet, 1999] NASA. (1999). Mars Climate Orbiter Fact Sheet.
https://mars.jpl.nasa.gov/msp98/orbiter/fact.html
[NASA Board, 1999] NASA. (1999). Mars Climate Orbiter Mishap Investigation Board Phase I
Report. ftp://ftp.hq.nasa.gov/pub/pao/reports/1999/MCO_report.pdf