The Video Game Asset Pipeline A Pattern Approach to Visualization James Lear Student ID: 91002002 [email protected] The University of the West of England Faculty of Environment and Technology This thesis is submitted for the degree of Doctor of Philosophy March 2021 Director of Studies Supervisor Professor Richard McClatchey Dr Simon Scarle [email protected] [email protected] a Abstract Video games consist of virtual worlds modelled as an approximation of either a real or imaginary environment. The amount of content required to populate the environments for Triple-A (AAA) video games doubles every few years to satisfy the expectations of the end-users. For this reason, the art and design discipline now constitute the maJority of those employed in a video game studio. The artists use Digital Content Creation (DCC) tools to design and create their content; tools not originally designed for video game asset creation. Ultimately the artists require to preview their content in the form of source assets in the runtime environment, the game engine, to ensure they provide an accurate rendering of their original vision. However, there exists a barrier to achieve this workflow; the original source assets are persisted in a proprietary format, information rich to handle future edits, and the final runtime environment requires the assets to be lightweight ready for fast and efficient loading into the game engine. The video game industry has solved this problem by introducing a fast and efficient workflow known as the asset pipeline. The asset pipeline is recognized within video games technology as a general reusable solution to the common problem of converting source assets into their final runtime form as expected by the runtime game engine. Although the asset pipeline defines a series of stages that all content must follow from inception to their final realization a single solution does not exist to satisfy all projects. Likewise, within the discourse of patterns, a pattern is defined as a general reusable solution to a problem operating under a certain context. Originating in the field of architecture (Alexander, 1979) patterns have now been discovered and mined in numerous domains including software engineering (Gamma et al., 1995). Within the field of software engineering patterns exists at several levels of abstraction including architectural, design- level and low-level idioms. The world of video games technology and patterns intersects in the form of one set of patterns identified by Nystrom (2014), although these are very much low-level idioms and certainly do not encompass the challenge of the asset pipeline as found in video game production. This research addresses this shortfall and formalizes the asset pipeline into a catalogue of patterns for use both within the video game industry and to satisfy the wider audience of interactive real-time visualization in general. Interactive real-time visualizations consist of both the navigation and viewing application, executing in a runtime environment, and the digital content providing the data source for the visualization. Their workflow production draws parallels with that of the video game industry. The designers of such visualizations use the iterative process of create, review and modify. Creation of the source asset within the DCC tool, preview within the visualization runtime and subsequent modification in the DCC tool. However, the video game industry is tempered by a number of problems hindering proliferation of the asset pipeline as a general reusable solution. The video game industry is shrouded in secrecy preventing the natural dissemination of information. Software developers operating within the industry are subject to Non-Disclosure Agreements (NDAs) protecting intellectual property invested in software tools such as the asset pipeline. The video game industry is relatively young, being fifty years old, and as such a set of agreed-upon terms and their definition has been slow to develop. This is compounded by the asset pipeline technology operating at the fault line of two disciplines: the engineers developing the runtime and the artists creating digital content. In such an interdisciplinary field language barriers exist. The characteristics and properties of patterns address these problems. They identify, name and provide a common vocabulary for specific problem-solution abstractions. ii They capture expertise and make knowledge accessible to non-experts. The communication of which enables domain independent solutions. This novel research formalizes the asset pipeline into a catalogue of patterns consisting of an architectural pattern named the ASSET PIPELINE and the component patterns of DCC EXPORT, INTERMEDIATE FILE FORMAT and ASSET BUILD PROCESS. Under the iterative spiral model methodology aligned to the framework of the Pattern Languages of Programs (PLoP) workflow. This involves the unique approach of shepherding, pattern mining, a writers’ workshop and pattern writing. All of which culminated in the publication of the pattern catalogue in the Association for Computing Machinery (ACM) for wider consumption and use in further domains of visualization. The asset pipeline catalogue was instantiated and applied in two domains: architectural visualization (ArchViz) and graph visualization (GraphViz) under the process of sequential application of the pattern components. This resulted in real-time exploratory visualizations that not only validate the pattern application but serve for wider research opportunities in the future. Such opportunities include expansion of the pattern language, refinement of the instantiated visualizations, development of a software framework and further pattern mining in other avenues of games technology. iii Acknowledgements I would like to express my sincere gratitude to my supervisory team consisting of Professor Richard McClatchey and Dr Simon Scarle. Professor Richard McClatchey for his continual support and assistance navigating every stage of the doctoral Journey; an extremely difficult and winding path to navigate. Nevertheless, a fascinating and interesting learning experience. Dr Simon Scarle for his extensive technical guidance related to the field of video games technology. Chris Simons provided valuable review feedback during progress milestone stages of the PhD journey. I would like to thank the University of the West of England for funding both the Graduate Tutor Scheme and the European Conference on Pattern Languages of Programs 2019. The conference, its extensive year long process including shepherding and writers’ workshop, has undoubtedly afforded me a deep understanding of the discourse of patterns. I would like to attribute my love for patterns and everything pattern related to my time working as a Senior Software Engineer at Oracle Corporation UK Ltd in their Business Intelligence group. Oracle Corporation Ltd introduced me to the “Gang of Four” and I have been a pattern advocate ever since. Their professional grounding in true software engineering will undoubtedly last me a lifetime. The greatest of thanks go to my parents whose continual and undying support has enabled me to get through this journey. Without my father I would not have the tenacity to persevere and keep going. Without my mother I would not see the world through the wonderful lens of creativity. Both are essential on my journey through life. Ultimately, I would like to dedicate this piece of work to my daughters, Willow and Saffron. I wrote every word for them so that they can understand tenacity overcomes everything, no matter of the struggles they have been through in life, and that in the end perseverance and faith prevails. iv Table of Contents 1 Introduction .................................................................................................................................................. 1 1.1 Background and Context ....................................................................................................................... 1 1.2 Research Motivation .............................................................................................................................. 7 1.3 Research Problem Area ....................................................................................................................... 11 1.4 Research Aims and ObJectives ............................................................................................................ 13 1.5 Research Hypothesis and Associated Questions ................................................................................. 14 1.6 Contribution to Knowledge ................................................................................................................. 15 1.7 Methodology ........................................................................................................................................ 16 1.8 Thesis Structure ................................................................................................................................... 18 1.9 Publications .......................................................................................................................................... 19 1.10 Summary .............................................................................................................................................. 19 2 Literature Review ......................................................................................................................................