Michael Lewis and Jeffrey Jacobson GAME ENGINES IN SCIENTIFIC RESEARCH

SERIOUS COMPUTATIONAL province of the military and high-end VR labs seek- ing to drive costly peripherals such as Caves, data- RESULTS ARE DERIVED FROM gloves, and head-mounted displays (HMDs). In COMPUTER-BASED GAMES. the past two years, however, the situation has changed remarkably. Now the mass market for ix-figure workstations and custom soft- computer games, grown even larger than the movie ware are not producing the best graphics industry, has expropriated the best in computer or virtual reality simulations anymore. hardware and software for itself. The most sophisti- Today, the only way to have the fastest, cated rendering pipelines are now found not on most realistic simulations and sophisti- specialized scientific machines but on PC video catedS graphics is to trade down from the expensive cards costing less than $500. The most sophisti- gear to standard PCs running game software. Vir- cated, responsive interactive simulations are now tual reality (VR), augmented reality (AR), and found in the engines built to power games. high-fidelity physical simulation have long posed Despite the stigma of violence and gore associ- too high a barrier to entry for any but the most gen- ated with first-person games, there has been a long erously funded researchers. Significant advances in history of unpublicized cooperation between com- computer graphics in these areas have traditionally puter scientists and the game industry [1, 2]. depended on expensive, specialized graphics hard- Games have provided the first and sometimes the ware running on scientific workstations. High- only market for advanced graphics techniques, fidelity simulation/graphics software has also demonstrating the extent to which realism could remained an expensive niche market largely the be conjured up even from relatively weak graphics

COMMUNICATIONS OF THE ACM January 2002/Vol. 45, No. 1 27 environments using artistry and texture maps. Modular structure. The cost of developing ever Level editing more realistic simulations has Virtual Worlds grown so huge that even game Reprogramming Game Code developers can no longer rely on behaviors To other instances recouping their entire investment Net- Server from a single game. This has led Engine work of the game Code to the emergence of game Graphics Drivers engines—modular simulation code—written for a specific game Operating System but general enough to be used for a family of similar games. This separability of function from content is what now clever split of rendering pipelines between entities and allows game code to be repurposed for scientific environment was needed to attain sufficient speed. In research. place of peer-to-peer networking it provided a Early computer games consisted of little more than client/server architecture with the game state main- the event-loop, state tables, and graphic routines tained on the server while the compute-intensive 3D needed by simple 2D games such as Space Invaders, engine ran on the client. also extended its pro- Galaxians, or Raptor. The 1993 release of Doom by id grammability to provide the first game-independent Software ushered in a new era of game design and play. game engine providing both a level editor for changing Although not the first game to offer an immersive first- layouts and QuakeC, a byte-compiled scripting lan- person perspective, Doom was the first to do so suc- guage, for changing behavior in the simulation. cessfully. It achieved a sense of realism even on the Quake II extended the performance and fidelity of 80486-based PCs of the day by assiduous use of texture Quake, adding support for hardware-accelerated maps, 2-1/2 D/4 DF animation, and other program- graphics that were just beginning to appear on PC ming tricks. This tradition of coaxing more realism video cards. With the release of Quake III Arena and than possible by brute force continues to distinguish Epic Games Unreal Tournament in 1999, game gaming simulations from their scientific counterparts. engines reached their present mature state. Both games Two other features significant to scientific users intro- are strictly multiplayer with single-user play simulated duced by Doom were multiplayer play over a network through play against a synthetic opponent. They each and user/third-party programmability. provide extensive support for hardware acceleration, Because Doom and its descendants have been and support user modifications through level editors designed from inception for network play, their archi- and scripting languages. As with Quake II, Quake III tectures have more in common with large-scale distrib- Arena makes the language game code avail- uted military simulations such as SimNet or ModSAF able, keeping only the graphics engine code propri- than VR systems such as World Toolkit or languages etary. Unreal Tournament is set up in much the same such as VRML. By elevating the problems of updating way, except that its open source game code is written in and synchronization to a primary concern, game UnrealScript, a bytecode-compiled engines provide superior platforms for rendering mul- similar to Java. Although there are more than 600 com- tiple views and coordinating real and simulated scenes mercial game engines including the high-concept as well as supporting multiuser interaction. Despite its graphics of Everquest’s LithTech engine, Quake III initial success, Doom’s peer-to-peer architecture with Arena and Unreal Tournament provide the most devel- parallel games updating in lockstep proved cumber- oped, flexible, and usable engines for research purposes. some and has been replaced by client/server schemes in later games. The offered only the mini- What are Game Engines? mal degree of programmability by providing a generic In today’s modularly constructed games, the game’s game executable to which the user could add data in a engine refers to that collection of modules of simula- prescribed format. The new levels had changed layouts tion code that do not directly specify the game’s behav- but continued to look and play just like Doom. Later ior (game logic) or game’s environment (level data). games have significantly expanded programmability. The engine includes modules handling input, output Quake [1] was a lessons-learned follow-on from id (3D rendering, 2D drawing, sound), and generic Software. It was truly 3D with environment and actors physics/dynamics for game worlds. The figure here made from texture-mapped polygons, although a shows this overall structure.

28 January 2002/Vol. 45, No. 1 COMMUNICATIONS OF THE ACM At the top level are the virtual worlds or scenarios that object must exist for all the other players and be in with which the players interact. They come in a wide the same state. range of appearances and rules of interaction, even dif- ferent types of simulated physics. Many are authored by Scientific Use of Game Engines fans of the game, who primarily use an advanced pro- While control is the key to science, game engines are by gram development environment that comes free with design opaque. Speed-up strategies such as swapping the game itself. Often they include components of texture maps as distance increases produce the illusion of existing worlds or are built using knowledge the authors reality but can leave the experimenter without cleanly freely exchange via the Web. manipulatable parameters. Precise behavior of control The level below is the game code, which handles systems, as found in Microsoft Flight Simulator, or most of the basic mechanics of game itself, like simple response of materials to stress are not usually part of the physics, display parameters, networking, and the base- worlds simulated by game engines. If, on the other or atomic-level actions for animations such as hand, ecological validity is crucial to the research, pho- autonomous agents or behaviors in the player’s own torealistic game engine graphics and animations will avatar. Altering the game at this level requires a more work far better than well-controlled but more primitive detailed knowledge of its internals and is accomplished scenes constructed in a principled way. If research using a game-specific scripting language. requires only faithful maintenance of player, object, and The rendering engine is the terrain locations then game engine crown jewel of the game. It incor- fidelity is completely adequate. porates all of the complicated code While precise multimodal VE needed to efficiently identify and simulations such as those required render the player’s view from a Coaxing more for remote surgery will continue complex 3D model of the environ- realism than possible to demand custom solutions, for ment. The rendering engine is a by brute force most VE applications a game proprietary black box and not open continues to engine can add value by perform- to any kind of user modification. distinguish gaming ing 3D bookkeeping, providing The networking code supports simulations from networking and synchronization, a robust, built-in networking pro- their scientific or driving high-fidelity hardware- tocol similar to DIS/HLA, which counterparts. accelerated graphics. allows several users in remote loca- The research projects described tions to explore and interact in the in the short articles in this section same virtual environment (VE). illustrate the variety of ways in One computer acts as the server or host for the envi- which game engine capabilities can be repurposed for ronment, while the others support the individual users. research. Gal Kaminka’s GameBots, Bylund and Networking code is so efficient and well-developed in Espinoza’s QuakeSim, and John Laird’s artificial intelli- Unreal Tournament and Quake III that players on a gence systems all de-emphasize game graphics in favor low-latency local-area network can expect to get syn- of game engines’ capabilities as general-purpose 3D chronization that exceeds frame rate of their displays. simulations for keeping track of players, locations, and The graphics drivers translate generic requests from objects in complex 3D worlds. the rendering engine to the underlying graphics library, Research in robotic coordination requires common using APIs such as DirectX, OpenGL, and others. environments for simulating sensor data and motoric Because the drivers are open source they can easily be responses. The popular Robocup [3] soccer simulation modified to accommodate new display types such as does not provide the degree of complexity or challenge Caves or HMDs. needed to prepare teams of robots for real-life tasks. By Finally, the server is a separate process, usually on a defining socket-based interfaces to connect simulated different machine, which maintains information on robots to an Unreal Tournament server, Kaminka whichever virtual world it is supporting at the time. It opened up the entire library of Unreal Tournament communicates with the game clients used by the play- worlds to robotic play, test, and evaluation. QuakeSim ers to maintain global information about shared envi- takes advantage of this same access to simulated sensor ronments, player interactions (such as damage) and data but uses it to test software concepts involving con- synchronization information. For example, each player text-aware services. John Laird’s game-based research is has a different window into their shared VE. The scenes closest in spirit to the games themselves. Working in presented to the players must be consistent so if one both Quake and Unreal Tournament, he and his team player changes the state of some object, for example, at the University of Michigan are seeking to use the

COMMUNICATIONS OF THE ACM January 2002/Vol. 45, No. 1 29 Soar learning architecture to develop and test human- For research purposes it is hard to imagine an applica- level artificial intelligence through controlling synthetic tion for which one would be suited and the other not. characters within the games. However, because a choice must be made there are The other two research projects exploit both the some differences that should be considered. modeling and graphics capabilities of the game As the older game, the Quake III engine has devel- engines. In the augmented reality game ARQuake, oped a large collection of customized levels and modi- game levels are written to match actual physical loca- fications to play. Quake III also has a large cadre of tions and players wearing see-through HMDs users and user sites to use as resources for programming encounter and do battle with monsters superimposed questions and assistance. It is not unlikely that some- over real scenery. CaveUT takes advantage of both the one in your organization may have Quake program- graphical and networking capabilities of the Unreal ming experience. The has been engine by using multiple players’ viewpoints to con- optimized to the point where it is the fastest-running struct a panoramic Cave-like display. There are proba- game engine and has what many believe the best over- bly as many potential applications for game engines as all polygonal architecture. there are research problems requiring medium-fidelity As the newer engine, Unreal benefits from current 3D simulation or high-fidelity interactive graphics. trends in programming and technology—see the side- Our hope is to raise awareness of the high-power/low- bar “The New Cards.” Unlike Quake, which has cost alternative game engines can offer. descended from a succession of C language implemen- tations, the is strictly object-oriented. Choosing an Engine This pays big dividends through mutators and other A researcher’s choice between the Quake and Unreal programming constructs that allow a programmer to engines is similar to that between Coke and Pepsi. Both make robust changes in game behavior without requir- are more than adequate for their intended purpose yet ing detailed knowledge of the involved code. The Java- each has its avid fans and detractors. An examination of like scripting language, Unreal Script, is easy to use and contributors here indicates our researchers are evenly well documented as is the well-designed UnrealEd divided with Kaminka and Jacobson lining up with the development environment. Although the Quake Unreal engine, Bylund and Espinoza choosing Quake engine offers true curves and direct access to graphics III, and John Laird and colleagues working with both. functions such as deformation shading, the Unreal

The New Cards

✮ Michael Lewis open-canopy flight simulators, however, rely not on a single GPU but on systems built from many graphics he most significant recent development in gaming pipelines working together in parallel. The top-of-the- T and graphics-intensive computing has been the line Silicon Graphics Onyx 3800, for example, manages availability of supercomputer-level graphics on com- up to 16 simultaneous graphics pipelines for an upper modity-priced graphics processing units (GPUs). limit of 210 million polygons per second and a fill rate Leapfrogging earlier limits, Nvidia’s Quadro2 Pro GPU of 12 billion pixels per second. Achieving coherent delivers 31 million polygons per second at a fill rate of graphics on this scale requires elaborate coordination over one billion texture-mapped pixels per second, and management to rebalance loads among pipelines which makes it the world’s fastest GPU by some mea- at the crucial transition in processing between object sures. This speed comes despite a raft of new hardware parallelism and image parallelism. The SGI InfiniteRe- operations including multiple shadings per pixel, mul- ality architecture designs graphics pipelines to accept titexturing, texture and lighting, bump mapping and broadcast primitives at just this point. General-pur- other compute-intensive operations not found until pose GPUs designed for PC graphics and embedded recently in animated PC graphics. The new Nvidia chips applications do not have the luxury of this “sort-mid- are so effective they have found outside applications dle architecture” because they are designed to work as as the graphical heart of Microsoft’s new Xbox game self-contained standalone units. player and at the core of the multifunction avionics Making the transition from competing with desktop display for the F-22 fighter aircraft. workstations to taking on refrigerator-sized, rack- The really big graphics applications like Caves, mounted capital investments requires solving the Power Walls (high-resolution wall-size displays), and problem of parallel processing using commodity GPUs.

30 January 2002/Vol. 45, No. 1 COMMUNICATIONS OF THE ACM engine encapsulates a broader range of sophisticated is www.idsoftware.com, while www.averstar.com/ graphics including bump maps and other tweaks ~bowditch/QUAKE2/links.html contains links to recently added to graphics hardware. In the balance, the many secondary Quake sites. The manufacturer’s site Unreal engine is slightly slower, has more sophisticated for Unreal Tournament is www.epicgames. com; a “get- graphics, and is probably an easier environment for the ting started” FAQ is available at www.unreality. inexperienced game programmer. org/Cleaned/FAQ/FaqMain.htm. A quick search will turn up a multitude of sites with information, tools, Getting Started and programming tips for either of these engines. Now that you have chosen your engine all you need to While neither nor Epic Games is in the get started is a copy of the base game. Depending on business of supporting research, their user communities where you shop the game may cost anywhere from can provide active sources of help and information for $20–$50. You will need a license for each machine you game-using researchers. c use so, for instance, a QuakeSim simulation would take a single license, a GameBot installation with two teams References would require two licenses, while a five-screen CaveUT 1. Abrash, M. Quake’s game engine: The big picture. Dr. Dobb’s Journal (Spring, 1997). installation would require six licenses. A crucial feature 2. Bishop, L., Eberly, D., Whitted, T., Finch, M., and Shantz, M. Designing a of the game engines is that almost all code, except the PC game engine. IEEE Computer Graphics and Applications (1998), 46–53. 3. Kitano, H., Tambe, M., Stone, P., Veloso, M., Coradeschi, S., Osawa, E., proprietary graphics engine, is open source. Under its Matsubara, H., Noda, I., and Asada, M. The RoboCup synthetic agent chal- GNU license, this code may be freely distributed, lenge ’97. In Proceedings of the Fifteenth International Joint Conference on Arti- copied, and modified. Several of the systems discussed ficial Intelligence, Nagoya, Japan, (1997). here are already available online. CaveUT can be down- loaded from www2.sis.pitt.edu/~jacobson/ut/- Michael Lewis ([email protected]) is an associate professor in the CaveUT.html and GameBot code, documentation, Department of Information Science and Telecommunications at the University of Pittsburgh. discussion, and even opponents, are located at Jeffrey Jacobson ([email protected]) is a Ph.D. candidate in www.planetunreal.com/gamebots/. While not game- the Department of Information Science and Telecommunications at the based, the WireGL code for building PC- University of Pittsburgh. based Power Walls is available at graphics. stanford.edu/software/wiregl/. The primary Quake site © 2002 ACM 0002-0782/02/0100 $5.00

Because the graphics pipelines themselves are inac- mate of 29 million polygons per second for a 16- cessible and closely tied to other parts of the PC archi- pipeline SGI InfiniteReality configuration. Using PCs and tecture, the most convenient approach lies in data projectors found in most labs, WireGL can allow clustering PCs to achieve “sort-first” machine-wise researchers to experiment with previously rare and parallelization. While this architecture is limited by expensive large-scale high-resolution displays. network bandwidth and is less efficient than the The same hardware can be used to run CaveUT “sort-middle” approach of high-end graphics vendors, (described in this issue by Jacobson and Hwang) to cre- the huge cost savings and the speed advantage of ate an immersive panoramic Cave-like display. CaveUT commodity GPUs makes it hard to resist. follows the same cluster rendering approach as WireGL We are aware of two such solutions. WireGL (now but takes advantage of the Unreal engine’s networking Chromium),1 developed at Stanford University, is a and graphics synchronization capabilities to do the general-purpose system for scalable interactive ren- processing. Because game players’ views must be com- dering on a cluster of workstations. WireGL replaces the puted independently by clients, the opportunity for OpenGL libraries, which allows OpenGL calls to be paral- load balancing is lost. Nevertheless, given lightweight lelized and rendered through the cluster. Reported per- communication protocols, the power of the new com- formance of 70 million triangles per second for a 32-PC modity GPUs and gigahertz-plus GPUs, display refresh configuration (16 compute and 16 rendering nodes) rates have become the limiting factor. c compares well with the achievable performance esti-

1Humphreys, G., Eldridge, M., Buck, I., Stoll, G., Everett, M., and Hanrahan, P. Michael Lewis ([email protected]) is an associate professor in WireGL: A scalable graphics system for clusters. In Proceedings of the 2001 Conference the Department of Information Science and Telecommunications on Computer Graphics (Los Angeles, CA, 2001), 129–140. at the University of Pittsburgh.

COMMUNICATIONS OF THE ACM January 2002/Vol. 45, No. 1 31