POSTMORTEM STAR WARS: SHADOWS OF THE EMPIRE by Mark Haigh-Hutchinson

hadows of the Empire is an action

game originally developed for the

56 64 console. It

formed part of a multimedia Star

Wars event consisting of a novel,

soundtrack, toy line, comic books, SStrading cards, and other related merchan- dising. The Nintendo 64 version was

released in December of 1996, and has

proven to be very popular with over one

million copies shipped to date. The IBM

PC version was released in early September

of 1997, and has enhanced cut scenes, Red

Book audio (both music and voice), and

high-resolution graphics. It requires the Dinner in Kyoto, Japan, August 1996. (Left to right: Don James, Hiro Yamada, Mark Haigh-Hutchinson, , Kenji Miki.) use of a 3D accelerator card.

GAME DEVELOPER JANUARY 1997 http://www.gdmag.com Images courtesy of LucasArts Entertainment Company LLC, © 1997. 24-bit graphics. Eventually, we would have to change our programmers’ com- puters to INDYs (still powerful machines) to install the Nintendo 64 development systems. In addition, we were fortunate that LucasArts allowed us to obtain a Silicon Graphics ONYX supercomputer. This impressive and somewhat expensive refrigerator-sized computer boasted Reality Engine 2 graphics hardware, four the INDY. This later became known as Why Shadows? R4000 CPUs, and 256MB of RAM. It the Revision 1 board, but on inspection became an essential part of our develop- it was extremely “clean” — no wire ack in the summer of 1994, ment equipment, as it was the only wraps or other temporary items in B LucasArts was exploring the pos- hardware available that could possibly sight. Within three days, technical lead sibility of developing a new 3D title for emulate how the final Nintendo 64 Eric Johnston and second programmer one of the emerging “next-generation” hardware would perform. Indeed, Mark Blattel had ported the game to platforms. After some discussion, the Nintendo and SGI supplied us with soft- the actual hardware. It was an awe- Nintendo 64 was decided upon as the ware that emulated most of the features inspiring moment when we first saw platform of choice, even though there that the real hardware would support. the Battle of Hoth running on the 57 was no hardware available at the time. In late September, the programmers “real” machine. The first revision of Due to our close relationship with took a trip down to Silicon Graphics to the hardware was very close to the orig- Lucasfilm, we were aware that discuss the Nintendo 64 hardware inal specifications supplied by SGI. Lucasfilm Licensing was planning the design with its chief architect, Tim Van Other than the RCP (Reality Shadows of the Empire event. Jon Hook. The SGI engineers were rightly CoProcessor) not running at quite the Knoles, the lead artist and designer on proud of their design, and promised final speed, and one of the special the Nintendo 64 game, took an active that they would deliver hardware video “dither modes” not being avail- part in deciding the timeline of matching the ambitious specifications. able, it performed extremely well. Shadows. He suggested that it take Nine months later, we learned that they Over the next few weeks, we would place between The Empire Strikes Back had indeed met those specifications. receive an additional two boards, so and Return of the Jedi. By Christmas of 1994, we had the that all the programmers were devel- The Shadows story line deals mainly basis of the first level of the game, The oping in a similar fashion. Three with the criminal underworld of the Battle of Hoth, running quite nicely on months later, we would receive Galaxy, and the new period allowed us the ONYX — “quite nicely” being in Revision 2 boards, which brought the to explore some of the things that high resolution (1280×1024), 32-bit RCP up to full speed as well as fixing a weren’t explained in Return of the Jedi. color, and at 60 frames a second. By this few minor bugs. Another pleasant sur- It also opened up some new characters point, we had also received a very early prise was the doubling of the amount that were not bound to the original prototype of the Nintendo 64 con- of RAM to 4MB. story, which gave us more creative free- troller. This consisted of a modified A further development was the hard- dom than using established figures. A Super Nintendo controller with a primi- ware “dither modes” that perform sev- bonus was that it allowed us to make tive analogue joystick and Z trigger. Due eral different kinds of functions at the use of everyone’s favorite bounty to our strict nondisclosure agreement, video back end — mostly to reduce the hunter, Boba Fett. we were unable to discuss the hardware effect of Mach banding, which is com- Since we were developing one of the or the project with anyone outside the mon when using 16-bit color. premier titles for an entirely new game core team. Consequently, we would machine, there was a conscious deci- furtively hide the prototype controller sion to attempt to stretch out and cover in a cardboard box while we used it. In Technology a number of different game-play styles. answer to the inevitable questions about We wanted to ensure that the player what we were doing, we replied jokingly ince Eric Johnston and Mark would have as much variety as possible, that it was a new type of controller — a S Blattel had extensive experience yet still enjoy a satisfying experience. bowl of liquid that absorbed your with the SGI platform, we undertook to thoughts through your fingertips. Of prototype the game using the course, you had to think in Japanese…. Performer 3D API. This is an OpenGL- A Reality Engine for $200? In July of 1995, we received our first based system that is very flexible. actual hardware as a plug-in board for Eventually, we would write our own y early September 1994, we had B received our Silicon Graphics Mark Haigh-Hutchinson is a Project Leader and Senior Programmer at LucasArts workstations and the core team was Entertainment Company. He has been developing computer and video games as a working. Initially the three program- hobby since 1979, and professionally since 1984. Cutting his teeth on numerous 8-bit mers were using Indigo 2 Extremes, computers such as the Sinclair ZX Spectrum, he has contributed to 32 published with 200mhz CPUs, 64MB of RAM, and games, 16 as sole programmer. He may be reached at [email protected].

http://www.gdmag.com JANUARY 1997 GAME DEVELOPER POSTMORTEM

(effectively as RGBA values) However, the first problem that we caused the scene to appear came across was hardware incompati- as purely pastel colors. bilities between the MIDI keyboards used by our musicians and the Silicon Graphics computers used to develop Motion Capture the game. The theory was that the compositions could be previewed n the spring of 1995, we directly on the Nintendo 64 hardware Idecided to experiment as a musician played them on a key- with the use of motion cap- board. Naturally, this would provide ture to control the anima- the best possible feedback to the musi- tions of the main character cian. Unfortunately, for some Jon Knoles directs actor Amos Glick who recoils as well as enemies such as unknown reason(s), note on/off pairs from an imaginary shot. Mark Haigh-Hutchinson Stormtroopers. Fortunately were lost, causing chords to sound as wrangles the cables and provides a supporting for us, our sister company, one note. Additionally, note releases hand. Industrial Light & Magic were sometimes missed completely. (ILM), had a capture system Before long, other unwelcome behav- subset of Performer’s functionality on available for use. It was a tethered sys- iors surfaced. We worked around these the Nintendo 64. This allowed us to tem, using a magnetic field to deter- mysteries by having the musicians cap- 58 move the game from a $140,000 SGI mine the position of each of the sen- ture the sample set and play it solely ONYX to a $200 Nintendo 64 in a mat- sors. The sensors were attached to the on their keyboards. ter of just three days. actor at 11 locations using a combina- After some experimentation, though, Level designers used the tool set from tion of a climbing harness, sports joint we felt that the MIDI music was good, DARK FORCES to construct the first-per- supports, bandages, and Velcro strips. but didn’t capture the essence of the son levels for the game. This allowed a The nature of the system presented John Williams orchestral soundtrack crude form of preview using the actual several problems. First, the actor had to that is so closely associated with Star DARK FORCES engine on an IBM PC. This perform on a raised wooden platform, Wars. Furthermore, each additional worked fairly well, although later in the since the metal construction supports instrument channel would require more project we were able to have a single in the concrete floor would affect the CPU time than we wanted to allocate. SGI for dedicated use by the level capture system. Secondly, since the At this point, we tried an experiment designers. The PC solution, however, actor was on a platform as well as teth- using uncompressed digital samples of was also useful because the level design- ered, we couldn’t obtain a “clean” run the Star Wars main theme. The quality ers were already familiar with the cycle. Some of our more ambitious was extremely good, even after subse- processes involved. Unfortunately, motions also proved problematic. On quent compression with the ADPCM since the game engine wasn’t running the positive side, once the system was encoder provided by Nintendo. After a on the PC at that point, the develop- calibrated, we were able to capture over little persuasion, Nintendo generously ment cycle was somewhat slow. 100 motions in a single day, each with agreed to increase the amount of car- Additionally, the ONYX calculated two or three different “takes.” We tridge space from 8MB to 12MB. This the preculling visibility tree for each of viewed the motions in real time on a allowed us to include approximately 15 these levels. The way it works is quite SGI Indigo 2 Extreme computer run- minutes of 16-bit, 11khz, mono music elegant, thanks to Eric and Mark. The ning Alias PowerAnimator. This that sounded surprisingly good. world is subdivided into “sectors” — allowed us to quickly ensure that every Considering that most users would lis- that is, polygonal regions defined by capture was “clean” before continuing ten to the music through their televi- either geometry or some other criteria. with the next action. sions (rather than a sophisticated audio These sectors control collision detec- Unfortunately, we were to discover system), the results were close to that tion, have properties relating to game that after analysis, the motion data of an audio CD, thereby justifying the play, and perform several other related proved to be unusable. This was mainly extra cartridge space required. functions. The visibility program tra- because the angle information for the verses the world rendering the scene joints wasn’t consistent on its represen- from the center of every sector in a 360- tation of the direction around each Art Path degree arc as well as three elevations. axis. Consequently, all the animation For every polygon to be rendered in the for the characters was redone by hand, continuing problem throughout scene from a particular sector, an identi- a somewhat time-consuming task. A the development of SHADOWS was fying 32-bit value, rather than texture the inability to import and export data information, fills the appropriate pixels between the various 3D packages we in the frame buffer. It’s then a simple MIDI Music were using. Eventually, we managed to matter of reading the frame buffer to circumvent these problems with a determine which sectors are visible from ur initial approach to music for number of translation utilities as well that location. This process became O the game was similar to that as by using Alias Power Animator as known as “pastelization” because the taken on some of our PC titles — our central “hub” format. However, identifiers written into the frame buffer namely, a MIDI-based solution. there were still issues with scale, model

GAME DEVELOPER JANUARY 1997 http://www.gdmag.com hierarchies, and animation data. It was also meant an almost Herculean pro- core game players. A variety of game sometimes difficult for the artists to see gramming task in trying to write and play was important for a game that, for what their artwork really looked like debug what amounted to five different many players, would be one of their first until it had been through the hands of game engines. These consisted of low experiences in a fully 3D environment. our polygon wrangler (thanks Tom!). flight over terrain, gunnery action in Initially, it was difficult for our texture space, first/third person on foot or with artist to visualize the restrictions on jet pack (including a moving train Hardware Performance texture size required by the hardware, sequence), high-speed chases on a as well as color reduction issues. speeder bike, and full 360-degree space s mentioned before, for the first flight. Nonetheless, the result was that A nine months of SHADOWS, we had most players’ experiences with the game no real hardware with which to gauge New Hardware were always interesting, at the expense the performance of the game — other of displeasing some of the more hard- than a rather nice Silicon Graphics here were a number of other issues T that we had to deal with in devel- oping the game, not the least of which was that for the first nine months of The Core Team the project, we didn’t have any real The core team developing SHADOWS from inception to completion consisted of hardware on which to run the game. mainly six people, although twenty people contributed to the game for varying This deficiency wasn’t insurmountable lengths of time, and to varying degrees. Nonetheless, everyone played a vital role 59 by any means, but it restricted our in the production of the game. choices in certain ways, especially in Game Designer/Lead Artist - Jon Knoles 3D/Background Artist - Bill Stoneham level design. We were forced to make Project Leader/Senior Programmer - Storyboard Artist - Paul Topolos some assumptions, especially regarding Mark Haigh-Hutchinson Music Editor - Peter McConnell to performance. Fortunately, this was- Technical Lead - Eric Johnston Sound Designers - Larry the O and Clint n’t quite the bugbear that we anticipat- Programmer/Lycanthrope - Mark Blattel Bajakian ed. Still, as is well known, those on the Polygon Wrangler - Tom Harper Lead Tester - Darren Johnson. bleeding edge of technology are often Level Designer - Jim Current Production Manager - Brett Tosti sacrificed upon it. Level Designers - Matthew Tateishi and Extra thanks go to Don James, Henry Ingar Shu Sterchi, Hiro Yamada, Kensuke Tanabe, 3D Artists - Paul Zinnes, Andrew and Shigeru Miyamoto. Special thanks Other Issues Holdun, and Garry M. Gaber as always go to the staff at LucasArts, 3D Animator - Eric Ingerson and particularly to George Lucas for his here was considerable pressure to Texture Artist - Chris Hockabout gift of the Star Wars universe. T finish the game in time for the Christmas 1996 deadline. This reality meant many, many late nights, with some team members regularly working over 100 hours every week for the best part of a year. Hopefully, this sort of workload can be avoided in future pro- jects. Time pressure is, of course, a com- mon thing in the computer games industry — and we were certainly no strangers to the phenomenon. However, since we had to release our game shortly after launch of the machine, we were under more pressure than might usually have been encountered. Game testing also became an issue because there were very few machines with which to actu- ally test the game.

Game Play Variety Back Row (left to right): Steve Dauterman, Peter McConnell, Jon Knoles, Andrew Holdun, Paul Topolos, Mr. B. Fett; Middle Row (left to right): Jim e were able to include a very Current, Matthew Tateishi, Bill Stoneham, Brett Tosti, Ingar Shu, Tom Harper, W wide variety of game play styles Chris Hockabout; Front Row (left to right): Garry Gaber, Mark Blattel, Eric in SHADOWS. In retrospect, this meant Johnston, Mark Haigh-Hutchinson; Not shown: Paul Zinnes, Larry the O, Clint that we couldn’t tune each type of game Bajakian, Eric Ingerson, and Darren Johnson. play as much as we would have liked. It http://www.gdmag.com JANUARY 1997 GAME DEVELOPER POSTMORTEM

vary considerably depending upon scene complexity, as well as the simple fact that we didn’t have any real hardware from which to measure performance characteristics. Essentially, the program keeps track of the absolute time between each update of the game. This value, which we called delta time, became a multiplicand for any movement or other time-based quantity. By this method, the game runs independent of the video ONYX. Nonetheless, when we finally refresh rate, with all objects moving and we had to compress everything in the received the real hardware, we were responding at the correct frequency. game to fit it into the cartridge space pleased to find that the performance The other issue had to do with the available. This included the thin operat- estimates given to us by SGI proved to “letterbox” effect that is common to ing system that SGI provides as part of be very accurate. In fact, in large part many NTSC to PAL conversions. In most the development system. Therefore, due to the parallel nature of the graph- cases, there is no extra rendering or upon machine reset, it’s necessary to ics hardware, we were able to use float- increase in the vertical frame buffer size, decompress this OS to run the game. To ing-point mathematics throughout leaving unsightly black bands above and perform this decompression, we wrote a 60 SHADOWS with no significant impact below the visible game area. Since the small bootstrap program, which intro- upon performance. vertical resolution is now greater than duced a small amount of time between Additionally, SHADOWS was pro- the original NTSC display, the aspect the hardware being initialized and the grammed entirely using the C language ratio will also change, causing the graph- OS starting. This lag introduced a one- — it wasn’t necessary for us to use ics to appear stretched horizontally. time glitch on the screen as the video assembler (a first as far as I was con- While I wasn’t willing to accept this, I hardware started. Not very noticeable, cerned, and a pleasant surprise even had presumed that I couldn’t afford the except to me. After many late nights, I though I’m a long-time hardcore extra CPU time necessary to render a discovered a way to remove the glitch assembler fan). Since our scene com- larger frame buffer, even with the extra by directly accessing the Nintendo 64 plexity was relatively high (usually time available due to the 50hz video video hardware registers. kept to around 3,000 polygons or so, refresh rate. There was also a question of but variable according to the level type the additional RAM usage required by and design), the graphics task took our triple buffering of the frame buffer. Bad Idea longer to execute than the program My first attempt, therefore, was simply code (that is, we were graphics-bound). to change both the field of view and e then discovered that because Consequently, optimizations to the aspect ratios of the 3D engine. This sim- W we had accessed the hardware program code didn’t significantly ple fix solved the “stretching” problem directly, it caused an infrequent bug. improve overall performance. quite nicely, although the display Rarely (1 out of 50 times) the Nintendo remained letter-boxed, of course. 64 would crash if the reset button were Unfortunately, it also meant that any pressed at a particular point in the NTSC to PAL Conversion 2D-overlay status information remained game. Not only that, I couldn’t repeat “stretched.” There was the potential that the bug on my hardware (I hate it fter completing the American and game play could be affected because the when that happens). A Japanese versions of the game, it field of view, by definition, would affect After a number of very late nights was my task to convert the game so that the player’s perception of the 3D world. (over the Christmas holiday), with the it could run on the European PAL televi- Again, this just wasn’t good enough. help of Nintendo of America’s techni- sion standard. Being British, I had a vest- What I needed was a solution that did- cal staff (thanks Mark and Jim), we ed interest in making sure that the con- n’t require extra rendering, yet would finally resolved the problem: first, by version was a good one. This meant two fix the aspect ratio problems. After a lit- removing the code that directly things: first, that the game used the tle bit of research, I realized that I had accessed the video registers, and sec- whole of the vertical resolution of the discovered earlier that it was possible to ond, by restoring the registers control- PAL display (625 lines vs. 525 lines of change the size of the final visible dis- ling the scaling of the output in the NTSC); second, I wanted to ensure that play area on the output stage of the dis- vertical axis upon reset. Sometimes, the the speed of the PAL game was the same play hardware. In reality, it’s possible to simplest solution is the best. as the NTSC one, even though the PAL shrink or enlarge the display both hori- refresh rate is 50hz rather than 60hz. zontally and vertically. To compensate Fortunately, when we started work on for the letterboxing, all I had to do was Support from SGI and Nintendo SHADOWS, we realized that one of the change the vertical display size by a fac- most important things to consider was tor of 625/525 or 1.19. Once I did this, I e were very lucky to receive that it had to be a time-based game, immediately had a full-screen PAL ver- W excellent support from both SGI rather than a frame-based one. This sion. Or so I thought…. and Nintendo during the production of would allow for update rates that could One of things about SHADOWS is that the game. The SGI engineers (thanks in

GAME DEVELOPER JANUARY 1997 http://www.gdmag.com particular to “Acorn”) were very helpful a major problem (that continued n’t have to build all new tools, and would normally have an answer to throughout the duration of the pro- although a large number of data our questions within a day, sometimes ject) was the inability of various 3D conversion utilities were necessary. within the hour. I would like to thank packages to import and export data. In addition, by reusing familiar Nintendo for their assistance in the pro- Although we were able, for the most tools, our level designers could be duction of the game. Nintendo of part, to write our own conversion more productive earlier in the pro- America’s technical support and QA utilities, it still proved to be a stum- ject than otherwise might have been departments also proved invaluable. In bling block and prevented us from expected. addition, three of Nintendo of Japan’s having an efficient art path. 3) Our decision to use digitized music staff spent some time working directly Fortunately, the companies supply- proved to be a crucial one. Because with us at our offices. ing these tools now recognize the most users would listen to the music I was also fortunate enough to visit need for importing and exporting through their televisions, the quality Nintendo’s head quarters in Kyoto, data to other packages, and are tak- approximated that of an audio CD as Japan, to discuss SHADOWS with Shigeru ing steps to remedy the situation — far as many customers were con- Miyamoto, creator of MARIO 64. His VRML, for example, is proving to be cerned. This alone justified the extra insights were both fascinating and a useful format. cartridge space required and sur- extremely relevant. He is simply a 4) Time was the biggest enemy of all in prised many players who didn’t genius with an instinctive understand- producing the game. This is nothing expect that level of quality from a ing of video games. new, but was exacerbated by the fact cartridge game. that we were working on a non-exis- 4) The conversion of the game for the 61 tent machine for nine months. PAL television standard went Of Wampas and Men Nonetheless, even though this was, extremely well and was much appre- for the most part, out of our control, ciated by customers in those coun- hen developing a project on we were still able to produce a quali- tries. It would be fair to say that W the scale of SHADOWS, there will ty game. SHADOWS has set the standard in that always be some things that didn’t 5) With hindsight, probably the most it runs both full screen and full progress as smoothly as they could important lesson to be learned from speed. There is no reason why all have… the game’s development is that of games from this point on shouldn’t 1) The motion capture process proved focus. Do one or two things and do run just as well on PAL systems as to be a red herring for us. While them extremely well. Although our they do on NTSC. originally promising a much more ambitions were well placed in trying 5) Given that we were working on realistic animation solution, in our to provide the player with as much completely new hardware and for case the data proved unusable. variety as possible, we effectively the most part had to discover every- However, I still believe that it has had to write five different game thing that we needed to know by great potential and deserves further engines. Additionally, we could have ourselves, the support from both SGI investigation, even though we didn’t also used a fourth programmer dedi- and Nintendo was invaluable to us get to the point of dealing with the cated to all aspects of the front-end throughout the project. potential problems matching the of the game; that is, level selection, motions to the character’s environ- controller options, and so forth. This ment and so forth. Caveat emptor. would have taken some of the pres- Varying Shadows 2) Attempting to use a MIDI-based sure away from the main program- music solution also proved incorrect mers towards the end of the project. ven though we were not able to for this game. While it promised to E spend as much time as we would be an efficient solution in terms of have liked tuning the game, SHADOWS memory (an important considera- Out of the Shadows… does succeed in supplying the player tion for a cartridge-based game), it with a variety of game-play styles. Its simply wasn’t suitable for an orches- hanks to the talent, dedication, popularity is a testament to the creativ- tral soundtrack such as Star Wars. T and experience of the SHADOWS ity and talent of the team of which I 3) When we started work on SHADOWS, team, many things went well during was fortunate enough to be a part. ■ the development process. 1) By using the powerful SGI comput- ers (fairly uncommon in the games industry in 1994) to prototype, com- bined with our programmers’ knowl- edge of 3D technology, we were able to develop the game rapidly, yet remain flexible in terms of perfor- mance requirements. 2) Our ability to reuse tools from our earlier DARK FORCES title saved us time and resources because we did-

GAME DEVELOPER JANUARY 1997 http://www.gdmag.com