Repairing Games at Runtime Or, How We Learned to Stop Worrying And

FOCUS MULTIPARADIGM PROGRAMMING MULTIPARADIGM FOCUS PROGRAMMING

FOCUS: MULTIPARADIGM PROGRAMMING

FOCUS: ENGINEERING FUN

person shooter gameplay. IEEE Soft- Repairing Games ware magazine is not responsible for content placed on unaffiliated websites.) These situations are created by emergence, the result of multiple sim- at Runtime or, How ple interactions operating together to create a complex set of behaviors.1 Ac- cording to Katie Salen and Eric Zim- We Learned to merman, emergence is the key require- ment for games to remain continually engaging, as players will return to a game as long as it provides “variety, Stop Worrying and 2 novelty, and surprise.” Think of Tic- Tac-Toe. For a six-year-old, Tic-Tac- Toe is an exciting game of strategy and Love Emergence intrigue. Who will win when two great, young minds collide? Of course, as we grow older, we realize that playing Tic- Chris Lewis and Jim Whitehead, University of California, Santa Cruz Tac-Toe with any reasonable competi- tor will result in a draw. The game can no longer surprise, and we abandon it for more varied pursuits, such as chess // By repairing games as they execute, Mayet lets developers or Go. In software engineering, emer- focus on creating excellent gaming experiences without gence is often considered an undesir- able confluence of complex compo- worrying about edge cases and untraceable bugs. // nents, indicating an overengineered or poorly understood system, but in video games, emergence is king. Emergence presents a significant verification obstacle: If software cannot be predicted, how can it be verified? Most automated testing techniques only function when the developer knows both the possible inputs and their cor- responding outputs. However, those outputs are, by definition, unknow- CAST YOUR MIND back to some of justcausejump). However, for all the able (if we did know them for all pos- your greatest gaming moments (or good times, there are also the bad: sibilities, we wouldn’t have an emergent ask your gaming friends for theirs). maybe the time you fell through the floor system). Furthermore, even if we could Maybe they include jumping a quad (http://bit.ly/crysissink) or when your begin the onerous task of enumerating bike into an airplane (http://bit.ly/gta- AI buddy’s pathfinding sent him in the possible inputs and outputs, emergence jump). Perhaps it was the simple joy opposite direction (http://bit.ly/fallout- can only be achieved for an adult mind of being killed in an online game by a bugs). How about when you couldn’t do through state space explosion, and our traffic cone (http://bit.ly/halo3cone). any damage at all to opponents in an on- ability to achieve test coverage across Or maybe it was launching yourself line match (http://bit.ly/gtaonlinebug)? those states decays exponentially. Cur- into the air by riding an exploding gas (Note: some of the games mentioned rent automated techniques don’t work, canister, then hijacking a helicopter and referenced in this article feature and the video game industry has re- on the way back down (http://bit.ly/ violent content in the context of first- signed itself to hiring human testers,

FOCUS: MULTIPARADIGM PROGRAMMING

FOCUS: ENGINEERING FUN

sometimes numbering in the hundreds, in non-safety-critical software such has come under alien attack, with ex- to manually test games.3 as games, failures aren’t necessarily a plosions going off all around. A failure Complete verification of emergent problem if they don’t cause significant causes the player to glitch through the software is not yet possible. With this harm. For example, if a nonplayer char- floor to the level below, which is bad in mind, we realized that we had to acter (NPC) isn’t spawned in a game enough. Worse still, as the player at- turn the problem of testing games on world, this is indeed a deviation from tempts to go back to the starting point, its head: If we can’t know or enumer- the expected state and a failure has oc- he or she finds that a locker has fallen ate the extent of all possible outputs at curred. However, if the NPC has no into a doorway. From the video, it’s un- test time, what if we instead use the ac- critical interactions in the game, the clear why the failure occurred. Regard- tual output of the system at runtime? failure’s impact is negligible. In this ar- less, the player can’t return to the top With that output, we can then repair ticle, we talk about acceptable failures, of the aircraft carrier, and must reload a saved game. The astute reader might well com- Failures can occur in complex systems ment that the failure was the player falling through the floor, and the locker even if there are no faults in the issue is simply part of a failure cascade. individual software components. Although this assessment is indeed correct, it provides cold comfort for our poor player, who must deal with the situation presented, regardless of how the system if it displays emergent mis- when a failure has occurred but game- it occurred. The player’s inability to behavior, creating a system that can play isn’t significantly impacted, and move through a necessary doorway recover from failures.4,5 To do this, we unacceptable failures, where the failure presents an unacceptable failure, in created Mayet, a system for monitoring causes a strongly negative experience. which the game has essentially reached software at runtime. Mayet evaluates We adopt the term “system under an end state from which the player the running software’s state for invalid test,” which is used often in software can no longer progress. Any number states. If it detects an erroneous state, verification. SUT refers to whatever sys- of situations could lead to a particular it instructs the software to repair itself. tem we’re testing; in this article it will be failure. In this case, the physics engine a Mario-style game. Using SUT avoids might have created an explosion that The Basics of Bugs confusion when describing Mayet, as moved the locker to that position, the When we talk about “bugs,” what do often words like “system” and “pro- designer might have placed the locker we actually mean? Algirdas Avižienis gram” can get conflated together and no to prevent backtracking, a bug in the and his colleagues define bugs at dif- one is really sure what they refer to any- implementation might have spawned ferent points.5 A fault causes an error. more (authors included). the locker by accident, the locker might Faults can be caused by several things, Now armed with our knowledge of have been incorrectly flagged as im- such as human mistakes when coding bug definitions, we can investigate an movable—the list goes on. Instead or vulnerabilities that let external faults actual problem. of looking at causes, we focus on the harm the system. An error is the system problem: the player can’t progress be- state that might lead to a failure. Some Diving In: cause the critical path is blocked. Two errors are handled and corrected in the The Crysis Problem simple solutions present themselves: re- system (such as packet loss in the TCP/ Consider a single instance of the prob- move the locker from the game world, IP protocol), but some can reach the lems caused by emergence. While or teleport the player back to where we system’s external state, in which case browsing YouTube for video game fail- expect him or her to be. Detailing the a failure occurs. A failure occurs when ures (see the sidebar), one of particu- problem and possible repairs is far eas- errors cause a deviation from the sys- lar interest caught our eye. The video ier in this situation than trying to find tem’s expected behavior. is of a playthrough of Crysis, a popu- the causes. In fact, we hypothesize that Failures can occur in complex sys- lar first-person shooter game where the most problems in emergent software tems even if there are no faults in the player takes the role of protagonist, have many more possible causes than individual software components (in shooting enemies while trying to reach solutions. many domains, integration testing will the end of the level. The player is on The Crysis problem shows us two expose these problems). Additionally, the runway of an aircraft carrier that things:

54 IEEE SOFTWARE | WWW.COMPUTER.ORG/SOFTWARE • Problems can arise from compo- to predict what failures could occur is and rule engine communicate via mes- nent orchestration, not just com- roughly analogous to that required to sages, any refactoring of the SUT ponents themselves. Jonathan Blow create good unit tests that find likely won’t require altering the rules, pro- describes this as the cross-cutting problems at a function level. These vided the SUT conforms to the same concerns problem, noting that rules specify safety properties, which specification. bringing algorithms into a “harmo- express that something bad doesn’t Finally, SUT and rule engine exe- nious whole” is fraught with diffi- happen (such as getting stuck in a wall). cution are kept separate to ensure ex- culty and no existing programming We can’t specify liveness properties, ecution independence. Although rule paradigms address this problem.6 which are properties that say something engines are fast, they aren’t without • Although trying to find the cause good must eventually happen. Take the performance cost. Communication of a problem is difficult, identifying example that a player must eventually via messaging means the rule engine the problem and repairing it is com- pick up a key. For that “eventually” can be run asynchronously in a sepa- paratively easy. to be truly evaluated, we would need rate thread, CPU core, or even a com- to be able to verify an infinite trace. pletely different machine, allowing the We designed Mayet with these insights Model checkers can do this, but our SUT to run at full speed. Asynchronous in mind. dynamic evaluation can’t. If we bound execution trades off, with failures be- the property, such as a player must pick ing possibly visible for a few frames The Mayet Architecture up a key within 30 minutes, we actu- before they’re repaired. These failures Mayet can be characterized as a run- ally have a safety property.9 In practice, are acceptable because they only ap- time monitor. Runtime monitors watch most properties we want to specify are pear briefly. If so desired, synchronous the SUT as it executes via some means bounded. For example, knowing that a execution could be enabled, meaning of instrumentation and can take some player, given infinite time, will eventu- the system could fix easily recognizable action based upon the current execu- ally pick up a key doesn’t ensure an en- failures before they’re rendered to the tion. Systems such as Rainbow7 and joyable user experience. screen. ClearView8 have begun to repair ex- The execution of the SUT and the ecutions that are exhibiting some un rule engine are deliberately kept sepa- A Demonstration: Lakitu desirable behavior. Mayet takes such rate. There are several good reasons for Lakitu is a version of Super Mario an approach. this. World built on top of the Infinite Ma- The SUT is manually instrumented The first reason is language in- rio Bros open source 2D platform en- to produce events across a message dependence. Most games are writ- gine. We altered the engine by adding queue. Events are small messages indi- ten in unmanaged languages like C faults, instrumenting the game, attach- cating something important has hap- or C++. Rule engine implementa- ing Mayet, and enacting repairs. pened and are parameterized with use- tions, such as Drools (www.jboss. ful values. For example, a 2D platform org/drools) and Windows Work- Adding Faults game (such as Super Mario World) flow (http://bit.ly/windowsworkflow), We added faults by looking at our would have events for jumping, landing, are typically written in managed code, video game failure taxonomy (see the and changing the score (parameterized with the old and the new score). The SUT produces event messages and sends them to a message broker, which routes Events are small messages indicating them to a consuming rule engine running as a separate process. The rule en- that something important has happened. gine evaluates the messages and checks whether they violate human-authored integrity rules. If an integrity rule is vi- olated, the rule engine sends a message such as Java or C#. Separation lets us sidebar) and inserting various faults back to the SUT to perform a repair. use whichever language we choose for covering different categories. The bugs An experienced programmer writes the game, without needing to write a we inserted include accelerating Mario the rules beforehand to identify possi- new rule engine each time. too much when he jumps (invalid posi- ble failure scenarios and how they can A second reason is implementation), having coins equal two points in- be repaired. The experience required tion independence. Because the SUT stead of one (invalid value change), and

SEPTEMBER/OCTOBER 2011 | IEEE SOFTWARE 55 FOCUS MULTIPARADIGM PROGRAMMING MULTIPARADIGM FOCUS PROGRAMMING

FOCUS: MULTIPARADIGM PROGRAMMING

FOCUS: ENGINEERING FUN

GAME FAILURE TAXONOMY

To know how well a system-recovery tool performs, we need a disallowed or an action that must be taken isn’t possible. metric of the domain’s failure types. To help us evaluate Mayet— and future systems that attempt to aid game stability—we created Temporal categories include the following: a taxonomy of game failures.1 We divide the categories into • Invalid position over time categorizes invalid movements (for nontemporal (that is, they can be evaluated by a single inspection example, an object moving too quickly or too slowly, hovering of the state) or temporal (that is, multiple inspections are required when this isn’t possible). to ascertain their presence). We created these categories • Invalid context state over time observes objects as finite state independently of Mayet’s development. machines; an object is in incorrect state for a wrong period of Nontemporal categories include the following: time (such as being stunned for too long). • Invalid event occurrence over time is when an event happens • Invalid position: an object is in a place it shouldn’t be. too frequently or not frequently enough (events can be long • Invalid graphical representation: an object doesn’t appear sequences, not atomic events as with Mayet). correctly, or doesn’t appear at all.* • Interrupted event occurs when an event that previously hap- • Invalid value change: an internal counter (such as score and pened has stopped before reaching completion (events can health points) has been modified in a way that isn’t allowed be long sequences, not atomic events as with Mayet). (for example, score going up too quickly or health points not • Implementation response is when the game doesn’t provide going down). feedback quickly enough (for example, problems cause • Artificial stupidity: the AI has done something that doesn’t network or controller lag).* meet expectations, such as running over a land mine or standing in a doorway. Categories marked with an “*” are those that Mayet can’t yet • Bad information: players get information about what they cover. shouldn’t do, such as seeing through walls, don’t receive necessary information, or receive information out of order Reference (such as being asked to kill a character the player has already 1. C. Lewis, J. Whitehead, and N. Wardrip-Fruin, “What Went Wrong: A defeated). Taxonomy of Video Game Bugs,” Proc. 5th Int’l Conf. Foundations of Digital • Bad actions: a character takes an action when that action is Games (FDG 2010), ACM Press, 2010, pp. 108–115.

spawning Bullet Bills too frequently (in- intent is to expose details about im- sages and send repairs back, and valid event occurrence over time). Our portant events that occur during the • Mayet sends events as messages insertions took the form of if statements game’s execution. These are roughly across the producer queue. that checked whether a flag had been analogous to the same details and set for buggy operation, which would events that programmers would place A message broker manages the routing then modify various data structures in in a debugging-level log file for offline of messages to different subscribers Lakitu to have invalid values. analysis. Programmers are already depending on various rules. We don’t well-trained in finding these areas of need a message broker in this case Instrumenting the Game execution and identifying which values because we only run one instance of Instrumenting Lakitu manually sounds should be emitted. In Lakitu, instead of Mayet, so we could just route messages like it would be an alien and labori- writing a call to a logging framework, directly to a socket. However, if we ous task, wading through source code the call is a message to be sent. To do required more complexity, such as trying to find various points of impor- this, the game does four things: redundant Mayet instances, or queuing tance, packaging a message with vari- messages if no Mayet instance is found, ous values, and then sending it on the • starts an ActiveMQ instance to act the message broker would become message queue for the rule engine to as a message broker (optional), useful. In the actual Lakitu source, evaluate. In fact, the process is quite • creates a producer queue on the Ac- the producer and consumer queues similar to that all-too-familiar pro- tiveMQ instance that Lakitu sends are shared between Lakitu and Mayet gramming task of logging. messages out on, for simplicity. In production, different When we instrument a game, our • Mayet starts to consume the mes- queues should be used.

56 IEEE SOFTWARE | WWW.COMPUTER.ORG/SOFTWARE To create the messages, we serialize Lakitu event objects, which are immutable Version Rules objects of various values. A jump event, for example, records Mario’s x and y positions, as well as his acceleration. Some events don’t require values, such as when Mario dies (although the player might want to modify this to know exactly how he died). When developing Lakitu, we found that our instrumentation method was largely iterative. After adding the bugs, we began to think about the information the rule engine would need to detect them. In the real world, of course, game programmers would use their experience to identify possible failure scenarios and add the events required to detect them. Once we knew what was required, we could create a new class (or modify an old one) to store those values and then fi nd a suitable place for the event to be emitted. FIGURE 1. Lakitu in buggy operation, with the rule engine disabled. Notice how high Mario Lakitu uses a standard run loop as has jumped: a jump of such height should not be possible. Re-enabling the rule engine will seen in many simple games: input is prevent Mario from jumping so high again. read, the logical state is modifi ed, and the output is rendered to the screen. Mayet once per second, so Mayet could an event, it logs an error message and When a game object modifi es its state, also perform a boundary check and kill sends a repair message back. Not Lakitu generally emits an event mes- Mario if he falls off screen. shown in the rule is a little bookkeeping sage. These messages weren’t diffi cult that expires jump events so we don’t or invasive to insert. More complex Attaching Mayet continue evaluating the same jump over games might have more diffi cult state Let’s go through an example of a repair and over. representations that are across sepa- that Mayet can make if Mario jumps Let’s consider a more diffi cult rate threads and components. As long too high, as Figure 1 illustrates. It’s problem. Imagine Mario is hovering in as these components can send mes- easy to defi ne a categorization of what the air. He jumps up to a normal height sages to the queue, this shouldn’t pose should really happen: “Mario should and just stays there, fl uttering off the a concern. not be able to jump more than fi ve units ground. Any single event doesn’t give us Creating event messages and fi nding high.” The game must meet this specifi - enough information. A jump event only the correct places to insert them is a cation. To send this information for the says how fast and how high Mario will fairly simple and fast process. However, rule engine to monitor, we need to cre- go, but that’s completely normal. Can some thought might be required about ate an event when Mario jumps, with this hovering be detected? The key is to how best to represent something. For the jump height value attached. A value realize that the state must be inspected instance, we needed a way of knowing that is too high will break this specifi - over time, so we add a landing event, whether Mario had fallen into a pit. cation. When the rule engine receives too. What goes up, must come down. The pit itself isn’t an object, so no the jump event, it compares the in-game Our rule needs to know about both the function is called when Mario falls height with what it should be. Figure 2a jump and the landing. Drools specifi es down. Our buggy implementation shows how that rule could look. temporal rules as shown in Figure 2b. disabled an if statement that kills Inside the when clause is a search for Drools evaluates this rule by fi rst Mario when he falls off the screen. We a jump event where the height is greater looking for a jump event. When it fi nds decided to send out Mario’s location to than fi ve. If the rule engine fi nds such one, it waits to see if the next property

SEPTEMBER/OCTOBER 2011 | IEEE SOFTWARE 57 FOCUS MULTIPARADIGM PROGRAMMING MULTIPARADIGM FOCUS PROGRAMMING

FOCUS: MULTIPARADIGM PROGRAMMING

FOCUS: ENGINEERING FUN

Lakitu lets users turn bugs on or off rule “marioJumpTooHigh” and turn the rule engine on or off. when $jump : Jump(height > 5) Limitations then The Mayet architecture generally fo- logger.info(“Mario jumping too high”); cuses on repairing logical state. How- broker.send(newMessage(new MarioMovement(...))); ever, if there’s a problem with how that end state is represented—say, a bug in the (a) graphics pipeline that results in screen corruption—Mayet can’t repair (or rule “marioHover” perhaps even detect) the problem. This duration(2s) corresponds to the invalid graphical when representation category in our failure $jump : Jump() taxonomy. Further investigation in this not(Landing(this after[0s,2s] $jump)) area might well yield a modifi cation to then the Mayet architecture that would be logger.info(“Mario was hovering”); more suitable for this class of issue. broker.send(newMessage(new MarioMovement(...))); A common concern raised by those end who see the Mayet architecture is the (b) possibility that the system could get stuck in a repair cascade: one repair FIGURE 2. Examples of rules generated by the rule engine to create an event. (a) A rule results in the violation of a different specifying a maximum height for an event, and (b) a rule specifying a length of time between rule, which in turn issues a repair that two events. violates a different rule, and so on. The worst-case scenario would be a repair is matched—that a landing two sec- message and its parameters, then enacts loop, where repairs are constantly is- onds after the jump isn’t found. If Ma- the fi x. This is, by far, the most inva- sued. Related to this is the concern that rio hasn’t landed, Drools sends a repair sive change we made to Lakitu. To con- repairs result in something even more message asking that the game acceler- sume repairs, we modifi ed the loop so negative, such as a hovering character ate him down, just as before. Express- that after the logical state is modifi ed, being accelerated into a spike pit. ing rules temporally is very powerful, the game listens for repair messages and Certain design strategies, such as and lets us specify many useful and in- enacts them. Because Lakitu runs asyn- making the minimum viable change teresting properties about a game, be it chronously, these repair messages aren’t in a repair and keeping rules as con- a short-lived one like this or a property guaranteed to correspond with the logi- strained as possible to avoid broad spanning the entire game. cal state change in that loop’s execution. matches, will help. However, the only We believe this syntax is far eas- We implemented functionality for way to truly assert whether this is a ier to read than many programming repairs that could move Mario, add problem, and how it can be solved, is to languages. Thus, the rule fi le can be new level components, or remove en- try and scale up to a full game, which handed to the game designer for vali- emies. One useful aspect of having the we haven’t yet attempted. dation, ensuring that faults aren’t ac- SUT enact the repair, aside from the cidentally introduced, edge cases aren’t independence issues mentioned previ- missed, and the designer’s intentions ously, is that the SUT can choose when lthough emergent software are met correctly. a repair should occur. Some repairs is generally unverifi able, it could be delayed until they’re outside A doesn’t mean that we can’t Enacting Repairs the player’s cone of view or ignored if take steps to ensure positive user ex- The rule engine communicates via mes- they’re no longer relevant. periences. Games, like many user-ori- sages. It never actually modifi es the With repairs in place, we created a ented software products, have an ac- SUT’s state. What happens to those system that can recover from failures: ceptability bound. Using a system such messages? Lakitu listens for messages it can monitor for problems, react to as Mayet provides developers the means as part of its run loop. If a new repair them, and dynamically repair them, all to ensure that their games stay within message is waiting, it reads the type of within a few frames. By comparison, that bound, but bugs can still occur.

58 IEEE SOFTWARE | WWW.COMPUTER.ORG/SOFTWARE As software across all domains be- gins to show greater complexity and thus greater emergence, the ideas and CHRIS LEWIS is pursuing his PhD as a member of the Software techniques we’ve explored here will Introspection Lab, part of the computer science department at the Uni- become increasingly important. We versity of California, Santa Cruz. His research interests include testing methodologies, data mining, and software modeling. Lewis has an MS believe games herald the beginning of in engineering from the University of Bristol. Contact him at cfl ewis@ developers letting go of the notion of soe.ucsc.edu. being able to test everything upfront. We’re moving toward runtime recovery being an essential component of mod- ern software development. Although JIM WHITEHEAD is a professor in the computer science depart- not being able to verify software can be ment at the University of California, Santa Cruz. His research interests worrying, having the tools to deal with include software evolution, software bug prediction, procedural content generation, and level design of computer games. Whitehead has a PhD that reality is rather comforting. in information and computer science from the University of California, Lakitu is free and open sourced Irvine. He is a member of the ACM and the International Game Develop- ABOUT THE AUTHORS THE ABOUT under a BSD-style license, and can be ers Association (IGDA). Contact him at [email protected]. downloaded from its website at www. zenetproject.com/pages/lakitu.

References 1. H. Haken, “The Challenge of Complex Systems,” Information and Self-Organization, Springer, 2006, chap. 1, pp. 1–35. 2. K. Salen and E. Zimmerman, Rules of Play, MIT Press, 2004, pp. 350–353. 3. K. Starr, “Testing Video Games Can’t Possibly Be Harder Than an Afternoon With Xbox, Right?” Seattle Weekly, 11 July 2007; www. PURPOSE: The IEEE Computer Society is the world’s largest association of computing professionals and is the seattleweekly.com/2007-07-11/news/testing leading provider of technical information in the field. Visit our website at www.computer.org. -video-games-can-t-possibly-be-harder-than OMBUDSMAN: Email [email protected]. -an-afternoon-with-xbox-right.php. Next Board Meeting: 13–14 Nov., New Violet S. Doan; Director, Products & Services: Evan 4. J.C. Mogul, “Emergent (Mis)Behavior vs. Brunswick, NJ, USA Butterfield;Director, Sales & Marketing: Dick Price Complex Software Systems,” SIGOPS Operat- ing Systems Rev., vol. 40, no. 4, Apr. 2006, EXECUTIVE COMMITTEE COMPUTER SOCIETY OFFICES pp. 293–304. President: Sorel Reisman* Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C. 20036-4928 5. A. Avižienis et al., “Basic Concepts and President-Elect: John W. Walz;* Past President: James D. Isaak;* VP, Standards Activities: Roger U. Fujii;† Phone: +1 202 371 0101 • Fax: +1 202 728 9614 Taxonomy of Dependable and Secure Comput- Secretary: Jon Rokne (2nd VP);* VP, Educational Email: [email protected] ing,” IEEE Trans. Dependable and Secure Activities: Elizabeth L. Burd;* VP, Member & Los Alamitos: 10662 Los Vaqueros Circle, Los Alamitos, Computing, vol. 1, no. 1, 2004, pp. 11–33. Geographic Activities: Rangachar Kasturi;† VP, CA 90720-1314 • Phone: +1 714 821 8380 • Email: Publications: David Alan Grier (1st VP);* VP, Professional [email protected] 6. J. Blow, “Game Development: Harder Than Membership & Publication Orders You Think,” ACM Queue, vol. 1, no. 10, Activities: Paul K. Joannou;* VP, Technical & Conference Activities: Paul R. Croll;† Treasurer: James W. Moore, Phone: +1 800 272 6657 • Fax: +1 714 821 4641 • 2004, pp. 28–37. CSDP;* 2011–2012 IEEE Division VIII Director: Susan Email: [email protected] 7. D. Garlan, B. Schmerl, and S.-W. Cheng, K. (Kathy) Land, CSDP;† 2010–2011 IEEE Division V Asia/Pacific: Watanabe Building, 1-4-2 Minami- Director: Michael R. Williams;† 2011 IEEE Division Aoyama, Minato-ku, Tokyo 107-0062, Japan • Phone: “Software Architecture-Based Self-Adapta- +81 3 3408 3118 • Fax: +81 3 3408 3553 • Email: tion,” Autonomic Computing and Network- Director V Director-Elect: James W. Moore, CSDP* *voting member, †nonvoting member of the Board of Governors [email protected] ing, Springer, 2009, chap. 2, pp. 31–55. 8. J.H. Perkins et al., “Automatically Patching BOARD OF GOVERNORS IEEE OFFICERS Term Expiring 2011: Elisa Bertino, Jose Castillo- President: Moshe Kam; President-Elect: Gordon W. Errors in Deployed Software,” Proc. ACM Day; Past President: Pedro A. Ray; Secretary: Roger SIGOPS 22nd Symp. Operating Systems Velázquez, George V. Cybenko, Ann DeMarle, David S. Ebert, Hironori Kasahara, Steven L. Tanimoto D. Pollard; Treasurer: Harold L. Flescher; President, Principles (SOSP 2009), ACM Press, 2009, Term Expiring 2012: Elizabeth L. Burd, Thomas M. Standards Association Board of Governors: Steven pp. 87–102. Conte, Frank E. Ferrante, Jean-Luc Gaudiot, Paul K. M. Mills; VP, Educational Activities: Tariq S. Durrani; VP, Membership & Geographic Activities: Howard 9. “Liveness Manifestos,” Apr. 2004; www. Joannou, Luis Kun, James W. Moore E. Michel; VP, Publication Services & Products: cs.nyu.edu/acsys/beyond-safety/liveness.htm. Term Expiring 2013: Pierre Bourque, Dennis J. Frailey, Atsuhiro Goto, André Ivanov, Dejan S. Milojicic, Jane David A. Hodges; VP, Technical Activities: Donna Chu Prey, Charlene (Chuck) Walrad L. Hudson; IEEE Division V Director: Michael R. Williams; IEEE Division VIII Director: Susan K. (Kathy) EXECUTIVE STAFF Land, CSDP; President, IEEE-USA: Ronald G. Jensen Executive Director: Angela R. Burgess; Associate Executive Director, Director, Governance: Anne Marie Kelly; Director, Finance & Accounting: John Selected CS articles and columns Miller; Director, Information Technology & Services: are also available for free at Ray Kahn; Director, Membership Development: revised 18 July 2011 http://ComputingNow.computer.org.

SEPTEMBER/OCTOBER 2011 | IEEE SOFTWARE 59