Using Game Theory for Los Angeles Airport Security

Articles

James Pita, Manish Jain, Fernando Ordóñez, Christopher Portway, Milind Tambe, Craig Western, Praveen Paruchuri, and Sarit Kraus

n Security at major locations of economic or political importance is a key concern around P the world, particularly given the threat of ter- rotecting national infrastructure such as airports, historical rorism. Limited security resources prevent full landmarks, or a location of political or economic importance is security coverage at all times, which allows a challenging task for police and security agencies around the adversaries to observe and exploit patterns in world, a challenge that is exacerbated by the threat of terrorism. selective patrolling or monitoring; for exam- Such protection of important locations includes tasks such as ple, they can plan an attack avoiding existing monitoring all entrances or inbound roads and checking pa trols. Hence, randomized patrolling or monitoring is important, but randomization inbound traffic. However, limited resources imply that it is typ- must provide distinct weights to dif ferent ically impossible to provide full security coverage at all times. actions based on their complex costs and Furthermore, adversaries can observe security arrangements benefits. To this end, this article describes a over time and exploit any predictable patterns to their advan- promising transition of the latest in multia- tage. Randomizing schedules for pa trolling, checking, or moni- gent algorithms into a deployed application. toring is thus an important tool in the police arsenal to avoid In particular, it describes a software assistant the vulnerability that comes with predictability. Even beyond agent called ARMOR (assistant for random- protecting infrastructure, ran domized patrolling is important in ized monitoring over routes) that casts this tasks ranging from secu rity on university campuses to normal patrolling and monitoring problem as a Bayesian Stackelberg game, allowing the police beats to border or maritime security (Billante 2003, agent to appropriately weigh the different Paruchuri et al. 2007, Ruan et al. 2005). actions in randomization, as well as uncer- This article focuses on a deployed software assistant agent tainty over adversary types. ARMOR com- that can aid police or other security agencies in randomizing bines two key features. It uses the fastest their security schedules. We face at least three key chal lenges in known solver for Bayesian Stackelberg games building such a software assistant. First, the assistant must pro- called DOBSS, where the dominant mixed vide quality guarantees in randomization by appropriately strategies enable randomization; and its weighing the costs and benefits of the different options avail- mixed-initiative-based interface allows users able. For example, if an attack on one part of an infrastructure occasionally to adjust or override the auto- will cause economic damage while an attack on another could mated schedule based on their local constraints. ARMOR has been successfully potentially cost human lives, we must weigh the two options deployed since August 2007 at the Los Ange- differently—giving higher weight (probability) to guarding the les International Airport (LAX) to randomize latter. Second, the assistant must address the uncertainty in checkpoints on the roadways entering the air- information that security forces have about the adversary. port and canine patrol routes within the air- Third, the assistant must enable a mixed-initiative interaction port terminals. This article examines the with potential users rather than dictate a schedule; the assistant information, design choices, challenges, and may be unaware of users’ real-world constraints, and hence evaluation that went into designing ARMOR. users must be able to shape the schedule development. We have addressed these challenges in a software assis tant

agent called ARMOR (assistant for randomized six-month trial period of ARMOR deployment at mon itoring over routes). Based on game-theoretic LAX. The feedback from police at the end of this principles, ARMOR combines three key features to six-month pe riod was extremely positive; ARMOR address each of the challenges outlined above. will continue to be deployed at LAX and expand to Game theory is a well-established foundational other police activities at LAX. principle within multiagent systems to reason about multiple agents, each pursuing its own inter- ests (Fudenberg and Tirole 1991). We build on Security Domain Description these game-theoretic foundations to reason about We will now describe the specific challenges in the two agents—the police force and its adversary—in security problems faced by the LAWA police. LAX1 providing a method of randomization. In particu- is the fifth busiest airport in the United States and lar, the main contribution of our article is mapping the largest destination airport in the United States, the problem of security scheduling as a Bayesian serving 60–70 million passen gers per year (Stevens Stackelberg game (Conitzer and Sandholm 2006) et al. 2006). LAX is unfortunately also suspected to and solving it through the fastest optimal algo- be a prime terrorist target on the West Coast of the rithm for such games (Paruchuri et al. 2008), United States, with multiple arrests of plotters address ing the first two challenges. The algorithm attempting to attack LAX (Stevens et al. 2006). To used builds on several years of research regarding protect LAX, LAWA police have designed a securi- multiagent systems and security (Paruchuri et al. ty system that utilizes multiple rings of protection. 2005, 2006, 2007). In particular, ARMOR relies on As is evident to anyone traveling through an air- an optimal algorithm called DOBSS (decomposed port, these rings include such things as vehicular optimal Bayesian Stackelberg solver) (Paruchuri et checkpoints, police units patrolling the roads to al. 2008). the terminals and inside the terminals (with dogs), While a Bayesian game allows us to address and security screening and bag checks for passen- uncertainty over adversary types, by optimally gers. There are unfortunately not enough resources solving such Bayesian Stackelberg games (which (police offi cers) to monitor every single event at yield optimal randomized strategies as solutions), the airport; given its size and the number of pas- ARMOR provides quality guarantees on the sched- sengers served, such a level of screening would ules generated. These quality guarantees obviously require considerably more personnel and cause do not imply that ARMOR provides perfect securi- greater delays to travelers. Thus, assuming that all ty; instead, ARMOR guarantees optimality in the checkpoints and terminals are not being moni- utilization of fixed security resources (number of tored at all times, setting up available checkpoints, police or canine units) assuming the rewards are canine units, or other patrols on deterministic accurately modeled. In other words, given a schedules allows adversaries to learn the schedules specific number of security resources and areas to and plot an attack that avoids the police check- protect, ARMOR creates a schedule that random- points and patrols, which makes deterministic izes over the possible deployment of those schedules in effective. resources in a fashion that optimizes the expected Randomization offers a solution here. In partic- reward obtained in protecting LAX. ular, from among all the security measures to The third challenge is addressed by ARMOR’s use which randomization could be applied, LAWA of a mixed-initiative-based interface, where users police have so far posed two crucial problems to are allowed to graphically enter different con- us. First, given that there are many roads leading straints to shape the schedule generated. ARMOR is into LAX, they want to know where and when thus a collaborative assistant that it erates over gen- they should set up checkpoints to check cars driv- erated schedules rather than a rigid one-shot ing into LAX. For example, figure 1 shows a vehic- scheduler. ARMOR also alerts users in case over- ular checkpoint set up on a road inbound towards rides may potentially deteriorate schedule quality. LAX. Police officers examine cars that drive by, ARMOR thus represents a very promising transi- and if any car appears suspicious, they do a more tion of multiagent research into a deployed appli- detailed inspec tion of that car. LAWA police cation. ARMOR has been successfully deployed wished to obtain a randomized schedule for such since August 2007 at the Los Angeles Internation- checkpoints for a particular time frame. For exam- al Airport (LAX) to assist the Los An geles World ple, if we are to set up two checkpoints, and the Airport (LAWA) police in randomized schedul ing timeframe of interest is 8 AM to 11 AM, then a can- of checkpoints and since November 2007 for gen- didate schedule may suggest to the police that on erating randomized patrolling schedules for canine Monday, check points should be placed on route 1 units. In particu lar, it assists police in determining and route 2, whereas on Tuesday during the same where to randomly set up checkpoints and where time slot, they should be on route 1 and 3, and so to randomly allocate canines to terminals. Indeed, on. Second, LAWA police wished to obtain an February 2008 marked the successful end of the assignment of canines to patrol routes through the

44 AI MAGAZINE Articles terminals inside LAX. For example, if there are three canine units available, a possible assignment may be to place canines on terminals 1, 3, and 6 on the first day, but on terminals 2, 4, and 6 on another day, and so on based on the available information. Figure 2 illustrates a canine unit on patrol at LAX. Given these problems, our analysis revealed three key challenges: first, potential attackers can observe secu rity forces’ schedules over time and then choose their attack strategy—this fact makes deterministic schedules highly susceptible to attack; second, there is unknown and uncertain in - formation regarding the types of adversary we may face; and third, although randomization helps eliminate deterministic patterns, it must also account for the different costs and benefits associ- ated with particular targets. In summarizing the domain requirements, we Figure 1. LAX Checkpoint. emphasize the following key points. First, it is LAWA police, as domain experts, who expressed a requirement for randomization, leading us to design ARMOR. Second, there exist different rings of security (including canines and checkpoints that ARMOR schedules), which are not static and therefore may change independently of the other rings different times. The end result of such shift- ing randomized security rings is that adversary costs and uncertainty increase, particularly for well-planned attacks, which in turn may help deter and prevent attacks.

Approach We modeled the decisions of setting checkpoints or canine patrol routes at the LAX airport as Bayesian Stackelberg games. These games allow us to accomplish three impor tant tasks: they model the fact that an adversary acts with knowledge of security forces’ schedules, and thus randomize schedules appropriately; they allow us to define Figure 2. LAX Canine Patrol. mul tiple adversary types, meeting the challenge of our uncer tain information about our adversaries; and they enable us to weigh the significance of dif- leader. For example, given our security domain, the ferent targets differently. Since Bayesian Stackel- police force (leader) must first commit to a mixed berg games address the challenges posed by our strategy for placing checkpoints on roads in order domain, they are at the heart of generating mean- to be unpredictable to the adversaries (followers), ingfully randomized schedules. From this point we where a mixed strategy implies a probability distri- will explain what a Bayesian Stackelberg game con- bution over the actions of setting checkpoints. The sists of, how an LAX security problem can be adversaries, after observing check - points over mapped onto Bayesian Stackelberg games, some of time, can then choose their own strategy of attack- the previous methods for solving Bayesian Stackel- ing a specific road. To see the advantage of being berg games, and how we use DOBSS to optimally the leader in a Stackelberg game, consider a simple solve the problem at hand. game with the payoff table as shown in figure 3. The leader is the row player and the follower is the Bayesian Stackelberg Games column player. Given a si multaneous move game, In a Stackelberg game, a leader commits to a strat- that is, the leader and follower now act at the same egy first, and then a follower selfishly optimizes its time, the only pure-strategy Nash equilibrium for reward, considering the action chosen by the this game is when the leader plays a and the fol-

SPRING 2009 45 Articles

lower. We also assume knowing a priori probability pl, where l represents the type of adversary (1, 2, and so on), of the different follower types (that is l c d ∈ L). Our goal is to find the optimal mixed strate- a 2, 1 4,0 gy for the leader to commit to, given that the follower may know the leader’s mixed strategy when b 1,0 3,2 choosing its strategy and that the leader will not know the follower’s type in advance. Techniques for Solving Stackelberg Games Figure 3. Payoff Table for In previous work it has been shown that finding an Example Normal Form Game. optimal solution to a Bayesian Stackelberg game with multiple fol lower types is NP-hard (Conitzer and Sandholm 2006). Researchers in the past have identified an approach, which we will refer to as Follower Type 1 the multiple-LPs method, to solve Stackelberg games (Conitzer and Sandholm 2006), and this can c d be used to solve Bayesian Stackelberg games. This a 2, 1 4,0 approach, however, requires transforming a Bayesian game into a nor mal form game using the b 1,0 3,2 Harsanyi transformation (Harsanyi and Selten 1972). Similarly one may apply efficient algorithms for finding Nash equilibria (Sandholm, Follower Type 2 Gilpin, and Conitzer 2005), but they require the c' d' same Harsanyi transformation. Since our research crucially differs in its nonuse of the Harsanyi trans- a 1, 1 2,0 formation, it is important to understand this trans- b 1,0 3,2 formation and its impact. Harsanyi Transformation. The first step in solving Bayesian games for previous methods is to apply the Harsanyi transformation (Harsanyi and Selten 1972) that converts the incomplete infor- Figure 4. Security Agent Versus Followers 1 and 2. mation game into a normal form game. Given that the Harsanyi transformation is a standard concept lower plays c, which gives the leader a payoff of 2. in game theory, we explain it briefly through a However, if the leader commits to a uniform mixed simple example without introducing the mathe- strategy of playing a and b with equal (0.5) proba- matical for mulations. Consider the case of the two bility, then the follower will play d in order to max- follower types 1 and 2 as shown in figure 4. Fol- imize its payoff, leading to a payoff for the leader lower type 1 will be active with probability α, and of 3.5. Thus, by committing to a mixed strategy follower type 2 will be active with probability 1 – first, the leader is able to obtain a higher payoff α. Performing the Harsanyi transformation in - than could be obtained in a simultaneous move volves introducing a chance node that determines situation. the follower’s type, thus transforming the leader’s The Bayesian form of such a game, then, implies incomplete information regarding the follower that each agent must be of a given set of types. For into an imperfect information game. The trans- our security domain, we have two agents, the formed, normal form game is shown in figure 5. In police force and the adversary. While there is only the transformed game, the leader still has two one police force type, there are many different strategies while there is a single follower type with adversary types, such as serious terrorists, drug four (2 * 2) strategies. For example, consider the sit- smugglers, and petty criminals, denoted by L. Dur- uation in the transformed game where the leader ing the game, the adversary knows its type, but the takes action a and the follower takes action ccЈ. The police do not know the adversary’s type; this is an leader’s payoff in the new game is calculated as a incomplete in formation game. For each agent (the weighted sum of its payoffs from the two tables in police force and the adversary) i, there is a set of figure 4, that is, α times payoff of leader when fol- σ ϫ σ σ α strategies i and a utility function ui : L 1 × 2 lower type 1 takes action c plus 1 – times payoff → ᑬ. Figure 4 shows a Bayesian Stackelberg game of leader when follower type 2 takes action cЈ. All with two follower types. Notice that follower type the other entries in the new table, both for the 2 changes the payoff of both the leader and the fol- leader and the follower, are derived in a similar

46 AI MAGAZINE Articles

cc' cd' dc' dd' a 2α + (1 – α), 1 2, α 4α + (1 – α), (1 – α) 4α + 2(1 – α), 0 b α, (1 – α) α + 3(1 – α), 2(1 – α) 3α, 2α + (1 – α) 3, 2

Figure 5. Harsanyi Transformed Payoff Table. fashion. In general, for n follower types with k strategies per follower type, the transformation results in a game with kn strategies for the follow- l l l er, thus causing an exponential blowup losing max xqa,, ∑∑∑ pRxqij ij compactness. iX∈ lL∈ jQ∈ Methods such as those described in Conitzer s.t.∑ iX∈ x i= 1 and Sandholm (2006) and Sandholm, Gilpin, and l Conitzer (2005) must use this Harsanyi transfor- ∑ jQ∈ qij = 1 mation, which implies the game loses its compact aCxqMl l l structure. Nonetheless, the solutions their meth- 01≤−()∑ iX∈ ij ij≤−() ods obtain can be trans formed back into the origi- … xi ∈[]01 nal game. l qj ∈ {}0,11 DOBSS a ∈ℜ One key advantage of the DOBSS approach is that it operates directly on the Bayesian representation, without re quiring the Harsanyi transformation. In particular, DOBSS obtains a decomposition scheme by exploiting the property that follower types are independent of each other. The key to the DOBSS Problem 1. decomposition is the observation that evaluat ing the leader strategy against a Harsanyi-transformed game matrix is equivalent to evaluating against probabilities pl, with l ∈ L, of facing each follower each of the game matrices for the individual fol- type, the leader solves problem 1. lower types. Here for a set of leader’s actions x and actions for We first present DOBSS in its most intuitive form each follower ql, the objective represents the as a mixed-integer quadratic program (MIQP); we expected reward for the agent considering the a then illustrate how it may be transformed into a priori distribution over different follower types pl. linearized equivalent mixed-integer linear program Constraints with free indices mean they are repeat- (MILP). While a more de tailed discussion of the ed for all values of the index. For example, the MILP is available in Paruchuri et al. (2008), the cur- fourth constraint means x ∈ [0 ... 1] for all i ∈ X. rent section may at least serve to explain at a high i The first and the fourth constraints define the set level the key idea of the decomposition used in this of feasible solutions x as a probability distribution MILP. X The model we propose explicitly represents the over the set of actions . The second and fifth con- actions by the leader and the optimal actions for straints limit the vector of actions of follower type l the follower types in the problem solved by the l, q to be a pure distribution over the set Q (that is l agent. We denote by x the leader’s policy (mixed each q has exactly one coordinate equal to one strategy), which consists of a vector of prob ability and the rest equal to zero). Note that we need to distributions over the leader’s pure strategies. consider only the reward-maximizing pure strate- Hence, the value xi is the proportion of times in gies of the follower types, since for a given fixed which pure strategy i is used in the policy. We mixed strategy x of the leader, each follower type denote by ql the vector of strategies of follower type faces a problem with fixed linear rewards. If a l ∈ L. We also denote by X and Q the index sets of mixed strategy is optimal for the follower, then so leader and follower l’s pure strategies, respectively. are all the pure strategies in support of that mixed We also index the payoff matrices of the leader and strategy. each of the follower types l by the matrices Rl and The two inequalities in the third constraint l 1 C . Let M be a large positive number. Given a priori ensure that q j = 1 only for a strategy j that is opti-

SPRING 2009 47 Articles

mal for follower type l. Indeed this is a linearized adversary type observes the LAWA police check- form of the optimality conditions for the linear pro- point policy and then decides where to attack. gramming problem solved by each follower type. Since adversaries can observe the LAWA police pol- We explain these constraints as follows: note that icy before deciding their actions, this can be mod- the leftmost inequality ensures that for all j ∈ Q, eled through a Stackelberg game with the police as the leader. aCxl ≥ l ∑ iX∈ ij i . In this setting, the set X of possible actions for This means that given the leader’s vector x, al is an LAWA po lice is the set of possible checkpoint com- upper bound on follower type l’s reward for any binations. If, for instance, LAWA police were set- action. The rightmost inequality is inactive for ting up one checkpoint then X = {1, ..., k}. If LAWA 1 police were setting up a combination of two check- every action where q j = 0, since M is a large positive 1 points, then X = {(1, 2), (1, 3)...(k − 1, k)}, that is, all quantity. For the action that has q j = 1 this inequality states that the adversary’s payoff for this action combinations of two checkpoints. Each adversary ∈ must be ≥ al, which combined with the previous type l L = {1, ..., m} can decide to attack one of the inequality shows that this action must be optimal k roads or maybe not attack at all (none), so its set for follower type l. Notice that problem one is a of ac tions is Q = {1, …, k, none}. If LAWA police decomposed MIQP in the sense that it does not uti- select road i to place a checkpoint on and adver- ∈ lize a full-blown Harsanyi transformation; instead sary type l L selects road j to attack then the l it solves multiple smaller problems using individ- police receive a reward Rij and the adversary l ual adversaries’ payoffs (indexed by l). Further- receives a reward Cij. These reward values vary more, this decomposition does not cause any sub- based on three considerations: the chance that the optimality (Paruchuri et al. 2008). LAWA police checkpoint will catch the adversary We can linearize the quadratic programming on a particular inbound road; the damage the l adversary will cause if it attacks by means of a par- problem 1 through the change of variables z ij = l ticular inbound road; and the type of adversary, xiq j. The substitution of this one variable allows us to create an MILP. The details of this transforma- that is, adversary capability. If LAWA police catch l tion and its equivalence to problem 1 are present- the adversary when i = j, we make Rij a large posi- l ed in Paruchuri et al. (2008). DOBSS refers to this tive value and Cij a large negative value. However, equivalent mixed-integer linear program, which the probability of catching the adversary at a can be solved with efficient integer programming checkpoint is based on the volume of traffic packages. Although DOBSS still remains as an through the checkpoint (significant traffic will exponential solution to solving Bayesian Stackel- increase the difficulty of catching the adversary), berg games, by avoiding the Harsanyi transforma- which is an input to the system. If the LAWA police tion it obtains significant speedups over the previ- are unable to catch the ad versary, then the adver- l ous approaches as shown in the experimental sary may succeed; that is, we make Rij a large nega- l results and proofs in Paruchuri et al. (2008). tive value and Cij a large positive value. Cer tainly, if the adversary attacks from an inbound road Bayesian Stackelberg Game for the Los where no checkpoint was set up, there is no chance Angeles International Airport that the po lice will catch the adversary. The mag- l l We now illustrate how the security problems set nitude of Rij and Cij vary based on the adversary’s forth by LAWA police can be cast in terms of a potential target, given the road from which the Bayesian Stackelberg game. We focus on the check- adversary attacks. Some roads lead to higher val- point problem for illustra tion, but the case of the ued targets for the adversary than others. The game canine problem is similar. Given the checkpoint is not a zero sum game, however, as even if the problem, our game consists of two players, the adversary is caught, the adversary may benefit due LAWA police (the leader) and the adversary (the to publicity. follower), in a situation consisting of a specific The reason we consider a Bayesian Stackelberg number of inbound roads on which to set up game is because LAWA police face multiple adver- checkpoints, say roads 1 through k. LAWA police’s sary types. Thus, differing values of the reward set of pure strategies consists of a particular subset matrices across the different adversary types l ∈ L of those roads to place checkpoints on prior to represent the different objectives and valuations of adversaries selecting which roads to attack. LAWA the different attackers (for example, smugglers, police can choose a mixed strategy so that the criminals, terrorists). For example, a hard-core, adversary will be unsure of exactly where the well-financed adversary could inflict significant checkpoints may be set up, but the adversary will damage on LAX; thus, the negative rewards to the know the mixed strategy LAWA police have cho- LAWA police are much higher in magnitude than sen. We assume that there are m different types of an amateur attacker who may not have suffi cient adversaries, each with different attack capabilities, resources to carry out a large-scale attack. If these planning constraints, and financial ability. Each are the only two types of adversaries faced, then a

48 AI MAGAZINE Articles

External Information

Armor System

User Input Bayesian Stackelberg Game

Interface

DOBSS

S1=30% S2=10% S3=60%

Finalized Schedule Suggested Schedule Mixed Strategy

Front End Back End

Figure 6. ARMOR System Flow Diagram.

20-80 split of probability implies that while there System Architecture is a 20 percent chance that the LAWA police face the former type of adversary, there is an 80 percent There are two separate versions of ARMOR, chance that they face an amateur attacker. Our ex - ARMOR-checkpoint and ARMOR-canine. While in perimental data provides initial results about the the following we focus on ARMOR-checkpoint for sensitivity of our algorithms to the probability dis- illustration, both these versions use the same tributions over these two different adversary types. underlying architecture with different inputs. As While the number of adver sary types has varied shown in ﬁgure 6, this architecture consists of a based on inputs from LAWA police, for any one front end and a back end, integrating four key adversary type the largest game that has been con- components: a front-end interface for user interac- structed, which was done for canine deployment, tion; a method for creating Bayesian Stackelberg consisted of 784 actions for the LAWA police game matrices; an im plementation of DOBSS; and (when multiple canine units were active) for the a method for producing suggested schedules for eight possible terminals within the airport and 8 the user. They also contain two major forms of actions per adversary type (one for a possible attack external input. First, they allow for direct user on each terminal). input into the system through the interface. Sec-

SPRING 2009 49 Articles

Figure 7. ARMOR Interface.

ond, they allow for file input of relevant informa- inbound roads and the number of checkpoints al- tion for checkpoints or canines, such as traffic/pas- lowed during that time slot determines the avail- senger volume by time of day, which can greatly able actions for the LAWA police, whereas the affect the security measures taken and the values action space of the adver sary is determined by the of certain actions. At this point we will discuss in number of inbound roads. Thus, the system can set detail what each component consists of and how up the foundation for the Bayesian Stackelberg they interact with each other. game by providing all the actions possible in the game. Once the action space has been generated, it Interface can be sent to the back end to be set up as a The ARMOR interface, seen in figure 7, consists of Bayesian Stackelberg game, solved, and returned as a file menu, options for local constraints, options a suggested schedule, which is displayed to the to alter the ac tion space, a monthly calendar and a user by means of a spreadsheet. main spreadsheet to view any day(s) from the cal- There are three options that serve to restrict cer- endar. Together these compo nents create a work- tain actions in the generated schedule: forced ing interface that meets all the key re quirements checkpoint; forbidden checkpoint; at least one set forth by LAWA officers for checkpoint and checkpoint. These constraints are intended to be canine deployment at LAX. used sparingly to accommo date situations where a The base of the interface is designed around six user, faced with exceptional circumstances and possi ble adjustable options; three of them alter the extra knowledge, wishes to modify the output of action space and three impose local constraints. the game. The user may impose these restrictions The three options to alter the action space are the by forcing specific actions in the schedule. In par- number of check points allowed during a particular ticular, the forced checkpoint option schedules a time slot; the time interval of each time slot; and checkpoint at a specific time on a specific day. The the number of days to schedule over. For each giv- forbidden checkpoint option designates a specific en time slot, the system constructs a new game. time on a specific day when a checkpoint should As discussed previously, the total num ber of not be scheduled. Finally, the at least one check-

50 AI MAGAZINE Articles point option designates a set of time slots and between the combinations with the given proba- ensures that a checkpoint is scheduled in at least bilities. Each time a selection is made, that combi- one of the slots. na tion is sent to the user interface to be reviewed The spreadsheet in the interface serves as the by the user as necessary. So if, for instance, combi- main mechanism for viewing, altering, and con- nation one was chosen, the user would see check- straining schedules. The columns correspond to point A and B as scheduled for the given time slot. the possible checkpoints, and the rows correspond In rare cases, as mentioned previously, a user to the time frames in which to schedule them. Up may have forbidden a checkpoint or required a to a full week can be viewed at a single time as seen checkpoint. ARMOR accommodates such user in figure 7. Once a particular day is in view, the directives when creating its schedule; for example, user can assign to that day any desired constraints. if checkpoint C is forbidden, then all the probabil - Each constraint is represented by a specific color ity in our example shifts to the combination A and within the spreadsheet, namely green, red, and yel- B. Un fortunately, by constraining the schedule fre- low for the forced, for bidden, and at least con- quently, a user can completely alter the mixed straints, respectively. strategy produced as the output of DOBSS, defeat- ing DOBSS’s guarantee of optimality. To avoid such Matrix Generation and DOBSS a possibility, ARMOR incorporates certain alerts Given the submitted user information, the system (warnings) to encourage noninterference in its must create a meaningful Bayesian Stackelberg sched ule generation. For example, if a combina- game matrix. Pre viously we illustrated the generation has zero or very low probability of being cho- tion of the action space in this game. Based on the sen and the user has forced that checkpoint com- prespecified rewards as discussed earlier, we can bination to occur, ARMOR will alert the user. provide the rewards for the LAWA police and the Similarly, if a combination has a very high likeli- adversaries to generate a game matrix for each hood and the user has forbidden that event, adversary type. After the final game matrices are ARMOR will again alert the user. However, ARMOR constructed for each adversary type, they are sent only alerts the user; it does not autonomously to the DOBSS implementation, which calculates remove the user’s constraints. Resolving more sub- the optimal mixed strategy over the current action tle interactions between the user’s imposed con- space. straints and DOBSS’s output strategy remains an To demonstrate the process, assume there are issue for future work. three pos sible checkpoint locations (A, B, C), one When a schedule is presented to the user with possible time slot to schedule over, and two check- alerts, the user may alter the schedule by altering points available for scheduling. Given this sce- the forbidden or required checkpoints, or possibly nario, the unique combinations possible include by directly altering the schedule. Both possibilities scheduling checkpoints A and B, A and C, and B are accommodated in ARMOR. If the user simply and C, over the given time frame. We will assume adds or removes constraints, ARMOR can create a that check points A and B are highly valuable while new schedule. Once the schedule is finalized, it can C, although not completely devoid of value, has a be saved for actual use, thus completing the system very low value. Based on this information, a likely cycle. This full process was designed specifically to mixed strategy generated by DOBSS would be to meet the re quirements at LAX for checkpoint and assign a high probability to choosing action A and canine allocation. B, say 70 percent, and a low probability to both the other actions, say 15 percent each. Whatever the Design Challenges mixed strategy actually comes out to be, it is the optimal strategy a user could take to maximize Designing and deploying the ARMOR software on security based on the given information. This a trial ba sis at LAX posed numerous challenges and mixed strategy is then stored and used for the actu- problems to our research group. Some key lessons al schedule generation. learned during the design and deployment of ARMOR include the importance of tools for ran- Mixed Strategy and Schedule Generation domization, the importance of manual schedule Once an optimal mixed strategy has been chosen overrides, and the importance of providing police by DOBSS and stored within the system, a particu- officers with operational flexibility. lar combination of actions must be chosen to be displayed to the user. Consider our example from Importance of Tools for Randomization the previous section involving three pos sibilities There is a crit ical need for randomization in secu- (checkpoints A and B, A and C, B and C) and their rity operations. Security officials are aware that probabilities of 70 percent, 15 percent, and 15 per- requiring humans to generate randomized sched- cent. Knowing this probability distribution, the ules is unsatisfactory because, as psychological system can formulate a method to randomly select studies have often shown (Wagenaar 1972),

SPRING 2009 51 Articles

humans have difficulty randomizing. Instead, Our run-time analysis adds to the above results, mathemati cal randomization that appropriately focusing specifically on the current security weighs the costs and benefits of different actions domain for which this work has been applied. For and randomizes accordingly leads to improved this reason we compare the run-time results of results. Security officials were hence extremely DOBSS versus multiple LPs, described previously, enthusiastic in their reception of our research and given the specific domain used for canine deploy- eager to apply it to their domain. In addition, these ment at LAX. MIP-Nash (Sandholm, Gilpin, and officials have indicated that obtaining schedules Conitzer 2005) has not been included in this automat ically reduces their burden of having to analysis of run times as it only provides the best construct such schedules manually taking all the Bayes-Nash equilibrium as opposed to the optimal relevant factors into ac count. mixed strategies provided by the multiple-LPs method and the DOBSS method. The aim of this Importance of analysis is to show that DOBSS is indeed the most Manual Schedule Overrides suitable procedure for application to real domains While ARMOR incorporates all the knowledge that such as the LAX canine and checkpoint allocation. we could obtain from LAWA police and provides the To that end, we used the data from a full week of best output possible, it may not be aware of dynam- canine deployment to analyze the time necessary ic developments on the ground. For example, police to generate a schedule given the DOBSS method officers may have very spe cific intelligence for and the multiple-LPs method. For completeness we requiring a checkpoint on a particular inbound road. show the results given one to four adversary types Hence, it was crucial to allow LAWA police officers where four adversary types is the minimum (in rare instances when it is necessary) to manually amount LAWA has set forth as necessary. selectively override the schedule provided. In figure 8 we summarize the run-time results for our Bayesian games using DOBSS and multiple LPs. Importance of Providing Police Officers We tested our results on the Bayesian games pro- with Operational Flexibility vided from the canine domain with number of When initially generating schedules for canine adversary types varying between one to four. Each patrols, the system created a very detailed sched- game between LAWA and one adversary type is ule, micromanaging the patrols. This did not get as modeled as a normal form game. Thus, there are positive a reception from the officers. Instead, an four normal form games designed for the game abstract sched ule that afforded the officers some between LAWA and the various adversary types for flexibility to respond to dynamic situations on the the base case. The size of each of these normal ground was better received. form games is (784, 8), corresponding to 784 strategies for LAWA and 8 for the adversary. We then used the seven generated instances, taken Experimental Results from an arbitrary week of canine deployment, of Our experimental results explore the run-time effi- this base case to obtain averaged results. ciency of DOBSS and evaluate the solution quality The x-axis in figure 8 shows the number of fol- and implementa tion of the ARMOR system. lower types the leader faces starting, and the y-axis of the graph shows the run time in seconds. All the Run-Time Analysis experiments that were not concluded in 20 min- It has been shown in Paruchuri et al. (2008) that utes (1,200 seconds) were cut off. From the graph DOBSS significantly outperforms its competitors, we summarize that DOBSS outperforms the multi- which include MIP-Nash (Sandholm, Gilpin, and ple-LPs method by a significant margin given our Conitzer 2005) and multiple LPs (Conitzer and real canine domain. In the graph, while multiple Sandholm 2006) in an experimental domain that LPs could solve the problem only for up to two involves a security agent patrolling a world consist- adversary types, DOBSS could solve for all four ing of m houses, 1 … m and a robber trying to rob adversary types within 80 seconds. these houses. These results show that even for a Hence we see that the DOBSS method is faster world as small as three houses, MIP-Nash and mul- than the multiple-LPs method. Consequently, we tiple LPs are unable to con verge on a solution with- conclude that DOBSS is the algorithm of choice for in the allowed time of 30 minutes when there are 8 Bayesian Stackelberg games (Paruchuri et al. 2008), or more adversary types. DOBSS, however, is able to especially given the partic ular games created by achieve a solution in less than 10 seconds for up to real security domains such as the ca nine patrolling 14 adversary types. Also, as the number of houses problem presented in this article. increases up to five, Paruchuri and colleagues find that MIP-Nash and multiple LPs are unable to con- Evaluation of ARMOR verge on a solution within 30 minutes for even low We now evaluate the solution quality obtained numbers of adversary types (Paruchuri et al. 2008). when DOBSS is applied to the LAX security

52 AI MAGAZINE Articles

DOBSS Multiple LPs

1400 1200 1000 800 600 400 Runtime (in secs) 200 0 1 2 3 4 Number of Adversary Types

Figure 8. Run Times: DOBSS and Multiple-LPs Methods. domain. We offer three types of evaluation. While two checkpoints are placed. Here also the reward our first evaluation is in the lab, ARMOR is a in the case of the DOBSS strategy is greater than deployed assistant, and hence our remaining two the reward of the uniform random strategy. When evaluations are of its deployment in the field. With we have two checkpoints, the type 2 adversary respect to our first evaluation, we conducted four chooses the action none (to not attack). This leads experiments. The first three compared ARMOR’s to the observation that the reward of the DOBSS randomization with a uni form randomization strategy and the reward of the uniform strategy are technique that does not use ARMOR’s weights in the same when only the type 2 adversary is pres- randomization. ent. Figure 9c presents the case of three check- The results of the first experiment are shown in points. Here the reward values obtained by DOBSS figures 9a, 9b, and 9c. The x-axis represents the are always positive—this is because the chances of probabilities of occurrence of the two adversary catching the adversary of type 1 improve signifi- types we chose to fo cus on. Since the actual num- cantly with three checkpoints. This also leads to ber of adversary types used for LAX is secure infor- the reward of DOBSS decreasing with the decrease mation, we use two adversary types for simplicity in the probability of occurrence of the ad versary of in this analysis. The x-axis shows the probability p type 1. Note that the type 2 adversary, as with the of adversary type 2 (the probability of adversary case of two checkpoints, decides none and hence type 1 is then obtained on 1 – p). The y-axis repre- the reward of the DOBSS strategy and the uni- sents the reward obtained by LAWA. This reward formly random strategy are the same when only represents the expected reward LAWA would type 2 adversary is present. obtain given the optimal adversary re sponse to the The three experiments reported allow us to con- strategy adopted by LAWA. Figure 9a shows the clude that DOBSS weighted randomization pro- comparison when one checkpoint is placed. For vides signifi cant improvements over uniform ran- exam ple, when adversary of type 1 occurs with a domization in the same domain, thus illustrating probability of 0.1 and type 2 occurs with a proba- the utility of our algorithms. We continue these bility of 0.9, the reward obtained by the DOBSS results in the following fourth experiment, focus- strategy is −1.72 whereas the re ward obtained by a ing now on canine units. Figure 9d shows the com- uniform random strategy is −2.112. It is important parison of the reward obtained between schedul- to note that the reward of the DOBSS strategy is ing canine units with DOBSS and scheduling them strictly greater than the reward of the uniform ran- with a uniform random strategy (denoted URS). In dom strat egy for all probabilities of occurrence of the uniform random strategy, canines are random- the adversary types. ly assigned to terminals with equal probability. The Figure 9b also has the probability distribution on x-axis represents the weekday and the y-axis repre- the x-axis and the reward obtained on the y-axis. It sents the reward obtained. We can see that DOBSS shows the difference in the obtained reward when performs better even with three canine units as

SPRING 2009 53 Articles

0 1.5

-0.5 1 DOBSS Strategy Uniform Random Strategy

-1 0.5

-1.5 0

Reward -2 Reward -0.5

-2.5 -1

-3 DOBBS Strategy Uniform Random Strategy -1.5

-3.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 -2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Probability of Adversaries (type2) Probability of Adversaries (type2)

1.6 6 DOBSS Strategy Uniform Random Strategy DOBSS (5 Canines) DOBSS (6 Canines) 1.4 5 DOBSS (3 Canines) URS (6 Canines)

1.2 4

1 3

0.8 2 Reward Reward 0.6 1

0.4 0

0.2 -1

0 -2 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Mon Tue Wed Thu Fri Sat Sun Probability of Adversaries (type2) Days

c d

Figure 9. DOBSS Strategy Versus Uniformly Random Strategy. a. One checkpoint. b. Two checkpoints. c. Three checkpoints. d. Canines.

compared to six canine units being scheduled The numbers 1 to 5 in the table denote the check- using the uniform random strategy. For example, point number (we have assigned arbitrary identi- on Friday, the reward of a uniformly random strat- fication numbers to all checkpoints for the purpose egy with six canine units is −1.47, whereas the of this experiment) and the values of the table reward of three, five, and six canine units with show the percentage of times this checkpoint was DOBSS is 1.37, 3.50, and 4.50 respectively. These used. For example, in week 1, checkpoint 2 was results show that DOBSS weighted randomization used just less than 5 percent of times, while check- with even three canine units provides better results point 2 was used about 25 percent of the times in against uniform randomization in the same week 2. We can make two observations from these domain with six canine units. Thus our algorithm two weeks: First, we do not appear to have uniform provides better rewards and can help in reducing use of these checkpoints; that is, there is great vari- the cost of resources needed. ance in the percentage of times check points are In the next evaluation, we examine ARMOR’s deployed. Second, the checkpoint deployment setting of checkpoints at LAX. The first experiment varies from week to week; for example, checkpoint examines the change in checkpoint deployment 4 was not used in week 1, but it was used 15 per- during a fixed shift (that is, keeping the time fixed) cent of the times in week 2. over two weeks. The results are shown in table 1. The goal of the next experiment was to provide

54 AI MAGAZINE Articles

Checkpoint Number 1 2 3 4 5 Week 1 33.33 4.76 33.33 0 28.57 Week 2 19.04 23.80 23.80 14.28 19.05

Table 1. Variation in Usage Percentage. results on the sensitivity analysis, specifically, how Related Work and Summary the probabilities of different actions will change if we change the proportion of adversary types. Fig- The patrolling problem itself has received signifi- ure 10 shows the variation in strategy for placing cant attention in multiagent literature due to its two checkpoints together when the probability of wide variety of ap plications ranging from robot occurrence of the adversary changes. The x-axis patrol to border patrolling of large areas (Ruan et shows the variation in the probability of occur- al. 2005, Billante 2003). The key idea behind the rence of the adversary types, whereas the y-axis policies provided by these techniques is random- shows the variation in the probabilities in the ization, which decreases the amount of informa- DOBSS strategy. For example, when adversary of tion given to an adversary. However, no specific type 1 occurs with a probability of 1, the probabil- algorithm/procedure has been provided for the ity of placing both checkpoints 1 and 4 is 0.353; generation of randomized policies; hence, they can when adversaries 1 and 2 occur with probabilities lead to highly suboptimal policies. Two exceptions 0.4 and 0.6, respectively, then the probability of are Praveen Paruchuri and colleagues and their ear- placing checkpoints 3 and 4 is 0.127. We can ly work (Paruchuri et al. 2006), which provides observe that there is no variation in the probabili- algorithms for an alyzing randomization-reward ties in the DOBSS strategies when the probabilities trade-offs, and Noa Agmon and colleagues and of occurrence of the two adversary types vary from their recent work (Agmon, Kraus, and Kamink .1 to .9. This indicates that our results are not par- 2008), which provides algorithms for reducing the ticularly sensitive to variations in probabilities of probability of penetration. However, unlike our opponents except at the extremes. work, neither model any adversaries or adversary Our final evaluation is a more informal evalua- types. tion based on feedback from the LAWA police. First, Finally, the sequence from Koller and Pfeffer they have pro vided very positive feedback about (1997) provides an alternative compact representa- the deployment. They suggest that the technique tion to normal form representation. However, rep- they had previously used was not one of random- resenting commitment to a mixed strategy, as ization, but one of alternating checkpoints; such a required in our Stackelberg games, is dif ficult in routine can bring about determinism in the sched- this representation, making its use difficult. Fur- ul ing, which we have avoided. Second, ARMOR has thermore, Koller and Pfeffer (1997) have not reduced routine work in scheduling, which allows focused on com puting optimal response in Stack- LAWA police to focus on more important tasks. elberg games, but rather in only finding equilibria. Third, several arrests have been made at check- While ARMOR is a game-theoretic security points scheduled by ARMOR. Typically these scheduler, there are many other competing non- involved cars attempting to carry weapons into game-theoretic tools in use for related applications. LAX. Finally, Director James Butts of LAX police has For example, the “Hypercube Queuing Model” commented that the new random placement (Larson 1974) based on queuing theory depicts the “makes travelers safer” and even gives them “a detailed spatial operation of urban police depart- greater feeling of police presence” by making the ments and emergency medical services and has police appear more numerous (Murr 2007). Also found application in police beat design, allocation Chief Erroll Southers in testimony to a congres- of patrolling time, and so on. However, this mod- sional committee has commented, “LAX is safer el does not take specific adversary models into today than it was 18 months ago,” citing ARMOR account; ARMOR, on the other hand, tailors poli- as one of the key factors.2 This does not necessarily cies to combat various potential adversaries. suggest that ARMOR’s schedule was responsible Two different approaches have been presented because this is not a controlled experiment per se. previously to find solutions to Bayesian Stackel- Nonetheless, it illustrates that the first line of berg games efficiently. One of the approaches, defense at the outer airport perimeter is helping named ASAP (Paruchuri et al. 2007), is able to alleviate the threat of violence at the airport. operate on the Bayesian form of Stackelberg

SPRING 2009 55 Articles

0.45

0.4

0.35

0.3 C1,C3 C1,C4 0.25

0.2

0.15 C2,C3 C3,C4

0.1

Probability of Action 0.05

0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Probabilities of Adversaries (type2)

Figure 10. Sensitivity Analysis.

games, but it provides an approximate solution. 2008) for the past two years into the real world. The second approach, the multiple-LPs method, requires a Bayesian game to be transformed into a Acknowledgements normal form game using the Harsanyi transfor- ARMOR’s deployment at LAX has only been possible mation (Harsanyi and Selten 1972). DOBSS is due to the exceptional effort by LAWA police to superior to ASAP in that it provides exact solu- strike a collabo ration. This research was supported tions, and as shown, it also outperforms the mul- by the United States Department of Homeland Secu- tiple-LPs method for our domain of interest. rity through the Center for Risk and Economic In summary, establishing security around infra- Analysis of Terrorism Events (CREATE) un der grant structure of economic or political importance is a number 2007-ST-061-000001. However, any opin- challenge that is faced today by police forces ions, findings, and conclusions or recommendations around the world. While randomized monitoring in this document are those of the authors and do not is important—as adversaries can observe and necessarily reflect views of the United States Depart- exploit any predictability—randomization must ment of Homeland Security. We would also like to use different weighing functions to reflect the thank the National Science Foundation for their complex costs and benefits of different police contributions under grant number IS0705587. actions. This article describes a deployed agent assistant called ARMOR that casts the monitoring Notes problem as a Bayesian Stackelberg game, where 1. www.lawa.org/lax/justTheFact.cfm randomized schedule generation can appropriate- 2. homeland.house.gov/SiteDocuments/20080506102233 ly weigh the costs and benefits as well as uncer- -79198.pdf tainty over adver sary types. ARMOR combines two key features: first, it uses the fastest known References solver for Bayesian Stackelberg games called DOB- Agmon, N.; Kraus, S.; and Kamink, G. A. 2008. Multi- SS, where the dominant mixed strategies provide Robot Perimeter Patrol in Adversarial Settings. In Pro ceed- schedule randomization; and second, its mixed- ings of the 2008 IEEE International Conference on Robotics and Automation. Piscataway, NJ: Institute of Electrical and initiative-based interface allows users to occa- Electronics Engineers. sionally adjust or override the automated sched- Billante, N. 2003. The Beat Goes On: Policing for Crime ule based on their local constraints. ARMOR has Prevention. Leonards, NSW, Australia: Center for Inde- been successfully deployed at the Los Angeles In- pendent Studies (www.cis.org.au/issueanalysis/IA38/IA38 ternational Airport, randomizing allocation of .HTM). checkpoints since August 2007 and canine Conitzer, V., and Sandholm, T. 2006. Computing the Op- deployment since November 2007. ARMOR thus timal Strategy to Commit To. In Proceedings of the Seventh represents a successful transition of multiagent ACM Conference on Electronic Commerce (ACM-EC). New algorithmic advances (Paruchuri et al. 2006, 2007, York: Association for Computing Machinery.

56 AI MAGAZINE Articles

Fudenberg, D., and Tirole, J. 1991. Game Theory. Cam- bridge, MA: The MIT Press. Harsanyi, J. C., and Selten, R. 1972. A Generalized Nash Solution for Two-Person Bargaining Games with Incom- plete Information. Management Science 18(5): 80–106. Theoretical Aspects of Rationality Koller, D., and Pfeffer, A. 1997. Representations and Solu- and Knowledge (TARK XII) tions for Game-Theoretic Problems. Artificial Intelligence 94(1–2): 167–215. Stanford University Larson, R. C. 1974. A Hypercube Queuing Model for Fa - July6-8,2009 cility Location and Redistricting in Urban Emergency Ser- vices. Computer and Operations Research 1(1): 67–95. TARK brings together researchers from a wide va- Murr, A. 2007. The Element of Surprise. Newsweek Nation- riety of fields, including Artificial Intelligence, Dis- al News (September 28) (www.newsweek.com/id/41845). tributed Computing, Cryptography, Economics and Paruchuri, P.; Pearce, J. P.; Marecki, J.; Tambe, M.; Ordóñez, F.; and Kraus, S. 2008. Playing Games for Secu- Game Theory, Linguistics, Philosophy, and Psychol- rity: An Efficient Exact Algorithm for Solving Bayesian ogy, in order to further our understanding of interdis- Stackelberg Games. In Proceedings of the 2008 Internation- ciplinary issues involving reasoning about rationality al Conference on Autonomous Agents and Multiagent Sys- tems. Richland, SC: International Foundation for and knowledge. For more information see: Autonomous Agents and Multiagent Systems. ai.stanford.edu/∼epacuit/tark09 Paruchuri, P.; Pearce, J. P.; Tambe, M.; Ordóñez, F.; and Kraus, S. 2007. An Efficient Heuristic Approach for Secu- rity against Multiple Adversaries. In Proceedings of the Key Dates 2007 International Conference on Autonomous Agents and Paper submission deadline: March 2, 2009 Multiagent Systems. Richland, SC: International Founda- Notification of authors: April 27, 2009 tion for Autonomous Agents and Multiagent Systems. Camera ready version due: June 10, 2009 Paruchuri, P.; Tambe, M.; Ordóñez, F.; and Kraus, S. 2005. Safety in Multiagent Systems by Policy Randomization. Paper presented at the Second International Workshop Organizers: on Safety and Security, Utrecht, the Netherlands, 26 July. Joseph Halpern (Conference Chair) Paruchuri, P.; Tambe, M.; Ordóñez, F.; and Kraus, S. 2006. Aviad Heifetz (PC Chair) Security in Multiagent Systems by Policy Randomization. In Proceedings of the 2006 International Conference on Eric Pacuit (Local Organizer) Autonomous Agents and Multiagent Systems. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems. Ruan, S.; Meirina, C.; Yu, F.; Pattipati, K. R.; and Popp, R. L. 2005. Patrolling in a Stochastic Environment. In Pro- ceedings of the Tenth International Command and Control Research and Technology Symposium. Washington, DC: Department of Defense. Sandholm, T.; Gilpin, A.; and Conitzer, V. 2005. Mixed- Integer Programming Methods for Finding Nash Equilib- ria. In Proceedings of the Twentieth National Conference on appointments in the Department of Industrial and Sys- Artificial Intelligence. Menlo Park, CA: AAAI Press. tems Engineering and the Department of Computer Sci- Stevens, D.; Hamilton, T.; Schaffer, M.; Dunham-Scott, ence at the Viterbi School of Engineering, University of D.; Medby, J. J.; Chan, E.; Gibson, J.; Eisman, M.; Mesic, Southern California. R.; Kelley, C.; Kim, J.; LaTourrette, T.; and Riley, K. J. 2006. Implementing Security Improve ment Options at Los Angeles Christopher Portway is a student in the Computer Sci- International Airport. Rand Documented Briefing. Santa ence Department at the Viterbi School of Engineering, Monica, CA: The Rand Corporation. University of Southern California. Wagenaar, W. A. 1972. Generation of Random Sequences Milind Tambe is a professor in the Computer Science by Human Subjects: A Critical Survey of Literature. Psy- Department at the Viterbi School of Engineering, Uni- chological Bulletin 77(1): 65–72. versity of Southern California.

James Pita is a graduate student in the Computer Science Craig Western is a student in the Computer Science Department at the Viterbi School of Engineering, Uni- Department at the Viterbi School of Engineering, Uni- versity of Southern California. versity of Southern California.

Manish Jain is a graduate student in the Computer Sci- Praveen Paruchuri is a research scientist with Intelligent ence Department at the Viterbi School of Engineering, Automation, Inc., Rockville, Maryland. University of Southern California. Saris Kraus is a professor in the Computer Science Fernando Ordóñez is an associate professor with joint Department at Bar-Ilan University.

SPRING 2009 57