Computers & Industrial Engineering 111 (2017) 216–227 Contents lists available at ScienceDirect Computers & Industrial Engineering journal homepage: www.elsevier.com/locate/caie A branch and price algorithm for a Stackelberg Security Game ⇑ Felipe Lagos a, Fernando Ordóñez b, , Martine Labbé c,d a Georgia Institute of Technology, United States b Universidad de Chile, Chile c Université Libre de Bruxelles, Belgium d INRIA, Lille, France article info abstract Article history: Mixed integer optimization formulations are an attractive alternative to solve Stackelberg Game prob- Received 14 June 2017 lems thanks to the efficiency of state of the art mixed integer algorithms. In particular, decomposition Received in revised form 26 June 2017 algorithms, such as branch and price methods, make it possible to tackle instances large enough to rep- Accepted 28 June 2017 resent games inspired in real world domians. Available online 29 June 2017 In this work we focus on Stackelberg Games that arise from a security application and investigate the use of a new branch and price method to solve its mixed integer optimization formulation. We prove that Keywords: the algorithm provides upper and lower bounds on the optimal solution at every iteration and investigate Column generation the use of stabilization heuristics. Our preliminary computational results compare this solution approach Stackelberg games Security with previous decomposition methods obtained from alternative integer programming formulations of Stackelberg games. Ó 2017 Published by Elsevier Ltd. 1. Introduction to attack. Such Stackelberg Security Game models have been used in the deployment of decision support systems with specialized Stackelberg games model the strategic interaction between algorithms in real security domain applications (Jain et al., 2010; players, where one participant – the leader – is able to commit Pita, Tambe, Kiekintveld, Cullen, & Steigerwald, 2011; Shieh et al., to a strategy first, knowing that the remaining players – the follow- 2012). ers – will take this strategy into account and respond in an optimal Recent work has developed efficient integer optimization solu- manner. These games have been used to represent markets in tion algorithms for different variants of the SSGs (Hochbaum, Lyu, which a participant has significant market share and can commit & Ordóñez, 2014; Jain, Kardes, Kiekintveld, Ordóñez, & Tambe, to a strategy (von Stackelberg, Bazin, Hill, & Urch, 2010), where 2010; Jain, Kiekintveld, & Tambe, 2011; Jain et al., 2011; government decides tolls or capacities in a transportation network Kiekintveld et al., 2009). In general terms these optimization prob- (Labbé, Marcotte, & Savard, 1998), and of late have been used to lems are formulated with the defender committing to a mixed represent the attacker-defender interaction in security domains (randomized) strategy and the attacker(s) responding with a pure (Jain et al., 2010). These games are examples of bilevel optimiza- strategy after conducting surveillance of the defender’s mixed tion problems, which are in general non convex optimization prob- strategy. A mixed strategy refers to a probability distribution over lems that are difficult to solve. the possible actions while a pure strategy corresponds to selecting In this work we focus on a specific class of Stackelberg games one of the possible actions. In this SSG, the defender mixed strate- which we refer to as Stackelberg Security Games (SSG) that arise gies are probability distributions over possible patrolling strate- in security domains and have a particular payoff structure (Yin, gies, while the attacker’s pure strategy corresponds to selecting a Korzhyk, Kiekintveld, Conitzer, & Tambe, 2010). In a SSG, the secu- specific target to attack. In addition, the number of actions of the rity (or defender) behaves as the leader selecting a patrolling strat- defender can be exponential in size, with respect to the targets egy first and then, possibly many attackers act as the follower, and defense resources, due to the combinatorics of using N observing the defender’s patrolling strategy and deciding where resources to patrol m targets. This illustrates that to solve SSGs we have to address mixed integer optimization problems with ⇑ exponential number of variables. Addressing the combinatorial Corresponding author. size of defender strategies has led to both development of branch E-mail addresses: [email protected] (F. Ordóñez), [email protected] (M. Labbé). and price methods (Kiekintveld et al., 2009) and constraint gener- http://dx.doi.org/10.1016/j.cie.2017.06.034 0360-8352/Ó 2017 Published by Elsevier Ltd. F. Lagos et al. / Computers & Industrial Engineering 111 (2017) 216–227 217 ation methods (Yang, Jiang, Tambe, & Ordóñez, 2013). There are, follower will choose an optimal response and will break ties in favor however, problem instances that arise from real security applica- of the leader. In other words, following the formal definition of a tions that still challenge existing solution methods. Here we inves- SSE in Kiekintveld et al. (2009), a pair of strategies x and h tigate a new branch and price method developed for a novel ðq ðxÞÞh2H form a SSE if they satisfy: formulation of Stackelberg games (MIPSG), introduced in Casorrán-Amilburu, Fortz, Labbé, and Ordóñez (2017). This new ; h P 1. The leader (defender) maximizes utility: uDðx ðq ðxÞÞh2HÞ formulation has been shown to provide tighter linear relaxations 0; h 0 0 uDðx ðq ðx ÞÞh2HÞ for any feasible x than other existing mixed integer formulations and to give the con- h ; h P 2. The followers (attackers) play a best response: uAðx q ðxÞÞ vex hull of the feasible integer solutions when there is only one h u ðx; gÞfor any feasible g. follower. A 3. The follower breaks ties in favor of the leader: We begin by introducing notation and describing the integer u ðx; ðqhðxÞÞ Þ P u ðx; ðqhÞ Þ for any ðqhÞ that is optimal optimization formulations that have been considered previously D h2H D h2H h2H for the followers, that is for any h; qh 2 argmax uh ðx; gÞ. in the next section. We also introduce the equivalent MIPSG for- g A mulation. In Section 3 we present the column generation algorithm for the solution of the linear relaxation of MIPSG, along with a This can be formulated as the following bilevel optimization speed up that can be obtained by aggregating subproblems, and problem, where e is the vector of all ones of appropriate the existence of upper and lower bounds at every iteration. We dimension: also describe the branching strategies used in adapting this column ; h max uDðx ðq Þh2HÞ generation to a Branch and Price method and how to apply dual s:t: eT x ¼ 1; x P 0 stabilization techniques. We present our preliminary computa- h h ; T ; P h H: tional results in Section 4 and provide concluding remarks in q ¼ argmaxgfuAðx gÞje g ¼ 1 g 0g 2 Section 5. Given that the inner optimization problem is a linear optimization problem over the jQj dimensional simplex, there always exists an 2. Integer optimization formulations of SSG optimal pure-strategy response for the attacker, so in the integer optimization formulations we present now we restrict our attention In a Stackelberg security game we consider that the leader is the to the set of pure strategies for the attacker. As we see below, the defender and the attacker (of possibly many types) is the follower. optimality condition of the inner optimization problem can be We let H be the set of possible attacker types and assume that ph expressed with linear constraints and integer variables when we corresponds to a known a priori probability distribution that the make use of the fact that the followers respond with an optimal defender is facing an attacker of type h 2 H. The attacker may pure strategy. Although this leads to being able to use efficient decide to attack any one of a set of targets Q. The mixed strategy mixed integer optimization solution procedures, the problem for the hth attacker is the vector of probabilities over this set of tar- remains theoretically difficult as the problem of choosing the opti- gets, which we denote as qh ¼ðqhÞ . The defender allocates up to mal strategy for the leader to commit to in a Bayesian Stackelberg j j2Q game is NP-hard (Conitzer & Sandholm, 2006). N resources to protect targets, with N < jQj. Each resource can be The payoffs for agents depend only on the target attacked, the assigned to a patrol that protects multiple targets, s # Q, so the # adversary type and whether or not a defender resource is covering set of feasible patrols for one resource is a set S PðQÞ, where h PðQÞ represents the power set of Q. The defender’s pure strategies, the target. Let the parameter Rdj denote the defender’s utility, or or joint patrols, are combinations of up to N such patrols, one for reward, if j 2 Q is attacked by adversary h 2 H when it is covered each available resource. In addition we assume that in a joint by a defender resource. If j 2 Q is not covered, the defender patrol a target is covered by at most one resource. Let X denote h receives a penalty Pdj . Likewise, the attacker’s utilities are denoted the set of joint patrols, or defender strategies. A joint patrol i 2 X, h by a reward Raj when target j is attacked and not covered and pen- jQj can be represented by the vector a ¼½a ; a ; ...; a 2f0; 1g h i i1 i2 ijQj alty Pa , when j is attacked while protected. Therefore if we let j 2 i where a represents whether or not target j is covered in strategy j ij denote when target j 2 Q is protected by patrol i 2 X, then we con- i.