AND/OR Branch-and-Bound Search for Pure 0/1 Integer Linear Programming Problems Radu Marinescu and Rina Dechter School of Information and Computer Science University of California, Irvine, CA 92697-3425 {radum,dechter}@ics.uci.edu Abstract. AND/OR search spaces have recently been introduced as a unify- ing paradigm for advanced algorithmic schemes for graphical models. The main virtue of this representation is its sensitivity to the structure of the model, which can translate into exponential time savings for search algorithms. In this paper we extend the recently introduced AND/OR Branch-and-Bound algorithm (AOBB) [1] for solving pure 0/1 Integer Linear Programs [2]. Since the variable selection can have a dramatic impact on search performance, we introduce a new dynamic AND/OR Branch-and-Bound algorithm able to accommodate variable ordering heuristics. The effectiveness of the dynamic AND/OR approach is demonstrated on a variety of benchmarks for pure 0/1 integer programming, including instances from the MIPLIB library, real-world combinatorial auctions and random unca- pacitated warehouse location problems. 1 Introduction A constraint optimization problem is the minimization/maximization of an objective function subject to a set of constraints on the possible values of a set of independent decision variables. An important class of constraint optimization problems are the Inte- ger Linear Programming problems (ILP) [2] where the objective is to optimize a linear function of integer-valued variables, subject to a set of linear equality or inequality con- straints defined on subsets of variables. The classical approach to solving ILPs is the branch-and-bound method [3] which maintains the best solution found so far, while discarding partial solutions which cannot improve on the best. The AND/OR search space for graphical models [4] is a newly introduced frame- work for search that is sensitive to the independencies in the model, often resulting in exponentially reduced complexities. It is based on a pseudo-tree that captures indepen- dencies in the graphical model, resulting in a search tree exponential in the depth of the pseudo-tree, rather than in the number of variables. The AND/OR Branch-and-Bound algorithm (AOBB) is a new search method that explores the AND/OR search tree for solving optimization tasks in graphical models [1]. In this paper we present an extension of the algorithm for solving optimization problems from the class of pure 0/1 Integer Linear Programs [2]. A pure 0/1 integer linear program is a linear program where all the decision variables are restricted to be either 0 or 1 at the optimal solution. Since variable selection can have a dramatic impact on search performance [2], we introduce a dynamic AND/OR Branch-and-Bound search algorithm that combines the AND/OR decomposition principle with variable ordering heuristics. There are two orthogonal approaches to incorporating dynamic orderings into AOBB. The first one im- proves AOBB by applying an independent semantic variable ordering heuristic whenever the partial order dictated by the decomposition principle allows. The second, orthogonal approach gives priority to the semantic variable ordering heuristic and applies problem decomposition as a secondary principle. We demonstrate empirically the efficiency of the dynamic AND/OR Branch-and-Bound approach on several benchmarks for pure 0/1 integer linear programming problems, including test instances from the MIPLIB library, combinatorial auctions simulating radio spectrum allocation and random uncapacitated warehouse location problems. The paper is organized as follows. In Section 2 we present background on con- straint optimization problems and integer linear programming. Section 3 presents the AND/OR search space as well as an efficient heuristic for constructing low depth bal- anced pseudo-trees. In Section 4 we introduce the AND/OR Branch-and-Bound algo- rithm, specialized for solving pure 0/1 integer linear programs. In Section 5 we intro- duce the dynamic AND/OR Branch-and-Bound algorithm. Section 6 shows our empir- ical evaluation and Section 7 concludes. 2 Background 2.1 Constraint Optimization Problems A finite Constraint Optimization Problem (COP) is a four-tuple hX , D, C,zi, where X = {X1, ..., Xn} is a set of variables, D = {D1, ..., Dn} is a set of finite domains, C = {C1, ..., Cm} is a set of constraints on the variables and z is a global cost func- tion (i.e. objective function) to be optimized. The scope of a constraint Ci, denoted scope(Ci) ⊆ X , is the set of arguments of Ci. Constraints can be expressed exten- sionally, through relations, or intentionally, by a mathematical formula (equality or inequality). An optimal solution to a COP is a complete value assignment to all the variables such that every constraint is satisfied and the objective function is minimized or maximized. With every COP instance we can associate a constraint graph G which has a node for each variable and connects any two nodes whose variables appear in the scope of the same constraint. The induced graph of G relative to an ordering d of its variables, denoted G∗(d), is obtained by processing the nodes in reverse order of d. For each node all its earlier neighbors are connected, including neighbors connected by previously added edges. Given a graph and an ordering of its nodes, the width of a node is the number of edges connecting it to nodes lower in the ordering. The induced width of a graph, denoted w∗(d), is the maximum width of nodes in the induced graph. 2.2 Integer Linear Programming A Linear Program (LP) consists of a set of continuous variables and a set of linear con- straints (equalities or inequalities). The goal is to optimize a global linear cost function subject to the constraints. One of the standard forms of a linear program is: min{c>x | Ax ≤ b, x ≥ 0} (1) where c ∈ Rn, b ∈ Rm, A ∈ Rm×n and x ∈ Rn. Here c represents the cost vector and x is the vector of decision variables. The vector b and the matrix A define the m linear constraints. Linear programs are usually solved by Dantzig’s SIMPLEX method [5]. A Mixed Integer Linear Programming (MILP) problem is a linear program where some of the decision variables are constrained to have only integer values at the optimal solution. An important special case is a decision variable xi that is integer with 0 ≤ xi ≤ 1. This forces xi to be either 0 or 1 at the solution. Variables like xi are called 0/1 or binary integer variables. Subsequently, a MILP problem with binary integer variables is also called a 0/1 Mixed Integer Linear Programming problem. A pure 0/1 Integer Linear Programming problem is a MILP where all the decision variables are binary. Pure 0/1 ILPs can formulate many practical problems such as capital budgeting [6], cargo loading [7], processor allocation in distributed systems [8] or combinatorial auctions [9, 10]. Clearly, any pure 0/1 integer linear program can be viewed as a finite COP instance hX , D, C,zi with linear constraints and a linear objective function. In the remaining z n c X of the paper we will consider a minimization problem defined by = Pi=1 i i subject to m linear constraints C = {C1, ..., Cm}, over n binary decision variables X = {X1, ..., Xn}. 2.3 Branch-and-Bound Search for Constraint Optimization Branch-and-Bound (BB) is a general search method for solving constraint optimization problems [3]. It traverses the search tree defined by the problem, where internal nodes represent partial assignments and leaf nodes denote complete ones, which may or may not be optimal. During the traversal, which is usually depth first, BB maintains un upper bound ub, the cost of the best solution found so far. At each internal node the algorithm computes a lower bound lb on the optimal extension of the current partial assignment. When lb ≥ ub, the current best cost cannot be improved and the algorithm backtracks pruning the subtree below the current node. Otherwise, the algorithm moves forward and tries to instantiate the next variable in the ordering. In the context of pure 0/1 integer linear programs, the lower bound of a subproblem is obtained by solving its linear relaxation (i.e. relaxing the integrality restrictions). In this case the branching process can fail at a particular node for one of the following reasons: (i) the LP solution can be integer; or (ii) the LP problem can be infeasible; or (iii) the lower bound exceeds the upper bound (for more details see [2,3]). 3 AND/OR Search Spaces The classical way to do search is to instantiate variables one at a time, following a static/dynamic variable ordering. In the simplest case, this process defines a search tree (called here OR search tree), whose nodes represent states in the space of partial as- signments. The traditional search space does not capture independencies that appear in = + − + − + minimize : z 7A 3B 2C 5D 6E 8F A subject to : − + ≤ 3 A 12 B C 3 B − 2B + 5C − 3D ≤ −2 2 A + B − 4E ≤ 2 C E A − 3E + F ≤1 A, B,C, D, E, F ∈{}1,0 D F (a) (b) OR A AND 0 1 OR B B AND 0 1 0 1 OR C E C E C E C E AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 OR D F F D F F D F F D F AND 1 0 1 0 1 0 1 0 1 0 1 1 0 0 1 0 1 0 1 (c) Fig. 1. The AND/OR search space.