Combinatorial optimization Combinatorial optimization problems Heuristic Algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Combinatorial optimization Optimization In general an optimization problem can be formulated as: minimize z(x) subject to x ∈ X where • x is a vector of variables; • z(x) is the objective function; • X is the feasible region, i.e. the set of solutions satisfying the constraints. A solution is an assignment of values to the variables. Combinatorial optimization Combinatorial optimization In general a combinatorial optimization problem can be formulated as: minimize z(x) subject to x ∈{0, 1}|E| The feasible region X is defined as a subset of the set of all possible subsets of a given ground set (!). Let’s say it again with an example... Combinatorial optimization Combinatorial optimization: example Ground set E: the set of edges of a given graph G. All possible subsets of E are the 2|E| subsets of edges. Only a subset X of them are, for instance, spanning trees. The condition x ∈ X describes the feasible region of any problem involving the search of an optimal spanning tree. Although the ground set E is rather small, the number of its subsets is exponential in its cardinality (2|E|) and hence the number of solutions can be very large. Even restricting the search to feasible solutions, the cardinality of X can be combinatorial: it grows as a combinatorial number when |E| grows. Combinatorial optimization The combinatorial problems structure The variables, the constraints and the objective function of a combinatorial optimization problem define its “combinatorial structure”. This is a semi-informal way to indicate the main characteristics of the problem, that affect the effectiveness of different solution procedures. The analysis of the combinatorial structure of any given problem gives useful indications on the most suitable algorithm to solve it. Moreover it can uncover similarities between seemingly different problems. Combinatorial optimization Problems on weighted sets: the knapsack problem (KP) From a ground set of items, select a subset such that the value of the selected items is maximum and their weight does not exceed a given capacity. We are given • a ground set N of items • a weight function a : N → N • a capacity b ∈ N • a value function c : N → N We can associate a binary variable x with each element of the ground set: the solution space is {0, 1}n, where n = |N|. In this way every solution corresponds to a subset and to its binary characteristic vector x. The feasible region X contains the subsets with total weight not larger than b: n X = {x ∈{0, 1} : X aj xj ≤ b} j∈N The objective is the maximization of the total value: max z(x)= X cj xj . j∈N Combinatorial optimization Example D N ABCDEF F E c 724541 A a 532311 B b 8 Knapsack = C x ′ = (0, 0, 1, 1, 1, 0, 0) ∈ X x ′′ = (1, 0, 1, 1, 0, 0, 0) 6∈ X z(x ′)= 13 z(x ′′)= 16 D C E D A Knapsack Knapsack C Combinatorial optimization Problems on subsets with a metric: the Max Diversity Problem (MDP) Given a ground set of items, we want to select a subset of given cardinality, maximizing a measure of the pairwise distances between the selected items. We are given • a ground set N, • a distance function d : N × N → N, • a positive integer number k ∈{1,..., |N|}. A possible choice of the variables is analogous to the previous case: x is the binary characteristic vector of the selected subset. The feasible region X contains all the subsets of cardinality k: n X = {x ∈{0, 1} : X xj = k}. j∈N The objective is to maximize the sum of the pairwise distances: max z(x)= X dij xi xj . i,j∈N It is a quadratic function. Combinatorial optimization Example C B E D A G F x ′ = (0, 0, 1, 1, 1, 0, 0) ∈ X x ′′ = (1, 0, 1, 0, 0, 0, 1) ∈ X z(x ′)= 24 z(x ′′)= 46 C C B B E E D D A A G G F F Combinatorial optimization Additive and non-additive objective functions In general the objective function associates rational or integer values with (feasible) subsets of the ground set. z : X → N Computing its value can be more or less difficult. • The KP has an additive (linear) objective function: its value is the sum of the values of another value function c whose domain is the ground set: c : N → N • The MDP has a non-additive (quadratic) objective-function. Both of them are easy to compute, but the additive objective function of the KP is easier to update when an item is inserted or deleted from the solution. It is enough to • add cj for each inserted item j ∈ N; • subtract cj for each deleted item j ∈ N. For the non-additive objective function of the MDP this is not true. Combinatorial optimization Set partitioning problems: the Bin Packing Problem (BPP) A set of weighted items must be partitioned into the minimum number of subsets, so that the total weight of each subset is within a given capacity. We are given: • a set N of items, • a set M of bins, • a weight function a : N → N, • a capacity b of the bins. The ground set of the problem contains all the item-bin pairs. E = N × M A solution is represented by nm binary variables with two indices i ∈ N and j ∈ M. The feasible region contains the partitions of the items, complying with the capacity constraints: nm X = {x ∈{0, 1} : X xij = 1 ∀i ∈ N, X ai xij ≤ b ∀j ∈ M}. j∈M i∈N Combinatorial optimization Set partitioning problems: the Bin Packing Problem (BPP) The objective is to minimize the number of bins used. min z(x)= |{j ∈ M : X xij > 0}|. i∈N To get rid of the function “cardinality of”, we need additional binary variables: the characteristic vector y of the bin subset. We obtain: nm+m X = {(x, y) ∈{0, 1} : X xij = 1 ∀i ∈ N, X ai xij ≤ byj ∀j ∈ M}. j∈M i∈N min z(y)= X yj . j∈M Combinatorial optimization Example ′ S = (A, 1) , (B, 1) , (C, 2) , (D, 2) , (E, 2) , (F, 3) , (G, 4) , (H, 5) , (I, 5) ∈ X y ′ = (1, 1, 1, 1, 1) z(y ′)= 5 ′′ S = (A, 1) , (B, 1) , (C, 2) , (D, 2) , (E, 2) , (F, 3) , (G, 4) , (H, 1) , (I, 4) 6∈ X y ′′ = (1, 1, 1, 1, 0) z(y ′′)= 4 Combinatorial optimization Set partitioning problems: the Parallel Machine Scheduling Problem (PMSP) A set of indivisible jobs of given duration must be assigned to a set of machines, minimizing the overall completion time. We are given: • a set N of jobs, • a set M of machines, • a processing time function p : N → N. The ground set contains all job-machine pairs. We can use the same variable choice as for the BPP. The feasible region X contains the partitions of N into subsets. nm X = {x ∈{0, 1} : X xij = 1 ∀i ∈ N.} j∈M The objective function is the minimization of the maximum working time among all machines. min z(x)= max X pi xij . j∈M i∈N Combinatorial optimization Example N = {L1, L2, L3, L4, L5, L6} M = {M1, M2, M3} Job L1 L2 L3 L4 L5 L6 p 80 40 20 30 15 80 M1 L1 L5 ′ S = (L1, M1) , (L2, M2) , (L3, M2) , M2 L2 L3 L4 (L4, M2) , (L5, M1) ∈ X ′ M3 L6 z(x )= 95 95 M1 L1 L2 ′′ S = (L1, M1) , (L2, M1) , (L3, M2) , M2 L3 L4 L5 (L4, M2) , (L5, M2) ∈ X ′′ M3 L6 z(x )= 120 120 Combinatorial optimization Sensitive and insensitive objective functions The objective functions of the BPP and the PMSP • are not additive, • are not easy to compute. Small changes in the solution x may have different impact on the objective function value: • variation equal to the duration of the changed job (e.g. L5 on M1); • no variation (e.g., L5 on M3); • intermediate variation (e.g., L2 on M2). This is because the effect of the change depends both • on the modified elements, • on the non-modified elements. In both problems the objective function is “flat”: many different feasible solutions have the same value. Combinatorial optimization Problems on matrices: the Set Covering Problem (SCP) Given a binary matrix and a vector of costs associated with the columns, select a minimum cost subset of columns covering all the rows. We are given: • a binary matrix a ∈ Bm,n with a set R or m rows and a set C of n columns, • a cost function c : C → N. A column j ∈ C covers a row i ∈ R if and only if aij = 1. The ground set is the set of columns C. The feasible region contains the subsets of columns that cover all the rows. n X = {x ∈{0, 1} : X aij xj ≥ 1 ∀i ∈ R}. j∈C The objective is to minimize the total cost of the selected columns: min z(x)= X cj xj . j∈C Combinatorial optimization Example c 4 6 10 14 5 6 011 110 001 100 a 110 001 000 111 111 010 0 1 1 1 1 0 2 0 0 1 1 0 0 1 x ′ = (1, 0, 1, 0, 1, 0) ∈ X a 1 1 0 0 0 1 1 0 0 0 1 1 1 1 z(x ′)= 19 1 1 1 0 1 0 3 0 1 1 1 1 0 1 0 0 1 1 0 0 0 x ′′ = (1, 0, 0, 0, 1, 1) 6∈ X a 1 1 0 0 0 1 2 0 0 0 1 1 1 2 z(x ′′)= 15 1 1 1 0 1 0 2 Combinatorial optimization The feasibility test In a heuristic algorithm the following sub-problem may often occur: Given a solution x, is it feasible or not? x ∈ X? This is a decision problem.