Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Winter 2018 Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Knapsack Knapsack Problem Knapsack (Optimization) Problem Instance: Profits p0; p1; : : : ; pn−1 Weights w0; w1; : : : ; wn−1 Knapsack capacity M n Find: and n-tuple [x0; x1; : : : ; xn−1] 2 f0; 1g Pn−1 such that P = i=0 pixi is maximized, Pn−1 subject to i=0 wixi ≤ M. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Knapsack Example Objects: 1 2 3 4 weight (lb) 8 1 5 4 profit $500 $1,000 $ 300 $ 210 Knapsack capacity: M = 10 lb. Two feasible solutions and their profit: x1 x2 x3 x4 profit 1 1 0 0 $ 1,500 0 1 1 1 $ 1,510 This problem is NP-hard. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Knapsack Naive Backtracking Algorithm for Knapsack Examine all 2n tuples and keep the ones with maximum profit. Global Variables X; OptP; OptX. Algorithm Knapsack1 (l) if (l = n) then Pn−1 Pn−1 if i=0 wixi ≤ M then CurP i=0 pixi; if (CurP > OptP ) then OptP CurP ; OptX [x0; x1; : : : ; xn−1]; else xl 1; Knapsack1 (l + 1); xl 0; Knapsack1 (l + 1); First call: OptP −1; Knapsack1 (0). Running time: 2n n-tuples are checked, and it takes Θ(n) to check each solution. The total running time is Θ(n2n). ExhaustiveNote: Generation: not all Backtrackingn-tuples and are Branch-and-bound feasible but the algorithm will test all (theLucia whole Moura search tree is examined). We will improve this algorithm!!! Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound A General Backtracking Algorithm A General Backtracking Algorithm Represent a solution as a list: X = [x0; x1; x2;:::]. Each xi 2 Pi (possibility set) Given a partial solution: X = [x0; x1; : : : ; xl−1], we can use constraints of the problem to limit the choice of xl to Cl ⊆ Pl (choice set). By computing Cl we prune the search tree, since for all y 2 Pl n Cl the subtree rooted on [x0; x1; : : : ; xl−1; y] is not considered. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound A General Backtracking Algorithm Part of the search tree for the previous Knapsack example: w 8 1 5 4 i M = 10. pi $500 $1,000 $ 300 $ 210 [ ] C1={0,1} [1] [0] C2={0,1} [1,1] this part C3={0} [1,0] C3={0} not shown [1,1,0] C4={0} [1,0,1] [1,0,0] C4={0} [1,1,0,0] [1,0,0,0] profit=$1,500 profit= $500 : pruning Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Backtracking Algorithm with Pruning General Backtracking Algorithm with Pruning Global Variables X = [x0; x1;:::], Cl, for l = 0; 1;:::). Algorithm Backtrack (l) if (X = [x0; x1; : : : ; xl−1] is a feasible solution) then \Process it" Compute Cl; for each x 2 Cl do xl x; Backtrack(l + 1); Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Backtracking Algorithm with Pruning Backtracking with Pruning for Knapsack Global Variables X; OptP; OptX. Algorithm Knapsack2 (l; CurW ) Pn−1 if (l = n) then if ( i=0 pixi > OptP ) then Pn−1 OptP i=0 pixi; OptX [x0; x1; : : : ; xn−1]; if (l = n) then Cl ; else if (CurW + wl ≤ M) then Cl f0; 1g; else Cl f0g; for each x 2 Cl do xl x; Knapsack2 (l + 1; CurW + wlxl); First call: Knapsack2 (0; 0). Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Backtracking: Generating all Cliques Problem: All Cliques Instance: a graph G = (V; E). Find: all cliques of G without repetition 0 1 2 6 5 4 3 Cliques (and maximal cliques): ;; f0g; f1g;:::; f6g; f0; 1g; f0; 6g; f1; 2g; f1; 5g; f1; 6g; f2; 3g; f2; 4g; f3; 4g; f5; 6g; f0; 1; 6g; f1; 5; 6g; f2; 3; 4g: Definition Clique in G(V; E): C ⊆ V such that for all x; y 2 C, x 6= y, fx; yg 2 E. Maximal clique: a clique not properly contained into another clique. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v: find a largest clique in a special graph. Find all intersecting set systems: find all cliques in a special graph. Etc. Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Find a Steiner triple system of order v: find a largest clique in a special graph. Find all intersecting set systems: find all cliques in a special graph. Etc. Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G. Exact cover of sets by subsets: find clique with special property. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Find all intersecting set systems: find all cliques in a special graph. Etc. Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G. Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v: find a largest clique in a special graph. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Etc. Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G. Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v: find a largest clique in a special graph. Find all intersecting set systems: find all cliques in a special graph. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G. Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v: find a largest clique in a special graph. Find all intersecting set systems: find all cliques in a special graph. Etc. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques In a Backtracking algorithm, X = [x0; x1; : : : ; xl−1] is a partial solution () fx0; x1; : : : ; xl−1g is a clique. But we don't want ot get the same k-clique k! times: [0; 1] extends to [0; 1; 6] [0; 6] extends to [0; 6; 1] So we require partial solutions for be in sorted order: x0 < x1 < x2 < : : : < xl−1. Let Sl−1 = fx0; x1; : : : ; xl−1g for X = [x0; x1; : : : ; xl−1]. The choice set of this point is: if l = 0 then C0 = V if l > 0 then Cl = fv 2 V n Sl−1 : v > xl−1 and fv; xg 2 E for all x 2 Sl−1g = fv 2 Cl−1 n fxl−1g : fv; xl−1g 2 E and v > xl−1g Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques So, C0 = V Cl = fv 2 Cl−1 n fxl−1g : fv; xl−1g 2 E and v > xl−1g, for l > 0 To compute Cl, define: Av = fu 2 V : fu; vg 2 Eg (vertices adjacent to v) Bv = fv + 1; v + 2; : : : ; n − 1g (vertices larger than v) Cl = Axl−1 \ Bxl−1 \Cl−1. To detect if a clique is maximal (set inclusionwise): Calculate Nl, the set of vertices that can extend Sl−1: N0 = V Nl = Nl−1 \ Axl−1 . Sl−1 is maximal () Nl = ;. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Algorithm AllCliques(l) Global: X, Cl(l = 0; : : : ; n − 1), Al, Bl pre-computed. if (l = 0) then output ([ ]); else output ([x0; x1; : : : ; xl−1]); if (l = 0) then Nl V ; else Nl Axl−1 \ Nl−1; if (Nl = ;) then output (\maximal"); if (l = 0) then Cl V ; else Cl Axl−1 \ Bxl−1 \Cl−1; for each (x 2 Cl) do xl x; AllCliques(l + 1); First call: AllCliques(0).