Branch and Bound Now that we have studied the state-space traversal methods and seen the benefit of LC-Search using the 15-puzzle problem, let us now formally introduce . We should remember the various types of nodes in state-space tree: 1. Live-Node: node that is yet to be expanded 2. E-Node: Node that is currently being expanded 3. Dead-Node: Node that is already expanded or need not be expanded The term branch-and-bound refers to all state space search methods in which: 1. [Branch] All children of the E-Node are generated before any other live node can become the E-node. 2. [Bound] Bounding functions are used to help avoid the generation of subtrees that do not contain an answer node. Question: Out of the following traversal methods, pick the method which does not apply to Branch and Bound: a. DFS b. BFS c. D-Search d. LC-Search Answer: DFS. In DFS you never expand an E-Node fully. In B&B it is mandatory that all children of an E- Node are generated before any other live node can become the E-node. Let us consider an example of 4-queens problem given its portion of state space tree below:

The nodes are numbered (inside circles) in DFS order that generated. Outside the numbers indicate the BFS/FIFO order. However this time note that we are also bounding the nodes, i.e. we do not expand nodes that do not lead to the solution. When we use bounding along with the search methods they lead to following types of branch and bound 1. FIFOBB: First in First our Branch and Bound is where we use FIFO/BFS search while we use bounding functions to kill nodes that do not lead to solution or lead to non-optimal solutions. 2. LIFOBB: Last in First our Branch and Bound is where we use LIFO/D-search while we use bounding functions to kill nodes that do not lead to solution or lead to non-optimal solutions. 3. LCBB: Least Cost Branch and Bound is where we use LC-search (priority BFS) while we use bounding functions to kill nodes that do not lead to solution or lead to non-optimal solutions. This is most commonly used instead of FIFOBB or LIFOBB as it leads to solutions quicker. (see notes on LC-Search) Bounding We have already discussed search methods i.e. branching earlier. Let us cover point 2 of Branch and Bound, i.e. bounding in this section. When we think of bounding any mathematical entity, we refer to generating upper and lower bounds. We did this in analysis of algorithms using asymptotic notations. Let us define following entities for any node x ion state-space tree to solve any minimization problem. 1. C(x) = cost of solution passing through x. This is the final cost for which we need to solve the problem through this node x. This cannot be estimated and is presented for theoretical reasons. 2. LC(x) = lower bound on the cost of solution passing through x. We did this in LC-Search also by computing the least cost through x. This can be estimated as we did in 15-puzzle. 3. UC(x) = upper bound on the cost of the solution passing through x. 4. U = a variable upper, which is not a function of x, but contains a number that tells the till-now known best upper bound of the solution to the problem being solved. Initial value of U = ∞ Now whenever we expand an E-Node, if any child has a lower bound greater than the best known upper bound we kill that node and thus remove the subtree of those nodes from search space. Question 1. C(x) <= LC(x) True/False? Answer 1. False. LC(x) is a lower bound so the actual solution C(x) will be greater than or equal to this lower bound. Question 2. For any node x, U > LC(x) True/False? Answer 2. True. U is an upper bound so any lower bounds or the actual solution can be less than or equal to the upper bound Good! Now you understand upper and lower bounds. Congratulations! Question 3. What is minimization problem? Answer 3. Problem where a certain objective function has to be minimized subject to certain constraints, for example a. Finding the minimum cost of spanning tree in a weighted graph. b. Finding the minimum number of colors to color the nodes of a graph so that no to adjacent nodes share same color. c. Finding the minimum steps to solve 15-puzzle. d. Finding the minimum cost of a Hamiltonian cycle in a graph. e. Optimal merge pattern f. Finding minimum length codes to represent a message ( g. Etc. Question 4. What is maximization problem? Answer 4. Problem where a certain objective function has to be maximized subject to certain constraints, for example a. Finding the maximum profit in 0/1 knapsack problem. b. Finding the maximum profit in Job-Scheduling. c. Etc. Question 5. Can the same strategy of LCBB be applied as it is to maximization problems? Answer 5: No. The maximization problem has to be inverted into minimization problem by multiplying the objective function with -1. For example 10 > 5 but -10 < -5. So by multiplying max cost by -1 we convert it to min cost. This is the method given in your text book. Alternatively, we can modify the bounding to compute UC(x) and L instead of LC(x) and U. where UC(x) is the upper bound of a solution passing through x and L is the global lower bound on a solution. And instead of LC-Search we do MC-Search or maximum cost search, where we use a max- instead of a min-heap. 0/1 Knapsack using LCBB

Review of problem:

• Input – Capacity m – n items with weights wi and values vi • Goal – Output a set of items S such that • the sum of weights of items in S is at most m • and the sum of values of items in S is maximized

This is a maximization problem. We need to convert it to minimization problem by multiplying objective function (sum of profits) by -1. i.e our new goal is:

– Output a set of items S such that • the sum of weights of items in S is at most m • and the -1*(sum of values of items in S) is minimized

Just as in backtracking we create a fixed-tuple state-space tree of 0/1 knapsack by defining a root node with

 profit p = 0  weight in knapsack w = 0  items considered till now k = 0

Question: How many levels this tree can have? Answer: n, where n is number of items. Root is level 0. At each level i, we decide whether to include or exclude item i.

Question: How many children each node can have? Answer: 2. because we have to make a binary choice. Whether to include or exclude item i.

Question: If we are at a node at level l, how many items have we decided upon? Answer: l. from 1 to l.

Question: If we are at a node at level l, how many items are yet to be chosen in its subtree? Answer: n - l. from l + 1 to n.

Computing the bounds

Upper Bound: At a given node x at level l, let p be the profit till now in knapsack and let w be the weight in knapsack. If m was the capacity of knapsack, remaining weight in knapsack is m-w. If items are listed in order of decreasing per unit price (as we did in greedy strategy), we can consider the remaining items l+1 to n and see which all of them in that order can be fully accommodated in the knapsack. The resulting profit is our minimum estimate for the maximization problem. We multiply it by -1 to convert this into an upper bound, i.e. the algorithm for upper bound of a node can be written as:

UC(x) 1. x.u = x.p; // set upper bound to current profit at this node 2. w = x.w // set w to current weight in knapsack at this node 3. l = x.level + 1 // start from the next level 4. while (w