Informed (Or Heuristic) Search Methods

US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Informed (or Heuristic) Search Methods

• Heuristics • Best-ﬁrst search • The algorithm A∗ • Properties of heuristic functions • Branch-and-bound

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Heuristics

All blind search algorithms that we discussed have time complexity of order O(bd) or something similar. This is unacceptable in real problems! In large search spaces, one can do a lot better by using domain-speciﬁc information to speed-up search. Heuristics are “rules of thumb” for selecting the next node to be expanded by a search algorithm.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Best-First Search

A blind search algorithm could be improved if we knew the best (or “seemingly best”) node to expand.

function BestFirstSearch(problem,EvalFn) returns a solution sequence QueuingF n ← a function that orders nodes in ascending order of EvalFn return TreeSearch(problem, QueuingF n)

The function EvalFn is called the evaluation function. Note: GraphSearch can be used instead of TreeSearch.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Evaluation Functions and Heuristic Functions

There is a whole family of best-first search algorithms with different evaluation functions. A key component of many of these algorithms is a heuristic function h such that h(n) = estimated cost of the cheapest path from the state at node n to a goal state h can be any function such that h(n) = 0 if n is a goal node. But in order to find a good heuristic function, we need domain specific information.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Greedy Best-First Search

GreedyBestFirstSearch tries to expand the node that it is closest to the goal, on the grounds that this is likely to lead to a solution quickly. Thus nodes are evaluated using the heuristic function h i.e., f(n) = h(n).

function GreedyBestFirstSearch(problem) returns a solution or failure return BestFirstSearch(problem, h)

The algorithm is greedy because it prefers to take the biggest possible bite out of the remaining cost to reach the goal.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Example - On the Road to Bucharest

Oradea 71 Neamt

Zerind 87 151 75 Iasi Arad 140 92 Sibiu 99 Fagaras 118 Vaslui 80 Rimnicu Vilcea Timisoara 142 111 Pitesti 211 Lugoj 97 70 98 Hirsova 146 85 Mehadia 101 Urziceni 75 86 138 Bucharest Dobreta 120 90 Craiova Eforie Giurgiu & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Example (cont’d)

hSLD(n)= straight line distance between n and the goal location. Distances for Bucharest are shown below:

Arad 366 Mehadia 241 Bucharest 0 Neamt 234 Craiova 160 Oradea 380 Dobreta 242 Pitesti 100 Eforie 161 Rimnicu Vilcea 193 Fagaras 176 Sibiu 253 Giurgiu 77 Timisoara 329 Hirsova 151 Urziceni 80 Iasi 226 Vaslui 199 Lugoj 244 Zerind 374

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Example (cont’d)

(a) The initial state Arad 366

(b) After expanding Arad Arad

Sibiu Timisoara Zerind 253 329 374

Sibiu Timisoara Zerind 329 374

Arad Fagaras Oradea Rimnicu Vilcea 366 176 380 193

(d) After expanding Fagaras Arad

Sibiu Timisoara Zerind 329 374

Arad Fagaras Oradea Rimnicu Vilcea 366 380 193

Sibiu Bucharest 253 0 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Greedy Best-First Search (cont’d)

Evaluation: • Complete? No (consider the problem of getting from Iasi to Fagaras; search can oscillate between Iasi and Neamt). • Time: O(bm) where m is the maximum depth of the search space. • Space: O(bm) • Optimal? No (The path Arad-Sibiu-Rimnicu Vilcea-Pitesti-Bucharest is optimal with cost 418. The path through Sibiu and Fagaras has cost 450. ) A good choice of h can reduce space and time substantially.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Greedy Best-First Search (cont’d)

The problem with greedy BFS is that it does not take into account the cost of getting to a node n that has minimum h(n).

Shall we have an improved algorithm if we take this cost into account?

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Uniform-cost search (UCS) - Revision

Modiﬁes BFS by always expanding the lowest cost node on the fringe (as measured by the path cost). Example:

0 S

ABC 1 5 15 S

A ABC 1 10 5 15 G 5 B 5 S SG 11

15 5 ABC C 15 G G 11 10 (a) (b) & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Uniform-cost search (cont’d)

Evaluation: • Complete? Yes.

∗ • Time: O(bdC /²e) where b is the branching factor, C∗ is the cost of the optimal solution and every action costs at least ² > 0. • Space: same as time. • Optimal? Yes. Completeness and optimality hold under the assumption that the branching factor is ﬁnite and the cost never decreases as we go along a path i.e., g(Successor(n)) ≥ g(n) for every node n. The last condition holds e.g., when each action costs at least ² > 0. & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The A∗ Search Algorithm

Greedy Best-First Search: • Searches by minimizing the estimated cost h(n) to the goal • Neither optimal nor complete Uniform Cost Search: • Searches by minimizing the cost g(n) of the path so far • Optimal, complete A∗ combines the above algorithms.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The A∗ Algorithm (cont’d)

A∗ is a best-ﬁrst search algorithm with evaluation function f(n) = g(n) + h(n). In this case f(n) is the estimated cost of the cheapest solution through n. function A∗Search(problem) returns a solution or failure return BestFirstSearch(problem, g + h)

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

A∗ Goes to Bucharest

See illustration in accompanying ﬁle astar-progress.ps or in AIMA.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The A∗ Algorithm (cont’d)

Let us assume that A∗ uses TreeSearch as its main subroutine and also: • The function h is chosen such that it never overestimates the cost to reach a goal. Such an h is called an admissible heuristic. If h is admissible then f(n) never overestimates the actual cost of the best solution through n. • The branching factor b is ﬁnite. • Every action costs at least δ > 0.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The A∗ Algorithm (cont’d)

Evaluation (under the previous assumptions): • Complete? Yes. • Time: exponential, unless the error in the heuristic function h grows no faster than the logarithm of the actual path cost. For most heuristics used in practice, the error is at least proportional to the path cost. But even when A∗ takes exponential time, it oﬀers a huge improvement compared to blind search.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The A∗ Algorithm (cont’d)

Evaluation (cont’d): • Space: O(bd). This is the main drawback of A∗. The algorithm iterative deepening A∗ (IDA∗) addresses the large space requirements of A∗. • Optimal? Yes.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Optimality and Completeness of A∗

Proposition. A∗ is optimal. Proof: Let us assume that the cost of the optimal solution is C∗

and a suboptimal goal node G2 appears on the fringe. Then

because G2 is suboptimal and h(G2) = 0, we have:

∗ f(G2) = g(G2) + h(G2) = g(G2) > C

Now consider a fringe node n which is on the optimal path. Because h does not overestimate the cost to the goal, we have:

f(n) = g(n) + h(n) ≤ C∗

So G2 will not be chosen for expansion!

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Optimality and Completeness of A∗ (cont’d)

The proof of optimality breaks down when A∗ uses GraphSearch as its main subroutine because GraphSearch can discard the optimal path to a repeated state if it is not the ﬁrst one to be generated. To guarantee optimality in this case, we have two options: • Modify GraphSearch so that it discards the most expensive path found to a node. • Impose an extra requirement of consistency or monotonicity on h.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Consistent Heuristics

Deﬁnition. A heuristic h is called consistent if for all nodes n, n0 such that n0 is a successor of n generated by any action a, h(n) ≤ c(n, a, n0) + h(n0). This is a form of the general triangle inequality: the sum of the lengths of any two sides of a triangle is greater than the length of the remaining side. Proposition. Every consistent heuristic is also admissible. Most admissible heuristics that one can think of are also consistent

(e.g., hSLD)!

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Optimality and Completeness of A∗ (cont’d)

Proposition. If h is consistent then the values of f for nodes expanded by A∗ along any path are non-decreasing. Proof: Let n be a node and n0 its successor. Then

g(n0) = g(n) + c(n, a, n0)

for some action a, and we have

f(n0) = g(n0)+h(n0) = g(n)+c(n, a, n0)+h(n0) ≥ g(n)+h(n) = f(n).

Thus we can conceptually draw contours in the state space like contours in a topographic map.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The behaviour of A∗

N Z

I A 380 S F V 400 T R

L P

H M U B 420 D E C G

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Optimality and Completeness of A∗ (cont’d)

A∗ search is complete: as we add contours of increasing f, we must eventually reach a contour where f is equal to the cost of the path to a goal state.

In fact, A∗ works as follows: • It expands all nodes with f(n) < C∗ • It may then expand some of the nodes right on the “goal contour”, for which f(n) = C∗, before selecting a goal node.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Optimality and Completeness of A∗ (cont’d)

A∗ expands no nodes with cost f(n) > C∗ where C∗ is the cost of the optimal solution. This is called pruning: eliminating possibilities from consideration without having to examine them.

Important: There is no other optimal algorithm of this type that is guaranteed to expand fewer nodes than A∗.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Heuristic Functions

What is a good heuristic for the 8-puzzle problem?

5 44 514 2 3

6 1 88 68 84

7 33 22 7 6 25

Start State Goal State & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

The 8-puzzle Problem

Formal speciﬁcation: • States: a state description speciﬁes the location of each tile and blank • Actions: blank moves L, R, U, D • Goal state • Path cost: length of the path

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Heuristic Functions (cont’d)

Heuristics for the 8-puzzle:

• h1 = the number of tiles in the wrong position

• h2 = the sum of the horizontal and vertical distances of all tiles from their goal positions (Manhattan distance). Both heuristics are admissible. Which one is better?

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Heuristic Functions (cont’d)

A way of characterizing the quality of a heuristic is to ﬁnd its eﬀective branching factor b∗. If the total number of nodes expanded by A∗ for a particular problem is N, and the solution depth is d then

N = 1 + b∗ + (b∗)2 + ··· + (b∗)d.

Usually, b∗ is fairly constant over a large number of instances. A well-deﬁned heuristic would have a value of b∗ close to 1.

& % US02 Teqnht NohmosÔnh M. Koumparkhc

Comparing A∗ and IDS

Search Cost Effective Branching Factor

d IDS A*(h1) A*(h2) IDS A*(h1) A*(h2) 2 10 6 6 2.45 1.79 1.79 4 112 13 12 2.87 1.48 1.45 6 680 20 18 2.73 1.34 1.30 8 6384 39 25 2.80 1.33 1.24 10 47127 93 39 2.79 1.38 1.22 12 364404 227 73 2.78 1.42 1.24 14 3473941 539 113 2.83 1.44 1.23 16 ± 1301 211 ± 1.45 1.25 18 ± 3056 363 ± 1.46 1.26 20 ± 7276 676 ± 1.47 1.27 22 ± 18094 1219 ± 1.48 1.28 24 ± 39135 1641 ± 1.48 1.26 US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Heuristic Functions (cont’d)

If h2(n) ≥ h1(n) for all nodes n then h2 dominates h1 (or h2 is

more informed) than h1.

Example: In the 8-puzzle h2 dominates h1. ∗ Theorem: If h2 dominates h1 then A using h2 will expand fewer ∗ nodes, on average, than A using h1. Lesson: It is always better to use an admissible heuristic function with higher values.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Heuristic Functions: How do we ﬁnd them?

It is possible to ﬁnd heuristic functions by considering relaxed versions of the given problem.

The cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem.

Relaxed problems can sometimes be generated automatically and then heuristics can be discovered automatically! Otherwise, we have to consider the problem at hand carefully and use our brain!

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Heuristic Functions: How do we ﬁnd them?

If we have admissible heuristics h1, . . . , hn such that no one dominates any of the others then we can choose

h = max(h1, . . . , hn).

Final note: The cost of computing the heuristic function for each node must be taken into account.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Extensions of A∗

The main problem with A∗ is its excessive use of memory for large problems. Several algorithms have been invented to tackle this problem: IDA∗, RBFS, MA∗, SMA∗. See AIMA for more details.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound

Another class of traditional intelligent search algorithms pioneered originally in the Operations Research community is branch-and-bound. Branch-and-bound has been designed for optimization problems. The main idea is to eliminate (prune) parts of the search space where we know that a solution cannot be found. In Operations Research courses branch-and-bound is usually presented in the context of solving integer linear programming problems. We will present a general formulation of branch-and-bound and examples from the following book: Christos Papadimitriou and Kenneth H. Steiglitz. Combinatorial Optimization - Algorithms and Complexity. Prentice-Hall, 1982. & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Branch-and-Bound (cont’d)

In branch-and-bound the search space is organized as a tree with the following two features: • Branching or partitioning. Each node represents a set of solutions which can be partitioned into mutually exclusive sets. Each subset in the partition is represented by a child of the node. • Lower bounding. There is an algorithm for computing a lower-bound on the cost of each solution in a given subset (i.e., obtained as a child of a node). Actually, we need a lower-bound if we are minimizing but an upper bound if we are maximizing.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Branch-and-Bound (cont’d)

The tree can be searched in any way we choose (e.g., DFS or BFS). However, if we have already found a solution with cost c by traversing the tree, and we are at a node with lower bound ≥ c, then we can safely ignore (prune) this branch of the tree and carry on our search with another branch. This is the step in branch-and-bound where heuristic knowledge about the problem domain is used. Notice the differences with the Artificial Intelligence terminology we have used so far: • Partition the current solution set – refine – branch (OR). • Extend a partial solution – create – generate-and-test (AI). & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound (cont’d)

algorithm BranchAndBound(problem) activeset := {problem} U := ∞; currentbest := anything while activeset is not empty do choose a branching node k ∈ activeset remove node k from activeset

generate the children of node k: child i, i = 1, . . . , nk, and the corresponding lower bounds zi for i = 1, . . . , nk do if zi ≥ U then kill child i else if child i is a complete solution and zi < U then U := zi; currentbest := i else add child i to activeset end end & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Notes

• The previous algorithm is for minimization problems. • activeset holds the live nodes at any point (corresponds to the fringe in earlier search algorithms). • U is used to hold the cost of the best complete solution obtained at any given time. U is an upper bound on the optimal cost.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Example

The shortest-path problem for directed weighted graphs. Deﬁnition. Let G = (V,E) be a directed graph with non-negative

weight cj ≥ 0 associated with each arc ej ∈ E. The shortest-path problem is to ﬁnd a directed path from a distinguished source node s to distinguished terminal node t, with the minimum total weight. We already know an algorithm for this problem: Dijkstra’s algorithm for single-source shortest-path problems in time O(n2) where n is the number of nodes in the graph.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Applying Branch-and-Bound

• Feasible solutions: the set of all paths from s to t. We want to ﬁnd the shortest path i.e., we are solving a minimization problem. • Branching is determined by considering which arc to choose to continue the path. I.e., a subset of the feasible solutions corresponds to all paths from s to t that start by the choices already made. • The lower bound used is the cumulative length of the partial path up to the current node. • We will branch from the node with the lowest lower bound.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Example Graph

1 10 100

2 30 5

10 50 60

3 4 20 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Applying Branch-and-Bound

Let us apply branch-and-bound to ﬁnd the shortest path from node 1 to node 5.

Step 1:

U = ∞

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound - Step 2

2 4 5 10 30 100

U = 100. currentbest is shown by an arrow. & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound - Step 3

2 4 5 10 30 100

3 60

U = 100 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound - Step 4

2 4 5 10 30 100

3 3 5 60 50 90

U = 90 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound - Step 5

2 4 5 10 30 100

3 3 5 60 50 90

5 60

U = 60 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Branch-and-Bound - Step 6

2 4 5 10 30 100

3 3 5 60 50 90

5 5 70 60

U = 60 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Dijkstra’s Algorithm

algorithm Dijkstra S := { 1 }; for i = 2 to n do D(i) := C(1, i); for i = 1 to n − 1 do choose a vertex w in V \ S such that D(w) is minimum; add w to S; for each vertex v in V − S do D(v) := min(D(v),D(w) + C(w, v)) end end

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Notes on Dijkstra’s Algorithm

• Dijkstra’s algorithm is greedy (“greed” pays oﬀ in this case). • The algorithm maintains a set S of vertices whose shortest distance from the source (node 1) is known. • C is a two dimensional array giving the weights of the edges. • At each step of the algorithm, D(i) contains the length of the shortest path from the source to node i that goes only through the nodes in S. • When the algorithm ﬁnishes, D(i) contains the length of the shortest path from the source to node i.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Example Graph - Revisited

1 10 100

2 30 5

10 50 60

3 4 20 & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Applying Dijkstra’s Algorithm

Iteration S w D(2) D(3) D(4) D(5) 1 { 1 } - 10 ∞ 30 100 2 { 1, 2} 2 10 60 30 100 3 { 1, 2, 4} 4 10 50 30 90 4 { 1, 2, 3, 4 } 5 10 50 30 60

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $ Dijkstra’s Algorithm vs. Branch-and-Bound

The way branch-and-bound works on the above shortest path problem is very similar to Dijkstra’s algorithm. But notice the differences: • Dijkstra’s algorithm computes the shortest path from the source to all nodes. • Branch-and-bound proves that the solution is optimal by making sure that no part of the search space remains that deserves to be explored. For Dijkstra’s algorithm, we prove this when we show correctness. • In the branch-and-bound case we may have several nodes of the same state, instead of several cost updates of a state that we may have in Dijkstra’s algorithm. • The time and space complexity is different because the parameters of interest are different. & % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Notes on Branch-and-Bound (cont’d)

If we always choose as branching node the one with the shortest partial path (as in the previous ﬁgures), we have an algorithm similar to UCS (or A∗ with h = 0).

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Example: k-way Number Partitioning (kNUMP)

Deﬁnition. Let S be a ﬁnite bag (multi-set) of positive integers.

Partition S into k bags A1,...,Ak ⊆ S so as to minimize the following diﬀerence: X X ∆(A1,...,Ak) = max x − min x i i x∈Ai x∈Ai

Let us deal with 2NUMP for simplicity. Example: How do we partition the bag of numbers {8, 7, 6, 5, 4} into two bags so that the diﬀerence of the numbers in these two bags is minimized?

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

A Greedy Algorithm

Order the given numbers in descending order. Initially, the two subsets are empty. Then, repeatedly take the next input number and assign it to the subset with the smallest sum so far.

For the input set {8, 7, 6, 5, 4}, this greedy algorithm will return the subsets {8, 5} and {7, 6, 4} with diﬀerence 4 which is not optimal. The optimal diﬀerence is 0.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Search Tree for Branch-and-Bound

• Feasible solutions: set of partitions • Each level of the search tree corresponds to choosing a number from the (sorted) input bag and putting it into one of the output bags. At each node of the search tree, we branch depending on which output bag the number will go.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Search Tree (cont’d)

{}{}

{8} {}

{8} {7} {8,7} {}

{8} {7,6} {8,6} {7} {8,7} {6}

{8,5} {7,6} {8,6} {7,5} {8,7} {6,5}

{8,5,4} {7,6} {8,6} {7,5,4} {8,7} {6,5,4}

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Executing the Search Algorithm

• We can search the tree in a DFS fashion. • We can order the search by always trying first the branch where the next number is put in the smallest subset. • This search strategy will actually return the greedy solution first. We can call this algorithm anytime: it starts with a greedy solution, and given more time, it improves it until it finds and proves the optimal solution.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Pruning the Search Tree

• Let us assume that we are at a search tree node n with current subset sum diﬀerence d0, sum of remaining numbers s and d0 ≥ s. Then we can add the remaining numbers in the smaller sum and not explore any other possibilities. This pruning does not depend on previously found solutions.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Pruning (cont’d)

• If at any point in the search we find a perfect partition then we terminate the search. A partition will be called perfect if it gives difference 0 when the sum of the given numbers is even and 1 if the sum of the given numbers is odd. The difference corresponding to a perfect partition is a lower-bound on any other possible difference.

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Pruning (cont’d)

• The ﬁrst number should be assigned only to one subset. • The last number should only be assigned to the smallest subset. • When the two current subsets have equal sums, the next number should only be assigned to one subset. Note: The above algorithm did not follow exactly the algorithm BranchAndBound given earlier (so that we could do more pruning). Thus, branch-and-bound should be understood to be a family of algorithms with the features we presented (as opposed to a single ﬁxed algorithm).

& % US02 Teqnht NohmosÔnh M. Koumparkhc ' $

Readings

• Chapter 4 of AIMA (Sections 4.1 and 4.2). • Section 18.2 from Christos Papadimitriou and Kenneth H. Steiglitz. Combinatorial Optimization - Algorithms and Complexity. Prentice-Hall, 1982.

& %