Problem-solving agents

function Simple-Problem-Solving-Agent( percept) returns an action static: seq, an action sequence, initially empty state, some description of the current world state Solving Problems by Searching goal, a goal, initially null problem, a problem formulation state ← Update-State(state, percept) if seq is empty then Chapter 3 goal ← Formulate-Goal(state) problem ← Formulate-Problem(state, goal) seq ← Search( problem) if seq is failure then return a null action action ← First(seq) seq ← Rest(seq) return action

Note: this is offline problem solving; solution executed “eyes closed.” Online problem solving involves acting without complete knowledge.

Chapter 3 1 Chapter 3 3

Outline Example: Romania

♦ Problem-solving agents On holiday in Romania; currently in Arad. Flight leaves tomorrow from Bucharest ♦ Problem types Formulate goal: ♦ Problem formulation be in Bucharest ♦ Example problems Formulate problem: ♦ Basic search algorithms states: various cities actions: drive between cities Find solution: sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest

Chapter 3 2 Chapter 3 4 Example: Romania Example: vacuum world Oradea 71 Neamt Single-state, start in #5. Solution?? 1 2 Zerind 87 75 151 Iasi 3 4 Arad 140 92 Sibiu 99 Fagaras 5 6 118 Vaslui 80 Timisoara Rimnicu Vilcea 7 8 142 111 Pitesti 211 Lugoj 97 70 98 85 Hirsova Mehadia 146 101 Urziceni 75 138 86 Bucharest Dobreta 120 90 Craiova Eforie Giurgiu

Chapter 3 5 Chapter 3 7

Problem types Example: vacuum world

Deterministic, fully observable =⇒ single-state problem Single-state, start in #5. Solution?? Agent knows exactly which state it will be in; solution is a sequence [Right, Suck] 1 2 Non-observable =⇒ conformant problem Conformant, start in {1, 2, 3, 4, 5, 6, 7, 8} 3 4 Agent may have no idea where it is; solution (if any) is a sequence e.g., Right goes to {2, 4, 6, 8}. Solution??

Nondeterministic and/or partially observable =⇒ contingency problem 5 6 percepts provide new information about current state

solution is a contingent plan or a policy 7 8 often interleave search, execution Unknown state space =⇒ exploration problem (“online”)

Chapter 3 6 Chapter 3 8 Example: vacuum world Single-state problem formulation

Single-state, start in #5. Solution?? A problem is defined by four items: [Right, Suck] 1 2 ♦ initial state e.g., “at Arad” Conformant, start in {1, 2, 3, 4, 5, 6, 7, 8} 3 4 e.g., Right goes to {2, 4, 6, 8}. Solution?? ♦ successor function S(x) = set of action–state pairs [Right,Suck,Left,Suck] e.g., S(Arad)= {hArad → Zerind, Zerindi,...} 5 6 Contingency ♦ goal test, can be ♦ Murphy’s Law (non-deterministic): Suck explicit, e.g., x = “at Bucharest” 7 8 can dirty a clean carpet; start in #5 implicit, e.g., NoDirt(x) ♦ Local sensing (partially-observable): dirt, ♦ path cost (additive) location only, start in {#5,#7}. e.g., sum of distances, number of actions executed, etc. Solution?? c(x, a, y) is the step cost, assumed to be ≥ 0 A solution is a sequence of actions leading from the initial state to a goal state

Chapter 3 9 Chapter 3 11

Example: vacuum world Selecting a state space

Single-state, start in #5. Solution?? Real world is absurdly complex [Right, Suck] 1 2 ⇒ state space must be abstracted for problem solving Conformant, start in {1, 2, 3, 4, 5, 6, 7, 8} ♦ (Abstract) state = set of real states 3 4 e.g., Right goes to {2, 4, 6, 8}. Solution?? [Right,Suck,Left,Suck] ♦ (Abstract) action = complex combination of real actions 5 6 e.g., “Arad → Zerind” represents a complex set Contingency of possible routes, detours, rest stops, etc. ♦ Murphy’s Law (non-deterministic): Suck 7 8 can dirty a clean carpet; start in #5 ♦ Local sensing (partially-observable): dirt, ♦ For guaranteed realizability, any real state “in Arad” location only, start in {#5,#7}. must get to some real state “in Zerind” Solution?? ♦ (Abstract) solution = [Right, while dirt do Suck] set of real paths that are solutions in the real world [Right, if dirt then Suck] ♦ Each abstract action should be “easier” than the original problem!

Chapter 3 10 Chapter 3 12 Example: vacuum world state space graph Example: vacuum world state space graph

R R L R L R

L L

S S S S

R R R R L R L R L R L R

L L L L SS SS S S S S

R R L R L R

L L

S S S S states?? states??: integer dirt and robot locations (ignore dirt amounts etc.) actions?? actions??: Left, Right, Suck, NoOp goal test?? goal test?? path cost?? path cost??

Chapter 3 13 Chapter 3 15

Example: vacuum world state space graph Example: vacuum world state space graph

R R L R L R

L L

S S S S

R R R R L R L R L R L R

L L L L SS SS S S S S

R R L R L R

L L

S S S S states??: integer dirt and robot locations (ignore dirt amounts etc.) states??: integer dirt and robot locations (ignore dirt amounts etc.) actions?? actions??: Left, Right, Suck, NoOp goal test?? goal test??: no dirt path cost?? path cost??

Chapter 3 14 Chapter 3 16 Example: vacuum world state space graph Example: The 8-puzzle

R L R 7 2 4 51 2 3 L

S S

R R 5 6 4 5 6 L R L R

L L SS 8 3 1 7 8 S S

R L R Start State Goal State

L

S S states??: integer locations of tiles (ignore intermediate positions) actions?? states??: integer dirt and robot locations (ignore dirt amounts etc.) goal test?? actions??: Left, Right, Suck, NoOp path cost?? goal test??: no dirt path cost??: 1 per action (0 for NoOp)

Chapter 3 17 Chapter 3 19

Example: The 8-puzzle Example: The 8-puzzle

7 2 4 51 2 3 7 2 4 51 2 3

5 6 4 5 6 5 6 4 5 6

8 3 1 7 8 8 3 1 7 8

Start State Goal State Start State Goal State states?? states??: integer locations of tiles (ignore intermediate positions) actions?? actions??: move blank left, right, up, down (ignore unjamming etc.) goal test?? goal test?? path cost?? path cost??

Chapter 3 18 Chapter 3 20 Example: The 8-puzzle Example: robotic assembly

P 7 2 4 51 2 3 R R

R R 5 6 4 5 6

R 8 3 1 7 8

Start State Goal State states??: real-valued coordinates of robot joint angles states??: integer locations of tiles (ignore intermediate positions) parts of the object to be assembled actions??: move blank left, right, up, down (ignore unjamming etc.) goal test??: = goal state (given) actions??: continuous motions of robot joints path cost?? goal test??: complete assembly with no robot included! path cost??: time to execute

Chapter 3 21 Chapter 3 23

Example: The 8-puzzle Tree search algorithms

7 2 4 51 2 3 Basic idea: offline, simulated exploration of state space 5 6 4 5 6 by generating successors of already-explored states (a.k.a. expanding states) 8 3 1 7 8 function Tree-Search( problem, strategy) returns a solution, or failure problem Start State Goal State initialize the frontier using the initial state of loop do states??: integer locations of tiles (ignore intermediate positions) if the frontier is empty then return failure choose a leaf node and remove it from the frontier based on strategy actions??: move blank left, right, up, down (ignore unjamming etc.) if the node contains a goal state then return the corresponding solution goal test??: = goal state (given) else expand the chosen node and add the resulting nodes to the frontier path cost??: 1 per move end [Note: optimal solution of n-Puzzle family is NP-hard]

Chapter 3 22 Chapter 3 24 Tree search example Tree search example

Arad Arad

Sibiu Timisoara Zerind Sibiu Timisoara Zerind

Arad Fagaras Oradea Rimnicu Vilcea Arad Lugoj Arad Oradea Arad Fagaras Oradea Rimnicu Vilcea Arad Lugoj Arad Oradea

Chapter 3 25 Chapter 3 27

Tree search example Implementation: states vs. nodes

Arad A state is a (representation of) a physical configuration A node is a data structure constituting part of a search tree includes parent, children, depth, path cost g(x) Sibiu Timisoara Zerind States do not have parents, children, depth, or path cost! parent, action

Arad Fagaras Oradea Rimnicu Vilcea Arad Lugoj Arad Oradea

depth = 6 State5 4 Node g = 6 6 1 88 state 7 3 22

The Expand function creates new nodes, filling in the various fields and using the SuccessorFn of the problem to create the corresponding states.

Chapter 3 26 Chapter 3 28 Search strategies Breadth-first search

A strategy is defined by picking the order of node expansion Expand shallowest unexpanded node Strategies are evaluated along the following dimensions: Implementation: completeness—does it always find a solution if one exists? frontier is a FIFO queue, i.e., new successors go at end time complexity—number of nodes generated/expanded A space complexity—maximum number of nodes in memory optimality—does it always find a least-cost solution? Time and space complexity are measured in terms of B C b—maximum branching factor of the search tree d—depth of the “shallowest” solution m—maximum depth of the state space (may be ∞) D E F G

Chapter 3 29 Chapter 3 31

Uninformed search strategies Breadth-first search

Uninformed strategies use only the information available Expand shallowest unexpanded node in the problem definition Implementation: ♦ Breadth-first search frontier is a FIFO queue, i.e., new successors go at end ♦ Uniform-cost search A ♦ Depth-first search B C ♦ Depth-limited search ♦ Iterative deepening search D E F G

Chapter 3 30 Chapter 3 32 Breadth-first search Properties of breadth-first search

Expand shallowest unexpanded node Complete?? Implementation: frontier is a FIFO queue, i.e., new successors go at end A

B C

D E F G

Chapter 3 33 Chapter 3 35

Breadth-first search Properties of breadth-first search

Expand shallowest unexpanded node Complete?? Yes (if b is finite) Implementation: Time?? frontier is a FIFO queue, i.e., new successors go at end A

B C

D E F G

Chapter 3 34 Chapter 3 36 Properties of breadth-first search Properties of breadth-first search

Complete?? Yes (if b is finite) Complete?? Yes (if b is finite) Time (# of visited nodes)?? 1+ b + b2 + b3 + ... + bd = O(bd) Time (# of generated nodes)?? b+b2 +b3 +...+bd +(bd+1 −b)= O(bd+1) Time (# of generated nodes)?? b+b2 +b3 +...+bd +(bd+1 −b)= O(bd+1) Space?? O(bd+1) (keeps every node in memory) Space?? Optimal?? Yes (if cost = 1 per step); not optimal in general Space is the big problem; can easily generate nodes at 100MB/sec so 24hrs = 8640GB.

Chapter 3 37 Chapter 3 39

Properties of breadth-first search Uniform-cost search

Complete?? Yes (if b is finite) Expand least-cost unexpanded node Time (# of generated nodes)?? b+b2 +b3 +...+bd +(bd+1 −b)= O(bd+1) Implementation: frontier = queue ordered by path cost, lowest first Space?? O(bd+1) (keeps every node in memory) Equivalent to breadth-first if step costs all equal Optimal?? Complete?? Yes, if step cost ≥ ǫ (lowest step cost)

∗ Time?? # of nodes with g ≤ cost of optimal solution, O(b⌈C /ǫ⌉) where C∗ is the cost of the optimal solution

∗ Space?? # of nodes with g ≤ cost of optimal solution, O(b⌈C /ǫ⌉) Optimal?? Yes—nodes expanded in increasing order of g(n)

Chapter 3 38 Chapter 3 40 Depth-first search Depth-first search

Expand deepest unexpanded node Expand deepest unexpanded node Implementation: Implementation: frontier = LIFO queue, i.e., put successors at front frontier = LIFO queue, i.e., put successors at front A A

B C B C

D E F G D E F G

H I J K L M N O H I J K L M N O

Chapter 3 41 Chapter 3 43

Depth-first search Depth-first search

Expand deepest unexpanded node Expand deepest unexpanded node Implementation: Implementation: frontier = LIFO queue, i.e., put successors at front frontier = LIFO queue, i.e., put successors at front A A

B C B C

D E F G D E F G

H I J K L M N O H I J K L M N O

Chapter 3 42 Chapter 3 44 Depth-first search Depth-first search

Expand deepest unexpanded node Expand deepest unexpanded node Implementation: Implementation: frontier = LIFO queue, i.e., put successors at front frontier = LIFO queue, i.e., put successors at front A A

B C B C

D E F G D E F G

H I J K L M N O H I J K L M N O

Chapter 3 45 Chapter 3 47

Depth-first search Depth-first search

Expand deepest unexpanded node Expand deepest unexpanded node Implementation: Implementation: frontier = LIFO queue, i.e., put successors at front frontier = LIFO queue, i.e., put successors at front A A

B C B C

D E F G D E F G

H I J K L M N O H I J K L M N O

Chapter 3 46 Chapter 3 48 Depth-first search Depth-first search

Expand deepest unexpanded node Expand deepest unexpanded node Implementation: Implementation: frontier = LIFO queue, i.e., put successors at front frontier = LIFO queue, i.e., put successors at front A A

B C B C

D E F G D E F G

H I J K L M N O H I J K L M N O

Chapter 3 49 Chapter 3 51

Depth-first search Depth-first search

Expand deepest unexpanded node Expand deepest unexpanded node Implementation: Implementation: frontier = LIFO queue, i.e., put successors at front frontier = LIFO queue, i.e., put successors at front A A

B C B C

D E F G D E F G

H I J K L M N O H I J K L M N O

Chapter 3 50 Chapter 3 52 Properties of depth-first search Properties of depth-first search

Complete?? Complete?? Yes: in finite spaces No: fails in infinite-depth spaces, spaces with loops Modify to avoid repeated states along path

Time?? O(bm): terrible if m is much larger than d but if solutions are dense, may be much faster than breadth-first Space??

Chapter 3 53 Chapter 3 55

Properties of depth-first search Properties of depth-first search

Complete?? Yes: in finite spaces Complete?? Yes: in finite spaces No: fails in infinite-depth spaces, spaces with loops No: fails in infinite-depth spaces, spaces with loops Modify to avoid repeated states along path Modify to avoid repeated states along path

Time?? Time?? O(bm): terrible if m is much larger than d but if solutions are dense, may be much faster than breadth-first Space?? O(bm), i.e., linear space! Optimal??

Chapter 3 54 Chapter 3 56 Properties of depth-first search Iterative deepening search

Complete?? Yes: in finite spaces function Iterative-Deepening-Search( problem) returns a solution No: fails in infinite-depth spaces, spaces with loops inputs: problem, a problem Modify to avoid repeated states along path for depth ← 0 to ∞ do result ← Depth-Limited-Search( problem, depth) Time?? O(bm): terrible if m is much larger than d if result 6= cutoff then return result end but if solutions are dense, may be much faster than breadth-first Space?? O(bm), i.e., linear space! Optimal?? No

Chapter 3 57 Chapter 3 59

Depth-limited search Iterative deepening search l =0

= depth-first search with depth limit l, Limit = 0 A A i.e., nodes at depth l have no successors Recursive implementation:

function Depth-Limited-Search( problem, limit) returns soln/fail/cutoff Recursive-DLS(Make-Node(Initial-State[problem]), problem, limit) function Recursive-DLS(node, problem, limit) returns soln/fail/cutoff cutoff-occurred? ← false if Goal-Test(problem, State[node]) then return node else if Depth[node] = limit then return cutoff else for each successor in Expand(node, problem) do result ← Recursive-DLS(successor, problem, limit) if result = cutoff then cutoff-occurred? ← true else if result 6= failure then return result if cutoff-occurred? then return cutoff else return failure

Chapter 3 58 Chapter 3 60 Iterative deepening search l =1 Iterative deepening search l =3

Limit = 1 A A A A Limit = 3 A A A A

B C B C B C B C B C B C B C B C

D E F G D E F G D E F G D E F G

H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O

A A A A

B C B C B C B C

D E F G D E F G D E F G D E F G

H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O

A A A A

B C B C B C B C

D E F G D E F G D E F G D E F G

H I J K L M N O H I J K L M N O H I J K L M N O H I J K L M N O

Chapter 3 61 Chapter 3 63

Iterative deepening search l =2 Properties of iterative deepening search

Complete?? Limit = 2 A A A A

B C B C B C B C

D E F G D E F G D E F G D E F G

A A A A

B C B C B C B C

D E F G D E F G D E F G D E F G

Chapter 3 62 Chapter 3 64 Properties of iterative deepening search Properties of iterative deepening search

Complete?? Yes Complete?? Yes Time?? Time (# of generated nodes)?? db1 +(d − 1)b2 + ... + bd = O(bd) Space?? O(bd) Optimal??

Chapter 3 65 Chapter 3 67

Properties of iterative deepening search Properties of iterative deepening search

Complete?? Yes Complete?? Yes Time (# of generated nodes)?? db1 +(d − 1)b2 + ... + bd = O(bd) Time (# of generated nodes)?? db1 +(d − 1)b2 + ... + bd = O(bd) Space?? Space?? O(bd) Optimal?? Yes, if step cost = 1 Can be modified to explore uniform-cost tree Numerical comparison for b = 10 and d =5, solution at far right leaf: N(IDS) = 50+400+3, 000 + 20, 000 + 100, 000 = 123, 450

Nvisited(BFS) = 10+100+1, 000 + 10, 000 + 100, 000 = 111, 110 Ngenerated(BFS) = 10+100+1, 000 + 10, 000 + 100, 000 + 999, 990 = 1, 111, 100

BFS can be modified to apply goal test when a node is generated

Chapter 3 66 Chapter 3 68 Summary of algorithms Graph search

Criterion Breadth- Uniform- Depth- Depth- Iterative function Graph-Search( problem) returns a solution, or failure First Cost First Limited Deepening frontier ← a list with node from the initial state of problem Complete? Yes Yes No Yes,if l ≥ d Yes explored ← an empty set ∗ Time (big-O) bd b⌈C /ǫ⌉ bm bl bd loop do d ⌈C∗/ǫ⌉ if frontier is empty then return failure Space (big-O) b b bm bl bd Remove-Front ∗ ∗ node ← (frontier) Optimal? Yes Yes No No Yes if node contains a goal state then return Solution(node) add State[node] to explored expand node add to frontier the resulting nodes that are not in explored or not in frontier or [better than the corresponding nodes in frontier in some algs] end

frontier (aka fringe or open); explored (aka visited or closed)

Chapter 3 69 Chapter 3 71

Repeated states Informed Search

Failure to detect repeated states can turn a linear problem into an exponential ♦ So far the search algorithms are “uninformed”—independent to the prob- one! lems A A ♦ Informed search–incorporating knowledge related to the problem for guid- ing search B B B

C C C C C

D

Chapter 3 70 Chapter 3 72 Best-first search Greedy search

Idea: use an evaluation function for each node Evaluation function h(n) (heuristic) – estimate of “desirability” = estimate of cost from n to the closest goal

⇒ Expand most desirable unexpanded node E.g., hSLD(n) = straight-line distance from n to Bucharest Implementation: Greedy search expands the node that appears to be closest to goal frontier is a queue sorted in decreasing order of desirability Special cases: greedy search A∗ search

Chapter 3 73 Chapter 3 75

Romania with step costs in km Greedy search example

Straight−line distance Oradea Arad 71 to Bucharest Neamt Arad 366 366 Bucharest 0 Zerind 87 75 151 Craiova 160 Iasi Dobreta 242 Arad 140 Eforie 161 92 Fagaras 178 Sibiu Fagaras 99 Giurgiu 77 118 Hirsova 151 80 Vaslui Iasi 226 Rimnicu Vilcea Timisoara Lugoj 244 142 Mehadia 241 111 211 Neamt 234 Lugoj 97 Pitesti Oradea 380 70 98 Pitesti 98 146 85 Hirsova Mehadia 101 Urziceni Rimnicu Vilcea 193 Sibiu 75 138 86 253 Bucharest Timisoara 120 329 Dobreta 90 Urziceni 80 Craiova Eforie Vaslui 199 Giurgiu Zerind 374

Chapter 3 74 Chapter 3 76 Greedy search example Greedy search example

Arad Arad

Sibiu Timisoara Zerind Sibiu Timisoara Zerind 253 329 374 329 374

Arad Fagaras Oradea Rimnicu Vilcea 366 380 193

Sibiu Bucharest 253 0

Chapter 3 77 Chapter 3 79

Greedy search example Properties of greedy search

Complete?? Arad

Sibiu Timisoara Zerind 329 374

Arad Fagaras Oradea Rimnicu Vilcea 366 176 380 193

Chapter 3 78 Chapter 3 80 Properties of greedy search Properties of greedy search

Complete?? Yes–Complete in finite space with repeated-state checking Complete?? Yes–Complete in finite space with repeated-state checking No–can get stuck in loops, e.g., with Oradea as goal, No–can get stuck in loops, e.g., Iasi → Neamt → Iasi → Neamt → Iasi → Neamt → Iasi → Neamt →

Time?? Time?? O(bm), but a good heuristic can give dramatic improvement Space?? O(bm), but a good heuristic can give dramatic improvement Optimal??

Chapter 3 81 Chapter 3 83

Properties of greedy search Properties of greedy search

Complete?? Yes–Complete in finite space with repeated-state checking Complete?? Yes–Complete in finite space with repeated-state checking No–can get stuck in loops, e.g., No–can get stuck in loops, e.g., Iasi → Neamt → Iasi → Neamt → Iasi → Neamt → Iasi → Neamt →

Time?? O(bm), but a good heuristic can give dramatic improvement Time?? O(bm), but a good heuristic can give dramatic improvement Space?? Space?? O(bm), but a good heuristic can give dramatic improvement Optimal?? No

Chapter 3 82 Chapter 3 84 A∗ search A∗ search example

Idea: avoid expanding paths that are already expensive Arad Evaluation function f(n)= g(n)+ h(n) g(n) = cost so far to reach n Sibiu Timisoara Zerind 393=140+253 447=118+329 449=75+374 h(n) = estimated cost to goal from n f(n) = estimated total cost of path through n to goal A∗ search uses an admissible heuristic i.e., h(n) ≤ h∗(n) where h∗(n) is the true cost from n. (Also require h(n) ≥ 0, so h(G)=0 for any goal G.)

E.g., hSLD(n) never overestimates the actual road distance Theorem: A∗ search is optimal

Chapter 3 85 Chapter 3 87

A∗ search example A∗ search example

Arad Arad 366=0+366

Sibiu Timisoara Zerind 447=118+329 449=75+374

Arad Fagaras Oradea Rimnicu Vilcea 646=280+366415=239+176 671=291+380 413=220+193

Chapter 3 86 Chapter 3 88 A∗ search example A∗ search example

Arad Arad

Sibiu Timisoara Zerind Sibiu Timisoara Zerind 447=118+329 449=75+374 447=118+329 449=75+374

Arad Fagaras Oradea Rimnicu Vilcea Arad Fagaras Oradea Rimnicu Vilcea 646=280+366 415=239+176 671=291+380 646=280+366 671=291+380

Craiova Pitesti Sibiu Sibiu Bucharest Craiova Pitesti Sibiu 526=366+160 417=317+100 553=300+253 591=338+253 450=450+0 526=366+160 553=300+253

Bucharest Craiova Rimnicu Vilcea 418=418+0 615=455+160 607=414+193

Chapter 3 89 Chapter 3 91

A∗ search example Optimality of A∗ (standard proof)

Suppose some suboptimal goal G2 has been generated and is in the queue. Arad Let n be an unexpanded node on a shortest path to an optimal goal G. Start

Sibiu Timisoara Zerind 447=118+329 449=75+374 n Arad Fagaras Oradea Rimnicu Vilcea 646=280+366 671=291+380 G G Sibiu Bucharest Craiova Pitesti Sibiu 2 591=338+253 450=450+0 526=366+160 417=317+100 553=300+253 ∗ Want to prove: f(G2) >f(n) [A will never select G2 for expansion]

f(G2) = g(G2) since h(G2)=0 g(G2) > g(G) since G2 is suboptimal g(G) ≥ f(n) since h is admissible

Chapter 3 90 Chapter 3 92 Properties of A∗ Properties of A∗

Complete?? Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G) Time?? Exponential in [relative error in h × d] Space??

Chapter 3 93 Chapter 3 95

Properties of A∗ Properties of A∗

Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G) Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G) Time?? Time?? Exponential in [relative error in h × d] Space?? Exponential Optimal??

Chapter 3 94 Chapter 3 96 Properties of A∗ A* vs Uniform-cost Search

Complete?? Yes, unless there are infinitely many nodes with f ≤ f(G) f(n)= g(n)+ h(n) Time?? Exponential in [relative error in h × d] Isn’t UCS just A* with h(n) being zero (admissible)? Space?? Exponential Both are optimal, why is A* usually “faster?”

Optimal?? Yes—cannot expand fi+1 until fi is finished, where fi+1 >fi Consider h(n) is perfect, f(n) is ? C∗ is the cost for the optimal solution: A∗ expands all nodes with f(n) C∗

Chapter 3 97 Chapter 3 99

A* vs Uniform-cost Search A* vs Uniform-cost Search f(n)= g(n)+ h(n) f(n)= g(n)+ h(n) Isn’t UCS just A* with h(n) being zero (admissible)? Isn’t UCS just A* with h(n) being zero (admissible)? Both are optimal, why is A* usually “faster?” Both are optimal, why is A* usually “faster?” Consider h(n) is perfect, f(n) is the actual total path cost.

Chapter 3 98 Chapter 3 100 A* vs Uniform-cost Search A* vs Uniform-cost Search f(n)= g(n)+ h(n) f(n)= g(n)+ h(n) Isn’t UCS just A* with h(n) being zero (admissible)? Isn’t UCS just A* with h(n) being zero (admissible)? Both are optimal, why is A* usually “faster?” Both are optimal, why is A* usually “faster?” Consider h(n) is perfect, f(n) is the actual total path cost. Consider h(n) is perfect, f(n) is the actual total path cost.

• If n doesn’t lead to a goal state, f(n)= ∞ • If n doesn’t lead to a goal state, f(n)= ∞ – A* ? – A* doesn’t explore n. – UCS ? – UCS doesn’t know and keeps on exploring n (and its sucessors). • If n doesn’t lead to the optimal goal, but n∗ does: f(n) >f(n∗) – A* doesn’t explore n and *only* explores n∗ ! – UCS doesn’t know and keeps on exploring n (and its sucessors).

In terms of speed, the worst case for A* is when h(n) is zero, but we don’t use h(n)=0.

Chapter 3 101 Chapter 3 103

A* vs Uniform-cost Search Consistency f(n)= g(n)+ h(n) Consider n′ is a successor of n, a heuristic is consistent if Isn’t UCS just A* with h(n) being zero (admissible)? h(n) ≤ c(n, a, n′)+ h(n′), n Both are optimal, why is A* usually “faster?” Let’s find the relationship between f(n) and f(n′): c(n,a,n') h(n) Consider h(n) is perfect, f(n) is the actual total path cost. f(n′) = g(n′)+ h(n′) n' f(n′) = g(n)+ c(n, a, n′)+ h(n′) • If n doesn’t lead to a goal state, f(n)= ∞ f(n′) ≥ g(n)+ h(n) [since h is consistent] h(n') – A* doesn’t explore n. f(n′) ≥ f(n) G – UCS doesn’t know and keeps on exploring n (and its sucessors). i.e. f(n) values for a sequence of nodes along *any* path are nondecreasing ∗ ∗ • If n doesn’t lead to the optimal goal, but n does: f(n) >f(n ) (similar to g(n) values in UCS). – A* ? A* using GRAPH-SEARCH is optimal if h(n) is consistent (using a similar – UCS ? argument as UCS).

Chapter 3 102 Chapter 3 104 Optimality of A∗ (consistent heuristics) Admissible heuristics

Lemma: A∗ expands nodes in order of increasing f value∗ E.g., for the 8-puzzle:

Gradually adds “f-contours” of nodes (cf. breadth-first adds layers) h1(n) = number of misplaced tiles Contour i has all nodes with f = fi, where fi

L P h1(S) =?? H M U h2(S) =?? B 420 D E C G

Chapter 3 105 Chapter 3 107

Admissible vs Consistent Heuristics Admissible heuristics

Consistency is a slightly stronger/stricter requirement than admissibility. E.g., for the 8-puzzle: consistentHeuristics ⊂ admissibleHeuristics h1(n) = number of misplaced tiles h2(n) = total Manhattan distance Admissible heuristics are usually consistent. (i.e., no. of squares from desired location of each tile) Not easy to concort admissible, but not consistent heuristics. 7 2 4 51 2 3

5 6 4 5 6

8 3 1 7 8

Start State Goal State

h1(S) =?? 6 h2(S) =?? 4+0+3+3+1+0+2+1 = 14

Chapter 3 106 Chapter 3 108 Dominance Relaxed problems contd.

If h2(n) ≥ h1(n) for all n (both admissible) Well-known example: travelling salesperson problem (TSP) then h2 dominates h1 and is faster for search Find the shortest tour visiting all cities exactly once Typical search costs: d = 14 IDS = 3,473,941 nodes ∗ A (h1) = 539 nodes ∗ A (h2) = 113 nodes d = 24 IDS ≈ 54,000,000,000 nodes ∗ A (h1) = 39,135 nodes ∗ A (h2) = 1,641 nodes

Given any admissible heuristics ha, hb, Minimum spanning tree can be computed in O(n2) and is a lower bound on the shortest (open) tour h(n) = max(ha(n),hb(n)) is also admissible and dominates ha, hb

Chapter 3 109 Chapter 3 111

Relaxed problems Summary

Admissible heuristics can be derived from the exact Problem formulation usually requires abstracting away real-world details to solution cost of a relaxed version of the problem define a state space that can feasibly be explored If the rules of the 8-puzzle are relaxed so that a tile can move anywhere, Variety of uninformed search strategies then h1(n) gives the shortest solution Iterative deepening search uses only linear space If the rules are relaxed so that a tile can move to any adjacent square, and not much more time than other uninformed algorithms then h2(n) gives the shortest solution Graph search can be exponentially more efficient than tree search Key point: the optimal solution cost of a relaxed problem is no greater than the optimal solution cost of the real problem

Chapter 3 110 Chapter 3 112 Summary

Heuristic functions estimate costs of shortest paths Good heuristics can dramatically reduce search cost Greedy best-first search expands lowest h – incomplete and not always optimal A∗ search expands lowest g + h – complete and optimal – also optimally efficient (up to tie-breaks, for forward search) Admissible heuristics can be derived from exact solution of relaxed problems

Chapter 3 113