INFOB2KI 2019-2020 Utrecht University The Netherlands

ARTIFICIAL INTELLIGENCE

Informed search

Lecturer: Silja Renooij

These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html Shakey (1966-1972)

Shakey is a robot which navigates using?

a) Dijkstra’s algorithm b) A1 c) A2 d) A*

2 Recap: Search

. Search problem: . States (configurations of the world) . Actions and costs . Successor function . Start state and goal test

. Search tree: . Nodes represent how to reach states . Cost of reaching state = sum of action costs

. Search algorithm: . Systematically builds a search tree . Chooses an ordering of the fringe . Optimal: find least‐cost plans

3 Search Heuristics

A heuristic is: . a function that estimates how close a state is to a goal . non‐trivial (trivial == no real further search required) . designed for a particular search problem

Examples:

. Manhatten distance . Euclidean distance

4 Search Heuristics

A heuristic is:

. easy to compute, ‘on the fly’: otherwise overhead of computing the heuristic could outweigh time saved by reducing search!

Why not pre‐compute and store? . huge state space . dynamic goal state

5 Heuristic search (outline)

. Best‐first search –GREEDY SEARCH –A* search

. Heuristic functions

. Local search algorithms – Hill‐climbing search – Simulated annealing search – Local beam search – Genetic algorithms

6 Best-first search

. TREE/GRAPH‐SEARCH with an evaluation function f(n) for each node n – estimate of "desirability"  Expand most desirable unexpanded node  Should direct search toward goal

. Implementation: Order the nodes in fringe in decreasing order of desirability f(n)  Finds best solution, according to evaluation function

Special cases: Both also use heuristic h(n) with –GREEDY SEARCH • –A* search h(n) ≥ 0 for all n • h(n) = 0 for state[n] = goal 7 Romania with step costs in km

and straight‐line distances between city map‐coordinates 8 GREEDY SEARCH . Evaluation function f(n) = h(n) where –heuristic h(n) = estimated cost from n to goal

. e.g., hSLD(n) = straight‐line distance from n to Bucharest

Note: hSLD(n) ≥ 0 for all n; hSLD(Bucharest) = 0

. GREEDY SEARCH expands the node that appears to be closest to goal . Goal‐test: upon generation == upon expansion 9 GREEDY SEARCH example

f(Arad) = hsld(Arad) = 366

 Expand Arad

10 GREEDY SEARCH example

11 GREEDY SEARCH example

12 GREEDY SEARCH example

Done, or continue? Resembles DFS? 13 Properties of GREEDY SEARCH

. Complete? TREE‐SEARCH: No; 1 GRAPH‐SEARCH: Yes (in finite state spaces) . Optimal? No

. Time? O(bm) (but a good heuristic can give dramatic improvement)

. Space? O(bm) (keeps all nodes in memory)

Properties similar to DFS without space benefit; behaviour not necessarily

1 consider e.g. finding a path from Neamt to Fagaras with SLD heuristic, then Iasi  Neamt  Iasi  Neamt … since from Iasi neighbour Neamt is closer to Fagaras than neighbour Vaslui

14 GREEDY SEARCH

Note(!) GREEDY SEARCH is a variant of best‐first search; it is indeed a very greedy search algorithm, but not every greedy approach to searching is the same as GREEDY SEARCH.

15 A* search

. Combines benefits of greedy best‐first search and uniform‐cost search

. Evaluation function f(n) = g(n) + h(n) where – g(n) = cost so far to reach n (= path cost; dynamic) – h(n) = estimated cost from n to goal (= static heuristic)  f(n) = estimated total cost of path through n to goal

. A* search avoids expanding paths that are already expensive

. Goal‐test upon expansion!

16 A* search example

f(Arad) = g(Arad) + hsld(Arad) = 0 + 366

 Expand Arad

17 A* search example

min g(n)

Expand n with minimal f(n)

18 A* search example

min g(n)

Expand n with minimal f(n)

19 A* search example

min g(n)

Expand n with minimal f(n)

20 A* search example

min g(n)

Done, or continue?

For Bucharest: f(n) <> min f(n) g(n) <> min g(n) h(n) = 0

21 A* search example

min g(n)

Done, or continue? For Bucharest: f(n) = min f(n) g(n) <> min g(n) h(n) = 0

A* vs Dijkstra: https://www.youtube.com/watch?v=g024lzsknDo 22 Admissible heuristics . Definition: a heuristic h(n) is admissible if

h(n) ≤ h*(n) for every node n

where h*(n) is the optimal heuristic, i.e. gives true cost to reach the goal state from n.

. An admissible heuristic never overestimates the actual cost to reach the goal, i.e., it is optimistic (property transfers to f(n))

. Example: hSLD(n) (never overestimates the actual road distance)

. Theorem: If h(n) is admissible then A* using TREE‐SEARCH is optimal

23 Optimality of A* (proof tree)

Consider: . Optimal goal G . A fringe with (generated, but not yet expanded):

– suboptimal goal G2 –noden, on a shortest path to G

To prove:

G2 will never be expanded before n.

First we show that f(G2) > f(G): 1) f(G) = g(G) since h(G) = 0

2) f(G2) = g(G2) since h(G2) = 0

3) g(G2) > g(G) G2 suboptimal, so higher (path) costs

 f(G2) > f(G) from 1—3) 24 Optimality of A* (proof-cntd) Recall:

G optimal; fringe with suboptimal G2 and n on shortest path to G

To prove: G2 will never be expanded before n.

Established: f(G2) > f(G)

Next we show that f(G2) > f(n): 1) h(n) ≤ h*(n) assumption h is admissible ↔ h(n) + g(n) ≤ h*(n) + g(n) add g(n) to both sides ↔ f(n) ≤ g(G) n on shortest path to G; def f 2) f(G) = g(G) + 0 h(G) = 0; def f  f(G) ≥ f(n) 1,2

* Hence f(G2) > f(n), and A will never select G2 for expansion 25 Consistent heuristics . A heuristic is consistent (or monotonic) if for every node n and every successor n' of n, generated by any action a :

h(n) ≤ c(n,a,n') + h(n') where c(n,a,n') is the step cost

. If h is consistent, we have

f(n') = g(n') + h(n') (def) = g(n) + c(n,a,n') + h(n') ≥ g(n) + h(n) (consistent) = f(n) (def)

i.e., f(n) is non‐decreasing along any path. Intuition: h should be optimistic, but not to the extent that total costs f can drop by taking a next step 26 Optimality of A* (again) Theorem: If h(n) is consistent, A* using GRAPH‐SEARCH is optimal

. A* expands nodes in order of increasing f value . Gradually adds "f‐contours" of nodes

. Contour fc has all nodes n with f(n) ≤ c

Contours

f380 f400 f420

27 Properties of A*

. Complete? Yes (unless there are infinitely many nodes with f ≤ f(G) = f *)

. Optimal? Yes (if heuristic is consistent (GRAPH) /admissible (TREE) )

. Time? Exponential in d

. Space? Keeps all nodes in memory

28 Admissible heuristics Recall the properties of (admissible) heuristic h: . h(goal node) = 0 . 0 ≤ h(n) ≤ h*(n) for every node n (admissible: should never overestimate!)

E.g., two common heuristics for the 8‐puzzle:

. h1(n) = number of misplaced tiles . h2(n) = total Manhattan distance (i.e., no. of squares from desired location of each tile (only horizontal and vertical moves!!)))

. h1(Start) = ? . h2(Start) = ?

29 Admissible heuristics

E.g., two common heuristics for the 8‐puzzle:

. h1(n) = number of misplaced tiles . h2(n) = total Manhattan distance

. h1(Start) = 8 . h2(Start) = 3+1+2+2+2+3+3+2 = 18

30 Which is better? - Dominance

. If h2(n) ≥ h1(n) for all n (and both admissible) then h2 dominates h1

 h2 is better for search

. Typical search costs (average number of nodes expanded over 100 instances of 8‐puzzle per solution length d ): . d=12 IDS = 3,644,035 nodes * A (h1) = 227 nodes * A (h2) = 73 nodes . d=24 IDS = too many nodes * A (h1) = 39,135 nodes * A (h2) = 1,641 nodes 31 Good heuristics

. h*(n) ≥ h(n) ≥ 0 for all n (h is admissible)

. The closer h is to h*, the less nodes are kept open for search (more efficient)

. If h(n)=h(n)* for all n then A* leads direct to optimal solution without search

. If h(n)=0 for all n then A* is uniform cost search (= Dijkstra’s algorithm + goal test)

. However, if h is not admissible then optimal solution can be missed!

32 Relaxed problems

Relaxed problem: a problem with fewer restrictions on the actions E.g. allow moves to occupied slots in 8‐puzzle

The cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem

If the rules of the 8‐puzzle are further relaxed so that . a tile can move anywhere in one step

 h1(n) gives the best (shortest) solution . a tile can move to any adjacent square

 h2(n) gives the shortest solution 33 Variations on A*

. Hierarchical A* . IDA* (Iterative Deepening A*) . SMA* (Simplified Memory‐bounded A*)

Specific for dynamic environments: . LPA* (Lifelong Planning A*) . D* (A* for dynamic goals)

34 Summary best-first search

. Similar to uninformed search: . TREE/GRAPH‐SEARCH . goal‐test . step & path costs . solution = (least‐cost) path through state‐ space

. more problem specific ingredients (beyond definition of problem):

. Heuristic function h(n)  Evaluation function f(n) 35 Local search algorithms

In many optimization problems, the path to the goal is irrelevant; the goal state itself is the solution  State space = set of "complete‐state” configurations (rather than partial/incremental)

 Search: find configuration satisfying constraints

36 Example: TSP

BC

A

E D

Which is the shortest route that visits all cities exactly once, and returns at the starting point?

37 Local search algorithms

In many optimization problems, the path to the goal is irrelevant; the goal state itself is the solution  State space = set of "complete‐state” configurations (rather than partial/incremental)

 Search: find configuration satisfying constraints e.g. n‐queens In such cases, we can use local search algorithms  keep a single "current" state, and try to improve it by moving to neighboring states  find goal state, or state that optimizes some objective function (an evaluation function)

38 8-queens problem

. Evaluation function f: # of pairs of queens that attack each other, either directly or indirectly . Objective: f=0 (minimization)

• State s, with f(s) = 17

• neighbour/successor state of s: any state n where 1 queen is moved to different position in her own column

• f(n) shown in squares

39 Hill-climbing search

. No search tree . Terminates when it reaches “peak” (steepest‐ascent for maximization; for minimization use gradient descent version)

. No look‐ahead beyond immediate neighbors (greedy) "Like climbing Everest in thick fog with amnesia"

VALUE = value from objective function 40 Hill-climbing search

. Problem: depending on initial state, can easily get stuck in local optima .

41 Hill-climbing: 8-queens problem

. A local minimum with f = 1 (every neighbor has higher cost)

42 Simulated annealing search . Combines hill‐climbing with random walk . Idea: escape local max/min by allowing some “random" moves (‘shake’) but gradually decrease their frequency and ‘intensity’

43 Simulated annealing search

# T decreases over time!

# instead of best

# go if better # possibly try worse anyway

. Property: if T decreases slowly enough, a global optimum is found with probability approaching 1

. Widely used in VLSI layout, airline scheduling, etc 44 Local beam search

. Keep track of k states in memory rather than just one

. Start with k randomly generated states

. At each iteration, all the successors of all k states are generated

. If any one is a goal state, stop; else select the k best successors from the complete list and repeat.

“Come on over here, the grass is greener!”

45 Genetic algorithms

. A successor state is generated by combining two parent states

. Start with k randomly generated states (population)

. A state (or, individual) is represented as a string over a finite alphabet (often a string of 0s and 1s)

. Objective function (or, fitness function): higher values for better states.

. Produce the next generation of states by selection, crossover, and mutation

46 Summary Informed Search

. Use domain knowledge to guide search towards goal. . If path to goal is relevant: – best first search . If only finding the goal is relevant: – local search

. What about ?: search vs optimization vs learning

47 Shakey (1966-1972)

Shakey is a robot which navigates using?

a) Dijkstra’s algorithm b) A1 c) A2 d) A*

48