Outline Artificial intelligence 1: Informed = use problem-specific knowledge informed search Which search strategies? – Best-first search and its variants Heuristic functions? – How to invent them Lecturer: Tom Lenaerts Local search and optimization SWITCH, Vlaams Interuniversitair Instituut voor Biotechnologie – Hill climbing, local beam search, genetic algorithms,… Local search in continuous spaces Online search agents AI 1 6 februari Pag.2 2008 Previously: tree-search Best-first search function TREE-SEARCH(problem,fringe) return a solution or failure General approach of informed search: fringe ← INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe) – Best-first search: node is selected for expansion based on loop do an evaluation function f(n) if EMPTY?(fringe) then return failure node ← REMOVE-FIRST(fringe) Idea: evaluation function measures distance if GOAL-TEST[problem] applied to STATE[node] succeeds to the goal. then return SOLUTION(node) – Choose node which appears best fringe ← INSERT-ALL(EXPAND(node, problem), fringe) Implementation: A strategy is defined by picking the order of – fringe is queue sorted in decreasing order of desirability. node expansion – Special cases: greedy search, A* search AI 1 AI 1 6 februari Pag.3 6 februari Pag.4 2008 2008 A heuristic function Romania with step costs in km [dictionary]“A rule of thumb, simplification, hSLD=straight-line or educated guess that reduces or limits the distance heuristic. hSLD can NOT be search for solutions in domains that are computed from the difficu lt and poor ly under stood. ” problem descr ipt ion – h(n) = estimated cost of the cheapest path from node n to itself goal node. In this example –If n is goal then h(n)=0 f(n)=h(n) – Expand node that is closest to goal More information later. = Greedy best-first search AI 1 AI 1 6 februari Pag.5 6 februari Pag.6 2008 2008 1 Greedy search example Greedy search example Arad Arad (366) Zerind(374) Sibiu(253) Timisoara (329) Assume that we want to use greedy search The first expansion step produces: to solve the problem of travelling from Arad – Sibiu, Timisoara and Zerind to Bucharest. Greedy best-first will select Sibiu. The initial state=Arad AI 1 AI 1 6 februari Pag.7 6 februari Pag.8 2008 2008 Greedy search example Greedy search example Arad Arad Sibiu Sibiu Fagaras Arad Rimnicu Vilcea Fagaras Oradea Sibiu Bucharest (366) (193) (176) (380) (253) (0) If Sibiu is expanded we get: If Fagaras is expanded we get: – Sibiu and Bucharest – Arad, Fagaras, Oradea and Rimnicu Vilcea Goal reached !! Greedy best-first search will select: Fagaras – Yet not optimal (see Arad, Sibiu, Rimnicu Vilcea, Pitesti) AI 1 AI 1 6 februari Pag.9 6 februari Pag.10 2008 2008 Greedy search, evaluation Greedy search, evaluation Completeness: NO (cfr. DF-search) Completeness: NO (cfr. DF-search) – Check on repeated states Time complexity? – Minimizing h(n) can result in false starts, e.g. Iasi to m – Cfr. Worst-case DF-search O(b ) Fagaras. (w it h m is maximum dept h of search space) – Good heuristic can give dramatic improvement. AI 1 AI 1 6 februari Pag.11 6 februari Pag.12 2008 2008 2 Greedy search, evaluation Greedy search, evaluation Completeness: NO (cfr. DF-search) Completeness: NO (cfr. DF-search) m Time complexity: O(bm ) Time complexity: O(b ) O(bm ) Space complexity:O(bm ) Space complexity: – Keeps all nodes in memory Optimality? NO – Same as DF-search AI 1 AI 1 6 februari Pag.13 6 februari Pag.14 2008 2008 A* search A* search Best-known form of best-first search. A* search uses an admissible heuristic Idea: avoid expanding paths that are already – A heuristic is admissible if it never overestimates the expensive. cost to reach the goal – Are optimistic Evaluation function f(n)=g(n) + h(n) – g(n) the cost (so far) to reach the node. – h(n) estimated cost to get from the node to the goal. Formally: 1. h(n) <= h*(n) where h*(n) is the true cost from n – f(n) estimated total cost of path through n to goal. 2. h(n) >= 0 so h(G)=0 for any goal G. e.g. hSLD(n) never overestimates the actual road distance AI 1 AI 1 6 februari Pag.15 6 februari Pag.16 2008 2008 Romania example A* search example Find Bucharest starting at Arad – f(Arad) = c(??,Arad)+h(Arad)=0+366=366 AI 1 AI 1 6 februari Pag.17 6 februari Pag.18 2008 2008 3 A* search example A* search example Expand Arrad and determine f(n) for each node Expand Sibiu and determine f(n) for each node – f(Sibiu)=c(Arad,Sibiu)+h(Sibiu)=140+253=393 – f(Arad)=c(Sibiu,Arad)+h(Arad)=280+366=646 – f(Fagaras)=c(Sibiu,Fagaras)+h(Fagaras)=239+179=415 – f(Timisoara)=c(Arad,Timisoara)+h(Timisoara)=118+329=447 – f(Oradea)=c(Sibiu,Oradea)+h(Oradea)=291+380=671 – f(Zerind)=c(Arad,Zerind)+h(Zerind)=75+374=449 – f(Rimnicu Vilcea)=c(Sibiu,Rimnicu Vilcea)+ Best choice is Sibiu h(Rimnicu Vilcea)=220+192=413 Best choice is Rimnicu Vilcea AI 1 AI 1 6 februari Pag.19 6 februari Pag.20 2008 2008 A* search example A* search example Expand Rimnicu Vilcea and determine f(n) for Expand Fagaras and determine f(n) for each each node node – f(Craiova)=c(Rimnicu Vilcea, Craiova)+h(Craiova)=360+160=526 – f(Sibiu)=c(Fagaras, Sibiu)+h(Sibiu)=338+253=591 – f(Pitesti)=c(Rimnicu Vilcea, Pitesti)+h(Pitesti)=317+100=417 – f(Bucharest)=c(Fagaras,Bucharest)+h(Bucharest)=450+0=450 – f(Sibiu)=c(Rimnicu Vilcea,Sibiu)+h(Sibiu)=300+253=553 Best choice is Pitesti !!! Best choice is Fagaras AI 1 AI 1 6 februari Pag.21 6 februari Pag.22 2008 2008 A* search example Optimality of A*(standard proof) Expand Pitesti and determine f(n) for each node Suppose suboptimal goal G2 in the queue. – f(Bucharest)=c(Pitesti,Bucharest)+h(Bucharest)=418+0=418 Let n be an unexpanded node on a shortest to Best choice is Bucharest !!! optimal goal G. f(G ) = g(G ) since h(G )=0 – Optimal solution (only if h(n) is admissable) 2 2 2 > g(G) since G2 is suboptimal Note values along optimal path !! >= f(n) since h is admissible Since f(G2) > f(n), A* will never select G2 for expansion AI 1 AI 1 6 februari Pag.23 6 februari Pag.24 2008 2008 4 BUT … graph search Consistency Discards new paths to repeated A heuristic is consistent if state. h(n) ≤ c(n,a,n')+ h(n') – Previous proof breaks down If h is consistent, we have Solution: f (n') = g(n')+ h(n') = g(n) + c(n,a,n')+ h(n') – Add extra bookkeeping i.e. remove more expsive of two paths. ≥ g(n) + h(n) – Ensure that optimal path to any repeated ≥ f (n) state is always first followed. i.e. f(n) is nondecreasing along any path. – Extra requirement on h(n): consistency (monotonicity) AI 1 AI 1 6 februari Pag.25 6 februari Pag.26 2008 2008 Optimality of A*(more usefull) A* search, evaluation A* expands nodes in order of increasing f value Contours can be drawn in state space Completeness: YES – Uniform-cost search adds circles. – Since bands of increasing f are added – Unless there are infinitly many nodes with f<f(G) – F-contours are gradually Added: 1) nodes with f(n)<C* 2) Some nodes on the goal Contour (f(n)=C*). Contour I has all Nodes with f=fi, where fi < fi+1. AI 1 AI 1 6 februari Pag.27 6 februari Pag.28 2008 2008 A* search, evaluation A* search, evaluation Completeness: YES Completeness: YES Time complexity: Time complexity: (exponential with path – Number of nodes expppanded is still exponential in the length) length of the solution. Space complexity: – It keeps all generated nodes in memory – Hence space is the major problem not time AI 1 AI 1 6 februari Pag.29 6 februari Pag.30 2008 2008 5 A* search, evaluation Memory-bounded heuristic search Completeness: YES Some solutions to A* space problems Time complexity: (exponential with path (maintain completeness and optimality) length) – Iterative-deepening A* (IDA*) Space complexit y:(all nodes are stored) – Here cutoff information is the f-cost (g+h) instead of depth Optimality: YES – Recursive best-first search(RBFS) – Cannot expand fi+1 until fi is finished. – Recursive algorithm that attempts to mimic standard best-first – A* expands all nodes with f(n)< C* search with linear space. – A* expands some nodes with f(n)=C* – (simple) Memory-bounded A* ((S)MA*) – A* expands no nodes with f(n)>C* – Drop the worst-leaf node when memory is full Also optimally efficient (not including ties) AI 1 AI 1 6 februari Pag.31 6 februari Pag.32 2008 2008 Recursive best-first search Recursive best-first search function RECURSIVE-BEST-FIRST-SEARCH(problem) return a solution or failure return RFBS(problem,MAKE-NODE(INITIAL-STATE[problem]),∞) Keeps track of the f-value of the function RFBS( problem, node, f_limit) return a solution or failure and a new f- cost limit best-alternative path available. if GOAL-TEST[problem](STATE[node]) then return node successors ← EXPAND(node, problem) – If current f-values exceeds this alternative f- if successors is empty then return failure, ∞ value than backtrack to alternative path. for each s in successors do f [s] ← max(g(s) + h(s), f [node]) – Upon backtracking change f-value to best f- repeat best ← the lowest f-value node in successors value of its children.