Principles of Ar Ficial Intelligence

Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory

Principles of Arﬁcial Intelligence

Vasant Honavar Arﬁcial Intelligence Research Laboratory College of Informaon Sciences and Technology Bioinformacs and Genomics Graduate Program The Huck Instutes of the Life Sciences Pennsylvania State University

[email protected] hp://vhonavar.ist.psu.edu hp://faculty.ist.psu.edu/vhonavar

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Summary of uninformed (blind) search algorithms Criterion Breadth Depth- Iterative Bi- -First First deepening directional Complete? YES NO YES YES* Time bd+1 bm bd bd/2

Space bd+1 bm bd bd/2

Admissible? YES NO YES YES*

Optimal? NO NO YES NO

Assuming all arc costs are equal m – max depth of search d – depth of soluon, b – ﬁnite branching factor * Assuming forward and backward search are BFS

• All operator applicaons may not be equally expensive • Suppose we have a cost funcon c: S x O à ℜ+ • c (q,o,r) = cost of applying operator o in state q to reach state r • Path cost is typically assumed to be the sum of costs of operator applicaons along the path • An opmal soluon is one with the lowest cost path from the speciﬁed start state s to a goal g ∈ G

• Branch and bound search (BBS) with dynamic programming • Maintain a list of nodes sorted by cost g(n) of cheapest known paral paths to the respecve nodes • Terminate when a node picked from the open list happens to be a goal node A 1 2 ((A,0)) B C ((AB, 1) (AC, 2) 3 8 1 3 D G E F ((AC, 2) (ABD, 4)(ABG,9)) 2 1 H I ((ACE, 3) (ABD, 4)(ACF, 5) (ABG,9))

((ACEI, 4) (ABD, 4)(ACEH, 5)(ACF, 5) (ABG,9))

• Branch and bound search (BBS) with dynamic programming • Maintain a list of nodes sorted by cost g(n) of cheapest known paral paths to the respecve nodes • Terminate when a node picked from the open list happens to be a goal node • Quesons: – Is BBS complete? • Yes – Is BBS admissible? • Yes – Under the assumpon that each arc cost is bounded from below by some posive constant δ

• Informed search uses problem-speciﬁc knowledge • Which search strategies? – Best-ﬁrst search and its variants • Heurisc funcons – How to design them • Local search and opmizaon – Hill climbing, local beam search, genec algorithms,… • Local search in connuous spaces • Online search agents

• [diconary]“A rule of thumb, simpliﬁcaon, or educated guess that reduces or limits the search for soluons in domains that are diﬃcult and poorly understood.”

Heuriscs, those rules of thumb, Oen scorned as sloppy, dumb, Yet slowly commonsense become! – Judea Pearl, in Heuriscs

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Why we need heuriscs • Combinatorial explosion • Uninformed search runs out of me, space, or both quickly • Soluon – incorporate heuriscs to guide search • Perfect heurisc would guide search directly from start state to a goal state yielding the least expensive soluon – No need to search! • In pracce, we have imperfect but useful heuriscs • A heurisc funcon h(n), roughly speaking, esmates the cost of compleng a paral path (s..n) to obtain a soluon, i.e., a path (s..n…g) where g is a goal state

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Why we need heuriscs • Combinatorial explosion • Uninformed search runs out of me, space, or both quickly • Perfect raonality unaﬀordable • Can we come up with a compromise? – Use a heurisc funcon to guide choice of acons • Suppose we could esmate the cost of the cheapest soluon obtainable by expanding each node that could be chosen

+ A h : S → ℜ ∀n∈ S, h(n)≥ 0 ∀g ∈G, h g = 0 s s ( ) 1 s2 3 h(s1 ) = 0.8 h(s2 ) = 2.0 h(s3 ) =1.6

In problems requiring the cheapest path from start state to a goal state – h(n) = esmated cost of the cheapest path from node n to a goal node – h(g1)=h(g2)=0 s

h(m) m n h(n)

g2 g1

• Choose “locally best” moves, guided by the heurisc funcon, with random choice to break es • Terminates when a goal state is reached • Does not look beyond the immediate successors of the current state in deciding which move to make • Essenally DFS, where at each step, successors are ordered by heurisc evaluaon • a.k.a. greedy local search

• E.g for the 8-puzzle – Avg. soluon cost is about 22 steps – Exhausve search to depth 22 ≈ 3.1 x 1010 states – A good heurisc funcon can reduce search

Two commonly used heuriscs

• h1 = the number of misplaced les (not counng blank) relave to the goal (why?)

h1(s)=8

• h2 = the sum of the distances of the les (not counng blank) from their desired posions (Manhaan distance) (why?)

h2(s)=3+1+2+2+2+3+3+2=18 Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Drawbacks of hill-climbing search

• Ridge – sequence of local maxima diﬃcult for greedy algorithms to navigate • Plateau – an area of the state space where the evaluaon funcon is ﬂat

• Stochasc hill-climbing – Random selecon among the uphill moves – The selecon probability can vary with the steepness of the uphill move • First-choice hill-climbing – stochasc hill climbing by generang successors randomly unl a beer one is found • Random-restart hill-climbing – Tries to avoid geng stuck in local maxima

• General approach to informed search: – Best-ﬁrst search: node is selected for expansion based on an evaluaon funcon f(n) • Idea: evaluaon funcon measures esmated cost of compleng the paral soluon (s..n) – Choose node which appears best • Implementaon: – Open-list is a queue sorted in decreasing order of desirability – Special cases: greedy search, A* search

• Best ﬁrst search – List of paral paths is ordered by h values of the nodes • A* search – BBS-like search with dynamic programming – Open list nodes ordered by

f (n)= g(n)+ h(n)

g(n) =2 g(m)=3 s 3 2 4 h(m)=6 3 h(n)=8 n 6 m 5 8

g2 g1

• hSLD= straight-line distance heurisc.

• hSLD cannot be computed from the problem descripon, requires addional informaon or knowledge on the part of the agent • In this example f(n)=h(n) – Expand node that is closest to goal = Greedy best-ﬁrst search

• Best-known form of best-ﬁrst search • Idea: avoid expanding paths that are already expensive • Evaluaon funcon f(n)=g(n) + h(n) – g(n) the cost (so far) to reach the node. – h(n) esmated cost to get from the node to the goal – f(n) esmated total cost of path through n to goal

• A* search using an admissible heurisc – A heurisc is admissible if it never overesmates the cost to reach the goal – Opmisc

Formally: 1. h(n) <= h*(n) where h*(n) is the true cost from n 2. h(n) >= 0 so h(G)=0 for any goal G.

e.g. hSLD(n) never overesmates the actual road distance

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Admissible heuriscs • A heurisc h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n. • An admissible heurisc never overesmates the cost to reach the goal, i.e., it is opmisc

• Example: hSLD(n) (never overesmates the actual road distance) • Theorem: If h(n) is admissible, A* using TREE-SEARCH is admissible • (provided each arc cost is bounded from below by a posive constant δ) • Proof Sketch: – Show that A* terminates with a soluon – Show that A* terminates with an opmal soluon

• Assumpon: Cost of each arc is bounded from below by a posive constant

∀ni,n j ,c(ni,n j ) ≥ δ > 0 • Any me a path gets extended, its cost increases by at least • There is a soluon, i.e., a bounded cost path to a goal (by assumpon) • Terminaon condion: a path terminang in a goal is picked oﬀ the front of € the list of paral paths L • There are two kinds of paths other than those that terminate in a goal: – Paths that lead to dead ends – discovered and discarded (if their cost is lower than that of a path to a goal) – Paths that are inﬁnite (demoted behind a path leading to a goal because sooner or later their cost exceeds the cost of a path leading to a goal) – See notes for a formal proof.

Suppose some subopmal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an opmal goal G.

f(G2) = g(G2) since h(G2) = 0

g(G2) > g(G) since G2 is subopmal f(G) = g(G) since h(G) = 0

f(G2) > f(G) from above

Suppose some subopmal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an opmal goal G.

f(G2) > f(G) from above h(n) ≤ h*(n) since h is admissible g(n) + h(n) ≤ g(n) + h*(n) f(n) ≤ f(G)

* Hence f(G2) > f(n), and A will never select G2 for expansion That is, A* tree search is admissible provided the heurisc funcon is admissible See notes for a more formal proof. Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory What if the search space is not a tree?

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Graph search • Previous proof breaks down • Possible soluons: – Discard more expensive paths when there are mulple paths to a node • Adds messy book-keeping • Admissible if h is admissible – Ensure that opmal path to any repeated state is always ﬁrst followed • Extra requirement on h(n): consistency ∀n,∀n' where n' is a child of n, h(n) ≤ h(n')+ c(n, n')

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Consistency If h is consistent, we have n h ( n ) c( n ; m ) h(n) ≤ c(n,m) + h(m) c( n ; m ) f(n) is non-decreasing along any path

f (m) = g(m) + h(m) m = g(n) + c(n,m) + h(m) h ( m ) ≥ g(n) + h(n) +δ

G > f (n) à First goal selected for expansion must be an opmal goal à A* using graph search is admissible provided the heurisc used is consistent

• Complete?: YES – Since bands of increasing f are added – Unless there are inﬁnitely many nodes with f < f(G) • Not possible when the branching factor is ﬁnite and arc costs are bounded from below by δ • Time complexity: – exponenal in the length of the soluon (in the worst case) • Space complexity: – Need to maintain all generated nodes in memory – Space is a more serious problem than me

• Some soluons to A*’s space problems – Iterave-deepening A* (IDA*) • Cutoff informaon is the f-cost (g+h) instead of depth • Can expand too many nodes when every node has a different f value – (simple) Memory-bounded A* ((S)MA*) • Drop the worst-leaf node when memory is full – Recursive best-first search (RBFS) • Recursive algorithm that aempts to mimic standard best- first search with linear space

• Use all available memory. – i.e. expand best leafs unl available memory is full – When full, SMA* drops worst leaf node (highest f-value) – Backup the f-value of the forgoen node to its parent

• What if all leafs have the same f-value? – Same node could be selected for expansion and deleon – SMA* solves this by expanding newest best leaf and deleng oldest worst leaf. • SMA* is complete if soluon is reachable, admissible if opmal soluon is reachable within the available memory bound

• Similar to IDA* but keeps track of the f value of the best- alternave path available from any ancestor of the current node – If current f values exceeds this alternave f value then backtrack to the best alternave path – During backtracking (unwinding of recursion) change f value of each node to best f-value of its children – Remembers the f value of the best leaf in the “forgoen” sub-tree and hence can revisit that sub-tree, if warranted, at some point in the future

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory Recursive best-ﬁrst search funcon RECURSIVE-BEST-FIRST-SEARCH(problem) return a soluon or failure return RFBS(problem,MAKE-NODE(INITIAL-STATE[problem]),∞) funcon RFBS( problem, node, f_limit) return a soluon or failure and a new f- cost limit if GOAL-TEST[problem](STATE[node]) then return node successors ← EXPAND(node, problem) if successors is empty then return failure, ∞ for each s in successors do f [s] ← max(g(s) + h(s), f [node]) repeat best ← the lowest f-value node in successors if f [best] > f_limit then return failure, f [best] alternave ← the second lowest f-value among successors result, f [best] ← RBFS(problem, best, min(f_limit, alternave)) if result ≠ failure then return result

• Path unl Rumnicu Vilcea is already expanded • f-limit for every recursive call is shown above the node • f value is shown below the node • The path is followed unl Pites which has a f value worse than the f-limit.

• Unwind recursion and store best f-value for current best leaf Pites result, f [best] ← RBFS(problem, best, min(f_limit, alternave)) • best is now Fagaras. Call RBFS for new best – best value is now 450

• Unwind recursion and store best f-value for current best leaf Fagaras • best is now Rimnicu Viclea (again). Call RBFS for new best – Subtree is again expanded. – Best alternave subtree is now through Timisoara. • Soluon is found because 447 > 417. Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar Pennsylvania State University College of Information Sciences and Technology Artificial Intelligence Research Laboratory RBFS

• RBFS is a bit more efficient than IDA* – Sll excessive node generaon – Like A*, admissible if h(n) is admissible • Space complexity is O(bd). • Time complexity difficult to characterize – Depends on accuracy if h(n) and how oen best path changes • IDA* and RBFS suffer from too lile memory • Other variants of A* aim to address these limitaons

• All previous algorithms use ﬁxed strategies. • Agents can learn to improve their search by exploing experience gained during the search (e.g., by analyzing missteps) • We will see examples of this when we consider learning agents

Principles of Artificial Intelligence, IST 597F, Fall 2014, (C) Vasant Honavar