Artificial Intelligence Heuristic (Informed) Search

Artificial Intelligence

Heuristic (Informed) Search

Prof. Dr. habil. Jana Koehler Dr. Sophia Saller, M. Sc. Annika Engel

Deep thanks goes to Summer 2020 Prof. Jörg Hoffmann for sharing his course material

. Algorithms 1) Greedy (Best-First) Search 2) A* and IDA* 3) Bidirectional Search

. Finding good heuristics

2 Artificial Intelligence: Heuristic Search © JK Recommended Reading . AIMA Chapter 3: Solving Problems by Searching – 3.4 Uninformed Search Strategies, the following subchapters: • 3.4.6 Bidirectional search – 3.5 Informed (Heuristic) Search Strategies • 3.5.1 Greedy best-first search • 3.5.2 A* search: Minimizing the total estimated solution cost – 3.6 Heuristic Functions • 3.6.1 The effect of heuristic accuracy on performance • 3.6.2 Generating admissible heuristics from relaxed problems • 3.6.3 Generating admissible heuristics from subproblems: Pattern databases

. Optional reading: – R. C. Holte, A. Felner, G. Sharon, N. R. Sturtevant: Bidirectional Search That Is Guaranteed to Meet in the Middle, AAAI-2016

3 Artificial Intelligence: Heuristic Search © JK How to Determine the next Node for Expansion? . Uninformed Search – rigid procedure, no knowledge of the cost from a node to the goal – e.g. FIFO, LIFO queues

. Informed Search – "value" of expanding a node (state) used as guidance that steers the search algorithm through the search space – evaluation function f(s) assigns a number to each state

4 Artificial Intelligence: Heuristic Search © JK The Evaluation Function ( ) = ( ) + ( ) C(3) s 𝒇𝒇4 𝒔𝒔 𝒈𝒈 𝒔𝒔 𝒉𝒉 𝒔𝒔 s1 A(2) g=5 g=2 𝑔𝑔푔 s2 h*=7 𝑠𝑠 G(0) G(0) C(3) I C(3) s5 G(0) g=7 𝑠𝑠𝑔𝑔푔 B(4) s 3 E(0) h*=0 G(0) g=4 F(1) H(2) E(0) 𝑔𝑔푔 s6 𝑠𝑠 s7

( ) corresponds to the costs ( ) corresponds to the from the initial state to the + estimated costs from the current current𝑔𝑔 𝑠𝑠 state stateℎ 𝑠𝑠 to the goal state  a precise value  an estimated value 𝑔𝑔 5 𝑠𝑠 Artificial Intelligence: Heuristic Search 𝑠𝑠 𝑠𝑠 © JK Heuristic Functions and ∗ Let be a problem with state𝒉𝒉 space𝒉𝒉 . A heuristic function, short heuristic, for is a function so that, for every goal state , we have ( )Π= 0. + Θ Θ ℎ ∶ 𝑆𝑆 ↦ ℝ0 ∪ ∞ 𝑠𝑠 ℎThe𝑠𝑠 perfect heuristic is the function assigning every the cost of a cheapest path from to∗ a goal state, or if no such path exists. ℎ 𝑠𝑠 ∈ 𝑆𝑆 𝑠𝑠 ∞ 0, if is a goal state . = > 0, otherwise 𝑠𝑠 . ℎ 𝑠𝑠 =� for dead-end states, from which the goal is unreachable . ∗ is also called the goal distance of ℎ 𝑠𝑠 ∞ . The∗ value of depends only on the state , not on the path that we ℎ 𝑠𝑠 𝑠𝑠 followed so far to construct the partial solution (and the costs of this path) ℎ 𝑠𝑠 𝑔𝑔

6 © JK Artificial Intelligence: Heuristic Search Desirable Properties of Heuristic Function ( ) 1) Efficient to compute ( = 0 as extreme case) 𝒉𝒉 𝒔𝒔 2) Informative ( ( ) = ( ) as extreme case) ℎ 𝑠𝑠 ∗ 0,ℎ 𝑠𝑠 ℎif 𝑠𝑠is a goal state = 3) > 0, otherwise 𝑠𝑠 4) ℎ is𝑠𝑠 admissible� = 5) ℎ for dead-end states 6) is𝑑𝑑 consistent 𝑑𝑑 ℎ 𝑠𝑠 𝑠𝑠  GOOD heuristics∞ should satisfy a balanced compromise of properties (1) to ℎ(4) at least, better of all 6  Properties (5) ensures effective dead-end recognition and (6) is a prerequisite for algorithms to guarantee minimal-cost (optimal) solutions

7 Artificial Intelligence: Heuristic Search © JK Admissiblity of ( )

Let be a problem𝒉𝒉 𝒔𝒔with state space and let be a heuristic function for . We say that is admissible if, for all , we haveΠ ( ). Θ ℎ Θ ∗ ℎ 𝑠𝑠 ∈ 𝑆𝑆 The functionℎ 𝑠𝑠 ≤ ℎ 𝑠𝑠 corresponds to the real cost of the optimal path from node∗ to a goal state. ℎ 𝑠𝑠 The function is𝑛𝑛 an optimistic estimation of the costs that actually occur. It underestimates the real costs and provides the search algorithmℎ with a lower bound on the goal distance.

8 Artificial Intelligence: Heuristic Search © JK Consistency (Monotonicity) of h(s) Let be a problem with state space , and let be a heuristic function for . We say that is consistent if, for all transitions s in , we have Π Θ ℎ 𝑎𝑎 , . ′ Θ ′ ℎ → 𝑠𝑠 Θ ℎThe𝑠𝑠 value− ℎ 𝑠𝑠 ≤, 𝑐𝑐 𝑠𝑠is𝑎𝑎 the action cost of getting from to with action . We reformulate the inequality from above to: , + ( ). 𝑐𝑐 𝑠𝑠 𝑎𝑎 𝑠𝑠 𝑠𝑠푠 𝑎𝑎 ℎ 𝑠𝑠 ≤ 𝑐𝑐 𝑠𝑠 𝑎𝑎 ℎ 𝑠𝑠푠 ( , ) Triangle inequality: The sum of the lengths of 𝑠𝑠 any two sides of a triangle must be greater or equal than the length of the remaining side. 𝑐𝑐 𝑠𝑠 𝑎𝑎 ( ) ( )𝑠𝑠푠 ℎ 𝑠𝑠 Applying an action to the state , the heuristic value cannot decrease by more ℎ 𝑠𝑠푠 than the cost , 𝑎𝑎of . 𝑠𝑠 goal state

9 Artificial Intelligence: Heuristic Search 𝑐𝑐 𝑠𝑠 𝑎𝑎 𝑎𝑎 © JK Consistency Admissibility

Let be a problem⟹ with state space and let be a heuristic function for . If is consistent, then is admissible. Π Θ ℎ Θ ℎ ℎ To show: , , ( ) ( ), . 𝑎𝑎 This means that we need′ to show that a′ consistent heuristic∗ never overestimatesℎ 𝑠𝑠 the− ℎ costs𝑠𝑠 ≤ to𝑐𝑐 the𝑠𝑠 𝑎𝑎 goal.∀𝑠𝑠 → 𝑠𝑠 ⟹ ℎ 𝑠𝑠 ≤ ℎ 𝑠𝑠 ∀𝑠𝑠 ∈ 𝑆𝑆

Observation: The value of can at most decrease by the action costs. ( ) ( ) ℎ ℎ 𝑠𝑠 0 ( ) , ( , 1) ( ) ℎ 𝑠𝑠 1 , + ( ) 0 𝑐𝑐 𝑠𝑠 𝑎𝑎 (ℎ 𝑠𝑠, 2) ℎ 𝑠𝑠 − ℎ 𝑠𝑠푠 ≤ 𝑐𝑐 𝑠𝑠 𝑎𝑎 ( 2) 1 . ⇔ ℎ 𝑠𝑠 ≤ 𝑐𝑐 𝑠𝑠 𝑎𝑎 ℎ 𝑠𝑠푠 𝑐𝑐 𝑠𝑠 𝑎𝑎 ℎ 𝑠𝑠 . .

10 1 2 3 © JK 𝑎𝑎 𝑎𝑎 𝑎𝑎 Artificial Intelligence: Heuristic Search 𝑠𝑠0 𝑠𝑠1 𝑠𝑠2 𝑠𝑠3 → ⋯ → 𝑠𝑠𝑔𝑔 Proof: We need to show that for all . For states (dead ends) where ( ) = ∗ , this is trivial as any number is . Now let be the set of non dead𝒉𝒉 ∗𝒔𝒔-end≤ 𝒉𝒉states𝒔𝒔 with a𝒔𝒔 shortest cheapest path to a goal state of length𝑠𝑠 . ℎ 𝑠𝑠 ∞ ≤ ∞ 𝑘𝑘 We will prove𝒮𝒮 for all that for all by induction over . 𝑘𝑘 ∗ 𝑘𝑘 Base case: is a goal𝑘𝑘 state,ℎ 𝑠𝑠 so≤ ℎ 𝑠𝑠 ( = 0)𝑠𝑠. ∈By𝒮𝒮 the definition of heuristic𝑘𝑘 functions then ( ) = 0 and so ( ) ( ) = 0 as required. 0 𝑠𝑠 ∗ 𝑠𝑠 ∈ 𝒮𝒮 𝑘𝑘 Inductiveℎ 𝑠𝑠 Hypothesis:ℎ 𝑠𝑠For≤ allℎ 𝑠𝑠 we have that . ∗ 𝑘𝑘 Inductive step: Let . Then𝑠𝑠 ∈ 𝒮𝒮 the cheapest pathℎ 𝑠𝑠 from≤ ℎ to𝑠𝑠 a goal state has length + 1. Let be the successor state of in this cheapest path, so . We thus know 𝑠𝑠 ∈ 𝒮𝒮𝑘𝑘+1 𝑠𝑠 that and therefore 𝑎𝑎 𝑘𝑘 1) By𝑠𝑠푠 the consistency of we have:𝑠𝑠 𝑠𝑠 → 𝑠𝑠푠 ( , ) 𝑘𝑘 𝑠𝑠푠2)∈By𝒮𝒮 the Inductive Hypothesis: ′ 3) Since the cheapest pathℎ has the cheapest costs: ℎ 𝑠𝑠′ −=ℎ 𝑠𝑠∗ ≤+𝑐𝑐 𝑠𝑠( 𝑎𝑎, ) ℎ∗ 𝑠𝑠 ≤ ℎ∗ 𝑠𝑠′′ Combining these three statements, we get ℎ 𝑠𝑠 ℎ 𝑠𝑠 𝑐𝑐 𝑠𝑠 𝑎𝑎 + , + , = ( ) ′ ∗ ∗ QED

11 𝒉𝒉 𝒔𝒔 ≤ ℎ1)𝑠𝑠 𝑐𝑐 𝑠𝑠 𝑎𝑎 ≤ ℎ 2)𝑠𝑠′ 𝑐𝑐 𝑠𝑠 𝑎𝑎 𝒉𝒉3)𝒔𝒔 © JK (1) Greedy Best-First Search . Uses only the heuristic part of the evaluation function ( ) = ( )

𝑓𝑓 𝑠𝑠 ℎ 𝑠𝑠 . Expands the node first that is estimated as being closest to the goal

. Does not consider the current path costs – "counterpart" to uniform-cost search, which uses ( ) = ( )

𝑓𝑓 𝑠𝑠 𝑔𝑔 𝑠𝑠

12 Artificial Intelligence: Heuristic Search © JK GBFS Algorithm

. Frontier ordered by ascending . Duplicates checked at successor generation, against both the frontier and the explored setℎ

13 Artificial Intelligence: Heuristic Search © JK GBFS on the Romania Travel Example

h: Aerial Distances to Bucharest

14 Artificial Intelligence: Heuristic Search © JK 15 Artificial Intelligence: Heuristic Search © JK Properties of GBFS . Complete – for finite state spaces and with duplicate elimination

. Not optimal

. Time complexity is ( ) . Space complexity is 𝑚𝑚 𝑂𝑂 𝑏𝑏 𝑚𝑚 where is the maximum depth𝑂𝑂 𝑏𝑏 of the search space

𝑚𝑚

16 Artificial Intelligence: Heuristic Search © JK (2) A* (Hart, Nilsson, Raphael 1968) . Greedy search only uses ( )

ℎ 𝑠𝑠 . Uniform-Cost search uses ( ) – finds an optimal solution if path costs grow monotonically: (𝑔𝑔)𝑠𝑠 ( )

𝑔𝑔 𝑠𝑠 ≤ 𝑔𝑔 𝑠𝑠푠 . A* uses ( ) = ( ) + ( )

𝑓𝑓 𝑠𝑠 𝑔𝑔 𝑠𝑠 ℎ 𝑠𝑠 . A* combines both using preferably admissible and consistent heuristics

. http://theory.stanford.edu/~amitp/GameProgramming/AStarComparison.html gives a good introduction

17 Artificial Intelligence: Heuristic Search © JK A* Algorithm

Frontier ordered by ascending + , duplicates handled as in UCS (nodes replaced by duplicates with cheaper costs)

18 Artificial Intelligence:𝑔𝑔 Heuristicℎ Search © JK Properties of A* . A* is complete – if a solution exists, A* will find it provided that 1) every node has a finite number of successor nodes, and 2) each action has positive and finite costs

. A* is optimal – first solution found has minimum path cost if is admissible (on trees) or if is consistent (on graphs) • under an admissible heuristics on graphs, A* needsℎ to expand all nodes with ( ) (the costℎ of an optimal solution), called “re-opening” ∗ • For any path,𝑓𝑓 𝑛𝑛re-opening≤ 𝐶𝐶 checks if it is the cheapest to a state and puts explored states back into the frontier if a cheaper path was found 𝑠𝑠

19 Artificial Intelligence: Heuristic Search © JK Properties of A*

. Time Complexity is ( ) 𝑑𝑑 𝑂𝑂 𝑏𝑏 . Space Complexity is ( ), where is the maximum 𝑚𝑚 depth of the search space𝑂𝑂 𝑏𝑏 𝑚𝑚 – subexponential growth requires that the error in the heuristic function grows no faster than the logarithm of the actual path cost ( ) log ( ) ∗ ∗ ℎ 𝑠𝑠 − ℎ 𝑠𝑠 ≤ 𝑂𝑂 ℎ 𝑠𝑠

20 Artificial Intelligence: Heuristic Search © JK A* on the Romania Travel Example

h: Aerial Distances to Bucharest

21 Artificial Intelligence: Heuristic Search © JK Computed f-Values in the Example Frontier:

S,T,Z

R,F,T,Z,O

F,P,T,Z,C,O

P,T,Z,B,C´,O Bucharest is inserted after Pitesti in the frontier! B´,T,Z,C´,O

22 Artificial Intelligence: Heuristic Search © JK f-based Contours in the Search Space . A* fans out from the start node, adding nodes in concentric bands of increasing f-costs – with good heuristics the bands stretch towards the goal state and are more narrowly focused around the optimal path

23 Artificial Intelligence: Heuristic Search © JK Proof of Optimality of A* under Consistent Heuristics . The general idea for the proof is to encode the consistent heuristic function as action costs and establish a correspondence to uniform cost search – UCS is optimal for non-negative action costs (Dijkstra´s algorithm)

. We then show that the original and the transformed problem have the same optimal solutions and isomorphic search spaces . Finally, we can prove that every optimal solution found by A* on the transformed problem, is also an optimal solution for the original problem

24 Artificial Intelligence: Heuristic Search © JK Step 1: Encoding Heuristic Values as Action Costs

Definition: Let be a problem with state space = , , , , , , and let be a consistent heuristic function for . We define the -weighted𝐺𝐺 state space as Π = , , , , , where: Θ 𝑆𝑆 𝐴𝐴 𝑐𝑐 𝑇𝑇 𝐼𝐼 𝑆𝑆 • ℎ , ℎ| , ℎ, ℎ ℎ, ,𝐺𝐺 , Π , ℎ • ℎ: ′Θis defined𝑆𝑆 𝐴𝐴 by𝑐𝑐 ′ 𝑇𝑇 𝐼𝐼 ,𝑆𝑆 ′ , , • 𝐴𝐴ℎ ≔=ℎ 𝑎𝑎 ,𝑠𝑠 +𝑠𝑠 , 𝑎𝑎, ∈ |𝐴𝐴 𝑠𝑠, 𝑠𝑠, ∈ℎ 𝑆𝑆 𝑠𝑠. 𝑎𝑎 𝑠𝑠 ∈ 𝑇𝑇 ′ 0 𝑐𝑐 ℎ 𝐴𝐴 → ℝ 𝑐𝑐 𝑎𝑎 𝑠𝑠 𝑠𝑠푠 ≔ 𝑐𝑐 𝑠𝑠 𝑎𝑎 − ℎ 𝑠𝑠 − ℎ 𝑠𝑠 𝑇𝑇 𝑠𝑠 𝑎𝑎 𝑠𝑠 𝑠𝑠푠 𝑠𝑠 𝑠𝑠 𝑎𝑎 𝑠𝑠푠 ∈ 𝑇𝑇 Remember consistency of : , + = ( ) ( , ′) ℎ ℎ 𝑠𝑠 ≤ 𝑐𝑐 𝑠𝑠 𝑎𝑎 ℎ 𝑠𝑠 ℎ 𝑠𝑠 − ℎ 𝑠𝑠푠 ≤ 𝑐𝑐 𝑠𝑠 𝑎𝑎 Lemma: is well-defined, i.e. ℎ , ( ) 0. Θ Proof: The assumption𝑐𝑐 𝑠𝑠follows𝑎𝑎 − immediatelyℎ 𝑠𝑠 − ℎ 𝑠𝑠푠 from≥ consistency.

25 Artificial Intelligence: Heuristic Search © JK Illustration of Encoding

700 = 400 - (2200 - 2500) 150 = 650 - (2200 -1700) 250 = 1950 - (1700 - 0)

Optimal Solution: SB - DD - M 650 + 1950 = 2600 = 150 + 250 + 2200

26 h(SB) © JK Identify the Correspondence (1) Lemma A: and have the same optimal solutions. ℎ Θ Θ Proof: Let , … , be the corresponding state path of a solution 𝑎𝑎1 𝑎𝑎𝑛𝑛 in , . The cost of the same path in is : = + 0 1 𝑛𝑛−1 𝑛𝑛 ′ ′ + 𝑠𝑠𝐺𝐺 , 𝑠𝑠 + ( 𝑠𝑠 ) + 𝑠𝑠 + , +ℎ ( ) + 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 − ℎ 𝑠𝑠 − ℎ 𝑠𝑠 −ℎ 𝑠𝑠 ℎ 𝑠𝑠 Θ 𝑠𝑠𝑛𝑛 ∈ 𝑆𝑆 Θ −ℎ 𝑠𝑠0 𝑐𝑐 𝑠𝑠0 𝑎𝑎1 ℎ 𝑠𝑠1 −ℎ 𝑠𝑠1 𝑐𝑐 𝑠𝑠1 𝑎𝑎2 ℎ+ 𝑠𝑠2 + , + ( ) = + , + + , + + ⋯ −ℎ 𝑠𝑠𝑛𝑛−1 𝑐𝑐 𝑠𝑠𝑛𝑛−1 𝑎𝑎𝑛𝑛 ℎ 𝑠𝑠𝑛𝑛 = 0 + , + ( ) −ℎ 𝑠𝑠0 𝑐𝑐 𝑠𝑠0 𝑎𝑎1 ℎ 𝑠𝑠1 − ℎ 𝑠𝑠1 𝑐𝑐 𝑠𝑠1 𝑎𝑎2 ℎ 𝑠𝑠2 − ℎ 𝑠𝑠2 ℎ 𝑠𝑠𝑖𝑖 − ℎ 𝑠𝑠𝑖𝑖 ⋯ − ℎ 𝑠𝑠𝑛𝑛−1 𝑐𝑐 𝑠𝑠𝑛𝑛−1 𝑎𝑎𝑛𝑛 ℎ 𝑠𝑠𝑛𝑛 ∀𝑠𝑠𝑖𝑖 ∈ 𝑆𝑆 = 𝑛𝑛 , + ( ) = 𝑛𝑛 , ,

� 𝑐𝑐 𝑠𝑠𝑖𝑖−1 𝑎𝑎𝑖𝑖 − ℎ 𝑠𝑠0 ℎ 𝑠𝑠𝑛𝑛 � 𝑐𝑐 𝑠𝑠𝑖𝑖−1 𝑎𝑎𝑖𝑖 − ℎ 𝑠𝑠0 since𝑖𝑖=1 is a goal state, it holds 𝑖𝑖=1 = 0.

𝑠𝑠𝑛𝑛 ℎ 𝑠𝑠𝑛𝑛 Thus, the costs of solution paths in are those of , minus a constant. The claim follows. ℎ

27 Artificial Intelligence:Θ Heuristic Search Θ © JK Identify the Correspondence (2)

Lemma B: The search space of A* on is isomorphic to that of uniform-

cost on . Θ ℎ Θ Proof: Let , … , be any state path in . The + value, 𝑎𝑎1 𝑎𝑎𝑛𝑛 used by A*,𝑠𝑠 0is 𝑠𝑠1 (𝑠𝑠𝑛𝑛−1, )𝑠𝑠𝑛𝑛+ ( ). The value Θin , used𝑔𝑔 ℎ by 𝑛𝑛 ℎ uniform-cost search∑𝑖𝑖=1 𝑐𝑐on𝑠𝑠𝑖𝑖 −1 ,𝑎𝑎 is𝑖𝑖 ℎ 𝑠𝑠𝑛𝑛( , )𝑔𝑔 +Θ ( ) (see Proof ℎ 𝑛𝑛 of Lemma A). The differenceΘ ∑(𝑖𝑖=1)𝑐𝑐is𝑠𝑠 𝑖𝑖constant,−1 𝑎𝑎𝑖𝑖 − ℎso𝑠𝑠 the0 orderingℎ 𝑠𝑠𝑛𝑛 of the frontier (open list) is the same.− ℎAs𝑠𝑠 0the duplicate elimination is identical, the assumption is shown.

28 Artificial Intelligence: Heuristic Search © JK Illustration of A* and UCS Searches

400+2500 650+1700

650+1950+0

29 Artificial Intelligence: Heuristic Search © JK Proving the Final Theorem

Theorem (Optimality of A*) Let be a problem with state space , and let be a heuristic function for . If is consistent, then the solution returned by A* (ifΠ any) is optimal. Θ ℎ Θ ℎ Proof Let be the solution returned by A* run on . Denote by the ∗ set of 𝐴𝐴all solutions that can possibly be returned by uniform cost𝑈𝑈𝑈𝑈𝑈𝑈 search𝜌𝜌 run on . Θ 𝕊𝕊 By Lemma B weℎ know that . ∗ By optimality ofΘ Uniform-Cost𝐴𝐴-Search,𝑈𝑈𝑈𝑈𝑈𝑈 every element of is an optimal solution for . 𝜌𝜌 ∈ 𝕊𝕊 𝑈𝑈𝑈𝑈𝑈𝑈 Thus is an optimalℎ solution for . 𝕊𝕊 ∗ Together𝐴𝐴 with LemmaΘ A, this impliesℎ that is an optimal solution ∗ for .𝜌𝜌 Θ 𝐴𝐴 𝜌𝜌

30 Θ Artificial Intelligence: Heuristic Search © JK IDA* (Korf 1985) . A* requires exponential memory in the worst case – combine with Depth-Limited Search . Idea: use successive iterations with increasing -costs – use -bounds instead of bounding the length of the path 𝑓𝑓 𝑓𝑓 . At each iteration, perform a depth-first search, cutting off a branch when its total cost ( + ) exceeds a given threshold – threshold starts at the estimate of the cost of the initial 𝑔𝑔 ℎ state, and increases for each iteration of the algorithm – at each iteration, the threshold used for the next iteration is the minimum cost of all values that exceeded the current threshold

31 Artificial Intelligence: Heuristic Search © JK Example of IDA* . initial -cost limit = 366 ( -costs of the initial state) – first expansion: T=118 +329=447, S=140+253=393, Z=75+374=449 𝑓𝑓 𝑓𝑓 . next -cost limit = 393 (S)

𝑓𝑓

32 Artificial Intelligence: Heuristic Search © JK Properties of IDA*

. IDA* is complete if every node has a finite number of successor nodes and if each action has positive and finite costs

. IDA* is optimal. The first solution found has minimal path costs if is

admissible (tree) or consistent (graph) ℎ

. Time Complexity is ( ) 𝑑𝑑 𝑂𝑂 𝑏𝑏 . Space Complexity is ( )

𝑂𝑂 𝑑𝑑

33 Artificial Intelligence: Heuristic Search © JK (3) Bidirectional Search (Concept of Searching)

. 2 simultaneous searches: 1 forward (starting at initial state) and 1 backward (starting at goal state) . Hoping that the two searches meet in an intersection state in the middle . Motivation: is exponentially less than . 𝑑𝑑⁄2 𝑑𝑑 Any search 𝑂𝑂technique𝑏𝑏 can be used 𝑂𝑂 𝑏𝑏 . Goal test test if two searches have intersection state 34 Artificial Intelligence: Heuristic Search © JK ⟶ Example

Forward search Backward search with BFS: E with DFS: A B C D K G D B I → → → F → → →

Initial state A C

G I Goal state D K H J

Problem: We do not necessary meet in the middle and there is no guarantee that the solution is optimal

35 ⟹ Artificial Intelligence: Heuristic Search © JK (3) Bidirectional Search – MM Algorithm (Holte et al 2016)

R. C. Holte, A. Felner, G. Sharon, N. R. Sturtevant: Bidirectional Search That Is Guaranteed to Meet in the Middle, AAAI-2016

. First letter indicates the distance from start (N=near, F=far, R=remote) and the second letter indicates the distance from goal (N=near, F=far) . NN includes only those states at the exact midpoint of optimal solutions

36 Artificial Intelligence: Heuristic Search © JK Measuring Heuristic Distances in Bidirectional Search

. A* search in both directions . Each search direction uses an admissible front-to-end heuristic that directly estimates the distance from node to the target of the search (target for forward search is the goal, target for backward search is the start state) 𝑛𝑛 . ( , ) is the distance (cost of a least-cost path) from state to state 𝑑𝑑 𝑢𝑢 𝑣𝑣 𝑢𝑢 . = ("start", "goal") is the cost of an optimal solution 𝑣𝑣 . State∗ is “near to goal” if ( , goal) 2, and “far from 𝐶𝐶 𝑑𝑑 goal” otherwise. For start, we make a 3-∗way distinction: is “near 𝑠𝑠to start” if (start, )𝑑𝑑 𝑠𝑠 2, “far≤ 𝐶𝐶 from⁄ start” if 2 < (start, ) , and “remote” if∗ (start, ) > ∗ 𝑠𝑠

37 ∗ 𝑑𝑑 Artificial Intelligence:𝑠𝑠 Heuristic≤ Search𝐶𝐶 ⁄ ∗ 𝐶𝐶 ⁄ © JK 𝑑𝑑 𝑠𝑠 ≤ 𝐶𝐶 𝑑𝑑 𝑠𝑠 𝐶𝐶 Properties of MM

. MM is complete

. MM is optimal for non-negative action costs, the first solution found has minimal path costs if is admissible

(tree) or consistent (graph) ℎ

. If there exists a path from start to goal and MM’s heuristics are consistent, MM never expands a state twice

38 Artificial Intelligence: Heuristic Search © JK Overview on Properties of Heuristic Algorithms Criterion Greedy Best- A* search IDA* search MM Algorithm First Search (Bidirectional search) Evaluation ( ) + ( ) + ( ) + ( ) function ( )

Complete𝒇𝒇 ?𝒔𝒔 Yesℎ 𝑠𝑠a,b 𝑔𝑔 𝑠𝑠Yes cℎ,d 𝑠𝑠 𝑔𝑔 𝑠𝑠Yes cℎ,d 𝑠𝑠 𝑔𝑔 𝑠𝑠 Yesℎe 𝑠𝑠 Time ?

𝑚𝑚 𝑑𝑑 𝑑𝑑 g Space 𝑂𝑂 𝑏𝑏 𝑂𝑂 𝑏𝑏 𝑂𝑂 𝑏𝑏 ? 𝑚𝑚 𝑚𝑚 e e f Optimal? 𝑂𝑂 No𝑏𝑏 𝑂𝑂Yes𝑏𝑏 𝑂𝑂Yes𝑑𝑑 Yes Superscripts: Where: a for finite state spaces depth of solution b with duplicate elimination maximum depth of the search space c if every node has a finite number of 𝑑𝑑 successor nodes 𝑚𝑚 d if each action has positive and finite costs e 1st solution found has minimal path costs if is admissible (tree) or consistent (graph) f for non-negative action costs ℎ g 39 with backtracking search, else ( ) © JK

𝑂𝑂 𝑏𝑏𝑏𝑏 Designing Heuristic Functions . The informedness of the heuristic is critical for the success of the search algorithm – steer the algorithm towards the most promising parts of the search space – recognize dead ends early – find a near-optimal solution under practical conditions

. Requires an understanding of the application domain – keep heuristic and search approach separate – try out different variants in empirical tests – an art form and a hot topic in AI research

corresponds to the number of tiles in the wrong position (Misplaced Tiles)

ℎ1 corresponds to the sum of the distances of the tiles from their goal positionsℎ2 (Manhattan distance)

h1 = 8 (all tiles are misplaced)

h2 = 3 + 1 + 2 + … = 18

initial state goal state . Distance between two points measured along axes at right angles goal position

current position

. Disadvantage: considers all tiles independently

LC: 2 + 2 = 4

MH: 2 + 0 = 2

. Two tiles and are in a linear conflict if and are in the same line, the goal positions of and are both in that 𝑗𝑗 𝑘𝑘 𝑗𝑗 𝑘𝑘 line, is to𝑡𝑡 the right𝑡𝑡 of and the goal position𝑡𝑡 of𝑡𝑡 is to the 𝑗𝑗 𝑘𝑘 left of the goal position of 𝑡𝑡 𝑡𝑡 𝑡𝑡𝑗𝑗 𝑡𝑡𝑘𝑘 𝑡𝑡𝑗𝑗 – LC will add a cost of 2 moves for each pair of conflicting 𝑡𝑡𝑘𝑘 tiles to the Manhattan Distance – Motivation: Tiles need to surround one another . 44 LC is consistent andArtificial Intelligence:more Heuristic informative Search than MH © JK Gaschnig's Heuristic

9 stands for the blank

transform 724596831 into 912345678

Relaxed Move: Any tile can be moved to the blank position (count the number of swaps) loop until solution is found: If 9 is not at the 1st position then swap 9 with the element whose target position 9 is taking else swap 9 with the rightmost element that is not in its proper place

724596831 – 729546831 – 792546831 – 712546839 – 712546938 – 712549638 – 712945638 – 712345698 – 912345678 = 8 swaps

724596831

794526831 724956831 724569831 724536891 . 4 successor nodes . Compute the number of swaps for each node . Expand the node with the fewest number of swaps first  Problem relaxation is a powerful idea and very successfully used to derive good heuristics

46 Artificial Intelligence: Heuristic Search © JK Problem Relaxation on Whitebox Description . Primitive Predicates in the N-Puzzle – ON(t, y) : tile t is on cell y – CLEAR(y) : cell y is clear of tiles – ADJ(y, z) : cell y is adjacent to cell z

. Move(t, y, z) preconditions : ON(t, y) & CLEAR(z) & ADJ(y, z) effects : ON(t, z) & CLEAR(y) & NOT ON(t, y) & NOT CLEAR(z)

. Remove CLEAR(z) & ADJ(y, z) – Misplaced Tile heuristic . Remove CLEAR(z) – Manhattan Distance heuristic . Remove ADJ(y, z) – Gaschnig's heuristic

47 Artificial Intelligence: Heuristic Search © JK Pattern Database . Apply the Divide and Conquer principle – decompose the problem into subproblems (subgoals) – store solutions to the subproblems with associated costs (patterns) – reuse these solutions

Divide the 15 puzzle into Fringe + 8 puzzle • map the current location of the fringe tiles into an index of the database • the data base tells us the minimal number of moves to achieve the fringe • achieve the fringe + solve the remaining 8 puzzle

. Famous example: end game libraries in chess 48 Artificial Intelligence: Heuristic Search © JK Learning Heuristic Functions

. Relaxed problem heuristic – problem with fewer restrictions on the actions is called a relaxed problem – cost of an optimal solution path to a relaxed problem is an admissible heuristic for the original problem

. Modern search algorithms analyze the domain and the given problem instance – learn a problem-instance specific heuristic before they start searching

49 Artificial Intelligence: Heuristic Search © JK Summary . Heuristic search is the preferred search method for medium size search spaces . The effectiveness of heuristic search depends on the properties of the heuristic function: efficient to compute, informative, admissible, consistent . Only recently, a bidirectional search algorithm was developed, which is guaranteed to meet in the middle, but does not gurantee to reduce search and memory costs in the worst case . Greedy best-first search often quickly finds good solutions in practice . A* is optimal on graphs when using consistent heuristics . Finding good heuristics can be done by analyzing the

50 search problem, e.g. using relaxation methods © JK Working Questions 1. Explain the underlying ideas of greedy (best-first) search, A* and IDA*. 2. What are the properties of A*? 3. Why can IDA* be more efficient than A* in practice? 4. What are heuristics, which role do they play in informed search? 5. Which properties have good heuristics? 6. What is an admissible/consistent heuristic function? 7. What can happen with A* if the heuristic function is non- admissible / non-consistent? 8. What is important for bidirectional search to work?