CS221 Section 3: Search Search Problems, UCS and A*

David Lin, Jerry Qu Recap of A* Search ● We want to avoid wasted effort (to go from SF to LA, we probably don’t want to end up looking at roads to Seattle, for example). ● To do this, we can use a heuristic to estimate how far is left until we reach our goal. ● The heuristic must be optimistic. It must underestimate the true cost. Why? Relaxation A good way to come up with a reasonable heuristic is to solve an easier (less constrained) version of the problem For example, we can use geographic distance as a heuristic for distance if we have the positions of nodes. Note: The main point of relaxation is to attain a problem that can be solved more efficiently. First, let’s see what happens when we run UCS. Shortest Paths in Germany

Hannover 0 110 85 90 Bremen ∞ Hamburg ∞ 155 120 200 270 Kiel ∞ Leipzig ∞ 320 255 Schwerin ∞ 365 185 Duesseldorf ∞ 240 140 410 Rostock ∞ Frankfurt ∞ 180 Dresden ∞ Berlin ∞ 410 200 435 Bonn ∞ Stuttgart ∞ Muenchen ∞ 210

Example from CS 4700 at Cornell University Shortest Paths in Germany

Hannover 0 85 110 90 Bremen 120 Hamburg 155 155 120 200 270 Kiel ∞ Leipzig 255 320 255 Schwerin 270 365 185 Duesseldorf 320 240 140 410 Rostock ∞ Frankfurt 365 180 Dresden ∞ Berlin ∞ 410 200 435 Bonn ∞ Stuttgart ∞ Muenchen ∞ 210

Example from CS 4700 at Cornell University Shortest Paths in Germany

Hannover 0 85 110 90 Bremen 120 Hamburg 155 155 120 200 270 Kiel 240 Leipzig 255 320 255 Schwerin 270 365 185 Duesseldorf 320 240 140 410 Rostock ∞ Frankfurt 365 180 Dresden ∞ Berlin ∞ 410 200 435 Bonn ∞ Stuttgart ∞ Muenchen ∞ 210

Example from CS 4700 at Cornell University Shortest Paths in Germany

Hannover 0 85 110 90 Bremen 120 Hamburg 155 155 120 200 270 Kiel 240 Leipzig 255 320 255 Schwerin 270 365 185 Duesseldorf 320 240 140 410 Rostock ∞ Frankfurt 365 180 Dresden 395 Berlin 440 410 200 435 Bonn ∞ Stuttgart ∞ Muenchen 690 210

Example from CS 4700 at Cornell University Shortest Paths in Germany

Hannover 0 85 110 90 Bremen 120 Hamburg 155 155 120 200 270 Kiel 240 Leipzig 255 320 255 Schwerin 270 365 185 Duesseldorf 320 240 140 410 Rostock 360 Frankfurt 365 180 Dresden 395 Berlin 440 410 200 435 Bonn ∞ Stuttgart ∞ Muenchen 690 210

Example from CS 4700 at Cornell University Shortest Paths in Germany

Example from CS 4700 at Cornell University We’re done! But we’ve explored every city on this map. Shortest Paths in Germany

Example from CS 4700 at Cornell University Now, define our heuristic to be the straight-line distance to Shortest Paths in Germany München. Let’s see how A* does. 750 740

110 85 720 90 720 680

155 120 200 270 610 590

320 255 540 365 185 410

240 140 410 380 400 180 480 410 200 435 180

210

Example from CS 4700 at Cornell University Shortest Paths in Germany

750 740 Hannover 0 + 610 = 610 110 85720 720 90 680 Bremen 120 + 720 = 840 Hamburg 155 + 720 = 875 155 120 200 270 590 Kiel ∞ + 750 = ∞ Leipzig 255 + 410 = 665 320 610 540 255 Schwerin 270 + 680 = 950 365 185 410 Duesseldorf 320 + 540 = 860 240 140 410 Rostock ∞ + 740 = ∞ 380 400 Frankfurt 365 + 380 = 745 180 480 Dresden ∞ + 400 = ∞ 410 200 Berlin ∞ + 590 = ∞ 435 180 Bonn ∞ + 480 = ∞

210 Stuttgart ∞ + 180 = ∞ Muenchen ∞ + 0 = ∞ Example from CS 4700 at Cornell University Shortest Paths in Germany

750 740 Hannover 0 + 610 = 610 110 85720 720 90 680 Bremen 120 + 720 = 840 Hamburg 155 + 720 = 875 155 120 200 270 590 Kiel ∞ + 750 = ∞ Leipzig 255 + 410 = 665 320 610 540 255 Schwerin 270 + 680 = 950 365 185 410 Duesseldorf 320 + 540 = 860 240 140 410 Rostock ∞ + 740 = ∞ 380 400 Frankfurt 365 + 380 = 745 180 480 Dresden 395 + 400 = 795 410 200 Berlin 440 + 590 = 1030 435 180 Bonn ∞ + 480 = ∞

210 Stuttgart ∞ + 180 = ∞ Muenchen 690 + 0 = 690 Example from CS 4700 at Cornell University Shortest Paths in Germany

750 740 Hannover 0 + 610 = 610 110 85720 720 90 680 Bremen 120 + 720 = 840 Hamburg 155 + 720 = 875 155 120 200 270 590 Kiel ∞ + 750 = ∞ Leipzig 255 + 410 = 665 320 610 540 255 Schwerin 270 + 680 = 950 365 185 410 Duesseldorf 320 + 540 = 860 240 140 410 Rostock ∞ + 740 = ∞ 380 400 Frankfurt 365 + 380 = 745 180 480 Dresden 395 + 400 = 795 410 200 Berlin 440 + 590 = 1030 435 180 Bonn ∞ + 480 = ∞

210 Stuttgart ∞ + 180 = ∞ Muenchen 690 + 0 = 690 Example from CS 4700 at Cornell University Warm Up

Which of the following algorithms can be used to find the minimum number of actions needed to reach an end state from the start state in an arbitrary search problem? Circle all that apply. a. depth-first search (fails because we just find a path that works, not necessarily shortest) b. breadth-first search - (Works: All we need is constant cost) c. uniform cost search - (Works: Set all costs to be nonnegative) d. dynamic programming Warm Up

You're going rafting! The river is modeled as a grid of positions (x, y), where x ∈ {−m, − (m − 1), . . . , m} is your offset from the middle of the river and y ∈ {0,...,n} is how far down river you are. Unfortunately, there are rocks in the river: For each (x, y), let R(x, y) = 1 if there is a rock and 0 otherwise. The start and end positions do not have rocks.

From any position (x, y), you can: • go straight down to (x, y + 1) (which takes 1 second), • veer left to (x − 1, y + 1) (which takes 2 seconds), or • veer right to (x + 1, y + 1) (which takes 2 seconds).

If you hit a rock, there's an extra 5s delay. The raft starts in (0, 0) and you want to end up in any position (x, y) where y = n. Practice Problem (2015 Q2) x ∈ {−m ... m} and y ∈ {0,...,n} Start: (0,0), You Goal: Get to (Any x, n)

R(x, y) = 1 if there is a rock, 0 otherwise If you hit a rock, there's an extra 5s delay.

Possible Actions: • go down to (x, y + 1) (1 second) • go left to (x − 1, y + 1) (2 seconds) • go right to (x + 1, y + 1) (2 seconds) Practice Problem (2015 Q2) x ∈ {−m ... m} and y ∈ {0,...,n} A. Let C(x,y) denote the minimum time that Start: (0,0), You Goal: Get to (Any x, n) it takes to get from position (x,y) to an end position. For convenience, let C(x, y) = ∞ R(x, y) = 1 if there is a rock, 0 otherwise for invalid positions (x, y) (those that are If you hit a rock, there's an extra 5s delay. off the grid).

Possible Actions: Write down the dynamic programming • go down to (x, y + 1) (1 second) recurrence (you should have a base case • go left to (x − 1, y + 1) (2 seconds) and a recursive case) • go right to (x + 1, y + 1) (2 seconds). Practice Problem (2015 Q2) x ∈ {−m ... m} and y ∈ {0,...,n} A. Let C(x,y) denote the minimum time to get Start: (0,0), You Goal: Get to (Any x, n) from position (x,y) to an end position. Write down the DP recurrence: R(x, y) = 1 if there is a rock, 0 otherwise If you hit a rock, there's an extra 5s delay. C(x,y) = 5R(x,y) + 0 if y = n

Possible Actions: mina∊{-1,0,1} (|a| + 1 + C((x+a, y+1))) otherwise • go down to (x, y + 1) (1 second) • go left to (x − 1, y + 1) (2 seconds) Key observations: • go right to (x + 1, y + 1) (2 seconds). Notice the 3 actions and their associated costs can be represented with action space a ∈ {-1,0,1} and cost |a| + 1 Practice Problem (2015 Q2)

During the rafting trip, you are planning to raft down not just one m × n section of the river, but K of them. You are given K maps R1, ... ,RK, where each Rj(x,y) = 1 if there is a rock in position (x,y) in the j-th map. Assume that each map has at most p rocks (where p is much smalller than mn). You could find the minimum cost path for each of the K maps separately, but that’s a lot of work. Upon inspection, you find that the maps are quite similar, so you suspect there’s a way to save some work.

Indeed there is! Precisely define a single heuristic h(x,y) that can be used to do A* search on each of the K maps. Your heuristic must be

• consistent in the search problem defined by each of the K maps, • non-trivial, which means you must use the information from the rocks somehow (not just define something like the Manhattan distance) • computable in O(kp + mn) time. Practice Problem (2015 Q2)

K maps R1, ... ,RK maps of size m x n. Rj(x,y) = 1 if there is a rock in position (x,y) in the j-th map. Each map has at most p rocks.

Precisely define a single heuristic h(x,y) that can be used to do A* search on each of the K maps.

Justify why your heuristic is consistent, and explain the running time for computing the heuristic for each position (x, y). Practice Problem (2015 Q2)

K maps R1, ... ,RK maps of size m x n. Rj(x,y) = 1 if We need some heuristic that’s there is a rock in position (x,y) in the j-th map. Each consistent and uses the rocks. map has at most p rocks. Any ideas?

Precisely define a single heuristic h(x,y) that can be used to do A* search on each of the K maps.

Justify why your heuristic is consistent, and explain the running time for computing the heuristic for each position (x, y). Practice Problem (2015 Q2)

Precisely define a single heuristic h(x,y) that can be used to do A* Let’s do some relaxation! search on each of the K maps.

Justify why your heuristic is consistent, and explain the running time for computing the heuristic for each position (x, y). Practice Problem (2015 Q2)

Precisely define a single heuristic h(x,y) that can be used to do A* Let’s do some relaxation! search on each of the K maps.

Justify why your heuristic is consistent, and explain the running time for computing the heuristic for each position (x, y). Practice Problem (2015 Q2)

Precisely define a single heuristic h(x,y) that can be used to do A* Let’s do some relaxation! search on each of the K maps.

• consistent in the search problem defined by each of the K maps, We can do that by removing • non-trivial, which means you must use the information from the rocks. But how? rocks somehow (not just define something like the Manhattan distance) • computable in O(kp + mn) time. We keep a rock only if it is in all K maps! Justify why your heuristic is consistent, and explain the running time for computing the heuristic for each position (x, y). Practice Problem (2015 Q2)

K maps R1, ... ,RK maps of size m x n. Rj(x,y) = 1 if Formally, define there is a rock in position (x,y) in the j-th map. Each That is, we keep a rock only if it map has at most p rocks. occurs in all K maps. Define h(x,y) to be the minimum time from Precisely define a single heuristic h(x,y) that can be used to do A* (x,y) to an end state under R. This search on each of the K maps. heuristic can be computed by the • consistent in the search problem defined by each of the K maps, dynamic program in part (a) in • non-trivial, which means you must use the information from the time O(mn). This heuristic is rocks somehow (not consistent for all the maps just define something like the Manhattan distance) • computable in O(kp + mn) time. because the intersection map R(x,y) is a relaxed problem with Justify why your heuristic is consistent, and explain the running respect to each of the K maps. time for computing the heuristic for each position (x, y). Practice Problem (2016 Q2)

You want to optimize a package delivery system. There are n locations numbered 1 to n connected by roads; the set of roads is denoted R. Each road (i, j) ∈ R connects location i with location j. Define N(i) = {j : (i,j) ∈ R} to be the locations reachable by a single road from i. It takes an integer number of minutes ti,j ≥ 0 to traverse the road from i to j, costing ci,j ≥ 0 dollars. We wish to find the cheapest path that goes from location 1 to location n that does not exceed a given time budget T (otherwise customers will be unhappy). Assume that you can only move forwards (i.e. the graph is acyclic).

For example, if n = 3 with roads R = {(1,2),(2,3)}, costs c1,2 = 10 and c2,3 = 30, and times t1,2 = 20 and t2,3 = 10, then there is one path with total cost 10 + 30 and total time 20+10. Practice Problem (2016 Q2) ● Locations (1 to n) A. Your first instinct is to define a search ● Road (i, j) connects problem to solve the delivery problem. States location i & j look like s = (i, t), where i is the current location and t is the number of minutes that has elapsed ○ ti,j ≥ 0 min, ci,j ≥ 0 $ ● N(i) = {j : (i,j) ∈ R} -- so far. Actions look like j, which means traveling reachable locations from i to location j.

Given a time budget T, find Fill in the specification of the search problem. the cheapest path from You must be precise about the valid actions. location 1 to n Practice Problem (2016 Q2) ● Locations (1 to n) A. Fill in the specification of the search ● Road (i, j) connects problem. You must be precise about the valid location i & j actions.

○ ti,j ≥ 0 min, ci,j ≥ 0 $ ● N(i) = {j : (i,j) ∈ R} -- We start at position 1 and time 0 reachable locations from i

Given a time budget T, find Sstart = (1,0) the cheapest path from location 1 to n Practice Problem (2016 Q2) ● Locations (1 to n) A. Fill in the specification of the search ● Road (i, j) connects problem. You must be precise about the valid location i & j actions.

○ ti,j ≥ 0 min, ci,j ≥ 0 $ ● N(i) = {j : (i,j) ∈ R} -- We pick actions from all reachable cities. An reachable locations from i action is only valid if within the time budget.

Given a time budget T, find Sstart = (1,0) ∈ the cheapest path from Actions((i,t)) = {j N(i): t + ti,j≤ T} (traveling to j) location 1 to n Practice Problem (2016 Q2) ● Locations (1 to n) A. Fill in the specification of the search ● Road (i, j) connects problem. You must be precise about the valid location i & j actions.

○ ti,j ≥ 0 min, ci,j ≥ 0 $ ● N(i) = {j : (i,j) ∈ R} -- Successor states are the reachable locations reachable locations from i and the additional time to get there

Given a time budget T, find Sstart = (1,0) ∈ the cheapest path from Actions((i,t)) = {j N(i): t + ti,j≤ T} (traveling to j) location 1 to n Succ((i,t), j) = (j, t + ti,j) Practice Problem (2016 Q2) ● Locations (1 to n) A. Fill in the specification of the search ● Road (i, j) connects problem. You must be precise about the valid location i & j actions.

○ ti,j ≥ 0 min, ci,j ≥ 0 $ ● N(i) = {j : (i,j) ∈ R} -- Cost follows from initial specifications reachable locations from i

Given a time budget T, find Sstart = (1,0) ∈ the cheapest path from Actions((i,t)) = {j N(i): t + ti,j≤ T} (traveling to j) location 1 to n Succ((i,t), j) = (j, t + ti,j)

Cost((i,t), j) = ci,j Practice Problem (2016 Q2) ● Locations (1 to n) A. Fill in the specification of the search ● Road (i, j) connects problem. You must be precise about the valid location i & j actions.

○ ti,j ≥ 0 min, ci,j ≥ 0 $ ● N(i) = {j : (i,j) ∈ R} -- We're done when we reach n! reachable locations from i

Given a time budget T, find Sstart = (1,0) ∈ the cheapest path from Actions((i,t)) = {j N(i): t + ti,j≤ T} (traveling to j) location 1 to n Succ((i,t), j) = (j, t + ti,j)

Cost((i,t), j) = ci,j IsEnd((i,t)) = [i == n] Practice Problem (2016 Q2)

ASDC wants to understand the tradeoff between time and money more precisely. They have asked you for a graph that plots a time budget T against the cost of the minimum cost path from location 1 to location n given time budget T. Let PT be the search problem with time budget T. Solving PT separately for each integer value T = 1, … ,Tmax turns out to be too slow. However, you realize that the search problems for adjacent values of T are quite similar, so maybe A* is applicable.

Assume you have access to: 1. An A* implementation that takes in a search problem and a consistent heuristic and returns the minimum cost path. 2. A dynamic programming (DP) implementation that takes in a search problem and computes the future cost for each state (that is, the minimum cost to get to an end state). Practice Problem (2016 Q2)

Your job is to devise an algorithm that computes the minimum cost path for each T

= 1, … ,Tmax. Your algorithm should loop over the values of T in some order. Recall that DP requires time linear in the number of states, which is too expensive to call at each timestep, so let’s run DP every 10 time steps—that is, for each T mod 10 = 0 (assume that Tmax is divisible by 10), run dynamic programming to obtain future costs. Then, for each T, construct a consistent heuristic based on the results of DP and run A* (which should hopefully run in time much less than the number of states). This is only a sketch of the algorithm. Work out the details, write the pseudocode for your algorithm, and justify why it works. Practice Problem (2016 Q2)

Work out the details, write the pseudocode for your algorithm, and justify why it works. Practice Problem (2016 Q2)

Compute the min cost path for T = 1, … ,Tmax. Let’s start with what’s already Loop over the values of T in some order. given to us: Run DP every 10 time steps. Then, for each For T = {1, …, Tmax}: 1. If 10 divides T, run DP on P T, construct a consistent heuristic based on T 2. Run A* on P with some the results of DP and run A* (which should T heuristic. What is this heuristic? Is hopefully run in time much less than the there anything we could use? number of states). We’re already running DP every 10 steps, we could use the min Work out the details, write the pseudocode for cost from that! your algorithm, and justify why it works. Practice Problem (2016 Q2)

Work out the details, write the pseudocode for Tmax and decrement instead of your algorithm, and justify why it works. starting from 1 and incrementing, since we need to reference DP on higher values of T. Practice Problem (2016 Q2)

Compute the min cost path for T = 1, … ,Tmax. For T = Tmax,...,1: Loop over the values of T in some order. 1. If 10 divides T, run DP on PT Run DP every 10 time steps. Then, for each 2. Run A* on PT with heuristic h(s) = future cost of s in P where T′ = T, construct a consistent heuristic based on T′ 10 ceil(T/10). the results of DP and run A* (which should hopefully run in time much less than the It is important to start withT = number of states). Tmax(least constrained) and proceed to T = 1 (most

Work out the details, write the pseudocode for constrained). Note PT′ is a your algorithm, and justify why it works. relaxation of PT because it has at least as many available actions. Therefore, the future costs of PT′ are a consistent heuristic for PT. Summary

● States Representation/Modelling ○ make state representation as compact as possible, remove unneccesary information ● DP ○ underlying graph cannot have cycles ○ visit all reachable states, but no log overhead ● UCS ○ actions cannot have negative cost ○ visit only a subset of states, log overhead ● A* ○ ensure that relaxed problem can be solved more efficiently