Single-Source Shortest Paths (Ch. 24) Single-Source Shortest Paths

The single-source shortest path problem (SSSP) Weights in SSSP algorithms include distances, times, hops, input: a directed graph G = (V, E) with edge weights, and a cost, etc. Used in lots of routing applications. specific source node s. goal: find a minimum weight (shortest) path from s to every other Note: BFS finds the shortest paths for the special case when all node in V (sometimes we only want the cost of these paths) edge weights are 1. Running time = O(V + E) It turns out that, in the worst case, finding the shortest path between s and a single The result of a SSSP algorithm can be viewed as a tree node t is no easier than finding the shortest paths between s and all other nodes rooted at s, containing a shortest (wrt weight) path from s à g = ¥ s to all other nodes. s à b = 2 2 3 s à c = 5 s à h = 7 a b c 2 3 5 6 a b 6 c s à d = 6 S 5 3 d 1 e S 3 s à e = 5 2 d 1 e 2 s à f = 3 f 4 g 2 h f 4 g 2 h 2

Negative-weight cycles Single-Source Shortest Paths

Some graphs may have negative-weight cycles and these are Question: Can a shortest path contain a cycle? a problem for SSSP algorithms.

What is the shortest path from a to d? No. b 9 · path a-c-e-d = -12+3-2 = -11 2 8 6 Suppose we have a shortest path p = áv0, v1, ..., vkñ and S -12 3 c = áv , v , ..., v ñ is a positive-weight cycle on p so that v = v · path a-c-e-d-a-c-e-d = a c e i i+1 j i j -12+3-2+(10-12+3-2)= -13 6 4 and w(c) > 0. Then the path (obtained from splicing out c) 10 p' = áv , v , ..., v , v , v ,...v ñ If we keep going around the cycle d -2 0 1 i j+1 j+2 k (d-a-c-e-d), we keep shortening the has w(p') = w(p) – w(c) < w(p). So p can’t be a shortest path. weight of the path. So the shortest path has weight -¥ Therefore, we can assume, wlog, that shortest paths have no cycles. To avoid this problem, we require that the graph has no negative weight cycles, otherwise the solution does not exist. 3 4

Dijkstra’s SSSP Algorithm Dijkstra’s SSSP Algorithm

Algorithm SSSP-Dijkstra (G, s) Algorithm SSSP-Dijkstra (G, s) 1. Q = Æ //** Q is a priority queue 1. Q = Æ //** Q is a priority queue Tree nodes are Trace execution 2. d[s] = 0 and insert s into Q 2. d[s] = 0 and insert s into Q of Dijkstra’s algorithm nodes extracted 3. for each v ¹ s 3. for each v ¹ s from Q (in S, on graph below. 4. d[v] = ¥ 4. d[v] = ¥ the set of 5. insert v into Q with key d[v] 5. insert v into Q with key d[v] shortest paths). 6. while Q ¹ Æ 6. while Q ¹ Æ 7. u = extract-min(Q) 7. u = extract-min(Q) 8. for each (outgoing) neighbor of u 8. for each outgoing neighbor of u 9. d[v] = min{d[v], d[u] + wt(u,v)} 9 9. d[v] = min{d[v], d[u] + wt(u,v)} 10. endfor b 11. endwhile 2 10. endfor Procedure relax (x, y) 8 6 d[y] = min{d[y], d[x] + wt(x, y)} S 12 3 11. endwhile a c e 4 Fringe nodes are nodes in Q with d[v] < ¥ 6 10 Unseen nodes are nodes in Q with d[v] = ¥ 5 d 2

1 Algorithm SSSP-Dijkstra (G, s) Algorithm SSSP-Dijkstra (G, s) iteration 0: 1. Q = Æ //** Q is a priority queue 1. Q = Æ //** Q is a priority queue Q a b c d e 2. d[s] = 0 and insert s into Q iteration 4: Q = Q – {e} 2. d[s] = 0 and insert s into Q 3. for each v ¹ s 3. for each v ¹ s d[v] 0 ¥ ¥ ¥ ¥ 4. d[v] = ¥ Q xa xb xc d xe 4. d[v] = ¥ iteration 1: Q = Q – {a} 5. insert v into Q with key d[v] d[v] 0 2 10 13 11 5. insert v into Q with key d[v] 6. while Q ¹ Æ 6. while Q ¹ Æ Q xa b c d e 7. u = extract-min(Q) 7. u = extract-min(Q) d[v] 0 2 12 ¥ ¥ 8. for each neighbor of u 8. for each neighbor of u 9. d[v] = min{d[v], d[u] + wt(u,v)} iteration 5: Q = Q – {d} 9. d[v] = min{d[v], d[u] + wt(u,v)} iteration 2: Q = Q – {b} 10. endfor 10. endfor 11. endwhile Q xa xb xc xd xe 11. endwhile Q xa xb c d e d[v] 0 2 10 13 11 d[v] 0 2 10 ¥ 11 b 9 b 9 2 2 8 6 8 6 iteration 3: Q = Q – {c} S 12 3 DONE S 12 3 a c e a c e Q xa xb xc d e 6 4 6 4 0 2 10 16 11 10 2 10 2 d[v] d d 8

Dijkstra’s SSSP Algorithm Dijkstra’s SSSP Algorithm iter u d[a] d[b] d[c] d[d] d[e] Algorithm SSSP-Dijkstra (G, s) Exercise: 0 - 0 ¥ ¥ ¥ ¥ 1. Q = Æ //** Q is a priority queue Example: 2. d[s] = 0 and insert s into Q 1 a x 2 12 ¥ ¥ 3. for each v ¹ s 1. Give a simple example of a directed graph with 2 b x x 10 ¥ 11 4. d[v] = ¥ 5. insert v into Q with key d[v] negative weight edges 3 c x x x 16 11 6. while Q ¹ Æ (but no neg wt cycles) 7. u = extract-min(Q) for which Dijkstra’s alg 4 e x x x 13 x 8. for each neighbor of u produces incorrect 5 d x x x x x 9. d[v] = min{d[v], d[u] + wt(u,v)} answers. 10. endfor b 9 11. endwhile 2 8 6 S 12 3 2. Suppose we change line 6 to “while |Q| > 1” a c e This change causes the while loop to execute |V| - 1 times 6 4 instead of |V| times. Is this proposed algorithm correct? Why or 10 d 2 why not? 10

Running Time of Dijkstra’s SSSP Alg Running Time of Dijkstra’s SSSP Alg

Algorithm SSSP-Dijkstra (G, s) Assume adjacency list 1. Q = Æ //** Q is a priority queue representation. The time If G is dense (i.e., q(V2) edges): 2. d[s] = 0 and insert s into Q will depend on what? 3. for each v ¹ s no point in being clever about extract-min. Store each 4. d[v] = ¥ Implementation of the d[v] in the vth entry of an array. Each insert and decrease- 5. insert v into Q with key d[v] priority queue. key takes O(1) time. Extract-min takes O(V) time (why?) 6. while Q ¹ Æ Total time: O(V2 + E) = O(V2) 7. u = extract-min(Q) In general, 8. for each neighbor of u 2 Steps 1-5: O(V) time If G is sparse (i.e., o(V ) edges: 9. d[v] = min{d[v], d[u] + wt(u,v)} Steps 6-11: try to minimize extract-min, use binary heap (heapsort) 10. endfor time for extract-min & decrease-key = O(lgV) 11. endwhile V iterations Suppose extract-min Total time: O(VlgV + ElgV) = O((V + E) lgV) Steps 8-10: E iterations overall takes O(X ) time (looks at each edge once) Q

Suppose decrease-key takes O(YQ) time.

Total: O(VXQ + EYQ) 11 12

2 Correctness of Dijkstra’s SSSP Alg Correctness of Dijkstra’s SSSP Alg

Let S = V – Q be the nodes that have been removed from Q. Lemma: For all nodes x Î V: Lemma: For all nodes x Î V: (a) if x Î S (x Ï Q), then the shortest s to x path only uses nodes in S and (a) if x Î S (x Ï Q), then the shortest s to x path only uses nodes in S and d[x] is its weight. d[x] is its weight. (b) if x Ï S (x Î Q), then d[x] is weight of the shortest s to x path, all of (b) if x Ï S (x Î Q), then d[x] is weight of the shortest s to x path, all of whose intermediate nodes are in S. whose intermediate nodes are in S. Induction Step: Show Lemma is true for iteration i. Proof: By induction on i, the number of iterations of the while loop. Let u be the node selected in the ith iteration (the node in Q with minimum d[u] value). Basis: i = 1, S = {s}, and d[x] = ¥ if x is not a neighbor of s, and Proof of (a): (Assume x is in S) otherwise d[x] = wt(s,x). So both (a) and (b) hold. case 1: x ¹ u. Then x was in S before iteration i, and by the IHOP, we already had the best s to x path. Inductive Hypothesis (IHOP): Assume true for iteration i -1.

13

Correctness of Dijkstra’s SSSP Alg Correctness of Dijkstra’s SSSP Alg Lemma: For all nodes x Î V: Proof of (b): (Assume x is not in S) (a) if x Î S (x Ï Q), then the shortest s to x path only uses nodes in S and d[x] is its weight. Choose x that is not in S after iteration i (x ¹ u): (b) if x Ï S (x Î Q), then d[x] is weight of the shortest s to x path, all of whose intermediate nodes are in S. S (before i) • d[x] is wt of shortest s to x path with all case 2: x = u. Suppose in contradiction the shortest s to u path uses intermed. nodes in S (IHOP) some node r not in S after iteration i. s u • d[u] is wt of shortest s to u path by (a) • d[u] is wt of shortest s to u path with all x intermed nodes in S (IHOP) case 1: x is not a neighbor of u. Then • d[r] is wt of shortest s to r path with all adding u to S doesn’t change the best s intermed nodes in S (IHOP) S (before i) s to x path with all internal nodes in S. u • d[u] £ d[r] since alg picks u = x in To see why not, suppose it did: iteration i. v s u • there is a better s to v obtained by r So...the shortest s to u path can’t go going outside S...contradicts part (a) S (before i) through r since there are no negative x • so best s to x path and d[x] can’t weight edges. change. 16

Correctness of Dijkstra’s SSSP Alg The Convergence property Proof of (b) (cont): • If s u v is a shortest path in G for some u,v V, and if Choose x that is not in S after iteration i (x ¹ u): ® Î d[u] = d(s, u) at any time prior to relaxing edge (u, v), then S (before i) d[v] = d(s, v) at all times afterward. case 2: x is a neighbor of u. s u The algorithms checks to see if it is Algorithm SSSP-Dijkstra (G, s) 1. Q = Æ //** Q is a priority queue better to go from s to x via u, or to Dijkstra's algorithm relies 2. d[s] = 0 and insert s into Q choose an edge from some other node on the property that a shortest x 3. for each v ¹ s path is made up of segments in S. 4. d[v] = ¥ that are smaller shortest paths 5. insert v into Q with key d[v] Exercise: 6. while Q ¹ Æ Why doesn’t the proof of correctness for Dijkstra’s algorithm 7. u = extract-min(Q) 8. for each neighbor of u go through when negative edge weights are allowed? Procedure relax (x,y) 9. relax (u, v) d[y] = min{d[y], d[x] + wt(x,y)} 10. endfor 11. endwhile 17 18

3 The Path-relaxation property Bellman-Ford SSSP Algorithm

• If p = v0, v1, ..., vk is a shortest path from s = v0 to vk, and the edges of p are relaxed in the order (v0, v1), (v1, v2),... • Computes single-source shortest paths even when some edges (vk-1, vk), then d[vk] = d(s, vk). This property holds regardless have negative weight. of any other relaxation steps that occur, even if these other steps are intermixed with relaxations of the edges of p. • Can detect if there are any negative-weight cycles in the graph.

Algorithm SSSP-Dijkstra (G, s) 1. Q = Æ //** Q is a priority queue Note: Dijkstra's algorithm does The algorithm has 2 parts: 2. d[s] = 0 and insert s into Q not ensure that the edges are Part 1: Computing shortest paths tree: 3. for each v ¹ s examined in any particular order. · |V| - 1 iterations. 4. d[v] = ¥ but it does ensure that, at the time 5. insert v into Q with key d[v] u is extracted from Q, · Iteration i computes the shortest path 6. while Q ¹ Æ d[u] = d(s, u). from s using paths of up to i edges. 7. u = extract-min(Q) Part 2: Checking for negative-weight cycles. 8. for each neighbor of u Procedure relax (x,y) 9. relax (u, v) d[y] = min{d[y], d[x] + wt(x,y)} 10. endfor 11. endwhile 19 20

Bellman-Ford SSSP Algorithm Bellman-Ford SSSP Algorithm

Algorithm SSSP-Bellman-Ford (G, s) Algorithm SSSP-Bellman-Ford (G, s) 1. d[s] = 0 Note: 1. d[s] = 0 2. for each v ¹ s 2. for each v ¹ s This is same Procedure relax (x,y) 3. d[v] = ¥ “decrease-key” 3. d[v] = ¥ d[y] = min{d[y], d[x] + wt(x,y)} 4. for i = 1 to |V| - 1 operation as in 4. for i = 1 to |V| - 1 5. for each edge (u, v) Î E Dijkstra’s alg. 5. for each edge (u, v) Î E 6. d[v] = min{d[v], d[u] + wt(u,v)} 6. relax(u,v) 7. for each edge (u, v) Î E 7. for each edge (u, v) Î E 8. if d[v] > d[u] + wt(u, v) 8. if d[v] > d[u] + wt(u, v) 9. return false 9. return false 10. return true 10. return true

21 22

Correctness of Bellman-Ford Correctness of Bellman-Ford Algorithm Algorithm • Theorem: • Theorem Suppose there are no negative-weight cycles in G. Suppose there are no negative-weight cycles in G. Then the Bellman-Ford algorithm returns true with After |V| - 1 iterations of the for loop, d[v] = d(s,v) for all d[v] = d(s,v) for all vertices v reachable from s. If there is vertices v that are reachable from s. a negative cycle, Bellman-Ford returns false.

• Proof: -If v is reachable from s, then there is an acyclic path from • Proof: -For the first case, see the previous theorem. s to v, say s = u0, u1, u2, ..., uk = v, where k < |V|. -There are k edges in this path. -For the second case, there is a path of length at least -By the path relaxation property, after the first pass, u , u is |V| whose cost is less than the shortest acyclic path, so 0 1 we have d[v] > d[u] + wt(u,v) for some u,v in the path. a shortest path; after the second pass, u 0, u1, u2 is a shortest path; after k passes, u0, u1, u2, ..., uk is a shortest path.

23 24

4 Complexity of Bellman-Ford SSSPs in DAGs Algorithm • If the graph is a DAG, we can use a topological sort • Initialization = O(V) on the vertices and compute the shortest path from a single source in O(V + E) time • decrease-key is called (|V| - 1) × |E| times

• Test for any negative-weight cycle = O(E) input: directed acyclic graph (DAG) output: ordering of nodes s.t. if (u,v) Î E, then u comes before v in ordering • Total: O(VE) -- so more expensive than Topological-Sort (G) Dijkstra's, but also more general, since it works 1. call DFS(G) to compute finishing times f[v] for each v for graphs with negative edge weights. 2. as each vertex is finished, insert it at front of a linked list 3. return the linked list of vertices

25 26

Alternate topological sort Property of a DAG

• The in-degree of vertex u is the number of incoming • Why does the previous algorithm work? edges incident on u. The out-degree of vertex u is Claim: a DAG G must have some vertex with no the number of outgoing edges incident on u. incoming edges. Why? Suppose, in contradiction, that every vertex in G has at least input: directed acyclic graph (DAG) one incoming edge. Choose a vertex v0. Trace the edge output: ordering of nodes s.t. if (u,v) Î E, then u comes before incoming at v 0 to its source, v 1. Since v1 must have an v in ordering incoming edge, we can follow that edge to its source, v2. If we Topological-Sort (G) continue backtracking in this fashion, since there are a finite 1. while V is not empty number of vertices, we will eventually return to a previously 2. remove a vertex u of in-degree 0 and all its outgoing edges. visited vertex. At this point, we will have discovered a cycle, 3. insert u at the tail of a linked list of vertices which is a contradiction to our assumption that G is a DAG. Therefore, a DAG has at least one vertex with no incoming edge (a similar argument holds for outgoing edges). How could we implement this to run in O(V + E) time?

27 28

SSSPs in DAGs SSSPs in DAGs

Note: This algorithm will work with negative weight edges • In a topologically sorted list of vertices, all edges will go from in the DAG. left to right. Once all outgoing edges at a node u have been input: directed acyclic graph (DAG) relaxed, u will never be revisited. Since we process the nodes output: ordering of nodes s.t. if (u,v) Î E, then u comes before in topologically-sorted order, the nodes at 1 hop from s will be v in ordering finished before those at 2 hops, etc. By the path-relaxation DAG-Shortest-Paths (G, s) property, all shortest paths will be found. 1. Topological-Sort(G) 2. d[s] = 0 5 3. for each v ¹ s m s q r u v z -3 4. d[v] = ¥ ¥ 0 2 ¥ ¥s ¥s ¥s ¥s 5. for each vertex u, taken in topologically sorted order 7 6. for each vertex v adjacent to u 1 6 7. relax(u, v) List the shortest path distance from s to every other node in the above graph. 29 30

5