Shortest path problems and algorithms
1. Shortest path problems . Shortest path problem . Single-source shortest path problem . Single-destination shortest path problem . All-pairs shortest path problem
2. Algorithms . Dijkstra’s algorithm for single source shortest path problem (positive weight) . Bellman–Ford algorithm for single source shortest path problem (allow negative weight) . Floyd–Warshall algorithm for all pairs shortest paths . Johnson's algorithm for all pairs shortest paths
CP412 ADAII 1. Shortest path problems
Shortest path problem: Input: weight graph G = (V, E, W), source vertex s and destination vertex t. Output: a shortest path of G from s to t.
Single-source shortest path problem: Output: shortest paths from source vertex s to all other vertices of G.
Single-destination shortest path problem: Output: shortest paths from all vertices of G to a single destination t.
All-pairs shortest path problem: Output: shortest paths between every pair of vertices u, v in the graph.
CP412 ADAII History of SPP and algorithms
Shimbel (1955). Information networks.
Ford (1956). RAND, economics of transportation.
Leyzorek, Gray, Johnson, Ladew, Meaker, Petry, Seitz (1957). Combat Development Dept. of the Army Electronic Proving Ground.
Dantzig (1958). Simplex method for linear programming.
Bellman (1958). Dynamic programming.
Moore (1959). Routing long-distance telephone calls for Bell Labs.
Dijkstra (1959). Simpler and faster version of Ford's algorithm.
CP412 ADAII Dijkstra’s algorithms
Edger W. Dijkstra’s quotes . The question of whether computers can think is like the question of whether submarines can swim. . In their capacity as a tool, computers will be but a ripple on the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind. . Do only what only you can do. (https://en.wikipedia.org/wiki/Turing_Award)
Dijkstra’s algorithm: solve the single-source shortest path problem GREEDY strategy . Starting from source s, set visited set R = {s} . Always pick the next closest vertex and add to R . Use priority queue to store unvisited vertices by distance from s . After deleteMin v, update distances of remaining vertices adjacent to v using decreaseKey
CP412 ADAII Example
4 12 8 7 1 8 2 7 3 1 2 3 4 9 4 9 2 0 2 4 4 14 0 11 8 14 4 0 11 8 4 7 7 6 8 6 10 8 1 7 6 5 7 6 5 0 1 2 8 1 2
4 12 4 12 8 7 1 2 3 1 8 2 7 3 4 9 4 9 0 2 0 2 4 14 4 0 11 15 8 4 0 11 15 8 14 4 7 7 6 8 1 8 6 10 7 6 5 0 7 6 5 8 1 9 2 8 1 9 2 11
4 CP412 ADAII 5 Example
4 12 12 25 8 7 4 1 2 3 1 8 2 7 3 4 9 2 4 9 0 0 2 21 15 4 14 4 4 0 11 8 0 11 15 8 14 4 7 7 8 6 10 8 6 10 7 6 5 7 6 5 8 1 2 1 2 9 11 8 9 11 4 12 19 4 12 19 1 8 2 7 3 1 8 2 7 3 4 9 21 4 9 0 2 0 2 21 4 4 0 11 14 8 14 4 0 11 14 8 14 4 7 7 8 6 10 8 6 10 7 6 5 7 6 5 1 2 1 2 8 9 11 8 9 11
4 CP412 ADAII 6 Example
12 19 4 4 12 19 8 2 7 3 1 1 8 2 7 3 4 9 2 21 4 9 0 0 2 21 14 4 14 4 4 0 11 8 0 11 14 8 14 4 7 7 8 6 10 8 6 10 6 5 7 7 6 5 8 1 9 2 1 2 11 8 9 11
Shortest Path Tree (SPT)
4 CP412 ADAII 7 Correctness of Dijkstra’s algorithm
Lemma: (optimal substructure property) A least-cost path from X to Y contains least-cost paths from X to every vertex on the path to Y. (prove by contradiction)
Let d(v) be the label found by the algorithm and let δ(v) be the shortest path distance from s-to-v. We want to show that d(v) = δ(v) for every vertex v at the end of the algorithm, showing that the algorithm correctly computes the distances. We prove this by induction on |R| via the following lemma
Lemma: For each x ∈ R, d(x) = δ(x).
CP412 ADAII Correctness of Dijkstra’s algorithm Proof by Induction on |R|. Base case (|R| = 1): Since R only grows in size, the only time |R| = 1 is when R = {s} and d(s) = 0 = δ(s), which is correct.
Suppose it is true for |R| < k. Now assume |R| = k. Let u be the last vertex added to R. We need only show that d(u) = δ(u). Suppose for a contradiction that the shortest path from s-to-u is Q and has length w(Q) < d(u). Let R0 = R \ {u}. Q starts in R0 and at some leaves R0 (to get to u which is not in R0 ). Let xy be the first edge along Q that leaves R0 . Let Qx be the s-to-x subpath of Q. Clearly: w(Qx) + w(xy) ≤ w(Q). Since d(x) is the length of the shortest s-to-x path, d(x) ≤ w(Qx), d(x) + w(xy) ≤ w(Q). Since y is adjacent to x, d(y) must have been updated by the algorithm, so d(y) ≤ d(x) + w(xy), and d(y) < d(u). Finally, since u was picked by the algorithm, u must have the smallest distance label: d(u) ≤ d(y). Combining these inequalities gives us the contradiction that d(u) < d(u). Therefore, no such shorter path Q must exist and so d(u) = δ(u).
CP412 ADAII Implementation and analysis
CP412 ADAII BuildHeap: O(|V|)
DeleteMin: O(|V| log |V|)
DecreaseKey: O(|E| log |V|)
Total running time: O(|E| log |V|)
CP412 ADAII What about graphs with negative edges?
Do not mark any vertex as “known”. Instead allow multiple updates.
CP412 ADAII Bellman-Ford Algorithm
The idea of Bellman-Ford algorithm is from dynamic programming.
Consider problem of shortest path using k number of edges for given source s and any destination v. Let dk(s, v) denote the weight of shortest path from s to v using at most k edges.
We can start by determining d0(s, v) for all v ϵ V, which is infinite for all vertices except s itself, i.e., d0(s, s) = 0, d0(s, v) = ∞, v≠s. Then we can use this to determine d1(s, v) for all v ϵ V. In general we want to determine dk+1(s, v) based on all dk(s, v).
Observation: if a dk+1(s, v) shortest path uses at most k edges, then dk+1(s, v) = dk(s, v), otherwise it uses k+1 edges, then it reaches of v’s neighbor x with k edges and plus edge (x, v). Therefore,
dk+1(s, v) = min{ dk(s, v), min{dk(s, x)+w(x, v) : x ϵ N-(v)} } CP412 ADAII Bellman-Ford algorithm and analysis
distance[|V|], predecessor[|V|], source vertex s // Step 1: initialize graph for each vertex v in V: // O(|V|) distance[v] := inf // At the beginning , all vertices have a weight of infinity predecessor[v] := null // And a null predecessor distance[s] := 0
// Step 2: relax edges repeatedly for i from 1 to |V|-1: for each edge (u, v) with weight w in edges: // O(|V||E|) if distance[u] + w < distance[v]: distance[v] := distance[u] + w predecessor[v] := u
// Step 3: check for negative-weight cycles for each edge (u, v) with weight w in edges: if distance[u] + w < distance[v]: error "Graph contains a negative-weight cycle" return distance[], predecessor[]
Time: O(|V||E|), Space : O(|V|) CP412 ADAII Example
CP412 ADAII Correctness of Bellman-Ford Algorithm
Theorem. Given a directed weighted graph G = (V, E, w), w : E → R, and a source s, the BellmanFord algorithm returns the shortest path length from s to every vertex or indicates that there is a negative weight cycle in G reachable from s.
Proof. By induction on the number of edges k in a path. The base case is correct since d0(s,s) = 0. For all v in V \ s, on each step a shortest (s, v) path with up to k +1 edges must consist of a shortest (s, u) path of up to k edges followed by a single edge (u, v). Therefore if we take the minimum of these we get the overall shortest path with up to k+1 edges.
The algorithm only proceeds to |V| rounds (step 3). If there is a reachable negative-weight cycle, error returns; otherwise shortest path to every v is simple and consist of at most n vertices and hence |V|-1 edges.
CP412 ADAII Floyd–Warshall algorithm Also known as Floyd's algorithm, the Roy–Warshall algorithm, the Roy–Floyd algorithm, is an algorithm for finding all pair shortest paths in a weighted graph with positive or negative edge weights (but with no negative cycles). Based on the idea of dynamic programming.
Let G be graph with vertices numbered 1 through N. Consider a function sp(i,j,k) that returns the shortest possible path from i to j using vertices only from the set {1,2, …,k} as intermediate points along the way. • Now, given this function, find the shortest path from each i to each j using only vertices in {1, …, k+1}. • For each of these pairs of vertices, the true shortest path could be either (1) a path that only uses vertices in the set {1,… ,k} or (2) a path that goes from i to k+1 and then from k+1 to j.
CP412 ADAII • The best path from i to j that only uses vertices 1 through k is defined by sp(i,j,k). If there were a better path from i to k+1 to j, then the length of this path is the concatenation of the shortest path from i to k+1 using vertices in {1, … ,k} and the shortest path from k+1 to j (also using vertices in {1, … ,k}.
Then sp(i, j, k+1) satisfies the following recursion formula:
sp(i,j,0) = w(i, j)
sp(i,j,k+1)=min{sp(i,j,k), sp(i,k+1,k)+sp(k+1,j,k) }.
CP412 ADAII 18 Floyd–Warshall algorithm and analysis D[|V|] [|V|] = { ∞ }
for each vertex v in V D[v][v] ← 0 for each edge (u,v) D[u][v] ← w(u,v) // the weight of the edge (u,v) // O(|V|2)
for k from 1 to |V| for i from 1 to |V| for j from 1 to |V| if D[i][j] > D[i][k] + D[k][j] D[i][j] ←D[i][k] + D[k][j] // O(|V|3) end if
Time: O(|V|3), Space: O(|V|2) CP412 ADAII 19 Floyd–Warshall algorithm and recovery path D[|V|] [|V|] = { ∞ }, n[|V|] [|V|] = { null } FloydWarshall() for each edge (u,v) D[u][v] ← w(u,v) n[u][v] ← u for k from 1 to |V| // standard Floyd-Warshall implementation for i from 1 to |V| for j from 1 to |V| if D[i][k] + D[k][j] < D[i][j] then D[i][j] ← D[i][k] + D[k][j] n[i][j] ← n[i][k] procedure Path(u, v) if n[u][v] = null then return [] path = [u] while u ≠ v u ← n[u][v] path.append(u) return path
CP412 ADAII 20 Example
Initialization: (k = 0)
CP412 ADAII 21 Iteration 1: (k = 1) Shorter paths from 2 ↝ 3 and 2 ↝ 4 are found through vertex 1
Iteration 2: (k = 2) Shorter paths from 4 ↝ 1, 5 ↝ 1, and 5 ↝ 3 are found through vertex 2
CP412 ADAII 22 Iteration 3: (k = 3) No shorter paths are found through vertex 3
CP412 ADAII 23 Iteration 4: (k = 4) Shorter paths from 1 ↝ 2, 1 ↝ 3, 2 ↝ 3, 3 ↝ 1, 3 ↝ 2, 5 ↝ 1, 5 ↝ 2, 5 ↝ 3, and 5 ↝ 4 are found through vertex 4
Iteration 5: (k = 5) No shorter paths are found through vertex 5
CP412 ADAII 24