Directed Graphs

Department of General and Computational Linguistics

Directed Graphs

Data Structures and Algorithms for CL III, WS 2019-2020

Corina Dima [email protected] MICHAEL GOODRICH Data Structures & Algorithms in Python ROBERTO TAMASSIA MICHAEL GOLDWASSER

14.1-3 Digraph Properties and Traversals 14.4 Transitive Closure 14.5 Topological Sorting

Directed Graphs | 2 Directed Graphs

• A directed graph or digraph ! is a set " of vertices b – together with a collection # of pairwise a connections between vertices from ", called edges where all the edges in the graph are directed c • An edge e = (', )) is directed from ' to ) if the pair (', )) is ordered, with ' preceding ) e u v • The first endpoint of a directed edge is called the origin, and the second endpoint is called the destination of the edge

- ' is the origin, ) is the destination of edge ,

Directed Graphs | 3 Directed Graphs - Terminology

• the outgoing edges of a vertex are the edges whose origin is that vertex - The outgoing edges of vertex C are 1 and 2 • The incoming edges of a vertex are the edges whose E 3 destination is that vertex 1 - The incoming edge of vertex C is 6 2 D • [ug] Two vertices ! and " are adjacent if there is an edge whose end vertices are ! and " 8 C 4 • [ug] An edge is called incident to a vertex if the vertex is one of the edge’s endpoints 5 B 6 • [ug] The degree of a vertex, deg("), is the number of incident 7 edges of " A • The in-degree and out-degree of a vertex " are the number of incoming and outgoing edges of ", - indeg("), outdeg(") - indeg(C) = 1, outdeg(C) = 2

Directed Graphs | 4 Directed Graph – Terminology (cont’d)

• A directed path is a path such that all edges are directed E and are traversed along their direction 3 1 P - " = (%, 7, (, 4, *, 3, ,) is a directed simple path 2 D

8 C 4

• A directed cycle is a cycle such that all edges are 6 5 B directed and are traversed along their direction 7 - . = ,, 8, %, 6, ., 1, , is a directed simple cycle C A

Directed Graphs | 5 Directed Graph – Terminology (cont’d) D

• A directed graph is called acyclic if it has E no directed cycles A F

- #⃗ is an acyclic graph #⃗ C • A directed graph is strongly connected if B for any two vertices $ and % of #⃗, $ G reaches % and % reaches $ - & is a strongly connected graph E

& C B G

Directed Graphs | 6 Directed Graph Properties

• Property 1. If "⃗ is a directed graph with # edges and vertex set $, then

% indeg(/) = % outdeg / = # &∈( & ∈( • Justification. In a directed graph each edge (5, /) contributes:

- One unit to the out-degree of its origin 5 - One unit to the in-degree of its destination / - The total contribution is equal to the number of edges

Directed Graphs | 7 Directed Graph Properties (cont’d)

• Property 2. If "⃗ is a simple directed graph with # vertices and $ edges, then

$ ≤ #(# − 1) • Justification. The graph is simple → it has no parallel edges or self-loops.

- No two edges can have the same origin and destination - There are no self-loops (edges with the same origin and destination) - Therefore the maximum degree of a vertex is n-1 - It follows from property 1 that $ ≤ #(# − 1)

Directed Graphs | 8 The Graph ADT

Graphs | 9 The Graph ADT – for directed graphs

• A graph is a collection of vertices and edges • Can be modelled as a combination of three data types: Vertex, Edge and Graph

• class Vertex

- Lightweight object storing the information provided by the user - The element() method provides a way to retrieve the stored information • class Edge

- Another lightweight object storing an associated object - the cost - The element() method provides a way to retrieve the cost of the edge

- endpoints() method: returns a tuple (", $) such that vertex " is the origin of the edge and vertex $ is the destination

- opposite(v) method: assuming vertex $ is one endpoint of an edge (either origin or destination), return the other endpoint

Graphs | 10 The Graph ADT – for directed graphs (cont’d)

• class Graph: can be either undirected or directed – flag provided to the constuctor vertex_count() returns the number of vertices of the graph vertices() returns an iteration of all the vertices of the graph edge_count() returns the number of edges of the graph edges() returns an interation of all the edges of the graph get_edge(u,v) returns the edge from vertex ! to vertex ", if one exists, otherwise None degree(v, out=True) returns the number of outgoing/incoming edges incident to vertex ", as designated by the optional parameter out incident_edges(v, out=True) returns outgoing edges incident to vertex " by default; report incoming edges if out=False insert_vertex(v, x=None) create and return a new Vertex storing element # insert_edge(u,v, x=None) create and return a new Edge from vertex ! to vertex ", storing # remove_vertex(v) remove vertex " and all its incident edges from the graph remove_edge(e) remove edge $ from the graph Graphs | 11 Traversals in a Directed Graph

Directed Graphs | 12 Traversals in a Directed Graph

• The DFS and BFS techniques presented in the previous lecture for undirected graphs can be used to perform traversals of directed graphs • The difference is that this time the edges can only be traversed from origin to destination, but not in the opposite direction • As in the undirected graphs case, traversal algorithms can solve interesting problems dealing with reachability in a directed graph "⃗:

- Computing a directed path from vertex # to vertex $, or report that no such path exists

- Finding all the vertices of "⃗ that are reachable from a given vertex %

- Determine whether "⃗ is acyclic

- Determine whether "⃗ is strongly connected

Directed Graphs | 13 DFS in a Directed Graph

Directed Graphs | 14 DFS in a Directed Graph - Example visited discovery edge D None D

E A F

C B G Current vertex: D Edges to consider: to A, F, G • Start from vertex D, which is marked as visited (red) • Assume that the outgoing edges of a vertex are considered in alphabetical order – e.g. for D: A, F, G

Directed Graphs | 15 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A)

E A F

C B G Current vertex: D Edges to consider: to F, G

Directed Graphs | 16 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A)

E A F

C B G Current vertex: A Edges to consider: - (no outgoing edges) Finished A, backtrack to D

Directed Graphs | 17 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E A F

C B G Current vertex: D Edges to consider: to F, G

Directed Graphs | 18 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E A F

C B G Current vertex: F Edges to consider: to A, C, D, G

Directed Graphs | 19 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F

C B G Current vertex: F Edges to consider: to C, D, G

Directed Graphs | 20 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F

C B G Current vertex: C Edges to consider: to A, B, E

Directed Graphs | 21 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B)

C B G Current vertex: C Edges to consider: to B, E

Directed Graphs | 22 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C B G Current vertex: B Edges to consider: to E

Directed Graphs | 23 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C B G Current vertex: E Edges to consider: to C, G

Directed Graphs | 24 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: E Edges to consider: to G

Directed Graphs | 25 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: G Edges to consider: to B, C

Directed Graphs | 26 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: G Edges to consider: to C

Directed Graphs | 27 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: G Edges to consider: - Finished G, backtracking to E

Directed Graphs | 28 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: E Edges to consider: - Finished E, backtracking to B

Directed Graphs | 29 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: B Edges to consider: - Finished B, backtracking to C

Directed Graphs | 30 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: C Edges to consider: E

Directed Graphs | 31 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: C Edges to consider: - Finished C, backtracking to F

Directed Graphs | 32 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: F Edges to consider: to D, G

Directed Graphs | 33 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: F Edges to consider: to G

Directed Graphs | 34 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: F Edges to consider: - Finished F, backtracking to D

Directed Graphs | 35 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: D Edges to consider: to G

Directed Graphs | 36 DFS in a Directed Graph - Example visited discovery edge D None D A (D,A) F (D,F) E C (F,C) A F B (C,B) E (B,E) C G (E,G) B G Current vertex: D Edges to consider: - Finished D – start vertex – stop.

Directed Graphs | 37 DFS Traversal – discovery edges

D visited discovery edge E D None A F A (D,A) F (D,F) C C (F,C) B (C,B) B E (B,E) G G (E,G)

Directed Graphs | 38 visited discovery edge DFS Tree D None A (D,A) D F (D,F) D C (F,C) A F B (C,B) E (B,E) E C A F G (E,G)

B C B G E

Tree edge discovery edge G back edge (connects a vertex to an ancestor in the DFS tree) Nontree edges forward edge (connects a vertex to a descendant in the DFS tree) cross edge (connects a vertex to another vertex that is neither its ancestor nor its descendant) Directed Graphs | 39 Properties of a DFS in a Digraph

• Proposition. A depth-first search in a directed graph "⃗ starting at a vertex # visits all the vertices of "⃗ that are reachable from #. Also, the DFS tree contains directed paths from # to every vertex reachable from #.

• Justification. Consider $% to be the subset of vertices of "⃗ visited by a DFS starting at #. Need to show that $% contains # and every vertex reachable from #.

- Suppose that there is a vertex & reachable from # that is not in $% - Consider a directed path from # to & and let ((, *) be the first edge on this path that goes out of $% → ( ∈ $%, v ∉ $% - When DFS reaches (, all outgoing edges of ( are explored – thus it must also reach * → then * ∈ $% (contradiction) - Second property – induction: each time a discovery edge ((, *) is identified, since ( was previously discovered, there exists a directed path from # to (; by appending the discovery edge to the existing path, a directed path from # to * is obtained

Directed Graphs | 40 Graph Class, part 1

Graphs | 41 Graph Class, part 2

Graphs | 42 Depth-First Search in a Directed Graph – Python Implementation

Directed Graphs | 43 DFS in a Directed Graph – Running Time

• Consider "⃗, a directed graph with # vertices and $ edges. A DFS traversal of "⃗ can be performed in %(# + $) time.

- provided the graph is represented using a data structure where the incident edges of a vertex (both incoming and outgoing) can be iterated in %(deg , ) time, and finding the opposite vertex takes %(1) time - The DFS procedure will be called at most once for every vertex of the graph - Each edge will be examined at most once in a directed graph, from its origin vertex

Directed Graphs | 44 Problems Solved using a DFS Traversal in a Directed Graph

1. Computing a directed path from vertex ! to vertex ", or report that no such path exists

2. Testing whether $⃗ is strongly connected

3. Computing the set of vertices of $⃗ that are reachable from a given vertex %

4. Computing a directed cycle in $⃗, or reporting that $⃗ is acyclic

5. Computing the transitive closure of $⃗

Directed Graphs | 45 1. Compute a Directed Path from ! to "

• Assume DFS was performed for the digraph • Exactly the same algorithm as in the undirected case - build the path from end to start

Directed Graphs | 46 2. Testing whether ! is strongly connected

• That is, if for every pair of vertices " and #, " reaches # and # reaches "

• Idea: start an independent DFS traversal from each vertex of %⃗. If the discovered dictionary of every of these independent DFS traversals has length & (the number of vertices), then %⃗ is strongly connected • Running time: ?

Directed Graphs | 47 2. Testing whether ! is strongly connected • That is, if for every pair of vertices " and #, " reaches # and # reaches "

- Start with doing a DFS from an arbitrary vertex ,. - If the discovered dictionary does not contain all the vertices – the digraph is not strongly connected - stop.

- Otherwise, construct a copy of the graph %⃗, but where the orientation of each edge is reversed. Perform a DFS on the reversed graph. If discovered contains all vertices – the digraph is strongly connected. Otherwise it is not. - Runs in '(& + *) time Directed Graphs | 48 3. Computing the Vertices Reachable from a Given Start Vertex !

• Perform a DFS traversal #⃗ starting from $

• The set of vertices reachable from $ are the keys of the discovered dictionary

Directed Graphs | 49 4. Compute a Directed Cycle in "⃗, or Report that "⃗ is Acyclic

• The DFS procedure was already performed for the graph " • A cycle exists if and only if a back edge exists with respect to the DFS traversal of that graph • In a directed graph DFS traversal, there are multiple types of nontree edges: back edges, forward edges and cross edges • When a directed edge is explored, leading to a previously visited vertex, keep track of whether that vertex is an ancestor of the current vertex • To obtain the cycle, take the back edge from the descendant to the ancestor and then follow DFS tree edges back to the descendant

Directed Graphs | 50 4. Compute a Directed Cycle in "⃗, or Report that "⃗ is Acyclic

D A F

E A F C

C B B G E

discovery edge back edge G forward edge cross edge Directed Graphs | 51 5. Computing the Transitive Closure of "⃗

• Particular graph applications benefit from being able to answer reachability questions more efficiently

- e.g. a service that computes driving destinations from point # to point $; a first step is to find about if $ can be reached starting from # • Precompute a more efficient representation for the graph that can answer such queries, and then reuse it for all the reachability queries

• A transitive closure of a directed graph "⃗ is itself a directed graph "⃗∗ such that

- the vertices of "⃗∗ are the same vertices of "⃗ and

- "⃗∗ has an edge ((, *) whenever "⃗ has a directed path from ( to *, including the case where ((, *) is an edge of the original graph "⃗

Directed Graphs | 52 5. Computing the Transitive Closure of "⃗: Method A

• If "⃗ is a graph with # vertices and $ edges represented as an adjacency list or an adjacency map, then • Compute the transitive closure by making # sequential DFS traversals of the graph, one starting at each vertex • E.g. the DFS starting at vertex % will determine all vertices reachable from % – the transitive closure includes all the edges starting at % to each of the vertices that are reachable from % • Thus computing the transitive closure of a digraph using several DFS traversals can be done in ? time

Directed Graphs | 53 5. Computing the Transitive Closure of "⃗: Method A

Directed Graphs | 54 5. Computing the Transitive Closure of "⃗: Method B

• If "⃗ is a graph with # vertices and $ edges represented by a data structure that supports O(1) lookup for get_edge(u,v) (e.g. an adjacency matrix), then • Compute the transitive closure of "⃗ in a series of rounds

- Initially, "⃗%= "⃗ - Define an arbitrary order over the vertices of "⃗, &', &), … , &+ i - Compute the rounds, starting with round 1

- At round ,, construct a directed graph "⃗ starting - j with "⃗- = "⃗-/', and adding to "⃗- the directed edge (&1, &2) if "⃗-/' contains both edges (&1, &-) and (&-, &2). • This method of computing the transitive closure of a k digraph is known as the Floyd-Warshall algorithm

Directed Graphs | 55 Floyd-Warshall Algorithm - Pseudocode

Directed Graphs | 56 Floyd-Warshall Algorithm – Running Time

• If the data structure supports get_edge and insert_edge in O(1) time • The main loop, indexed by !, is executed " times • The inner loop contains of #("%) pairs of vertices, for each of which an # 1 computation is performed • Total running time of the algorithm: #("() • Asymptotically, this is not better than running DFS " times, which is #("% + "*) • Floyd-Warshall matches the asymptotic bounds of repeated DFS when the graph is dense

Directed Graphs | 57 Floyd-Warshall Algorithm - Example i k j

!( G

!% D !# B F !'

A !$

!" E !&

*⃗ = *⃗, Directed Graphs | 58 Floyd-Warshall Algorithm - Example i k j 1 1 1 i=j, continue

!( G

!% D !# B F !'

A !$

!" E !&

*⃗" Directed Graphs | 59 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 j=k, continue !( G

!% D !# B F !'

A !$

!" E !&

*⃗" Directed Graphs | 60 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 i=j, continue !( G

!% D !# B F !'

A !$

!" E !&

*⃗" Directed Graphs | 61 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( + ≠ - ≠ . G 3

!% D !# B F Does *⃗ have the edges (!#, !") !' and (!", !$)? No, it doesn’t have either, continue. C

A !$

!" E !&

*⃗" Directed Graphs | 62 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 4 + ≠ - ≠ . !% D ! B F # Does *⃗ have: !' - (!#, !") - no - (!", !%) – yes C Continue.

A !$

!" E !&

*⃗" Directed Graphs | 63 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 4 !% 5 + ≠ - ≠ . D !# B F !'

⃗ C Does * have: - (!#, !") - no - (! , ! ) - no A !$ " & Continue. !" E !& 2-1-6, 2-1-7 will also not add edges because (!#, !") does not exist. *⃗" Directed Graphs | 64 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 + ≠ - ≠ . !( G 3

!% D !# B F !'

⃗ C Does * have: - (!$, !") - yes - (! , ! ) - no A !$ " # Continue. !" E !&

*⃗" Directed Graphs | 65 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( 3 3 + = -, G continue

!% D !# B F !'

A !$

!" E !&

*⃗" Directed Graphs | 66 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 3 4 / ≠ 1 ≠ 2 !% D !# B F !'

⃗ C Does * have: - (!$, !") - yes - (! , ! ) - yes A !$ " %

!" E A direct edge (!$, !%) already !& exists, continue.

*⃗" Directed Graphs | 67 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 3 4 !% 5 D / ≠ 1 ≠ 2 !# B F !'

⃗ C Does * have: - (!$, !") - yes - (! , ! ) - no A !$ " & ! E " Continue. 3-1-6 and 3-1-7 wont !& add edges, since 1-6 and 1-7 do not exist.

*⃗" Directed Graphs | 68 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 3 4 4 !% 5 D 6 !# B F !' 7

No edges added starting with C 4-1.

A !$

!" E !&

*⃗" Directed Graphs | 69 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 3 4 4 !% 5 5 D 6 !# B F !' 7

The edge 5-1-4 can be added, C a direct edge from 5 to 4 does not exist. A !$

!" E !&

*⃗" Directed Graphs | 70 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 3 4 4 !% 5 5 D 6 6 !# B F !' 7

No edges added starting with C 6-1.

A !$

!" E !&

*⃗" Directed Graphs | 71 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 3 4 4 !% 5 5 D 6 6 !# B F !' 7 7

No edges added starting with C 7-1.

A !$

!" E !&

*⃗" Directed Graphs | 72 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 !( G 3 4 !% 5 D 6 !# B F !' 7

C The are no outgoing edges for !# - so no edges are added to the transitive closure. A !$

!" E !&

*⃗# Directed Graphs | 73 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 2 !( G 3 3 3 4 4 !% 5 5 D 6 6 !# B F !' 7 7

4-3-1, add edge from 4 to 1 C 4-3-2, add edge from 4 to 2 5-3-1, a direct 5-1 edge exists A !$ E 5-3-2, add edge from 5 to 2 !" 5-3-4, a direct 5-4 edge exists !& 6-3-1, add edge from 6 to 1 6-3-2, a direct 6-2 edge exists 6-3-4, add edge from 6 to 4 *⃗$ Directed Graphs | 74 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 2 !( G 3 3 3 4 4 4 !% 5 5 D 6 6 !# B F !' 7 7 1-4-2, add edge from 1 to 2 C 1-4-3, add edge from 1 to 3 3-4-1, a direct 3-1 edge exists A !$ 3-4-2, a direct 3-2 edge exists

!" E 5-4-1, a direct 5-1 edge exists 5-4-2, a direct 5-2 edge exists !& 5-4-3, a direct 5-3 edge exists 6-4-1, a direct 6-1 edge exists *⃗% 6-4-2, a direct 6-2 edge exists 6-4-3, a direct 6-3 edge exists Directed Graphs | 75 Floyd-Warshall Algorithm - Example i k j 1 1 1 2 2 2 !( G 3 3 3 4 4 4 !% 5 5 5 D 6 6 !# B F !' 7 7 6-5-1, a direct 6-1 edge exists C 6-5-2, a direct 6-2 edge exists 6-5-3, a direct 6-3 edge exists A !$ 6-5-4, a direct 6-4 edge exists

!" E 7-5-1, add edge from 7 to 1 7-5-2, add edge from 7 to 2 !& 7-5-3, add edge from 7 to 3 7-5-4, add edge from 7 to 4 ⃗ *& *⃗&= *⃗'= *⃗(, stop. Directed Graphs | 76 Floyd-Warshall Algorithm – Python Implementation

Directed Graphs | 77 BFS in a Directed Graph

Directed Graphs | 78 BFS in a Directed Graph - Example visited discovery edge D None D

E A F

Directed Graphs | 79 BFS in a Directed Graph - Example visited discovery edge Level 0 D None D A (D,A) F (D,F) E G (D,G) A F

C B G Current level: D Edges to consider: to A, F, G • Start from vertex D, which is marked as visited (red) • Assume that the outgoing edges of a vertex are considered in alphabetical order – e.g. for D: A, F, G

Directed Graphs | 80 BFS in a Directed Graph - Example visited discovery edge Level 0 D None D A (D,A) F (D,F) E G (D,G) A F C (F,C) B (G,B) C B G Current level: A, F, G Level 1 Edges to consider: (F, A), (F,C), (F,D), (F,G), (G,B), (G,C)

Directed Graphs | 81 BFS in a Directed Graph - Example visited discovery edge Level 0 D None D A (D,A) F (D,F) E G (D,G) A F C (F,C) B (G,B) C E (B, E) B G Current level: B, C Level 2 Level 1 Edges to consider: (B,E), (C, A), (C,B), (C,E)

Directed Graphs | 82 BFS in a Directed Graph - Example visited discovery edge Level 0 D None D A (D,A) F (D,F) E G (D,G) A F Level 3 C (F,C) B (G,B) C E (B, E) B G Current level: E Level 2 Level 1 Edges to consider: (E,C), (E,G) No new nodes for next level, BFS stop.

Directed Graphs | 83 Topological Ordering in Directed Acyclic Graphs (DAGs)

Directed Graphs | 84 Directed Acyclic Graphs (DAGs)

• directed acyclic graphs are directed graphs without directed cycles • DAGs are encountered in many practical applications

- Prerequisites between the courses for a degree program - Inheritance between classes of an object-oriented program - Scheduling constrains between the tasks of a project

Directed Graphs | 85 https://www.xkcd.com/754/

Directed Graphs | 86 Introduction to Introduction to Computational Mathematics Linguistics for Linguists

Text Programming 1 Technology

Grammar Formalisms

Programming 2

Data Structures and Algorithms for CL

Parsing Directed Graphs | 87 Introduction to Introduction to Computational Mathematics Linguistics for Linguists A B

Text Programming 1 Technology C

Grammar D Formalisms F E

Programming 2 G

Data Structures and Algorithms H for CL

Parsing Directed Graphs | 88 Topological Ordering

• #⃗ is a directed graph with $ vertices

• A topological ordering of #⃗ is an ordering %&, %(, … , %* of the vertices of #⃗ such that for every edge (%,, %-) of #⃗, it is the case that / < 1. • A topological ordering is an ordering such that any directed path in #⃗ traverses vertices in an increasing order • A directed graph might have more than one topological orderings

Directed Graphs | 89 1 4 Introduction to Introduction to Computational Alterante Topological Orderings (1) Mathematics Linguistics for Linguists

2 Text Programming 1 Technology

5 Grammar Formalisms 3 6

Programming 2

Data Structures and Algorithms for CL 7 8

Parsing Directed Graphs | 90 2 1 Introduction to Introduction to Computational Alterante Topological Orderings (2) Mathematics Linguistics for Linguists

3 Text Programming 1 Technology

4 Grammar Formalisms 6 5

Programming 2

Data Structures and Algorithms for CL 7 8

Parsing Directed Graphs | 91 When does a Directed Graph Have a Topological Ordering?

• Proposition. "⃗ has a topological ordering if and only if it is acyclic. • Justification. - ⟹ Suppose "⃗ is topologically ordered. Assume that "⃗ has a cycle made of the edges ⃗ %&', %&) , %&), %&* , … , (%&-.), %&'). But " has a topological ordering, meaning that /0 < /2 < ⋯ < /452 < /0 - impossible, therefore "⃗ must be acyclic.

Directed Graphs | 92 When does a Directed Graph Have a Topological Ordering?

• Proposition. "⃗ has a topological ordering if and only if it is acyclic. • Justification. - ⟸ Suppose "⃗ is acyclic. A topological ordering can be built using the following algorithm:

• "⃗ is acyclic, therefore "⃗ must have a vertex with no incoming edges, %&

• if a vertex like %& would not exist, we would eventually encounter a vistied vertex when tracing a path from the start index - would contradict "⃗ being acyclic

• thus by removing %& and its outgoing edges we obtrain another acyclic graph; this graph has, again, a vertex %' with no incoming edges • repeat the process of removing the vertex with no incoming edges until "⃗ is empty

• %&, %', … , %* form an ordering of the vertices in "⃗; because of how it was constructed, if

%+, %, is an edge in "⃗, %+ must be deleted before %, can be deleted – thus - < / and %&, %', … , %* is a topological ordering

Directed Graphs | 93 • topological sorting is an Topological Sorting algorithm for computing a topological ordering of a directed graph

• incount is a dict, maps - vertex ! to - number of incoming edges to ! (excluding those from vertices that have been added to the topological order) • also tests if #⃗ is acyclic: if the algorithm terminates without ordering all the vertices, then the subgraph of vertices that have not been ordered must contain a cycle Directed Graphs | 94 Topological Sorting - Example

0 0 • in the left box, the current incount of each of the vertices in the graph A B • in the right box, the index of the 1 vertex in the topological ordering C D 3 F 1 E 2

2 G

H 3

Directed Graphs | 95 Topological Sorting - Example topo ready 0 0 B A B A 1 C D 3 F 1 E 2 • len(ready) > 0 == True

2 G

H 3

Directed Graphs | 96 Topological Sorting - Example topo ready 0 0 A B A B 1 C D 3 F 1 E 2 • pop A from ready, append it to topo • decrease the incount of all 2 G neighbours of A (on outgoing edges): C, D H 3

Directed Graphs | 97 Topological Sorting - Example topo ready 0 1 0 A B A B C 0 C D 2 F 1 E 2 • if after the decrease any of the vertices have an incount of 0, add it to ready 2 G • pop C, add it to topo H 3

Directed Graphs | 98 Topological Sorting - Example topo ready 0 1 0 A B A B C E 0 2 C D 1 F 0 E 2 • decrease the incount of D, E and H • add E to ready, since its incount is 0 2 G

H 2

Directed Graphs | 99 Topological Sorting - Example topo ready 0 1 0 A B A B C 0 2 E C D 1 F 0 3 E 2 • pop E from ready, add it to topo

2 G

H 2

Directed Graphs | 100 Topological Sorting - Example topo ready 0 1 0 A B A B C 0 2 E C D 1 F 0 3 E 2 • decrease the incount of G

1 G

H 2

Directed Graphs | 101 Topological Sorting - Example topo ready 0 1 0 A B A B C 0 2 E C D 1 F 0 3 E 2 • pop B from ready, add it to topo

1 G

H 2

Directed Graphs | 102 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D 1 F 0 3 E 2 • pop B from ready, add it to topo

1 G

H 2

Directed Graphs | 103 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D 0 F 0 3 E 1 • decrease the incount of D and F

1 G

H 2

Directed Graphs | 104 Topological Sorting - Example topo ready 0 1 0 4 A D A B C 0 2 E C B D 0 F 0 3 E 1 • add D to ready, since its incount is 0

1 G

H 2

Directed Graphs | 105 Topological Sorting - Example topo ready 0 1 0 4 A D A B C 0 2 E C B D 0 F 0 3 E 1 • pop D from ready, add it to topo

1 G

H 2

Directed Graphs | 106 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F 0 3 E 1 • decrement the incount of F

1 G

H 2

Directed Graphs | 107 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F 0 3 E 0 • decrement the incount of F

1 G

H 2

Directed Graphs | 108 Topological Sorting - Example topo ready 0 1 0 4 A F A B C 0 2 E C B D D 0 5 F 0 3 E 0 • add F to ready, its incount is 0

1 G

H 2

Directed Graphs | 109 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 • pop F from ready, add to topo

1 G

H 2

Directed Graphs | 110 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 • decrement the incounts of G and H

1 G

H 2

Directed Graphs | 111 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 • decrement the incounts of G and H

0 G

H 1

Directed Graphs | 112 Topological Sorting - Example topo ready 0 1 0 4 A G A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 • add G to ready, its incount is 0

0 G

H 1

Directed Graphs | 113 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 G

• pop G from ready, add to topo 0 7 G

H 1

Directed Graphs | 114 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 G

• decrement the incount of H 0 7 G

H 1

Directed Graphs | 115 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 G

• decrement the incount of H 0 7 G

H 0

Directed Graphs | 116 Topological Sorting - Example topo ready 0 1 0 4 A H A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 G

• add H to ready, its incount is 0 0 7 G

H 0

Directed Graphs | 117 Topological Sorting - Example topo ready 0 1 0 4 A A B C 0 2 E C B D D 0 5 F F 0 3 E 0 6 G H 0 7 G • pop H from ready, add to topo • H has no outgoing nodes H • ready is empty – stop. • the topological ordering of the 0 8 vertices is obtained in topo

Directed Graphs | 118 Introduction to Computational Linguistics

Programming 1 topo A C Programming 2 E Introduction to B Mathematics for Linguists D F Text Technology G

H Grammar Formalisms

Data Structures and Algorithms for CL

Parsing Directed Graphs | 119 Topological Sorting - Performance

• topological sorting runs in !(# + %) time, using !(#) auxiliary space • it either computes a topological ordering of (⃗ or fails to include some vertices – meaning that (⃗ has a directed cycle

Directed Graphs | 120 Thank you.