Graph Algorithms
Graphs and Graph Repre- sentations
Graph Graph Algorithms Traversals COMP4128 Programming Challenges Directed Graphs and Cycles
Strongly Connected Components
Example: School of Computer Science and Engineering 2SAT UNSW Australia Minimum Spanning Trees Table of Contents 2
Graph Algorithms
1 Graphs and Graph Representations Graphs and Graph Repre- sentations 2 Graph Graph Traversals Traversals
Directed Graphs and 3 Directed Graphs and Cycles Cycles
Strongly Connected 4 Strongly Connected Components Components
Example: 2SAT 5 Example: 2SAT Minimum Spanning Trees 6 Minimum Spanning Trees Graphs 3
Graph Algorithms
Graphs and Graph Repre- sentations A graph is a collection of vertices and edges connecting Graph pairs of vertices. Traversals
Directed Generally, graphs can be thought of as abstract Graphs and Cycles representations of objects and connections between those
Strongly objects, e.g. intersections and roads, people and Connected Components friendships, variables and equations. Example: Many unexpected problems can be solved with graph 2SAT techniques. Minimum Spanning Trees Graphs 4
Graph Algorithms
Graphs and Graph Repre- sentations Graph Many different types of graphs Traversals Directed graphs Directed Graphs and Acyclic graphs, including trees Cycles Weighted graphs Strongly Connected Flow graphs Components Other labels for the vertices and edges Example: 2SAT
Minimum Spanning Trees Graph Representations 5
Graph Algorithms
Graphs and Graph Repre- sentations
Graph You’ll usually want to either use an adjacency list or an Traversals adjacency matrix to store your graph. Directed Graphs and An adjacency matrix is just a table (usually implemented Cycles as an array) where the jth entry in the ith row represents Strongly Connected the edge from i to j, or lack thereof. Components
Example: Adjacency lists are a little more complex, but can be faster. 2SAT
Minimum Spanning Trees Adjacency List 6
Graph Algorithms
Graphs and Graph Repre- sentations The simplest way to implement adjacency lists is to create Graph a vector for every vertex. Traversals
Directed There exist other representations that are slightly faster Graphs and Cycles and more efficient, like implementing your own linked list,
Strongly but the time complexity of these is the same. Connected Components The simplest implementation with the best time Example: complexity, even if it slower by a constant factor, is usually 2SAT the best choice. Minimum Spanning Trees Adjacency List 7
Graph Algorithms Implementation for undirected graph
#include
Example: // iterate over edges from u (since C++11) 2SAT for (int v : edges[u]) cout << v << '\n';
Minimum // iterate over edges from u (before C++11) Spanning vector
Graph Algorithms
1 Graphs and Graph Representations Graphs and Graph Repre- sentations 2 Graph Graph Traversals Traversals
Directed Graphs and 3 Directed Graphs and Cycles Cycles
Strongly Connected 4 Strongly Connected Components Components
Example: 2SAT 5 Example: 2SAT Minimum Spanning Trees 6 Minimum Spanning Trees Graph Traversals 9
Graph Algorithms
Graphs and Graph Repre- sentations There are two main ways to traverse a graph, which differ
Graph in the order in which they visit new vertices: Traversals Depth-first search (DFS) - visit the first vertex insome Directed vertex’s adjacency list, and then recursively DFS on that Graphs and Cycles vertex, then move on; Strongly Breadth-first search (BFS) - visit the entire adjacency list Connected Components of some vertex, then recursively visit every unvisited vertex
Example: in the adjacency list of those vertices. 2SAT Both can be implemented in O(|V| + |E|) time. Minimum Spanning Trees Depth-First Search 10
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Traversals Depth-first search is a simple idea that can be extended to Directed Graphs and solve a huge amount of problems. Cycles Basic idea: for every vertex, recurse on everything it’s Strongly Connected adjacent to that hasn’t already been visited. Components
Example: 2SAT
Minimum Spanning Trees Depth-First Search 11
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Implementation Traversals // global arrays are initialised to zero for you Directed bool seen[N]; Graphs and Cycles void dfs(int u){ if (seen[u]) return; Strongly seen[u] = true; Connected for (int v : edges[u]) dfs(v); Components } Example: 2SAT
Minimum Spanning Trees Depth-First Search 12
Graph Algorithms In its simple form, it can already be used to solve several
Graphs and problems - undirected cycle detection, connectivity, flood Graph Repre- sentations fill, etc.
Graph It’s very useful, however, to introduce some more Traversals terminology to describe the parts of the DFS: Directed Graphs and Preorder(u) - the procedure to execute when u is reached, Cycles but before recursing on any of its neighbours; Strongly Inorder(u) - the procedure to execute after each neighbour Connected Components of u has finished its recursion; Example: Postorder(u) - the procedure to execute after u has 2SAT finished completely. Minimum Spanning Where in the implementation would each one of these go? Trees How would we modify the implementation to return a result found from the graph? Table of Contents 13
Graph Algorithms
1 Graphs and Graph Representations Graphs and Graph Repre- sentations 2 Graph Graph Traversals Traversals
Directed Graphs and 3 Directed Graphs and Cycles Cycles
Strongly Connected 4 Strongly Connected Components Components
Example: 2SAT 5 Example: 2SAT Minimum Spanning Trees 6 Minimum Spanning Trees Directed Graph Cycle Detection 14
Graph Algorithms A simple cycle is a path through a graph consisting of Graphs and ≥ Graph Repre- n 1 distinct vertices and n distinct edges connecting sentations them (no vertex or edge is visited twice), where the start Graph Traversals and end of the path are the same vertex. Directed Given a graph, report whether or not this graph contains a Graphs and Cycles cycle. Assume that the graph is connected — if not, we Strongly can solve separately on each connected component. Connected Components If the graph is undirected, we can simply run a DFS on the Example: 2SAT graph, and return true if any vertex marked seen is visited
Minimum again. (note: every undirected acyclic graph is a tree) Spanning Trees However, this doesn’t work for directed graphs, such as the diamond graph (1 → 2 → 3 ← 4 ← 1). Directed Graph Cycle Detection 15
Graph Algorithms
Graphs and Graph Repre- This doesn’t work for directed graphs, such as the sentations diamond graph (1 → 2 → 3 ← 4 ← 1). Why not? Graph Traversals What does the DFS actually do? Directed Graphs and When visiting each vertex in our DFS, we check to see if Cycles any of the vertices we’re recursing on has been visited Strongly Connected before. Components
Example: This doesn’t work because even if we’re visiting some 2SAT vertex that has been visited before, there’s no guarantee Minimum Spanning that that vertex can visit us. Trees Directed Graph Cycle Detection 16
Graph Algorithms
Graphs and Graph Repre- However, we can see that the only time that some sentations already-visited vertex that we encounter can reach us is Graph Traversals when we’re currently within that vertex’s recursion stack. Directed Graphs and Now, if we had some easy way to check if we’re currently Cycles inside the recursion stack of a specific vertex, we’d have a Strongly Connected complete algorithm. Components It turns out this is easy to do — just mark each vertex Example: 2SAT “active” in a table during its preorder step, and unmark it Minimum Spanning during its postorder step. Trees Directed Graph Cycle Detection 17
Graph Algorithms
Graphs and Graph Repre- sentations Implementation
Graph // the vertices that are still marked active when this returns are the Traversals ones in the cycle we detected bool has_cycle(int u){ Directed if (seen[u]) return false; Graphs and seen[u] = true; Cycles active[u] = true; Strongly for (int v : edges[u]){ Connected if (active[v] || has_cycle(v)) return true; Components } active[u] = false; Example: return false; 2SAT }
Minimum Spanning Trees Topological Sort 18
Graph Algorithms
Graphs and Graph Repre- sentations
Graph A topological sort of a graph is an ordering of the vertices Traversals that has the property that if some vertex u has a directed Directed Graphs and edge pointing to another vertex v, then v comes after u in Cycles the ordering. Strongly Connected Clearly, if the graph has a cycle, then there does not exist Components
Example: a valid topological ordering of the graph. 2SAT
Minimum Spanning Trees Topological Sort 19
Graph Algorithms
Assuming the graph is acyclic (we refer to directed acyclic Graphs and Graph Repre- graphs as DAGs), how do we compute a topological sentations
Graph ordering via DFS? Traversals We can directly use the reverse of the postorder sequence Directed Graphs and of the graph. Cycles
Strongly The postorder sequence of the graph is an ordering of the Connected Components vertices of the graph in the order that each vertex reaches
Example: its postorder procedure. 2SAT A vertex is only added after its children have been visited Minimum Spanning (and thus added), so the reverse order is a valid Trees topological ordering. Topological Sort 20
Graph Algorithms
Graphs and Graph Repre- Implementation sentations // if the edges are in ASCENDING order of node number , Graph // this produces the lexicographically GREATEST ordering Traversals void dfs(int u, vector
Graph Algorithms
1 Graphs and Graph Representations Graphs and Graph Repre- sentations 2 Graph Graph Traversals Traversals
Directed Graphs and 3 Directed Graphs and Cycles Cycles
Strongly Connected 4 Strongly Connected Components Components
Example: 2SAT 5 Example: 2SAT Minimum Spanning Trees 6 Minimum Spanning Trees Strongly Connected Components 22
Graph Algorithms
Graphs and Graph Repre- sentations
Graph A strongly connected component (SCC) is a maximal Traversals subset of the vertices of a directed graph such that every Directed Graphs and vertex in the subset can reach every other vertex in that Cycles component. Strongly Connected Condensing every strongly connected component to a Components
Example: single vertex results in a directed acyclic graph (DAG). 2SAT
Minimum Spanning Trees Strongly Connected Components 23
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Traversals
Directed Graphs and Cycles
Strongly Connected Components
Example: 2SAT
Minimum Spanning Trees Finding Strongly Connected Components 24
Graph Algorithms
Graphs and Graph Repre- sentations There are a few linear time algorithms known to compute Graph Traversals the strongly connected components of a graph based on
Directed DFS. Graphs and Cycles Kosaraju’s algorithm is simple to implement but hard to Strongly understand. Connected Components Another popular choice in these contests is Tarjan’s Example: 2SAT algorithm.
Minimum Spanning Trees Tarjan’s Algorithm 25
Graph Algorithms Do a regular DFS on the graph, but with an explicit stack. Graphs and Graph Repre- When an item is pushed onto the stack, mark it as sentations
Graph “in-stack”, and unmark it as such when it is popped. Traversals If we want to push a vertex that is already “in-stack”, Directed Graphs and then we’ve found a strongly connected component. Cycles
Strongly Each item on the stack after this vertex can be reached Connected Components from it, and can also reach that vertex. Example: Simply pop everything off the stack including that vertex, 2SAT and combine it into an SCC. Minimum Spanning Trees The actual algorithm is slightly more complicated because some bookkeeping is required. Tarjan’s Algorithm 26
Graph Algorithms Implementation
// we will number the vertices in the order we see them in the DFS Graphs and int dfs_index[MAX_VERTICES]; Graph Repre- // for each vertex , store the smallest number of any vertex we see sentations // in its DFS subtree int lowlink[MAX_VERTICES]; Graph Traversals // explicit stack Directed stack
// continued Tarjan’s Algorithm (continued) 27
Graph Algorithms Implementation
for (auto w : edges[v]){ // for each edge v -> w if (!dfs_index[w]){ // w hasn't been visited yet Graphs and connect(w); Graph Repre- // if w can see something , v can too sentations lowlink[v] = min(lowlink[v], lowlink[w]); Graph } Traversals else if (in_stack[w]){ // w is already in the stack , but we can see it Directed // this means v and w are in the same SCC Graphs and lowlink[v] = min(lowlink[v], dfs_index[w]); Cycles } } Strongly // v is the root of an SCC Connected if (lowlink[v] == dfs_index[v]){ Components ++scc_counter; int w; Example: do { 2SAT w = s.top(); s.pop(); in_stack[w] = false; Minimum which_scc[w] = scc_counter; Spanning } while (w != v); Trees } }
// call connect for each vertex once for (int v = 0; v < n; ++v) if (!dfs_index[v]) connect(v); Table of Contents 28
Graph Algorithms
1 Graphs and Graph Representations Graphs and Graph Repre- sentations 2 Graph Graph Traversals Traversals
Directed Graphs and 3 Directed Graphs and Cycles Cycles
Strongly Connected 4 Strongly Connected Components Components
Example: 2SAT 5 Example: 2SAT Minimum Spanning Trees 6 Minimum Spanning Trees Satisfiability 29
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Traversals Satisfiability (SAT) is the problem of determining, given
Directed some Boolean formula, if there exists some truthiness Graphs and Cycles assignment of variables which would result in the formula
Strongly evaluating to true. Connected Components Satisfiability is NP-hard in the general case. Example: 2SAT
Minimum Spanning Trees 2SAT 30
Graph Algorithms
Graphs and Graph Repre- sentations 2-satisfiability (2SAT) is the problem of determining, given Graph Traversals a set of constraints on pairs of Boolean variables, if there Directed exists some truthiness assignment of variables which would Graphs and Cycles result in the conjunction of all the constraints evaluating Strongly to true. Connected Components Unlike general satisfiability, 2-satisfiability can be solved in Example: 2SAT linear time
Minimum Spanning Trees 2SAT 31
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Traversals The inputs to a 2SAT problem are a set of constraints on
Directed Boolean variables with standard Boolean operators. Graphs and Cycles In this context, only these make sense:
Strongly x1 ∨ x2 (disjunction) Connected Components ¬x1 (negation) Example: 2SAT
Minimum Spanning Trees 2SAT 32
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Traversals We can write these in equivalent implicative normal form: Directed Graphs and x ∨ x ≡ (¬x → x ) ∧ (¬x → x ) Cycles 1 2 1 2 2 1 ¬x1 ≡ (x1 → ¬x1) Strongly Connected Components
Example: 2SAT
Minimum Spanning Trees 2SAT 33
Graph Algorithms
Graphs and Graph Repre- sentations Graph These implications now form a directed graph with the Traversals Boolean variables (and their negations) as vertices, called Directed Graphs and the implication graph. Cycles What does it mean when some variable x can reach some Strongly Connected other variable y in this implication graph? Components If x can reach y in this graph, then x → y. Example: 2SAT
Minimum Spanning Trees 2SAT 34
Graph Algorithms
Graphs and Graph Repre- sentations
Graph When do we have a valid solution to our 2SAT instance? Traversals As long as we don’t have any contradictions (i.e. Directed Graphs and x → ¬x and ¬x → x, we can solve our 2SAT instance. Cycles
Strongly In our implication graph, this is exactly the same as Connected ¬ Components checking to see if x and x are in the same strongly
Example: connected component! 2SAT
Minimum Spanning Trees 2SAT 35
Graph Algorithms
Graphs and Graph Repre- sentations We can then easily construct an actual solution to our Graph 2SAT instance after computing the strongly connected Traversals components by assigning to each variable whichever Directed Graphs and truthiness value comes second in the topological ordering Cycles of the SCC condensed graph. Strongly Connected Components If x comes after ¬x in the topological ordering of the
Example: condensed implication graph, then we say x is true. 2SAT Otherwise, we say it’s false. Minimum Spanning Trees Table of Contents 36
Graph Algorithms
1 Graphs and Graph Representations Graphs and Graph Repre- sentations 2 Graph Graph Traversals Traversals
Directed Graphs and 3 Directed Graphs and Cycles Cycles
Strongly Connected 4 Strongly Connected Components Components
Example: 2SAT 5 Example: 2SAT Minimum Spanning Trees 6 Minimum Spanning Trees Trees 37
Graph Algorithms
Graphs and Graph Repre- sentations A tree is a connected acyclic graph. Graph There are several equivalent definitions, all of which are Traversals useful: Directed Graphs and A connected graph with |V| − 1 edges Cycles An acyclic graph with |V| − 1 edges Strongly Connected There exists exactly one path between every pair of vertices Components An acyclic graph where adding any edge results in a cycle Example: 2SAT A connected graph where removing any edge would disconnect it Minimum Spanning Trees Minimum Spanning Trees 38
Graph Algorithms
Graphs and Graph Repre- sentations Graph A spanning tree for some graph G is a subgraph of G that Traversals
Directed is a tree, and also connects (spans) all of the vertices of G. Graphs and Cycles A minimum spanning tree (MST) is a spanning tree with Strongly minimum sum of edge weights. Connected Components There are several similar algorithms to solve this problem. Example: 2SAT
Minimum Spanning Trees Minimum Spanning Trees 39
Graph Algorithms
Graphs and Graph Repre- sentations
Graph To construct a minimum spanning tree of some graph G, Traversals we maintain a set of spanning forests, initially composed Directed Graphs and of just the vertices of the graph and no edges, and we Cycles keep adding edges until we have a spanning tree. Strongly Connected Clearly, if we add |V| − 1 edges and we avoid constructing Components
Example: any cycles, we’ll have a spanning tree. 2SAT
Minimum Spanning Trees Minimum Spanning Trees 40
Graph Algorithms
Graphs and Graph Repre- sentations
Graph Traversals How do we decide which edges to add, so that we end up
Directed with a minimum spanning tree? Graphs and Cycles We can’t add any edges to our spanning forest that has its Strongly endpoints in the same connected component of our Connected Components spanning forest, or we’ll get a cycle. Example: 2SAT
Minimum Spanning Trees Minimum Spanning Trees 41
Graph Algorithms
Graphs and Graph Repre- We can restrict ourselves to only the edges that cross sentations components that we haven’t connected yet. Graph Traversals Furthermore, to connect some component C and some Directed Graphs and other component D, we should always choose the Cycles minimum weight edge e that crosses from one to the Strongly Connected other; if we instead choose some higher weight edge f, Components then adding e and deleting the maximum weight edge Example: 2SAT from the resulting cycle will create a better spanning tree, Minimum contradicting minimality. Spanning Trees Minimum Spanning Trees 42
Graph Algorithms
Graphs and Graph Repre- sentations The distinction between MST algorithms is in the way Graph that they pick the next components to join together, and Traversals how they handle the joining. Directed Graphs and Cycles Prim’s algorithm only ever connects one large connected
Strongly component to single disconnected vertices in the spanning Connected Components forest. Example: Kruskal’s algorithm connects the two components that 2SAT contain the next globally minimum edge. Minimum Spanning Trees Prim’s Algorithm 43
Graph Algorithms Prim’s algorithm is typically faster for dense graphs, while Graphs and Graph Repre- Kruskal’s algorithm is typically faster for sparse graphs. sentations Both can be implemented to run in O(|E| log |V|) time. Graph Traversals Prim’s algorithm: Directed 1 Mark all vertices as not in the tree. Graphs and Cycles 2 Pick any vertex and add it to the tree. Put all of its Strongly outgoing edges in a min-heap, ordered by weight (so that Connected Components the lowest weight one is at the top). 3 → Example: Remove the top edge u v from the min-heap. If v is 2SAT already in the tree, ignore it and repeat this step. Minimum 4 Add v to the tree. Add all of v’s outgoing edges to the Spanning Trees min-heap. 5 Repeat from step 3 until the min-heap is empty. Prim’s Algorithm 44
Graph Algorithms Implementation #include