CS483-10 Elementary Graph Algorithms & Transform-and-Conquer Outline ➣ Depth-first Search – cont Instructor: Fei Li ➣ Topological Sort Room 443 ST II ➣ Transform-and-Conquer – Gaussian Elimination Office hours: Tue. & Thur. 1:30pm - 2:30pm or by appointments [email protected] with subject: CS483 http://www.cs.gmu.edu/∼ lifei/teaching/cs483_fall07/ Based on Introduction to the Design and Analysis of Algorithms by Anany Levitin, Jyh-Ming Lien’s notes, and Introduction to Algorithms by CLRS. CS483 Design and Analysis of Algorithms 1 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 2 Lecture 09, September 27, 2007 Depth-first Search (DFS) ➣ The correctness proof: Use an induction method ➣ The overall running time of DFS is O(|V | + |E|). | | The time initializing each vertex is O( V ). ∈ Each edge (u, v) E is examined twice, once exploring u and once exploring v. Therefore takes O(|E|) time. CS483 Design and Analysis of Algorithms 3 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 4 Lecture 09, September 27, 2007 Topological Sort Outline ➣ An application of DFS ➣ Depth-first Search – cont ➣ Input: a directed acyclic graph (DAG) ➣ Topological Sort ➣ Output: A linear ordering of all its vertices, such that if G contains an edge ➣ Transform-and-Conquer – Gaussian Elimination (u, v), then, u appears before v in the ordering. CS483 Design and Analysis of Algorithms 5 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 6 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 7 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 8 Lecture 09, September 27, 2007 Topological-Sort Algorithm 0.1: TOPOLOGICAL-SORT(G(V,E)) Call DFS(G) to compute finishing times f[v] for each vertex v As each vertex is finished, insert it onto the front of a linked list return (the linked list of vertices) CS483 Design and Analysis of Algorithms 9 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 10 Lecture 09, September 27, 2007 Transform-and-Conquer A problem is solved by a transformation ➣ Outline To a simpler/more convenient instance of the same problem (instance simplification) ➣ Depth-first Search – cont - Ex: transforming unsorted to sorted ➣ Topological Sort ➣ To a different representation of the same instance (representation change) ➣ Transform-and-Conquer – Gaussian Elimination - Ex: transforming list to tree, tree to balanced tree, ... , etc. ➣ To a different problem for which an algorithm is already available (problem reduction) - Ex: transforming multiplication to addition CS483 Design and Analysis of Algorithms 11 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 12 Lecture 09, September 27, 2007 Transform-and-Conquer ➣ Instance Simplification Instance Simplification - Element uniqueness ➣ Find if a given array contains unique elements. - Mode (the most repeated element) of an array - What is the transform? - Searching for a value in an array ➣ Find the most repeated element. ➣ Representation Change - What is the transform? - Gaussian elimination ➣ - AVL tree, 2-3 tree Search for a value in an array (including binary search). - Heap and heapsort - What is the transform? ➣ ➣ Problem Reduction Quicksort - Least common multiple - What is the transform? - Paths in a graph - Linear programming (Chapter 10) CS483 Design and Analysis of Algorithms 13 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 14 Lecture 09, September 27, 2007 Instance Simplification - Presorting Instance Simplification Instance Simplification: Solve a problem’s instance by transforming it into another simpler/easier instance of the same problem. ➣ Find if a given array contains unique elements. ➣ Presorting - What is the transform? Sorting Many problems involving lists are easier when list is sorted. ➣ Find the most repeated element. Searching - What is the transform? Sorting Computing the median (selection problem) ➣ Search for a value in an array (including binary search). Checking if all elements are distinct (element uniqueness) - What is the transform? Sorting Also: ➣ Quicksort Topological sorting helps solving some problems for directed Acyclic - What is the transform? Randomization graphs (DAGs). Presorting is used in many geometric algorithms. CS483 Design and Analysis of Algorithms 15 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 16 Lecture 09, September 27, 2007 Searching with Presorting How Fast Can We Sort? ➣ Problem: Search for a given K in A[1...n] ➣ Efficiency of algorithms involving sorting depends on efficiency of sorting. ➣ Presorting-based algorithm: 1. Sort the array by an efficient sorting algorithm ➣ Theorem (see Sec. 11.2): log2 n!≈n log2 n comparisons are necessary in the worst case to sort a list of size n by any comparison-based algorithm. 2. Apply binary search ➣ ➣ Note: About n log2 n comparisons are also sufficient to sort array of size n Efficiency: Θ(n log2 n)+O(log2 n)=Θ(n log2 n) (by mergesort). ➣ Good or bad? ➣ Why do we have our dictionaries, telephone directories, etc. sorted? CS483 Design and Analysis of Algorithms 17 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 18 Lecture 09, September 27, 2007 Searching with Presorting ➣ Problem: Search for a given K in A[1...n] Element Uniqueness with Presorting ➣ Presorting-based algorithm: ➣ Brute force algorithm 1. Sort the array by an efficient sorting algorithm Compare all pairs of elements 2 2. Apply binary search Efficiency: O(n ) ➣ Efficiency: Θ(n log2 n)+O(log2 n)=Θ(n log2 n) ➣ Presorting-based algorithm ➣ Good or bad? 1. Sort by efficient sorting algorithm (e.g. mergesort) 2. Scan array to check pairs of adjacent elements ➣ Why do we have our dictionaries, telephone directories, etc. sorted? Efficiency: Θ(n log n)+O(n)=Θ(n log n) Cumulative cost is reduced. 2 2 M · n vs. n log2 n + M · log2 n,givenM is large. CS483 Design and Analysis of Algorithms 19 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 20 Lecture 09, September 27, 2007 Transform-and-Conquer ➣ Instance Simplification - Element uniqueness Representation Change: Gaussian Elimination - Mode (the most repeated element) of an array Problem: Solve the linear system of a set of n linear equations and n variables. - Searching for a value in an array ➣ Example 1: ➣ Representation Change - Gaussian elimination - AVL tree, 2-3 tree a11x + a12y = b1 - Heap and heapsort a21x + a22y = b2 ➣ Problem Reduction (Chapter 10) - Least common multiple - Paths in a graph - Linear programming CS483 Design and Analysis of Algorithms 21 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 22 Lecture 09, September 27, 2007 Representation Change: Gaussian Elimination Representation Change: Gaussian Elimination Problem: Solve the linear system of a set of n linear equations and n variables. ➣ Given:Asystem of n linear equations in n unknowns with an arbitrary coefficient matrix. ➣ Example 1: ➣ Transform to: An equivalent system of n linear equations in n unknowns with an upper triangular coefficient matrix. a11x + a12y = b1 ➣ Solve the latter by substitutions starting with the last equation and moving up a21x + a22y = b2 to the first one. ➣ Example 2: ➣ Base: If we add a multiple of one equation to another, the overall system of + + ···+ = a11x1 a12x2 a1nxn b1 equations remains equivalent. a21x1 + a22x2 + ···+ a2nxn = b2 . + + ···+ = + + ···+ = . a11x1 a12x2 a1nxn b1 a11x1 a12x2 a1nxn b1 a21x1 + a22x2 + ···+ a2nxn = b2 a x2 + ···+ a xn = b an1x1 + an2x2 + ···+ annxn = bn 22 2n 2 . an1x1 + an2x2 + ···+ annxn = bn annxn = bn CS483 Design and Analysis of Algorithms 23 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 24 Lecture 09, September 27, 2007 Gaussian Elimination: Example Transformation by Gaussian Elimination The transformation is accomplished by a sequence of elementary operations on 2x1 − 4x2 + x3 =6 the system’s coefficient matrix (which do not change the system’s solution): 3x1 − x2 + x3 =11 + − = −3 ← − x1 x2 x3 for f 1 to n 1 do Replace each of the subsequent rows (i.e., rows i +1, ..., n) by a difference 2 −41 6 th between that row and an appropriate multiple of the i row to make the new 3 −11 11 th coefficient in the i column of that row 0. 11−1 −3 CS483 Design and Analysis of Algorithms 25 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 26 Lecture 09, September 27, 2007 Gaussian Elimination: Example Gaussian Elimination: Example 2x1 − 4x2 + x3 =6 3x1 − x2 + x3 =11 2x1 − 4x2 + x3 =6 x1 + x2 − x3 = −3 3x1 − x2 + x3 =11 x1 + x2 − x3 = −3 2 −41 6 3 −11 11 2 − 3 × 1 row 2 row 2 −41 6 11−1 −3 3 − 1 × 1 row 2 row 3 −11 11 2 − 3 × 1 row 2 row 11−1 −3 3 − 1 × 1 2 −41 6 row 2 row 05− 1 2 2 03− 3 −6 3 − 3 × 2 2 −41 6 2 row 5 row 05− 1 2 2 2 −41 6 03− 3 −6 2 05− 1 2 2 00− 6 − 36 5 5 CS483 Design and Analysis of Algorithms 27 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 28 Lecture 09, September 27, 2007 Gaussian Elimination: Example Gaussian Elimination Algorithm ➣ We have: 2 − 4 + =6 x1 x2 x3 Algorithm 0.2: GE(A[1 ···n, 1 ···n],b[1 ···n]) 5 − 1 =2 x2 2 x3 − 6 = − 36 5 x3 5 Append b to A as the last column for i ∈{1 ···n − 1} ➣ ⎧ Then we can solve x3,x2,x1 by backward substitution: ⎪ ∈{ ··· } ⎨⎪for j ⎧ i +1 n ⎨ 3= (− 36 ) (− 6 )=6 do ∈{ ··· } x 5 / 5 ⎪ for k j n 1 ⎩⎪ do x2=(2+() × 6)/5=1 ⎩ A[i,k]A[j,i] 2 do for A[j, k]=A[j, k] − A[i,i] x1= (66+4× 1)/2=2 CS483 Design and Analysis of Algorithms 29 Lecture 09, September 27, 2007 CS483 Design and Analysis of Algorithms 30 Lecture 09, September 27, 2007 Backward Substitution Algorithm Solve Ax = b using Gaussian Elimination Algorithm 0.3: BS(A[1 ···n, 1 ···n +1]) ➣ To solve a linear system Ax = b:WewillcallGE(A,b) and then x = BS(A) ←{ ··· } for j ⎧ n 1 ➣ Time Complexity: ⎪ ← ⎪t 0 ⎨⎪ for k ←{j +1···n} 2 2 2 do T (n)=n +(n − 1) + ···+(n − (n − 1)) = ⎪ ← × 2 2 2 1 3 3 ⎪ do t t + A[j, k] x[k] 1 +2 + ···+ n ≈ n =Θ(n ).