Computer Science III

Dr. Chris Bourke Department of Computer Science & Engineering University of Nebraska—Lincoln Lincoln, NE 68588, USA http://cse.unl.edu/~cbourke [email protected]

2016/08/05 21:48:04 Version 1.2.0

These are lecture notes used in CSCE 310 (Data Structures & Algorithms) at the University of Nebraska—Lincoln.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

Contents

1 Introduction1

2 Algorithm Analysis3 2.1 Introduction ...... 3 2.1.1 Example: Computing a Sum...... 6 2.1.2 Example: Computing a Mode ...... 8 2.2 Pseudocode...... 11 2.3 Analysis ...... 14 2.4 Asymptotics ...... 18 2.4.1 Big-O Analysis ...... 18 2.4.2 Other Notations...... 21 2.4.3 Observations...... 22 2.4.4 Limit Method...... 25 2.5 Examples...... 28 2.5.1 Linear Search ...... 28 2.5.2 Set Operation: Symmetric Diﬀerence ...... 29 2.5.3 Euclid’s GCD Algorithm...... 30 2.5.4 Selection Sort ...... 32 2.6 Other Considerations ...... 33 2.6.1 Importance of Input Size...... 33 2.6.2 Control Structures are Not Elementary Operations...... 36 2.6.3 Average Case Analysis ...... 37 2.6.4 Amortized Analysis...... 38 2.7 Analysis of Recursive Algorithms...... 39 2.7.1 The Master Theorem...... 40

3 Storing Things 45 3.1 Lists ...... 45 3.2 Sets...... 45 3.3 Hash-Tables ...... 45 3.3.1 Hash Functions ...... 45 3.3.2 Collisions...... 46 3.3.3 Eﬃciency Rehashing ...... 47 3.3.4 Other Applications ...... 48 3.3.5 Java Implementations...... 48

iii Contents

3.4 Bloom Filters ...... 49 3.5 Disjoint Sets...... 49 3.6 Exercises...... 49

4 Brute Force Style Algorithms 51 4.1 Introduction ...... 51 4.1.1 Examples...... 51 4.1.2 Backtracking...... 52 4.2 Generating Combinatorial Objects...... 52 4.2.1 Generating Combinations (Subsets)...... 52 4.2.2 Generating Permutations...... 53 4.2.3 Permutations with Repetition ...... 54 4.2.4 Set Partitions ...... 56 4.3 Problems & Algorithms...... 58 4.3.1 Satisﬁability...... 58 4.3.2 Hamiltonian Path/Cycle ...... 59 4.3.3 0-1 Knapsack ...... 62 4.3.4 Closest Pair ...... 65 4.3.5 Convex Hull...... 65 4.3.6 Assignment Problem ...... 67 4.3.7 Subset Sum ...... 68 4.4 Exercises...... 69

5 Divide & Conquer Style Algorithms 71 5.1 Introduction ...... 71 5.2 Problems & Algorithms...... 71 5.3 Repeated Squaring...... 71 5.4 Euclid’s GCD Algorithm ...... 73 5.5 Peasant Multiplication ...... 76 5.6 Karatsuba Multiplication...... 76 5.7 Strassen’s Matrix Multiplication ...... 78 5.8 Closest Pair Revisited...... 80 5.9 Convex Hull Revisited...... 80 5.10 Fast Fourier Transform ...... 81 5.11 Exercises...... 81

6 Linear Systems 83 6.1 Introduction ...... 83 6.2 Solving Linear Systems ...... 83 6.2.1 LU Decomposition ...... 87 6.2.2 Matrix Inverse...... 87 6.2.3 Determinants ...... 89 6.3 Linear Programming...... 90 6.3.1 Formulations...... 92

iv Contents

6.4 Exercises...... 95

7 Trees 97 7.1 Introduction ...... 97 7.2 Deﬁnitions & Terminology ...... 97 7.3 Tree Traversal ...... 99 7.3.1 Preorder Traversal...... 99 7.3.2 Inorder Traversal ...... 100 7.3.3 Postorder Traversal...... 101 7.3.4 Breadth-First Search Traversal...... 104 7.3.5 Implementations & Data Structures ...... 104 7.3.6 Operations...... 108 7.4 Binary Search Trees...... 110 7.4.1 Basic Operations ...... 110 7.5 Balanced Binary Search Trees ...... 112 7.5.1 AVL Trees...... 114 7.5.2 B-Trees...... 120 7.5.3 Red-Black Trees...... 122 7.6 Heaps...... 123 7.6.1 Operations...... 123 7.6.2 Implementations...... 124 7.6.3 Java Collections Framework ...... 128 7.6.4 Other Operations...... 128 7.6.5 Variations ...... 128 7.7 Applications ...... 128 7.7.1 Heap Sort ...... 128 7.7.2 Huﬀman Coding...... 129

8 Graph Algorithms 141 8.1 Introduction ...... 141 8.2 Depth First Search...... 142 8.2.1 DFS Example...... 144 8.2.2 DFS Artifacts...... 145 8.2.3 Analysis ...... 146 8.3 Breadth First Search ...... 148 8.4 DFS/BFS Applications ...... 150 8.4.1 Connectivity & Path Finding...... 151 8.4.2 Topological Sorting...... 151 8.4.3 Shortest Path ...... 151 8.4.4 Cycle Detection...... 152 8.4.5 Bipartite Testing ...... 153 8.4.6 Condensation Graphs...... 153 8.5 Minimum Spanning Tree Algorithms...... 154 8.5.1 Greedy Algorithmic Strategy...... 154

v Contents

8.5.2 Kruskal’s Algorithm...... 155 8.5.3 Prim’s Algorithm ...... 157 8.6 Minimum Distance Algorithms...... 160 8.6.1 Dijkstra’s Algorithm ...... 160 8.6.2 Floyd-Warshall Algorithm ...... 162 8.6.3 Huﬀman Coding...... 168 8.7 Exercises...... 168

9 Dynamic Programming 171 9.1 Introduction ...... 171 9.1.1 Optimal Substructure Property ...... 171 9.2 Binomial Coeﬃcients ...... 172 9.3 Optimal Binary Search Trees...... 174 9.3.1 Example...... 178 9.4 Dynamic Knapsack ...... 179 9.4.1 Example...... 182 9.4.2 Analysis ...... 182 9.5 Coin Change Problem...... 182 9.5.1 Example...... 183 9.6 Matrix Chain Multiplication ...... 184 9.6.1 Example...... 186 9.7 Exercises...... 188

10 Computational Models 191 10.1 Introduction ...... 191 10.1.1 Languages...... 192 10.2 Computational Models ...... 195 10.2.1 Finite-State Automata ...... 195 10.2.2 Turing Machines...... 197 10.2.3 Church-Turing Thesis...... 199 10.2.4 Halting Problem & Decidability ...... 202 10.3 Complexity Classes ...... 204 10.3.1 Deterministic Polynomial Time ...... 205 10.3.2 Nondeterminism...... 205 10.4 Reductions & NP-Completeness ...... 206 10.4.1 Satisﬁability...... 208 10.4.2 Satisﬁability to Clique ...... 210 10.4.3 Clique to Vertex Cover...... 211 10.5 Beyond P and NP ...... 213 10.6 Misc ...... 214 10.7 Exercises...... 216

11 More Algorithms 219 11.1 A∗ Search...... 219

vi Contents

11.2 Jump Point Search...... 219 11.3 Minimax ...... 219

Glossary 221

Acronyms 223

Index 227

References 228

vii

List of Algorithms

1 Computing the Mean ...... 12 2 Computing the Mode ...... 13 3 Trivial Sorting (Bad Pseudocode)...... 14 4 Trivially Finding the Minimal Element...... 14 5 Finding the Minimal Element...... 14 6 Linear Search...... 28 7 Symmetric Diﬀerence of Two Sets ...... 29 8 Euclid’s GCD Algorithm...... 30 9 Selection Sort...... 32 10 Sieve of Eratosthenes ...... 34 11 Fibonacci(n) ...... 40 12 Binary Search – Recursive...... 42 13 Merge Sort ...... 42

14 Next k-Combination...... 53 15 Next Lexicographic Permutation ...... 55 16 Next Repeated Permutation Generator...... 56 17 Base Conversion Algorithm ...... 56 18 Set Partition Generator ...... 57 19 Brute Force Iterative Algorithm for Satisﬁability ...... 59 20 Brute Force Recursive Algorithm for Satisﬁability...... 59 21 Brute Force Iterative Algorithm for Hamiltonian Cycle...... 61 22 Hamiltonian DFS Cycle Walk...... 61 23 Hamiltonian DFS Path Walk–Main Algorithm...... 62 24 Walk(G, p) – Hamiltonian DFS Path Walk...... 62 25 Knapsack(K,S) – Backtracking Brute Force 0-1 Knapsack...... 65 26 Brute Force Convex Hu