Computer Science III
Total Page:16
File Type:pdf, Size:1020Kb
Computer Science III Dr. Chris Bourke Department of Computer Science & Engineering University of Nebraska|Lincoln Lincoln, NE 68588, USA http://cse.unl.edu/~cbourke [email protected] 2016/08/05 21:48:04 Version 1.2.0 These are lecture notes used in CSCE 310 (Data Structures & Algorithms) at the University of Nebraska|Lincoln. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License i Contents 1 Introduction1 2 Algorithm Analysis3 2.1 Introduction .................................. 3 2.1.1 Example: Computing a Sum..................... 6 2.1.2 Example: Computing a Mode .................... 8 2.2 Pseudocode...................................11 2.3 Analysis ....................................14 2.4 Asymptotics ..................................18 2.4.1 Big-O Analysis ............................18 2.4.2 Other Notations............................21 2.4.3 Observations..............................22 2.4.4 Limit Method.............................25 2.5 Examples....................................28 2.5.1 Linear Search .............................28 2.5.2 Set Operation: Symmetric Difference ................29 2.5.3 Euclid's GCD Algorithm.......................30 2.5.4 Selection Sort .............................32 2.6 Other Considerations .............................33 2.6.1 Importance of Input Size.......................33 2.6.2 Control Structures are Not Elementary Operations.........36 2.6.3 Average Case Analysis ........................37 2.6.4 Amortized Analysis..........................38 2.7 Analysis of Recursive Algorithms.......................39 2.7.1 The Master Theorem.........................40 3 Storing Things 45 3.1 Lists ......................................45 3.2 Sets.......................................45 3.3 Hash-Tables ..................................45 3.3.1 Hash Functions ............................45 3.3.2 Collisions................................46 3.3.3 Efficiency Rehashing .........................47 3.3.4 Other Applications ..........................48 3.3.5 Java Implementations.........................48 iii Contents 3.4 Bloom Filters .................................49 3.5 Disjoint Sets..................................49 3.6 Exercises....................................49 4 Brute Force Style Algorithms 51 4.1 Introduction ..................................51 4.1.1 Examples................................51 4.1.2 Backtracking..............................52 4.2 Generating Combinatorial Objects......................52 4.2.1 Generating Combinations (Subsets).................52 4.2.2 Generating Permutations.......................53 4.2.3 Permutations with Repetition ....................54 4.2.4 Set Partitions .............................56 4.3 Problems & Algorithms............................58 4.3.1 Satisfiability..............................58 4.3.2 Hamiltonian Path/Cycle .......................59 4.3.3 0-1 Knapsack .............................62 4.3.4 Closest Pair ..............................65 4.3.5 Convex Hull..............................65 4.3.6 Assignment Problem .........................67 4.3.7 Subset Sum ..............................68 4.4 Exercises....................................69 5 Divide & Conquer Style Algorithms 71 5.1 Introduction ..................................71 5.2 Problems & Algorithms............................71 5.3 Repeated Squaring...............................71 5.4 Euclid's GCD Algorithm ...........................73 5.5 Peasant Multiplication ............................76 5.6 Karatsuba Multiplication...........................76 5.7 Strassen's Matrix Multiplication .......................78 5.8 Closest Pair Revisited.............................80 5.9 Convex Hull Revisited.............................80 5.10 Fast Fourier Transform ............................81 5.11 Exercises....................................81 6 Linear Systems 83 6.1 Introduction ..................................83 6.2 Solving Linear Systems ............................83 6.2.1 LU Decomposition ..........................87 6.2.2 Matrix Inverse.............................87 6.2.3 Determinants .............................89 6.3 Linear Programming..............................90 6.3.1 Formulations..............................92 iv Contents 6.4 Exercises....................................95 7 Trees 97 7.1 Introduction ..................................97 7.2 Definitions & Terminology ..........................97 7.3 Tree Traversal .................................99 7.3.1 Preorder Traversal...........................99 7.3.2 Inorder Traversal ..........................100 7.3.3 Postorder Traversal.........................101 7.3.4 Breadth-First Search Traversal...................104 7.3.5 Implementations & Data Structures . 104 7.3.6 Operations..............................108 7.4 Binary Search Trees.............................110 7.4.1 Basic Operations ..........................110 7.5 Balanced Binary Search Trees .......................112 7.5.1 AVL Trees..............................114 7.5.2 B-Trees................................120 7.5.3 Red-Black Trees...........................122 7.6 Heaps.....................................123 7.6.1 Operations..............................123 7.6.2 Implementations...........................124 7.6.3 Java Collections Framework ....................128 7.6.4 Other Operations..........................128 7.6.5 Variations ..............................128 7.7 Applications .................................128 7.7.1 Heap Sort ..............................128 7.7.2 Huffman Coding...........................129 8 Graph Algorithms 141 8.1 Introduction .................................141 8.2 Depth First Search..............................142 8.2.1 DFS Example............................144 8.2.2 DFS Artifacts............................145 8.2.3 Analysis ...............................146 8.3 Breadth First Search ............................148 8.4 DFS/BFS Applications ...........................150 8.4.1 Connectivity & Path Finding....................151 8.4.2 Topological Sorting.........................151 8.4.3 Shortest Path ............................151 8.4.4 Cycle Detection...........................152 8.4.5 Bipartite Testing ..........................153 8.4.6 Condensation Graphs........................153 8.5 Minimum Spanning Tree Algorithms....................154 8.5.1 Greedy Algorithmic Strategy....................154 v Contents 8.5.2 Kruskal's Algorithm.........................155 8.5.3 Prim's Algorithm ..........................157 8.6 Minimum Distance Algorithms.......................160 8.6.1 Dijkstra's Algorithm ........................160 8.6.2 Floyd-Warshall Algorithm .....................162 8.6.3 Huffman Coding...........................168 8.7 Exercises...................................168 9 Dynamic Programming 171 9.1 Introduction .................................171 9.1.1 Optimal Substructure Property ..................171 9.2 Binomial Coefficients ............................172 9.3 Optimal Binary Search Trees........................174 9.3.1 Example...............................178 9.4 Dynamic Knapsack .............................179 9.4.1 Example...............................182 9.4.2 Analysis ...............................182 9.5 Coin Change Problem............................182 9.5.1 Example...............................183 9.6 Matrix Chain Multiplication ........................184 9.6.1 Example...............................186 9.7 Exercises...................................188 10 Computational Models 191 10.1 Introduction .................................191 10.1.1 Languages..............................192 10.2 Computational Models ...........................195 10.2.1 Finite-State Automata .......................195 10.2.2 Turing Machines...........................197 10.2.3 Church-Turing Thesis........................199 10.2.4 Halting Problem & Decidability ..................202 10.3 Complexity Classes .............................204 10.3.1 Deterministic Polynomial Time ..................205 10.3.2 Nondeterminism...........................205 10.4 Reductions & NP-Completeness ......................206 10.4.1 Satisfiability.............................208 10.4.2 Satisfiability to Clique .......................210 10.4.3 Clique to Vertex Cover.......................211 10.5 Beyond P and NP ..............................213 10.6 Misc .....................................214 10.7 Exercises...................................216 11 More Algorithms 219 11.1 A∗ Search...................................219 vi Contents 11.2 Jump Point Search..............................219 11.3 Minimax ...................................219 Glossary 221 Acronyms 223 Index 227 References 228 vii List of Algorithms 1 Computing the Mean ...............................12 2 Computing the Mode ...............................13 3 Trivial Sorting (Bad Pseudocode).........................14 4 Trivially Finding the Minimal Element......................14 5 Finding the Minimal Element...........................14 6 Linear Search....................................28 7 Symmetric Difference of Two Sets ........................29 8 Euclid's GCD Algorithm..............................30 9 Selection Sort....................................32 10 Sieve of Eratosthenes ...............................34 11 Fibonacci(n) ....................................40 12 Binary Search { Recursive.............................42 13 Merge Sort .....................................42 14 Next k-Combination................................53