Algorithms {Fundamental Techniques} Contents
Total Page:16
File Type:pdf, Size:1020Kb
Algorithms {fundamental techniques} Contents 1 Introduction 1 1.1 Prerequisites ............................................. 1 1.2 When is Efficiency Important? .................................... 1 1.3 Inventing an Algorithm ........................................ 2 1.4 Understanding an Algorithm ..................................... 3 1.5 Overview of the Techniques ..................................... 3 1.6 Algorithm and code example ..................................... 4 1.6.1 Level 1 (easiest) ........................................ 4 1.6.2 Level 2 ............................................ 4 1.6.3 Level 3 ............................................ 4 1.6.4 Level 4 ............................................ 4 1.6.5 Level 5 ............................................ 4 2 Mathematical Background 5 2.1 Asymptotic Notation ......................................... 5 2.1.1 Big Omega .......................................... 6 2.1.2 Big Theta ........................................... 6 2.1.3 Little-O and Omega ..................................... 6 2.1.4 Big-O with multiple variables ................................ 7 2.2 Algorithm Analysis: Solving Recurrence Equations ......................... 7 2.2.1 Substitution method ..................................... 7 2.2.2 Summations ......................................... 7 2.2.3 Draw the Tree and Table ................................... 7 2.2.4 The Master Theorem ..................................... 7 2.3 Amortized Analysis .......................................... 8 3 Divide and Conquer 9 3.1 Merge Sort .............................................. 9 3.1.1 Analysis ........................................... 9 3.1.2 Iterative Version ....................................... 10 3.1.3 space inefficiency ....................................... 10 3.2 Binary Search ............................................. 10 3.3 Integer Multiplication ......................................... 11 i ii CONTENTS 3.3.1 Grade School Multiplication ................................. 11 3.3.2 Divide and Conquer Multiplication .............................. 11 3.4 Base Conversion ........................................... 12 3.4.1 Recursive Implementation .................................. 13 3.4.2 Iterative Implementation ................................... 13 3.5 Closest Pair of Points ......................................... 14 3.6 Closest Pair: A Divide-and-Conquer Approach ........................... 14 3.6.1 Introduction ......................................... 14 3.6.2 Closest Pair in the Plane ................................... 14 3.6.3 Summary and Analysis of the 2-D Algorithm ........................ 14 3.6.4 Improving the Algorithm ................................... 15 3.7 Towers Of Hanoi Problem ...................................... 15 3.7.1 Rules ............................................. 15 3.7.2 Solution ............................................ 15 3.8 Footnotes ............................................... 15 4 Randomization 17 4.1 Ordered Statistics ........................................... 17 4.1.1 find-max ........................................... 17 4.1.2 find-min-max ......................................... 17 4.1.3 find-median .......................................... 18 4.2 Quicksort ............................................... 19 4.3 Shuffling an Array .......................................... 20 4.4 Equal Multivariate Polynomials .................................... 20 4.5 Hash tables .............................................. 20 4.6 Skip Lists ............................................... 21 4.6.1 a method of skip list implementation : implement lookahead single-linked linked list, then test , then transform to skip list implementation , then same test, then performance comparison 22 4.6.2 Role of Randomness ..................................... 22 4.6.3 concurrent access application ................................. 23 4.6.4 Idea for an exercise ...................................... 23 4.7 Treaps ................................................. 23 4.7.1 java example of treap implementation ............................ 23 4.7.2 Treaps compared and contrasted to Splay trees ....................... 23 4.8 Derandomization ........................................... 24 4.9 Exercises ............................................... 24 4.10 Supplementary Topic: skip lists and multiprocessor algorithms ................... 24 5 Backtracking 26 5.1 Longest Common Subsequence (exhaustive version) ......................... 26 5.2 Shortest Path Problem (exhaustive version) .............................. 27 5.3 Largest Independent Set ....................................... 27 CONTENTS iii 5.4 Bounding Searches .......................................... 27 5.5 Constrained 3-Coloring ........................................ 27 5.6 Traveling Salesperson Problem .................................... 27 6 Dynamic Programming 28 6.1 Fibonacci Numbers .......................................... 28 6.2 Longest Common Subsequence (DP version) ............................. 29 6.3 Matrix Chain Multiplication ..................................... 29 6.3.1 Forming a Recursive Solution ................................ 30 6.4 Parsing Any Context-Free Grammar ................................. 31 7 Greedy Algorithms 32 7.1 Event Scheduling Problem ...................................... 32 7.1.1 = Longest Path solution to critical path scheduling of jobs .................. 33 7.2 Dijkstra’s Shortest Path Algorithm .................................. 33 7.3 Minimum spanning tree ........................................ 35 7.4 Maximum Flow in weighted graphs .................................. 35 8 Hill Climbing 36 8.1 Newton’s Root Finding Method .................................... 36 8.2 Network Flow ............................................. 37 8.3 The Ford-Fulkerson Algorithm .................................... 38 9 Unweighted Graph Algorithms 40 9.1 Representation of Graph ........................................ 40 9.1.1 Adjacency Matrix ....................................... 40 9.1.2 Adjacency List ........................................ 40 9.1.3 Comparison .......................................... 40 9.2 Depth First Search ........................................... 40 9.2.1 Pseudocode .......................................... 40 9.2.2 Properties ........................................... 40 9.2.3 Classification of Edge ..................................... 40 9.3 Breadth First Search .......................................... 40 9.3.1 Pseudocode .......................................... 40 9.3.2 Example ............................................ 40 9.3.3 Correctness .......................................... 40 9.3.4 Analysis ............................................ 40 9.3.5 Usage ............................................. 40 9.4 Classical Graph Problems ....................................... 40 9.4.1 Directed graph cycle detection ................................ 41 9.4.2 Topological Sort of Directed Graph ............................. 41 9.4.3 Strongly Connected Components in Directed Graphs ..................... 41 9.4.4 Articulation Vertex ...................................... 41 iv CONTENTS 9.4.5 Bridge ............................................. 41 9.4.6 Diameter ........................................... 41 10 Distance approximations 42 10.1 You don't need a square root to compare distances .......................... 42 10.2 Approximations of Euclidean distance ................................ 42 10.2.1 Taxicab/Manhattan/L1 .................................... 42 10.2.2 Octagonal ........................................... 42 11 Appendix A: Ada Implementation 44 11.1 Introduction .............................................. 44 11.2 Chapter 1: Introduction ........................................ 44 11.2.1 To Lower ........................................... 44 11.2.2 Equal Ignore Case ...................................... 44 11.3 Chapter 6: Dynamic Programming .................................. 44 11.3.1 Fibonacci numbers ...................................... 44 11.4 Text and image sources, contributors, and licenses .......................... 46 11.4.1 Text .............................................. 46 11.4.2 Images ............................................ 46 11.4.3 Content license ........................................ 47 Chapter 1 Introduction Top, Chapters: 1, 2, 3, 4, 5, 6, 7, 8, 9, A 1.1 Prerequisites This book covers techniques for the design and analysis of algorithms. The algorithmic techniques covered in- To understand the material presented in this book you clude: divide and conquer, backtracking, dynamic pro- need to know a programming language well enough to gramming, greedy algorithms, and hill-climbing. translate the pseudocode in this book into a working solu- tion. You also need to know the basics about the follow- Any solvable problem generally has at least one algorithm ing data structures: arrays, stacks, queues, linked-lists, of each of the following types: trees, heaps (also called priority queues), disjoint sets, and graphs. 1. the obvious way; Additionally, you should know some basic algorithms like binary search, a sorting algorithm (merge sort, heap sort, insertion sort, or others), and breadth-first or depth-first 2. the methodical way; search. If you are unfamiliar with any of these prerequisites you 3. the clever way; and should review the material in the Data Structures book first. 4.