UC San Diego UC San Diego Electronic Theses and Dissertations Title On the power of the basic algorithmic design paradigms Permalink https://escholarship.org/uc/item/52t3d36w Author Davis, Sashka Tchameva Publication Date 2008 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California UNIVERSITY OF CALIFORNIA, SAN DIEGO On the Power of the Basic Algorithmic Design Paradigms A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science by Sashka Tchameva Davis Committee in charge: Russell Impagliazzo, Chair Samuel Buss Fan C. Graham Daniele Micciancio Ramamohan Paturi 2008 Copyright Sashka Tchameva Davis, 2008 All rights reserved. The dissertation of Sashka Tchameva Davis is ap- proved, and it is acceptable in quality and form for publication on microfilm and electronically: Chair University of California, San Diego 2008 iii DEDICATION To my husband, my son, and my parents for their love and support. iv TABLE OF CONTENTS Signature Page . iii Dedication . iv Table of Contents . v List of Figures . vii Acknowledgements . viii Vita . ix Abstract of the Dissertation . x Chapter 1. Do We Need Formal Models of the Algorithmic Paradigms? . 1 1.1. History . 3 1.1.1. Priority Model . 3 1.1.2. Prioritized Branching Trees . 6 1.2. Techniques . 7 Chapter 2. Priority Algorithms . 9 2.1. Why Study the Greedy Paradigm? . 9 2.1.1. What Is a Greedy Algorithm? . 11 2.2. Priority Models . 14 2.3. A General Lower Bound Technique . 19 2.4. Results for Graph Problems . 23 2.4.1. Shortest Paths . 23 2.4.2. The Weighted Vertex Cover Problem . 28 2.4.3. The Metric Steiner Tree Problem . 31 2.4.4. Maximum Independent Set Problem . 48 2.5. Memoryless Priority Algorithms . 50 2.5.1. A Memoryless Adaptive Priority Game . 53 2.5.2. Separation Between the Class of Adaptive Priority Algorithms and Memoryless Adaptive Priority Algorithms . 58 2.6. Notes . 66 Chapter 3. Prioritized Branching Tree and Prioritized Free Branching Tree Models 67 3.1. The prioritized Branching Tree Model . 71 3.1.1. Complexity Measure for pBT Algorithms . 74 3.2. The prioritized Free Branching Tree Model . 75 3.3. Lower Bound for pBT Algorithms for 7-SAT . 78 3.3.1. Proof Framework . 82 v 3.3.2. Analysis of Good Intermediaries . 90 3.3.3. Analysis of Bad Intermediaries . 93 3.4. Lower Bound for pFBT Algorithms for 7-SAT . 97 3.5. Boundary Expander Matrix . 99 3.6. Notes . 106 Chapter 4. Can pBT Algorithms Solve the Shortest Path Problem? . 107 4.1. Notes . 117 Chapter 5. prioritized Branching Programs . 118 5.1. The pBP Model . 119 5.1.1. Definition of a General pBP Algorithm . 120 5.1.2. Submodels of the pBP Model . 124 5.2. Examples . 126 5.3. Simulations of pBPs by pBTs . 132 5.4. Notes . 136 Chapter 6. Open Questions . 137 6.1. Inside the Existing Models . 139 6.2. Are the Containments Proper? . 140 6.3. Beyond pFBP . 140 6.4. Beyond the Basic Algorithmic Paradigms . 141 Appendix A. Priority Formalization for Adaptive Contract . 143 Appendix B. Concentration Inequalities and Some Simple Bounds . 151 B.1. Chernoff Style Bounds and Chebyshev's Inequality . 151 B.2. Martingale and Submatringale Concentration Inequalities . 152 B.2.1. Properties of Conditional Expectation . 152 B.2.2. Doob Martingales . 153 B.2.3. Submartingales . 156 Bibliography . 158 vi LIST OF FIGURES Figure 2.1. Adversary selects Γ = x; y; z; u; v; w : . 25 1 f g Figure 2.2. The set of edges initially selected by the Adversary Γ = a; b; c; d; e; f 26 1 f g Figure 2.3. K6 with three required and three Steiner nodes . 33 Figure 2.4. Nemesis graphs chosen by the Adversary . 49 Figure 2.5. The nemesis graph for MIS problem. 59 Figure 2.6. Fraction of the cycle considered. 61 Figure 3.1. Example of a computation of a backtracking algorithm on an instance I = 223223. 69 Figure 3.2. Transition function of state B=D. 101 Figure 5.1. Submodel Lattice . 125 Figure 5.2. pBP DAG for LIS on instance 5; 1; 2; 3 . 128 Figure 5.3. Example SSSP instance . 130 Figure 5.4. pBP DAG for the instance graph on Figure 5.3 . 130 Figure 6.1. Lattice of the algorithmic models. SPN stands for the problem of finding shortest paths in graphs with negative weights; LCS stands for the longest common subsequence problem; MCM stands for matrix chain multiplication prob- lem; OBT stands for the optimal binary tree problem. 138 vii ACKNOWLEDGEMENTS My deepest gratitude goes to my advisor Russell Impagliazzo. The road of my doctoral studies was long and his guidance was invaluable. The work presented in Chapter 2 appeared in the proceedings of the 2004 SIAM SODA conference, “Models of Greedy Algorithms for Graph Problems”, by Sashka Davis and Russell Impagliazzo, [20]. The full version “Models of Greedy Algorithms for Graph Problems”, by Sashka Davis and Russell Impagliazzo, was published in Al- gorithmica in 2007, [21]. The results in Chapter 3 are part of manuscript “A General Model for Back- tracking and Dynamic Programming Algorithms”, by Josh Buresh-Oppenheim, Sashka Davis and Russell Impagliazzo, [16], to be published in the future. The results presented in Chapters 4 and 5 together with results which are not included in this dissertation are part of manuscript “A Stronger Model for Dynamic Programming Algorithms”, by Josh Buresh-Oppenheim, Sashka Davis and Russell Im- pagliazzo, [17], to be published in the future. viii VITA 1991 Bachelor of Science, Technical University Sofia, Bulgaria 1999 Master of Science, Rochester Institute of Technology, USA 2008 Doctor of Philosophy, University of California, San Diego, USA PUBLICATIONS Josh Buresh-Oppenheim, Sashka Davis, Russell Impagliazzo, “A Stronger Model for Dynamic Programming Algorithms”, 2008 manuscript in preparation. Josh Buresh-Oppenheim, Sashka Davis, Russell Impagliazzo, “A General Formal Model for Backtracking and Dynamic Programming Algorithms”, 2008 manuscript in prepara- tion. Sashka Davis, Russell Impagliazzo, “Models of Greedy Algorithms for Graph Optimiza- tion Problem”, Algorithmica 2007. Sashka Davis, “Evaluating Algorithmic Design Paradigms”, 2006 Grace Hopper Cele- bration of Women in Computing. Sashka Davis, Jeff Edmonds, Russell Impagliazzo, “Online Algorithms To Minimize Resource Reallocations and Network Communication”, Proceedings, Lecture Notes in Computer Science 4110, Springer 2006. Sashka Davis, Russell Impagliazzo, “Models of greedy algorithms for graph problems”, Proceedings of the 2004 SIAM SODA. Sashka Davis, “Hu-Tucker Algorithm for Building Optimal Alphabetic Binary Search Trees” Technical Report RIT-99-019, Computer Science Dept., Rochester Institute of Technology, 1999. FIELDS OF STUDY Bachelor of Science in Computer Engineering Master of Science in Computer Science Professor Stanislaw Radziszowski Doctor of Philosophy in Computer Science Professor Russell Impagliazzo ix ABSTRACT OF THE DISSERTATION On the Power of the Basic Algorithmic Design Paradigms by Sashka Tchameva Davis Doctor of Philosophy in Computer Science University of California, San Diego, 2008 Russell Impagliazzo, Chair This dissertation formalizes the intuitive notion of the basic algorithmic paradigms. We present three formal models which aim to capture the intrinsic power of greedy, back- tracking and dynamic programming algorithms. We develop lower bound techniques for proving negative results for all algorithms in all models, which allow us to make strong statements about the limitations of each paradigm. [14] designed the Priority algorithms, a formal model of greedy algorithms for scheduling problems. We generalized the priority model to arbitrary problem domain and in particular graph problems and develop a lower bound technique for proving neg- ative results for the class of all priority algorithms. We use the lower bound technique to show that finding shortest path in graphs with negative weights cannot be solved by a priority algorithm. We also prove that Dijkstra's algorithm is inherently adaptive and cannot be made non-adaptive. We show inapproximability results within the model for minimum weighted vertex cover, minimum metric Steiner tree, and maximum indepen- dent set problems. We develop a new 1:8-approximation scheme for the Steiner(1; 2) x problem. [1] presented a model of backtracking and dynamic programming algorithms called prioritized Branching Trees (pBT). We generalize their model to allow free branch- ing and call this new model prioritized Free Branching Tree (pFBT) algorithms and de- veloped a lower bound technique for proving negative results for randomized priority algorithms, pBT and pFBT algorithms. We use the technique to prove that pBT algo- rithms require exponential width to solve the 7-SAT problem and that pFBT algorithms require width 2Ω(pn) to solve the 7-SAT problem. Bellman-Ford is a classical dynamic programming algorithm and we show that pBT algorithms require width of 2Ω(n1=9) to solve the shortest path problem in graphs with negative weights exactly. Next we develop a stronger model of dynamic programming algorithms called prioritized Branching Programs (pBP). pBP algorithms can simulate pBT algorithms at no additional cost but also capture the notion of memoization which we believe is an essential part of the dynamic programming paradigm. We show that this class of algorithms can solve the shortest paths in graphs with negative weights but no negative cycles efficiently. We also show that two pBP sub-models can be simulated by pBT algorithms. xi Chapter 1 Do We Need Formal Models of the Algorithmic Paradigms? In an algorithm design class, we are taught the basic algorithm paradigms such as divide-and-conquer, greedy algorithms, backtracking and dynamic programming. The paradigm is taught by an intuitive example together with a number of counter examples. Intuitive formulations, while easy to understand, do not allow us to answer the following natural questions. Suppose we have an optimization problem that we want to solve.