DOCSLIB.ORG

Computational Complexity of Some Optimization Problems in Planning

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Linköping Studies in Science and Technology Dissertation No. 1854 Computational Complexity of some Optimization Problems in Planning Meysam Aghighi Linköping University Department of Computer and Information Science SE-581 85 Linköping, Sweden Linköping 2017 c Meysam Aghighi, 2017 ISBN 978-91-7685-519-5 ISSN 0345-7524 URL http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-136280/ Published articles have been reprinted with permission from the respective copyright holder. Typeset using LATEX Printed by LiU-Tryck, Linköping 2017 ii Computational Complexity of some Optimization Problems in Planning ABSTRACT Automated planning is known to be computationally hard in the general case. Propositional planning is PSPACE-complete and first-order planning is undecidable. One method for ana- lyzing the computational complexity of planning is to study restricted subsets of planning in- stances, with the aim of differentiating instances with varying complexity. We use this method- ology for studying the computational complexity of planning. Finding new tractable (i.e. polynomial-time solvable) problems has been a particularly impor- tant goal for researchers in the area. The reason behind this is not only to differentiate between easy and hard planning instances, but also to use polynomial-time solvable instances in order to construct better heuristic functions and improve planners. We identify a new class of tractable cost-optimal planning instances by restricting the causal graph. We also study the computational complexity of oversubscription planning (such as the net-benefit problem) under various restrictions and reveal strong connections with classical planning. Inspired by this, we present a method for compiling oversubscription planning prob- lems into the ordinary plan existence problem. We further study the parameterized complexity of cost-optimal and net-benefit planning under the same restrictions and show that the choice of numeric domain for the action costs has a great impact on the parameterized complexity. We finally consider the parameterized complexity of certain problems related to partial-order planning. In some applications, less restricted plans than total-order plans are needed. There- fore, a partial-order plan is being used instead. When dealing with partial-order plans, one important question is how to achieve optimal partial order plans, i.e. having the highest de- gree of freedom according to some notion of flexibility. We study several optimization problems for partial-order plans, such as finding a minimum deordering or reordering, and finding the minimum parallel execution length. The research presented in this thesis has been partially funded by the National Graduate School of Com- puter Science in Sweden (CUGS). iii Populärvetenskaplig sammanfattning Huvudtemat i denna avhandling är studiet av beräkningskomplexitet för automatisk planering. På en hög nivå kan man betrakta planeringsproblemet så här; man utgår från en värld eller ett system, som ett obemannat fordon eller en autonom robot, och antar att det finns vissa handlingar som ändrar tillståndet i systemet. Man antar dessutom att det finns ett initial- och ett måltillstånd. Planering är problemet att hitta en sekvens av handlingar som tranformerar initialtillståndet till måltillståndet. En sådan sekvens av handlingar kallas en plan. En planerare är ett datorprogram som löser planeringsproblem, och planerare baseras ofta på sökning genom tillstånden i systemet. Vanligtvis vill man optimera sökningen och hitta en plan som är optimal med avseende på något kriterium. Vad man vill optimera beror på tillämpningen men det är ofta att hitta en kortaste eller billigaste plan. En viktig fråga när man studerar automatisk planering är hur snabbt man kan hitta lösningar. Ett sätt att analysera denna fråga är att studera beräkningskomplexiteten för olika planeringsproblem. Komplexitetsteori är den gren av datavetenskapen som försöker klassificera hur svåra olika beräkningsproblem är, det vill säga analysera hur mycket resurser (såsom tid och minne) som behövs. Man likställer ofta effektivt lösbara problem med de som kan lösas i polynomisk tid: ett problem kan lösas i polynomisk tid om det finns en algoritm vars tidsåtgång begränsas av ett polynom i indatas storlek. Det grundläggande planeringsproblemet är exempelvis känt att vara PSPACE-fullständigt. PSPACE är den komplexitetsklass som innehåller alla de beräkningsproblem som kan lösas genom att använda polynomiskt mycket minne och de PSPACE-fullständiga problemen utgör de svåraste problemen i PSPACE. Man tror att det inte finns några PSPACE- fullständiga problem som kan lösas i polynomisk tid—detta är ett viktigt olöst problem inom teoretisk datalogi—och man förväntar sig därför inte att det generella planeringsproblemet kan lösas i polynomisk tid. En metod för att analysera beräkningskomplexitet av planering är att studera begränsade delmängder av planeringsinstanser i syfte att särskilja fall med varierande komplexitet. Vi använder denna metod genomgående i denna avhandling. Så kallad “oversubscription planning” är en typ av planering där man mäter hur nära ett givet måltillstånd v man kan komma. Vi studerar beräkningskomplexiteten för denna typ av planering under olika restriktioner och avslöjar starka kopplingar med vanlig planering. Kostnadsoptimal planering är planering där varje handling har en kostnad och målet är att hitta en plan med minimal kostnad. Dessa handlingskostnader kan hämtas från olika numeriska domäner såsom de positiva heltalen eller de rationella talen. Vi studerar effekter av valet av numeriska domäner och noterar intressanta komplexitetsmässiga skillnader. För kostnadsoptimal planering identifierar vi dessutom en ny klass av instanser som kan lösas i polynomisk tid. Slutligen studerar vi komplexiteten för olika problem inom så kallad partialordningsplanering där man tillåter planer som inte nödvändigtvis är sekvenser av handlingar. vi Acknowledgments First and foremost, I want to thank my advisors Peter Jonsson and Christer Bäckström. It has been an honor to work with and learn from them for the past three years. They have taught me how well the academic research is done. I am specially grateful to Peter for his great help and Christer for his valuable comments in preparing this thesis. The members of TCSLAB and IDA contributed immensely to my per- sonal and professional time as a PhD student at LiU. I specifically thank Si- mon Ståhlberg, who was both a colleague and a friend, Amir Aminifar, Bi- man Roy, Arian Maghazeh, Adrian Horga, Rouhollah Mahfouzi and Victor Lagerkvist. I am also grateful to the administration at IDA that facilitated my work in less direct ways. I specially thank Karin Baardsen, Anne Moe, Marie Jo- hansson, Inger Norén and the director of graduate studies, Professor Petru Eles. Lastly, I would like to thank my family for all their love and encourage- ment. For my parents who raised me with a love of science and supported me in all my pursuits. For my loving, supporting, encouraging and patient wife Zeinab whose support during my whole PhD is so appreciated. Thank you. MEYSAM AGHIGHI Linköping University May 2017 vii List of Papers The thesis is based on the following papers: 1. Meysam Aghighi and Peter Jonsson. Oversubscription planning: Complexity and compilability. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelli- gence, Québec City, Québec, Canada, July 27 -31, pages 2221-2227, 2014. 2. Meysam Aghighi, Peter Jonsson and Simon Ståhlberg. Tractable cost-optimal planning over restricted polytree causal graphs. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelli- gence, Austin, Texas, USA, January 25-30, pages 3225-3231, 2015. 3. Meysam Aghighi and Christer Bäckström. Cost-optimal and net-benefit planning - A parameterised complexity view. In Proceedings of the Twenty-Fourth International Joint Conference on Arti- ficial Intelligence (IJCAI), Buenos Aires, Argentina, July 25-31, pages 1487- 1493, 2015. 4. Meysam Aghighi and Christer Bäckström. A multi-parameter complexity analysis of cost-optimal and net-benefit planning. In Proceedings of the Twenty-Sixth International Conference on Automated Planning and Scheduling (ICAPS), London, UK, June 12-17, pages 2-10, 2016. 5. Meysam Aghighi and Christer Bäckström. Plan reordering and parallel execution – a parameterized complexity view. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, California, USA, February 4-9, pages 3540-3546, 2017. In addition to the above papers, the author has also contributed to the fol- lowing publications: 1. Meysam Aghighi, Christer Bäckström, Peter Jonsson and Simon Ståhlberg. Analysing Approximability and Heuristics in Planning Using the ix Exponential-time Hypothesis. In Proceedings of the Twenty-second European Conference on Artificial Intel- ligence (ECAI) , the Hague, Holland, August 29th - September 2nd, pages 184-192, 2016. 2. Meysam Aghighi, Christer Bäckström, Peter Jonsson and Simon Ståhlberg. Refining Complexity Analyses in Planning by Exploiting the Exponential-time Hypothesis. In Annals of Mathematics and Artificial Intelligence (AMAI), pages 157-175, 2016. x Contents Abstract iii Populärvetenskaplig Sammanfattning v Acknowledgments vii List of Papers ix Contents xi I Introduction 1 1 Thesis Overview 3 1 About this Thesis . 3 2 Brief Summary of Papers . 4 2 Computational Complexity Theory 7 1 Computational Complexity . 7 2 Parameterized Complexity . 11 3 Automated Planning 15 1 What is Planning? . 15 2 Planning
Recommended publications
  • 4. Convex Optimization Problems

    4. Convex Optimization Problems

    Convex Optimization — Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form • convex optimization problems • quasiconvex optimization • linear optimization • quadratic optimization • geometric programming • generalized inequality constraints • semidefinite programming • vector optimization • 4–1 Optimization problem in standard form minimize f0(x) subject to f (x) 0, i =1,...,m i ≤ hi(x)=0, i =1,...,p x Rn is the optimization variable • ∈ f : Rn R is the objective or cost function • 0 → f : Rn R, i =1,...,m, are the inequality constraint functions • i → h : Rn R are the equality constraint functions • i → optimal value: p⋆ = inf f (x) f (x) 0, i =1,...,m, h (x)=0, i =1,...,p { 0 | i ≤ i } p⋆ = if problem is infeasible (no x satisfies the constraints) • ∞ p⋆ = if problem is unbounded below • −∞ Convex optimization problems 4–2 Optimal and locally optimal points x is feasible if x dom f and it satisfies the constraints ∈ 0 ⋆ a feasible x is optimal if f0(x)= p ; Xopt is the set of optimal points x is locally optimal if there is an R> 0 such that x is optimal for minimize (over z) f0(z) subject to fi(z) 0, i =1,...,m, hi(z)=0, i =1,...,p z x≤ R k − k2 ≤ examples (with n =1, m = p =0) f (x)=1/x, dom f = R : p⋆ =0, no optimal point • 0 0 ++ f (x)= log x, dom f = R : p⋆ = • 0 − 0 ++ −∞ f (x)= x log x, dom f = R : p⋆ = 1/e, x =1/e is optimal • 0 0 ++ − f (x)= x3 3x, p⋆ = , local optimum at x =1 • 0 − −∞ Convex optimization problems 4–3 Implicit constraints the standard form optimization problem has an implicit
  • Metaheuristics1

    Metaheuristics1

    METAHEURISTICS1 Kenneth Sörensen University of Antwerp, Belgium Fred Glover University of Colorado and OptTek Systems, Inc., USA 1 Definition A metaheuristic is a high-level problem-independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms (Sörensen and Glover, To appear). Notable examples of metaheuristics include genetic/evolutionary algorithms, tabu search, simulated annealing, and ant colony optimization, although many more exist. A problem-specific implementation of a heuristic optimization algorithm according to the guidelines expressed in a metaheuristic framework is also referred to as a metaheuristic. The term was coined by Glover (1986) and combines the Greek prefix meta- (metá, beyond in the sense of high-level) with heuristic (from the Greek heuriskein or euriskein, to search). Metaheuristic algorithms, i.e., optimization methods designed according to the strategies laid out in a metaheuristic framework, are — as the name suggests — always heuristic in nature. This fact distinguishes them from exact methods, that do come with a proof that the optimal solution will be found in a finite (although often prohibitively large) amount of time. Metaheuristics are therefore developed specifically to find a solution that is “good enough” in a computing time that is “small enough”. As a result, they are not subject to combinatorial explosion – the phenomenon where the computing time required to find the optimal solution of NP- hard problems increases as an exponential function of the problem size. Metaheuristics have been demonstrated by the scientific community to be a viable, and often superior, alternative to more traditional (exact) methods of mixed- integer optimization such as branch and bound and dynamic programming.
  • On the Complexity of Numerical Analysis

    On the Complexity of Numerical Analysis

    On the Complexity of Numerical Analysis Eric Allender Peter B¨urgisser Rutgers, the State University of NJ Paderborn University Department of Computer Science Department of Mathematics Piscataway, NJ 08854-8019, USA DE-33095 Paderborn, Germany [email protected] [email protected] Johan Kjeldgaard-Pedersen Peter Bro Miltersen PA Consulting Group University of Aarhus Decision Sciences Practice Department of Computer Science Tuborg Blvd. 5, DK 2900 Hellerup, Denmark IT-parken, DK 8200 Aarhus N, Denmark [email protected] [email protected] Abstract In Section 1.3 we discuss our main technical contribu- tions: proving upper and lower bounds on the complexity We study two quite different approaches to understand- of PosSLP. In Section 1.4 we present applications of our ing the complexity of fundamental problems in numerical main result with respect to the Euclidean Traveling Sales- analysis. We show that both hinge on the question of under- man Problem and the Sum-of-Square-Roots problem. standing the complexity of the following problem, which we call PosSLP: Given a division-free straight-line program 1.1 Polynomial Time Over the Reals producing an integer N, decide whether N>0. We show that PosSLP lies in the counting hierarchy, and combining The Blum-Shub-Smale model of computation over the our results with work of Tiwari, we show that the Euclidean reals provides a very well-studied complexity-theoretic set- Traveling Salesman Problem lies in the counting hierarchy ting in which to study the computational problems of nu- – the previous best upper bound for this important problem merical analysis.
  • IEOR 269, Spring 2010 Integer Programming and Combinatorial Optimization

    IEOR 269, Spring 2010 Integer Programming and Combinatorial Optimization

    IEOR 269, Spring 2010 Integer Programming and Combinatorial Optimization Professor Dorit S. Hochbaum Contents 1 Introduction 1 2 Formulation of some ILP 2 2.1 0-1 knapsack problem . 2 2.2 Assignment problem . 2 3 Non-linear Objective functions 4 3.1 Production problem with set-up costs . 4 3.2 Piecewise linear cost function . 5 3.3 Piecewise linear convex cost function . 6 3.4 Disjunctive constraints . 7 4 Some famous combinatorial problems 7 4.1 Max clique problem . 7 4.2 SAT (satisfiability) . 7 4.3 Vertex cover problem . 7 5 General optimization 8 6 Neighborhood 8 6.1 Exact neighborhood . 8 7 Complexity of algorithms 9 7.1 Finding the maximum element . 9 7.2 0-1 knapsack . 9 7.3 Linear systems . 10 7.4 Linear Programming . 11 8 Some interesting IP formulations 12 8.1 The fixed cost plant location problem . 12 8.2 Minimum/maximum spanning tree (MST) . 12 9 The Minimum Spanning Tree (MST) Problem 13 i IEOR269 notes, Prof. Hochbaum, 2010 ii 10 General Matching Problem 14 10.1 Maximum Matching Problem in Bipartite Graphs . 14 10.2 Maximum Matching Problem in Non-Bipartite Graphs . 15 10.3 Constraint Matrix Analysis for Matching Problems . 16 11 Traveling Salesperson Problem (TSP) 17 11.1 IP Formulation for TSP . 17 12 Discussion of LP-Formulation for MST 18 13 Branch-and-Bound 20 13.1 The Branch-and-Bound technique . 20 13.2 Other Branch-and-Bound techniques . 22 14 Basic graph definitions 23 15 Complexity analysis 24 15.1 Measuring quality of an algorithm .
  • Interactions of Computational Complexity Theory and Mathematics

    Interactions of Computational Complexity Theory and Mathematics

    Interactions of Computational Complexity Theory and Mathematics Avi Wigderson October 22, 2017 Abstract [This paper is a (self contained) chapter in a new book on computational complexity theory, called Mathematics and Computation, whose draft is available at https://www.math.ias.edu/avi/book]. We survey some concrete interaction areas between computational complexity theory and different fields of mathematics. We hope to demonstrate here that hardly any area of modern mathematics is untouched by the computational connection (which in some cases is completely natural and in others may seem quite surprising). In my view, the breadth, depth, beauty and novelty of these connections is inspiring, and speaks to a great potential of future interactions (which indeed, are quickly expanding). We aim for variety. We give short, simple descriptions (without proofs or much technical detail) of ideas, motivations, results and connections; this will hopefully entice the reader to dig deeper. Each vignette focuses only on a single topic within a large mathematical filed. We cover the following: • Number Theory: Primality testing • Combinatorial Geometry: Point-line incidences • Operator Theory: The Kadison-Singer problem • Metric Geometry: Distortion of embeddings • Group Theory: Generation and random generation • Statistical Physics: Monte-Carlo Markov chains • Analysis and Probability: Noise stability • Lattice Theory: Short vectors • Invariant Theory: Actions on matrix tuples 1 1 introduction The Theory of Computation (ToC) lays out the mathematical foundations of computer science. I am often asked if ToC is a branch of Mathematics, or of Computer Science. The answer is easy: it is clearly both (and in fact, much more). Ever since Turing's 1936 definition of the Turing machine, we have had a formal mathematical model of computation that enables the rigorous mathematical study of computational tasks, algorithms to solve them, and the resources these require.
  • Quantum Computational Complexity Theory Is to Un- Derstand the Implications of Quantum Physics to Computational Complexity Theory

    Quantum Computational Complexity Theory Is to Un- Derstand the Implications of Quantum Physics to Computational Complexity Theory

    Quantum Computational Complexity John Watrous Institute for Quantum Computing and School of Computer Science University of Waterloo, Waterloo, Ontario, Canada. Article outline I. Definition of the subject and its importance II. Introduction III. The quantum circuit model IV. Polynomial-time quantum computations V. Quantum proofs VI. Quantum interactive proof systems VII. Other selected notions in quantum complexity VIII. Future directions IX. References Glossary Quantum circuit. A quantum circuit is an acyclic network of quantum gates connected by wires: the gates represent quantum operations and the wires represent the qubits on which these operations are performed. The quantum circuit model is the most commonly studied model of quantum computation. Quantum complexity class. A quantum complexity class is a collection of computational problems that are solvable by a cho- sen quantum computational model that obeys certain resource constraints. For example, BQP is the quantum complexity class of all decision problems that can be solved in polynomial time by a arXiv:0804.3401v1 [quant-ph] 21 Apr 2008 quantum computer. Quantum proof. A quantum proof is a quantum state that plays the role of a witness or certificate to a quan- tum computer that runs a verification procedure. The quantum complexity class QMA is defined by this notion: it includes all decision problems whose yes-instances are efficiently verifiable by means of quantum proofs. Quantum interactive proof system. A quantum interactive proof system is an interaction between a verifier and one or more provers, involving the processing and exchange of quantum information, whereby the provers attempt to convince the verifier of the answer to some computational problem.
  • Introduction to Integer Programming – Integer Programming Models

    Introduction to Integer Programming – Integer Programming Models

    15.053/8 March 14, 2013 Introduction to Integer Programming – Integer programming models 1 Quotes of the Day “Somebody who thinks logically is a nice contrast to the real world.” -- The Law of Thumb “Take some more tea,” the March Hare said to Alice, very earnestly. “I’ve had nothing yet,” Alice replied in an offended tone, “so I can’t take more.” “You mean you can’t take less,” said the Hatter. “It’s very easy to take more than nothing.” -- Lewis Carroll in Alice in Wonderland 2 Combinatorial optimization problems INPUT: A description of the data for an instance of the problem FEASIBLE SOLUTIONS: there is a way of determining from the input whether a given solution x’ (assignment of values to decision variables) is feasible. Typically in combinatorial optimization problems there is a finite number of possible solutions. OBJECTIVE FUNCTION: For each feasible solution x’ there is an associated objective f(x’). Minimization problem. Find a feasible solution x* that minimizes f( ) among all feasible solutions. 3 Example 1: Traveling Salesman Problem INPUT: a set N of n points in the plane FEASIBLE SOLUTION: a tour that passes through each point exactly once. OBJECTIVE: minimize the length of the tour. 4 Example 2: Balanced Partition INPUT: A set of positive integers a1, …, an FEASIBLE SOLUTION: a partition of {1, 2, … n} into two disjoint sets S and T. – S ∩ T = ∅, S∪T = {1, … , n} OBJECTIVE : minimize | ∑i∈S ai - ∑i∈T ai | Example: 7, 10, 13, 17, 20, 22 These numbers sum to 89 The best split is {10, 13, 22} and {7, 17, 20}.
  • Algorithmic Complexity in Computational Biology: Basics, Challenges and Limitations

    Algorithmic Complexity in Computational Biology: Basics, Challenges and Limitations

    Algorithmic complexity in computational biology: basics, challenges and limitations Davide Cirillo#,1,*, Miguel Ponce-de-Leon#,1, Alfonso Valencia1,2 1 Barcelona Supercomputing Center (BSC), C/ Jordi Girona 29, 08034, Barcelona, Spain 2 ICREA, Pg. Lluís Companys 23, 08010, Barcelona, Spain. # Contributed equally * corresponding author: [email protected] (Tel: +34 934137971) Key points: ● Computational biologists and bioinformaticians are challenged with a range of complex algorithmic problems. ● The significance and implications of the complexity of the algorithms commonly used in computational biology is not always well understood by users and developers. ● A better understanding of complexity in computational biology algorithms can help in the implementation of efficient solutions, as well as in the design of the concomitant high-performance computing and heuristic requirements. Keywords: Computational biology, Theory of computation, Complexity, High-performance computing, Heuristics Description of the author(s): Davide Cirillo is a postdoctoral researcher at the Computational biology Group within the Life Sciences Department at Barcelona Supercomputing Center (BSC). Davide Cirillo received the MSc degree in Pharmaceutical Biotechnology from University of Rome ‘La Sapienza’, Italy, in 2011, and the PhD degree in biomedicine from Universitat Pompeu Fabra (UPF) and Center for Genomic Regulation (CRG) in Barcelona, Spain, in 2016. His current research interests include Machine Learning, Artificial Intelligence, and Precision medicine. Miguel Ponce de León is a postdoctoral researcher at the Computational biology Group within the Life Sciences Department at Barcelona Supercomputing Center (BSC). Miguel Ponce de León received the MSc degree in bioinformatics from University UdelaR of Uruguay, in 2011, and the PhD degree in Biochemistry and biomedicine from University Complutense de, Madrid (UCM), Spain, in 2017.
  • Computational Complexity

    Computational Complexity

    Computational Complexity The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Vadhan, Salil P. 2011. Computational complexity. In Encyclopedia of Cryptography and Security, second edition, ed. Henk C.A. van Tilborg and Sushil Jajodia. New York: Springer. Published Version http://refworks.springer.com/mrw/index.php?id=2703 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:33907951 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#OAP Computational Complexity Salil Vadhan School of Engineering & Applied Sciences Harvard University Synonyms Complexity theory Related concepts and keywords Exponential time; O-notation; One-way function; Polynomial time; Security (Computational, Unconditional); Sub-exponential time; Definition Computational complexity theory is the study of the minimal resources needed to solve computational problems. In particular, it aims to distinguish be- tween those problems that possess efficient algorithms (the \easy" problems) and those that are inherently intractable (the \hard" problems). Thus com- putational complexity provides a foundation for most of modern cryptogra- phy, where the aim is to design cryptosystems that are \easy to use" but \hard to break". (See security (computational, unconditional).) Theory Running Time. The most basic resource studied in computational com- plexity is running time | the number of basic \steps" taken by an algorithm. (Other resources, such as space (i.e., memory usage), are also studied, but they will not be discussed them here.) To make this precise, one needs to fix a model of computation (such as the Turing machine), but here it suffices to informally think of it as the number of \bit operations" when the input is given as a string of 0's and 1's.
  • Integer Optimization Methods for Machine Learning by Allison an Chang Sc.B

    Integer Optimization Methods for Machine Learning by Allison an Chang Sc.B

    Integer Optimization Methods for Machine Learning by Allison An Chang Sc.B. Applied Mathematics, Brown University (2007) Submitted to the Sloan School of Management in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Operations Research at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2012 c Massachusetts Institute of Technology 2012. All rights reserved. Author............................................. ................. Sloan School of Management May 18, 2012 Certified by......................................... ................. Dimitris Bertsimas Boeing Leaders for Global Operations Professor Co-Director, Operations Research Center Thesis Supervisor Certified by......................................... ................. Cynthia Rudin Assistant Professor of Statistics Thesis Supervisor Accepted by......................................... ................ Patrick Jaillet Dugald C. Jackson Professor Department of Electrical Engineering and Computer Science Co-Director, Operations Research Center 2 Integer Optimization Methods for Machine Learning by Allison An Chang Submitted to the Sloan School of Management on May 18, 2012, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Operations Research Abstract In this thesis, we propose new mixed integer optimization (MIO) methods to ad- dress problems in machine learning. The first part develops methods for supervised bipartite ranking, which arises in prioritization tasks in diverse domains such as infor- mation retrieval, recommender systems, natural language processing, bioinformatics, and preventative maintenance. The primary advantage of using MIO for ranking is that it allows for direct optimization of ranking quality measures, as opposed to current state-of-the-art algorithms that use heuristic loss functions. We demonstrate using a number of datasets that our approach can outperform other ranking methods. The second part of the thesis focuses on reverse-engineering ranking models.
  • Notes for Lecture Notes 2 1 Computational Problems

    Notes for Lecture Notes 2 1 Computational Problems

    Stanford University | CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation for search problems is equivalent to its formulation for decision problems, and we prove the time hierarchy theorem, which remains essentially the only known theorem which allows to show that certain problems are not in P. 1 Computational Problems In a computational problem, we are given an input that, without loss of generality, we assume to be encoded over the alphabet f0; 1g, and we want to return as output a solution satisfying some property: a computational problem is then described by the property that the output has to satisfy given the input. In this course we will deal with four types of computational problems: decision problems, search problems, optimization problems, and counting problems.1 For the moment, we will discuss decision and search problem. In a decision problem, given an input x 2 f0; 1g∗, we are required to give a YES/NO answer. That is, in a decision problem we are only asked to verify whether the input satisfies a certain property. An example of decision problem is the 3-coloring problem: given an undirected graph, determine whether there is a way to assign a \color" chosen from f1; 2; 3g to each vertex in such a way that no two adjacent vertices have the same color. A convenient way to specify a decision problem is to give the set L ⊆ f0; 1g∗ of inputs for which the answer is YES.
  • CMSC 451 Design and Analysis of Computer Algorithms1

    CMSC 451 Design and Analysis of Computer Algorithms1

    CMSC 451 Design and Analysis of Computer Algorithms1 David M. Mount Department of Computer Science University of Maryland Fall 2003 1Copyright, David M. Mount, 2004, Dept. of Computer Science, University of Maryland, College Park, MD, 20742. These lecture notes were prepared by David Mount for the course CMSC 451, Design and Analysis of Computer Algorithms, at the University of Maryland. Permission to use, copy, modify, and distribute these notes for educational purposes and without fee is hereby granted, provided that this copyright notice appear in all copies. Lecture Notes 1 CMSC 451 Lecture 1: Course Introduction Read: (All readings are from Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 2nd Edition). Review Chapts. 1–5 in CLRS. What is an algorithm? Our text defines an algorithm to be any well-defined computational procedure that takes some values as input and produces some values as output. Like a cooking recipe, an algorithm provides a step-by-step method for solving a computational problem. Unlike programs, algorithms are not dependent on a particular programming language, machine, system, or compiler. They are mathematical entities, which can be thought of as running on some sort of idealized computer with an infinite random access memory and an unlimited word size. Algorithm design is all about the mathematical theory behind the design of good programs. Why study algorithm design? Programming is a very complex task, and there are a number of aspects of program- ming that make it so complex. The first is that most programming projects are very large, requiring the coor- dinated efforts of many people.