DOCSLIB.ORG

The Complexity of Gradient Descent: CLS = PPAD ∩ PLS

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Complexity of Gradient Descent: CLS = PPAD ∩ PLS The Complexity of Gradient Descent: CLS = PPAD ∩ PLS ALEXANDROS HOLLENDER JOINT WORK WITH JOHN FEARNLEY, PAUL GOLDBERG AND RAHUL SAVANI Some interesting computational problems Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. Some interesting computational problems NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: Find a prime factor of a number 푛 ≥ 2. BROUWER: Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: They are NP Total Search (TFNP) problems! Find a prime factor of a number 푛 ≥ 2. • Total: there is always a solution BROUWER: • NP: it is easy to verify solutions Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: They are NP Total Search (TFNP) problems! Find a prime factor of a number 푛 ≥ 2. • Total: there is always a solution BROUWER: • NP: it is easy to verify solutions Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. Can a TFNP problem be NP-hard? PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Some interesting computational problems What do these problems have in common? NASH: Find a mixed Nash equilibrium of a game. FACTORING: They are NP Total Search (TFNP) problems! Find a prime factor of a number 푛 ≥ 2. • Total: there is always a solution BROUWER: • NP: it is easy to verify solutions Find a fixpoint of a continuous function 푓: 0,1 3→ [0,1]3. CONTRACTION: Find the unique fixpoint of a contraction 푓: 0,1 푛→ [0,1]푛. Can a TFNP problem be NP-hard? PURE-CONGESTION: Find a pure Nash equilibrium of a congestion game. Not unless co-NP = NP… The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution TFNP lies between P and NP (search versions) The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… 3-SAT ≤ NASH ⇒ certificate for unsatisfiable 3-SAT formulas The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… The class TFNP [Megiddo-Papadimitriou, 1991] Total NP search problems: • “search” : looking for a solution, not just YES or NO • “NP”: any solution can be checked efficiently • “total”: there always exists at least one solution How do we show that a TFNP-problem is hard: ▪ No TFNP-problem can be NP-hard, unless NP = coNP… ▪ Believed that no TFNP-complete problems exists… The TFNP landscape TFNP P The TFNP landscape Pigeonhole Principle TFNP PPP P The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA PPP P The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA PPP PLS Local Search Argument P PURE-CONGESTION LOCAL-MAX-CUT The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA PPP PPAD PLS Local Search Argument Directed Graph Argument P PURE-CONGESTION NASH LOCAL-MAX-CUT BROUWER The TFNP landscape Pigeonhole Principle Parity Argument TFNP Borsuk-Ulam PPA FACTORING PPP PPAD PLS Local Search Argument Directed Graph Argument P PURE-CONGESTION NASH LOCAL-MAX-CUT BROUWER TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? • many seemingly hard problems lie in PPAD, PLS etc… TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? • many seemingly hard problems lie in PPAD, PLS etc… • oracle separations between the classes (in particular PPAD ≠ PLS) TFNP subclasses What reasons are there to believe that PPAD ≠ P, PLS ≠ P, etc? • many seemingly hard problems lie in PPAD, PLS etc… • oracle separations between the classes (in particular PPAD ≠ PLS) • hard under cryptographic assumptions TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P-LCP P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P-LCP TARSKI P TFNP PPAD PLS NASH LOCAL-MAX-CUT BROUWER PURE-CONGESTION PPAD ∩ PLS CONTRACTION MIXED-CONGESTION P-LCP TARSKI SSGs P PPAD ∩ PLS seems unnatural… PPAD ∩ PLS seems unnatural… Problem 퐴 : PPAD-complete Problem 퐵 : PLS-complete PPAD ∩ PLS seems unnatural… Problem 퐴 : PPAD-complete Problem 퐵 : PLS-complete EITHER-SOLUTION(푨,푩): Input: instance 퐼퐴 of 퐴, instance 퐼퐵 of 퐵 Goal: find a solution of 퐼퐴, or a solution of 퐼퐵 PPAD ∩ PLS seems unnatural… Problem 퐴 : PPAD-complete Problem 퐵 : PLS-complete EITHER-SOLUTION(푨,푩): Input: instance 퐼퐴 of 퐴, instance 퐼퐵 of 퐵 Goal: find a solution of 퐼퐴, or a solution of 퐼퐵 → EITHER-SOLUTION(푨,푩) is (PPAD ∩ PLS)-complete! PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛 Goal: find a fixpoint 푥 푓 푥 = 푥 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 Goal: find a local minimum of 푝 with respect to 푔 푝 푔 푥 ≥ 푝 푥 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 Goal: find a local minimum of 푝 with respect to 푔 푝 푔 푥 ≥ 푝 푥 − 휀 PPAD ∩ PLS seems unnatural… BROUWER: Input: a continuous function 푓: 0,1 푛→ [0,1]푛, precision 휀 > 0 Goal: find an approximate fixpoint 푥 푓 푥 − 푥 ≤ 휀 REAL-LOCAL-OPT: Input: - a continuous function 푝: [0,1]푛→ 0,1 - a (possibly non-continuous) function 푔: 0,1 푛→ [0,1]푛 Goal: find a local minimum of 푝 with respect to 푔 푝 푔 푥 ≥ 푝 푥 − 휀 → EITHER-SOLUTION(BROUWER,LOCAL-OPT) is (PPAD ∩ PLS)-complete.
Load more

Recommended publications
  • Paradigms of Combinatorial Optimization
    W657-Paschos 2.qxp_Layout 1 01/07/2014 14:05 Page 1 MATHEMATICS AND STATISTICS SERIES Vangelis Th. Paschos Vangelis Edited by This updated and revised 2nd edition of the three-volume Combinatorial Optimization series covers a very large set of topics in this area, dealing with fundamental notions and approaches as well as several classical applications of Combinatorial Optimization. Combinatorial Optimization is a multidisciplinary field, lying at the interface of three major scientific domains: applied mathematics, theoretical computer science, and management studies. Its focus is on finding the least-cost solution to a mathematical problem in which each solution is associated with a numerical cost. In many such problems, exhaustive search is not feasible, so the approach taken is to operate within the domain of optimization problems, in which the set of feasible solutions is discrete or can be reduced to discrete, and in which the goal is to find the best solution. Some common problems involving combinatorial optimization are the traveling salesman problem and the Combinatorial Optimization minimum spanning tree problem. Combinatorial Optimization is a subset of optimization that is related to operations research, algorithm theory, and computational complexity theory. It 2 has important applications in several fields, including artificial intelligence, nd mathematics, and software engineering. Edition Revised and Updated This second volume, which addresses the various paradigms and approaches taken in Combinatorial Optimization, is divided into two parts: Paradigms of - Paradigmatic Problems, which discusses several famous combinatorial optimization problems, such as max cut, min coloring, optimal satisfiability TSP, Paradigms of etc., the study of which has largely contributed to the development, the legitimization and the establishment of Combinatorial Optimization as one of the most active current scientific domains.
    [Show full text]
  • Solving Non-Boolean Satisfiability Problems with Stochastic Local Search
    Solving Non-Boolean Satisfiability Problems with Stochastic Local Search: A Comparison of Encodings ALAN M. FRISCH †, TIMOTHY J. PEUGNIEZ, ANTHONY J. DOGGETT and PETER W. NIGHTINGALE§ Artificial Intelligence Group, Department of Computer Science, University of York, York YO10 5DD, UK. e-mail: [email protected], [email protected] Abstract. Much excitement has been generated by the success of stochastic local search procedures at finding solutions to large, very hard satisfiability problems. Many of the problems on which these procedures have been effective are non-Boolean in that they are most naturally formulated in terms of variables with domain sizes greater than two. Approaches to solving non-Boolean satisfiability problems fall into two categories. In the direct approach, the problem is tackled by an algorithm for non-Boolean problems. In the transformation approach, the non-Boolean problem is reformulated as an equivalent Boolean problem and then a Boolean solver is used. This paper compares four methods for solving non-Boolean problems: one di- rect and three transformational. The comparison first examines the search spaces confronted by the four methods then tests their ability to solve random formulas, the round-robin sports scheduling problem and the quasigroup completion problem. The experiments show that the relative performance of the methods depends on the domain size of the problem, and that the direct method scales better as domain size increases. Along the route to performing these comparisons we make three other contri- butions. First, we generalise Walksat, a highly-successful stochastic local search procedure for Boolean satisfiability problems, to work on problems with domains of any finite size.
    [Show full text]
  • Random Θ(Log N)-Cnfs Are Hard for Cutting Planes
    Random Θ(log n)-CNFs are Hard for Cutting Planes Noah Fleming Denis Pankratov Toniann Pitassi Robert Robere University of Toronto University of Toronto University of Toronto University of Toronto noahfl[email protected] [email protected] [email protected] [email protected] Abstract—The random k-SAT model is the most impor- lower bounds for random k-SAT formulas in a particular tant and well-studied distribution over k-SAT instances. It is proof system show that any complete and efficient algorithm closely connected to statistical physics and is a benchmark for based on the proof system will perform badly on random satisfiability algorithms. We show that when k = Θ(log n), any Cutting Planes refutation for random k-SAT requires k-SAT instances. Furthermore, since the proof complexity exponential size in the interesting regime where the number of lower bounds hold in the unsatisfiable regime, they are clauses guarantees that the formula is unsatisfiable with high directly connected to Feige’s hypothesis. probability. Remarkably, determining whether or not a random SAT Keywords-Proof complexity; random k-SAT; Cutting Planes; instance from the distribution F(m; n; k) is satisfiable is controlled quite precisely by the ratio ∆ = m=n, which is called the clause density. A simple counting argument shows I. INTRODUCTION that F(m; n; k) is unsatisfiable with high probability for The Satisfiability (SAT) problem is perhaps the most ∆ > 2k ln 2. The famous satisfiability threshold conjecture famous problem in theoretical computer science, and sig- asserts that there is a constant ck such that random k-SAT nificant effort has been devoted to understanding randomly formulas of clause density ∆ are almost certainly satisfiable generated SAT instances.
    [Show full text]
  • The Classes FNP and TFNP
    Outline The classes FNP and TFNP C. Wilson1 1Lane Department of Computer Science and Electrical Engineering West Virginia University Christopher Wilson Function Problems Outline Outline 1 Function Problems defined What are Function Problems? FSAT Defined TSP Defined 2 Relationship between Function and Decision Problems RL Defined Reductions between Function Problems 3 Total Functions Defined Total Functions Defined FACTORING HAPPYNET ANOTHER HAMILTON CYCLE Christopher Wilson Function Problems Outline Outline 1 Function Problems defined What are Function Problems? FSAT Defined TSP Defined 2 Relationship between Function and Decision Problems RL Defined Reductions between Function Problems 3 Total Functions Defined Total Functions Defined FACTORING HAPPYNET ANOTHER HAMILTON CYCLE Christopher Wilson Function Problems Outline Outline 1 Function Problems defined What are Function Problems? FSAT Defined TSP Defined 2 Relationship between Function and Decision Problems RL Defined Reductions between Function Problems 3 Total Functions Defined Total Functions Defined FACTORING HAPPYNET ANOTHER HAMILTON CYCLE Christopher Wilson Function Problems Function Problems What are Function Problems? Function Problems FSAT Defined Total Functions TSP Defined Outline 1 Function Problems defined What are Function Problems? FSAT Defined TSP Defined 2 Relationship between Function and Decision Problems RL Defined Reductions between Function Problems 3 Total Functions Defined Total Functions Defined FACTORING HAPPYNET ANOTHER HAMILTON CYCLE Christopher Wilson Function Problems Function
    [Show full text]
  • Lecture: Complexity of Finding a Nash Equilibrium 1 Computational
    Algorithmic Game Theory Lecture Date: September 20, 2011 Lecture: Complexity of Finding a Nash Equilibrium Lecturer: Christos Papadimitriou Scribe: Miklos Racz, Yan Yang In this lecture we are going to talk about the complexity of finding a Nash Equilibrium. In particular, the class of problems NP is introduced and several important subclasses are identified. Most importantly, we are going to prove that finding a Nash equilibrium is PPAD-complete (defined in Section 2). For an expository article on the topic, see [4], and for a more detailed account, see [5]. 1 Computational complexity For us the best way to think about computational complexity is that it is about search problems.Thetwo most important classes of search problems are P and NP. 1.1 The complexity class NP NP stands for non-deterministic polynomial. It is a class of problems that are at the core of complexity theory. The classical definition is in terms of yes-no problems; here, we are concerned with the search problem form of the definition. Definition 1 (Complexity class NP). The class of all search problems. A search problem A is a binary predicate A(x, y) that is efficiently (in polynomial time) computable and balanced (the length of x and y do not differ exponentially). Intuitively, x is an instance of the problem and y is a solution. The search problem for A is this: “Given x,findy such that A(x, y), or if no such y exists, say “no”.” The class of all search problems is called NP. Examples of NP problems include the following two.
    [Show full text]
  • The Relative Complexity of NP Search Problems
    Journal of Computer and System Sciences 57, 319 (1998) Article No. SS981575 The Relative Complexity of NP Search Problems Paul Beame* Computer Science and Engineering, University of Washington, Box 352350, Seattle, Washington 98195-2350 E-mail: beameÄcs.washington.edu Stephen Cook- Computer Science Department, University of Toronto, Canada M5S 3G4 E-mail: sacookÄcs.toronto.edu Jeff Edmonds Department of Computer Science, York University, Toronto, Ontario, Canada M3J 1P3 E-mail: jeffÄcs.yorku.ca Russell Impagliazzo9 Computer Science and Engineering, UC, San Diego, 9500 Gilman Drive, La Jolla, California 92093-0114 E-mail: russellÄcs.ucsd.edu and Toniann PitassiÄ Department of Computer Science, University of Arizona, Tucson, Arizona 85721-0077 E-mail: toniÄcs.arizona.edu Received January 1998 1. INTRODUCTION Papadimitriou introduced several classes of NP search problems based on combinatorial principles which guarantee the existence of In the study of computational complexity, there are many solutions to the problems. Many interesting search problems not known to be solvable in polynomial time are contained in these classes, problems that are naturally expressed as problems ``to find'' and a number of them are complete problems. We consider the question but are converted into decision problems to fit into standard of the relative complexity of these search problem classes. We prove complexity classes. For example, a more natural problem several separations which show that in a generic relativized world the than determining whether or not a graph is 3-colorable search classes are distinct and there is a standard search problem in might be that of finding a 3-coloring of the graph if it exists.
    [Show full text]
  • Computational Complexity; Slides 9, HT 2019 NP Search Problems, and Total Search Problems
    Computational Complexity; slides 9, HT 2019 NP search problems, and total search problems Prof. Paul W. Goldberg (Dept. of Computer Science, University of Oxford) HT 2019 Paul Goldberg NP search problems, and total search problems 1 / 21 Examples of total search problems in NP FACTORING NASH: the problem of computing a Nash equilibrium of a game (comes in many versions depending on the structure of the game) PIGEONHOLE CIRCUIT: Input: a boolean circuit with n input gates and n output gates Output: either input vector x mapping to 0 or vectors x, x0 mapping to the same output NECKLACE SPLITTING SECOND HAMILTONIAN CYCLE (in 3-regular graph) HAM SANDWICH: search for ham sandwich cut Search for local optima in settings with neighbourhood structure The above seem to be hard. (of course, many search probs are in P, e.g. input a list L of numbers, output L in increasing order) Paul Goldberg NP search problems, and total search problems 2 / 21 Search problems as poly-time checkable relations NP search problem is modelled as a relation R(·; ·) where R(x; y) is checkable in time polynomial in jxj, jyj input x, find y with R(x; y)( y as certificate) total search problem: 8x9y (jyj = poly(jxj); R(x; y)) SAT: x is boolean formula, y is satisfying bit vector. Decision version of SAT is polynomial-time equivalent to search for y. FACTORING: input (the \x" in R(x; y)) is number N, output (the \y") is prime factorisation of N. No decision problem! NECKLACE SPLITTING (k thieves): input is string of n beads in c colours; output is a decomposition into c(k − 1) + 1 substrings and allocation of substrings to thieves such that they all get the same number of beads of each colour.
    [Show full text]
  • On Total Functions, Existence Theorems and Computational Complexity
    Theoretical Computer Science 81 (1991) 317-324 Elsevier Note On total functions, existence theorems and computational complexity Nimrod Megiddo IBM Almaden Research Center, 650 Harry Road, Sun Jose, CA 95120-6099, USA, and School of Mathematical Sciences, Tel Aviv University, Tel Aviv, Israel Christos H. Papadimitriou* Computer Technology Institute, Patras, Greece, and University of California at Sun Diego, CA, USA Communicated by M. Nivat Received October 1989 Abstract Megiddo, N. and C.H. Papadimitriou, On total functions, existence theorems and computational complexity (Note), Theoretical Computer Science 81 (1991) 317-324. wondeterministic multivalued functions with values that are polynomially verifiable and guaran- teed to exist form an interesting complexity class between P and NP. We show that this class, which we call TFNP, contains a host of important problems, whose membership in P is currently not known. These include, besides factoring, local optimization, Brouwer's fixed points, a computa- tional version of Sperner's Lemma, bimatrix equilibria in games, and linear complementarity for P-matrices. 1. The class TFNP Let 2 be an alphabet with two or more symbols, and suppose that R G E*x 2" is a polynomial-time recognizable relation which is polynomially balanced, that is, (x,y) E R implies that lyl sp(lx()for some polynomial p. * Research supported by an ESPRIT Basic Research Project, a grant to the Universities of Patras and Bonn by the Volkswagen Foundation, and an NSF Grant. Research partially performed while the author was visiting the IBM Almaden Research Center. 0304-3975/91/$03.50 @ 1991-Elsevier Science Publishers B.V.
    [Show full text]
  • Arxiv:1910.02319V2 [Cs.CV] 10 Nov 2020
    Covariance-free Partial Least Squares: An Incremental Dimensionality Reduction Method Artur Jordao, Maiko Lie, Victor Hugo Cunha de Melo and William Robson Schwartz Smart Sense Laboratory, Computer Science Department Federal University of Minas Gerais, Brazil Email: {arturjordao, maikolie, victorhcmelo, william}@dcc.ufmg.br Abstract latent space [23][8]. Previous works have demonstrated that dimensionality reduction can improve not only com- Dimensionality reduction plays an important role in putational cost but also the effectiveness of the data rep- computer vision problems since it reduces computational resentation [19] [35] [33]. In this context, Partial Least cost and is often capable of yielding more discriminative Squares (PLS) has presented remarkable results when com- data representation. In this context, Partial Least Squares pared to other dimensionality reduction methods [33]. This (PLS) has presented notable results in tasks such as image is mainly due to the criterion through which PLS finds the classification and neural network optimization. However, low dimensional space, which is by capturing the relation- PLS is infeasible on large datasets, such as ImageNet, be- ship between independent and dependent variables. An- cause it requires all the data to be in memory in advance, other interesting aspect of PLS is that it can operate as a fea- which is often impractical due to hardware limitations. Ad- ture selection method, for instance, by employing Variable ditionally, this requirement prevents us from employing PLS Importance in Projection (VIP) [24]. The VIP technique on streaming applications where the data are being contin- employs score matrices yielded by NIPALS (the standard uously generated. Motivated by this, we propose a novel algorithm used for traditional PLS) to compute the impor- incremental PLS, named Covariance-free Incremental Par- tance of each feature based on its contribution to the gener- tial Least Squares (CIPLS), which learns a low-dimensional ation of the latent space.
    [Show full text]
  • A Tour of the Complexity Classes Between P and NP
    A Tour of the Complexity Classes Between P and NP John Fearnley University of Liverpool Joint work with Spencer Gordon, Ruta Mehta, Rahul Savani Simple stochastic games T A two player game I Maximizer (box) wants to reach T I Minimizer (triangle) who wants to avoid T I Nature (circle) plays uniformly at random Simple stochastic games 0.5 1 1 0.5 0.5 0 Value of a vertex: I The largest probability of winning that max can ensure I The smallest probability of winning that min can ensure Computational Problem: find the value of each vertex Simple stochastic games 0.5 1 1 0.5 0.5 0 Is the problem I Easy? Does it have a polynomial time algorithm? I Hard? Perhaps no such algorithm exists This is currently unresolved Simple stochastic games 0.5 1 1 0.5 0.5 0 The problem lies in NP \ co-NP I So it is unlikely to be NP-hard But there are a lot of NP-intermediate classes... This talk: whare are these complexity classes? Simple stochastic games Solving a simple-stochastic game lies in NP \ co-NP \ UP \ co-UP \ TFNP \ PPP \ PPA \ PPAD \ PLS \ CLS \ EOPL \ UEOPL Simple stochastic games Solving a simple-stochastic game lies in NP \ co-NP \ UP \ co-UP \ TFNP \ PPP \ PPA \ PPAD \ PLS \ CLS \ EOPL \ UEOPL This talk: whare are these complexity classes? TFNP PPAD PLS CLS Complexity classes between P and NP NP P There are many problems that lie between P and NP I Factoring, graph isomorphism, computing Nash equilibria, local max cut, simple-stochastic games, ..
    [Show full text]
  • Total Functions in QMA
    Total Functions in QMA Serge Massar1 and Miklos Santha2,3 1Laboratoire d’Information Quantique CP224, Université libre de Bruxelles, B-1050 Brussels, Belgium. 2CNRS, IRIF, Université Paris Diderot, 75205 Paris, France. 3Centre for Quantum Technologies & MajuLab, National University of Singapore, Singapore. December 15, 2020 The complexity class QMA is the quan- either output a circuit with length less than `, or tum analog of the classical complexity class output NO if such a circuit does not exist. The NP. The functional analogs of NP and functional analog of P, denoted FP, is the subset QMA, called functional NP (FNP) and of FNP for which the output can be computed in functional QMA (FQMA), consist in ei- polynomial time. ther outputting a (classical or quantum) Total functional NP (TFNP), introduced in witness, or outputting NO if there does [37] and which lies between FP and FNP, is the not exist a witness. The classical com- subset of FNP for which it can be shown that the plexity class Total Functional NP (TFNP) NO outcome never occurs. As an example, factor- is the subset of FNP for which it can be ing (given an integer n, output the prime factors shown that the NO outcome never occurs. of n) lies in TFNP since for all n a (unique) set TFNP includes many natural and impor- of prime factors exists, and it can be verified in tant problems. Here we introduce the polynomial time that the factorisation is correct. complexity class Total Functional QMA TFNP can also be defined as the functional ana- (TFQMA), the quantum analog of TFNP.
    [Show full text]
  • 4.1 Introduction 4.2 NP And
    601.436/636 Algorithmic Game Theory Lecturer: Michael Dinitz Topic: Hardness of Computing Nash Equilibria Date: 2/6/20 Scribe: Michael Dinitz 4.1 Introduction Last class we saw an exponential time algorithm (Lemke-Howson) for computing a Nash equilibrium of a bimatrix game. Today we're going to argue that we can't really expect a polynomial-time algorithm for computing Nash, even for the special case of two-player games (even though for two- player zero-sum we saw that there is such an algorithm). Formally proving this was an enormous accomplishment in complexity theory, which we're not going to do (but might be a cool final project!). Instead, I'm going to try to give a high-level point of view of what the result actually says, and not dive too deeply into how to prove it. Today's lecture is pretty complexity-theoretic, so if you haven't seen much complexity theory before, feel free to jump in with questions. The main problem we're going to be analyzing is what I'll call Two-Player Nash: given a bimatrix game, compute a Nash equilibrium. Clearly this is a special case of the more general problem of Nash: given a simultaneous one-shot game, compute a Nash equilibrium. So if we can prove that Two-Player Nash is hard, then so is Nash. 4.2 NP and FNP As we all know, the most common way of proving that a problem is \unlikely" to have a polynomial- time algorithm is to prove that it is NP-hard.
    [Show full text]