Testing Closeness of Discrete Distributions
Total Page:16
File Type:pdf, Size:1020Kb

Load more
Recommended publications
-
Resource-Competitive Algorithms1 1 Introduction
Resource-Competitive Algorithms1 Michael A. Bender Varsha Dani Department of Computer Science Department of Computer Science Stony Brook University University of New Mexico Stony Brook, NY, USA Albuquerque, NM, USA [email protected] [email protected] Jeremy T. Fineman Seth Gilbert Department of Computer Science Department of Computer Science Georgetown University National University of Singapore Washington, DC, USA Singapore [email protected] [email protected] Mahnush Movahedi Seth Pettie Department of Computer Science Electrical Eng. and Computer Science Dept. University of New Mexico University of Michigan Albuquerque, NM, USA Ann Arbor, MI, USA [email protected] [email protected] Jared Saia Maxwell Young Department of Computer Science Computer Science and Engineering Dept. University of New Mexico Mississippi State University Albuquerque, NM, USA Starkville, MS, USA [email protected] [email protected] Abstract The point of adversarial analysis is to model the worst-case performance of an algorithm. Un- fortunately, this analysis may not always reflect performance in practice because the adversarial assumption can be overly pessimistic. In such cases, several techniques have been developed to provide a more refined understanding of how an algorithm performs e.g., competitive analysis, parameterized analysis, and the theory of approximation algorithms. Here, we describe an analogous technique called resource competitiveness, tailored for dis- tributed systems. Often there is an operational cost for adversarial behavior arising from band- width usage, computational power, energy limitations, etc. Modeling this cost provides some notion of how much disruption the adversary can inflict on the system. In parameterizing by this cost, we can design an algorithm with the following guarantee: if the adversary pays T , then the additional cost of the algorithm is some function of T . -
FOCS 2005 Program SUNDAY October 23, 2005
FOCS 2005 Program SUNDAY October 23, 2005 Talks in Grand Ballroom, 17th floor Session 1: 8:50am – 10:10am Chair: Eva´ Tardos 8:50 Agnostically Learning Halfspaces Adam Kalai, Adam Klivans, Yishay Mansour and Rocco Servedio 9:10 Noise stability of functions with low influences: invari- ance and optimality The 46th Annual IEEE Symposium on Elchanan Mossel, Ryan O’Donnell and Krzysztof Foundations of Computer Science Oleszkiewicz October 22-25, 2005 Omni William Penn Hotel, 9:30 Every decision tree has an influential variable Pittsburgh, PA Ryan O’Donnell, Michael Saks, Oded Schramm and Rocco Servedio Sponsored by the IEEE Computer Society Technical Committee on Mathematical Foundations of Computing 9:50 Lower Bounds for the Noisy Broadcast Problem In cooperation with ACM SIGACT Navin Goyal, Guy Kindler and Michael Saks Break 10:10am – 10:30am FOCS ’05 gratefully acknowledges financial support from Microsoft Research, Yahoo! Research, and the CMU Aladdin center Session 2: 10:30am – 12:10pm Chair: Satish Rao SATURDAY October 22, 2005 10:30 The Unique Games Conjecture, Integrality Gap for Cut Problems and Embeddability of Negative Type Metrics Tutorials held at CMU University Center into `1 [Best paper award] Reception at Omni William Penn Hotel, Monongahela Room, Subhash Khot and Nisheeth Vishnoi 17th floor 10:50 The Closest Substring problem with small distances Tutorial 1: 1:30pm – 3:30pm Daniel Marx (McConomy Auditorium) Chair: Irit Dinur 11:10 Fitting tree metrics: Hierarchical clustering and Phy- logeny Subhash Khot Nir Ailon and Moses Charikar On the Unique Games Conjecture 11:30 Metric Embeddings with Relaxed Guarantees Break 3:30pm – 4:00pm Ittai Abraham, Yair Bartal, T-H. -
A Polynomial-Time Approximation Algorithm for All-Terminal Network Reliability
A Polynomial-Time Approximation Algorithm for All-Terminal Network Reliability Heng Guo School of Informatics, University of Edinburgh, Informatics Forum, Edinburgh, EH8 9AB, United Kingdom. [email protected] https://orcid.org/0000-0001-8199-5596 Mark Jerrum1 School of Mathematical Sciences, Queen Mary, University of London, Mile End Road, London, E1 4NS, United Kingdom. [email protected] https://orcid.org/0000-0003-0863-7279 Abstract We give a fully polynomial-time randomized approximation scheme (FPRAS) for the all-terminal network reliability problem, which is to determine the probability that, in a undirected graph, assuming each edge fails independently, the remaining graph is still connected. Our main contri- bution is to confirm a conjecture by Gorodezky and Pak (Random Struct. Algorithms, 2014), that the expected running time of the “cluster-popping” algorithm in bi-directed graphs is bounded by a polynomial in the size of the input. 2012 ACM Subject Classification Theory of computation → Generating random combinatorial structures Keywords and phrases Approximate counting, Network Reliability, Sampling, Markov chains Digital Object Identifier 10.4230/LIPIcs.ICALP.2018.68 Related Version Also available at https://arxiv.org/abs/1709.08561. Acknowledgements We thank Mark Huber for bringing reference [8] to our attention, Mark Walters for the coupling idea leading to Lemma 12, and Igor Pak for comments on an earlier version. We also thank the organizers of the “LMS – EPSRC Durham Symposium on Markov Processes, Mixing Times and Cutoff”, where part of the work is carried out. 1 Introduction Network reliability problems are extensively studied #P-hard problems [5] (see also [3, 22, 18, 2]). -
Arxiv:1611.01647V4 [Cs.DS] 15 Jan 2019 Resample to Able of Are Class We Special Techniques, Our a with for Inst Extremal Solutions Answer
UNIFORM SAMPLING THROUGH THE LOVASZ´ LOCAL LEMMA HENG GUO, MARK JERRUM, AND JINGCHENG LIU Abstract. We propose a new algorithmic framework, called “partial rejection sampling”, to draw samples exactly from a product distribution, conditioned on none of a number of bad events occurring. Our framework builds new connections between the variable framework of the Lov´asz Local Lemma and some classical sampling algorithms such as the “cycle-popping” algorithm for rooted spanning trees. Among other applications, we discover new algorithms to sample satisfying assignments of k-CNF formulas with bounded variable occurrences. 1. Introduction The Lov´asz Local Lemma [9] is a classical gem in combinatorics that guarantees the existence of a perfect object that avoids all events deemed to be “bad”. The original proof is non- constructive but there has been great progress in the algorithmic aspects of the local lemma. After a long line of research [3, 2, 30, 8, 34, 37], the celebrated result by Moser and Tardos [31] gives efficient algorithms to find such a perfect object under conditions that match the Lov´asz Local Lemma in the so-called variable framework. However, it is natural to ask whether, under the same condition, we can also sample a perfect object uniformly at random instead of merely finding one. Roughly speaking, the resampling algorithm by Moser and Tardos [31] works as follows. We initialize all variables randomly. If bad events occur, then we arbitrarily choose a bad event and resample all the involved variables. Unfortunately, it is not hard to see that this algorithm can produce biased samples. -
Are All Distributions Easy?
Are all distributions easy? Emanuele Viola∗ November 5, 2009 Abstract Complexity theory typically studies the complexity of computing a function h(x): f0; 1gn ! f0; 1gm of a given input x. We advocate the study of the complexity of generating the distribution h(x) for uniform x, given random bits. Our main results are: • There are explicit AC0 circuits of size poly(n) and depth O(1) whose output n P distribution has statistical distance 1=2 from the distribution (X; i Xi) 2 f0; 1gn × f0; 1; : : : ; ng for uniform X 2 f0; 1gn, despite the inability of these P circuits to compute i xi given x. Previous examples of this phenomenon apply to different distributions such as P n+1 (X; i Xi mod 2) 2 f0; 1g . We also prove a lower bound independent from n on the statistical distance be- tween the output distribution of NC0 circuits and the distribution (X; majority(X)). We show that 1 − o(1) lower bounds for related distributions yield lower bounds for succinct data structures. • Uniform randomized AC0 circuits of poly(n) size and depth d = O(1) with error can be simulated by uniform randomized circuits of poly(n) size and depth d + 1 with error + o(1) using ≤ (log n)O(log log n) random bits. Previous derandomizations [Ajtai and Wigderson '85; Nisan '91] increase the depth by a constant factor, or else have poor seed length. Given the right tools, the above results have technically simple proofs. ∗Supported by NSF grant CCF-0845003. Email: [email protected] 1 Introduction Complexity theory, with some notable exceptions, typically studies the complexity of com- puting a function h(x): f0; 1gn ! f0; 1gm of a given input x. -
On the Complexity of Partial Derivatives Ignacio Garcia-Marco, Pascal Koiran, Timothée Pecatte, Stéphan Thomassé
On the complexity of partial derivatives Ignacio Garcia-Marco, Pascal Koiran, Timothée Pecatte, Stéphan Thomassé To cite this version: Ignacio Garcia-Marco, Pascal Koiran, Timothée Pecatte, Stéphan Thomassé. On the complexity of partial derivatives. 2016. ensl-01345746v2 HAL Id: ensl-01345746 https://hal-ens-lyon.archives-ouvertes.fr/ensl-01345746v2 Preprint submitted on 30 May 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. On the complexity of partial derivatives Ignacio Garcia-Marco, Pascal Koiran, Timothée Pecatte, Stéphan Thomassé LIP,∗ Ecole Normale Supérieure de Lyon, Université de Lyon. May 30, 2017 Abstract The method of partial derivatives is one of the most successful lower bound methods for arithmetic circuits. It uses as a complexity measure the dimension of the span of the partial derivatives of a polynomial. In this paper, we consider this complexity measure as a computational problem: for an input polynomial given as the sum of its nonzero monomials, what is the complexity of computing the dimension of its space of partial derivatives? We show that this problem is ♯P-hard and we ask whether it belongs to ♯P. We analyze the “trace method”, recently used in combinatorics and in algebraic complexity to lower bound the rank of certain matri- ces. -
Typical Stability
Typical Stability Raef Bassily∗ Yoav Freundy Abstract In this paper, we introduce a notion of algorithmic stability called typical stability. When our goal is to release real-valued queries (statistics) computed over a dataset, this notion does not require the queries to be of bounded sensitivity – a condition that is generally assumed under differential privacy [DMNS06, Dwo06] when used as a notion of algorithmic stability [DFH+15b, DFH+15c, BNS+16] – nor does it require the samples in the dataset to be independent – a condition that is usually assumed when generalization-error guarantees are sought. Instead, typical stability requires the output of the query, when computed on a dataset drawn from the underlying distribution, to be concentrated around its expected value with respect to that distribution. Typical stability can also be motivated as an alternative definition for database privacy. Like differential privacy, this notion enjoys several important properties including robustness to post-processing and adaptive composition. However, privacy is guaranteed only for a given family of distributions over the dataset. We also discuss the implications of typical stability on the generalization error (i.e., the difference between the value of the query computed on the dataset and the expected value of the query with respect to the true data distribution). We show that typical stability can control generalization error in adaptive data analysis even when the samples in the dataset are not necessarily independent and when queries to be computed are not necessarily of bounded- sensitivity as long as the results of the queries over the dataset (i.e., the computed statistics) follow a distribution with a “light” tail. -
Interactions of Computational Complexity Theory and Mathematics
Interactions of Computational Complexity Theory and Mathematics Avi Wigderson October 22, 2017 Abstract [This paper is a (self contained) chapter in a new book on computational complexity theory, called Mathematics and Computation, whose draft is available at https://www.math.ias.edu/avi/book]. We survey some concrete interaction areas between computational complexity theory and different fields of mathematics. We hope to demonstrate here that hardly any area of modern mathematics is untouched by the computational connection (which in some cases is completely natural and in others may seem quite surprising). In my view, the breadth, depth, beauty and novelty of these connections is inspiring, and speaks to a great potential of future interactions (which indeed, are quickly expanding). We aim for variety. We give short, simple descriptions (without proofs or much technical detail) of ideas, motivations, results and connections; this will hopefully entice the reader to dig deeper. Each vignette focuses only on a single topic within a large mathematical filed. We cover the following: • Number Theory: Primality testing • Combinatorial Geometry: Point-line incidences • Operator Theory: The Kadison-Singer problem • Metric Geometry: Distortion of embeddings • Group Theory: Generation and random generation • Statistical Physics: Monte-Carlo Markov chains • Analysis and Probability: Noise stability • Lattice Theory: Short vectors • Invariant Theory: Actions on matrix tuples 1 1 introduction The Theory of Computation (ToC) lays out the mathematical foundations of computer science. I am often asked if ToC is a branch of Mathematics, or of Computer Science. The answer is easy: it is clearly both (and in fact, much more). Ever since Turing's 1936 definition of the Turing machine, we have had a formal mathematical model of computation that enables the rigorous mathematical study of computational tasks, algorithms to solve them, and the resources these require. -
An Efficient Boosting Algorithm for Combining Preferences Raj
An Efficient Boosting Algorithm for Combining Preferences by Raj Dharmarajan Iyer Jr. Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Master of Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 1999 © Massachusetts Institute of Technology 1999. All rights reserved. A uth or .............. ..... ..... .................................. Department of E'ectrical ngineering and Computer Science August 24, 1999 C ertified by .. ................. ...... .. .............. David R. Karger Associate Professor Thesis Supervisor Accepted by............... ........ Arthur C. Smith Chairman, Departmental Committe Graduate Students MACHU OF TEC Lo NOV LIBRARIES An Efficient Boosting Algorithm for Combining Preferences by Raj Dharmarajan Iyer Jr. Submitted to the Department of Electrical Engineering and Computer Science on August 24, 1999, in partial fulfillment of the requirements for the degree of Master of Science Abstract The problem of combining preferences arises in several applications, such as combining the results of different search engines. This work describes an efficient algorithm for combining multiple preferences. We first give a formal framework for the problem. We then describe and analyze a new boosting algorithm for combining preferences called RankBoost. We also describe an efficient implementation of the algorithm for certain natural cases. We discuss two experiments we carried out to assess the performance of RankBoost. In the first experi- ment, we used the algorithm to combine different WWW search strategies, each of which is a query expansion for a given domain. For this task, we compare the performance of Rank- Boost to the individual search strategies. The second experiment is a collaborative-filtering task for making movie recommendations. -
An Introduction to Johnson–Lindenstrauss Transforms
An Introduction to Johnson–Lindenstrauss Transforms Casper Benjamin Freksen∗ 2nd March 2021 Abstract Johnson–Lindenstrauss Transforms are powerful tools for reducing the dimensionality of data while preserving key characteristics of that data, and they have found use in many fields from machine learning to differential privacy and more. This note explains whatthey are; it gives an overview of their use and their development since they were introduced in the 1980s; and it provides many references should the reader wish to explore these topics more deeply. The text was previously a main part of the introduction of my PhD thesis [Fre20], but it has been adapted to be self contained and serve as a (hopefully good) starting point for readers interested in the topic. 1 The Why, What, and How 1.1 The Problem Consider the following scenario: We have some data that we wish to process but the data is too large, e.g. processing the data takes too much time, or storing the data takes too much space. A solution would be to compress the data such that the valuable parts of the data are kept and the other parts discarded. Of course, what is considered valuable is defined by the data processing we wish to apply to our data. To make our scenario more concrete let us say that our data consists of vectors in a high dimensional Euclidean space, R3, and we wish to find a transform to embed these vectors into a lower dimensional space, R<, where < 3, so that we arXiv:2103.00564v1 [cs.DS] 28 Feb 2021 ≪ can apply our data processing in this lower dimensional space and still get meaningful results. -
The Complexity of Nash Equilibria by Constantinos Daskalakis Diploma
The Complexity of Nash Equilibria by Constantinos Daskalakis Diploma (National Technical University of Athens) 2004 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Christos H. Papadimitriou, Chair Professor Alistair J. Sinclair Professor Satish Rao Professor Ilan Adler Fall 2008 The dissertation of Constantinos Daskalakis is approved: Chair Date Date Date Date University of California, Berkeley Fall 2008 The Complexity of Nash Equilibria Copyright 2008 by Constantinos Daskalakis Abstract The Complexity of Nash Equilibria by Constantinos Daskalakis Doctor of Philosophy in Computer Science University of California, Berkeley Professor Christos H. Papadimitriou, Chair The Internet owes much of its complexity to the large number of entities that run it and use it. These entities have different and potentially conflicting interests, so their interactions are strategic in nature. Therefore, to understand these interactions, concepts from Economics and, most importantly, Game Theory are necessary. An important such concept is the notion of Nash equilibrium, which provides us with a rigorous way of predicting the behavior of strategic agents in situations of conflict. But the credibility of the Nash equilibrium as a framework for behavior-prediction depends on whether such equilibria are efficiently computable. After all, why should we expect a group of rational agents to behave in a fashion that requires exponential time to be computed? Motivated by this question, we study the computational complexity of the Nash equilibrium. We show that computing a Nash equilibrium is an intractable problem. -
Adversarial Robustness of Streaming Algorithms Through Importance Sampling
Adversarial Robustness of Streaming Algorithms through Importance Sampling Vladimir Braverman Avinatan Hassidim Yossi Matias Mariano Schain Google∗ Googley Googlez Googlex Sandeep Silwal Samson Zhou MITO CMU{ June 30, 2021 Abstract Robustness against adversarial attacks has recently been at the forefront of algorithmic design for machine learning tasks. In the adversarial streaming model, an adversary gives an algorithm a sequence of adaptively chosen updates u1; : : : ; un as a data stream. The goal of the algorithm is to compute or approximate some predetermined function for every prefix of the adversarial stream, but the adversary may generate future updates based on previous outputs of the algorithm. In particular, the adversary may gradually learn the random bits internally used by an algorithm to manipulate dependencies in the input. This is especially problematic as many important problems in the streaming model require randomized algorithms, as they are known to not admit any deterministic algorithms that use sublinear space. In this paper, we introduce adversarially robust streaming algorithms for central machine learning and algorithmic tasks, such as regression and clustering, as well as their more general counterparts, subspace embedding, low-rank approximation, and coreset construction. For regression and other numerical linear algebra related tasks, we consider the row arrival streaming model. Our results are based on a simple, but powerful, observation that many importance sampling-based algorithms give rise to adversarial robustness which is in contrast to sketching based algorithms, which are very prevalent in the streaming literature but suffer from adversarial attacks. In addition, we show that the well-known merge and reduce paradigm in streaming is adversarially robust.