Simultaneous Private Learning of Multiple Concepts∗
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
FOCS 2005 Program SUNDAY October 23, 2005
FOCS 2005 Program SUNDAY October 23, 2005 Talks in Grand Ballroom, 17th floor Session 1: 8:50am – 10:10am Chair: Eva´ Tardos 8:50 Agnostically Learning Halfspaces Adam Kalai, Adam Klivans, Yishay Mansour and Rocco Servedio 9:10 Noise stability of functions with low influences: invari- ance and optimality The 46th Annual IEEE Symposium on Elchanan Mossel, Ryan O’Donnell and Krzysztof Foundations of Computer Science Oleszkiewicz October 22-25, 2005 Omni William Penn Hotel, 9:30 Every decision tree has an influential variable Pittsburgh, PA Ryan O’Donnell, Michael Saks, Oded Schramm and Rocco Servedio Sponsored by the IEEE Computer Society Technical Committee on Mathematical Foundations of Computing 9:50 Lower Bounds for the Noisy Broadcast Problem In cooperation with ACM SIGACT Navin Goyal, Guy Kindler and Michael Saks Break 10:10am – 10:30am FOCS ’05 gratefully acknowledges financial support from Microsoft Research, Yahoo! Research, and the CMU Aladdin center Session 2: 10:30am – 12:10pm Chair: Satish Rao SATURDAY October 22, 2005 10:30 The Unique Games Conjecture, Integrality Gap for Cut Problems and Embeddability of Negative Type Metrics Tutorials held at CMU University Center into `1 [Best paper award] Reception at Omni William Penn Hotel, Monongahela Room, Subhash Khot and Nisheeth Vishnoi 17th floor 10:50 The Closest Substring problem with small distances Tutorial 1: 1:30pm – 3:30pm Daniel Marx (McConomy Auditorium) Chair: Irit Dinur 11:10 Fitting tree metrics: Hierarchical clustering and Phy- logeny Subhash Khot Nir Ailon and Moses Charikar On the Unique Games Conjecture 11:30 Metric Embeddings with Relaxed Guarantees Break 3:30pm – 4:00pm Ittai Abraham, Yair Bartal, T-H. -
The Limits of Post-Selection Generalization
The Limits of Post-Selection Generalization Kobbi Nissim∗ Adam Smithy Thomas Steinke Georgetown University Boston University IBM Research – Almaden [email protected] [email protected] [email protected] Uri Stemmerz Jonathan Ullmanx Ben-Gurion University Northeastern University [email protected] [email protected] Abstract While statistics and machine learning offers numerous methods for ensuring gener- alization, these methods often fail in the presence of post selection—the common practice in which the choice of analysis depends on previous interactions with the same dataset. A recent line of work has introduced powerful, general purpose algorithms that ensure a property called post hoc generalization (Cummings et al., COLT’16), which says that no person when given the output of the algorithm should be able to find any statistic for which the data differs significantly from the population it came from. In this work we show several limitations on the power of algorithms satisfying post hoc generalization. First, we show a tight lower bound on the error of any algorithm that satisfies post hoc generalization and answers adaptively chosen statistical queries, showing a strong barrier to progress in post selection data analysis. Second, we show that post hoc generalization is not closed under composition, despite many examples of such algorithms exhibiting strong composition properties. 1 Introduction Consider a dataset X consisting of n independent samples from some unknown population P. How can we ensure that the conclusions drawn from X generalize to the population P? Despite decades of research in statistics and machine learning on methods for ensuring generalization, there is an increased recognition that many scientific findings do not generalize, with some even declaring this to be a “statistical crisis in science” [14]. -
Individuals and Privacy in the Eye of Data Analysis
Individuals and Privacy in the Eye of Data Analysis Thesis submitted in partial fulfillment of the requirements for the degree of “DOCTOR OF PHILOSOPHY” by Uri Stemmer Submitted to the Senate of Ben-Gurion University of the Negev October 2016 Beer-Sheva This work was carried out under the supervision of Prof. Amos Beimel and Prof. Kobbi Nissim In the Department of Computer Science Faculty of Natural Sciences Acknowledgments I could not have asked for better advisors. I will be forever grateful for their close guidance, their constant encouragement, and the warm shelter they provided. Without them, this thesis could have never begun. I have been fortunate to work with Raef Bassily, Amos Beimel, Mark Bun, Kobbi Nissim, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. I enjoyed working with them all, and I thank them for everything they have taught me. iii Contents Acknowledgments . iii Contents . iv List of Figures . vi Abstract . vii 1 Introduction1 1.1 Differential Privacy . .2 1.2 The Sample Complexity of Private Learning . .3 1.3 Our Contributions . .5 1.4 Additional Contributions . 10 2 Related Literature 15 2.1 The Computational Price of Differential Privacy . 15 2.2 Interactive Query Release . 18 2.3 Answering Adaptively Chosen Statistical Queries . 19 2.4 Other Related Work . 21 3 Background and Preliminaries 22 3.1 Differential privacy . 22 3.2 Preliminaries from Learning Theory . 24 3.3 Generalization Bounds for Points and Thresholds . 29 3.4 Private Learning . 30 3.5 Basic Differentially Private Mechanisms . 31 3.6 Concentration Bounds . 33 4 The Generalization Properties of Differential Privacy 34 4.1 Main Results . -
Privacy Loss in Apple's Implementation of Differential
Privacy Loss in Apple’s Implementation of Differential Privacy on MacOS 10.12 Jun Tang Aleksandra Korolova Xiaolong Bai University of Southern California University of Southern California Tsinghua University [email protected] [email protected] [email protected] Xueqiang Wang Xiaofeng Wang Indiana University Indiana University [email protected] [email protected] ABSTRACT 1 INTRODUCTION In June 2016, Apple made a bold announcement that it will deploy Differential privacy [7] has been widely recognized as the lead- local differential privacy for some of their user data collection in ing statistical data privacy definition by the academic commu- order to ensure privacy of user data, even from Apple [21, 23]. nity [6, 11]. Thus, as one of the first large-scale commercial de- The details of Apple’s approach remained sparse. Although several ployments of differential privacy (preceded only by Google’s RAP- patents [17–19] have since appeared hinting at the algorithms that POR [10]), Apple’s deployment is of significant interest to privacy may be used to achieve differential privacy, they did not include theoreticians and practitioners alike. Furthermore, since Apple may a precise explanation of the approach taken to privacy parameter be perceived as competing on privacy with other consumer com- choice. Such choice and the overall approach to privacy budget use panies, understanding the actual privacy protections afforded by and management are key questions for understanding the privacy the deployment of differential privacy in its desktop and mobile protections provided by any deployment of differential privacy. OSes may be of interest to consumers and consumer advocate In this work, through a combination of experiments, static and groups [16]. -
Are All Distributions Easy?
Are all distributions easy? Emanuele Viola∗ November 5, 2009 Abstract Complexity theory typically studies the complexity of computing a function h(x): f0; 1gn ! f0; 1gm of a given input x. We advocate the study of the complexity of generating the distribution h(x) for uniform x, given random bits. Our main results are: • There are explicit AC0 circuits of size poly(n) and depth O(1) whose output n P distribution has statistical distance 1=2 from the distribution (X; i Xi) 2 f0; 1gn × f0; 1; : : : ; ng for uniform X 2 f0; 1gn, despite the inability of these P circuits to compute i xi given x. Previous examples of this phenomenon apply to different distributions such as P n+1 (X; i Xi mod 2) 2 f0; 1g . We also prove a lower bound independent from n on the statistical distance be- tween the output distribution of NC0 circuits and the distribution (X; majority(X)). We show that 1 − o(1) lower bounds for related distributions yield lower bounds for succinct data structures. • Uniform randomized AC0 circuits of poly(n) size and depth d = O(1) with error can be simulated by uniform randomized circuits of poly(n) size and depth d + 1 with error + o(1) using ≤ (log n)O(log log n) random bits. Previous derandomizations [Ajtai and Wigderson '85; Nisan '91] increase the depth by a constant factor, or else have poor seed length. Given the right tools, the above results have technically simple proofs. ∗Supported by NSF grant CCF-0845003. Email: [email protected] 1 Introduction Complexity theory, with some notable exceptions, typically studies the complexity of com- puting a function h(x): f0; 1gn ! f0; 1gm of a given input x. -
Calibrating Noise to Sensitivity in Private Data Analysis
Calibrating Noise to Sensitivity in Private Data Analysis Cynthia Dwork1, Frank McSherry1, Kobbi Nissim2, and Adam Smith3? 1 Microsoft Research, Silicon Valley. {dwork,mcsherry}@microsoft.com 2 Ben-Gurion University. [email protected] 3 Weizmann Institute of Science. [email protected] Abstract. We continue a line of research initiated in [10, 11] on privacy- preserving statistical databases. Consider a trusted server that holds a database of sensitive information. Given a query function f mapping databases to reals, the so-called true answer is the result of applying f to the database. To protect privacy, the true answer is perturbed by the addition of random noise generated according to a carefully chosen distribution, and this response, the true answer plus noise, is returned to the user. Previous work focused on the case of noisy sums, in which f = P i g(xi), where xi denotes the ith row of the database and g maps database rows to [0, 1]. We extend the study to general functions f, proving that privacy can be preserved by calibrating the standard devi- ation of the noise according to the sensitivity of the function f. Roughly speaking, this is the amount that any single argument to f can change its output. The new analysis shows that for several particular applications substantially less noise is needed than was previously understood to be the case. The first step is a very clean characterization of privacy in terms of indistinguishability of transcripts. Additionally, we obtain separation re- sults showing the increased value of interactive sanitization mechanisms over non-interactive. -
Differential Privacy, Property Testing, and Perturbations
Differential Privacy, Property Testing, and Perturbations by Audra McMillan A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Mathematics) in The University of Michigan 2018 Doctoral Committee: Professor Anna Gilbert, Chair Professor Selim Esedoglu Professor John Schotland Associate Professor Ambuj Tewari Audra McMillan [email protected] ORCID iD: 0000-0003-4231-6110 c Audra McMillan 2018 ACKNOWLEDGEMENTS First and foremost, I would like to thank my advisor, Anna Gilbert. Anna managed to strike the balance between support and freedom that is the goal of many advisors. I always felt that she believed in my ideas and I am a better, more confident researcher because of her. I was fortunate to have a large number of pseudo-advisors during my graduate ca- reer. Martin Strauss, Adam Smith, and Jacob Abernethy deserve special mention. Martin guided me through the early stages of my graduate career and helped me make the tran- sition into more applied research. Martin was the first to suggest that I work on privacy. He taught me that interesting problems and socially-conscious research are not mutually exclusive. Adam Smith hosted me during what became my favourite summer of graduate school. I published my first paper with Adam, which he tirelessly guided towards a pub- lishable version. He has continued to be my guide in the differential privacy community, introducing me to the right people and the right problems. Jacob Abernethy taught me much of what I know about machine learning. He also taught me the importance of being flexible in my research and that research is as much as about exploration as it is about solving a pre-specified problem. -
Amicus Brief of Data Privacy Experts
Case 3:21-cv-00211-RAH-ECM-KCN Document 99-1 Filed 04/23/21 Page 1 of 27 EXHIBIT A Case 3:21-cv-00211-RAH-ECM-KCN Document 99-1 Filed 04/23/21 Page 2 of 27 UNITED STATES DISTRICT COURT FOR THE MIDDLE DISTRICT OF ALABAMA EASTERN DIVISION THE STATE OF ALABAMA, et al., ) ) Plaintiffs, ) ) v. ) Civil Action No. ) 3:21-CV-211-RAH UNITED STATES DEPARTMENT OF ) COMMERCE, et al., ) ) Defendants. ) AMICUS BRIEF OF DATA PRIVACY EXPERTS Ryan Calo Deirdre K. Mulligan Ran Canetti Omer Reingold Aloni Cohen Aaron Roth Cynthia Dwork Guy N. Rothblum Roxana Geambasu Aleksandra (Seša) Slavkovic Somesh Jha Adam Smith Nitin Kohli Kunal Talwar Aleksandra Korolova Salil Vadhan Jing Lei Larry Wasserman Katrina Ligett Daniel J. Weitzner Shannon L. Holliday Michael B. Jones (ASB-5440-Y77S) Georgia Bar No. 721264 COPELAND, FRANCO, SCREWS [email protected] & GILL, P.A. BONDURANT MIXSON & P.O. Box 347 ELMORE, LLP Montgomery, AL 36101-0347 1201 West Peachtree Street, NW Suite 3900 Atlanta, GA 30309 Counsel for the Data Privacy Experts #3205841v1 Case 3:21-cv-00211-RAH-ECM-KCN Document 99-1 Filed 04/23/21 Page 3 of 27 TABLE OF CONTENTS STATEMENT OF INTEREST ..................................................................................... 1 SUMMARY OF ARGUMENT .................................................................................... 1 ARGUMENT ................................................................................................................. 2 I. Reconstruction attacks Are Real and Put the Confidentiality of Individuals Whose Data are Reflected -
Calibrating Noise to Sensitivity in Private Data Analysis
Calibrating Noise to Sensitivity in Private Data Analysis Cynthia Dwork1, Frank McSherry1, Kobbi Nissim2, and Adam Smith3? 1 Microsoft Research, Silicon Valley. {dwork,mcsherry}@microsoft.com 2 Ben-Gurion University. [email protected] 3 Weizmann Institute of Science. [email protected] Abstract. We continue a line of research initiated in [10, 11] on privacy- preserving statistical databases. Consider a trusted server that holds a database of sensitive information. Given a query function f mapping databases to reals, the so-called true answer is the result of applying f to the database. To protect privacy, the true answer is perturbed by the addition of random noise generated according to a carefully chosen distribution, and this response, the true answer plus noise, is returned to the user. P Previous work focused on the case of noisy sums, in which f = i g(xi), where xi denotes the ith row of the database and g maps data- base rows to [0, 1]. We extend the study to general functions f, proving that privacy can be preserved by calibrating the standard deviation of the noise according to the sensitivity of the function f. Roughly speak- ing, this is the amount that any single argument to f can change its output. The new analysis shows that for several particular applications substantially less noise is needed than was previously understood to be the case. The first step is a very clean characterization of privacy in terms of indistinguishability of transcripts. Additionally, we obtain separation re- sults showing the increased value of interactive sanitization mechanisms over non-interactive. -
Efficiently Querying Databases While Providing Differential Privacy
Epsolute: Efficiently Querying Databases While Providing Differential Privacy Dmytro Bogatov Georgios Kellaris George Kollios Boston University Canada Boston University Boston, MA, USA [email protected] Boston, MA, USA [email protected] [email protected] Kobbi Nissim Adam O’Neill Georgetown University University of Massachusetts, Amherst Washington, D.C., USA Amherst, MA, USA [email protected] [email protected] ABSTRACT KEYWORDS As organizations struggle with processing vast amounts of informa- Differential Privacy; ORAM; differential obliviousness; sanitizers; tion, outsourcing sensitive data to third parties becomes a necessity. ACM Reference Format: To protect the data, various cryptographic techniques are used in Dmytro Bogatov, Georgios Kellaris, George Kollios, Kobbi Nissim, and Adam outsourced database systems to ensure data privacy, while allowing O’Neill. 2021. Epsolute: Efficiently Querying Databases While Providing efficient querying. A rich collection of attacks on such systems Differential Privacy. In Proceedings of the 2021 ACM SIGSAC Conference on has emerged. Even with strong cryptography, just communication Computer and Communications Security (CCS ’21), November 15–19, 2021, volume or access pattern is enough for an adversary to succeed. Virtual Event, Republic of Korea. ACM, New York, NY, USA, 15 pages. https: In this work we present a model for differentially private out- //doi.org/10.1145/3460120.3484786 sourced database system and a concrete construction, Epsolute, that provably conceals the aforementioned leakages, while remaining 1 INTRODUCTION efficient and scalable. In our solution, differential privacy ispre- Secure outsourced database systems aim at helping organizations served at the record level even against an untrusted server that outsource their data to untrusted third parties, without compro- controls data and queries. -
An Introduction to Johnson–Lindenstrauss Transforms
An Introduction to Johnson–Lindenstrauss Transforms Casper Benjamin Freksen∗ 2nd March 2021 Abstract Johnson–Lindenstrauss Transforms are powerful tools for reducing the dimensionality of data while preserving key characteristics of that data, and they have found use in many fields from machine learning to differential privacy and more. This note explains whatthey are; it gives an overview of their use and their development since they were introduced in the 1980s; and it provides many references should the reader wish to explore these topics more deeply. The text was previously a main part of the introduction of my PhD thesis [Fre20], but it has been adapted to be self contained and serve as a (hopefully good) starting point for readers interested in the topic. 1 The Why, What, and How 1.1 The Problem Consider the following scenario: We have some data that we wish to process but the data is too large, e.g. processing the data takes too much time, or storing the data takes too much space. A solution would be to compress the data such that the valuable parts of the data are kept and the other parts discarded. Of course, what is considered valuable is defined by the data processing we wish to apply to our data. To make our scenario more concrete let us say that our data consists of vectors in a high dimensional Euclidean space, R3, and we wish to find a transform to embed these vectors into a lower dimensional space, R<, where < 3, so that we arXiv:2103.00564v1 [cs.DS] 28 Feb 2021 ≪ can apply our data processing in this lower dimensional space and still get meaningful results. -
Adversarial Robustness of Streaming Algorithms Through Importance Sampling
Adversarial Robustness of Streaming Algorithms through Importance Sampling Vladimir Braverman Avinatan Hassidim Yossi Matias Mariano Schain Google∗ Googley Googlez Googlex Sandeep Silwal Samson Zhou MITO CMU{ June 30, 2021 Abstract Robustness against adversarial attacks has recently been at the forefront of algorithmic design for machine learning tasks. In the adversarial streaming model, an adversary gives an algorithm a sequence of adaptively chosen updates u1; : : : ; un as a data stream. The goal of the algorithm is to compute or approximate some predetermined function for every prefix of the adversarial stream, but the adversary may generate future updates based on previous outputs of the algorithm. In particular, the adversary may gradually learn the random bits internally used by an algorithm to manipulate dependencies in the input. This is especially problematic as many important problems in the streaming model require randomized algorithms, as they are known to not admit any deterministic algorithms that use sublinear space. In this paper, we introduce adversarially robust streaming algorithms for central machine learning and algorithmic tasks, such as regression and clustering, as well as their more general counterparts, subspace embedding, low-rank approximation, and coreset construction. For regression and other numerical linear algebra related tasks, we consider the row arrival streaming model. Our results are based on a simple, but powerful, observation that many importance sampling-based algorithms give rise to adversarial robustness which is in contrast to sketching based algorithms, which are very prevalent in the streaming literature but suffer from adversarial attacks. In addition, we show that the well-known merge and reduce paradigm in streaming is adversarially robust.