DOCSLIB.ORG

Arxiv:1809.07750V3 [Cs.CR] 4 May 2021 Ible with Mechanisms Like This One—It Is Impossible to Run Analysis of the Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Arxiv:1809.07750V3 [Cs.CR] 4 May 2021 Ible with Mechanisms Like This One—It Is Impossible to Run Analysis of the Data CHORUS: a Programming Framework for Building Scalable Differential Privacy Mechanisms Noah Johnson Joseph P. Near Joseph M. Hellerstein Dawn Song UC Berkeley University of Vermont UC Berkeley UC Berkeley [email protected] [email protected] [email protected] [email protected] Abstract—Differential privacy is fast becoming the gold stan- [64], [67] to special-purpose analytics tasks such as graph dard in enabling statistical analysis of data while protecting analysis [24], [42], [46], [47], [72], linear queries [8], [25], the privacy of individuals. However, practical use of differ- [41], [50]–[53], [68], [81], [82], [85], and analysis of data ential privacy still lags behind research progress because streams [31], [74]. research prototypes cannot satisfy the scalability require- Despite extensive academic research and an abundant ments of production deployments. To address this challenge, supply of mechanisms, differential privacy has not been we present CHORUS, a framework for building scalable dif- widely adopted in practice. Existing applications of dif- ferential privacy mechanisms which is based on cooperation ferential privacy in practice are limited to specialized use between the mechanism itself and a high-performance pro- cases [2], [34]. duction database management system (DBMS). We demon- A major challenge for the practical adoption of differ- strate the use of CHORUS to build the first highly scal- ential privacy is the ability to deploy differential privacy able implementations of complex mechanisms like Weighted mechanisms at scale. Today’s data analytics infrastruc- PINQ, MWEM, and the matrix mechanism. We report on tures include industrial-grade database management sys- our experience deploying CHORUS at Uber, and evaluate its tems (DBMSs) carefully tuned for performance and reli- scalability on real-world queries. ability, designed to process datasets consisting of billions of rows. Index Terms—privacy, differential privacy, SQL queries, The simplest mechanisms for differential privacy, like query rewriting, security the Laplace mechanism [30], answer an analyst’s query by adding noise to the final result of the query. This 1. Introduction mechanism can be easily deployed atop an existing high- performance DBMS by leveraging the DBMS to execute Organizations are collecting more and more sensitive the analyst’s query, then adding the right amount of noise information about individuals. As this data is highly valu- to the result. Since it uses the DBMS to perform the actual able for a broad range of business interests, organizations data processing tasks, this approach scales well. Some ex- are motivated to provide analysts with flexible access to isting work, such as FLEX [44], takes this approach, which the data. At the same time, the public is increasingly we call the post-processing architecture. For appropriate concerned about privacy protection. There is a growing mechanisms, the post-processing architecture solves the and urgent need for technology solutions that balance scalability problem. these interests by supporting general-purpose analytics However, more advanced differential privacy mecha- while guaranteeing privacy protection. nisms require fundamental changes to the way queries exe- Differential privacy [28], [32] is widely recognized by cute. Summation queries, for example, require clipping the experts as the most rigorous theoretical solution to this data before summing it to control the influence of outliers. problem. Differential privacy provides a formal guarantee The post-processing approach is fundamentally incompat- of privacy for individuals while allowing general statistical arXiv:1809.07750v3 [cs.CR] 4 May 2021 ible with mechanisms like this one—it is impossible to run analysis of the data. In short, it states that the presence or the analyst’s query unmodified and then achieve differen- absence of any single individual’s data should not have a tial privacy by post-processing the results. Unfortunately, large effect on the results of a query. This allows precise the vast majority of recently-developed mechanisms fall answers to questions about populations in the data while into this category (e.g. [8], [24], [25], [41], [42], [46], guaranteeing the results reveal little about any individual. [47], [50]–[53], [68], [72], [81], [82], [85]), and cannot Unlike alternative approaches such as anonymization and be implemented using the post-processing architecture. k-anonymity, differential privacy protects against a wide range of attacks, including attacks using auxiliary infor- As a result, no scalable implementation exists for mation [27], [61], [65], [77]. many of the exciting differential privacy mechanisms de- Current research on differential privacy focuses on veloped in recent years. The implementations which have development of new algorithms, called mechanisms, to been developed (e.g. [49], [55], [59], [67], [80]) modify achieve differential privacy for a particular class of or replace the DBMS with a custom engine, which is un- queries. Researchers have developed dozens of mecha- likely to offer performance on par with modern production nisms covering a broad range of use cases, from general- DBMSs. purpose statistical queries [19], [30], [54], [55], [59], The CHORUS Framework. This paper describes CHO- 1 RUS, a framework for developing and deploying cutting- tributions: edge differential privacy mechanisms at scale. CHO- 1) We present the CHORUS framework, which enables a RUS makes it easy to develop mechanism implementa- novel cooperative architecture for implementing dif- tions which work in cooperation with an existing high- ferential privacy mechanisms atop high-performance performance DBMS, even for mechanisms which require DBMSs (§ 3). modifying queries or generating entirely new ones. CHO- 2) We demonstrate CHORUS’s flexibility by developing RUS supports scalable implementations by leveraging the scalable implementations for a number of advanced DBMS for data processing tasks, instead of custom code. differential privacy mechanisms (§ 5). We call this the cooperative architecture for differential privacy mechanisms. 3) We release CHORUS as open source [3] and describe CHORUS provides a programming framework to sup- how to deploy it to provide differential privacy in port implementing mechanisms in the cooperative ar- production settings (§ 6). chitecture. The framework has three major components: 4) We report on our experience deploying CHORUS to rewriting, for modifying queries to perform functions enforce differential privacy at Uber, where it processes like clipping; analysis, for analyzing queries to determine more than 10,000 queries per day (§ 6). properties like how much noise is required for differential 5) We demonstrate the scalability of CHORUS by evaluat- privacy; and post-processing, for processing the results of ing it on 18,774 real-world queries with a database of executing queries. To implement a summation mechanism 300 million rows (§ 7). with clipping, for example, we can use CHORUS’s rewrit- ing component to modify the analyst’s query so that the 2. Background DBMS performs the clipping as well as the summation. With this modification, the rest of the mechanism can Differential privacy provides a formal guarantee of be implemented via analysis and post-processing of the indistinguishability. This guarantee is defined in terms of rewritten query. a privacy budget —the smaller the budget, the stronger CHORUS supports integration with any standard SQL the guarantee. The formal definition of differential privacy database. The framework is designed to facilitate working is written in terms of the distance d(x; y) between two directly with SQL queries, since SQL is the most com- databases, i.e. the number of entries on which they differ: monly used language for high-performance production d(x; y) = jfi : xi 6= yigj. Two databases x and y DBMSs. By using a standard SQL database engine instead are neighbors if d(x; y) = 1. A randomized mechanism of a custom runtime, CHORUS can leverage the reliability, K : Dn ! R preserves (, δ)-differential privacy if for scalability and performance of modern databases, which any pair of neighboring databases x; y 2 Dn and set S of are built on decades of research and engineering experi- possible outputs: ence. The cooperative architecture applies to all of the Pr[K(x) 2 S] ≤ e Pr[K(y) 2 S] + δ recently-developed differential privacy mechanisms—even Differential privacy can be enforced by adding noise to ones that require significant changes to the way queries the non-private results of a query. The scale of this noise execute or generate entirely new queries. We demon- depends on the sensitivity of the query. The global sensi- strate the flexibility of the approach, and of the CHORUS tivity of a query f : Dn ! R is defined as: framework, by implementing both simple mechanisms like GSf = max jf(x) − f(y)j summation with clipping and complex mechanisms like x;y:d(x;y)=1 wPINQ, MWEM, and the matrix mechanism. In all of Importantly, differential privacy mechanisms satisfy a se- these implementations, CHORUS supports scalability by quential composition property: if F satisfies ( ; δ )- moving data processing tasks to the DBMS. 1 1 1 differential privacy, and F2 satisfies (2; δ2)-differential Deployment. We have made CHORUS available as open privacy, then running both F1 and F2 satisfies (1+2; δ1+ source software [3], and it is designed for integration δ2)-differential privacy. For more on differential privacy, in production environments.
Load more

Recommended publications
  • From Local to Central Differential Privacy Via Anonymity
    Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity Ulfar´ Erlingsson∗ Vitaly Feldman∗ Ilya Mironov∗ Ananth Raghunathan∗ Kunal Talwar∗ Abhradeep Thakurtay Abstract Sensitive statistics are often collected across sets of users, with repeated collection of reports done over time. For example, trends in users’ private preferences or software usage may be monitored via such reports. We study the collection of such statistics in the local differential privacy (LDP) model, and describe an algorithm whose privacy cost is polylogarithmic in the number of changes to a user’s value. More fundamentally—by building on anonymity of the users’ reports—we also demonstrate how the privacy cost of our LDP algorithm can actually be much lower when viewed in the central model of dif- ferential privacy. We show, via a new and general privacy amplification technique, that any permutation- p invariant algorithm satisfying "-local differential privacy will satisfyp (O(" log(1/δ)=n); δ)-central dif- ferential privacy. By this, we explain how the high noise and n overhead of LDP protocols is a con- sequence of them being significantly more private in the central model. As a practical corollary, our results imply that several LDP-based industrial deployments may have much lower privacy cost than their advertised " would indicate—at least if reports are anonymized. 1 Introduction A frequent task in data analysis is the monitoring of the statistical properties of evolving data in a manner that requires repeated computation on the entire evolving dataset. Software applications commonly apply online monitoring, e.g., to establish trends in the software configurations or usage patterns.
    [Show full text]
  • The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space
    The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space Adam Smith Shuang Song Abhradeep Thakurta Boston University Google Research, Brain Team Google Research, Brain Team [email protected] [email protected] [email protected] Abstract We revisit the problem of counting the number of distinct elements F0(D) in a data stream D, over a domain [u]. We propose an ("; δ)-differentially private algorithm that approximates F0(D) within a factor of (1 ± γ), and with additive error of p O( ln(1/δ)="), using space O(ln(ln(u)/γ)/γ2). We improve on the prior work at least quadratically and up to exponentially, in terms of both space and additive p error. Our additive error guarantee is optimal up to a factor of O( ln(1/δ)), n ln(u) 1 o and the space bound is optimal up to a factor of O min ln γ ; γ2 . We assume the existence of an ideal uniform random hash function, and ignore the space required to store it. We later relax this requirement by assuming pseudo- random functions and appealing to a computational variant of differential privacy, SIM-CDP. Our algorithm is built on top of the celebrated Flajolet-Martin (FM) sketch. We show that FM-sketch is differentially private as is, as long as there are p ≈ ln(1/δ)=(εγ) distinct elements in the data set. Along the way, we prove a structural result showing that the maximum of k i.i.d. random variables is statisti- cally close (in the sense of "-differential privacy) to the maximum of (k + 1) i.i.d.
    [Show full text]
  • Efficiently Querying Databases While Providing Differential Privacy
    Epsolute: Efficiently Querying Databases While Providing Differential Privacy Dmytro Bogatov Georgios Kellaris George Kollios Boston University Canada Boston University Boston, MA, USA [email protected] Boston, MA, USA [email protected] [email protected] Kobbi Nissim Adam O’Neill Georgetown University University of Massachusetts, Amherst Washington, D.C., USA Amherst, MA, USA [email protected] [email protected] ABSTRACT KEYWORDS As organizations struggle with processing vast amounts of informa- Differential Privacy; ORAM; differential obliviousness; sanitizers; tion, outsourcing sensitive data to third parties becomes a necessity. ACM Reference Format: To protect the data, various cryptographic techniques are used in Dmytro Bogatov, Georgios Kellaris, George Kollios, Kobbi Nissim, and Adam outsourced database systems to ensure data privacy, while allowing O’Neill. 2021. Epsolute: Efficiently Querying Databases While Providing efficient querying. A rich collection of attacks on such systems Differential Privacy. In Proceedings of the 2021 ACM SIGSAC Conference on has emerged. Even with strong cryptography, just communication Computer and Communications Security (CCS ’21), November 15–19, 2021, volume or access pattern is enough for an adversary to succeed. Virtual Event, Republic of Korea. ACM, New York, NY, USA, 15 pages. https: In this work we present a model for differentially private out- //doi.org/10.1145/3460120.3484786 sourced database system and a concrete construction, Epsolute, that provably conceals the aforementioned leakages, while remaining 1 INTRODUCTION efficient and scalable. In our solution, differential privacy ispre- Secure outsourced database systems aim at helping organizations served at the record level even against an untrusted server that outsource their data to untrusted third parties, without compro- controls data and queries.
    [Show full text]
  • ACM SIGACT Announces 2017 Awards Gödel, Knuth and Distinguished Service Prizes Recognize Contributions to Algorithms and Compu
    Contact: Michael Mitzenmacher Jim Ormond ACM SIGACT Chair ACM 617-496-7172 212-626-0505 [email protected] [email protected] ACM SIGACT Announces 2017 Awards Gödel, Knuth and Distinguished Service Prizes Recognize Contributions to Algorithms and Computation Theory Community NEW YORK, June 19, 2017 – The Association for Computing Machinery’s Special Interest Group on Algorithms and Computation Theory (SIGACT) recently announced the recipients of three significant awards. The 2017 Gödel Prize is awarded to Cynthia Dwork, Frank McSherry, Kobbi Nissim and Adam Smith for their work on differential privacy in their paper “Calibrating Noise to Sensitivity in Private Data Analysis.” The 2017 Donald E. Knuth Prize is awarded to Oded Goldreich of the Weizmann Institute of Science for fundamental and lasting contributions to many areas of theoretical computer science, including cryptography, probabilistically checkable proofs, inapproximability, property testing, and several other areas in complexity theory. Also this year, Alistair Sinclair of the University of California, Berkeley is the recipient of the 2017 ACM SIGACT Distinguished Service Prize for his role in the spectacular success of The Simons Institute for the Theory of Computing. These awards will be presented at the 49th Annual ACM Symposium on the Theory of Computing (STOC 2017), to be held June 19-23, in Montreal, Canada. 2017 Gödel Prize Differential privacy is a powerful theoretical model for dealing with the privacy of statistical data. The intellectual impact of differential privacy has been broad, influencing thinking about privacy across many disciplines. The work of Cynthia Dwork (Harvard University), Frank McSherry (independent researcher), Kobbi Nissim (Harvard University), and Adam Smith (Harvard University) launched a new line of theoretical research aimed at understanding the possibilities and limitations of differentially private algorithms.
    [Show full text]
  • Answering Multi-Dimensional Range Queries Under Local Differential Privacy
    Answering Multi-Dimensional Range Queries under Local Differential Privacy Jianyu Yang1,2∗, Tianhao Wang2, Ninghui Li2, Xiang Cheng1, Sen Su1 1State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China {jyyang, chengxiang, susen}@bupt.edu.cn 2Department of Computer Science, Purdue University, West Lafayette, USA {yang1896, tianhaowang}@purdue.edu, [email protected] ABSTRACT a meaningful total order. A typical kind of fundamental analysis In this paper, we tackle the problem of answering multi-dimensional over users’ records is multi-dimensional range query, which is a range queries under local differential privacy. There are three key conjunction of multiple predicates for the attributes of interest and technical challenges: capturing the correlations among attributes, asks the fraction of users whose record satisfies all the predicates. avoiding the curse of dimensionality, and dealing with the large In particular, a predicate is a restriction on the range of values of an domains of attributes. None of the existing approaches satisfacto- attribute. However, users’ records regarding these ordinal attributes rily deals with all three challenges. Overcoming these three chal- are highly sensitive. Without strong privacy guarantee, answering lenges, we first propose an approach called Two-Dimensional Grids multi-dimensional range queries over them will put individual pri- (TDG). Its main idea is to carefully use binning to partition the vacy in jeopardy. Thus, developing effective approaches to address two-dimensional (2-D) domains of all attribute pairs into 2-D grids such privacy concerns becomes an urgent need. that can answer all 2-D range queries and then estimate the an- In recent years, local differential privacy (LDP) has come to swer of a higher dimensional range query from the answers of the be the de facto standard for individual privacy protection.
    [Show full text]
  • Differential Privacy at Risk: Bridging Randomness and Privacy Budget Ashish Dandekar, Debabrota Basu, Stéphane Bressan
    Differential Privacy at Risk: Bridging Randomness and Privacy Budget Ashish Dandekar, Debabrota Basu, Stéphane Bressan To cite this version: Ashish Dandekar, Debabrota Basu, Stéphane Bressan. Differential Privacy at Risk: Bridging Ran- domness and Privacy Budget. Proceedings on Privacy Enhancing Technologies, De Gruyter Open, 2021, 2021 (1), pp.64-84. 10.2478/popets-2021-0005. hal-02942997 HAL Id: hal-02942997 https://hal.inria.fr/hal-02942997 Submitted on 18 Sep 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Proceedings on Privacy Enhancing Technologies ..; .. (..):1–21 Ashish Dandekar*, Debabrota Basu*, and Stéphane Bressan Differential Privacy at Risk: Bridging Randomness and Privacy Budget Abstract: The calibration of noise for a privacy- 1 Introduction preserving mechanism depends on the sensitivity of the query and the prescribed privacy level. A data steward Dwork et al. [12] quantify the privacy level ε in ε- must make the non-trivial choice of a privacy level that differential privacy (or ε-DP) as an upper bound on the balances the requirements of users and the monetary worst-case privacy loss incurred by a privacy-preserving constraints of the business entity. mechanism.
    [Show full text]
  • Differential Privacy: a Primer for a Non-Technical Audience
    Differential Privacy: A Primer for a Non-Technical Audience Alexandra Wood, Micah Altman, Aaron Bembenek, Mark Bun, Marco Gaboardi, James Honaker, Kobbi Nissim, David R. O’Brien, Thomas Steinke & Salil Vadhan* ABSTRACT Differential privacy is a formal mathematical framework for quantifying and managing privacy risks. It provides provable privacy protection against a wide range of potential attacks, including those * Alexandra Wood is a Fellow at the Berkman Klein Center for Internet & Society at Harvard University. Micah Altman is Director of Research at MIT Libraries. Aaron Bembenek is a PhD student in computer science at Harvard University. Mark Bun is a Google Research Fellow at the Simons Institute for the Theory of Computing. Marco Gaboardi is an Assistant Professor in the Computer Science and Engineering department at the State University of New York at Buffalo. James Honaker is a Research Associate at the Center for Research on Computation and Society at the Harvard John A. Paulson School of Engineering and Applied Sciences. Kobbi Nissim is a McDevitt Chair in Computer Science at Georgetown University and an Affiliate Professor at Georgetown University Law Center; work towards this document was completed in part while the Author was visiting the Center for Research on Computation and Society at Harvard University. David R. O’Brien is a Senior Researcher at the Berkman Klein Center for Internet & Society at Harvard University. Thomas Steinke is a Research Staff Member at IBM Research – Almaden. Salil Vadhan is the Vicky Joseph Professor of Computer Science and Applied Mathematics at Harvard University. This Article is the product of a working group of the Privacy Tools for Sharing Research Data project at Harvard University (http://privacytools.seas.harvard.edu).
    [Show full text]
  • Differentially-Private Software Analytics for Mobile Apps: Opportunities and Challenges
    Differentially-Private Software Analytics for Mobile Apps: Opportunities and Challenges Hailong Zhang Sufian Latif Ohio State University Ohio State University Columbus, OH, USA Columbus, OH, USA [email protected] [email protected] Raef Bassily Atanas Rountev Ohio State University Ohio State University Columbus, OH, USA Columbus, OH, USA [email protected] [email protected] ABSTRACT concern is amplified in the current environment, in which both Software analytics libraries are widely used in mobile applications, software users and legislative bodies are becoming increasingly which raises many questions about trade-offs between privacy, proactive in demanding privacy protections. In this context, it is utility, and practicality. A promising approach to address these essential to understand and enforce meaningful trade-offs between questions is differential privacy. This algorithmic framework has the benefits of data gathering and the privacy of software users. emerged in the last decade as the foundation for numerous al- Such trade-offs can be studied through the lens of privacy-pre- gorithms with strong privacy guarantees, and has recently been serving data analysis. In particular, we consider differential pri- adopted by several projects in industry and government. This paper vacy [8], an approach that quantifies these trade-offs and provides discusses the benefits and challenges of employing differential pri- a framework for disciplined algorithm design. Due to its attractive vacy in software analytics used in mobile apps. We aim to outline an properties, differential privacy has reigned as the gold standard of initial research agenda that serves as the starting point for further statistical data privacy.
    [Show full text]
  • Local Differential Privacy for Evolving Data
    Local Differential Privacy for Evolving Data Matthew Joseph Aaron Roth Computer and Information Science Computer and Information Science University of Pennsylvania University of Pennsylvania [email protected] [email protected] Jonathan Ullman Bo Waggoner Computer and Information Sciences Computer and Information Science Northeastern University University of Pennsylvania [email protected] [email protected] Abstract There are now several large scale deployments of differential privacy used to collect statistical information about users. However, these deployments periodically recollect the data and recompute the statistics using algorithms designed for a single use. As a result, these systems do not provide meaningful privacy guarantees over long time scales. Moreover, existing techniques to mitigate this effect do not apply in the “local model” of differential privacy that these systems use. In this paper, we introduce a new technique for local differential privacy that makes it possible to maintain up-to-date statistics over time, with privacy guarantees that degrade only in the number of changes in the underlying distribution rather than the number of collection periods. We use our technique for tracking a changing statistic in the setting where users are partitioned into an unknown collection of groups, and at every time period each user draws a single bit from a common (but changing) group-specific distribution. We also provide an application to frequency and heavy-hitter estimation. 1 Introduction After over a decade of research, differential privacy [12] is moving from theory to practice, with notable deployments by Google [15, 6], Apple [2], Microsoft [10], and the U.S. Census Bureau [1].
    [Show full text]
  • Learning with Differential Privacy
    Journal of Machine Learning Research 17 (2016) 1-40 Submitted 6/15; Revised 3/16; Published 4/16 Learning with Differential Privacy: Stability, Learnability and the Sufficiency and Necessity of ERM Principle Yu-Xiang Wang1;2 [email protected] Jing Lei2 [email protected] Stephen E. Fienberg1;2 [email protected] 1 Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213 2 Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213 Editor: Moritz Hardt Abstract While machine learning has proven to be a powerful data-driven solution to many real- life problems, its use in sensitive domains has been limited due to privacy concerns. A popular approach known as differential privacy offers provable privacy guarantees, but it is often observed in practice that it could substantially hamper learning accuracy. In this paper we study the learnability (whether a problem can be learned by any algorithm) under Vapnik's general learning setting with differential privacy constraint, and reveal some intricate relationships between privacy, stability and learnability. In particular, we show that a problem is privately learnable if an only if there is a private algorithm that asymptotically minimizes the empirical risk (AERM). In contrast, for non-private learning AERM alone is not sufficient for learnability. This result suggests that when searching for private learning algorithms, we can restrict the search to algorithms that are AERM. In light of this, we propose a conceptual procedure that always finds a universally consistent algorithm whenever the problem is learnable under privacy constraint. We also propose a generic and practical algorithm and show that under very general conditions it privately learns a wide class of learning problems.
    [Show full text]
  • Asymmetric Differential Privacy
    Asymmetric Differential Privacy Shun Takagi Yang Cao Masatoshi Yoshikawa Kyoto University Kyoto University Kyoto University Kyoto, Japan Kyoto, Japan Kyoto, Japan [email protected] [email protected] [email protected] ABSTRACT cannot know the true answer is whether yes or no due to the pos- Recently, differential privacy (DP) is getting attention as a privacy sible error (i.e., two-sided error). A one-sided error means that the definition when publishing statistics of a dataset. However, when randomized mechanism correctly answers for a one-sided answer answering a decision problem with a DP mechanism, it causes a (e.g., yes), but the opposite answer may include error. If we answer two-sided error. This characteristic of DP is not desirable when pub- yes using Mangat’s randomized response [26], we can know the true lishing risk information such as concerning COVID-19. This paper answer is yes because the output of the algorithm has the one-sided proposes relaxing DP to mitigate the limitation and improve the util- characteristic. ity of published information. First, we define a policy that separates As an example of a problem of the two-sided characteristic, we information into sensitive and non-sensitive. Then, we define asym- consider publishing risky location information for epidemic disease metric differential privacy (ADP) that provides the same privacy monitoring. Specifically, we want to publish each location is safeor guarantee as DP to sensitive information. This partial protection not safe, which means whether it has been visited by many infected induces asymmetricity in privacy protection to improve utility and people (i.e., counting query).
    [Show full text]
  • Arxiv:2103.05472V2 [Cs.CV] 7 Apr 2021
    Differentially Private Imaging via Latent Space Manipulation Tao Li and Chris Clifton Department of Computer Science, Purdue University, West Lafayette, Indiana, USA ftaoli,[email protected] Abstract There is growing concern about image privacy due to the popularity of social media and photo devices, along with increasing use of face recognition systems. However, es- tablished image de-identification techniques are either too subject to re-identification, produce photos that are insuf- ficiently realistic, or both. To tackle this, we present a novel approach for image obfuscation by manipulating la- tent spaces of an unconditionally trained generative model that is able to synthesize photo-realistic facial images of high resolution. This manipulation is done in a way that sat- isfies the formal privacy standard of local differential pri- Figure 1: Can you identify the authors? These are images vacy. To our knowledge, this is the first approach to image of the authors, with noise added that satisfies differential privacy that satisfies "-differential privacy for the person. privacy sufficient to prevent identification of the authors if you do not already know who they are. 1. Introduction and some privacy guarantees are provided. A pixelization Image obfuscation techniques have been used to protect method proposed in [3] satisfies pixel-wise -differential sensitive information, such as human faces and confidential privacy [4]. However, the utility of the pixelized images texts. However, recent advances in machine learning, es- is far from satisfactory, due to the high perturbation noise pecially deep learning, make standard obfuscation methods needed to reasonably hide the original; the images appear such as pixelization and blurring less effective at protecting like traditional pixelization or blurring techniques.
    [Show full text]