Cryptϵ: Crypto-Assisted Differential Privacy on Untrusted Servers Amrita Roy Chowdhury Chenghong Wang Xi He University of Wisconsin-Madison Duke University University of Waterloo [email protected] [email protected] [email protected] Ashwin Machanavajjhala Somesh Jha Duke University University of Wisconsin-Madison [email protected] [email protected]

ABSTRACT optimizations leveraging the fact that the output is noisy. Differential privacy (DP) is currently the de-facto standard We demonstrate Cryptϵ’s practical feasibility with extensive for achieving privacy in data analysis, which is typically im- empirical evaluations on real world datasets. plemented either in the “central” or “local” model. The local ACM Reference Format: model has been more popular for commercial deployments Amrita Roy Chowdhury, Chenghong Wang, Xi He, Ashwin Machanava- as it does not require a trusted data collector. This increased jjhala, and Somesh Jha. 2020. Cryptϵ: Crypto-Assisted Differen- privacy, however, comes at the cost of utility and algorithmic tial Privacy on Untrusted Servers. In 2020 ACM SIGMOD Inter- expressibility as compared to the central model. national Conference on Management of Data (SIGMOD’20), June In this work, we propose, Cryptϵ, a system and program- 14–19, 2020, Portland, OR, USA. ACM, New York, NY, USA, 24 pages. ming framework that (1) achieves the accuracy guarantees https://doi.org/10.1145/3318464.3380596 and algorithmic expressibility of the central model (2) with- out any trusted data collector like in the local model. Cryptϵ 1 INTRODUCTION achieves the “best of both worlds” by employing two non- Differential privacy (DP) is a rigorous privacy definition that colluding untrusted servers that run DP programs on en- is currently the gold standard for data privacy. It is typically crypted data from the data owners. In theory, straightforward implemented in one of two models – centralized differential implementations of DP programs using off-the-shelf secure privacy (CDP) and local differential privacy (LDP). In CDP, multi-party computation tools can achieve the above goal. data from individuals are collected and stored in the clear However, in practice, they are beset with many challenges in a trusted centralized data curator which then executes like poor performance and tricky security proofs. To this end, DP programs on the sensitive data and releases outputs to Cryptϵ allows data analysts to author logical DP programs an untrusted data analyst. In LDP, there is no trusted data that are automatically translated to secure protocols that curator. Rather, each individual perturbs his/her own data work on encrypted data. These protocols ensure that the using a (local) DP algorithm. The data analyst uses these untrusted servers learn nothing more than the noisy outputs, noisy data to infer aggregate statistics of the datasets. In thereby guaranteeing DP (for computationally bounded ad- practice, CDP’s assumption of a trusted server is ill-suited versaries) for all Cryptϵ programs. Cryptϵ supports a rich for many applications as it constitutes a single point of fail- class of DP programs that can be expressed via a small set ure for data breaches, and saddles the trusted curator with of transformation and measurement operators followed by arXiv:1902.07756v5 [cs.CR] 10 Mar 2020 legal and ethical obligations to uphold data privacy. Hence, arbitrary post-processing. Further, we propose performance recent commercial deployments of DP [42, 51] have preferred LDP over CDP. However, LDP’s attractive privacy properties Permission to make digital or hard copies of all or part of this work for comes at a cost. Under the CDP model, the expected additive personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear error for a aggregate count over a dataset of size n is at most this notice and the full citation on the first page. Copyrights for components Θ(1/ϵ) to√ achieve ϵ-DP. In contrast, under the LDP model, at of this work owned by others than ACM must be honored. Abstracting with least Ω( n/ϵ) additive expected error must be incurred by credit is permitted. To copy otherwise, or republish, to post on servers or to any ϵ-DP program [16, 28, 36], owing to the randomness of redistribute to lists, requires prior specific permission and/or a fee. Request each data owner. The LDP model in fact imposes additional permissions from [email protected]. penalties on the algorithmic expressibility; the power of LDP SIGMOD ’20, June 14–19, 2020, Portland, OR, USA © 2020 Association for Computing Machinery. is equivalent to that of the statistical query model [66] and ACM ISBN 978-1-4503-6735-6/20/06...$15.00 there exists an exponential separation between the accuracy https://doi.org/10.1145/3318464.3380596 and sample complexity of LDP and CDP algorithms [64]. In this paper, we strive to bridge the gap between LDP Data Owners Analytics Server ˜ Data Analyst DAAACAXicbVDLSsNAFJ3UV62vqBvBzWARXJVEBF0WdeGygq2FJoTJZNIOncyEmYlQQtz4K25cKOLWv3Dn3zhps9DWA8MczrmXe+8JU0aVdpxvq7a0vLK6Vl9vbGxube/Yu3s9JTKJSRcLJmQ/RIowyklXU81IP5UEJSEj9+H4qvTvH4hUVPA7PUmJn6AhpzHFSBspsA+8ULBITRLz5Z6mLCL5dRG4RWA3nZYzBVwkbkWaoEInsL+8SOAsIVxjhpQauE6q/RxJTTEjRcPLFEkRHqMhGRjKUUKUn08vKOCxUSIYC2ke13Cq/u7IUaLKJU1lgvRIzXul+J83yHR84eeUp5kmHM8GxRmDWsAyDhhRSbBmE0MQltTsCvEISYS1Ca1hQnDnT14kvdOW67Tc27Nm+7KKow4OwRE4AS44B21wAzqgCzB4BM/gFbxZT9aL9W59zEprVtWzD/7A+vwBPXaXYA== 1 3.Program Executor Crypt� and CDP. We propose, Cryptϵ, a system and a programming 2.Aggregator Program . ˜ framework for executing DP programs that: . AAACCXicbVDLSsNAFJ3UV62vqEs3g0VwVRIRdFnUhcsK9gFNKJPJbTt0MgkzE6GEbN34K25cKOLWP3Dn3zhps9DWC8MczrmXe+4JEs6Udpxvq7Kyura+Ud2sbW3v7O7Z+wcdFaeSQpvGPJa9gCjgTEBbM82hl0ggUcChG0yuC737AFKxWNzraQJ+REaCDRkl2lADG3ua8RAyL4h5qKaR+TIvInpMCc9u8jwf2HWn4cwKLwO3BHVUVmtgf3lhTNMIhKacKNV3nUT7GZGaUQ55zUsVJIROyAj6BgoSgfKz2SU5PjFMiIexNE9oPGN/T2QkUoVL01mYVItaQf6n9VM9vPQzJpJUg6DzRcOUYx3jIhYcMglU86kBhEpmvGI6JpJQbcKrmRDcxZOXQees4ToN9+683rwq46iiI3SMTpGLLlAT3aIWaiOKHtEzekVv1pP1Yr1bH/PWilXOHKI/ZX3+AD5Gm04= . D Program3.Program translated Executor to • underlying secure never stores or computes on sensitive data in the clear computation protocols Differentially Private Output • achieves the accuracy guarantees and algorithmic express- Data Collection Phase Program Execution Phase ibility of the CDP model 1.Key 4.Privacy 5. Data Crypt� Manager Engine Decryption Program

sk Key✏AAACAHicbVDLSsNAFJ3UV62vqAsXbgaL4KokIuiy1I3LCvYBTSiTyU07dDIJMxOhhGz8FTcuFHHrZ7jzb5y2WWjrgWEO59zLvfcEKWdKO863VVlb39jcqm7Xdnb39g/sw6OuSjJJoUMTnsh+QBRwJqCjmebQTyWQOODQCya3M7/3CFKxRDzoaQp+TEaCRYwSbaShfeIFCQ/VNDZf7kGqGDdyqxjadafhzIFXiVuSOirRHtpfXpjQLAahKSdKDVwn1X5OpGaUQ1HzMgUpoRMygoGhgsSg/Hx+QIHPjRLiKJHmCY3n6u+OnMRqtqOpjIkeq2VvJv7nDTId3fg5E2mmQdDFoCjjWCd4lgYOmQSq+dQQQiUzu2I6JpJQbTKrmRDc5ZNXSfey4ToN9/6q3myVcVTRKTpDF8hF16iJ7lAbdRBFBXpGr+jNerJerHfrY1FascqeY/QH1ucPqXyXEw== ManagerB Cryptϵ employs a pair of untrusted but non-colluding servers pk – Analytics Server (AS) and Cryptographic Service Provider Cryptographic Service Provider (CSP). The AS executes DP programs (like the data curator in CDP) but on encrypted data records. The CSP initializes Figure 1: Cryptϵ System and manages the cryptographic primitives, and collaborates with the AS to generate the program outputs. Under the as- order of magnitude. A novel contribution of this work is a sumption that the AS and the CSP are semi-honest and do DP indexing optimization that leverages the fact that noisy not collude (a common assumption in cryptographic systems intermediate statistics about the data can be revealed. [45, 46, 48, 67, 80, 83, 84]), Cryptϵ ensures ϵ-DP guarantee for • Practical for Real World Usage: For the same tasks, its programs via two cryptographic primitives – linear homo- Cryptϵ programs achieve accuracy comparable to CDP morphic encryption (LHE) and garbled circuits. One caveat and 50× more than LDP for a dataset of size ≈ 30K. Cryptϵ here is that due to the usage of cryptographic primitives, the runs within 3.6 hours for a large class of programs on a DP guarantee obtained in Cryptϵ is that of computational dataset with 1 million rows and 4 attributes. differential privacy or SIM-CDP [79] (details in Section 7). • Generalized Multiplication Using LHE: Our implemen- Cryptϵ provides a data analyst with a programming frame- tation uses an efficient way for performing n-way multipli- work to author logical DP programs just like in CDP. Like cations using LHE which maybe of independent interest. in prior work [40, 77, 106], access to the sensitive data is restricted via a set of predefined transformations operators 2 CRYPTϵ OVERVIEW (inspired by relational algebra) and DP measurement oper- ators (Laplace mechanism and Noisy-Max [38]). Thus, any 2.1 System Architecture program that can be expressed as a composition of the above Figure 1 shows Cryptϵ’s system architecture. Cryptϵ has operators automatically satisfies ϵ-DP (in the CDP model) two servers: Analytics server (AS) and Cryptographic Ser- giving the analyst a proof of privacy for free. Cryptϵ pro- vice Provider (CSP). At the very outset, the CSP records grams support constructs like looping, conditionals, and can the total privacy budget, ϵB (provided by the data owners), arbitrarily post-process outputs of measurement operators. and generates the key pair, ⟨sk,pk⟩ (details in Section 3), for The main contributions of this work are: the encryption scheme. The data owners, DOi ,i ∈ [m] (m • New Approach: We present the design and implementa- = number of data owners), encrypt their data records, Di , tion of Cryptϵ, a novel system and programming frame- in the appropriate format with the public key, pk, and send ˜ work for executing DP programs over encrypted data on the encrypted records, Di , to the AS which aggregates them two non-colluding and untrusted servers. into a single encrypted database, D˜ . Next, the AS inputs log- • Algorithm Expressibility: Cryptϵ supports a rich class ical programs from the data analyst and translates them to ϵ of state-of -the-art DP programs expressed in terms of a Crypt ’s implementation specific secure protocols that work D˜ small set of transformation and measurement operators. on . A Cryptϵ program typically consists of a sequence Thus, Cryptϵ achieves the accuracy guarantees of the CDP of transformation operators followed by a measurement op- model without the need for a trusted data curator. erator. The AS can execute most of the transformations on its own. However, each measurement operator requires an • Ease Of Use: Cryptϵ allows the data analyst to express interaction with the CSP for (a) decrypting the answer, and the DP program logic using high-level operators. Cryptϵ (b) checking that the total privacy budget, ϵB , is not exceeded. automatically translates this to the underlying implemen- In this way, the AS and the CSP compute the output of a tation specific secure protocols that work on encrypted Cryptϵ program with the data owners being offline. data and provides a DP guarantee (in the CDP model) for free. Thus, the data analyst is relieved of all concerns 2.2 Cryptϵ Design Principles regarding secure computation protocol implementation. Minimal Trust Assumptions: As mentioned above, the • Performance Optimizations: We propose optimizations overarching goal of Cryptϵ is to mimic the CDP model but that speed up computation on encrypted data by at least an without a trusted server. A natural solution for dispensing with the trust assumption of the CDP model is using crypto- using budget ϵ1. This is followed by post-processing steps of graphic primitives [9, 14, 17, 20, 29, 31, 37, 41, 92, 93]. Hence, thresholding, sorting and clustering resulting in H¯. Then a to accommodate the use of cryptographic primitives, we final histogram, H˜ , is computed with privacy budget ϵ − ϵ1. assume a computationally bounded adversary in Cryptϵ. An implementation of the entire algorithm in a single SMC However, a generic m-party SMC would be computation- protocol using the EMP toolkit [2] takes 810s for a dataset ally expensive. This necessitates a third-party entity that can of size ≈ 30K and histogram size 100. In contrast, Cryptϵ capture the requisite secure computation functionality in a uses SMC protocols only for the first and third steps. ϵCrypt 2-party protocol instead. This role is fulfilled by the CSP in automatically detects that the second post-processing step Cryptϵ. For this two-server model, we assume semi-honest can be performed in the clear. A Cryptϵ program for this runs behaviour and non-collusion. This is a very common assump- in 238s (3.4× less time than that of the EMP implementation) tion in the two-server model [45, 46, 48, 67, 80, 83, 84]. for the same dataset and histogram sizes. Programming Framework: Conceptually, the aforemen- Last, the security (privacy) proofs for just stand-alone tioned goal of achieving the best of both worlds can be ob- cryptographic and DP mechanisms can be notoriously tricky tained by implementing the required DP program using off- [19, 75]. Combining the two thus exacerbates the technical the-self secure multi-party computation (SMC) tools like complexity, making the design vulnerable to faulty proofs [2–5]. However, when it comes to real world usage, Cryptϵ [59]. For example, given any arbitrary DP program written outperforms such approaches due to the following reasons. under the CDP model, the distinction between intermedi- First, without the support of a programming framework ate results that can be released and the ones which have to like that of Cryptϵ, every DP program must be implemented be kept private is often ambiguous. An instance of this is from scratch. This requires the data analyst to be well versed observed in the Noisy-Max algorithm, where the array of in both DP and SMC techniques; he/she must know how to intermediate noisy counts is private. However, these inter- implement SMC protocols, estimate sensitivity of transforma- mediate noisy counts correspond to valid query responses. tions and track privacy budget across programs. In contrast, Thus, an incautious analyst, in a bid to improve performance, Cryptϵ allows the data analyst to write the DP program us- might reuse a previously released noisy count query out- ing a high-level and expressive programming framework. put for a subsequent execution of the Noisy-Max algorithm Cryptϵ abstracts out all the low-level implementation details leading to privacy leakage. In contrast, Cryptϵ is designed like the choice of input data format, translation of queries to reveal nothing other than the outputs of the DP programs to that format, choice of SMC primitives and privacy budget to the untrusted servers; every Cryptϵ program comes with monitoring from the analyst thereby reducing his/her burden an automatic proof of security (privacy). Referring back to of complex decision making. Thus, every Cryptϵ program is the aforementioned example, in Cryptϵ, the Noisy-Max al- automatically translated to protocols corresponding to the gorithm is implemented as a secure measurement operator underlying implementation. thereby preventing any accidental privacy leakage. The ad- Second, SMC protocols can be prohibitively costly in prac- vantages of a programming framework is further validated tice unless they are carefully tuned to the application. Cryptϵ by the popularity of systems like PINQ [77], Featherweight supports optimized implementations for a small set of oper- PINQ [40], Ektelo [106] - frameworks for the CDP setting. ators, which results in efficiency for all Cryptϵ programs. Data Owners are Offline: Recall, Cryptϵ’s goal is to mimic Third, a DP program can be typically divided into seg- the CDP model with untrusted servers. Hence, it is designed ments that (1) transform the private data, (2) perform noisy so that the data owners are offline after submitting their measurements, and (3) post-process the noisy measurements encrypted records to the AS. without touching the private data. A naive implementa- Low burden on CSP: Cryptϵ views the AS as an extension tion may implement all the steps using SMC protocols even of the analyst; the AS has a vested interest in obtaining the though post-processing can be performed in the clear. Given result of the programs. Thus, we require the AS to perform a DP program written in a general purpose programming the majority of the work for any program; interactions with language (like Python), automatically figuring out what can the CSP should be minimal and related to data decryption. be done in the clear can be subtle. In Cryptϵ programs, how- Keeping this in mind, the AS performs most of the data trans- ever, transformation and measurement are clearly delineated, formations by itself (Table 3). Specifically, for every Cryptϵ as the data can be accessed only through a pre-specified set program, the AS processes the whole database and trans- of operators. Thus, SMC protocols are only used for transfor- forms it into concise representations (an encrypted scalar or mations and measurements, which improves performance. a short vector) which is then decrypted by the CSP. An ex- For example, the AHP algorithm for histogram release ample real world setting can be when Google and Symantec ˆ [107] works as follows: first, a noisy histogram, H, is released assumes the role of the AS and the CSP respectively. Separation of logical programming framework and un- • Decryption (Dec): This uses the secret key, sk , to recover derlying physical implementation: The programming the plaintext, m, from the ciphertext, c, deterministically. framework is independent of the underlying implementation. In addition, LHE supports the operator ⊕ that allows the This allows certain flexibility in the choice for implementa- summation of ciphers as follows: tion. For example, we use one-hot-encoding as the input data Operator ⊕: Let c1 ← Encpk (m1),...,ca ← Encpk (ma) and format (Section 2). However, any other encoding scheme like a ∈ Z>0. Then we have Decsk (c1 ⊕c2...⊕ca) = m1 +...+ma. range based encoding can be used instead. Another example One can multiply a cipher c ← Encsk (m) by a plaintext posi- is that in this paper, we use ϵ-DP (pure DP) for our privacy tive integer a by a repetitions of ⊕. We denote this operation analysis. However, other DP notions like (ϵ, δ)-DP, Rènyi DP
by cMult(a,c) such that Decsk cMult(a,c) = a · m. [78] can also be used instead. Similarly, it is straightforward Labeled Homomorphic Encryption(labHE): Any LHE to replace LHE with the optimized HE scheme in [21] or scheme can be extended to a labHE scheme [12] with the help garbled circuits with the ABY framework [35]. of a pseudo-random function. In addition to the operations Yet another alternative implementation for Cryptϵ could supported by an LHE scheme, labHE supports multiplication be where the private database is equally shared between the of two labHE ciphers (details in Appendix A ). two servers and they engage in a secret share-based SMC protocol for executing the DP programs. This would require Garbled Circuit: Garbled circuit [73, 103] is a generic method both the servers to do almost equal amount of work for for secure computation. Two data owners with respective each program. Such an implementation would be justified private inputs x1 and x2 run the protocol such that, no data only if both the servers are equally invested in learning the owner learns more than f (x1, x2) for a function f . One of DP statistics and is ill-suited for our context. A real world the data owners, called generator, builds a "garbled" version analogy for this can be if Google and Baidu decide to compute of a circuit for f and sends it to the other data owner, called some statistics on their combined user bases. evaluator, alongside the garbled input values for x1. The evaluator, then, obtains the garbled input for x2 from the 3 BACKGROUND generator via oblivious transfer and computes f (x1, x2). 3.1 Differential Privacy Definition 1. An algorithm A satisfies ϵ-differential pri- 4 CRYPTϵ SYSTEM DESCRIPTION vacy (ϵ-DP), where ϵ > 0 is a privacy parameter, iff for any In this section, we describe Cryptϵ’s workflow (Section 4.1), two neighboring datasets D and D′ such that D = D′ − t or modules (Section 4.2), and trust assumptions (Section 4.3). ′ D = D − t, we have 4.1 Cryptϵ Workflow S ⊂ Ranдe(A), Pr A(D) ∈ S ≤ eϵ Pr A(D′) ∈ S (1) ∀ Cryptϵ operates in three phases: The above definition is sometimes called unbounded DP. (1) Setup Phase: At the outset, data owners initialize the B A variant is bounded-DP where neighboring datasets D and CSP with a privacy budget, ϵ , which is stored in its Privacy D′ have the same number of rows and differ in one row. Any Engine module. Next, the CSP’s Key Manager module gener- ϵ-DP algorithm also satisfies 2ϵ-bounded DP [72]. ates key pair (sk,pk) for labHE, publishes pk and stores sk. (2) Data Collection Phase: In the next phase, each data Theorem 1. (Sequential Composition) If A1 and A2 are ϵ1- owner encodes and encrypts his/her record using the Data DP and ϵ2-DP algorithms with independent randomness, then Encoder and Data Encryption modules and sends the en- releasing A1(D) and A2(D) on database D satisfies ϵ1 +ϵ2-DP. Theorem 2. (Post-Processing) Let A : D 7→ R be a random- crypted data records to the AS. The data owners are relieved ized algorithm that is ϵ-DP. Let f : R 7→ R′ be an arbitrary of all other duties and can go completely offline. The Aggre- randomized mapping. Then f ◦ A : D 7→ R′ is ϵ- DP. gator module of the AS, then, aggregates these encrypted records into a single encrypted database, D˜ . 3.2 Cryptographic Primitives (3) Program Execution Phase: In this phase, the AS exe- cutes a Cryptϵ program provided by the data analyst. Cryptϵ Linearly Homomorphic Encryption (LHE): If (M, +) is programs (details in Sections 5 and 6) access the sensitive data a finite group, an LHE scheme for messages in M is: via a restricted set of transformation operators, that filter, • Key Generation (Gen): This algorithm takes the security count or group the data, and measurement operators, which parameter κ as input and outputs a pair of secret and are DP operations to release noisy answers. Measurement public keys, ⟨sk ,pk ⟩ ← Gen(κ). operators need interactions with the CSP as they require (1) • Encryption (Enc): This is a randomized algorithm that decryption of the answer, and (2) a check that the privacy encrypts a message, m ∈ M, using the public key, pk , to budget is not exceeded. These functionalities are achieved generate the ciphertext, c ← Encpk (m). by CSP’s Data Decryption and Privacy Engine modules. The Setup and Data Collection phases occur just once at rds, D˜ i , from each of the data owners, DOi , and collates them the very beginning, every subsequent program is handled into a single encrypted database, D˜ . via the corresponding Program Execution phase. (2) Program Executor: This module inputs a logical Cryptϵ program, P, and privacy parameter, ϵ, from the data analyst, 4.2 Cryptϵ Modules translates P to the implementation specific secure protocol and computes the noisy output with the CSP’s help. Cryptographic Service Provider (CSP) (1) Key Manager: The Key Manager module initializes the 4.3 Trust Model ⟨ ⟩ labHE scheme for Cryptϵ by generating its key pair, sk,pk . There are three differences in Cryptϵ from the LDP setting: It stores the secret key, sk, with itself and releases the public (1) Semi-honest Model: We assume that the AS and the key, pk. The CSP has exclusive access to the secret key, sk, CSP are semi-honest, i.e., they follow the protocol honestly, and is the only entity capable of decryption in Cryptϵ. but their contents and computations can be observed by an (2) Privacy Engine: Cryptϵ starts off with a total privacy adversary. Additionally, each data owner has a private chan- budget of ϵB chosen by the data owners. The choice of value B nel with the AS. For real world scenarios, the semi-honest for ϵ should be guided by social prerogatives [7, 61, 69] behaviour can be imposed via legal bindings. Specifically, and is currently outside the scope of Cryptϵ. For executing both the AS and the CSP can swear to their semi-honest any program, the AS has to interact with the CSP at least behavior in legal affidavits; there would be loss of facein once (for decrypting the noisy answer), thereby allowing the public and legal implications in case of breach of conduct. CSP to monitor the AS’s actions in terms of privacy budget (2) Non-Collusion: We assume that the AS and the CSP are expenditure. The Privacy Engine module gets the program, P, non-colluding, i.e., they avoid revealing information [63] to and its allocated privacy budget, ϵ, from the data analyst, and each other beyond what is allowed by the protocol definition. maintains a public ledger that records the privacy budget This restriction can be imposed via strict legal bindings as spent in executing each such program. Once the privacy cost B well. Additionally, in our setting the CSP is a third-party incurred reaches ϵ , the CSP refuses to decrypt any further entity with no vested interested in learning the program answers. This ensures that the total privacy budget is never outputs. Hence, the CSP has little incentive to collude with exceeded. The ledger is completely public allowing any data the AS. Physical enforcement of the non-collusion condition owner to verify it. can be done by implementing the CSP inside a trusted exe- (3) Data Decryption: The CSP being the only entity capable cution environment (TEE) or via techniques which involve of decryption, any measurement of the data (even noisy) has using a trusted mediator who monitors the communications to involve the CSP. The Data Decryption module is tasked between the servers [11]. with handling all such interactions with the AS. (3) Computational Boundedness: The adversary is compu- Data Owners (DO) tationally bounded. Hence, the DP guarantee obtained is that (1) Data Encoder: Each data owner, DOi ,i ∈ [m], has a pri- of computational differential privacy or SIM-CDP79 [ ]. There vate data record, Di , of the form ⟨A1,...Al ⟩ where Aj is an is a separation between the algorithmic power of computa- attribute. At the very outset, every data owner, DOi , repre- tional DP and information-theoretic DP in the multi-party sents his/her private record, Di , in its respective per attribute setting [79]. Hence, this assumption is inevitable in Cryptϵ. one-hot-encoding format. The one-hot-encoding is a way of representation for categorical attributes and is illustrated 5 CRYPTϵ OPERATORS by the following example. If the database schema is given D˜ by ⟨Aдe,Gender⟩, then the corresponding one-hot-encoding Let us consider an encrypted instance of a database, , with schema ⟨A1,..., Al ⟩. In this section, we define the Cryptϵ representation for a data owner, DOi ,i ∈ [m], with the record operators (summarized in Table 1) and illustrate how to write ⟨30, Male⟩, is given by D˜i = ⟨[0,..., 0, 1, 0,..., 0], [1, 0]⟩. | {z } | {z } logical Cryptϵ programs for DP algorithms on D˜ . The design 29 70 (2) Data Encryption: The Data Encryption module stores of Cryptϵ operators are inspired by previous work [77, 106]. the public key pk of labHE which is announced by the CSP. Each data owner, DOi ,i ∈ [m], performs an element-wise 5.1 Transformation operators encryption of his/her per attribute one-hot-encodings using Transformation operators input encrypted data and output a ˜ pk and sends the encrypted record, Di, to the AS via a secure transformed encrypted data. These operators thus work com- channel. This is the only interaction that a data owner ever pletely on the encrypted data without expending any privacy participates in and goes offline after this. budget. Three types of data are considered in this context: Analytics Server (AS) (1) an encrypted table, T˜ , of x rows and y columns/attributes (1) Aggregator: The Aggregator collects the encrypted reco- where each attribute value is represented by its encrypted Table 1: Cryptϵ Operators

Types Name Notation Input Output Functionality ′ ′ Generates a new attribute A (in one-hot-coding) to represent ′ ˜ ˜ CrossProduct ×Ai,Aj →A (·) T T the data for both the attributes Ai and Aj ′ ∗ Project πA∗ (·) T˜ T˜ Discards all attributes but A ′ Transformation Filter σϕ (·) T˜ B Zeros out records not satisfying ϕ in B Count count(·) T˜ c Counts the number of 1s in B count (·) ˜ GroupByCount γA T V Returns encrypted histogram of A count (·) ˜ ˜ GroupByCountEncoded γ˜A T V Returns encrypted histogram of A in one-hot-encoding CountDistinct countD(·) V c Counts the number of non-zero values in V Laplace Lap (·) V \c Vˆ Adds Laplace noise to V Measurement ϵ,∆ k (·) Pˆ NoisyMax NoisyMaxϵ,∆ V Returns indices of the top k noisy values

′ one-hot-encoding; (2) an encrypted vector, V ; and (3) an en- bits and outputs the table, T˜ , with the updated indicator crypted scalar, c. In addition, every encrypted table, T˜ , of vector. x rows has an encrypted bit vector, B, of size x to indicate (4) Count count(T˜ ): This operator simply counts the number whether the rows are relevant to the program at hand. The of rows in T˜ that are pertinent to the program at hand, i.e. ˜ i-th row in T will be used for answering the current program the number of 1s in its associated bit vector B. This operator only if the i-th bit value of B is 1. The input to the first trans- outputs an encrypted scalar, c. formation operator in Cryptϵ program is D˜ with all bits of (5) GroupByCount γ count (T˜ ): The GroupByCount opera- B set to 1. For brevity, we use just T˜ to represent both the A tor partitions the input table, T˜ , into groups of rows having encrypted table, T˜ , and B. The transformation operators are: the same value for an attribute, A. The output of this trans- × ′ ( ˜ ) (1) CrossProduct (Ai,Aj )→A T : This operator transforms formation is an encrypted vector, V, that counts the number the two encrypted one-hot-encodings for attributes Ai and of unfiltered rows for each value of A. This operator serves ˜ Aj in T into a single encrypted one-hot-encoding of a new as a preceding transformation for other Cryptϵ operators ′ ′ attribute, A . The domain of the new attribute, A , is the cross specifically, NoisyMax, CountDistinct and Laplace. product of the domains for A and A . The resulting table, i j (6) GroupByCountEncoded γ˜count (T˜ ): This operator is ˜ ′ ˜ A T , has one column less than T . Thus, the construction of similar to GroupByCount. The only difference between the the one-hot-encoding of the entire y-dimensional domain two is that GroupByCountEncoded outputs a new table that can be computed by repeated application of this operator. has two columns – the first column corresponds to A and the ˜ ˜ (2) Project πA¯(T ): This operator projects T on a subset of second column corresponds to the number of rows for every attributes, A¯, of the input table. All the attributes that are value of A (in one-hot-encoding). This operator is useful for ′ not in A¯ are discarded from the output table T˜ . expressing computations of the form “count the number of age values having at least 200 records" (see P7 in Table 2). (3) Filter σϕ (T˜ ): This operator specifies a filtering condition that is represented by a Boolean predicate, ϕ, and defined (7) CountDistinct countD(V): This operator is always pre- over a subset of attributes, A¯, of the input table, T˜ . The pred- ceded by GroupByCount. Hence the input vector, V, is an icate can be expressed as a conjunction of range conditions encrypted histogram for attribute, A, and this operator re- ¯ ˜ Ó turns the number of distinct values of A that appear in D˜ by over A, i.e., for a row r ∈ T , ϕ(r) = A ∈A¯ (r.Ai ∈ VAi ), i counting the non-zero entries of V . where r.Ai is value of attribute Ai in row r and VA is a subset of values (can be a singleton) that Ai can take. For example, Aдe ∈ [30, 40] ∧ Gender = M can be a filtering condition. 5.2 Measurement operators The Filter operator affects only the associated encrypted The measurement operators take encrypted vector of counts, bit vector of T˜ and keeps the actual table untouched. If any V (or a single count, c), as input and return noisy measure- r ∈ T˜ ϕ row, , does not satisfy the filtering condition, , the ments on it in the clear. These two operators correspond to B labEnc ( ) corresponding bit in will be set to pk 0 ; otherwise, two classic DP mechanisms – Laplace mechanism and Noisy- B the corresponding bit value in is kept unchanged. Thus Max [38]. Both mechanisms add Laplace noise, η, scaled the Filter transformation suppresses all the records that are according to the transformations applied to D˜ . extraneous to answering the program at hand (i.e., does not Let the sequence of transformations applied on D˜ to satisfy ϕ) by explicitly zeroing the corresponding indicator get V be T(¯ D) = Tl (···T2((T1(D)))). The sensitivity of a sequence of transformations is defined as the maximum change to the output of this sequence of transformations 6 IMPLEMENTATION [77] when changing a row in the input database, i.e., ∆T¯ = In this section, we describe the implementation of Cryptϵ. ¯ ¯ ′ ′ maxD,D′ ∥T(D) − T(D )∥1 where D and D differ in a single First, we discuss our proposed technique for extending the ¯ row. The sensitivity of T can be upper bounded by the prod- multiplication operation of labHE to support n > 2 multi- uct of the stability [77] of these transformation operators, plicands which will be used for the CrossProduct operator. = Îl T i.e., ∆T¯ =(Tl , ..., T1) i=1 ∆ i . The transformations in Table 1 Then, we describe the implementations of Cryptϵ operators. have a stability of 1, except for GroupByCount and Group- ByCountEncoded which are 2-stable. Given ϵ and ∆T¯ , we 6.1 General labHE n-way Multiplication define the measurement operators: The labHE scheme is an extension of a LHE scheme where (1) Laplace Lapϵ,∆(V/c): This operator implements the clas- every ciphertext is now associated with a “label” [12]. This sic Laplace mechanism [38]. Given an encrypted vector, V , or extension enables labHE to support multiplication of two an encrypted scalar, c, a privacy parameter ϵ and sensitivity labHE ciphertexts via the labMult() operator (without in- ∆ of the preceding transformations, the operator adds noise volving the CSP). However, it cannot support multiplication 2∆ drawn from Lap( ϵ ) to V or c and outputs the noisy answer. of more than two ciphertexts because the“multiplication” ciphertext e = labMult(c , c ),(c and c are labHE cipher- (2) NoisyMax NoisyMaxk (V): Noisy-Max is a differen- 1 2 1 2 ϵ,∆ texts) does not have a corresponding label, i.e., it is not in tially private selection mechanism [38, 47] to determine the the correct labHE ciphertext format. Hence, we propose an top k highest valued queries. This operator takes in an en- algorithm дenLabMult to generate a label for every interme- crypted vector V and adds independent Laplace noise from diary product of two multiplicands to enable generic n-way Lap( 2k∆ ) to each count. The indices for the top k noisy val- ϵ multiplication (details are in theAlgorithm 1) . ues, Pˆ , are reported as the desired answer. 5.3 Program Examples 6.2 Operator Implementation A Cryptϵ program is a sequence of transformation oper- We now summarize how Cryptϵ operators are translated to ators followed by a measurement operator and arbitrary protocols that the AS and CSP can run on encrypted data. ⟨ ˜ post-processing. Consider a database schema Aдe, Gender, Project πA¯(T ): The implementation of this operator simply NativeCountry, Race⟩. We show 7 Cryptϵ program exam- drops off all but the attributes in A¯ from the input table, T˜ , ′ ples in Table 2 over this database. and returns the truncated table, T˜ . We will use P1 in Table 2 to illustrate how a Cryptϵ pro- Filter σ (T˜ ): Let ϕ be a predicate of the form r.A ∈ V . gram can be written and analyzed. Program P1 aims to com- ϕ j Aj Row i satisfies the filter if one of the bits corresponding to pute the cumulative distribution function (c.d.f.) of attribute positions inVA is 1. Thus, the bit corresponding to rowi is set Aдe with domain [1, 100]. The first step is to compute 100 j É as: B[i] = labMult(B[i], ∈ v˜ j [l]). The multi-attribute range queries, where the i-th query computes the the number l VAj of records in D˜ having Aдe ∈ [1,i] with privacy parame- implementation is detailed in Appendix D.2 . ϵ ′ ˜ ter i . The sequence of transformation operators for each CrossProduct ×Ai,Aj →A (T ): The crossproduct between two range query is count(σAдe ∈[1,i](πAдe (D˜ ))). All these three attributes are computed usingдenLabMult() described above. operators are 1-stable and hence, the sensitivity of the result- Count count(T˜ ): This operator simply adds up the bits in B ing range query is upper bounded by the product of these corresponding to input table T˜ , i.e., É B[i]. stability values, 1 [77]. Thus, the subsequent measurement i GroupByCountγ count (T˜ ): The implementations for Project, operator Laplace for the i-th range query takes in privacy A budget ϵi and sensitivity ∆ = 1, and outputs a noisy plain- Filter and Count are reused here. First, Cryptϵ projects the Í100 ˜ ˜ ( ˜ ) text count, cˆi . At this stage the program is i=1 ϵi /2-DP input table T on attribute A, i.e. T 1 = πA T . Then, Cryptϵ by Theorem 1[38] (recall we add noise from Lap(2 · ∆/ϵi ) loops each possible value of A. For each value v, Cryptϵ ini- ′ in Section 5.2). After looping over the 100 ranges, P1 ob- tializes a temporary Bv = B and filters T˜ on A = v to get an ˆ ′ ′ tains a noisy plaintext output V = [cˆ1,...,cˆ100] and applies updated Bv . Finally, Cryptϵ outputs the number of 1s in Bv . ˆ count ( ˜ ) a post-processing step, denoted by postc .d .f (V ). This oper- GroupByCountEncoded γ˜A T : For this operator, the ator inputs a noisy histogram, Vˆ , for attribute A and com- AS first uses GroupByCount to generate the encrypted his- ˆ ′ ′ ′ putes its c.d.f V = [cˆ1,...,cˆ100] via isotonic regression [57] togram, V , for attribute A. Since each entry of V is a count of ˆ ′ ˆ ′ ′ ˜ ˜ minVˆ ′ ∥V − V ∥2 s.t. 0 ≤ cˆ1 ≤ · · · ≤ cˆ100 ≤ |D|. Hence, by rows, its value ranges from {0,..., |T |}. The AS, then, masks Í100 Theorem 2, P1 is ϵ/2-DP, where ϵ = i=1 ϵi . However, since V and sends it to the CSP. The purpose of this mask is to hide Cryptϵ also reveals the total dataset size, the total privacy the true histogram from the CSP. Next, the CSP generates guarantee is ϵ-bounded DP (see Section 7 for details). the encrypted one-hot-coding representation for this masked Table 2: Examples of Cryptϵ Program Cryptϵ Program Description ˜ P1: i ∈ [1, 100],cˆi ← Lapϵi,1(count(σAдe ∈(0,i](πAдe (D)))); postc .d .f ([cˆ1,...,cˆ100]) Outputs the c.d.f of Aдe with domain [1, 100]. ∀ˆ ← 5 ( count (D˜ )) P2: P NoisyMaxϵ,1 γAдe Outputs the 5 most frequent age values. ˆ ← ( count ( (× (D˜ )))) P3: V Lapϵ,2 γRace×Gender πRace×Gender Race,Gender →Race×Gender Outputs the marginal over the attributes Race and Gender. ˆ ← ( count ( ( (× (D˜ ))))) P4: V Lapϵ,2 γAдe×Gender σN ativeCountry=Mexico πAдe×Gender,N ativeCountry Aдe,Gender →Aдe×Gender Outputs the marginal over Aдe and Gender for Mexican employees. P5: cˆ ← Lapϵ,1(count(σAдe=30∧Gender =Male∧N ativeCountry=Mexico(πAдe,Gender,N ativeCountry (D˜ )))) Counts the number of male employees of Mexico having age 30. ← ( ( count ( ( (D˜ ))))) P6: cˆ Lapϵ,2 countD γAдe σGender =Male πAдe,Gender Counts the number of distinct age values for the male employees. ← ( ( ( count ( (D˜ ))))) P7: cˆ Lapϵ,2 count σCount ∈[200,m] γ˜Aдe πAдe Counts the number of age values having at least 200 records. histogram V˜ and returns it to the AS. The AS can simply the number of records in the dataset, D. Let PCDP (D, ϵ/2) rotate V˜ [i],i ∈ [|V |] by its respective mask value M[i] and denote the random variable corresponding to the output of get back the true encrypted histogram in one-hot-coding V˜ . running P in the CDP model under ϵ/2-DP (Definition 1). The details are presented in in in Algorithm 2 Appendix D.2 We make the following claims: . • The views and outputs of the AS and CSP are computation- CountDistinct countD(V ): This operator is implemented ally indistinguishable from that of simulators with access by a garbled circuit (details are in Appendix D.2 ). to only PCDP (D, ϵ/2) and the total dataset size |D|.

Laplace Lapϵ,∆(V \c): The Laplace operator has two phases • For every P that satisfies ϵ/2-DP (Definition 1), revealing (since both the AS and the CSP adds Laplace noise). In the its output (distributed identical to PCDP (D, ϵ/2)) as well as first phase, the AS adds an instance of encrypted Laplace |D| satisfies ϵ-bounded DP, where neighboring databases 2∆ have the same size but differ in one row. noise, η1 ∼ Lap( ϵ ), to the encrypted input to generate ˆc. Ít In the second phase, the CSP first checks whether i=1 ϵi + • Thus, the overall protocol satisfies computational differ- B ϵ ≤ ϵ where ϵi represents the privacy budget used for a ential privacy under the SIM-CDP model. previously executed program, Pi (presuming a total of t ∈ N CDP (D ) Now, let PB , ϵ denote the random variable correspond- programs have been executed hitherto the details of which ing to the output of running P in the CDP model under ϵ- are logged into the CSP’s public ledger). Only in the event CDP (D ) ≡ ( CDP (D / ) |D|) bounded DP such that PB , ϵ P , ϵ 2 , . the above check is satisfied, the CSP proceeds to decrypt ˆc, We state the main theorems here and refer the reader to the and records ϵ and the current program details (description, Appendix A for formal proofs. sensitivity) in the public ledger. Next, the CSP adds a second Theorem 3. Let protocol Π correspond to the execution 2∆ instance of the Laplace noise, η2 ∼ Lap( ϵ ), to generate the of program P in Cryptϵ. The views and outputs of the AS and final noisy output, cˆ, in the clear. The Laplace operator with Π Π the CSP are denoted as View1 (P, D, ϵ),Output1 (P, D, ϵ) an encrypted scalar, V , as the input is implemented similarly. Π Π andView2 (P, D, ϵ),Output2 (P, D, ϵ) respectively. There ex- k ( ) NoisyMax NoisyMaxϵ,∆ V : This operator is implemented ists Probabilistic Polynomial Time (PPT) simulators, Sim1 via a two-party computation between the AS and the CSP and Sim2, such that: using garbled circuits (details are in Appendix D.2 ). ( CDP (D )) • Sim1 PB , ϵ is computationally indistinguishable (≡c ) Π Π Note: Cryptϵ programs are grouped into three classes based from (View1 (P, D, ϵ),Output (P, D, ϵ)), and on the number and type of interaction between the AS and ( CDP (D )) ( Π( D ) Π( D )) • Sim2 PB , ϵ is ≡c to View2 P, , ϵ ,Output P, , ϵ . the CSP. For example, P1, P2 and P3 (Table 2) have just one Output Π(P, D, ϵ)) is the combined output of the two parties1. interaction with the CSP for decrypting the noisy output. P4 The main ingredient for the proof is the composition the- and P5, on the other hand, require additional interactions д for the n-way multiplication of ciphers in the CrossProduct orem [86], which informally states: suppose a protocol, Πf , operator. Finally, P6 and P7 require intermediate intercations implements functionality f and uses function д as an oracle due to operators CountDistinct and GroupByCountEncoded (uses only input-output behavior of д). Assume that proto- д respectively. The details are presented in Appendix E . col Πд implements д and calls to д in Πf are replaced by instances of Πд (referred to as the composite protocol). If Πf 7 CRYPTϵ SECURITY SKETCH and Πд are correct (satisfy the above simulator definition), In this section, we provide a sketch of the security proof in then the composite protocol is correct. Thus, the proof can the semi-honest model using the well established simulation be done in a modular fashion as long as the underlying oper- argument [86]. Cryptϵ takes as input a DP program, P, and ators are used in a blackbox manner (only the input-output a privacy parameter, ϵ, and translates P into a protocol, Π, behavior are used and none of the internal state are used). 1 CDP (D )) which in turn is executed by the AS and the CSP. In addition Note that the simulators are passed a random variable PB , ϵ , i.e., to revealing the output of the program P, Π also reveals the simulator is given the ability to sample from this distribution. Next, every Cryptϵ program expressed as a sequence of on A (4) re-encrypt the sorted database using pk and out- transformation operators followed by a measurement opera- puts D˜ s = labEncpk (sort(D)). The mapping, F , must be tor, satisfies ϵ/2-DP (as in Definition 1). It is so because recall learned under DP, and we present a method for that below. 2∆ that the measurement operators add noise from Lap( ϵ ) (Sec- Let P = (P1,..., Pk ) be an equi-width partition on the sorted sA tion 5.2) where ∆ denotes the sensitivity of P (computed w.r.t domain of A such that each partition (bin) contains k con- to Definition 1)38 [ , 47]. However, Cryptϵ reveals both the secutive domain values where sA is the domain size of A. The output of the program as well as the total size of dataset D. index is constructed using a Cryptϵ program that firstly com- Í While revealing the size exactly would violate Definition 1, putes the noisy prefix counts, Vˆ [i] = ∈∪i ctA,v + ηi v l=1Pl it does satisfy bounded-DP albeit with twice the privacy pa- for i ∈ [k], where ηi ∼ Lap(2k/ϵA) and ctA,v denotes the rameter, ϵ – changing a row in D is equivalent to adding a number of rows with value v for A. Next, the program uses row and then removing a row. isotonic regression [57] on Vˆ to generate a noisy cumulative Finally, since every program P executed on Cryptϵ sat- histogram C˜ with non-decreasing counts. Thus, each prefix isfies ϵ-bounded DP, it follows from Theorem 3 that every count in C˜ gives an approximate index for the sorted data- execution of Cryptϵ satisfies computational DP. base where the values of attribute A change from being in Corollary 4. Protocol Π satisfies computational differen- Pi to a value in Pi+1. When a Cryptϵ program starts with tial privacy under the SIM-CDP notion [79]. a filter ϕ = A ∈ [vs ,ve ], we compute two indices for the sorted database, is and ie , as follows. Let vs and ve fall in Note that Theorem 3 assumes that AS and the CSP do partitions Pi and Pj respectively. If Pi is the first partition, not collude with the users (data owners). However, if the AS then we set is = 0; otherwise set is to be 1 more than the colludes with a subset of the users, U , then Sim1 (Sim2) has i − 1-th noisy prefix count from C˜. Similarly, if Pj is the last to be given the data corresponding to users in U as additional partition, then we set ie = |D˜ |; otherwise, we set ie to be parameters. This presents no complications in the proof (see the j + 1-th noisy prefix count from C˜. This gives us the DP the proof in [48]). If a new user u joins, their data can be mapping F . We then run the program on the subset of rows encrypted and simply added to the database. We discuss in [is ,ie ]. For example, in Figure 2, the indexing attribute extensions to handle malicious adversaries in Section 10. with domain {v1, ··· ,v10} has been partitioned into k = 5 bins and if ϕ ∈ [v3,v6], is = C[˜ 1] + 1 = 6 and ie = C[˜ 3] = 13. 8 CRYPTϵ OPTIMIZATIONS Lemma 5. Let P be the program that computes the mapping In this section, we present the optimizations used by Cryptϵ. F . Let Π be the Cryptϵ protocol corresponding to the construc- tion of the DP index. The views and outputs of the AS and 8.1 DP Index Optimization Π Π the CSP are denoted as View1 (P, D, ϵA), Output1 (P, D, ϵA) This optimization is motivated by the fact that several pro- Π Π and View2 (P, D, ϵA), Output2 (P, D, ϵA) respectively. There grams, first, filter out a large number of rows in the dataset. exists PPT simulators Sim1 and Sim2 such that: For instance, P5 in Table 2 constructs a histogram over Aдe • Sim (PCDP (D, ϵ )) ≡ (ViewΠ(P, D, ϵ ),Output Π(D, ϵ )), and and Gender on the subset of rows for which NativeCountry 1 B A c 1 A A • ( CDP (D )) ≡ ( Π( D ) Π(D )) is Mexico. Cryptϵ’s filter implementation retains all the rows Sim2 PB , ϵ c View2 P, , ϵA ,Output , ϵA . Π as the AS has no way of telling whether the filter condition Output (P, D, ϵA)) is the combined output of the two parties is satisfied. As a result, the subsequent GroupbyCount is run The proof of the above lemma is presented in Appendix on the full dataset. If there were an index on NativeCountry, D.3. Here we present the intuition behind it. From the secure Cryptϵ could run the GroupbyCount on only the subset sorting algorithm (steps 1 and 4), it is evident that the servers of rows with NativeCountry=Mexico. But an exact index cannot associate the records of the encrypted sorted dataset, would violate DP. Hence, we propose a DP index to bound D˜ , with the data owners. The AS can learn nothing from D˜ the information leakage while improving the performance. s s due to the semantic security of the LHE scheme used. This At a high-level, the DP index on any ordinal attribute A is ensures that the DP index construction of Cryptϵ satisfies constructed as follows: (1) securely sort the input encrypted the SIM-CDP privacy guarantee. database, D˜ , on A and (2) learn a mapping, F , from the domain of A to [1, |D˜ |] such that most of the rows with in- • Optimized feature: This optimization speeds up the pro- dex less than F(v),v ∈ domain(A), have a value less than v. gram execution by reducing the total number of rows to The secure sorting is done via the following garbled circuit be processed for the program. that (1) inputs D˜ (just the records without any identifying • Trade-off: The trade-off is a possible increase in error as features) and indexing attribute A from the AS (2) inputs some of the rows that satisfy the filter condition may not the secret key sk from the CSP (3) decrypts and sort D be selected due to the noisy index. 8.2 Crypto-Engineering Optimizations (1) DP Range Tree: If range queries are common, pre-compu- ted noisy range tree is a useful optimization. For example, building a range tree on Aдe attribute can improve the ac- Figure 2: Illustrative example for DP Index curacy for P1 and P2 in Table 2. The sensitivity for such a noisy range tree is logsA where sA is the domain size of the attribute on which the tree is constructed. Any arbitrary • Privacy Cost: Assuming the index is constructed with range query requires access to at most 2 logsA nodes on the privacy parameter ϵA, the selection of a subset of rows tree. Thus to answer all possible range queries on A, the total 2 2 using it will be ϵA-bounded DP (Lemma 5). If ϵL is the pa- s (log sA) squared error accumulated is O( ϵ ). In contrast for the rameter used for the subsequent measurement primitives, s3 O( A ) then by Theorem 1, the total privacy parameter is ϵA + ϵL. naive case, we would have incurred error ϵ [57]. Note that, if we already have a DP index on A, then the DP range Discussion: Here we discuss the various parameters in the tree can be considered to be a secondary index on A. construction of a DP index. The foremost parameter is the • Optimized Feature: The optimization reduces both exe- indexing attribute A which can be chosen with the help of cution time and expected error when executed over multi- the following two heuristics. First, A should be frequently ple range queries. queried so that a large number of queries can benefit from this optimization. Second, choose A such that the selectiv- • Trade-off: The trade-off for this optimization is the stor- ity of the popularly queried values of A is high. This would age cost of the range tree (O(2 · sA)). ensure that the first selection performed alone on A will fil- • Privacy Cost: If the range tree is constructed with pri- ter out the majority of the rows, reducing the intermediate vacy parameter ϵR , then any measurement on it is post- dataset size to be considered for the subsequent operators. processing. Hence, the privacy cost is ϵR -bounded DP. The next parameter is the fraction of the program privacy (2) Precomputation: The CrossProduct primitive generates budget, ρ (ϵ = ρ · ϵ where ϵ is the total program privacy A the one-hot-coding of data across two attributes. However, budget) that should be used towards building the index. The this step is costly due to the intermediate interactions with higher the value of ρ, the better is the accuracy of the in- the CSP. Hence, a useful optimization is to pre-compute the dex (hence better speed-up). However, the privacy budget one-hot-codings for the data across a set of frequently used allocated for the rest of the program decreases resulting in attributes A¯ so that for subsequent program executions, the increased noise in the final answer. This trade-off is studied AS can get the desired representation via simple look-ups. in Figures 4a and 4b in Section 9. Another parameter is the For example, this benefits P3 (Table 2). number of bins k. Finer binning gives more resolution but leads to more error due to DP noise addition. Coarser binning • Optimized Feature: This reduces the execution time of introduces error in indexing but has lower error due to noise. Cryptϵ programs. The multi-attribute one-hot-codings can We explore this trade-off in Figures 4c and 4d. To increase be re-used for all subsequent programs. accuracy we can also consider bins preceding is and bins • Trade-off: The trade-off is the storage costm (O( · sA¯ = Î succeeding ie . This is so because, since the index is noisy, it m · A∈A¯ sA), m = the number of data owners) incurred might miss out on some rows that satisfy the filter condition. to store the multi-attribute one-hot-codings for A¯. ˜ For example, in Figure 2, both the indices is = C[1] + 1 = 6 • Privacy Cost: The computation is carried completely on ˜ and ie = C[3] = 13 miss a row satisfying the filter condition the encrypted data, no privacy budget is expended. ϕ = A ∈ [v3,v6]; hence including an extra neighboring bin would reduce the error. (3) Offline Processing: For GroupByCountEncoded, the Thus, in order to gain in performance, the proposed DP in- CSP needs to generate the encrypted one-hot-codings for dex optimization allows some DP leakage of the data. This is the masked histogram. Note that the one-hot-encoding rep- in tune with the works in [27, 52, 59, 76]. However, our work resentation for any such count would simply be a vector of ˜ differs from earlier work in the fact that we can achieve pure (|D| − 1) ciphertexts for ‘0’, labEncpk (0) and 1 ciphertext for DP (albeit SIM-CDP). In contrast, previous work achieved ‘1’, labEncpk (1). Thus one useful optimization is to gener- a weaker version of DP, approximate DP [18], and added ate these ciphertexts offline (similar to offline generation of one-sided noise (i.e., only positive noise). One-sided noise Beaver’s multiplication triples [15] used in SMC). Hence, the requires addition of dummy rows in the data, and hence in- program execution will not be blocked by encryption. creases the data size. However, in our Cryptϵ programs, all • Optimized Feature: This optimization results in a reduc- the rows in the noisy set are part of the real dataset. tion in the run time of Cryptϵ programs. • Trade-off: A storage cost of O(m · sA) is incurred to store Configuration: We implemented Cryptϵ in Python with the the ciphers for attribute A. garbled circuit implemented via EMP toolkit [2]. We use Pail- • Privacy Cost: The computation is carried completely on lier encryption scheme [87]. All the experiments have been the encrypted data, no privacy budget is expended. performed on the Google Cloud Platform [1] with the config- uration c2-standard-8. For Adult dataset, Cryptϵ constructs 9 EXPERIMENTAL EVALUATION a DP index optimization over the attribute NativeCountry that benefits programs like P4 and P5. Our experiments as- In this section, we describe our evaluation of Cryptϵ along sign 20% of the total program privacy parameter towards two dimensions, accuracy and performance of Cryptϵ pro- constructing the index and the rest is used for the remaining grams. Specifically, we address the following questions: program execution. Cryptϵ also constructs a DP range tree • Q1: Do Cryptϵ programs have significantly lower errors over Aдe. This helps programs like P1, P2 and P3. This is our than that for the corresponding state-of-the-art LDP im- default Cryptϵ implementation. plementations? Additionally, is the accuracy of Cryptϵ programs comparable to that of the corresponding CDP 9.2 End-to-end Accuracy Comparison implementations? In this section, we evaluate Q1 by performing a comparative • Q2: Do the proposed optimizations provide substantial analysis between the empirical accuracy of the aforemen- performance improvement over unoptimized Cryptϵ? tioned four Cryptϵ programs (both optimized and unopti- • Q3: Are Cryptϵ programs practical in terms of their exe- mized) and that of the corresponding state-of-the-art LDP cution time and do they scale well? [95] and CDP (under bounded DP; specifically, using the CDP view Cryptϵ is computationally indistinguishable from Evaluation Highlights: as shown in Section 7) [38] implementations. • Cryptϵ can achieve up to 50× smaller error than the cor- The first observation with respect to accuracy is that the responding LDP implementation on a data of size ≈ 30K mean error for a single frequency count for Cryptϵ is at least (Figure 3). Additionally, Cryptϵ errors are at most 2× more 50× less than that of the corresponding LDP implementation. than that of the corresponding CDP implementation. For example, Figure 3b shows that for P3, ϵ = 0.1 results in a • The optimizations in Cryptϵ can improve the performance mean error of 599.7 as compared to an error of 34301.02 for of unoptimized Cryptϵ by up to 5667× (Table 3). the corresponding LDP implementation. Similarly, P5 (Figure • A large class of Cryptϵ programs execute within 3.6 hours 3c) gives a mean error of only 58.7 for ϵ = 0.1. In contrast, the for a dataset of size 106, and they scale linearly with the corresponding LDP implementation has an error of 3199.96. dataset size (Figure 5). The AS performs majority of the For P1 (c.d.f on Aдe), the mean error for Cryptϵ for ϵ = 0.1 is work for most programs (Table 3). given by 0.82 while the corresponding LDP implementation has an error of 9.2. The accuracy improvement on P7 (Figure 9.1 Methodology 3d) by Cryptϵ is less significant as compared to the other Programs: To answer the aforementioned questions, we ran programs, because P7 outputs the number of age values the experiments on the Cryptϵ programs previously outlined ([1 − 100]) having 200 records. At ϵ = 0.1, at least 52 age in Table 2. Due to space limitations, we present the results values out of 100 are reported incorrectly on whether their of only four of them in the main paper namely P1, P3, P5 counts pass the threshold. Cryptϵ reduces the error almost by and P7. The rationale behind choosing these four is that they half. Note that the additive error for a single√ frequency count cover all three classes of programs (Section 6) and showcase query in the LDP setting is at least Ω( n/ϵ), thus the error the advantages for all of the four proposed optimizations. increases with dataset size. On the other hand, for Cryptϵ the Dataset: We ran our experiments on the Adult dataset from error is of the order Θ(1/ϵ), hence with increasing dataset the UCI repository [6]. The dataset is of size 32, 651. For the size the relative the error improvement for Cryptϵ over that scaling experiments (Figure 5), we create toy datasets of sizes of an equivalent implementation in LDP would increase. 100K and 1 million by copying over the Adult dataset. For P1 (Figure 3a), we observe that the error of Cryptϵ Accuracy Metrics: Programs with scalar outputs (P5, P7) is around 5× less than that of the unoptimized implemen- use absolute error |c −cˆ| where c is the true count and cˆ is the tation. The reason is that P1 constructs the c.d.f over the noisy output. Programs with vector outputs (P1, P3) use the attribute Aдe (with domain size 100) by first executing 100 Í L1 error metric given by Error = i |V [i] − Vˆ [i]|,i ∈ [|V |] range queries. Thus, if the total privacy budget for the pro- where V is the true vector and Vˆ is the noisy vector. We gram is ϵ, then for unoptimized Cryptϵ, each query gets a ϵ report the mean and s.t.d of error values over 10 repetitions. privacy parameter of just 100 . In contrast, the DP range tree Performance Metrics: We report the mean total execution is constructed with the full budget ϵ and sensitivity ⌈log 100⌉ time in seconds for each program, over 10 repetitions. thereby resulting in lesser error. For P5 (Figure 3c) however, 105 104 60 1 Unoptimized Crypt 10 Crypt LDP 4 3 50 CDP 10 10

0 40 10 103 102 30 2 982.2 X Speed Up 1 10 10 41 X Speed Up 102 X Speed Up 1 5667 X Speed Up 10− 20

101 100 Mean Absolute10 Error Mean L1 Error in Log Scale 2 Mean L1 Error in Log Scale 10 0 1 − 10 10− 0 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 Mean Absolute Error in Log Scale 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 Privacy Parameter  Privacy Parameter  Privacy Parameter  Privacy Parameter  (a) Program 1 (b) Program 3 (c) Program 5 (d) Program 7 Figure 3: Accuracy Analysis of Cryptϵ Programs

Table 3: Execution Time Analysis for Cryptϵ Programs 20 35

33 Time in (s) Program 15

1 3 5 7 31 AS 1756.71 6888.23 650.78 290 10 29 Unoptimized Cryptϵ CSP 0.26 6764.64 550.34 30407.73 5 Total 1756.97 13652.87 1201.12 30697.73 27 Mean Absolute Error

Total 0.31 13.9 29.21 299.5 Mean Execution Time in s Cryptϵ 0 25 Speed Up × 5667.64 982.2 41.1 102.49 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 ρ = Fraction of Privacy Parameter used for Index ρ = Fraction of Privacy Parameter used for Index the unoptimized implementation has slightly better accuracy (a) (b) . × 20 80 (around 1 4 ) than Cryptϵ. It is because of two reasons; first, 70 16 the noisy index on NativeCountry might miss some of the 60 12 50 rows satisfying the filter condition (NativeCountry=Mexico). 40 8 30

20 Second, since only 0.8% of the total privacy parameter is bud- 4

Mean Absolute Error 10 geted for the Laplace operator in the optimized program 0 Mean Execution Time0 in s 2 4 8 10 20 40 2 4 8 10 20 40 execution, this results in a higher error as compared to that Total Number of Bins in CDF Total Number of Bins in CDF of unoptimized Cryptϵ. However, this is a small cost to pay (c) (d) for achieving a performance gain of 41×. The optimizations Figure 4: Accuracy and performance of P5 at different for P3 (Figure 3b) and P7 (Figure 3d) work completely on settings of the DP index optimization the encrypted data and do not expend the privacy budget. Hence they do not hurt the program accuracy in any way. uses ϵ = 2.2, ρ = 0.2, total 10 bins and considers no extra Another observation is that for frequency counts the error neighboring bin. of Cryptϵ is around 2× higher than that of the corresponding In Figure 4a and 4b we study how the mean error and CDP implementation. This is intuitive because we add two execution time of the final result varies with ρ for P5. From instances of Laplace noise in Cryptϵ (Section 6.2). For P1, the Figure 4a, we observe that the mean error drops sharply from CDP implementation also uses a range tree. ρ = 0.1 to ρ = 0.2, stabilises till ρ = 0.5, and starts increasing 9.3 Performance Gain From Optimizations again. This is because, at ρ = 0.2, the index correctly iden- tifies almost all the records satisfying the Filter condition. In this section, we evaluate Q2 (Table 3) by analyzing how However, as we keep increasing ρ, the privacy budget left for much speed-up is brought about by the proposed optimiza- the program after Filter (Laplace operator) keeps decreasing tions in the program execution time. resulting in higher error in the final answer. From Figure DP Index: For P5, we observe from Table 3 that the unopti- 4b, we observe that the execution time increases till ρ = 0.5 mized implementation takes around 20 minutes to run. How- and then stabilizes; the reason is that the number of rows ever, a DP index over the attribute NativeCountry reduces returned after ρ = 0.5 does not differ by much. the execution time to about 30s giving us a 41× speed-up. We plot the mean error and execution time for P5 by It is so because, only about 2% of the data records satisfy varying the total number of bins from 2 to 40 (domain size NativeCountry=Mexico. Thus the index drastically reduces of NativeCountry is 40) in Figure 4c and 4d respectively. the number of records to be processed for the program. From Figure 4c, we observe that the error of P5 increases Additionally, we study the dependency of the accuracy as the number of bins increase. It is so because from the and execution time of P5 implemented with the DP index computation of the prefix counts (Section 8.1), the amount of k on three parameters – (1) fraction of privacy budget ρ used noise added increases with k (as noise is drawn from Lap( ϵ )). for the index (2) total number of domain partitions (bins) Figure 4d shows that the execution time decreases with k. considered (3) number of neighboring bins considered. The This is intuitive because increase in k results in smaller bins, default configuration for Cryptϵ presented in this section hence the number of rows included in [is ,ie ] decreases. To avoid missing relevant rows, more bins that are adja- the CSP. The execution time for the P3 and P7 is dominated cent to the chosen range [is ,ie ] can be considered for the by the ⊕ operation for the GroupByCount operator. The subsequent operators. Thus, as the number of bins considered cost of ⊕ is linear to the data size. Hence, the execution time increases, the resulting error decreases at the cost of higher for P3 and P7 increases linearly with the data size. For P5, execution time. The experimental results are presented in the execution time depends on the % of the records in the Figure 7a and Figure 7b in Appendix F. dataset that satisfy the condition NativeCountry = Mexico DP Range Tree: For P1, we see from Table 3 that the total (roughly this many rows are retrieved from the noisy index). execution time of the unoptimized Cryptϵ implementation is about half an hour. However, using the range tree opti- 10 EXTENSION OF CRYPTϵ TO THE mization reduces the execution time by 5667×. The reason MALICIOUS MODEL behind this huge speed-up is that the time required by the AS In this section, we briefly discuss how to extend the cur- in the optimized implementation becomes almost negligible rent Cryptϵ system to account for malicious adversaries. because it simply needs to do a memory fetch to read off the We present one approach for the extension here and detail answer from the pre-computed range tree. another approach in Appendix C.The first approach imple- Pre-computation: For P3, the unoptimized execution time ments the CSP inside a trusted execution environment (TEE) on the dataset of 32561 records is around 4 hours (Table 3). [10, 20, 83]. This ensures non-collusion (as the CSP cannot This is so because the CrossProduct operator has to perform collude with the AS since its operations are vetted). The mea- 10 · 32561 labMult operations which is very time consuming. surement operators are implemented as follows (the privacy Hence, pre-computing the one-hot-codings for 2-D attribute budget over-expenditure checking remains unchanged from over Race and Gender is very useful; the execution time that in Section 6.2 and we skip re-describing it here). × reduces to less than a minute giving us a 982.2 speed up. Laplace Lapϵ,∆(V \c): The new implementation requires only Offline Processing: The most costly operator for P7 is the a single instance of noise addition by the CSP. The AS sends GroupByCountEncoded operator since the CSP has to gen- the ciphertext c to the CSP. The CSP decrypts the ciphertext, 2·∆ erate ≈ 3300K ciphertexts of 0 and 1 for the encrypted one- adds a copy of noise, η ∼ Lap( ϵ ), and sends it to the AS. k ( ) hot-codings. This results in a total execution time of about NoisyMax NoisyMaxϵ,∆ V : The new implementation works 8.5 hours in unoptimized Cryptϵ. However, by generating without the garbled circuit as follows. The AS sends the vec- the ciphertexts off-line, the execution time can be reduced tor of ciphertexts, V , to the CSP. The CSP computes V˜ [i] = to just 5 minutes giving us a speed up of 102.49×. labDecryptsk (V [i])+η[i],i ∈ [|V |], where η[i] ∼ Lap(2k∆/ϵ) Another important observation from Table 3 is that the and outputs the indices of the top k values of V˜ . AS performs the major chunk of the work for most program Malicious AS: Recall that a Cryptϵ program, P, consists of executions. This conforms with our discussion in Section 2.2. a series of transformation operators that transform the en- crypted database, D˜ , to a ciphertext, c (or an encrypted vec- 104 tor, V ). This is followed by applying a measurement operator 103 on c (or V ). Let P represent the first part of the program P up 102 1

Program 1 c P 1 to the computation of and let 2 represent the subsequent 10 Program 3 Program 5 Program 7 100 measurement operator (performed by the CSP inside a TEE). Time in seconds in Log Scale 1 In the malicious model, the AS is motivated to misbehave. For 10− 103 104 32561 105 106 ˜ Dataset size in Log Scale example, instead of submitting the correct cipher c = P1(D) ′ Figure 5: Scalability of Cryptϵ Programs the AS could run a different program P on the record of a single data owner only. Such malicious behaviour can be 9.4 Scalability prevented by having the CSP validate the AS’s work via In this section, we evaluate Q3 by observing the execu- zero knowledge proofs (ZKP) [86] as follows (similar proof tion times of the aforementioned four Cryptϵ programs for structure as prior work [84]). Specifically, the ZKP statement dataset sizes up to 1 million. As seen from Figure 5, the should prove that the AS 1) runs the correct program P1 2) longest execution time (P7) for a dataset of 1 million records on the correct dataset D˜ . For this, the CSP shares a random is ≈ 3.6 hours; this shows the practical scalability of Cryptϵ. one-time MAC key, mki ,i ∈ [m] with each of the data own- All the reported execution times are for default setting. For ers, DOi . Along with the encrypted record D˜ i , DOi sends a P1 we see that the the execution time does not change with Pedersen commitment [88] Comi to the one-time MAC [65] the dataset size. This is so because once the range tree is on D˜ i and a short ZKP that the opening of this commitment constructed, the program execution just involves reading the is a valid one-time MAC on D˜ i . The AS collects all the ci- answer directly from the tree followed by a decryption by phertexts and proofs from the data owners and computes c = P1(D˜ 1, ··· , D˜ m). Additionally, it constructs a ZKP that c Alternatively each data owner can provide a zero knowledge Í is indeed the output of executing P1 on D˜ = {D˜ 1, ··· , D˜ m } proof for j,k, Dij [k] ∈ {0, 1} ∧ k Dij [k] = 1. [24]. Formally, the proof statement is ∀

c = P1(D˜ 1, ··· , D˜ m) ∧ i Open(Comi ) = MACmk (D˜ i ) (2) 11 RELATED WORK ∀ i The AS submits the ciphertext c along with all the commit- Differential Privacy: Introduced by Dwork et al. in [38], ments and proofs to the CSP. By validating the proofs, the differential privacy has enjoyed immense attention from both CSP can guarantee c is indeed the desired ciphertext. The academia and industry in the last decade. Interesting work one-time MACs ensure that the AS did not modify or drop has been done in both the CDP model [8, 22, 30, 33, 34, 39, 54– any of the records received from the data owners. 56, 58, 70, 71, 75, 85, 89, 90, 100–102, 104, 105, 107] and the Efficient proof construction: Our setting suits that of des- LDP model [13, 32, 42, 44, 91, 95–97, 108]. Recently, it has ignated verifier non-interactive zero knowledge (DV NIZK) been showed that augmenting the LDP setting by a layer proofs [26]. In a DV NIZK setting, the proofs can be verified of anonymity improves the privacy guarantees [20, 31, 41]. by a single designated entity (as opposed to publicly verifi- It is important to note that the power of this new model able proofs [53]) who possesses some secret key for the NIZK (known as shuffler/mixnet model) lies strictly between that system. Thus in Cryptϵ, clearly the CSP can assume the role of LDP and CDP. Cryptϵ differs from this line of work in of the designated verifier. The framework for efficient DV three ways, namely expressibility, precise DP guarantee and NIZKs proposed by Chaidos and Couteau [26] can be ap- trust assumptions (details are in the Appendix G ). plied to prove Eq. (2), as this framework enables proving Two-Server Model: The two-server model is popularly used arbitrary relations between cryptographic primitives, such for privacy preserving machine learning approaches where as Pedersen commitment or Paillier encryption. A detailed one of the servers manages the cryptographic primitives construction is given in Appendix Cwhich shows that all while the other handles computation [46, 48, 67, 80, 83, 84]. the steps of the proof involve simple arithmetic operations Homomorphic Encryption: Recently, there has been a N 2 N modulo where is an RSA modulus. To get an idea of surge in privacy preserving solutions using homomorphic en- the execution overhead for the ZKPs, consider constructing cryptions due to improved primitives. A lot of the aforemen- a DV NIZK for proving that a Paillier ciphertext encrypts tioned two-server models employ homomorphic encryption the products of the plaintexts of two other ciphertexts (this [48, 67, 83, 84]. Additionally, it is used in [23, 25, 49, 50, 60]. could be useful for proving the validity of our Filter operator, for example). In [26], this involves 4loдN bits of communica- 12 CONCLUSIONS tion and the operations involve addition and multiplication of group elements. Each such operation takes order of 10−5 In this paper, we have proposed a system and programming seconds to execute, hence for proving the above statement framework, Cryptϵ, for differential privacy that achieves the for 1 million ciphertexts will take only a few tens of seconds. constant accuracy guarantee and algorithmic expressibility Malicious CSP: Recall that our extension implements the of CDP without any trusted server. This is achieved via two CSP inside a TEE. Hence, this ensures that the validity of each non-colluding servers with the assistance of cryptographic of CSP’s actions in the TEE can be attested to by the data primitives, specifically LHE and garbled circuits. Our pro- owners. Since the measurement operators (P2) are changed posed system Cryptϵ can execute a rich class of programs to be implemented completely inside the CSP, this guaran- that can run efficiently by virtue of four optimizations. tees the bounded ϵ-DP guarantee of Cryptϵ programs even Recall that currently the data analyst spells out the explicit under the malicious model. Additionally sending the CSP Cryptϵ program to the AS. Thus, an interesting future work ϵ the true ciphers c = P1(D˜ ) also does not cause any privacy is constructing a compiler for Crypt that inputs a user speci- violation as it is decrypted inside the TEE. fied query in a high-level-language. The compiler should next Validity of the data owner’s records: The validity of the formalize a Cryptϵ program expressed in terms of Cryptϵ one-hot-coding of the data records, D˜ i , submitted by the operators with automated sensitivity analysis. Another di- rection is to support a larger class of programs in Cryptϵ. data owners DOi can be checked as follows. Let D˜ ij repre- For example, inclusion of aggregation operators such as sum, sent the encrypted value for attribute Aj in one-hot-coding median, average is easily achievable. Support for multi-table for DOi . The AS selects a set of random numbers R = {rk | k ∈ queries like joins would require protocols for computing [|domain(Aj )|]} and computes the set Pij = {labMult(D˜ ij [k], sensitivity [62] and data truncation [68]. labEncpk (rk ))}. Then it sends sets Pij and R to the CSP who validates the record only if |Pij ∩ R| = 1 Aj . Note that since Acknowledgement: This work was supported by NSF un- ∀ the CSP does not have access to the index information of Pij der grants 1253327, 1408982; and by DARPA and SPAWAR and R (since they are sets), it cannot learn the value of Dij . under contract N66001-15-C-4067. REFERENCES Symposium on Foundations of Computer Science, Oct 2012. [1] Google cloud platform. https://cloud.google.com. [23] J. W. Bos, W. Castryck, I. Iliashenko, and F. Vercauteren. Privacy- [2] https://github.com/emp-toolkit. friendly forecasting for the smart grid using homomorphic encryption [3] https://github.com/encryptogroup/aby. and the group method of data handling. In M. Joye and A. Nitaj, [4] https://github.com/kuleuven-cosic/scale-mamba. editors, Progress in Cryptology - AFRICACRYPT 2017, pages 184–201, [5] http://www.multipartycomputation.com/mpc-software. Cham, 2017. Springer International Publishing. [6] A.Asuncion and D. Newman. Uci machine learning repository, 2010. [24] J. Camenisch and M. Michels. Proving in zero-knowledge that a [7] J. M. Abowd and I. M. Schmutte. An economic analysis of privacy pro- number is the product of two safe primes. In Proceedings of the 17th tection and statistical accuracy as social choices. American Economic International Conference on Theory and Application of Cryptographic Review, 109(1):171–202, January 2019. Techniques, EUROCRYPT’99, pages 107–122, Berlin, Heidelberg, 1999. [8] G. Acs, C. Castelluccia, and R. Chen. Differentially private histogram Springer-Verlag. publishing through lossy compression. In 2012 IEEE 12th International [25] H. Chabanne, A. de Wargny, J. Milgram, C. Morel, and E. Prouff. Conference on Data Mining, pages 1–10, Dec 2012. Privacy-preserving classification on deep neural network. IACR [9] A. Agarwal, M. Herlihy, S. Kamara, and T. Moataz. Encrypted Cryptology ePrint Archive, 2017:35, 2017. databases for differential privacy, 2018. https://eprint.iacr.org/2018/ [26] P. Chaidos and G. Couteau. Efficient designated-verifier non- 860. interactive zero-knowledge proofs of knowledge. IACR Cryptology [10] E. Aïmeur, G. Brassard, J. M. Fernandez, and F. S. Mani Onana. Alam- ePrint Archive, 2017:1029, 2017. bic: a privacy-preserving recommender system for electronic com- [27] T.-H. H. Chan, K.-M. Chung, B. M. Maggs, and E. Shi. Foundations merce. International Journal of Information Security, 7(5), Oct 2008. of differentially oblivious algorithms. In Proceedings of the Thirtieth [11] J. Alwen, J. Katz, Y. Lindell, G. Persiano, a. shelat, and I. Visconti. Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’19, Collusion-free multiparty computation in the mediated model. In pages 2448–2467, Philadelphia, PA, USA, 2019. Society for Industrial S. Halevi, editor, Advances in Cryptology - CRYPTO 2009, pages 524– and Applied Mathematics. 540, Berlin, Heidelberg, 2009. Springer Berlin Heidelberg. [28] T.-H. H. Chan, E. Shi, and D. Song. Optimal lower bound for differ- [12] M. Barbosa, D. Catalano, and D. Fiore. Labeled homomorphic encryp- entially private multi-party aggregation. In Proceedings of the 20th tion - scalable and privacy-preserving processing of outsourced data. Annual European Conference on Algorithms, ESA’12, pages 277–288, In ESORICS, 2017. Berlin, Heidelberg, 2012. Springer-Verlag. [13] R. Bassily and A. Smith. Local, private, efficient protocols for suc- [29] T. H. H. Chan, E. Shi, and D. Song. Privacy-preserving stream ag- cinct histograms. In Proceedings of the Forty-seventh Annual ACM gregation with fault tolerance. In A. D. Keromytis, editor, Financial Symposium on Theory of Computing, STOC ’15, pages 127–135, New Cryptography and Data Security, pages 200–214, Berlin, Heidelberg, York, NY, USA, 2015. ACM. 2012. Springer Berlin Heidelberg. [14] J. Bater, X. He, S. Y. Tendryakova, A. Machanavajjhala, and J. Duggan. [30] T. Chanyaswad, A. Dytso, H. V. Poor, and P. Mittal. Mvg mechanism: Shrinkwrap: Differentially-private query processing in private data Differential privacy under matrix-valued query. In Proceedings of federations. CoRR, abs/1810.01816, 2018. the 2018 ACM SIGSAC Conference on Computer and Communications [15] D. Beaver. Precomputing oblivious transfer. In Proceedings of the 15th Security, CCS ’18, pages 230–246, New York, NY, USA, 2018. ACM. Annual International Cryptology Conference on Advances in Cryptol- [31] A. Cheu, A. D. Smith, J. Ullman, D. Zeber, and M. Zhilyaev. Distributed ogy, CRYPTO ’95, pages 97–109, Berlin, Heidelberg, 1995. Springer- differential privacy via mixnets. CoRR, abs/1808.01394, 2018. Verlag. [32] G. Cormode, T. Kulkarni, and D. Srivastava. Marginal release under [16] A. Beimel, K. Nissim, and E. Omri. Distributed private data analysis: local differential privacy. In Proceedings of the 2018 International Simultaneously solving how and what. In Proceedings of the 28th Conference on Management of Data, SIGMOD ’18, pages 131–146, Annual Conference on Cryptology: Advances in Cryptology, CRYPTO New York, NY, USA, 2018. ACM. 2008, pages 451–468, Berlin, Heidelberg, 2008. Springer-Verlag. [33] G. Cormode, C. Procopiuc, D. Srivastava, E. Shen, and T. Yu. Differ- [17] A. Beimel, K. Nissim, and E. Omri. Distributed private data analysis: entially private spatial decompositions. 2012 IEEE 28th International On simultaneously solving how and what. CoRR, abs/1103.2626, 2011. Conference on Data Engineering, Apr 2012. [18] A. Beimel, K. Nissim, and U. Stemmer. Private learning and sanitiza- [34] W.-Y. Day and N. Li. Differentially private publishing of high- tion: Pure vs. approximate differential privacy. CoRR, abs/1407.2674, dimensional data using sensitivity control. In Proceedings of the 2014. 10th ACM Symposium on Information, Computer and Communications [19] M. Bellare and P. Rogaway. The security of triple encryption and Security, ASIA CCS ’15, pages 451–462, New York, NY, USA, 2015. a framework for code-based game-playing proofs. In Advances in ACM. Cryptology - EUROCRYPT 2006, pages 409–426, Berlin, Heidelberg, [35] D. Demmler, T. Schneider, and M. Zohner. Aby - a framework for 2006. Springer Berlin Heidelberg. efficient mixed-protocol secure two-party computation. In NDSS, [20] A. Bittau, U. Erlingsson, P. Maniatis, I. Mironov, A. Raghunathan, 2015. D. Lie, M. Rudominer, U. Kode, J. Tinnes, and B. Seefeld. Prochlo: [36] J. C. Duchi, M. I. Jordan, and M. J. Wainwright. Local privacy and Strong privacy for analytics in the crowd. In Proceedings of the 26th statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Symposium on Operating Systems Principles, SOSP ’17, pages 441–459, Foundations of Computer Science, pages 429–438, Oct 2013. New York, NY, USA, 2017. ACM. [37] C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. [21] M. Blatt, A. Gusev, Y. Polyakov, K. Rohloff, and V. Vaikuntanathan. Our data, ourselves: Privacy via distributed noise generation. In Optimized homomorphic encryption solution for secure genome- Proceedings of the 24th Annual International Conference on The Theory wide association studies. IACR Cryptology ePrint Archive, 2019:223, and Applications of Cryptographic Techniques, EUROCRYPT’06, pages 2019. 486–503, Berlin, Heidelberg, 2006. Springer-Verlag. [22] J. Blocki, A. Blum, A. Datta, and O. Sheffet. The johnson-lindenstrauss [38] C. Dwork and A. Roth. The algorithmic foundations of differential transform itself preserves differential privacy. 2012 IEEE 53rd Annual privacy. Found. Trends Theor. Comput. Sci., 9(3–4):211–407, Aug. 2014. [39] C. Dwork, G. N. Rothblum, and S. Vadhan. Boosting and differential of the VLDB Endowment, 3(1-2):1021âĂŞ1032, Sep 2010. privacy. In Proceedings of the 2010 IEEE 51st Annual Symposium on [59] X. He, A. Machanavajjhala, C. Flynn, and D. Srivastava. Composing Foundations of Computer Science, FOCS ’10, pages 51–60, Washington, differential privacy and secure computation: A case study on scaling DC, USA, 2010. IEEE Computer Society. private record linkage. In Proceedings of the 2017 ACM SIGSAC Con- [40] H. Ebadi and D. Sands. Featherweight pinq, 2015. ference on Computer and Communications Security, CCS ’17, pages [41] Ú. Erlingsson, V. Feldman, I. Mironov, A. Raghunathan, K. Talwar, 1389–1406, New York, NY, USA, 2017. ACM. and A. Thakurta. Amplification by shuffling: From local to central [60] E. Hesamifard, H. Takabi, and M. Ghasemi. Cryptodl: Deep neural differential privacy via anonymity. CoRR, abs/1811.12469, 2018. networks over encrypted data, 2017. [42] Ú. Erlingsson, V. Pihur, and A. Korolova. Rappor: Randomized aggre- [61] J. Hsu, M. Gaboardi, A. Haeberlen, S. Khanna, A. Narayan, B. C. Pierce, gatable privacy-preserving ordinal response. In CCS, 2014. and A. Roth. Differential privacy: An economic method for choosing [43] A. Evfimievski, J. Gehrke, and R. Srikant. Limiting privacy breaches in epsilon. 2014 IEEE 27th Computer Security Foundations Symposium, privacy preserving data mining. In Proceedings of the Twenty-second pages 398–410, 2014. ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database [62] N. M. Johnson, J. P. Near, and D. X. Song. Practical differential privacy Systems, PODS ’03, pages 211–222, New York, NY, USA, 2003. ACM. for SQL queries using elastic sensitivity. CoRR, abs/1706.09479, 2017. [44] G. Fanti, V. Pihur, and ÃŽlfar Erlingsson. Building a rappor with [63] S. Kamara, P. Mohassel, and M. Raykova. Outsourcing multi-party the unknown: Privacy-preserving learning of associations and data computation. Cryptology ePrint Archive, Report 2011/272, 2011. dictionaries, 2015. https://eprint.iacr.org/2011/272. [45] A. Gascón, P. Schoppmann, B. Balle, M. Raykova, J. Doerner, S. Zahur, [64] S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and and D. Evans. Secure linear regression on vertically partitioned A. Smith. What can we learn privately? In 2008 49th Annual IEEE datasets. IACR Cryptology ePrint Archive, 2016:892, 2016. Symposium on Foundations of Computer Science, pages 531–540, Oct [46] A. Gascón, P. Schoppmann, B. Balle, M. Raykova, J. Doerner, S. Zahur, 2008. and D. Evans. Privacy-preserving distributed linear regression on [65] J. Katz and Y. Lindell. Introduction to Modern Cryptography, Second high-dimensional data. PoPETs, 2017:345–364, 2017. Edition. Chapman & Hall/CRC, 2nd edition, 2014. [47] C. Ge, X. He, I. F. Ilyas, and A. Machanavajjhala. Apex: Accuracy- [66] M. Kearns. Efficient noise-tolerant learning from statistical queries. aware differentially private data exploration. In Proceedings of the J. ACM, 45(6):983–1006, Nov. 1998. 2019 International Conference on Management of Data, SIGMOD ’19, [67] S. Kim, J. Kim, D. Koo, Y. Kim, H. Yoon, and J. Shin. Efficient privacy- pages 177–194, New York, NY, USA, 2019. ACM. preserving matrix factorization via fully homomorphic encryption: [48] I. Giacomelli, S. Jha, M. Joye, C. D. Page, and K. Yoon. Privacy- Extended abstract. In Proceedings of the 11th ACM on Asia Conference preserving ridge regression with only linearly-homomorphic encryp- on Computer and Communications Security, ASIA CCS ’16, pages tion. In B. Preneel and F. Vercauteren, editors, Applied Cryptography 617–628, New York, NY, USA, 2016. ACM. and Network Security, pages 243–261, Cham, 2018. Springer Interna- [68] I. Kotsogiannis, Y. Tao, X. He, M. Fanaeepour, A. Machanavajjhala, tional Publishing. M. Hay, and G. Miklau. Privatesql: A differentially private sql query [49] I. Giacomelli, S. Jha, R. Kleiman, D. Page, and K. Yoon. Privacy- engine. Proc. VLDB Endow., 12(11):1371–1384, July 2019. preserving collaborative prediction using random forests, 2018. [69] J. Lee and C. Clifton. How much is enough? choosing ϵ for differ- [50] R. Gilad-Bachrach, N. Dowlin, K. Laine, K. E. Lauter, M. Naehrig, and ential privacy. In Proceedings of the 14th International Conference on J. R. Wernsing. Cryptonets: Applying neural networks to encrypted Information Security, ISC’11, pages 325–340, Berlin, Heidelberg, 2011. data with high throughput and accuracy. In ICML, 2016. Springer-Verlag. [51] A. Greenberg. Apple’s ‘differential privacy’ is about collecting your [70] C. Li, M. Hay, G. Miklau, and Y. Wang. A data- and workload-aware data—but not your data. Wired, Jun 13 2016. algorithm for range queries under differential privacy. Proceedings of [52] A. Groce, P. Rindal, and M. Rosulek. Cheaper private set intersection the VLDB Endowment, 7(5):341âĂŞ352, Jan 2014. via differentially private leakage. Cryptology ePrint Archive, Report [71] C. Li, M. Hay, V. Rastogi, G. Miklau, and A. McGregor. Optimizing 2019/239, 2019. https://eprint.iacr.org/2019/239. linear counting queries under differential privacy. In Proceedings [53] J. Groth and A. Sahai. Efficient non-interactive proof systems for of the Twenty-ninth ACM SIGMOD-SIGACT-SIGART Symposium on bilinear groups. Cryptology ePrint Archive, Report 2007/155, 2007. Principles of Database Systems, PODS ’10, pages 123–134, New York, https://eprint.iacr.org/2007/155. NY, USA, 2010. ACM. [54] M. Hardt, K. Ligett, and F. McSherry. A simple and practical algo- [72] N. Li, M. Lyu, D. Su, and W. Yang. Differential Privacy: From Theory rithm for differentially private data release. In Proceedings of the 25th to Practice. Morgan and Claypool, 2016. International Conference on Neural Information Processing Systems - [73] Y. Lindell and B. Pinkas. A proof of security of yao’s protocol Volume 2, NIPS’12, pages 2339–2347, USA, 2012. Curran Associates for two-party computation. J. Cryptol., 22(2):161–188, Apr. 2009. Inc. [74] Y. Lindell and B. Pinkas. A proof of security of yao’s protocol [55] M. Hardt and G. N. Rothblum. A multiplicative weights mechanism for two-party computation. J. Cryptol., 22(2):161–188, Apr. 2009. for privacy-preserving data analysis. In 2010 IEEE 51st Annual Sym- [75] M. Lyu, D. Su, and N. Li. Understanding the sparse vector technique posium on Foundations of Computer Science, pages 61–70, Oct 2010. for differential privacy. PVLDB, 10:637–648, 2017. [56] M. Hay, A. Machanavajjhala, G. Miklau, Y. Chen, and D. Zhang. Prin- [76] S. Mazloom and S. D. Gordon. Secure computation with differentially cipled evaluation of differentially private algorithms using dpbench. private access patterns. In Proceedings of the 2018 ACM SIGSAC In Proceedings of the 2016 International Conference on Management of Conference on Computer and Communications Security, CCS ’18, pages Data, SIGMOD ’16, pages 139–154, New York, NY, USA, 2016. ACM. 490–507, New York, NY, USA, 2018. ACM. [57] M. Hay, V. Rastogi, G. Miklau, and D. Suciu. Boosting the accuracy [77] F. D. McSherry. Privacy integrated queries: An extensible platform of differentially private histograms through consistency. Proc. VLDB for privacy-preserving data analysis. In Proceedings of the 2009 ACM Endow., 3(1-2):1021–1032, Sept. 2010. SIGMOD International Conference on Management of Data, SIGMOD [58] M. Hay, V. Rastogi, G. Miklau, and D. Suciu. Boosting the accuracy ’09, pages 19–30, New York, NY, USA, 2009. ACM. of differentially private histograms through consistency. Proceedings [78] I. Mironov. Renyi differential privacy. CoRR, abs/1702.07476, 2017. [79] I. Mironov, O. Pandey, O. Reingold, and S. Vadhan. Computational [94] J. Van Bulck, M. Minkin, O. Weisse, D. Genkin, B. Kasikci, F. Piessens, differential privacy. In S. Halevi, editor, Advances in Cryptology M. Silberstein, T. F. Wenisch, Y. Yarom, and R. Strackx. Foreshadow: - CRYPTO 2009, pages 126–142, Berlin, Heidelberg, 2009. Springer Extracting the keys to the intel sgx kingdom with transient out-of- Berlin Heidelberg. order execution. In Proceedings of the 27th USENIX Conference on [80] P. Mohassel and Y. Zhang. Secureml: A system for scalable privacy- Security Symposium, SEC’18, pages 991–1008, Berkeley, CA, USA, preserving machine learning. In 2017 IEEE Symposium on Security 2018. USENIX Association. and Privacy (SP), pages 19–38, May 2017. [95] T. Wang, J. Blocki, N. Li, and S. Jha. Locally differentially private [81] A. Narayan and A. Haeberlen. Djoin: Differentially private join protocols for frequency estimation. In Proceedings of the 26th USENIX queries over distributed databases. In Proceedings of the 10th USENIX Conference on Security Symposium, SEC’17, pages 729–745, Berkeley, Conference on Operating Systems Design and Implementation, OSDI’12, CA, USA, 2017. USENIX Association. pages 149–162, Berkeley, CA, USA, 2012. USENIX Association. [96] T. Wang, N. Li, and S. Jha. Locally differentially private heavy hitter [82] T. T. Nguyên, X. Xiao, Y. Yang, S. C. Hui, H. Shin, and J. Shin. Collect- identification, 2017. ing and analyzing data from smart device users with local differential [97] T. Wang, N. Li, and S. Jha. Locally differentially private frequent privacy. CoRR, abs/1606.05053, 2016. itemset mining. In 2018 IEEE Symposium on Security and Privacy (SP), [83] V. Nikolaenko, S. Ioannidis, U. Weinsberg, M. Joye, N. Taft, and pages 127–143, May 2018. D. Boneh. Privacy-preserving matrix factorization. In Proceedings of [98] X. Wang, S. Ranellucci, and J. Katz. Authenticated garbling and the 2013 ACM SIGSAC Conference on Computer & Communications efficient maliciously secure two-party computation. In Proceedings of Security, CCS ’13, pages 801–812, New York, NY, USA, 2013. ACM. the 2017 ACM SIGSAC Conference on Computer and Communications [84] V. Nikolaenko, U. Weinsberg, S. Ioannidis, M. Joye, D. Boneh, and Security, CCS ’17, pages 21–37, New York, NY, USA, 2017. ACM. N. Taft. Privacy-preserving ridge regression on hundreds of millions [99] S. L. Warner. âĂIJrandomized response: A survey technique for elim- of records. In 2013 IEEE Symposium on Security and Privacy, pages inating evasive answer bias.âĂİ. Journal of the American Statistical 334–348, May 2013. Association, 60 60, no. 309:63âĂŞ69, 1965. [85] A. Nikolov, K. Talwar, and L. Zhang. The geometry of differential [100] X. Xiao, G. Bender, M. Hay, and J. Gehrke. ireduct: Differential privacy privacy. Proceedings of the 45th annual ACM symposium on Symposium with reduced relative errors. In Proceedings of the 2011 ACM SIGMOD on theory of computing - STOC âĂŹ13, 2013. International Conference on Management of Data, SIGMOD ’11, pages [86] G. Oded. Foundations of Cryptography: Volume 2, Basic Applications. 229–240, New York, NY, USA, 2011. ACM. Cambridge University Press, New York, NY, USA, 1st edition, 2009. [101] X. Xiao, G. Wang, and J. Gehrke. Differential privacy via wavelet [87] P. Paillier. Public-key cryptosystems based on composite degree resid- transforms. 2010 IEEE 26th International Conference on Data Engineer- uosity classes. In Proceedings of the 17th International Conference on ing (ICDE 2010), 2010. Theory and Application of Cryptographic Techniques, EUROCRYPT’99, [102] J. Xu, Z. Zhang, X. Xiao, Y. Yang, and G. Yu. Differentially private pages 223–238, Berlin, Heidelberg, 1999. Springer-Verlag. histogram publication. In 2012 IEEE 28th International Conference on [88] T. P. Pedersen. Non-interactive and information-theoretic secure ver- Data Engineering, pages 32–43, April 2012. ifiable secret sharing. In Proceedings of the 11th Annual International [103] A. C. Yao. How to generate and exchange secrets. In 27th Annual Cryptology Conference on Advances in Cryptology, CRYPTO ’91, pages Symposium on Foundations of Computer Science (sfcs 1986), pages 129–140, London, UK, UK, 1992. Springer-Verlag. 162–167, Oct 1986. [89] W. Qardaji, W. Yang, and N. Li. Differentially private grids for geospa- [104] G. Yuan, Z. Zhang, M. Winslett, X. Xiao, Y. Yang, and Z. Hao. Low-rank tial data. In 2013 IEEE 29th International Conference on Data Engineer- mechanism. Proceedings of the VLDB Endowment, 5(11):1352âĂŞ1363, ing (ICDE), pages 757–768, April 2013. Jul 2012. [90] W. Qardaji, W. Yang, and N. Li. Understanding hierarchical methods [105] G. Yuan, Z. Zhang, M. Winslett, X. Xiao, Y. Yang, and Z. Hao. Opti- for differentially private histograms. Proc. VLDB Endow., 6(14):1954– mizing batch linear queries under exact and approximate differential 1965, Sept. 2013. privacy. ACM Trans. Database Syst., 40(2):11:1–11:47, June 2015. [91] Z. Qin, Y. Yang, T. Yu, I. Khalil, X. Xiao, and K. Ren. Heavy hitter [106] D. Zhang, R. McKenna, I. Kotsogiannis, M. Hay, A. Machanavajjhala, estimation over set-valued data with local differential privacy. In and G. Miklau. EKTELO: A framework for defining differentially- Proceedings of the 2016 ACM SIGSAC Conference on Computer and private computations. In Proceedings of the 2018 International Confer- Communications Security, CCS ’16, pages 192–203, New York, NY, ence on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, 2016. ACM. USA, June 10-15, 2018, pages 115–130, 2018. [92] V. Rastogi and S. Nath. Differentially private aggregation of dis- [107] X. Zhang, R. Chen, J. Xu, X. Meng, and Y. Xie. Towards accurate tributed time-series with transformation and encryption. In Proceed- histogram publication under differential privacy. In Proceedings of the ings of the 2010 ACM SIGMOD International Conference on Manage- 2014 SIAM International Conference on Data Mining, pages 587–595. ment of Data, SIGMOD ’10, pages 735–746, New York, NY, USA, 2010. [108] Z. Zhang, T. Wang, N. Li, S. He, and J. Chen. Calm: Consistent adaptive ACM. local marginal for marginal release under local differential privacy. [93] E. Shi, T.-H. Hubert Chan, E. G. Rieffel, R. Chow, and D. Song. Privacy- In Proceedings of the 2018 ACM SIGSAC Conference on Computer and preserving aggregation of time-series data. volume 2, 01 2011. Communications Security, CCS ’18, pages 212–229, New York, NY, USA, 2018. ACM. Table 4: Comparative analysis of different DP models • labMultDecsk (d1,d2, e) - On input two encrypted masks d1,d2 of two labHE ciphers c1, c2, this algorithm decryts Features LDP CDP Cryptϵ the output e oflabMult(c1, c2) asm3 = Decsk (e)+Decsk (d1)· # Centralized Servers 1 1 2 Decsk (d2) which is equals to m1 · m2. Trust Assumption Untrusted for Centralized Untrusted Trusted Semi Honest Server Non-Colluding B SECURITY PROOF Data Storage N/A Clear Encrypted in Server In this section we present the formal proof for Theorem 3. Information Information Computationally Adversary Theoretic√ Theoretic Bounded Proof. We have nine operators in our paper (see Table 1).  (n)      Error on Statistical Counting Query O O 1 O 1 ϵ ϵ ϵ • NoisyMax and CountDistinct use “standard” garbled circuit construction and their security proof follows from the proof of these schemes. A BACKGROUND CNTD. • All other operators except Laplace essentially use ho- The stability of a transformation operation is defined as momorphic properties of our encryption scheme and Definition 2. A transformation T is defined to be t-stable thus there security follows from semantic-security of if for two datasets D and D′, we have these scheme. • The proof for the Laplace operator is given below. |T (D) ⊖ T (D′)| ≤ t · |D ⊖ D′| (3) The proof for an entire program P (which is a composition of where (i.e., D ⊖ D′ = (D − D′) ∪ (D′ − D). these operators) follows from the composition theorem [86, Section 7.3.1] Transformations with bounded stability scale the DP guar- We will prove the theorem for the Laplace operator. In antee of their outputs, by their stability constant [77]. this case the views are as follows (the outputs of the two Theorem 6. If T is an arbitrary t-stable transformation on parties can simply computed from the views): A T dataset D and is an ϵ-DP algorithm which takes output of ViewΠ(P, D, ϵ) = (pk, D˜ ,η , P(D) + η + η ) as input, the composite computation A ◦ T provides (ϵ · t)-DP. 1 1 2 1 Π View2 (pk,sk, P, D, ϵ) = (η2,labEncpk (P(D) + η1)) Labeled Homomorphic Encryption(labHE). Let (Gen, The random variables η1 and η2 are random variables gen- Enc, Dec) be an LHE scheme with security parameter κ and 2·∆ erated according to the Laplace distribution Lap( ϵ ) where message space M. Assume that a multiplication operation ∆ is the program sensitivity (computed w.r.t Definition 1). s exists in M, i.e., is a finite ring. Let F : {0, 1} × L → M The simulators Sim (zB ) (where z = (y , |D|) is the random s 1 1 1 1 be a pseudo-random function with seed space {0, 1} ( s= CDP variable distributed according to P (D, ϵ)), y1 being the κ L labHE B poly( )) and the label space .A scheme is defined random variable distributed as PCDP (D, ϵ/2)) performs the as following steps: • labGen(κ) : Runs Gen(κ) and outputs (sk,pk). • Generates a pair of keys (pk1,sk1) for the encryption • localGen(pk) : For each user i and with the public key scheme and generates random data set D1 of the same s as input, it samples a random seed σi ∈ {0, 1} and com- size as D and encrypts it using pk1 to get D˜ 1. putes pk = Enc (σ ) where σ is an encoding of σ as an ′ i pk i i i • Generates η1 according to the Laplace distribution element of M. It outputs (σ ,pk ). 2·∆ i i Lap( ϵ ). ′ ′ • labEncpk (σi ,m,τ ) : On input a message m ∈ M with ˜ The output of Sim1(z1) is (D1,η1,y1+η1). Recall that the view label τ ∈ L from user i, it computes b = F(σ ,τ ) (mask) i of the AS is (D˜ ,η1, P(D) +η2 +η1). The computational indis- and outputs the labeled ciphertext c = (a,d) ∈ M × C tinguishability of D˜ 1 and D˜ follows from the semantic secu- with a = m − b (hidden message) in M and d = Enc (b). ′ ′ pk rity of the encryption scheme. The tuple (η ,y1 + η ) has the For brevity we just use notation labEnc (m) to denote 1 1 pk same distribution as (η1, P(D)+η2 +η1) and hence the tuples the above functionality, in the rest of paper. are computationally indistinguishable. Therefore, Sim1(z1) • ( ) ( ) ∈ Π labDecsk c : This functions inputs a cipher c = a,d is computational indistinguishable from View1 (P, D, ϵ). M × C and decrypts it as m = a − Decsk (d). The simulators Sim2(z2) (where z2 = (y2, |D|) is the ran- CDP (D, )) • labMult(c1, c2) - On input two labHE ciphers c1 = (a1,d1) dom variable distributed according to PB ϵ , y2 being CDP and c2 = (a2,d2), it computes a "multiplication" ciphertext the random variable distributed as P (D, ϵ/2)) performs e = labMult(c1, c2) = Encpk (a1, a2) ⊕ cMult(d1, a2) ⊕ the following steps: cMult(d2, a1). Observe that Decsk (e) = m1 · m2 − b1 · b2. • Generates a pair of keys (pk2,sk2) for our encryption scheme. ′ • Generates η2 according to the Laplace distribution quite evidently an efficient DV NIZK proof for eq (2) can 2·∆ Lap( ϵ ). be supported by the framework in [26]. To get an idea of ′ ′ the execution overhead for the ZKPs, consider constructing The output of Sim2(z2) is (η2,labEncpk (y2) + η2). By simi- a DV NIZK for proving that a Paillier ciphertext encrypts lar argument as before Sim2(z2) is computationally indistin- Π the products of the plaintexts of two other ciphertexts re- guishable from View2 (P, D, ϵ). □ quires (this could be useful for proving the validity of our C EXTENSION TO MALICIOUS MODEL Filter operator). In the framework proposed in in [26], this CNTD. involves 4loдN bits of communication and the operations involve addition and multiplication of group elements. Each C.1 First Approach Cntd. such operation takes order of 10−5s execution time, hence Efficient proof construction: Here we will outline an effi- for proving the above statement for 1 mil ciphers will take cient construction for the aforementioned proof. First note only a few tens of seconds. that our setting suits that of designated verifier non-interactive zero knowledge (DV NIZK) proofs. In a DV NIZK setting, C.2 Second Approach the proofs can be verified by a single designated entity (as In this section, we describe the second approach to extend opposed to publicly verifiable proofs) who possesses some Cryptϵ to account for a malicious adversary. For this we secret key for the NIZK system. Thus in Cryptϵ, clearly the propose the following changes to the implementation of the CSP can assume the role of the designated verifier. This re- measurement operator. laxation of public verifiability leads to boast in efficiency for Laplace Lap∆,ϵ (c\V ): Instead of having both the servers, AS the proof system. and CSP add two separate instances of Laplace noise to the The authors in [26] present a framework for efficient DV true answer, single instance of the Laplace noise is jointly NIZKs for a group-dependent language L where the abelian computed via a SMC protocol [37, 81] as follows. First the AS group L is initiated on is of order N and ZN is the plaintext- adds a random mask M to the encrypted input c to generate space of an homomorphic cryptosystem. In other words, cˆ and sends it to the CSP. Next the CSP generates a garbled this framework enables proving arbitrary relations between circuit that 1) inputs two random bit strings S1 and R1 from cryptographic primitives such as Pedersen commitments the AS 2) inputs another pair of random strings S2 and R2 ′ or Paillier encryptions via DV NIZK proofs, in an efficient and a mask M from the CSP 3) uses S1 and S2 to generate an 2·∆ way. In what follows, we show that the proof statement instance of random noise, η ∼ Lap( ϵ ) using the fundamen- given by eq (2) falls in the language L and consists of simple tal law of transformation of probabilities 4) uses R1 ⊕ R2 as arithmetic computations. the randomness for generating a Pedersen commitment for ′ ′ ′ ′ The construction of the proof works as follows. First the M ,Com(M ) 4) outputs c˜ = cˆ +labEncpk (η)+labEncpk (M ), ′ AS creates linearly homomorphic commitments (we use Ped- Com(M ), and labEncpk (r) (r is the randomness used for gen- e ( ′) erson commitments) Comi on the encrypted data records erating Com M ). The CSP sends this circuit to the AS who e e ′ ( ′) ( ) D˜i and proves that Comc = P1(Com ,...,Com ) where evaluates the circuit and sends c˜ , Com M , and labEncpk r 1 m ′ Open(Comc ) = c. This is possible because of the homomor- back to the CSP. Now, the CSP decrypts c˜ and subtracts the ′ ′ ′ ′ phic property of the Pederson commitment scheme; all the mask M to return c˜ = labDecsk (c˜ ) − M to the AS. Finally e the AS can subtract out M to compute the answer c˜ = c˜′ − M. operations in P1 can be applied to {Comi } instead. We use Paillier encryption scheme [87] for our prototype Cryptϵ con- Note that one can create an alternative circuit to the one struction and hence base the rest of the discussion on it. Pail- given above which decrypts ˜c inside the circuit. However, lier ciphertexts are elements in the group (Z/N 2Z)∗ where decrypting Pailler ciphertexts inside the garbled circuit is N is an RSA modulus. Pedersen commitments to such values costly. The circuit design given above hence results in a sim- x r ∗ 2 pler circuit at the cost of an extra round of communication. can be computed as Com = д h ∈ Fp, 0 ≤ r < N ,д,h ∈ ∗ 2 NoisyMax NoisyMaxk (·): The AS sends a masked encrypted Fp, Order(д) = Order(h) = N ,p is a prime such that p = ϵ,∆ 1mod N 2. This allows us to prove arithmetic relations on vector, Vˆ to the CSP Vˆ [i] = V [i] + M[i],i ∈ [|V |]. The CSP committed values modulo N 2. Finally, the AS just needs to generates a garbled circuit that 1) inputs the mask vector e show that Comi opens to a MAC of the opening of Comi . M, a vector of random strings S1, a random number r and For this, the MACs we use are built from linear hash func- its ciphertext cr = labEncpk (r) from the AS 2) inputs the tions H(x) = ax + b [84] where the MAC signing key is secret key sk and another vector of random strings S2 the 2 the pair of random values (a,b) ∈ (Z/N Z). Proving to the from the CSP 3) checks if labDecsk (cr) == r, proceed to CSP that the opening of Comi is a valid MAC on the open- the next steps only if the check succeeds else return −1 4) ing of Come is a simple proof of arithmetic relations. Thus, 2·k ·∆ i uses S1 and S2 to generate a vector η[i] ∼ Lap( ϵ ) using the fundamental law of transformation of probabilities 5) Recall that the CSP receives a masked cipher cˆ from the computes V [i] = labDecsk (Vˆ [i]) + η[i] − M[i] 6) finds the AS at the beginning of the measurement operators. The mask indices of the top k highest values of V and outputs them. M protects the value of c from the CSP. We discuss the set- The CSP sends this circuit to the AS who evaluates it to get ting of a malicious CSP separately for the two measurement the answer. Note that here we are forced to decrypt Paillier operators as follows. ciphertexts inside the circuit because in order to ensure DP Laplace: In case of the Laplace operator, a malicious CSP in the Noisy-Max algorithm, the noisy intermediate counts can cheat by 1) the generated garbled circuit does not corre- cannot be revealed. spond to the correct functionality 2) reports back incorrect Malicious AS: Recall that a Cryptϵ program P consists of decryption results. The correctness of the garbled circuit a series of transformation operators that transforms the en- can be checked by standard mechanisms [98] where the AS crypted database D˜ to a ciphertext c (or a vector of cipher- specifically checks that a) the circuit functionality is correct texts V ). This is followed by applying a measurement opera- b) the circuit uses the correct value for cˆ. For the second case, tor on c (V ). Additionally, as shown in the above discussion, the CSP provides the AS with a zero knowledge proof for in the very first step of the measurement operators the AS the following statement ′ ′ ′ adds a mask to c and sends the masked ciphertext cˆ = c + M Open(Com(M ),r) = labDecsk (c˜ ) − c˜ to the CSP. For a given program P, let P1 represent first part of the program up till the computation of c (V ). The zero NoisyMax: The garbled circuit for the NoisyMax operator knowledge proof structure is very similar to the one dis- is validated similarly by standard mechanisms [98] where cussed in Section 7.2 except for the following changes. Now the AS checks a) whether the circuit implements the correct k the CSP sends a one-time MAC key AS to the AS as well functionality b) the correct value of Vˆ is used. Note that the ˆ ) and the AS sends the masked ciphertext cˆ(or V , along with equality check of step (3) in the circuit validates if the CSP the commitments and zero knowledge proofs from the data has provided the correct secret key sk thereby forcing it to owners and an additional commitment to the one-time MAC decrypt the ciphertexts correctly. on the mask M, ComAS and a proof for the statement Note that certain operators like CrossProduct, GroupBy- ˜ ˜ ˜ c = P1(D1, ··· , Dm) ∧ i Open(Comi ) = MACmki (Di ) Count* and CountDistinct the involve interactions with the ∀ CSP as well but their validity can also be proven by standard ∧cˆ = c + labEncpk (M) ∧ Open(ComAS ) = MACk (M) AS techniques similar to the ones discussed above. Specifically The CSP proceeds with the rest of the computation only if it CrossProduct and GroupByCount* can use zero knowledge can validate the above proof. As long as one of the bit strings proof in the framework [26] while the garbled circuit in { } { } (or vectors of bit strings) in S1, S2 (and R2, R2 in case of CountDistinct can use [98]. the Laplace operator) is generated truly at random (in this case the honest CSP will generate truly random strings), the D ADDITIONAL IMPLEMENTATION garbled circuits for the subsequent measurement operators DETAILS will add the correct Laplace noise. Additionally, the mask M ′ prevents the AS from cheating in the last round of communi- D.1 General n-way Multiplication for cation with the CSP in the protocol. It is so because, if the AS labHE does not submit the correct ciphertext to the CSP in the last The labMult() operator of a labHE scheme allows the multi- round, it will get back garbage values (thereby thwarting any plication of two ciphers. However, it cannot be used directly privacy leakage). Hence, this prevents a malicious AS from for a n-way muplication where n > 2. It is so because the ϵ cheating during any Crypt program execution. Note that "multiplication" cipher e = labMult(c1, c2) does not have a the construction of the ZKP is similar to the one discussed corresponding label, i.e., it is not in the correct labHE cipher in Section 7.2 and can be done efficiently via the framework representation. Hence we propose Algorithm 1 to generate a in [26]. label τ ′ and a seed b′ for every intermediary product of two Malicious CSP: As discussed in Section 3, the CSP main- multiplicands so that it we can do a generic n-way multiplica- tains a public ledger with the following information tion on the ciphers. Note that the mask r protects the value of B ′ (1) total privacy budget ϵ which is publicly known (m1 ·m2) from the CSP (Step 3) and b hides (m1 ·m2) from the (2) the privacy budget ϵ used up every time the AS submits AS (Step 6). For example, suppose we want to multiply the 4 a ciphertext for decryption respective ciphers of 4 messages {m1,m2,m3,m4} ∈ M and Since the ledger is public, the AS can verify whether the per obtain e = labEncpk (m1·m2·m3·m4). For this, the AS first gen- program reported privacy budget is correct preventing any erates e1,2 = labEncpk (m1 ·m2) and e3,4 = labEncpk (m3 ·m4) disparities in the privacy budget allocation. using Algorithm 1. Both operations can be done in parallel in just one interaction round between the AS and the CSP. First, we will show how to evaluate whether a row r satis-

In the next round, the AS can again use Algorithm 1 with fies r.Aj ∈ VAj . Let v˜ j be the encrypted one-hot-encoding of inputs e1,2 and e3,4 to obtain the final answer e. Thus for a Aj , then the indicator function can be computed as generic n − way multiplication the order of multiplication Ê Ir .A ∈V = v˜ j [l]. can be, in fact, parallelized as shown in Figure 6 to require a j Aj l ∈V total of ⌈logn⌉ rounds of communication with the CSP. Aj A r V I If the attribute of j in has a value in Aj , then r .Aj ∈VAj D.2 Operator Implementation equals 1; otherwise, 0. Next, we can multiply all the indicators usingдenLabMult() CrossProduct × → ′ (·): This operator replaces the two Ai,Aj A (Section 6.1) to check whether all attributes in A ∈ A¯ of r attributes A and A by a single attribute A′. Given the en- j i j satisfy the conditions in ϕ. Let A¯ = {A ,..., A }, then crypted input table T˜ , where all attributes are in one-hot- 1 m ˜ ϕ(r) = дenLabMult(I ∈ ,..., I ∈ ). encoding and encrypted, the attributes of T except Ai and A1 VA1 Am VAm ′ Aj remain the same. For every row in T˜ , we denote the en- Last, we update the bit ofr in B, i.e., B [i] = labMult(B[i], ϕ(r)), crypted one-hot-encoding for Ai and Aj by v˜ 1 and v˜ 2. Let s1 given r is the ith row in the input table. This step zeros out and s2 be the domain sizes of Ai and Aj respectively. Then some additional records which were found to be extraneous the new one-hot-encoding for A′, denoted by v˜, has a length by some preceding filter conditions. of s = s1 · s2. For l ∈ {0, 1,...,s − 1}, we have Note that when the Filter transformation is applied for the very first time in a Cryptϵ program and the input predicate v˜[l] = labMult(v˜ 1[l/s2], v˜ 2[l%s2]). is conditioned on a single attribute A ∈ VA, we can directly Only one bit in v˜ for A′ will be encrypted 1 and the others will compute the new bit vector using Ir .A∈VA , i.e., for the ith be encrypted 0s. When merging more than two attributes, ′ É record r in input table T˜ , we have B [i] = v˜ j [l]. This Cryptϵ can use the дenLabMult() described in Section 6.1 to l ∈VA avoids the unnecessary multiplication labMult(B[i], ϕ(r)). speed up computation. Count count(·): To evaluate this operator on its input table Project π ¯(·): The implementation of this operator simply A T˜ , Cryptϵ simply adds up the bits in the corresponding B, drops off all but the attributes in A¯ from the input table T˜ Ém ′ i.e., B[i]. and returns the truncated table T˜ . i GroupByCount γ count (·): The implementation steps for (·) A Filter σϕ : The predicate ϕ in this operator is a conjunction Project, Filter and Count are reused here. First, Cryptϵ projects of range conditions over A¯, defined as: for a row r in input the input tableT˜ on attribute A, i.e.T˜ 1 = πA(T˜ ). Then, Cryptϵ table T˜ , ϕ(r) = Ó ¯ (r.A ∈ V ), where r.A is the value Aj ∈A j Aj j loops each possible value of A. For each value v, Cryptϵ ini- ′ of attribute Aj in row r and VA ⊆ {0, 1,...,sA } (the indices j j tializes a temporary Bv = B and filters T˜ on A = v to get an for attribute values of Aj with domain size sAj ). ′ ′ updated Bv . Last, Cryptϵ counts the number of 1s in Bv and release the counts. Algorithm 1 дenLabMult - generate label for labMult count (·) GroupByCountEncodedγA : The implementation de- Input: c1 = (a1,d1) = labEncpk (m1) and c2 = labEncpk (m2) tail of this operator is given by Algorithm 2. − ( ) − where a1 = m1 b1,d1 = Encpk b1 , a2 = m2 b2,d2 = CountDistinct countD(·): The implementation of this oper- Enc (b ) pk 2 ator involves both AS and CSP. Given the input encrypted Output: e = labEnc (m · m ) pk 1 2 V s V AS: vector of counts of length , the AS first masks to form ′ a new encrypted vector V with a vector of random numbers 1: Computes e = labMult(c1, c2)⊕Encpk (r) where r is a random mask M, i.e., for i ∈ {0, 1,...,s −1}, V[i] = V [i]⊕labEncpk (M[i]). ′ //e corresponds to m1 · m2 − b1 · b2 + r This masked encrypted vector is then sent to CSP and de- ′ 2: Sends e ,d1,d2 to CSP crypted by CSP to a plaintext vector V using the secret CSP: key. ′′ ′ 3: Computes e = Decsk (e ) + Decsk (d1)· Decsk (d2) Next, CSP generates a garbled circuit which takes (i) the ′′ //e corresponds to m1 · m2 + r mask M from the AS, and (ii) the plaintext masked vector ′ ′ ′ ′ ′ 4: Picks a seed σ and label τ and computes b = F(σ ,τ ) V ′ ′′ ′ ′ ′ and a random number r from the CSP as the input. This 5: Sends e¯ = (a¯,d ) to AS, where a¯ = e − b and d = Encpk (b ) circuit first removes the mask M from V to get V and then · − ′ //a¯ corresponds to m1 m2 + r b . counts the number of non-zero entries in V , denoted by c. AS: A masked count c ′ = c + r is outputted by this circuit. CSP 6: Computes true cipher e = (a′,d ′) where a′ = a¯ − r send both the circuit and the encrypted random number labEncpk (r) to AS. and CSP are denoted follows: Π Π View1 (P, D, ϵA) Output1 (P, D, ϵA) Π Π View2 (P, D, ϵA) Output2 (P, D, ϵA) There exists Probabilistic Polynomial Time (PPT) simulators Sim1 and Sim2 such that: • ( CDP (D )) Sim1 PB , ϵA is computationally indistinguishable (≡ ) from (ViewΠ(P, D, ϵ ),Output Π(D, ϵ )), and Figure 6: дenLabMult() - Batching of multiplicands for c 1 A A • ( CDP (D )) ≡ ( Π( D ) Π(D )) labHE Sim2 PB , ϵA is c to View2 P, , ϵA ,Output , ϵ . Π Output (P, D, ϵA)) is the combined output of the two parties ′ Last, the AS evaluates this circuit to the masked count c Proof. Recall that protocol Π consists of two parts; in the c = labEnc (c ′)− and obtains the final output to this operator: pk first part Π1, the AS obtains the sorted encrypted database labEnc (r). pk D˜ s via a garbled circuit. Next Π2 computes F via a Cryptϵ Laplace Lapϵ,∆(V): The implementation of this operator is program. The security of the garbled circuit in Π1 follows presented in the main paper in sec 5.2. from standard approaches [74]. Hence in this section we concentrate on Π . The proof of the entire protocol Π follows NoisyMax NoisyMaxk (·): The input to this operator is an 2 ϵ,∆ from the composition theorem [88,Section 7.3.1]. The views encrypted vector of counts V of size s. Similar to Laplace of the servers for Π are as follows: operator, both AS and CSP are involved. First, the AS adds to 2 Π2 D˜ D˜ V an encrypted Laplace noise vector and a mask M, i.e., for View1 (P, D, ϵA) = (pk, , s , F) Π i ∈ {0, 1,...,s}, Vˆ [i] = V [i] ⊕ labEncpk (ηi ) ⊕ M[i], where 2 View2 (pk,sk, P, D, ϵA) = (F) η ∼ Lap(2 · k · ∆/ϵ). This encrypted noisy, masked vector Vˆ i The simulators Sim (z ) (where z = (y , |D|) is the random is then sent to the CSP. 1 1 1 1 variable distributed as PCDP (D, ϵ ), y being the random The CSP first checks whether Ít ϵ + ϵ ≤ ϵB where ϵ B A 1 i=1 i i variable distributed as PCDP (D, ϵ /2)) performs the follow- represents the privacy budget used for a previously executed A ing steps: program Pi (we presume that a total of t ∈ N programs have been executed hitherto the details of which are logged into (1) Generates a pair of keys (pk1,sk1) for the encryption the CSP’s public ledger). Only in the event the above check scheme and generates random data set D1 of the same D D˜ is satisfied, the CSP proceeds to decrypt Vˆ using the secret size as and encrypts it using pk1 to get 1 D key, i.e., for i ∈ {0, 1,...,s}, Vˆ [i] = labDec (Vˆ [i]). Next the (2) Generates another random dataset 2 of the same size sk D˜ CSP records ϵ and the current program details in the public and encrypts it with pk to get 2. ˜ ˜ ledger. This is followed by the CSP adding another round of The computational indistinguishability of D1 and D fol- ˆ′ ˆ ′ lows directly from the semantic security of the encryption Laplace noise to generate V [i] = V [i] ⊕labEncpk (ηi ), where η′ ∼ Lap(2 · k · ∆/ϵ),i ∈ {0, 1,...,s}. (This is to ensure that scheme. From the construction of the secure sorting algo- i ˜ as long as one of the parties is semi-honest, the output does rithm, it is evident that the records in Ds cannot be as- not violate DP.) Finally, the CSP generates a garbled circuit sociated back with the data owners by the AS. This along which takes (i) the noisy, masked vector Vˆ from the CSP, with the semantic security of the encryption scheme ensures ˜ ˜ and (ii) the mask M from the AS as the input. This circuit that D2 and Ds are computationally indistinguishable as ˜ ˜ will remove the mask from Vˆ to get the noisy counts Vˆ ′ and well. The tuples (pk1, D1, D2,y1) has the same distribution as ˜ ˜ find the indices of thek top- values in Vˆ ′. (pk, D, Ds , F) and hence are computationally indistinguish- Finally, the AS evaluates the circuit above and returns the able. Therefore, Sim1(z1) is computational indistinguishable Π2 indices as the output of this operator. from View1 (P, D, ϵA). For the simulator Sim2(z2) (where z2 = (y2, |D|) is the ran- CDP (D ) D.3 DP Index Optimization Cntd. dom variable distributed according to PB , ϵA , y2 being CDP Lemma 5. Let P be the program that computes the map- the random variable distributed as P (D, ϵA/2)), clearly ping F . Let Π be the protocol corresponding to the construc- tuples (y2) and (F) have identical distribution. Thus, Sim2(z2) tion of the DP index in Cryptϵ. The views and outputs of AS is also computationally indistinguishable from Π2 View2 (P, D, ϵA) thereby concluding our proof. □ 5 120 110 Class III: Other Interaction Programs 4 100 90 The GroupByCountEncoded operator requires an interme- 3 80

70 2 diate interaction with the CSP. The CountDistinct operator 60 1 50 also uses a garbled circuit (details in Appendix D.2 ) and

Mean Absolute Error 40

0 Mean Execution Time30 in s CSP 0 2 4 6 8 0 2 4 6 8 hence requires interactions with the . Therefore, any Number of Neighboring Bins Considered Number of Neighboring Bins Considered program with the above two operators, like P6 and P7 (Ta- (a) (b) ble 2), requires at least two rounds of interaction.

Figure 7: Accuracy and performance of P5 with vary- F ADDITIONAL EVALUATION ing number of neighboring bins considered for the DP In this section we present some additional evaluation results index optimization for Cryptϵ programs.

count ( ˜ ) F.1 DP Index Analysis Cntd. Algorithm 2 GroupByCountEncoded γ˜A T Here we discuss the effect of including neighboring bins Input: T˜ in the program execution. To avoid missing relevant rows, Output: V˜ more bins that are adjacent to the chosen range [i ,i ] can AS: s e be considered for the subsequent operators. We increase 1: Computes V = γ count (T˜ ). A the number of neighbouring bins from 0 to 8. As shown in 2: Masks the encrypted histogram V for attribute A as fol- Figure 7a, the error decreases and all the relevant rows are lows included when 4 neighbouring bins are considered. However, V[i] = V[i] ⊕ labEncpk (M[i]) the execution time naturally increases with extra neighbour-

M[i] ∈R [m],i ∈ [|V |] ing bins as shown in Figure 7b. 3: Sends V to CSP. F.2 Communication Costs CSP: We use Paillier encryption scheme [87] in our prototype 4: Decrypts V as V[i] = labDecsk (V),i ∈ [|V |]. Cryptϵ (Section 9.1). This means that each ciphertext is a 5: Converts each entry of V to its corresponding one-hot- random number in the group (Z/N 2Z)∗ where N is a RSA coding and encrypts it, V˜ [i] = labEnc (V[˜ i]),i ∈ [|V |] pk moduli. Thus sending an encrypted data record entails in 6: Sends V˜ to AS. each data owner sending Í |domain(A )|, where A is an AS: j j j attribute of the database schema, such numbers to the AS. 7: Rotates every entry by its corresponding mask value to Communication is also needed for the measurement oper- ˜ [ ] obtain the desired encrypted one-hot-coding V i . ators and GroupByCountEncoded where the AS needs to V˜ [i] = RiдhtRotate(V˜ , M[i]),i ∈ [|V |] send a ciphertext (or a vector of ciphertexts) to the CSP. Ad- ditionally operators like NoisyMax and CountDistinct need a round of communication for the garbled circuit however E CLASSIFICATION OF CRYPTϵ these circuits are simple and dataset size independent. The PROGRAMS most communication intensive operator is the CrossProduct Cryptϵ programs are grouped into three classes based on the which requires loд2m (Appendix D.1) where m is the dataset number and type of interaction between the AS and the CSP. size rounds of interactions. However, this can be done as Class I: Single Decrypt Interaction Programs a part of pre-processing (Section and hence does not affect ϵ For releasing any result (noisy) in the clear, the AS needs to the actual program execution time. Hence overall, Crypt interact at least once with the CSP (via the two measurement programs are not communication intensive. operators) as the latter has exclusive access to the secret key. G RELATED WORK Cryptϵ programs like P1, P2 and P3 (Table 2) that require only a single interaction of this type fall in this class. G.1 Differential Privacy Class II: LabHE Multiplication Interaction Programs Introduced by Dwork et al. [38], differential privacy has Cryptϵ supports a n-way multiplication of ciphers for n > 2 enjoyed immense attention from both academia and industry as described in Section 6.1 which requires intermediate inter- in the last decade. We will discuss the recent directions in actions with the CSP. Thus all Cryptϵ programs that require two models of differential privacy: the centralized differential multiplication of more than two ciphers need interaction privacy (CDP), and local differential privacy (LDP). with the CSP. Examples include P4 and P5 (Table 2). The CDP model assumes the presence of a trusted server A parallel line of work involves efficient use of crypto- which can aggregate all users’ data before perturb the query graphic primitives for differentially private functionalities. answers. This allows the design of a complex algorithm that Agarwal et al. [9] proposed an algorithm for computing his- releases more accurate query answers than the basic DP togram over encrypted data. Rastogi et al. [92] and Shi et mechanisms. For example, an important line of work in the al. [93] proposed algorithms that allow an untrusted aggre- CDP model has been towards proposing "derived" mecha- gator to periodically estimate the sum of n users’ values in a nisms” [30] or "revised algorithms" [22] from basic DP mech- privacy preserving fashion.However, both schemes are irre- anisms (like exponential mechanism, Laplace mechanism, silient to user failures. Chan et al. [29] tackled this issue by etc.). The design of these mechanisms leverages on specific constructing binary interval trees over the users. properties of the query and the data, resulting in a better utility than the basic mechanisms. One such technique is G.2 Two-Server Model based on data partition and aggregation [8, 33, 58, 89, 90, 101, The two-server model is a popular choice for privacy pre- 102, 107] and is helpful in answering histogram queries. The serving machine learning techniques. Researchers have pro- privacy guarantees of these mechanisms can be ensured via posed privacy preserving ridge regression systems with the the composition theorems and the post-processing property help of a cryptographic service provider [46, 48, 84]. While of differential privacy [38]. We would like to build Cryptϵ the authors in [46] use a hybrid multi-party computation that can support many of these algorithms. scheme with a secure inner product technique, Nikolaenko The notion of LDP and related ideas has been around et al. propose a hybrid approach in [84] by combining ho- for a while [43, 64, 99]. Randomized response proposed by momorphic encryptions and Yao’s garbled circuits. Gascon Warner in 1960s [99] is one of the simplest LDP techniques. et al. [45] extended the results in [84] to include vertically The recent LDP research techniques [13, 42? ] focus on con- partitioned data and the authors in [48] solve the problem structing a frequency oracle that estimates the frequency using just linear homomorphic encryption. Zhang et al. in of any value in the domain. However, when the domain [80] also propose secure machine learning protocols using size is large, it might be computationally infeasible to con- a privacy-preserving stochastic gradient descent method. struct the histogram over the entire domain. To tackle this Their main contribution includes developing efficient algo- challenge, specialized and efficient algorithms have been rithms for secure arithmetic operations on shared decimal proposed to compute heavy hitters [44, 96], frequent item- numbers and proposing alternatives to non-linear functions sets [91, 97], and marginal tables [32, 108]. As the LDP model such as sigmoid and softmax tailored for MPC computations. does not require a trusted data curator, it enjoyed significant In [83] and [67] the authors solve the problem of privacy- industrial adoption, such as Google [42, 44], Apple [51], and preserving matrix factorization. In both the papers, use a Samsung [82]. hybrid approach combining homomorphic encryptions and Recently it has been showed that augmenting randomized Yao’s garbled circuits for their solutions. response mechanism with an additional layer of anonymity in the communication channel can improve the privacy guar- G.3 Homomorphic Encryption antees. The first work to study this was PROCHLO [20] With improvements made in implementation efficiency and implementation by Google. PROCHLO necessitates this in- new constructions developed in the recent past, there has termediary to be trusted, this is implemented via trusted been a surge in practicable privacy preserving solutions em- hardware enclaves (Intel’s SGX). However, as showcased ploying homomorphic encryptions. A lot of the aforemen- by recent attacks [94], it is notoriously difficult to design a tioned two-server models employ homomorphic encryption truly secure hardware in practice. Motivated by PROCHLO, [48, 67, 83, 84]. In [25, 50, 60] the authors enable neural net- the authors in [41], present a tight upper-bound on the works to be applied to homomorphic-ally encrypted data. worst-case privacy loss. Formally, they show that any per- Linear homomorphic encryption is used in [49] to enable mutation invariant algorithm satisfying ϵ-LDP will satisfy privacy-preserving machine learning for ensemble meth- q ( 1 ) log δ ods while uses fully-homomorphic encryption to approxi- O(ϵ n , δ)-CDP, where n is the data size. Cheu et al. [31] demonstrate privacy amplification by the same factor for1- mate the coefficients of a logistic-regression model. [23] uses bit randomized response by using a mixnet architecture to somewhat- homomorphic encryption scheme to compute provide the anonymity. This work also proves another im- the forecast prediction of consumer usage for smart grids. portant result that the power of the mixnet model lies strictly between those of the central and local models.