A preliminary version of this paper appears in Advances in Cryptology - CRYPTO ’07 Proceedings. Lecture Notes in Computer Science, Vol. 4622, pp. 535–552, A. Menezes ed., Springer, 2007. This is the full version. Deterministic and Efficiently Searchable Encryption Mihir Bellare∗ Alexandra Boldyreva† Adam O’Neill‡ Abstract We present as-strong-as-possible definitions of privacy, and constructions achieving them, for public-key encryption schemes where the encryption algorithm is deterministic. We obtain as a consequence database encryption methods that permit fast (i.e. sub-linear, and in fact logarithmic, time) search while provably providing privacy that is as strong as possible subject to this fast search constraint. One of our constructs, called RSA-DOAEP, has the added feature of being length preserving, so that it is the first example of a public-key cipher. We generalize this to obtain a notion of efficiently-searchable encryption schemes which permit more flexible privacy to search-time trade-offs via a technique called bucketization. Our results answer much- asked questions in the database community and provide foundations for work done there. Keywords: Public-key encryption, deterministic encryption, searchable encryption, database se- curity. ∗Dept. of Computer Science & Engineering, University of California at San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA. E-Mail: [email protected]. URL: http://www-cse.ucsd.edu/users/mihir. Supported in part by NSF grants CNS-0524765, CNS-0627779, and a gift from Intel Corporation. †School of Computer Science, College of Computing, Georgia Institute of Technology, 266 Ferst Drive, Atlanta, GA 30332, USA. E-mail: [email protected]. URL: http://www.cc.gatech.edu/∼ aboldyre. Supported in part by NSF CAREER award 0545659. ‡School of Computer Science, College of Computing, Georgia Institute of Technology, 266 Ferst Drive, Atlanta, GA 30332, USA. E-mail: [email protected]. URL: http://www.cc.gatech.edu/∼ amoneill. Supported in part by the grant of the second author. 1 Contents 1 Introduction 3 2 Notation and Conventions 6 3 Deterministic Encryption and its Security 7 4 A Useful Fact 9 5 Secure Deterministic Encryption Schemes 10 5.1 Encrypt-with-Hash ............................... ...... 10 5.2 RSA-DOAEP, A length-preserving deterministic scheme . ............... 11 6 Efficiently Searchable Encryption (ESE) 13 6.1 Encrypt-and-HashESE ............................. ..... 14 7 CCA and Other Extensions 16 8 Acknowledgments 20 A Proof of Theorem 5.2 23 B Proof of Theorem 6.1 27 C Proof of Theorem 6.2 29 D Proof of Theorem 7.1 34 2 1 Introduction The classical notions of privacy for public-key encryption schemes, namely indistinguishability or semantic security under chosen-plaintext or chosen-ciphertext attack [35, 44, 47, 28, 10], can only be met when the encryption algorithm is randomized. This paper treats the case where the encryption algorithm is deterministic. We begin by discussing the motivating application. Fast search. Remote data storage in outsourced databases is of increasing interest [51]. Data will be stored in encrypted form. (The database service provider is not trusted.) We are interested in a public key setting, where anyone can add to the database encrypted data which a distinguished “receiver” can retrieve and decrypt. The encryption scheme must permit search (by the receiver) for data retrieval. Public-key encryption with keyword search (PEKS) [16, 1, 18] is a solution that provably provides strong privacy but search takes time linear in the size of the database. Given that databases can be terabytes in size, this is prohibitive. The practical community indicates that they want search on encrypted data to be as efficient as on unencrypted data, where a record containing a given field value can be retrieved in time logarithmic in the size of the database. (For example, via appropriate tree-based data structures.) Deterministic encryption allows just this. The encrypted fields can be stored in the data structure, and one can find a target ciphertext in time logarithmic in the size of the database. The question is what security one can expect. To answer this, we need a definition of privacy for deterministic encryption. A definition. One possibility is to just ask for one-wayness, but we would like to protect partial information about the plaintext to the maximum extent possible. To gauge what this could be, we note two inherent limitations of deterministic encryption. First, no privacy is possible if the plaintext is known to come from a small space. Indeed, knowing that c is the encryption under public key pk of a plaintext x from a set X, the adversary can compute the encryption cx of x under pk for all x ∈ X, and return as the decryption of c the x satisfying cx = c. We address this by only requiring privacy when the plaintext is drawn from a space of large min-entropy. Second, and more subtle, is that the ciphertext itself is partial information about the plaintext. We address this by only requiring non-leakage of partial information when the plaintext and partial information do not depend on the public key. This is reasonable because in real life public keys are hidden in our software and data does not depend on them. We provide a semantic-security style definition of privacy for deterministic encryption that takes these issues into account. While certainly weaker than the classical notions met by randomized schemes, our notion, which we call PRIV, is still quite strong. The next question is how to achieve this new notion. Constructions. Our first construction is generic and natural: Deterministically encrypt plaintext x by applying the encryption algorithm of a randomized scheme but using as coins a hash of (the public key and) x. We show that this “Encrypt-with-Hash” deterministic encryption scheme is PRIV secure in the random oracle (RO) model of [12] assuming the starting randomized scheme is IND-CPA secure. Our second construction is an extension of RSA-OAEP [13, 31]. The padding transform is deterministic but uses three Feistel rounds rather than the two of OAEP. RSA-DOAEP is proven PRIV secure in the RO model assuming RSA is one-way. This construction has the attractive feature of being length-preserving. (The length of the ciphertext equals the length of the plaintext.) This is important when bandwidth is expensive —senders in the database setting could be power-constrained devices— and for securing legacy code. Historical context. Diffie and Hellman [26] suggested that one encrypt plaintext x by applying to it an injective trapdoor function. A deterministic encryption scheme is just a family of injective 3 trapdoor functions, so our definition is an answer to the question of how much privacy Diffie- Hellman encryption can provide. (We clarify that not all trapdoor functions meet our definition. For example, plain RSA does not.) In the symmetric setting, deterministic encryption is captured by ciphers including block ci- phers. So far there has been no public key analog. Deterministic encryption meeting our definition provides one, and in particular RSA-DOAEP is the first length-preserving public-key cipher. Efficiently searchable encryption. We introduce the notion of efficiently searchable encryp- tion (ESE) schemes. These are schemes permitting fast (i.e. logarithmic time) search. Encryption may be randomized, but there is a deterministic function of the plaintext that can also be com- puted from the ciphertext and serves as a “tag,” permitting the usual (fast) comparison-based search. Deterministic encryption schemes are a special case and the notion of security remains the same. (Our PRIV definition does not actually require encryption to be deterministic.) The bene- fit of the generalization is to permit schemes with more flexible privacy to search-time trade-offs. Specifically, we analyze a scheme from the database literature that we call “Hash-and-Encrypt.” It encrypts the plaintext with a randomized scheme but also includes in the ciphertext a determin- istic, collision-resistant hash of the plaintext. (This is an ESE scheme with the hash playing the role of the tag, and so permits fast search.) We prove that this scheme is PRIV secure in the RO model when the underlying encryption scheme is IND-CPA. With this scheme, loss of privacy due to lack of entropy in the plaintext space can be compensated for by increasing the probability of hash collisions. (This can be done, for example, by using truncated output of the hash function.) The trade-off is that the receiver then also gets “false positives” in response to a search query and must spend the time to sift through them to obtain the true answer. This technique is known as bucketization in the database literature, but its security was not previously rigorously analyzed. Discussion. Our schemes only provide privacy for plaintexts that have high min-entropy. (This is inherent in being deterministic or efficiently searchable, not a weakness of our particular constructs.) We do not claim database fields being encrypted have high min-entropy. They might or they might not. The point is that practitioners have indicated that they will not sacrifice search time for privacy. Our claim is to provide the best possible privacy subject to allowing fast search. In some cases, this may very well mean no privacy. But we also comment that bucketization can increase privacy (at the cost of extra processing by the receiver) when the database fields being encrypted do not have high min-entropy. Extensions. Our basic PRIV definition, and the above-mentioned results, are all for the CPA (chosen-plaintext attack) case. The definition easily extends to the CCA (chosen-ciphertext attack) case, and we call the resulting notion PRIV-CCA. Our Encrypt-with-Hash deterministic encryption scheme is not just PRIV, but in fact PRIV-CCA, in the RO model even if the underlying randomized encryption scheme is only IND-CPA, as long as the latter has the extra property that no ciphertext is too likely.