Noname manuscript No. (will be inserted by the editor) Efficient Number Theoretic Transform Implementation on GPU for Homomorphic Encryption Ozg¨un¨ Ozerk¨ · Can Elgezen · Ahmet Can Mert · Erdin¸c Ozt¨urk¨ · Erkay Sava¸s Abstract Lattice-based cryptography forms the mathematical basis for ho- momorphic encryption, which allows computation directly on encrypted data. Homomorphic encryption enables privacy-preserving applications such as se- cure cloud computing; yet, its practical applications suffer from the high com- putational complexity of homomorphic operations. Fast implementations of the homomorphic encryption schemes heavily depend on efficient polynomial arithmetic; multiplication of very large degree polynomials over polynomial rings, in particular. Number theoretic transform (NTT) accelerates polyno- mial multiplication significantly and therefore, it is the core arithmetic op- eration in the majority of homomorphic encryption scheme implementations. Therefore, practical homomorphic applications require efficient and fast imple- mentations of NTT in different computing platforms. In this work, we present an efficient and fast implementation of NTT, inverse NTT (INTT) and NTT- based polynomial multiplication operations for GPU platforms. To demon- strate that our GPU implementation can be utilized as an actual accelerator, we experimented with the key generation, the encryption and the decryption operations of the Brakerski/Fan-Vercauteren (BFV) homomorphic encryption scheme implemented in Microsoft's SEAL homomorphic encryption library on GPU, all of which heavily depend on the NTT-based polynomial multiplica- tion. Our GPU implementations improve the performance of these three BFV operations by up to 141.95×, 105.17× and 90.13×, respectively, on Tesla v100 GPU compared to the highly-optimized SEAL library running on an Intel i9-7900X CPU. Keywords Lattice-based Cryptography · Homomorphic Encryption · SEAL · Number Theoretic Transform · Polynomial Multiplication · GPU · CUDA This work is supported by TUB¨ ITAK_ under Grant Number 118E725. O.¨ Ozerk,¨ C. Elgezen, A. C. Mert, E. Ozt¨urk,E.¨ Sava¸s Faculty of Engineering and Natural Sciences, Sabanci University, Istanbul, Turkey E-mail: fozgunozerk, celgezen, ahmetcanmert, erdinco, [email protected] 2 Ozg¨un¨ Ozerk¨ et al. 1 Introduction Lattice-based cryptography is conjectured to be secure against attacks from quantum computers and thus supports post-quantum cryptography (PQC). Also, it provides the mathematical basis for fully homomorphic encryption (FHE) schemes, as demonstrated by Gentry in 2009 [20]. FHE allows com- putation on the encrypted data requiring neither decryption nor secret key, and therefore, enables secure processing of sensitive data. FHE offers a va- riety of applications ranging from private text classification to secure cloud computing [2]. Since Gentry's breakthrough, homomorphic encryption has gained tremen- dous amount of attention and different homomorphic encryption schemes are proposed in the literature such as Brakerski-Gentry-Vaikuntanathan (BGV) [12], Brakerski/Fan-Vercauteren (BFV) [18] and Cheon-Kim-Kim-Song (CKKS) [14]. There are also various efforts for developing their practical implementations. As such, there are different open-source and highly optimized software libraries such as SEAL [39], HElib [24] and PALISADE [34] for homomorphic encryp- tion and computation. The SEAL library is developed by Microsoft Research and it supports the BFV and the CKKS schemes. HElib supports the BGV and the CKKS schemes while PALISADE supports the BGV, the BFV and the CKKS schemes. Although the potential applications of FHE are of groundbreaking nature, its high algorithmic complexity is a standing impediment for efficient and practical implementations thereof. Among different core arithmetic operations in various FHE schemes, multiplication over polynomial rings is probably the most time-consuming. Therefore, there are different methods in the literature proposed for the efficient implementation of multiplication of two very large degree polynomials over polynomial ring Rq;n, where n and q represent the degree of polynomials in the ring and coefficient modulus, respectively. The Toom-Cook [42] or Karatsuba [25] multiplications have been methods in use for a long time and generally utilized in schemes with polynomial rings, for which NTT is not suitable [31]. NTT-based polynomial multiplication is, on the other hand, highly utilized in lattice-based cryptosystems and it reduces the O(n2) computational complexity of the schoolbook polynomial multiplication to O(n · log n) [15]. Although utilizing NTT improves the performance of polynomial multipli- cation operation, it is still inefficient for real life applications. Therefore, there are different NTT-based polynomial multiplication implementations proposed in the literature for efficient and practical lattice-based cryptosystems on dif- ferent platforms: hardware architectures [37,32,41,36], software implementa- tions [40,24,1,39], and implementations on GPUs [16,3,6,4,28,22,27,23,44, 5]. There are also hybrid approaches combining NTT-based and Karatsuba multiplication methods for the polynomial multiplication operation [7]. The NTT-based polynomial multiplication operation can be performed for a range of parameters n and q in different applications. For example, FHE applica- tions require usually large n and q parameters while PQC utilizes smaller Title Suppressed Due to Excessive Length 3 parameters. Therefore, an efficient implementation of NTT-based polynomial multiplication requires flexibility of supporting both FHE and PQC in addition to offering high-performance. With a similar motivation of the works in the literature, we propose efficient NTT, INTT and NTT-based polynomial multiplication implementations on GPU in this work. The proposed implementations on GPU support a wide range of polynomial rings. The proposed implementations can perform a single NTT and INTT operations in 39µs and 23µs, respectively, for the largest ring with n = 32768 and log2(q) = 61 in Tesla V100 GPU including overhead of kernel calls. In order to show that the proposed GPU implementations can be useful as actual accelerators in the homomorphic encryption schemes, for proof of con- cept, the proposed implementations are utilized to implement and accelerate the key generation, the encryption and the decryption operations of the BFV homomorphic encryption scheme on GPU. For a quick recap, our contributions are listed as follows: 1. We present high-performance and efficient GPU implementations for NTT, INTT and NTT-based polynomial multiplication operations. The proposed GPU implementations support polynomials of degrees ranging from 2048 to 32768 with 30-bit and 61-bit coefficients1. We run the implementations on three different GPU platforms, Nvidia GTX 980, Nvidia GTX 1080, Nvidia Tesla V100; and a single NTT operation for polynomials of degree 32768 with 61-bit coefficients is performed in 73µs, 36µs, 39µs on Nvidia GTX 980, Nvidia GTX 1080 and Nvidia Tesla V100, respectively. 2. In order to parallelize the NTT and INTT operations on GPU, we modi- fied their algorithmic structures significantly, which consist of three nested for loops and removed the dependencies in between as much as possible. A GPU block can process a block of 2048 array elements in NTT oper- ation and our implementations benefit from the fast shared memory for polynomials of moderate degrees (i.e., n = 8192 for 30-bit modulus and n = 4096 for 61-bit modulus). For larger polynomial degrees, global mem- ory is used. To support large polynomial degrees in our implementations, we propose a hybrid approach, whereby we make a separate kernel call for each NTT iteration when the NTT block is large; but as soon as the NTT block becomes sufficiently small we switch to a different working mode, in which the rest of the NTT iterations are completed in a single kernel call. The hybrid approach achieves up to 9× speed up for different polynomial degrees compared to baseline design, which performs log2(n) kernel calls for one NTT operation. 3. The key generation, encryption and decryption operations of the BFV scheme are fully implemented on GPU and, compared to the BFV im- plementation on Microsoft's SEAL library running on an Intel i9-7900X CPU; and we observed up to 60.31×, 43.84×, 33.89× speed-up values on Nvidia GTX 980; 56.07×, 40.85×, 25.05× speed-up values on Nvidia GTX 1 A sample code is available at https://github.com/SU-CISEC/gpu-ntt 4 Ozg¨un¨ Ozerk¨ et al. 1080; and 141.95×, 105.17×, 90.13× speed-up values on Nvidia Tesla V100, for key generation, encryption and decryption operations, respectively. 4. Since key generation and encryption operations of the BFV scheme require random polynomials from uniform, ternary and discrete Gaussian distribu- tions, we also introduced an implementation of random polynomial sampler for these distributions on GPU. We utilized Salsa20 implementation [21] for pseudo-random number generation for uniform distribution and inverse cumulative standard distribution function normcdfinvf in CUDA Math API. The rest of the paper is organized as follows. Section 2 presents the nota- tion, the background on NTT and polynomial multiplication operations and summarizes prior works in the literature. Section 3 introduces SEAL library. Section 4 presents the proposed GPU implementations. Section 5 presents the results and Section 6 concludes the paper. 2 Background In this section, we present the notation used in the rest of the paper, brief descriptions of NTT, INTT,