On the Security of Lattice-Based Cryptography Against Lattice Reduction and Hybrid Attacks Vom Fachbereich Informatik der Technischen Universit¨atDarmstadt genehmigte Dissertation zur Erlangung des Grades Doktor rerum naturalium (Dr. rer. nat.) von Dipl.-Ing. Thomas Wunderer geboren in Augsburg. Referenten: Prof. Dr. Johannes Buchmann Dr. Martin Albrecht Tag der Einreichung: 08. 08. 2018 Tag der m¨undlichen Pr¨ufung: 20. 09. 2018 Hochschulkennziffer: D 17 Wunderer, Thomas: On the Security of Lattice-Based Cryptography Against Lattice Reduction and Hybrid Attacks Darmstadt, Technische Universit¨atDarmstadt Jahr der Ver¨offentlichung der Dissertation auf TUprints: 2018 Tag der m¨undlichen Pr¨ufung:20.09.2018 Ver¨offentlicht unter CC BY-SA 4.0 International https://creativecommons.org/licenses/ Abstract Over the past decade, lattice-based cryptography has emerged as one of the most promising candidates for post-quantum public-key cryptography. For most current lattice-based schemes, one can recover the secret key by solving a corresponding instance of the unique Shortest Vector Problem (uSVP), the problem of finding a shortest non-zero vector in a lattice which is unusually short. This work is concerned with the concrete hardness of the uSVP. In particular, we study the uSVP in general as well as instances of the problem with particularly small or sparse short vectors, which are used in cryptographic constructions to increase their efficiency. We study solving the uSVP in general via lattice reduction, more precisely, the Block-wise Korkine-Zolotarev (BKZ) algorithm. In order to solve an instance of the uSVP via BKZ, the applied block size, which specifies the BKZ algorithm, needs to be sufficiently large. However, a larger block size results in higher runtimes of the algorithm. It is therefore of utmost interest to determine the minimal block size that guarantees the success of solving the uSVP via BKZ. In this thesis, we provide a theoretical and experimental validation of a success condition for BKZ when solving the uSVP which can be used to determine the minimal required block size. We further study the practical implications of using so-called sparsification techniques in combination with the above approach. With respect to uSVP instances with particularly small or sparse short vectors, we investigate so-called hybrid attacks. We first adapt the \hybrid lattice reduction and meet-in-the-middle attack" (or short: the hybrid attack) by Howgrave-Graham on the NTRU encryption scheme to the uSVP. Due to this adaption, the attack can be applied to a larger class of lattice-based cryptosystems. In addition, we enhance the runtime analysis of the attack, e.g., by an explicit calculation of the involved success probabilities. As a next step, we improve the hybrid attack in two directions as described in the following. To reflect the potential of a modern attacker on classical computers, we show how to parallelize the attack. We show that our parallel version of the hybrid attack scales well within realistic parameter ranges. Our theoretical analysis is supported by practical experiments, using our implementation of the parallel hybrid attack which employs Open Multi-Processing and the Message Passing Interface. iii Abstract To reflect the power of a potential future attacker who has access to a large-scale quantum computer, we develop a quantum version of the hybrid attack which replaces the classical meet-in-the-middle search by a quantum search. Not only is the quantum hybrid attack faster than its classical counterpart, but also applicable to a wider range of uSVP instances (and hence to a larger number of lattice-based schemes) as it uses a quantum search which is sensitive to the distribution on the search space. Finally, we demonstrate the practical relevance of our results by using the tech- niques developed in this thesis to evaluate the concrete security levels of the lattice- based schemes submitted to the US National Institute of Standards and Technology's process of standardizing post-quantum public-key cryptography. iv Publications Publications used in this thesis [1] Johannes A. Buchmann, Florian G¨opfert,Rachel Player, and Thomas Wun- derer. On the Hardness of LWE with Binary Error: Revisiting the Hybrid Lattice-Reduction and Meet-in-the-Middle Attack. In: Progress in Cryptology - AFRICACRYPT 2016 - 8th International Conference on Cryptology in Africa, Fes, Morocco, April 13-15, 2016, Proceedings. 2016, pp. 24-43. [2] Thomas Wunderer. A Detailed Analysis of the Hybrid Lattice-Reduction and Meet-in-the-Middle Attack. In: Journal of Mathematical Cryptology, to appear. [3] Florian G¨opfert,Christine van Vredendaal, and Thomas Wunderer. A Hybrid Lattice Basis Reduction and Quantum Search Attack on LWE. In: Post- Quantum Cryptography - 8th International Workshop, PQCrypto 2017, Utrecht, The Netherlands, June 26-28, 2017, Proceedings. 2017, pp. 184-202. [4] Martin R. Albrecht, Florian G¨opfert,Fernando Virdia, and Thomas Wunderer. Revisiting the Expected Cost of Solving uSVP and Applications to LWE. In: Advances in Cryptology - ASIACRYPT 2017 { 23rd International Conference on the Theory and Applications of Cryptology and Information Security, Hong Kong, China, December 3-7, 2017, Proceedings, Part I. 2017, pp. 297-322. [5] Martin R. Albrecht, Benjamin R. Curtis, Amit Deo, Alex Davidson, Rachel Player, Eamonn W. Postlethwaite, Fernando Virdia, and Thomas Wunderer. Estimate all the fLWE, NTRUg schemes!. In: Security and Cryptography for Networks { 11th International Conference, SCN 2018, Amalfi, Italy, September 5 - September 7, 2018, Proceedings. Lecture Notes in Computer Science, Springer 2018, to appear. [6] Yuntao Wang and Thomas Wunderer. Revisiting the Sparsification Technique in Kannan's Embedding Attack on LWE. In: Information Security Practice and Experience { 14th International Conference, ISPEC 2018, Tokyo, Japan, v Publications September 25-27, 2018, Proceedings. Lecture Notes in Computer Science, Springer 2018, to appear. [7] Martin R. Albrecht, Benjamin R. Curtis, and Thomas Wunderer. An Explo- ration of the Hybrid Attack on Small-secret LWE. Work in progress. [8] Thomas Wunderer, Michael Burger, and Giang Nam Nguyen. Parallelizing the Hybrid Lattice Reduction and Meet-in-the-Middle Attack. In: CSE-2018 { 21st IEEE International Conference on Computational Science and Engineering, Bucharest, Romania, October 29 - 31, 2018, to appear. Other publications [9] Patrick Holzer, Thomas Wunderer, and Johannes A. Buchmann. Recovering Short Generators of Principal Fractional Ideals in Cyclotomic Fields of Conduc- tor pαqβ. In: Progress in Cryptology - INDOCRYPT 2017 - 18th International Conference on Cryptology in India, Chennai, India, December 10-13, 2017, Proceedings. 2017, pp. 346-368. [10] Michael Burger, Christian Bischof, Alexandru Calotoiu, Thomas Wunderer, and Felix Wolf. Exploring the Performance Envelope of the LLL Algorithm. In: CSE-2018 { 21st IEEE International Conference on Computational Science and Engineering, Bucharest, Romania, October 29 - 31, 2018, to appear. vi Contents Abstract iii Publicationsv 1 Introduction1 1.1 Contribution and Organization . .2 2 Background7 2.1 Notation . .7 2.2 Lattices and Lattice Bases . .8 2.3 Lattice Problems . 10 2.3.1 Shortest Vector Problems . 10 2.3.2 Closest Vector Problems . 10 2.3.3 Learning with Errors . 10 2.3.4 NTRU . 12 2.4 Lattice Algorithms . 13 2.4.1 Runtime Estimates . 13 2.4.2 Lattice Reduction . 13 2.4.3 SVP Algorithms . 15 2.4.4 Kannan's Embedding Technique . 16 2.4.5 Babai's Nearest Plane . 16 2.4.6 Other Lattice Algorithms and Attacks . 17 3 On the Expected Cost of Solving uSVP via Lattice Reduction 19 3.1 Estimates . 20 3.1.1 2008 Estimate . 20 3.1.2 2016 Estimate . 22 3.2 Solving uSVP . 22 3.2.1 Prediction . 23 3.2.2 Observation . 24 3.2.3 Explaining Observation . 31 vii Contents 3.3 Applications . 35 3.3.1 Bai and Galbraith's embedding . 36 3.4 Security Estimates . 37 3.4.1 Lizard . 37 3.4.2 HElib . 37 3.4.3 SEAL . 38 3.4.4 TESLA . 38 3.4.5 BCIV17 . 38 4 On the Use of Sparsification when Embedding BDD into uSVP 43 4.1 The Sparsified Embedding Attack . 44 4.2 Analysis . 45 4.2.1 Heuristics for Kannan's Embedding . 46 4.2.2 Heuristics for the Sparsified Embedding . 47 4.2.3 Comparison . 48 5 Revisiting the Hybrid Lattice Reduction and Meet-in-the-Middle Attack 51 5.1 Tools for q-ary Lattices . 52 5.1.1 Constructing a Suitable Basis for the Hybrid Attack . 52 5.1.2 Modifying the GSA for q-ary Lattices . 53 5.2 The Hybrid Attack . 55 5.3 Analysis . 58 5.3.1 Runtime Analysis . 59 5.3.2 Determining the Success Probability. 67 5.3.3 Optimizing the Runtime . 68 5.4 Security Estimates Against the Hybrid Attack . 71 5.4.1 NTRU . 72 5.4.2 NTRU prime . 75 5.4.3 R-BinLWEEnc . 77 5.4.4 BLISS . 79 5.4.5 GLP . 82 6 Parallelizing the Hybrid Lattice Reduction and Meet-in-the-Middle At- tack 85 6.1 The Hybrid Attack on Binary LWE . 86 6.2 Parallelizing the Hybrid Attack . 86 6.2.1 Running Multiple Instances in Parallel . 88 6.2.2 Using Parallel BKZ . 88 6.2.3 Parallel Meet-in-the-Middle Search . 89 6.2.4 Runtime Analysis . 89 6.3 Experiments and Results . 93 6.3.1 Our Implementation . 98 viii Contents 6.3.2 Test Environment . 100 6.3.3 Test Cases . 100 6.3.4 Reducing the Runtime of the Meet-in-the-Middle Phase of the Attack . 101 6.3.5 Reducing the Overall Runtime of the Attack . 103 6.3.6 Analysis of the Hybrid Efficiency . 104 7 The Hybrid Lattice Reduction and Quantum Search Attack 107 7.1 The Quantum Hybrid Attack . 108 7.1.1 Amplitude Amplification . 108 7.1.2 The Attack . 109 7.2 Analysis .