Cryptanalysis of Public Key Cryptosystems Abderrahmane Nitaj To cite this version: Abderrahmane Nitaj. Cryptanalysis of Public Key Cryptosystems. Cryptography and Security [cs.CR]. Université de Caen Normandie, 2016. tel-02321087 HAL Id: tel-02321087 https://hal-normandie-univ.archives-ouvertes.fr/tel-02321087 Submitted on 20 Oct 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. MEMOIRE D'HABILITATION A DIRIGER DES RECHERCHES Sp´ecialit´e: Math´ematiques Pr´epar´eau sein de l'Universit´ede Caen Normandie Cryptanalysis of Public Key Cryptosystems Pr´esent´eet soutenu par Abderrahmane NITAJ Soutenu publiquement le 1er d´ecembre 2016 devant le jury compos´ede Mr Thierry BERGER Professeur, Universit´ede Limoges Rapporteur Mr Marc GIRAULT Chercheur HDR, Orange Labs, Caen Examinateur Mr Marc JOYE Chercheur HDR, NXP Semiconductors Rapporteur Mr Fabien LAGUILLAUMIE Professeur, Universit´ede Lyon 1 Rapporteur Mr Denis SIMON Professeur, Universit´ede Caen Examinateur Mme Brigitte VALLEE Directrice de Recherche, CNRS Examinatrice M´emoirepr´epar´eau Laboratoire de Math´ematiquesNicolas Oresme (LMNO) Contents Remerciements ix List of Publications xi 1 Introduction 1 2 Cryptanalysis of RSA 9 2.1 Introduction . .9 2.2 Continued Fractions . 10 2.3 Lattice Reduction . 12 2.4 Coppersmith's Method . 14 2.5 Attacks on RSA Using a Variant of the Key Equation . 16 2.5.1 An Attack for Small Difference jap − bqj ..................... 18 2.5.2 An Attack for Medium Difference jap − bqj ................... 18 2.5.3 An Attack for Large Difference jap − bqj ..................... 19 2.6 An Attack on RSA Unbalanced Moduli . 20 2.6.1 Implicit factorization of two RSA Moduli . 21 2.6.2 Implicit factorization of k RSA Moduli . 22 3 Cryptanalysis of Variants of RSA 25 3.1 Cryptanalysis of KMOV . 25 3.2 Cryptanalysis of Demytko's cryptosystem . 29 3.3 Cryptanalysis of Some RSA Type cryptosystems . 32 4 Cryptanalysis of NTRU 35 4.1 Introduction . 35 i ii CONTENTS 4.2 Description of NTRU . 36 4.3 The attack of Coppersmith and Shamir on NTRU . 37 4.4 An attack of NTRU with two public keys: Case 1 . 39 4.5 An attack of NTRU with two public keys: Case 2 . 40 5 Cryptanalysis of the DGHV Cryptosystem 43 5.1 Introduction . 43 5.2 Description of the Parameters in DGHV . 44 5.3 The First Proposed attack on DGHV . 46 5.4 The Second Proposed attack on DGHV . 47 Appendices 49 A Another Generalization of Wiener's Attack on RSA 55 A.1 Introduction . 56 A.2 Preliminaries . 57 A.2.1 Continued fractions and Wiener's attack . 57 A.2.2 Coppersmith's method . 58 A.2.3 Smooth numbers . 59 A.2.4 ECM . 60 A.3 Useful lemmas . 61 A.4 Properties of (u; v).................................... 63 A.5 The new attack . 66 A.6 The number of exponents for the new method . 70 A.7 Conclusion . 74 B Cryptanalysis of RSA Using the Ratio of the Primes 77 B.1 Introduction . 78 B.2 Preliminaries on Continued Fractions, Coppersmith's Method and The Elliptic Curve Method (ECM) . 80 B.2.1 Continued Fractions and the Euclidean Algorithm . 80 B.2.2 Coppersmith's Method . 82 B.2.3 The Elliptic Curve Method of Factorization . 82 B.3 Useful Lemmas and Properties . 83 CONTENTS iii B.4 The New Attacks on RSA . 85 B.4.1 An Attack for Small Difference jap − bqj ..................... 86 B.4.2 An Attack for Medium Difference jap − bqj ................... 87 B.4.3 An Attack for Large Difference jap − bqj ..................... 89 B.5 Estimation of the Public Exponents for which the Attacks Apply . 92 B.6 Conclusion . 98 C A New Attack on RSA with Two or Three Decryption Exponents 99 C.1 Introduction . 100 C.2 Former Attacks . 101 C.2.1 Guo's attack for two exponents . 101 C.2.2 Guo's attack for three exponents . 102 C.2.3 The Bl¨omerand May attack . 103 C.3 Useful Lemmas . 104 C.4 The New Attack on RSA with Two Exponents . 107 C.5 The New Attack on RSA with Three Exponents . 110 C.6 Conclusion . 112 D An Attack on RSA Using LSBs of Multiples of the Prime Factors 113 D.1 Introduction . 114 D.2 Preliminaries . 116 D.2.1 Lattices . 116 D.2.2 Useful Lemmas . 117 D.3 The New Attack . 118 D.4 Experimental Results . 126 D.5 Conclusion . 128 E Implicit Factorization of Unbalanced RSA Moduli 129 E.1 Introduction . 130 E.2 Preliminaries . 136 E.2.1 Continued fractions . 136 E.2.2 Lattice reduction . 137 E.3 Factoring two RSA Moduli in the MSB Case . 137 E.3.1 The general attack for two RSA Moduli in the MSB Case . 138 iv CONTENTS E.3.2 Application to unbalanced RSA and RSA for Paranoids . 139 E.4 Factoring k RSA Moduli in the MSB Case . 140 E.5 Factoring Two RSA Moduli in the LSB Case . 142 E.5.1 The general attack . 142 E.5.2 Application to unbalanced RSA and RSA for Paranoids . 143 E.6 Factoring k RSA Moduli in the LSB Case . 144 E.7 Experiments . 146 E.8 Conclusion . 147 F Factoring RSA Moduli with Weak Prime Factors 149 F.1 Introduction . 150 F.2 Preliminaries . 152 F.2.1 Integer factorization: the state of the art . 152 F.2.2 Lattice reduction . 153 F.2.3 Coppersmith's Method . 154 F.3 The Attack with One Weak Prime Factor . 155 F.3.1 The Attack . 155 F.3.2 Numerical Examples . 156 F.3.3 The Number of Single Weak Primes in an Interval . 158 F.4 The Attack with Two Weak Prime factors . 161 F.4.1 The Attack . 161 F.4.2 Examples . 163 F.4.3 The Number of Double Weak Primes in an Interval . 164 F.5 Conclusions . 165 G New attacks on RSA with Moduli N = prq 167 G.1 Introduction . 168 G.2 Preliminaries . 170 G.2.1 Linear Modular Polynomial Equations . 170 G.2.2 The Continued Fractions Algorithm . 171 G.3 The First Attack on Prime Power RSA with Modulus N =.