Applications of Search Techniques to Cryptanalysis and the Construction of Cipher Components. James David Mclaughlin Submitted F
Total Page:16
File Type:pdf, Size:1020Kb
Applications of search techniques to cryptanalysis and the construction of cipher components. James David McLaughlin Submitted for the degree of Doctor of Philosophy (PhD) University of York Department of Computer Science September 2012 2 Abstract In this dissertation, we investigate the ways in which search techniques, and in particular metaheuristic search techniques, can be used in cryptology. We address the design of simple cryptographic components (Boolean functions), before moving on to more complex entities (S-boxes). The emphasis then shifts from the construction of cryptographic arte- facts to the related area of cryptanalysis, in which we first derive non-linear approximations to S-boxes more powerful than the existing linear approximations, and then exploit these in cryptanalytic attacks against the ciphers DES and Serpent. Contents 1 Introduction. 11 1.1 The Structure of this Thesis . 12 2 A brief history of cryptography and cryptanalysis. 14 3 Literature review 20 3.1 Information on various types of block cipher, and a brief description of the Data Encryption Standard. 20 3.1.1 Feistel ciphers . 21 3.1.2 Other types of block cipher . 23 3.1.3 Confusion and diffusion . 24 3.2 Linear cryptanalysis. 26 3.2.1 The attack. 27 3.3 Differential cryptanalysis. 35 3.3.1 The attack. 39 3.3.2 Variants of the differential cryptanalytic attack . 44 3.4 Stream ciphers based on linear feedback shift registers . 48 3.5 A brief introduction to metaheuristics . 52 3.5.1 Hill-climbing . 55 3.5.2 Simulated annealing . 57 3.5.3 Memetic algorithms . 58 3.5.4 Ant algorithms . 64 3.5.5 Metaheuristics in cryptography . 67 3.6 Cryptanalysis with metaheuristic search. 69 3.6.1 Metaheuristic attacks on knapsack ciphers . 69 2 3.6.2 The Permuted Perceptron Problem . 69 3.6.3 Metaheuristic attacks on block ciphers . 80 3.6.4 An attack with a quasi-metaheuristic methodology. 92 3.6.5 Attacks using non-metaheuristic search algorithms. 94 4 Evolving balanced Boolean functions with optimal resistance to alge- braic and fast algebraic attacks, maximal algebraic degree, and very high nonlinearity. 95 4.1 Introduction . 95 4.2 Preliminaries . 99 4.3 The experiments . 102 4.3.1 Representing candidates as truth tables . 102 4.3.2 Choosing a cost function . 105 4.3.3 Adding algebraic degree to the cost function. 109 4.3.4 Equivalence classes . 110 4.4 Summary of achievements, and avenues for future research . 111 5 Using evolutionary computation to create vectorial Boolean functions with low differential uniformity and high nonlinearity. 113 5.1 Introduction . 113 5.1.1 Technical background . 114 5.1.2 The use of metaheuristics in this field. 117 5.2 Experiments with simulated annealing . 119 5.2.1 Refining the annealing cost functions for differential uniformity . 120 5.2.2 Relating nonlinearity and differential cost functions. 122 5.3 Experiments with memetic algorithms . 126 5.4 Experiments with ant colony optimisation . 131 5.5 Conclusions, and directions for future research. 137 6 Nonlinear cryptanalysis and metaheuristic search for S-box approxima- tions. 139 6.1 Introduction. 140 6.1.1 Linear cryptanalysis - the three main phases of an Algorithm 2 attack.144 6.2 How nonlinear approximations affect the attack . 153 3 6.2.1 How unbalanced nonlinear components in the approximation affect the attack. 153 6.2.2 How the related approximations affect the attack. 157 6.3 New statistical frameworks and cryptanalytic techniques. 161 6.3.1 Adapting the new analysis phase to nonlinear cryptanalysis of substitution- permutation networks. 161 6.3.2 The theoretical complexity of the new attack. 168 6.3.3 When the cipher is not a substitution-permutation network. 175 6.4 The use of simulated annealing to evolve nonlinear approximations. 179 6.5 Experiments on various ciphers, and application to their cryptanalyses. 184 6.5.1 AES. 184 6.5.2 Serpent. 185 6.5.3 DES. 202 6.5.4 PRESENT. 207 6.6 Conclusions, and directions for future research. 208 7 Evaluation and conclusions 210 7.1 Single-output Boolean functions (Chapter 4). 210 7.2 Vectorial Boolean functions (Chapter 5). 211 7.3 Nonlinear approximations and nonlinear cryptanalysis (Chapter 6). 212 7.4 A unified view... 214 A Truth tables of evolved filter functions 216 B The proof of Theorem 5.1.18 218 B.0.1 Preliminaries. 218 B.0.2 Constructing the equivalent bijection. 219 B.0.3 Consequences for genetic/memetic and ant algorithms . 240 C Errors in the description of the Dunkelman/Keller approximation 243 4 List of Figures 3.1 The Data Encryption Standard (DES). 22 3.2 The Heys substitution-permutation network. 25 3.3 The linear approximation table for the example S-box in the Heys toy cipher. 30 3.4 A linear approximation to the first three rounds of the Heys SPN. 31 3.5 Difference distribution table (DDT) for the Heys S-box. 36 3.6 A differential characteristic linked across several rounds of the Heys SPN. 38 3.7 Diagram showing the operation of a linear feedback shift register. 49 3.8 Diagram showing the operation of a combiner-based stream cipher. 50 3.9 Diagram showing the operation of a filter-based stream cipher. 51 3.10 Diagram depicting a boomerang attack. 84 6.1 Diagram of a 1R nonlinear approximation to the Heys toy cipher. 143 6.2 Diagram showing the full 16-round DES and Matsui's approximation for rounds 2 to 15. 154 6.3 Diagram showing part of the Heys cipher during linear cryptanalysis. 158 6.4 Graphs comparing an approximated and an exact means of calculating ad- vantage in nonlinear attacks. 173 6.5 Diagram showing the first three rounds of DES in Matsui's attack. 176 6.6 Diagram showing the last three rounds of DES in Matsui's attack. 176 6.7 Graph showing mean advantages for linear and nonlinear attacks on four round SPN. 197 6.8 Graph showing average advantages based on mean rank for linear and non- linear attacks on four round SPN. 198 5 List of Tables 4.1 Annealed and previously-known Boolean functions for n ≤ 11. 107 4.2 First set of annealed Boolean functions for 12 ≤ n ≤ 16. 109 4.3 Best annealed and previously-known functions for 12 ≤ n ≤ 16. 110 4.4 Equivalence classes of functions with the best annealed profiles. 110 5.1 Differential properties of annealed S-boxes with different cost functions. 122 5.2 Nonlinearity properties of annealed S-boxes with different cost functions. 124 5.3 Annealed S-boxes with n = 5 in large-scale experiments. 125 5.4 Annealed S-boxes with n = 6 in large-scale experiments. 125 5.5 Comparison of best-known and evolved S-box nonlinearities for 5 ≤ n ≤ 8. 126 5.6 Results of memetic algorithm experiments using cycle crossover and roulette- wheel selection. 127 5.7 Results of memetic algorithm experiments using cycle crossover and rank selection. 128 5.8 Results of memetic algorithm experiments using PMX crossover and roulette- wheel selection. 129 5.9 Results of memetic algorithm experiments using PMX crossover and rank selection. 130 5.10 Results of varying crossover probabilities in memetic algorithm experiments. 131 5.11 Results of varying population sizes in memetic algorithm experiments. 131 5.12 First set of experiments with Ant System. 133 5.13 First set of experiments with Dorigo's Ant Colony System and cycle-index. 133 5.14 First set of experiments with Dorigo's Ant Colony System and increment- index. 133 5.15 First set of experiments with Luke's Ant Colony System and cycle-index. 133 6 5.16 First set of experiments with Luke's Ant Colony System and increment-index.134 5.17 Experiments varying the evaporation rate for Ant System and increment- index. 134 5.18 Experiments varying the evaporation rate for Ant System and cycle-index. 134 5.19 Experiments varying evaporation rate and elitist pheromone update in Luke ACS (cycle). 135 5.20 Experiments varying evaporation rate and elitist pheromone update in Luke ACS (increment). 135 5.21 Experiments varying evaporation rate and elitist pheromone update in Dorigo ACS (cycle). 135 5.22 Experiments varying evaporation rate and elitist pheromone update in Dorigo ACS (increment). 136 5.23 Performance of ant algorithms for different numbers of ants. 137 6.1 Linear approximation to DES S5. 159 6.2 First nonlinear approximation to Serpent S3. 159 6.3 Second nonlinear approximation to Serpent S3. 159 6.4 Positions of data bits in hypothetical nonlinear attack on DES. 177 6.5 Recurring key bits in hypothetical nonlinear attack on DES (i). 177 6.6 Recurring key bits in hypothetical nonlinear attack on DES (ii). 178 6.7 Comparison of success probabilities in linear cryptanalysis according to the formulae of Matsui and Sel¸cuk. 190 6.8 Linear cryptanalysis: success probabilities for 44 attacked key bits. 190 6.9 Linear cryptanalysis: success probabilities for 108 attacked key bits. 190 6.10 Linear cryptanalysis: success probabilities for 140 attacked key bits. 191 6.11 Comparison of attack complexities for linear, multidimensional linear, and nonlinear attacks on reduced-round Serpent (part 1). 193 6.12 Comparison of attack complexities for linear, multidimensional linear, and nonlinear attacks on reduced-round Serpent (part 2). 194 6.13 Comparison of attack complexities for linear, multidimensional linear, and nonlinear attacks on reduced-round Serpent (part 3). 195 6.14 Nonlinear approximations to Serpent S3 with output bitmask 0101. 199 6.15 Nonlinear approximations to Serpent S3 with output bitmask 1100. 200 6.16 Nonlinear approximations to Serpent S3 with input bitmask 0011.