Modern Cryptanalysis of Historical Ciphers November 1, 2019 George Lasry Agenda • Introduction – Motivation – Difficulty – Generic approaches • Case studies – Hagelin M-209 – Playfair – Double transposition – SIGABA George Lasry 2 Agenda • Introduction – Motivation – Difficulty – Generic approaches • Case studies – Hagelin M-209 – Playfair – Double transposition – SIGABA George Lasry 3 Motivation • Historical cryptanalysis • Undecrypted texts • Public challenges • Fun George Lasry 4 Difficulty - Factors • System design – Diffusion – Confusion – Weaknesses • Key – Key space/length • Ciphertext – Length – Language George Lasry 5 Difficulty Easy Moderate Hard Very hard Intractable? Monoalphabetic Playfair Playfair Playfair Fialka substitution (long ciphertext) (short ciphertext) (very short) Transposition Transposition ADFGVX Double transposition Double transposition (short key) (long key) (long random key) Vigenere Enigma Enigma SIGABA (long ciphertext) (short ciphertext) (known plaintext) Hagelin M-209 Hagelin M-209 (long ciphertext) (short ciphertext) Hagelin M-209 Sturgeon T52 Sturgeon T52 (known plaintext) (regular stepping) (irregular stepping) George Lasry 6 Generic Approaches - 1 Exhaustive Combinatorial Stochastic Search Search Search ● Simple brute force ● Backtracking ● Hill climbing ● Dictionary search ● Meet in the Middle ● Simulated annealing (MITM) ● Hybrid (e.g., nested) ● Match some ● Others (e.g., genetic constraints (e.g., ● Match some algorithms) known plaintext) constraints ● Or optimize a scoring ● Optimize a fitness or function scoring function George Lasry 7 Generic Approaches - 2 Frontal Attack Divide and Conquer Key Step 1 Part 1 Attack Key Key Step 2 Part 2 George Lasry 8 Scoring Functions - Resilience to Errors vs. Selectivity Long ciphertexts George Lasry 9 Scoring Functions - Resilience to Errors vs. Selectivity Short ciphertexts George Lasry 10 Agenda • Introduction – Motivation – Difficulty – Generic approaches • Case studies – Hagelin M-209 – Playfair – Double transposition – SIGABA George Lasry 11 Hagelin M-209 Boris Hagelin 1892-1983 George Lasry 12 Hagelin M-209 – Functional Diagram George Lasry 13 Hagelin M-209 - Key Space • Wheel pins – 2131 options • Lugs – 238 options • Total keyspace – 2169 George Lasry 14 Known-Plaintext Attack • Frontal attack – On full key space - pins and lugs Full Key • Hill climbing Hill climbing (Pins and • Specialized score Lugs) – ADE - Aggregate Displacement Score George Lasry 15 Known-Plaintext Attack - ADE Scoring Function ADE: requires 50 known-plaintext symbols Simple score: requires 300 known-plaintext symbols George Lasry 16 M-209 - Ciphertext-Only Attack • Frontal nested attack • Outer hill climbing - lugs Nested Hill Climbing Full Key – Inner simulated annealing - pins and (Pins and Simulated Lugs) • Log monograms Annealing • Requires only 500 letters – Vs. 1500 with previous attacks George Lasry 17 M-209 - Ciphertext-Only Attack • Frontal nested attack • Outer hill climbing - lugs – Inner simulated annealing - pins • Log monograms • Requires only 500 letters – Vs. 1500 with previous attacks • Challenges solved – 1035 letters - 1977 – 500 letters - 2012 George Lasry 18 Agenda • Introduction – Motivation – Difficulty – Generic approaches • Case studies – Hagelin M-209 – Playfair – Double transposition – SIGABA George Lasry 19 The Playfair Cipher – Key Square • Keyword: – PLAYFAIREXAMPLE George Lasry 20 The Playfair Cipher – Encryption Rule 1 George Lasry 21 The Playfair Cipher – Encryption Rule 2 George Lasry 22 The Playfair Cipher – Encryption Rule 3 George Lasry 23 The Playfair Cipher – Example George Lasry 24 Prior Attacks • Historical attacks – Ciphertext only: 800 letters (Mauborgne, 1918) – Key from keyword: 30 letters (Monge, 1936) – From crib • Modern attacks – Hillclimbing: hundreds of letters – Simulated annealing: 80 letters (Cowan, 2008) • 4-grams, logarithmic scale – Compression-based: 60 letters (Al-Kazaz et al., 2018) • Order 5, equivalent to 6-grams on log. scale George Lasry 25 Ciphertext-Only Attack - Short Ciphertexts • Integrated into CrypTool 2 – Java code – Analysis Connector API • Simulated Annealing – Enhanced • 6-grams • Rich transformations – Swaps of any 2 elements/rows/columns – Permutations of the rows/columns, inside row/column George Lasry 26 Simulated Annealing - Variable Temperature George Lasry 27 Simulated Annealing - Fixed Temperature George Lasry 28 Simulated Annealing - Minimal Acceptance Probabililty George Lasry 29 Klaus Schmeh’s Challenges George Lasry 30 Agenda • Introduction – Motivation – Difficulty – Generic approaches • Case studies – Hagelin M-209 – Playfair – Double transposition – SIGABA George Lasry 31 Double Transposition Cipher - The “Spy Cipher” K1 K2 Plaintext First Transposition Second Transposition Ciphertext George Lasry 32 Double Transposition - Attack • Divide and Conquer • Phase 1 Phase 1 K2 – Find K2 – Hillclimbing – Specialized scoring - IDP • Phase 2 Phase 2 K1 – Undo K2 and find K1 – Hillclimbing, 4-grams George Lasry 33 IDP - Index of Digraphic Potential K2 K1 undo second undo first ciphertext plaintext transposition transposition interim decryption IDP score ● Hillclimbing ● Dictionary attack George Lasry 34 The Double Transposition Cipher Challenge, 2007 • Otto Leiberich • Klaus Schmeh • Secure parameters – Different K1 and K2 – Key lengths 20 to 25 – Cryptogram length VESINTNVONMWSFEWNOEALWRNRNCFITEEICRHCODEEAHEACAEOHMYTONTDFIFMDANGTDRVAONRRTORMTDHE OUALTHNFHHWHLESLIIAOETOUTOSCDNRITYEELSOANGPVSHLRMUGTNUITASETNENASNNANRTTRHGUODAAAR AOEGHEESAODWIDEHUNNTFMUSISCDLEDTRNARTMOOIREEYEIMINFELORWETDANEUTHEEEENENTHEOOEAUEA EAHUHICNCGDTUROUTNAEYLOEINRDHEENMEIAHREEDOV... George Lasry 35 Solving the Challenge What exciting news, deciphering the “Doppelwürfel”! l congratulate you to this great success. Otto Leiberich, December 29, 2013 George Lasry 36 Agenda • Introduction – Motivation – Difficulty – Generic approaches • Case studies – Hagelin M-209 – Playfair – Double transposition – SIGABA George Lasry 37 Design of SIGABA George Lasry 38 SIGABA - Key Space • Cipher and control rotors – 278.8 options • Index Rotors – 216.8 options • Total keyspace – 295.6 George Lasry 39 Prior Attacks • WW2 – “U.S. 5-letter traffic: Work discontinued as unprofitable at this time.” • Savard and Pekelney – 1999 – Attack on messages “in depth” – Unrealistic operational scenario • Stamp and Chan – 2007, Stamp and Low – 2007 – Known-plaintext attack – 286.7 vs. 295.6 for brute-force attack George Lasry 40 New Known-Plaintext Attack – Concept Known Expected Plaintext Ciphertext Meet in the Middle George Lasry 41 New Known-Plaintext Attack – Outline Phase 1: Find matching Known Expected cipher rotor settings and Plaintext Ciphertext stepping sequences Meet in the Middle Phase 2: Find control and index rotor settings that generate one of the matching cipher stepping sequences George Lasry 42 Hash Table for Meet-in-the-Middle Attack George Lasry 43 Meet-in-the-Middle Attack – Complexity • Processing – Phase 1: 247.1 – Phase 2: 260.2 – Overall: max(247.1, 260.2) = 260.2 – Comparison: • 295.6 for brute-force attack • 286.7 for best prior attack (Stamp & al. 2007) • 256 for DES – cracked in 1997 • Space – About 80GB RAM for hash table • Feasible with modern technology George Lasry 44 Solving MysteryTwisterC3 Challenges George Lasry 45 New Challenges George Lasry 46 Reference Source Code for SIGABA Simulator • Used to create the challenges • Validated against: – Pekelney (1998) • Itself validated against real machine – Sullivan (2002) George Lasry 47 Other Projects Project Method Results ADFGVX Divide and conquer, hillclimbing, IC and 600 original Eastern Front ngrams German cryptograms, 1918 Sturgeon T52 Divide and conquer, two phases, Original German cryptograms specialized scoring and monograms from 1942 Backtracking for known-plaintext attack Vatican ciphers Manual and computerized methods (e.g. Homophonic and polyphonic simulated annealing) ciphertexts, from 16-18th cent. WW1 Diplomatic codes Mostly manual methods 1913-1915 German messages Enigma - double indicators Hillclimbing, specialized scoring 5-10 indicators required Single transposition with Hillclimbing, two phases, specialized Key length up to 1000 long keys scoring and 4-grams George Lasry 48 ● George Lasry, Solving a 40-Letter Playfair Challenge with CrypTool 2, Proceedings of the 2nd International Conference on Historical Cryptology, HistoCrypt 2019, June 23-26, 2019, Mons, Belgium, fulltext ● George Lasry, A Practical Meet-in-the-Middle Attack on SIGABA, Proceedings of the 2nd International Conference on Historical Cryptology, HistoCrypt 2019, June 23-26, 2019, Mons, Belgium, fulltext ● George Lasry, A Methodology for the Cryptanalysis of Classical Ciphers with Search Metaheuristics, Kassel University Press, Ph.D. Thesis, 2018, fulltext Thank You ● Full list of publications November 1, 2019 George Lasry, Ph.D. [email protected] Classes of Active Pins 1 0 1 1 0 1 George Lasry May 2017 50 Classes of Active Pins George Lasry May 2017 51 Classes of Active Pins Plaintext: P Class Displacement Beaufort D Encryption Beaufort: 25 – P 25 – P + D Modulo 26 000000 0 25 – P 000001 7 18 – P … … … 101100 1 24 – P Beaufort: 22 – P 101101 3 22 – P … … … 111110 12 13 – P 111111 25 0 – P George Lasry May 2017 52 ADFGVX Fritz Nebel 1891-1967 Georges Painvin Substitution + Fractionation + Columnar Transposition 1886-1980 53 Before SIGABA – Hebern Cipher Machines – 1920s • 5 cipher rotors • Regular stepping George Lasry 54 History of SIGABA – William Friedman’s Design • 5 cipher rotors • Irregular stepping – Punched taps – Plugboard George Lasry 55 SIGABA • 5 cipher rotors • Irregular stepping – 5 „control“ rotors – Plugboard George Lasry 56 History of SIGABA – Final Design – US Navy • 5 cipher rotors • Irregular stepping – 5 „control“ rotors • No plugboard – 5 „index“ rotors George Lasry 57 Time-Memory Trade-off • Process only 8 known-plaintext symbols – Rotors step up to 7 times • Less than 8 - slower – More false positives • More than 8 - more memory – More matching sequences • Pruning false positives – Use additional known-plaintext symbols – Or Index of Coincidence after decrypting ciphertext George Lasry 58.