Breaking the Enigma

Dmitri Gabbasov

June 2, 2015

1 Introduction

Enigma was an electro-mechanical machine that was used before and during the World War II by Germany to encrypt and decrypt secret messages. Invented by Arthur Scherbius in 1918 and meant initially as a commercial product for the enterprise community, Enigma turned out to be more successful with the German military forces [2]. Enigma had evolved throughout several years, gaining better cryptographic strength, but also being broken time and again. In this report we give a general description of an Enigma machine. We describe the shortcomings of the machine and its early operating procedures. Based on a paper by M. Rajewski [1] we outline the key methods used to break the Enigma before World War II.

2 The Enigma machine

The Enigma machine is a combination of mechanical and electrical subsystems. Its main com- ponents are a keyboard, a plugboard, a lampboard and a set of rotating disks called rotors arranged adjacently on a spindle (figure 1). The mechanical parts act in such a way as to form a varying electrical circuit. When a key is pressed, one or more rotors move to form a new rotor configuration, ultimately lighting up one display lamp, which shows the output letter (figure 2).

2.1 Rotors

The rotors were one of the most important parts of an Enigma machine. An Enigma had typically three rotors, along with a so called reflector and an entry wheel. The rotors had 26 contacts on both sides – one for each alphabet letter (figure 3). Inside a rotor, small wires connected the contacts on one side with the contacts on the other side in some irregular order – a rotor effectively represented a permutation of the alphabet. Electrical current entering through a contact on one side would leave through some contact on the other side. The reflector had contacts only on one side, and a current entering one of the contacts would leave through some other contact. The entry wheel also had 26 contacts on one side, those were connected to the keyboard. Altogether, a current would enter through the entry wheel, go through a number of rotors, then through the reflector, again through the rotors (in reverse order and using different contact paths) and then leave through the entry wheel.

1 Figure 1: A typical Enigma machine.

The entry wheel was stationary. The rotors, however, could be rearranged and each one could be turned into one of 26 possible positions. The reflector was mostly stationary, although there were also Enigma models where the reflector could be set into different positions just like rotors. With each keypress, the leftmost rotor advances by one position, this we refer to as the stepping movement. At certain positions it also makes the second rotor advance by one, which in turn can make the third rotor advance. The movement is similar to that of an odometer. Each rotor consists of two parts – the rotor core and the alphabet ring. The electrical contacts are attached to the core. The ring has letters on its outside surface. These letters can be seen through the holes when the Enigma machine is being used, and they were used to identify the position of the rotor. The alphabet ring can be rotated around the core, meaning there are in fact 26 ways to combine the core with the ring. The alphabet ring also has a notch (or several notches) that determines when the neighboring rotor is to be turned. Early models of Enigma came with a set of three rotors that could be placed into the machine in any order, giving a total of 6 rotor placement combinations. Later, the set was increased to five rotors with three placed into the machine at any given time – this increased the number of combinations to 60. Most of the rotors were identified by Roman numerals, and each issued copy of rotor I was wired identically to all others. By itself, a rotor performs only a very simple type of encryption – a simple substitution cipher. For example, the contact corresponding to the letter E might be wired to the contact for letter T

2 Figure 2: Internal wiring of Enigma (only 4 keys/lamps shown for simplicity). The A key is pressed and the lamp corresponding to the letter D lights up. Red lines and arrowheads show the flow of the current. on the opposite side, and so on. Enigma’s security came from using several rotors in series and the regular stepping movement of the rotors, thus implementing a polyalphabetic substitution cipher. It is also important to note that because of the reflector the Enigma machine was reciprocal, meaning that encryption and decryption procedures were identical. One could type in plaintext and get the ciphertext, and similarly type in the ciphertext and get the plaintext; this is assuming that certain starting settings, which we will describe later, are the same in both cases.

2.2 Plugboard

The plugboard allowed for variable rewiring of certain connections by the operator. It consisted of 26 sockets – one for each letter. A wire could be inserted into any two sockets, the effect was to swap the corresponding letters before and after the main rotor scrambling unit. For example, if E and Q are swapped, when an operator presses E, the signal was diverted to Q before entering the rotors.

3 Figure 3: Two Enigma rotors. Electrical contacts on both sides can be seen. There is a notch on the alphabet ring of the right rotor next to the letter D.

Initially, only 6 pairs of letters were swapped during normal operation of the machine, meaning that 14 letters were unaffected. Later, the number was increased to 10, leaving only 6 letters unaffected.

2.3 Mathematical analysis

The Enigma transformation for each letter can be specified mathematically as a product of permutations. Let S denote the plugboard transformation, L, M, R the transformations of the left, middle and right rotors respectively, U the reflector transformation and P a simple rotation (a → b, b → c,..., z → a). Then the encryption E can be represented as

E = S(P xRP −x)(P yMP −y)(P zLP −z)U(P zL−1P −z)(P yM −1P −y)(P xR−1P −x)S−1, where x, y, z ∈ {0,..., 25} represent the starting positions of the rotors. Note, that even though there are 263 = 17576 ways to choose x, y and z and 3! = 6 ways to arrange the three rotors, it is the U (plugboard) transformation that can have ca. 1011 (6 swapped pairs) or 1.5 × 1014 (10 swapped pairs) different forms [4]. For an Enigma with a set of three rotors and a plugboard with 6 swappable pairs the total number of possible transformations E is 10 586 916 764 424 000 ≈ 253. The previous number merely represents the possible transformations of a single letter. Because with each keypress E changes, we will need to also count the number of ways that E can change in order to know how many different polyalphabetic substitution ciphers an Enigma machine can represent. This depends on the position of the alphabet rings on two of the rotors (because the rings have the notches that make the neighboring rotor turn). There are 262 = 767 ways to set the alphabet rings, thus together there are ca. 263 different polyalphabetic ciphers an Enigma may implement.

4 3 Operating procedures

For a message to be correctly encrypted and decrypted, both the sender and the receiver had to configure their Enigma in the same way – the rotor order, rotor starting positions and plugboard connections must be identical. Some of these settings were established beforehand and distributed to different German military units in codebooks, others were chosen by the operator to be different for each message. An Enigma machine’s initial state, or what today might be called the cryptographic key, con- sisted of a number of things: • rotor order – in later Enigma models not just the order, but also the chosen subset of rotors (e.g. rotors III, I and VI out of possible 8, in that order), • initial positions of the rotors – usually represented by letters that would be visible through the holes in the machine (e.g. AOH), • ring setting – the positions of alphabet rings with relation to the rotor cores, also repre- sented via three letters, • plugboard connections – the 6 (or in later models 10) pairs of letters that were swapped on the plugboard (e.g. EG DO LP). Most of the key was kept constant for a set time period, typically a day. However, a different initial rotor position was used for each message, a concept similar to an initialization vector in modern cryptography. The starting position for the rotors, referred to as the message key, was transmitted just before the ciphertext, usually after having been enciphered. The exact method used was termed the indicator procedure and changed over time. Design weakness and operator sloppiness in these indicator procedures were two of the main weaknesses that made breaking Enigma possible. In the period 1930–1938 the procedure was for the operator to set up his machine in accordance with the daily settings that he received from the codebook. This included (in addition to the rotor order, the ring setting and the plugboard connections) a global initial position for the rotors – the so called ground setting (e.g. AOH). The operator turned the rotors into that position, he then chose his own arbitrary starting position (e.g. EIN) – the message key – and typed it in twice to get six letters of ciphertext (e.g. XHTLOA). Finally he set the rotors into the position that he had come up with (EIN) and typed the message. The resulting enciphered message key and the enciphered message would be transmitted together. The receiving party would first similarly set up the machine using the daily settings and would then type in the first six letters of the ciphertext (XHTLOA). The resulting plaintext should then contain the message key repeated twice (EINEIN). The receiver would then set the rotor positions to the ones given by the message key (EIN), and would proceed with decrypting the remaining message. The weakness in this indicator scheme came from two factors. First, use of a global ground setting – this was later changed so the operator selected his initial position to encrypt the indicator, and sent the initial position in the clear. The second problem was the repetition of the indicator, which was a serious security flaw. The message setting was encoded twice, resulting in a relation between first and fourth, second and fifth, and third and sixth character. This enabled the Poles to break into the Enigma system as early as 1932.

5 4 Polish efforts

4.1 Beginnings

The Polish Cipher Bureau began intercepting German military Enigma-enciphered messages in 1928. The bureau was already in possession of a commercial Enigma, however the German military used an Enigma model with modified rotor and reflector wirings. They tried to read the messages, but their efforts were fruitless [3]. In 1930 the complexity of the machine was increased further by the addition of a plugboard, which, at the time, had six swappable pairs. In September 1932 the bureau hired three mathematicians – Marian Rejewski, Jerzy Różycki and Henryk Zygalski – who eventually started working on the Enigma. In 1931 and 1932 the bureau was aided by the French intelligence who provided them with operating instructions for Enigma and two sheets of monthly key settings. At the time, the procedures that were used by Germans entailed the double encipherment of the message key. This gave Rajewski the chance to analyze the first six letters of encrypted messages from which he managed to work out the wiring of each of the rotors as well as the reflector. To do this, he used his characteristic method, which we will describe shortly. After Rajewski had worked out the logical structure of the military Enigma, the Polish Cipher Bureau had replicas built – the so called Enigma doubles.

4.2 Rajewski’s characteristic

Rajewski discovered the following property of the daily Enigma keys. Let p1, . . . , p6 be the first six plaintext letters, and c1, . . . , c6 the corresponding cyphertext letters. We can then write down the following equations

c1 = p1A c4 = p4D

c2 = p2B c5 = p5E

c3 = p3C c6 = p6F where A, . . . , F are permutations that represent the collective effect of the Enigma on each letter. Equivalently, we can write

−1 −1 p1 = c1A p4 = c4D −1 −1 p2 = c2B p5 = c5E −1 −1 p3 = c3C p6 = c6F

If the first six letters represent the double-enciphered message key, then p1 = p4, p2 = p5 and p3 = p6, and therefore

−1 c1A D = c4 −1 c2B E = c5 −1 c3C F = c6

6 It is also known that Enigma is reciprocal – encryption and decryption are identical. This means that AA = I, or, equivalently, A−1 = A. Thus

c1AD = c4

c2BE = c5

c3CF = c6

Above, A, . . . , F only depend on the logical structure of the Enigma and the daily settings. With a sufficient amount of intercepted messages, Rajewski was able to determine the permutation products AD, BE and CF . The result would be written down in cyclic notation, for example:

AD = (pjxroquctwzsy)(kvgledmanhfib) BE = (kxtcoigweh)(zvfbsylrnp)(ujd)(mqa) CF = (yvxqtdhpim)(skgrjbcolw)(un)(fa)(e)(z)

Rajewski called the three permutations “the characteristic of the day”. The reciprocity of Enigma implies that the permutations A, . . . , F consist of simple transpo- sitions, i.e. they consist of 13 cycles of length 2. This, in turn, implies that the permutation products AD, BE and CF consist of pairs of cycles of equal length (e.g. CF , above, has 2 cycles of length 10, 2 cycles of length 2 and 2 cycles of length 1). From the above example it is already possible to tell that C = (ez) ... and F = (ez) .... Using further analysis and exploiting weak message keys it was possible to reconstruct all 6 permutations in full.

4.3 The grill method

We can now express the permutations A, . . . , F in terms of separate permutations corresponding to different parts of the Enigma:

A = S(P xNP −x)Q(P xN −1P −x)S−1 B = S(P x+1NP −x−1)Q(P x+1N −1P −x−1)S−1 . . where S represents the plugboard, N the rightmost rotor, Q the combined effect of the two other rotors and the reflector, P is a simple rotation (a → b, b → c,..., z → a) and x ∈ {0,..., 25} represents the starting position of the leftmost rotor. This assumes that the two slower rotors do not move during the encryption of the message key (i.e. Q is the same for all 6 permutations), which was true with probability 80% when the rightmost rotor had only one notch. It is also assumed that the plugboard is connected to the first rotor in alphabetical order, which was the case for the military Enigma. We can now write

(P xN −1P −x)S−1AS(P xNP −x) = Q (P x+1N −1P −x−1)S−1BS(P x+1NP −x−1) = Q . .

7 Even though Q is unknown, it is the same for all six permutations A, . . . , F . N is one of the rotors, which are all known. With three possible rotors to choose from and 26 starting positions there are only 84 possible combinations of N and x. We could try them all and, if the plugboard was not used (if S was identity), one of them would yield the same Q in all six cases. With only six swapped letter-pairs on the plugboard, S was “similar” to identity. Thus, with some further work and analysis – referred to as the grill method – it was possible to determine both – N as well as S.

4.4 Remaining rotors

After having found out the identity of the leftmost rotor, it was necessary to find the identity of the other two. With three rotors in total, there were only two that could be tried. Together there were 2 × 262 = 1352 possibilities how to place the remaining two rotors. The Poles tried all of the combinations by applying brute force.

4.5 The ring setting

By now, one has determined the plugboard configuration, the rotor order and the position of the core of each rotor (in the ground setting). With this information one can now decrypt the message key of any message for the day. However, in order to decrypt the message body, one also needs to know the ring setting. There are 263 = 17576 ways to set the alphabet rings on the three rotors. From the messages that the Poles decrypted thanks to the 2 sheets of daily keys that were delivered to them by the French intelligence, they new that in principle all messages began with the letters ANX, from the word an (German for to) and the spacer X. Based on this, one had to pick an intercepted message and, having correctly set up the rotor order and the plugboard (using any rotor core positions), start repeatedly pressing the first letter of the message body until the letter A would light up. Once that happened, one would press the next two letters of the message in hope that the letters N and X would light up next. If they did, then there was a high chance that the correct rotor core positions of the message had been found. The message key (which is known) and the rotor core positions determine the ring setting. Rajewski describes this method of finding the ring setting as very primitive and tiresome, but still effective. The entire daily key was now recovered, and all messages from the same day on the same network could be decrypted directly.

4.6 Major setback

The Poles were able to reliably decrypt German Enigma traffic in the years 1933–1938. Even though operating procedures began to change as early as 1936, the Poles were still able to come up with ways to continuously break Enigma traffic. This included building machines like the cyclometer and the Polish bomba, which helped reduce manual effort. In 1938 however, a change was made not in the indicator procedures, but in the machine – two new rotors were added to the set of existing three, this increased the number of possible rotor orders from 6 to 60. For the Poles this mostly meant that they had to build many more machines (e.g. there were 6 different bombas – one for each rotor order). The Poles did not have

8 the resources to commission more machines, and could therefore only read a small minority of messages that did not use either of the new rotors, and also messages on some networks where the old double-indicator procedure was still in use. Furthermore, in January 1939, the number of swappable plugboard pairs was increased to 10, which made the grill method useless.

5 Further efforts and consequences

In July 1939 at a conference near Warsaw, the Poles revealed to the French and British that they had broken Enigma. They provided the British with a reconstructed Enigma including the five rotors used at that time. They also described their methods of breaking the Enigma. Shortly after, Germany invaded Poland, and the Cipher Bureau had to flee from the country. The British deduced new methods for breaking Enigma traffic, that relied less on the indicator procedures. They also built new bigger machines, inspired by The Polish bombas. However, the precise details of their work deserve a separate report. The consequences of the breaking of Enigma are considered far reaching and sometimes credited with shortening the war by as much as four years [5].

References

[1] M. Rajewski, “How Polish Mathematicians Deciphered the Enigma,” IEEE Annals of the History of Computing, vol. 3, no. 3, pp. 213–234, July 1981. [2] K. Gaj, and A. Orłowski, “Facts and Myths of Enigma: Breaking Stereotypes,” Advances in Cryptology — EUROCRYPT 2003, pp. 106–122. Springer-Verlag, Berlin 2003. [3] T. Sale, “The Breaking of Enigma by the Polish Mathematicians,” The Enigma cipher machine. http://www.codesandciphers.org.uk/virtualbp/poles/poles.htm [4] T. Sale, “Military Use of the Enigma,” The Enigma cipher machine. http://www.codesandciphers.org.uk/enigma/enigma3.htm [5] H. Hinsley, “The Influence of ULTRA in the Second World War,” a lecture given in 1993 at Cambridge University http://www.cdpa.co.uk/UoP/HoC/Lectures/HoC_08e.PDF (transcript)

9