Code Breaking in the Pacific

Code Breaking in the Pacific Peter Donovan • John Mack Code Breaking in the Pacific 123 Peter Donovan John Mack School of Mathematics and Statistics School of Mathematics and Statistics University of New South Wales University of Sydney Sydney, NSW, Australia Sydney, NSW, Australia Additional material to this book can be downloaded from http://extras.springer.com ISBN 978-3-319-08277-6 ISBN 978-3-319-08278-3 (eBook) DOI 10.1007/978-3-319-08278-3 Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2014945227 © Springer International Publishing Switzerland 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Margaret Donovan and Vanessa Mack have had to put up with a great deal while the research into the cipher war in the Pacific Ocean was being carried out. And so this book is dedicated to them. Preface Will no one tell me what she sings? Perhaps the plaintive numbers flow For old, unhappy, far-off things, And battles long ago. W. Wordsworth, The Solitary Reaper. President Roosevelt, in his January 1943 State of the Union address, said: The Axis Powers knew that they must win the war in 1942—or eventually lose everything. I do not need to tell you that our enemies did not win the war in 1942. In the Pacific area, our most important victory in 1942 was the air and naval battle off Midway Island. That action is historically important because it secured for our use communications lines stretching thousands of miles in every direction. In placing this emphasis on the Battle of Midway, I am not unmindful of other successful actions in the Pacific, in the air and afloat—especially those on the Coral Sea and New Guinea and in the Solomon Islands. But these actions were essentially defensive. They were part of the delaying strategy that characterised this phase of the war. The President could not reveal that Allied interception and decoding of Japanese naval radio messages played a significant role in determining the events of the Pacific War in 1942 and their outcomes. Indeed, the role of Allied Signals Intelligence in WW2 was suppressed for many years afterwards, with the earliest authoritative accounts appearing only in the late 1970s. Even later, publications on the events of WW2 commonly failed to address underlying contributions from this source. Since then Bletchley Park has become a great British icon and is (at long last) receiving appropriate maintenance. European and Atlantic Signals Intelligence has been investigated and explained often enough for its significance to be accepted and appreciated. For example, it is clear that the 1942 Battle of the Atlantic and the 1944 invasion of Normandy involved much use of what is generically called codebreaking and that this was applied to codes based on encryption machines, notably the Enigma and Lorenz SZ42. vii viii Preface The situation with regard to Signals Intelligence in the WW2 Pacific Theatre is different. There is no iconic single site associated with this activity—it involved units around the world and some of these are almost unknown. It is well known for its crucial contributions to the 1942 Battle of Midway and the 1943 shooting down of the aircraft carrying Admiral Yamamoto. Yet it is chastised by some writers for its claimed failure to warn of the 1941 raid on Pearl Harbor, and its overall contribution to the evolution of the Pacific War is much less well appreciated than is the case with its European counterpart. The methods used in the code breaking behind the successes of Allied Pacific Signals Intelligence are quite different to those used against encryption machine ciphers such as the Enigma. The reason is that the main cipher systems used by both the Imperial Japanese Army (IJA) and the Imperial Japanese Navy (IJN) were based on code books rather than a machine. This book is the first to provide a complete description of those ‘additive cipher systems’ and the development of the techniques used to break various implementations of them. In doing so, it addresses the last major gap in the literature of WW2 cryptography and most likely the last major gap in the literature of WW2. The IJA and IJN in fact made little use of codes based on a Latinised alphabet with which most of their thousands of radio operators would have lacked familiarity. Just as Morse code was invented to enable plain language messages written in the 26-letter alphabet to be converted into radio signals, the Japanese devised Kana Morse for the same purpose in relation to the use of Kana syllabary for writing Japanese. Both types of Morse had capacity for transmitting the ten digits 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9. In the code books used by the IJA and the IJN for their major cipher systems, plain language entries in these were matched with code groups (or ‘words’) that were purely numerical—3-digits long for some simple codes, 4-digits long in others and, in the case of the all-important JN-25 naval ciphers, all of length 5 digits. Having numerical code groups made it possible to employ a second-stage encryption process before transmission. This process concealed each code group in a message by combining it with another group of the same length selected from a second book, called here the additive table, according to prescribed rules. In this book, the code book taken with its superencryption process is called an additive cipher system and much of the book is directed to an explanation of the procedures used to decrypt intercepts of messages sent in such systems. The code books and encryption systems were normally changed every few months. Numerous minor systems were used as well. In particular, this book explains why JN-25, the principal series of operational cipher systems used by the IJN, was broken almost immediately after its introduction and continually thereafter. It was the sole source of highest-level Allied Signals Intelligence in the Pacific throughout 1942 and was providing some 60 % of this as late as mid-1944. This fact demands explanation, given that all the principal operational cipher systems used by the IJA were of a type similar to JN-25 but defied attack until mid-1943, when one important such system was broken. The Preface ix others remained unreadable until early 1944 when capture at Sio in New Guinea of a full set of current code documents transformed the situation. The explanation is both simple and profound. The IJN used only multiples of three for ‘code groups’ in most JN-25 code books. The IJA did not. This book exposes in considerable detail the disastrous consequences of that IJN practice. The all-important Signals Intelligence successes obtained from JN-25 and other early Japanese cipher systems depended totally on separate decisions made by both the UK and the USA to develop Signals Intelligence capabilities against Japan soon after the end of WW1. Although these were initially directed at diplomatic messages, concerns regarding Japanese intentions towards China, South-East Asia and the Pacific stimulated development of this capacity in the military and naval sectors. The build-up of Japanese naval power was another source of concern. The 1930s saw both the UK and the USA setting up teams of cryptanalysts with a mathematical and/or scientific background. WW2 was to see the introduction of large-scale professional attacks on Axis communications by high-level profession- als with massive (by the standards of the day) data processing facilities and support staff. The cryptanalytic developments explained in this book are based on extensive reading of archival documents held in several countries, including oral history interviews, and much published material. Another, totally reliable, source of information used was computer-based experimentation. This is identified wherever it occurs. Both authors have strong backgrounds in classical mathematics coupled with initially quite separate interests in WW2 dating back at least 40 years. Our research generally supports the conclusion that little useful knowledge came from JN-25 decryption and decoding prior to the raid on Pearl Harbor in December 1941.

Load more