The Cultural Contradictions of Cryptography: a History of Secret Codes in Modern America

The Cultural Contradictions of Cryptography: A History of Secret Codes in Modern America Charles Berret Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy under the Executive Committee of the Graduate School of Arts and Sciences Columbia University 2019 © 2018 Charles Berret All rights reserved Abstract The Cultural Contradictions of Cryptography Charles Berret This dissertation examines the origins of political and scientific commitments that currently frame cryptography, the study of secret codes, arguing that these commitments took shape over the course of the twentieth century. Looking back to the nineteenth century, cryptography was rarely practiced systematically, let alone scientifically, nor was it the contentious political subject it has become in the digital age. Beginning with the rise of computational cryptography in the first half of the twentieth century, this history identifies a quarter-century gap beginning in the late 1940s, when cryptography research was classified and tightly controlled in the US. Observing the reemergence of open research in cryptography in the early 1970s, a course of events that was directly opposed by many members of the US intelligence community, a wave of political scandals unrelated to cryptography during the Nixon years also made the secrecy surrounding cryptography appear untenable, weakening the official capacity to enforce this classification. Today, the subject of cryptography remains highly political and adversarial, with many proponents gripped by the conviction that widespread access to strong cryptography is necessary for a free society in the digital age, while opponents contend that strong cryptography in fact presents a danger to society and the rule of law. I argue that cryptography would not have become invested with these deep political commitments if it had not been suppressed in research and the media during the postwar years. The greater the force exerted to dissuade writers and scientists from studying cryptography, the more the subject became wrapped in an aura of civil disobedience and public need. These positive political investments in cryptography have since become widely accepted among many civil libertarians, transparency activists, journalists, and computer scientists who treat cryptography as an essential instrument for maintaining a free and open society in the digital age. Likewise, even as opponents of widespread access to strong cryptography have conceded considerable ground in recent decades, their opposition is grounded in many of the same principles that defined their stance during cryptography’s public reemergence in the 1970s. Studying this critical historical moment reveals not only the origins of cryptography’s current politics, but also the political origins of modern cryptography. Table of Contents List of Figures ii Acknowledgements iii Introduction 1 1. Cryptography and Written Culture in Nineteenth-Century America 21 2. Communication and its Limits in Cybernetics and Information Theory 70 3. Classifying Cryptography in the Postwar Years 121 4. Thinking About Information Security in the 1960s 178 5. Cryptography’s Public Reemergence in the 1970s 221 Conclusion 261 Bibliography 273 Appendix 319 Figures 322 i List of Figures Figure 1. A letter matrix used for poly-alphabetic ciphers. 277 Figure 2. Pigpen cipher inscribed on the tombstone of James Leeson in 278 Trinity Cemetery, New York, NY, 1794. Figure 3. A print designed at Riverbank Laboratory to illustrate the use 279 of the Baconian biliteral cipher in a botanical drawing. Figure 4. A second print designed at Riverbank Laboratory to illustrate 280 the use of the Baconian biliteral cipher in a botanical drawing. Figure 5. A print designed at Riverbank to illustrate the use of the 281 Baconian biliteral cipher in numerical sequences. Figure 6. Claude Shannon working at a chalkboard in the West Street 282 office of Bell Labs, New York City. Figure 7. Harold Black’s negative feedback or feedforward amplifier. 283 Figure 8. A noise measuring set in use among AT&T engineers circa early 284 twentieth century. Figure 9. The Sig-Saly encrypted telephone system. 285 Figure 10. Claude Shannon’s diagram of a communication channel and 286 its components. Figure 11. Diagram of a secrecy system and its components. 287 Figure 12. Claude Shannon’s handwritten experiment in reconstructing 288 messages through probability. Figure 13. Peter Elias’s diagram for a computer module to perform 289 probabilistic error-correction using codes derived from Shannon’s theory of communication. Figure 14. Peter Elias’s diagram to illustrate noise entering a 290 transmission, and the use of redundant codes to ensure that the message can be corrected to its original state. ii Acknowledgments My first and deepest thanks go to Michael Schudson, who advised this dissertation with wisdom, care, and patience. I was fortunate to have a second mentor in Todd Gitlin, whose intellect drove my thinking forward, and whose good judgment guided me through critical junctures. I was likewise driven by Mark Hansen’s expansive interdisciplinary knowledge, infectious curiosity, and intellectual taste. For the generosity of these three, my closest mentors, I feel inexpressibly fortunate and grateful. I have also been fortunate to study in this truly unique doctoral program, and nobody deserves more credit than Andie Tucher for building and maintaining the virtues of this program. Among the program’s other faculty, I am particularly grateful to Richard John for fostering an appreciation of archival research and its rewards. The late Frank Moretti was one of the finest teachers and one of the finest human beings I have ever known. Years later, his lessons are no less vivid. Studying with Robbie McClintock imparted the conviction that a life of scholarship is a life in service to our highest principles. And even though I never had a chance to meet the late James Carey, his work and the spirit of his work remain a living presence in the Ph.D. program he founded. Several centers and institutes across campus have given generous support to my research throughout my time at Columbia. The Brown Institute for Media Innovation has served as a home base for long stretches of my graduate work, offering both support and intellectual nourishment. Beyond Brown’s director, Mark Hansen, it has been an incredible pleasure to work alongside Michael Krisch, Rosalie Yu, David Riordan, Ann Grimes, Juan Francisco Saldarriaga, Allison McCartney, Bernd Girod, Maneesh iii Agrawala, and the Institute’s many fellows and grantees during its five inaugural years. I was also fortunate to arrive at Columbia just as Emily Bell was launching the Tow Center for Digital Journalism, and over the years she has both supported my work and invited me to assist and collaborate with her own projects. During my time at Tow, I feel especially lucky to have worked with Taylor Owen, Susan McGregor, Fergus Pitt, Claire Wardle, Nushin Rashidian, Smitha Khorana, and Pete Brown. In the broader Journalism School, I have benefitted from the kindness, support, and opportunities presented by many distinguished faculty members. Sheila Coronel, in particular, has been a strong supporter, often entrusting me with tasks well beyond my rank or prior accomplishments. Likewise, Steve Coll has generously supported my work, even contributing the preface to a study published in my fourth year. I am also grateful for the support of the Knight Foundation and the Shorenstein Center at Harvard, who have funded and promoted my work at various points. Many other teachers, at Columbia and elsewhere, have left an indelible mark on my graduate education. Casey Blake, Pablo Boczkowski, Matthew Connelly, Jonathan Crary, Lisa Gitelman, Lydia Goehr, Philip Howard, Branden Joseph, Matthew L. Jones, Mary Kinzie, Paul Kockelman, Brinkley Messick, Mara Mills, Eli Noam, Ross Posnock, Dennis Tenen, Diane Vaughan, and Mark Wigley formed a constellation of influences during my coursework. For the final stretch of writing this dissertation, I served as writer-in-residence at Yeshiva University, where I enjoyed the company of many remarkable colleagues and students as the school launched a new undergraduate program in media studies. Special thanks go to Lauren Fitzgerald, Joanne Jacobson, Liesl Schwabe, and Rachel Mesch for iv bringing me into their department and inviting me to design a completely new course centered on my own research, an opportunity rarely offered to graduate students. Over the course of writing this dissertation, I was amazed at the amount of material that resurfaced from my undergraduate years, long before I found my way to Communication and Media Studies. For this, I must thank an ensemble of world-class teachers and scholars at the University of Michigan: Fred Amrine, Edwin Curley, Sam Epstein, Allan Gibbard, Jim Joyce, Michelle Kosch, Howard Lay, Eric Lormand, Dick Mann, Alex Potts, Ian Proops, Janet Richards, the late Omry Ronen, Silke Weineck, and Rebecca Zurier. I hope they know that their dedication to undergraduate teaching really matters. My colleagues in the Communications Ph.D. program at Columbia have been an anchor of camaraderie and source of inspiration throughout my time here. I want to thank the many remarkable friends and colleagues I met here: Colin Agur, Burcu Baykurt, Gal Beckerman, Lynn Berger, Jonah Bossewitch, Kathy Brown, Deani Citra, Andrea Dixon, Rosalind Donald, Elena Egawhary, Shant Fabricatorian, Kate Fink, Max Foxman, Tom Glasier,

Load more