An Analysis of the Transport Layer Security Protocol Thyla van der Merwe Thesis submitted to the University of London for the degree of Doctor of Philosophy Information Security Group School of Mathematics and Information Security Royal Holloway, University of London 2018 Declaration These doctoral studies were conducted under the supervision of Professor Kenneth G. Paterson. The work presented in this thesis is the result of original research I conducted, in collabo- ration with others, whilst enrolled in the School of Mathematics and Information Security as a candidate for the degree of Doctor of Philosophy. This work has not been submitted for any other degree or award in any other university or educational establishment. Thyla van der Merwe March, 2018 2 Dedication To my niece, Emma. May you always believe in your abilities, no matter what anybody tells you, and may you draw on the strength of our family for support, as I have done (especially your Gogo, she’s one tough lady). “If you’re going through hell, keep going.” Winston Churchill 3 Abstract The Transport Layer Security (TLS) protocol is the de facto means for securing commu- nications on the World Wide Web. Originally developed by Netscape Communications, the protocol came under the auspices of the Internet Engineering Task Force (IETF) in the mid 1990s and today serves millions, if not billions, of users on a daily basis. The ubiquitous nature of the protocol has, especially in recent years, made the protocol an attractive target for security researchers. Since the release of TLS 1.2 in 2008, the protocol has suffered many high-profile, and increasingly practical, attacks. Coupled with pressure to improve the protocol’s efficiency, this deluge of identified weaknesses prompted the IETF to develop a new version of the protocol, namely TLS 1.3. In the development of the new version of the protocol, the IETF TLS Working Group has adopted an “analysis-prior-to-deployment” design philosophy. This is in sharp contrast to all previous versions of the protocol. We present an account of the TLS standardisation narrative, commenting on the differences between the reactive development process for TLS 1.2 and below, and the more proactive design process for TLS 1.3. As part of this account, we present work that falls on both sides of this design transition. We contribute to the large body of work highlighting weaknesses in TLS 1.2 and below by presenting two classes of attacks against the RC4 stream cipher when used in TLS. Our attacks exploit statistical biases in the RC4 keystream to recover TLS-protected user passwords and cookies. Next we present a symbolic analysis of the TLS 1.3 draft specification, using the Tamarin prover, to show that TLS 1.3 meets the desired goals of authenticated key exchange, thus contributing to a concerted effort by the TLS community to ensure the protocol’s robustness prior to its official release. 4 Contents Publications 9 Acknowledgements 10 I Motivation and Background 12 1 Introduction 13 1.1 Motivation . 13 1.2 Thesis Structure . 16 2 Preliminaries 19 2.1 The TLS Protocol . 19 2.2 TLS 1.2 and Below . 21 2.2.1 The Handshake Protocol . 21 2.2.2 The Record Protocol . 24 2.2.3 Security Properties . 24 2.3 TLS 1.3 . 26 2.4 TLS 1.3 draft-10 ................................ 28 2.4.1 The Handshake Protocol . 28 2.4.2 The Record Protocol . 32 2.4.3 Security Properties . 32 2.5 TLS 1.3 draft-21 ................................ 34 2.5.1 The Handshake Protocol . 34 2.5.2 The Record Protocol . 38 2.5.3 Post-Handshake Mechanisms . 38 2.5.4 Security Properties . 39 3 Reactive and Proactive Standardisation of TLS 42 3.1 Post-Deployment Analysis . 42 5 CONTENTS 3.1.1 Design, Release, Break, Patch . 43 3.1.2 Fixes, Constraints and Time Lags . 46 3.1.3 Impact and Incentives . 49 3.2 Pre-Deployment Analysis . 49 3.2.1 Design, Break, Fix, Release . 49 3.2.2 Available Tools . 54 3.2.3 Impact and Incentives . 56 II Attacking TLS 1.2 and Below 58 4 Password Recovery Attacks Against RC4 59 4.1 Introduction . 59 4.2 Preliminaries . 64 4.2.1 Bayes’ Theorem . 65 4.2.2 The RC4 Algorithm . 65 4.2.3 Single-byte Biases in the RC4 Keystream . 66 4.2.4 Double-byte Biases in the RC4 Keystream . 67 4.2.5 RC4 and the TLS Record Protocol . 71 4.2.6 Passwords . 73 4.3 Plaintext Recovery via Bayesian Analysis . 74 4.3.1 Formal Bayesian Analysis . 74 4.3.2 Using a Product Distribution . 77 4.3.3 Double-byte-based Approximation . 79 4.4 Simulation Results . 82 4.4.1 Methodology . 82 4.4.2 Results . 84 4.5 Practical Validation . 94 4.5.1 The BasicAuth Protocol . 95 4.5.2 Attacking BasicAuth . 95 4.6 Conclusion . 98 5 Analysing and Exploiting the Mantin Biases in RC4 100 5.1 Introduction . 101 5.2 Preliminaries . 104 5.2.1 Inferential Form of Bayes’ Theorem . 104 6 CONTENTS 5.2.2 Order Statistics . 105 5.2.3 The Mantin Biases . 106 5.2.4 Dynamic Programming Algorithms . 106 5.3 Plaintext Recovery using the Mantin Biases . 107 5.3.1 Maximum Likelihood Estimation . 107 5.3.2 Plaintext Recovery Attack . 110 5.3.3 Distribution of the Maximum Likelihood Statistic and Attack Per- formance . 111 5.3.4 Incorporating Prior Information about Plaintext Bytes . 116 5.4 Recovering Multiple Plaintext Bytes . 119 5.4.1 A Likelihood Analysis for Multiple Plaintext Bytes . 120 5.4.2 Algorithms for Recovering Multiple Plaintext Bytes . 121 5.5 Simulation Results . 123 5.5.1 Methodology . 123 5.5.2 Results . 124 5.6 Conclusion . 126 III Verifying TLS 1.3 130 6 Automated Analysis and Verification of draft-10 131 6.1 Introduction . 131 6.2 Preliminaries . 135 6.2.1 Symbolic Analysis . 135 6.2.2 Tamarin Fundamentals . 136 6.3 draft-10 Analysis . 150 6.3.1 Building the Model . 153 6.3.2 Encoding Security Properties . 158 6.3.3 Analysis and Results . 162 6.3.4 Attacking Post-handshake Client Authentication . 165 6.4 Conclusion . 169 7 Automated Analysis and Verification of draft-21 171 7.1 Introduction . 171 7.2 Preliminaries . 174 7.3 draft-21 Analysis . 174 7 CONTENTS 7.3.1 Building the Model . 178 7.3.2 Encoding Security Properties . 186 7.3.3 Analysis and Results . 194 7.4 Conclusion . 198 IV Concluding Remarks 200 8 Conclusion 201 Bibliography 202 A STS .spthy File 221 8 Publications This thesis is based on the following five publications, to which each author contributed equally: 1. Christina Garman, Kenneth G. Paterson, and Thyla van der Merwe. Attacks Only Get Better: Password Recovery Attacks Against RC4 in TLS. In 24th USENIX Security Symposium, USENIX Security 15, Washington, D.C., USA, August 12-14, 2015., pages 113-128, 2015. 2. Cas Cremers, Marko Horvat, Sam Scott, and Thyla van der Merwe. Automated Analysis and Verification of TLS 1.3: 0-RTT, Resumption and Delayed Authentication. In IEEE Symposium on Security and Privacy, SP 2016, San Jose, CA, USA, May 22-26, 2016., pages 470-485, 2016. 3. Kenneth G. Paterson and Thyla van der Merwe. Reactive and Proactive Standardis- ation of TLS. In Security Standardisation Research - Third International Conference, SSR 2016, Gaithersburg, MD, USA, December 5-6, 2016., pages 160-186, 2016. 4. Cas Cremers, Marko Horvat, Jonathan Hoyland, Sam Scott, and Thyla van der Merwe. A Comprehensive Symbolic Analysis of TLS 1.3. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, pages 1773-1788, 2017. 5. Remi Bricout, Kenneth G. Paterson, Sean Murphy, and Thyla van der Merwe. Analysing and Exploiting the Mantin Biases in RC4. Designs, Codes and Cryptogra- phy, 86(4):743-770, April 2018. 9 Acknowledgements I would like to thank my supervisor, Kenny Paterson, for his guidance and support over the course of my Ph.D., and for facilitating a number of excellent research opportunities. Thank you for always making time for my work. I would also like to thank Cas Cremers for being an incredible mentor, collaborator, and friend. Your constant encouragement has meant the world to me. Thanks go to Eric Rescorla for hosting me at Mozilla as an intern (twice!), and for teaching me a great deal about the protocol upon which this thesis is based, TLS. Your advice, and help, over the years have been instrumental in dictating the direction of my research. I also thank Christine Swart for sparking my interest in cryptography many moons ago. Sam Scott and Jonathan Hoyland, I will never forget our Tamarin adventures, and I thank you for your patience and good humour. Sam, I can’t believe that we ventured down the Tamarin rabbit hole more than once! Thank you for teaching me so much about the tool, and for introducing me to dulce de leche during our time in California. I have been fortunate to have worked with a host of impressive and inspiring collaborators, I thank you all. Sheila Cobourne, I thank you for helping me to start the Women In the Security Domain and/Or Mathematics (WISDOM) group, and for instilling in me a love of cake. I also thank Thalia Laing for her calming influence on the group. I thank my Ph.D. office mates for creating a lively working environment, and my friends, James Hourston, Jayni Shah, Marc Nimmerrichter, Sarah Frewen and Marianne Jonassen for making me laugh when the journey seemed long, dark, and arduous.