Misuse Patterns for the SSL/TLS Protocol by Ali Alkazimi A Dissertation Submitted to the Faculty of College of Engineering & Computer Science In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Florida Atlantic University Boca Raton, FL May 2017 Copyright by Ali Alkazimi 2017 ii ACKNOWLEDGEMENTS I would like to thank my advisor, Dr. Eduardo B. Fernandez, for his guidance during these years. He has been a true mentor by supporting me not only during my research but also in my personal life. I would also like to express my gratitude to my committee members, Dr. Mohammad Ilyas, Dr. Maria Petrie, and the members of the Secure Systems Research Group for all their advice and constructive comments of this dissertation. I would like to thank my beloved wife, Abrar Almoosa, and my kids, Fadak, Mohammed, Jude and Lia for giving me the support during my studies. I want to thank my father and my mother because without them I would not accomplish anything. I also want to thank my brothers (Mohammad and Zaid) and my sister (Noor) for their continuous support. I want to thank the Central Bank of Kuwait for their generous support and for giving me the opportunity to achieve my goals. iv ABSTRACT Author: Ali Alkazimi Title: Misuse Patterns for the SSL/TLS Protocol Institution: Florida Atlantic University Dissertation Advisor: Dr. Eduardo B. Fernandez Degree: Doctor of Philosophy Year: 2017 The SSL/TLS is the main protocol used to provide secure data connection between a client and a server. The main concern of using this protocol is to avoid the secure connection from being breached. Computer systems and their applications are becoming more complex and keeping these secure connections between all the connected components is a challenge. To avoid any new security flaws and protocol connections weaknesses, the SSL/TLS protocol is always releasing newer versions after discovering security bugs and vulnerabilities in any of its previous version. We have described some of the common security flaws in the SSL/TLS protocol by identifying them in the literature and then by analyzing the activities from each of their use cases to find any possible threats. These threats are realized in the form of misuse cases to understand how an attack happens from the point of the attacker. This approach implies the development of some security patterns which will be added as a reference for designing secure systems using the SSL/TLS v protocol. We finally evaluate its security level by using misuse patterns and considering the threat coverage of the models vi MISUSE PATTERNS FOR SSL/TLS PROTOCOL TABLES ............................................................................................................................ xi FIGURES .......................................................................................................................... xii 1. INTRODUCTION ...................................................................................................... 1 2. SSL/TLS THREATS: STATUS AND PATTERNS .................................................. 6 2.1. Introduction ............................................................................................................. 6 2.1.1. Background ................................................................................................... 7 2.1.2. SSL/TLS Protocols ..................................................................................... 12 2.1.3. SSL/TLS Implementations.......................................................................... 14 2.2. Common Vulerabilities and Attacks in the SSL/TLS Protocol ............................ 15 2.3. Vulnerabilities of SSL/TLS .................................................................................. 15 2.4. Relatioship between Attacks, Misuse Threats, Vulnerabilites and their Countermeasures ................................................................................................... 20 2.5. Template of misuse pattern ................................................................................... 22 2.5.1. Name ........................................................................................................... 22 2.5.2. Context ........................................................................................................ 22 2.5.3. Problem ....................................................................................................... 22 2.5.4. Solution ....................................................................................................... 22 2.5.5. Affected system components (Where to look for evidence) (Targets) ....... 23 2.5.6. Known Uses ................................................................................................ 23 2.5.7. Consequences .............................................................................................. 23 vii 2.5.8. Countermeasures and Forensics .................................................................. 23 2.5.9. Related Patterns .......................................................................................... 24 3. MISUSE PATTERN: DENIAL OF SERVICE USING A MESSAGE DROP ATTACK ................................................................................................................. 25 3.1. Misuse Pattern: Denial of Service Using a Message Drop Attack ....................... 26 3.1.1. Intent ........................................................................................................... 26 3.1.2. Context ........................................................................................................ 27 3.1.3. Problem ....................................................................................................... 29 3.1.4. Solution ....................................................................................................... 30 3.1.5. Known Uses ................................................................................................ 31 3.1.6. Consequences .............................................................................................. 33 3.1.7. Countermeausre and Forensics ................................................................... 35 3.1.8. Related Patterns .......................................................................................... 36 4. CIPHER SUITE ROLLBACK: A MISUSE PATTERN FOR THE SSL/TLS CLIENT/SERVER AUTHENTICATION HANDSHAKE PROTOCOL............... 37 4.1. Intent ..................................................................................................................... 37 4.2. Context .................................................................................................................. 38 4.3. Problem ................................................................................................................. 39 4.4. Solution ................................................................................................................. 42 4.5. Known Uses .......................................................................................................... 44 4.6. Consequences ........................................................................................................ 44 4.7. Countermeasures ................................................................................................... 45 4.8. Related Patterns .................................................................................................... 46 viii 5. TRIPLE HANDSHAKE AUTHENTICATION ATTACK ...................................... 48 5.1. Intent ..................................................................................................................... 48 5.2. Context .................................................................................................................. 48 5.3. Problem for the attacker ........................................................................................ 49 5.4. Solution ................................................................................................................. 50 5.4.1. Structure (Affected system components) .................................................... 50 5.4.2. Dynamics .................................................................................................... 50 5.5. Affected system components ................................................................................ 53 5.6. Known uses ........................................................................................................... 54 5.7. Consequences for the attack.................................................................................. 54 5.8. Countermeasures ................................................................................................... 54 5.9. Related patterns ..................................................................................................... 55 6. “HEARTBLEED”: A MISUSE PATTERN FOR THE OPENSSL IMPLEMENTATION OF THE SSL/TLS PROTOCOL ......................................... 56 6.1. Intent ..................................................................................................................... 56 6.2. Context .................................................................................................................. 57 6.3. Problem for the attacker ........................................................................................ 58 6.4. Solution ................................................................................................................