Attacks on Hash Functions and Applications PROEFSCHRIFT ter verkrijging van de graad van Doctor aan de Universiteit Leiden, op gezag van Rector Magnificus prof. mr. P.F. van der Heijden, volgens besluit van het College voor Promoties te verdedigen op dinsdag 19 juni 2012 klokke 15.00 uur door Marc Martinus Jacobus Stevens, geboren te Hellevoetsluis in 1981 Samenstelling van de promotiecommissie: Promotores: Prof. dr. R. Cramer (CWI & Universiteit Leiden) Prof. dr. A.K. Lenstra (École Polytechnique Fédérale de Lausanne) Copromotor: Dr. B.M.M. de Weger (Technische Universiteit Eindhoven) Overige leden: Prof. dr. E. Biham (Technion) Dr. B. Schoenmakers (Technische Universiteit Eindhoven) Prof. dr. P. Stevenhagen (Universiteit Leiden) Prof. dr. X. Wang (Tsinghua University) The research in this thesis has been carried out at the Centrum Wiskunde & Informatica in the Netherlands and has been funded by the NWO VICI grant of Prof. dr. R. Cramer. Attacks on Hash Functions and Applications Attacks on Hash Functions and Applications Marc Stevens Printed by Ipskamp Drukkers AMS 2000 Subj. class. code: 94A60 NUR: 919 ISBN: 978-94-6191-317-3 Copyright © M. Stevens, Amsterdam 2012. All rights reserved. Cover design by Ankita Sonone. CONTENTS i Contents I Introduction 1 1 Hash functions 3 1.1 Authenticity and confidentiality . 3 1.2 Rise of a digital world . 4 1.3 Security frameworks . 5 1.4 Cryptographic hash functions . 8 1.5 Differential cryptanalysis . 15 2 MD5 collision attack by Wang et al. 17 2.1 Preliminaries . 17 2.2 Description of MD5 . 19 2.3 Collision attack overview . 21 2.4 Two message block collision . 23 2.5 Differential paths . 23 2.6 Sufficient conditions . 24 2.7 Collision finding . 25 2.8 Tables: differential paths and sufficient conditions . 27 II Contributions 31 3 Our contributions 33 3.1 Overview . 33 3.2 Chosen-prefix collision applications . 33 3.3 Results on MD5 . 34 3.4 Results on SHA-1 . 34 3.5 General results . 36 3.6 Recommendations . 38 3.7 Directions for further research . 39 3.8 Thesis outline . 40 4 Chosen-prefix collision abuse scenarios 43 4.1 Survey . 43 4.2 Creating a rogue Certification Authority certificate . 48 4.3 Nostradamus attack . 56 4.4 Colliding executables . 58 5 Differential cryptanalysis and paths 61 5.1 Introduction . 61 5.2 Definitions and notation . 63 5.3 Fmd4cf: MD4 based compression functions . 65 ii CONTENTS 5.4 Properties of the step functions . 69 5.5 Differential paths . 74 5.6 Differential path construction . 80 6 MD5 89 6.1 Overview . 89 6.2 Differential path construction . 89 6.3 Collision finding . 98 6.4 Identical-prefix collision attack . 101 6.5 Chosen-prefix collision attack . 102 7 SHA-0 and SHA-1 115 7.1 Overview . 116 7.2 Description of SHA-0 and SHA-1 . 117 7.3 Techniques towards collision attacks . 120 7.4 Differential path construction . 128 7.5 Differential cryptanalysis . 140 7.6 SHA-1 near-collision attack . 164 7.7 Chosen-prefix collision attack . 183 8 Detecting collision attacks 187 8.1 Extra line of defense . 187 8.2 Core principle . 188 8.3 Application to MD5 . 189 8.4 Application to SHA-1 . 192 Appendices 192 A MD5 compression function constants 193 B MD5 and SHA-1 bitconditions 195 C MD5 boolean function bitconditions 197 C.1 Bitconditions applied to boolean function F ............... 198 C.2 Bitconditions applied to boolean function G ............... 199 C.3 Bitconditions applied to boolean function H ............... 200 C.4 Bitconditions applied to boolean function I ............... 201 D MD5 chosen-prefix collision birthday search cost 203 E Rogue CA Construction 209 F SHA-1 disturbance vector analysis 219 CONTENTS iii References and indices 225 References 225 Index 234 Notation 239 List of Algorithms 240 List of Figures 240 List of Tables 240 List of Theorems 241 Nederlandse samenvatting 243 Acknowledgments 245 Curriculum Vitae 247 iv 1 Part I Introduction P g . N s . M b g g g. N g s s g. S g g g b s g s. W s , 128 s s g. Recipe 1: Old Hash Recipe 2 1 Hash functions 3 1 Hash functions Contents 1.1 Authenticity and confidentiality . 3 1.2 Rise of a digital world . 4 1.3 Security frameworks . 5 1.3.1 Information theoretic approach . 6 1.3.2 Complexity theoretic approach . 6 1.3.3 System based approach . 7 1.4 Cryptographic hash functions . 8 1.4.1 One-way functions . 8 1.4.2 Cryptographic requirements of hash functions . 8 1.4.3 General attacks . 10 1.4.4 From compression function to hash function . 11 1.4.5 The MD4 style family of hash functions . 13 1.5 Differential cryptanalysis . 15 1.1 Authenticity and confidentiality Authenticity and confidentiality have been big concerns in communications for mil- lennia. One of the first known examples of confidentially sending a message goes back to around 440 BC to “The Histories” written down by Herodotus. According to Herodotus, Histiaeus sent an important message to his son-in-law Aristagoras. As this message was very sensitive he tattooed it on the shaved head of his most-trusted slave. Later, when enough hair had grown back, he sent his slave to Aristagoras with instructions to shave the head of the slave again. There is a risk with hidden messages that somebody finds the hidden message and exposes it to unfriendly parties. This is especially the case for regular communications. Eventually the idea arose to encrypt messages so that they cannot be read even if intercepted and this has led to the development of cryptology, historically known as the art of building and breaking ciphers. Ciphers allow parties to securely communicate by encrypting their messages with some secret knowledge (the secret key) into unreadable ciphertext that can only be read by parties that possess the same secret knowledge. This form of encryption using a single secret key known to both sender and receiver is called symmetric encryption. The most famous ancient cipher, the Caesar cipher, is named after Julius Caesar who encrypted his most important messages with a secret number N by cyclically substituting each letter with the N-th next letter in the alphabet (if N = 3 then A goes to D, B to E, etc.). Authentication of messages, the act of confirming that a message really has been sent by a certain party, usually was achieved by a combination of inspecting the mes- sage (e.g., verify its signature), the ‘envelope’ (e.g., is the wax seal intact) and/or 4 1 HASH FUNCTIONS Figure 1: Symmetric Encryption the messenger (e.g., is the messenger known and/or in service of either party). Sym- metric encryption also provides some form of authentication. After all, no one else knows the secret key and is able to encrypt messages with it. However, this does not prevent the encrypted message from being purposely changed or simply repeated at an opportune moment by unfriendly parties. In general, authentication of someone (or the sender/creator of something) is achieved through mutual knowledge (such as a secret password), possession of a physical token (such as the king’s seal) and/or distinguished marks (such as a known birthmark). 1.2 Rise of a digital world With the invention of electronics, a new world was born. Digital information can be very cheaply stored and copied without loss of information. The rapid development of computers opened up a huge range of new possibilities in processing information. The explosive growth of telecommunication networks has made it very easy to quickly send information over vast distances to anywhere in the world. So not surprisingly, our society has become more and more dependent on information technologies in just a few decades. The main advantage of digital information is that there is no longer a link between the information and its carrier. It can be copied endlessly without any loss of quality. The same information can be stored on, e.g., a hard disk, USB disk, mobile phone or a DVD, or sent through an Ethernet, Wi-Fi, GSM or satellite link. This also introduces a large vulnerability as now authentication of information cannot be guaranteed anymore by simply looking at the physical carrier. Also, ciphers require two parties to first agree on a secret key. Given the vast number of possible parties in the world anyone might want to communicate privately with, establishing and storing secret keys for every possible connection between two parties becomes prohibitive. Solving these security problems and many more that arose has expanded the field of cryptography. Modern cryptography has grown into a science that encompasses much more than only ciphers and has strong links with mathematics, computer sci- ence and (quantum) physics. One of the key innovations of modern cryptography that 1.3 Security frameworks 5 Figure 2: Asymmetric or Public Key Encryption gave rise to new ideas and approaches is public key encryption (also called asymmetric encryption)[DH76, Mer78, RSA78]. Instead of using one secret key for both encryp- tion and decryption, public key encryption uses a key pair consisting of a public key for encryption and a private key for decryption. Suppose everyone publishes the public key part of their properly generated key pair in a public key listing (like a phone book). Now, securely sending a message to someone is possible without first exchanging a secret key in advance anymore, as you simply encrypt your message with his public key as found in the listing. Only he can now decrypt your ciphertext using the private key known only to him. This is a remarkable invention, although it depends heavily on the fact that no one can derive the private key from the public key.