DOCSLIB.ORG

Hashes for the Masses

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Hashes for the Masses #RSAC SESSION ID: AIR-R10F Hashes For the Masses Brad Antoniewicz Sr. Security Researcher Cisco Umbrella (formally OpenDNS) @brad_anton Agenda Hashes $ md5 -s a MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661 $ md5 -s a MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661 $ md5 -s b MD5 ("b") = 92eb5ffee6ae2fec3ad71c777531578f $ md5 -s a MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661 $ md5 -s b MD5 ("b") = 92eb5ffee6ae2fec3ad71c777531578f $ md5 -s A MD5 ("A") = 7fc56270e7a70fa81a5935b72eacbe29 md5 sha0 sha1 sha224 sha512 sha3-224 sha sha3-512 sha256 sha384 md4 md2 shake128 shake256 haval md5 md6 ripemd ripemd-128 ripemd-160 ripemd-256 ripemd-320 gost tiger whirlpool blake blake2 haval5 Cryptographic Hash Functions Uniquely identify items Uniquely identify Malware 000080e0 f7 63 64 06 16 04 02 05 00 30 0d 06 09 2a 86 48 |.cd......0...*.H| 000080f0 86 f7 0d 01 01 05 05 00 03 82 01 01 00 57 d4 c5 |.............W..| 00008100 21 57 1b 14 da 12 32 2c 11 05 0c 04 0f 95 aa d4 |!W....2,........| 00008110 e2 fa f2 c4 6d 68 de 52 52 0d 22 73 c8 85 73 1d |....mh.RR."s..s.| 00008120 ef 51 4e f0 8a 49 b0 68 94 65 51 22 df fd 40 8f |.QN..I.h.eQ"..@.| 00008130 35 b4 68 40 52 bb 25 8e cc fa ce 7c 12 56 d8 c0 |5.h@R.%....|.V..| 00008140 d0 55 05 b5 4a b1 b5 61 fc 95 7c 91 5e 72 d1 28 |.U..J..a..|.^r.(| 00008150 b4 1d 2b 6d da 4b 67 9f 05 fc e6 e7 59 4a 43 5e |..+m.Kg.....YJC^| 00008160One minor 1f change7b 84 37 defeatsbe fe ba everything8a 2b 33 05 94 50 83 b8 fe |.{.7....+3..P...| 00008170 9a db 03 bb 8d 62 f2 19 6d 77 93 e3 75 cf 36 59 |.....b..mw..u.6Y| 00008180 31 5e 08 6d 5d 01 1c 65 53 52 93 76 78 9c 17 0b |1^.m]..eSR.vx...| 00008190 10 1e 94 bc 36 90 56 4b 44 2e 98 d9 a6 31 a1 f0 |....6.VKD....1..| 000081a0 1c 66 94 99 54 73 13 b5 51 38 bc 24 a2 a0 30 3b |.f..Ts..Q8.$..0;| 000081b0 7a 65 87 ac c0 bc 15 f4 5e 30 c7 9d a3 2a 04 df |ze......^0...*..| 000081c0 c4 cf 47 84 6d 9f fd ff 45 c3 a0 19 ac 3c bb ad |..G.m...E....<..| 000081d0 8f 94 f9 b9 c7 c8 71 36 11 ed e8 01 ba b4 75 48 |......q6......uH| 000081e0 82 a4 2a 53 1a 05 05 9b 9d c6 47 1b f5 e8 20 72 |..*S......G... r| 000081f0 a0 1b 4b e5 be 64 fc 38 0d 9a ff 43 41 31 82 01 |..K..d.8...CA1..| $ python -c 'print "A"*10000 + "A"' > test1 $ ssdeep test1 3:Wttkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkZ:YLn,"test1” $ python -c 'print "A"*10000 + "B"' > test2 3:Wttkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkks:YYv, "test2" Similarity ssdeep tlsh bitshred sdhash (bitwise) >>> import pefile >>> pefile.PE(’evil.exe').get_imphash() '0580ee873f5698d357bc06bb429bc19f' https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html Practical Techniques Hash Strip whitespace subroutines Hash asm Strings hash instructions we have plenty of ways to find bad most files are good hash database Malware Hash Databases OWASP FHR Team Cymru MHR VirusShare VirusTotal Good/Benign-Hash Databases Kaspersky Whitelist Encase (Paid) HashKeeper (?) KnownGoods(?) Rolling your own Prior Work Shouts: • hashdog (Par!) • hfind • nsrlsvr • hashdig Requirements • Local and API Accessible • Super fast lookup • Lots of hash types • Feature Framework for ML • Store plaintext Bloom Filter https://www.nist.gov/sites/default/files/dw-2-aafs-2008-bloom.pdf pyhashdd case study demo! #RSAC Apply What You’ve Learned Today Identify known-common files in gold images, VMs, and software packages used in your organization Create a hash database using pyhashdd! Regularly use pyhashdd during investigations to weed out good files, and identify known-bad files. Tag any attacker-created files from systems or websites within the hash database to track actors. 26 #RSAC Thank You! Contact me on Twitter using @brad_anton.
Load more

Recommended publications
  • FIPS 140-2 Non-Proprietary Security Policy
    Kernel Crypto API Cryptographic Module version 1.0 FIPS 140-2 Non-Proprietary Security Policy Version 1.3 Last update: 2020-03-02 Prepared by: atsec information security corporation 9130 Jollyville Road, Suite 260 Austin, TX 78759 www.atsec.com © 2020 Canonical Ltd. / atsec information security This document can be reproduced and distributed only whole and intact, including this copyright notice. Kernel Crypto API Cryptographic Module FIPS 140-2 Non-Proprietary Security Policy Table of Contents 1. Cryptographic Module Specification ..................................................................................................... 5 1.1. Module Overview ..................................................................................................................................... 5 1.2. Modes of Operation ................................................................................................................................. 9 2. Cryptographic Module Ports and Interfaces ........................................................................................ 10 3. Roles, Services and Authentication ..................................................................................................... 11 3.1. Roles .......................................................................................................................................................11 3.2. Services ...................................................................................................................................................11
    [Show full text]
  • Increasing Cryptography Security Using Hash-Based Message
    ISSN (Print) : 2319-8613 ISSN (Online) : 0975-4024 Seyyed Mehdi Mousavi et al. / International Journal of Engineering and Technology (IJET) Increasing Cryptography Security using Hash-based Message Authentication Code Seyyed Mehdi Mousavi*1, Dr.Mohammad Hossein Shakour 2 1-Department of Computer Engineering, Shiraz Branch, Islamic AzadUniversity, Shiraz, Iran . Email : [email protected] 2-Assistant Professor, Department of Computer Engineering, Shiraz Branch, Islamic Azad University ,Shiraz ,Iran Abstract Nowadays, with the fast growth of information and communication technologies (ICTs) and the vulnerabilities threatening human societies, protecting and maintaining information is critical, and much attention should be paid to it. In the cryptography using hash-based message authentication code (HMAC), one can ensure the authenticity of a message. Using a cryptography key and a hash function, HMAC creates the message authentication code and adds it to the end of the message supposed to be sent to the recipient. If the recipient of the message code is the same as message authentication code, the packet will be confirmed. The study introduced a complementary function called X-HMAC by examining HMAC structure. This function uses two cryptography keys derived from the dedicated cryptography key of each packet and the dedicated cryptography key of each packet derived from the main X-HMAC cryptography key. In two phases, it hashes message bits and HMAC using bit Swapp and rotation to left. The results show that X-HMAC function can be a strong barrier against data identification and HMAC against the attacker, so that it cannot attack it easily by identifying the blocks and using HMAC weakness.
    [Show full text]
  • CHAPTER 9: ANALYSIS of the SHA and SHA-L HASH ALGORITHMS
    CHAPTER 9: ANALYSIS OF THE SHA AND SHA-l HASH ALGORITHMS In this chapter the SHA and SHA-l hash functions are analysed. First the SHA and SHA-l hash functions are described along with the relevant notation used in this chapter. This is followed by describing the algebraic structure of the message expansion algorithm used by SHA. We then proceed to exploit this algebraic structure of the message expansion algorithm by applying the generalised analysis framework presented in Chapter 8. We show that it is possible to construct collisions for each of the individual rounds of the SHA hash function. The source code that implements the attack is attached in Appendix F. The same techniques are then applied to SHA-l. SHA is an acronym for Secure Hash Algorithm. SHA and SHA-l are dedicated hash func- tions based on the iterative Damgard-Merkle construction [22] [23]. Both of the round func- tions utilised by these algorithms take a 512 bit input (or a multiple of 512) and produce a 160 bit hash value. SHA was first published as Federal Information Processing Standard 180 (FIPS 180). The secure hash algorithm is based on principles similar to those used in the design of MD4 [10]. SHA-l is a technical revision of SHA and was published as FIPS 180-1 [13]. It is believed that this revision makes SHA-l more secure than SHA [13] [50] [59]. SHA and SHA-l differ from MD4 with regard to the number of rounds used, the size of the hash result and the definition of a single step.
    [Show full text]
  • Fast Hashing and Stream Encryption with Panama
    Fast Hashing and Stream Encryption with Panama Joan Daemen1 and Craig Clapp2 1 Banksys, Haachtesteenweg 1442, B-1130 Brussel, Belgium [email protected] 2 PictureTel Corporation, 100 Minuteman Rd., Andover, MA 01810, USA [email protected] Abstract. We present a cryptographic module that can be used both as a cryptographic hash function and as a stream cipher. High performance is achieved through a combination of low work-factor and a high degree of parallelism. Throughputs of 5.1 bits/cycle for the hashing mode and 4.7 bits/cycle for the stream cipher mode are demonstrated on a com- mercially available VLIW micro-processor. 1 Introduction Panama is a cryptographic module that can be used both as a cryptographic hash function and a stream cipher. It is designed to be very efficient in software implementations on 32-bit architectures. Its basic operations are on 32-bit words. The hashing state is updated by a parallel nonlinear transformation, the buffer operates as a linear feedback shift register, similar to that applied in the compression function of SHA [6]. Panama is largely based on the StepRightUp stream/hash module that was described in [4]. Panama has a low per-byte work factor while still claiming very high security. The price paid for this is a relatively high fixed computational overhead for every execution of the hash function. This makes the Panama hash function less suited for the hashing of messages shorter than the equivalent of a typewritten page. For the stream cipher it results in a relatively long initialization procedure. Hence, in applications where speed is critical, too frequent resynchronization should be avoided.
    [Show full text]
  • MD5 Collisions the Effect on Computer Forensics April 2006
    Paper MD5 Collisions The Effect on Computer Forensics April 2006 ACCESS DATA , ON YOUR RADAR MD5 Collisions: The Impact on Computer Forensics Hash functions are one of the basic building blocks of modern cryptography. They are used for everything from password verification to digital signatures. A hash function has three fundamental properties: • It must be able to easily convert digital information (i.e. a message) into a fixed length hash value. • It must be computationally impossible to derive any information about the input message from just the hash. • It must be computationally impossible to find two files to have the same hash. A collision is when you find two files to have the same hash. The research published by Wang, Feng, Lai and Yu demonstrated that MD5 fails this third requirement since they were able to generate two different messages that have the same hash. In computer forensics hash functions are important because they provide a means of identifying and classifying electronic evidence. Because hash functions play a critical role in evidence authentication, a judge and jury must be able trust the hash values to uniquely identify electronic evidence. A hash function is unreliable when you can find any two messages that have the same hash. Birthday Paradox The easiest method explaining a hash collision is through what is frequently referred to as the Birthday Paradox. How many people one the street would you have to ask before there is greater than 50% probability that one of those people will share your birthday (same day not the same year)? The answer is 183 (i.e.
    [Show full text]
  • BLAKE2: Simpler, Smaller, Fast As MD5
    BLAKE2: simpler, smaller, fast as MD5 Jean-Philippe Aumasson1, Samuel Neves2, Zooko Wilcox-O'Hearn3, and Christian Winnerlein4 1 Kudelski Security, Switzerland [email protected] 2 University of Coimbra, Portugal [email protected] 3 Least Authority Enterprises, USA [email protected] 4 Ludwig Maximilian University of Munich, Germany [email protected] Abstract. We present the hash function BLAKE2, an improved version of the SHA-3 finalist BLAKE optimized for speed in software. Target applications include cloud storage, intrusion detection, or version control systems. BLAKE2 comes in two main flavors: BLAKE2b is optimized for 64-bit platforms, and BLAKE2s for smaller architectures. On 64- bit platforms, BLAKE2 is often faster than MD5, yet provides security similar to that of SHA-3: up to 256-bit collision resistance, immunity to length extension, indifferentiability from a random oracle, etc. We specify parallel versions BLAKE2bp and BLAKE2sp that are up to 4 and 8 times faster, by taking advantage of SIMD and/or multiple cores. BLAKE2 reduces the RAM requirements of BLAKE down to 168 bytes, making it smaller than any of the five SHA-3 finalists, and 32% smaller than BLAKE. Finally, BLAKE2 provides a comprehensive support for tree-hashing as well as keyed hashing (be it in sequential or tree mode). 1 Introduction The SHA-3 Competition succeeded in selecting a hash function that comple- ments SHA-2 and is much faster than SHA-2 in hardware [1]. There is nev- ertheless a demand for fast software hashing for applications such as integrity checking and deduplication in filesystems and cloud storage, host-based intrusion detection, version control systems, or secure boot schemes.
    [Show full text]
  • Cryptanalysis of MD4
    Cryptanalysis of MD4 Hans Dobbertin German Information Security Agency P. O. Box 20 03 63 D-53133 Bonn e-maih dobbert inQskom, rhein .de Abstract. In 1990 Rivest introduced the hash function MD4. Two years later RIPEMD, a European proposal, was designed as a stronger mode of MD4. Recently wc have found an attack against two of three rounds of RIPEMD. As we shall show in the present note, the methods developed to attack RIPEMD can be modified and supplemented such that it is possible to break the full MD4, while previously only partial attacks were known. An implementation of our attack allows to find collisions for MD4 in a few seconds on a PC. An example of a collision is given demonstrating that our attack is of practical relevance. 1 Introduction Rivest [7] introduced the hash function MD4 in 1990. The MD4 algorithm is defined as an iterative application of a three-round compress function. After an unpublished attack on the first two rounds of MD4 due to Merkle and an attack against the last two rounds by den Boer and Bosselaers [2], Rivest introduced the strengthened version MD5 [8]. The most important difference to MD4 is the adding of a fourth round. On the other hand the stronger mode RIPEMD [1] of MD4 was designed as a European proposal in 1992. The compress function of RIPEMD consists of two parallel lines of a modified version of the MD4 compress function. In [4] we have shown that if the first or the last round of its compress function is omitted, then RIPEMD is not collision-free.
    [Show full text]
  • Efficient Collision Attack Frameworks for RIPEMD-160
    Efficient Collision Attack Frameworks for RIPEMD-160 Fukang Liu1;6, Christoph Dobraunig2;3, Florian Mendel4, Takanori Isobe5;6, Gaoli Wang1?, and Zhenfu Cao1? 1 Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai, China [email protected],fglwang,[email protected] 2 Graz University of Technology, Austria 3 Radboud University, Nijmegen, The Netherlands [email protected] 4 Infineon Technologies AG, Germany [email protected] 5 National Institute of Information and Communications Technology, Japan 6 University of Hyogo, Japan [email protected] Abstract. RIPEMD-160 is an ISO/IEC standard and has been applied to gen- erate the Bitcoin address with SHA-256. Due to the complex dual-stream struc- ture, the first collision attack on reduced RIPEMD-160 presented by Liu, Mendel and Wang at Asiacrypt 2017 only reaches 30 steps, having a time complexity of 270. Apart from that, several semi-free-start collision attacks have been published for reduced RIPEMD-160 with the start-from-the-middle method. Inspired from such start-from-the middle structures, we propose two novel efficient collision at- tack frameworks for reduced RIPEMD-160 by making full use of the weakness of its message expansion. Those two frameworks are called dense-left-and-sparse- right (DLSR) framework and sparse-left-and-dense-right (SLDR) framework. As it turns out, the DLSR framework is more efficient than SLDR framework since one more step can be fully controlled, though with extra 232 memory complexi- ty. To construct the best differential characteristics for the DLSR framework, we carefully build the linearized part of the characteristics and then solve the cor- responding nonlinear part using a guess-and-determine approach.
    [Show full text]
  • Design Principles for Hash Functions Revisited
    Design Principles for Hash Functions Revisited Bart Preneel Katholieke Universiteit Leuven, COSIC bartDOTpreneel(AT)esatDOTkuleuvenDOTbe http://homes.esat.kuleuven.be/ preneel � October 2005 Outline Definitions • Generic Attacks • Constructions • A Comment on HMAC • Conclusions • are secure; they can be reduced to @ @ 2 classes based on linear transfor­ @ @ mations of variables. The properties @ of these 12 schemes with respect to @ @ weaknesses of the underlying block @ @ cipher are studied. The same ap­ proach can be extended to study keyed hash functions (MACs) based - -63102392168 on block ciphers and hash functions h based on modular arithmetic. Fi­ nally a new attack is presented on a scheme suggested by R. Merkle. This slide is now shown at the VI Spanish meeting on Information Se­ curity and Cryptology in a presenta­ tion on the state of hash functions. 2 Informal definitions (1) no secret parameters • x arbitrary length fixed length n • ) computation “easy” • One Way Hash Function (OWHF): preimage resistant: ! h(x) x with h(x) = h(x ) • 6) 0 0 2nd preimage resistant: • ! x; h(x) x (= x) with h(x ) = h(x) 6) 0 6 0 Collision Resistant Hash Function (CRHF) = OWHF + collision resistant: • x, x (x = x) with h(x) = h(x ). 6) 0 0 6 0 3 Informal definitions (2) preimage resistant 2nd preimage resistant 6) take a preimage resistant hash function; add an input bit b and • replace one input bit by the sum modulo 2 of this input bit and b 2nd preimage resistant preimage resistant 6) if h is OWHF, h is 2nd preimage resistant but not preimage • resistant 0 X if X n h(X) = 1kh(X) otherwise.j j < ( k collision resistant 2nd preimage resistant ) [Simon 98] one cannot derive collision resistance from ‘general’ preimage resistance 4 Formal definitions: (2nd) preimage resistance Notation: L = 0; 1 , l(n) > n f g A one-way hash function H is a function with domain D = Ll(n) and range R = Ln that satisfies the following conditions: preimage resistance: let x be selected uniformly in D and let M • be an adversary that on input h(x) uses time t and outputs < M(h(x)) D.
    [Show full text]
  • Security Analysis of BLAKE2's Modes of Operation
    Security Analysis of BLAKE2's Modes of Operation Atul Luykx, Bart Mennink, Samuel Neves KU Leuven (Belgium) and Radboud University (The Netherlands) FSE 2017 March 7, 2017 1 / 14 BLAKE2 m1 m2 m3 m 0∗ `k IV PB F F F F H(m) ⊕ t1 f1 t2 f2 t3 f3 t` f` Cryptographic hash function • Aumasson, Neves, Wilcox-O'Hearn, Winnerlein (2013) • Simplication of SHA-3 nalist BLAKE • 2 / 14 BLAKE2 Use in Password Hashing Argon2 (Biryukov et al.) • Catena (Forler et al.) • Lyra (Almeida et al.) • Lyra2 (Simplício Jr. et al.) • Rig (Chang et al.) • Use in Authenticated Encryption AEZ (Hoang et al.) • Applications Noise Protocol Framework (Perrin) • Zcash Protocol (Hopwood et al.) • RAR 5.0 (Roshal) • 3 / 14 BLAKE2 Guo et al. 2014 Hao 2014 Khovratovich et al. 2015 Espitau et al. 2015 ??? Even slight modications may make a scheme insecure! Security Inheritance? BLAKE cryptanalysis Aumasson et al. 2010 Biryukov et al. 2011 Dunkelman&K. 2011 generic Andreeva et al. 2012 Chang et al. 2012 4 / 14 ??? Even slight modications may make a scheme insecure! Security Inheritance? BLAKE BLAKE2 cryptanalysis Aumasson et al. 2010 Guo et al. 2014 Biryukov et al. 2011 Hao 2014 Dunkelman&K. 2011 Khovratovich et al. 2015 Espitau et al. 2015 generic Andreeva et al. 2012 Chang et al. 2012 4 / 14 Even slight modications may make a scheme insecure! Security Inheritance? BLAKE BLAKE2 cryptanalysis Aumasson et al. 2010 Guo et al. 2014 Biryukov et al. 2011 Hao 2014 Dunkelman&K. 2011 Khovratovich et al. 2015 Espitau et al. 2015 generic Andreeva et al.
    [Show full text]
  • Extending NIST's CAVP Testing of Cryptographic Hash Function
    Extending NIST’s CAVP Testing of Cryptographic Hash Function Implementations Nicky Mouha and Christopher Celi National Institute of Standards and Technology, Gaithersburg, MD, USA [email protected],[email protected] Abstract. This paper describes a vulnerability in Apple’s CoreCrypto library, which affects 11 out of the 12 implemented hash functions: every implemented hash function except MD2 (Message Digest 2), as well as several higher-level operations such as the Hash-based Message Authen- tication Code (HMAC) and the Ed25519 signature scheme. The vulnera- bility is present in each of Apple’s CoreCrypto libraries that are currently validated under FIPS 140-2 (Federal Information Processing Standard). For inputs of about 232 bytes (4 GiB) or more, the implementations do not produce the correct output, but instead enter into an infinite loop. The vulnerability shows a limitation in the Cryptographic Algorithm Validation Program (CAVP) of the National Institute of Standards and Technology (NIST), which currently does not perform tests on hash func- tions for inputs larger than 65 535 bits. To overcome this limitation of NIST’s CAVP, we introduce a new test type called the Large Data Test (LDT). The LDT detects vulnerabilities similar to that in CoreCrypto in implementations submitted for validation under FIPS 140-2. Keywords: CVE-2019-8741, FIPS, CAVP, ACVP, Apple, CoreCrypto, hash function, vulnerability. 1 Introduction The security of cryptography in practice relies not only on the resistance of the algorithms against cryptanalytical attacks, but also on the correctness and robustness of their implementations. Software implementations are vulnerable to software faults, also known as bugs.
    [Show full text]
  • The Missing Difference Problem, and Its Applications to Counter Mode
    The Missing Difference Problem, and its Applications to Counter Mode Encryption? Ga¨etanLeurent and Ferdinand Sibleyras Inria, France fgaetan.leurent,[email protected] Abstract. The counter mode (CTR) is a simple, efficient and widely used encryption mode using a block cipher. It comes with a security proof that guarantees no attacks up to the birthday bound (i.e. as long as the number of encrypted blocks σ satisfies σ 2n=2), and a matching attack that can distinguish plaintext/ciphertext pairs from random using about 2n=2 blocks of data. The main goal of this paper is to study attacks against the counter mode beyond this simple distinguisher. We focus on message recovery attacks, with realistic assumptions about the capabilities of an adversary, and evaluate the full time complexity of the attacks rather than just the query complexity. Our main result is an attack to recover a block of message with complexity O~(2n=2). This shows that the actual security of CTR is similar to that of CBC, where collision attacks are well known to reveal information about the message. To achieve this result, we study a simple algorithmic problem related to the security of the CTR mode: the missing difference problem. We give efficient algorithms for this problem in two practically relevant cases: where the missing difference is known to be in some linear subspace, and when the amount of data is higher than strictly required. As a further application, we show that the second algorithm can also be used to break some polynomial MACs such as GMAC and Poly1305, with a universal forgery attack with complexity O~(22n=3).
    [Show full text]