Rainbow Tables whoami ● Andrew Kramer ● Dakota State University ● Grad Student in Cyber Operations ● SFS Scholarship Recipient ● [email protected] INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information INDEX ● Hash Functions <<< ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information What Is a Hash Function? ● Different than “encryption” ● One-way ● Irreversible (sort of) ● Arbitrary length in, fixed length out ● Pseudo-random ● Few collisions Example Hash Function ● Convert all characters to numbers, i.e... – A=1 – B=2 – Etc... ● Add all the numbers ● Divide the sum mod 26 ● Represent as two digits (zero padded) Example Hash Function ● “foobar” – 6 + 15 + 15 + 2 + 1 + 18 = 57 – 57 mod 26 = 5 – Therefore: MYHASH(“foobar”) = 05 ● “catdog” – 3 + 1 + 20 + 4 + 15 + 7 = 50 – 50 mod 26 = 24 – Therefore: MYHASH(“catdog”) = 24 Real Hash Functions ● MD5 – 128 bits (16 bytes) – Commonly represented as 32 hex characters – md5(“foobar”) = 3858f62230ac3c915f300c664312c63f ● SHA1 – 160 bits (20 bytes) – Commonly represented as 40 hex characters – sha1(“foobar”) = 8843d7f92416211de9ebb963ff4ce28125932878 Why hash passwords? ● [email protected]:i<3computers [email protected]:Password1! [email protected]:kittiesrule Etc... ● [email protected]:5f306463f... [email protected]:ff4c22878... [email protected]:a97fe3c... Etc... INDEX ● Hash Functions ● Cracking Hashes <<< – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information What's a Poor Hacker To Do? ● Two simple ways to recover plaintext – On-the-fly bruteforce ● Hashcat (CPU or GPU) ● John the Ripper (CPU) ● Wordlists (CPU or GPU) – Precomputed lookup tables ● aaaaaa:0b4e7a0e5fe84ad35fb5f95b9ceeac79 ● aaaaab:9dcf6acc37500e699f572645df6e87fc ● aaaaac:52a0a42bc3e1675eccb123b56ea5e3c8 ● Etc... for gigs... and gigs... and gigs... INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce <<< – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information Raw Bruteforce? ● Sooooo slooow – 4GHz 8-core CPU = ~74Million MD5s / sec – [a-zA-Z0-9] ^ 10 = 83,929,936,586,800,000 – ~36 years :( ● Duplicate work for every hash ● The power bill... INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries <<< ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information Precomputed Lookup Lists? ● Requires storage space in the... – Terabytes? – Petabytes?? – Exabytes ??? – Forget it ● [10 chars] + [16 bytes] + [1 bytes '\n'] over [a-zA-Z0-9] ● (10 + 16 + 1) * (26 + 26 + 10)^10 = 2,266,108,287,840,000,000 bytes = ~ 2.266 exabytes :( INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables <<< – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information Rainbow Tables ● Best of both worlds ● Time-memory trade off ● A few hundred gigs + a few hours of CPU – Same coverage as a hundred years of CPU – Same coverage as a petabyte of lookup lists Or More Visually... INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions <<< – Table Generation – Hash Lookup – Potential Problems ● Related Information Reduction Function ● “Opposite” or a hash function ● Input hash... output a plaintext – NOT THE SAME PLAINTEXT ● Fixed length in... arbitrary length out ● Ideally also pseudo-random Example Reduction Function ● For charset=[0-9], length=1-10 – Find all digits in the hash – First digit represents length (0 = 10) – Other digits (up to length) represent plaintext – MYREDUCE(“52a0a42bc3e1675eccb123b56ea5e3c8”) = 20423 – MYREDUCE(“9dcf6acc37500e699f572645df6e87fc”) = 63700699 – MYREDUCE(“0b4e7a0e5fe84ad35fb5f95b9ceeac79”) = 4705843559 INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation <<< – Hash Lookup – Potential Problems ● Related Information We Can Chain Hash/Reduce! ● plaintext -hash()-> HASH -reduce()-> plaintext -hash()-> HASH ● Do this 1,000,000 times ● Only store the FIRST plaintext and LAST hash – 17829: ...lots of iterations... :52a0a42bc3e1675eccb123b56ea5e3c8 – 15186: ...lots of iterations... :9dcf6acc37500e699f572645df6e87fc – 123037: ...lots of iterations... :0b4e7a0e5fe84ad35fb5f95b9ceeac79 ● Each of these chains actually contains 1,000,000 password->hash transitions ● Now do THAT 1,000,000 times ● Congratulations! You stored 1,000,000,000,000 combinations in the space of 1,000,000 INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup <<< – Potential Problems ● Related Information Looking Up a Password ● Hash the password – If one of your chains ends in the hash, you know the password was the last iteration of that chain. ● Reduce and rehash – If one of your chains ends in the hash, you know the password was the second to last iteration of that chain. ● Reduce and rehash – If one of your chains ends in the hash,... third to last ● Etc... 1,000,000 times ● Once you find a matching hash, simply walk that chain by hashing and reducing until you hit your password ● Congratulations! You just covered 1,000,000,000,000 combinations with only 1,000,000 hash cycles INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems <<< ● Related Information Problem #1: Collisions ● When you go from a small plaintext to a large hash, it's unlikely that two plaintexts will produce the same hash. ● When you go from a large hash to a small plaintext, it's quite likely that two hashes will generate the same plaintext. ● Our chains will be constantly colliding (merging) Why Are Collisions Bad? ● LOTS of duplicate work ● Chains will have to be very small ● Hard to detect ● End up throwing out lots of hard work Varied Reduction Functions ● Reduce1(), reduce2(), reduce3(), repeat... ● Use them in sequence ● Lowers the chances of collision ● And if you represent each function with a color... ● That's why they call it a rainbow table :) Problem #2: Detecting Collisions ● Unfortunately, a few collisions WILL still occur ● We need to throw out those chains to achieve maximum efficiency ● How do we know if there has been a collision without wasting lots of CPU work? Collision Detection Solution ● Stop each chain at a predefined point ● For example, when the first 8 characters are “0” ● Every time a chain ends, search the table for a match – If you find one, throw out the shorter chain ● Slightly more work = way better efficiency ● Remember: your chains are different lengths, so you need to search up to the specific chain length Rainbow Table Drawback #1 ● You must do extra work up front – You will inevitably hash more than the keyspace ● Collisions ● Duplicates – However, once the rainbow table has been generated, the work pays off Rainbow Table Drawback #2 ● You will never cover 100% of your keyspace – Good design allows in the >=99.9% range – You WILL miss passwords here and there – The exact numbers and probability can be estimated. Rainbow Table Downside #3 ● Doesn't work for salted hashes :( – Because, two of the same password will generate different hashes ● Breaks our predictable keyspace ● Breaks our chains ● Breaks everything ● PSA: If you store other people's passwords, please make sure they are hashed AND salted! INDEX ● Hash Functions ● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information <<< Other Resources ● https://freerainbowtables.com/ – Distributed rainbow table generation ● Help out! Donate some CPU cycles! – rcracki_mt: Multi-threaded rainbow table lookup – Lots of tables available for free download ● MD5 / SHA1 / LM / NTLM ● Bittorrent downloads are lightning fast – Will ship you the same tables on HDD (for $) ● Total of about 12 TB ● $1,200 Other Great Information ● http://kestas.kuliukas.com/RainbowTables/ ● http://stichintime.wordpress.com/2009/04/09/rai nbow-tables-part-1-introduction/ ● https://www.freerainbowtables.com/en/faq/ EOF ● Thanks! ● Questions? ● Comments? ● Andrew Kramer – Dakota State University ● [email protected] .