Rainbow Tables
whoami
● Andrew Kramer ● Dakota State University ● Grad Student in Cyber Operations ● SFS Scholarship Recipient ● [email protected]
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information
INDEX
● Hash Functions <<<
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information
What Is a Hash Function?
● Different than “encryption” ● One-way ● Irreversible (sort of) ● Arbitrary length in, fixed length out ● Pseudo-random ● Few collisions
Example Hash Function
● Convert all characters to numbers, i.e... – A=1 – B=2 – Etc... ● Add all the numbers ● Divide the sum mod 26 ● Represent as two digits (zero padded)
Example Hash Function
● “foobar” – 6 + 15 + 15 + 2 + 1 + 18 = 57 – 57 mod 26 = 5 – Therefore: MYHASH(“foobar”) = 05 ● “catdog” – 3 + 1 + 20 + 4 + 15 + 7 = 50 – 50 mod 26 = 24 – Therefore: MYHASH(“catdog”) = 24
Real Hash Functions
● MD5 – 128 bits (16 bytes) – Commonly represented as 32 hex characters – md5(“foobar”) = 3858f62230ac3c915f300c664312c63f ● SHA1 – 160 bits (20 bytes) – Commonly represented as 40 hex characters – sha1(“foobar”) = 8843d7f92416211de9ebb963ff4ce28125932878
Why hash passwords?
● [email protected]:i<3computers [email protected]:Password1! [email protected]:kittiesrule Etc...
● [email protected]:5f306463f... [email protected]:ff4c22878... [email protected]:a97fe3c... Etc...
INDEX
● Hash Functions
● Cracking Hashes <<< – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information
What's a Poor Hacker To Do?
● Two simple ways to recover plaintext – On-the-fly bruteforce
● Hashcat (CPU or GPU) ● John the Ripper (CPU) ● Wordlists (CPU or GPU) – Precomputed lookup tables
● aaaaaa:0b4e7a0e5fe84ad35fb5f95b9ceeac79 ● aaaaab:9dcf6acc37500e699f572645df6e87fc ● aaaaac:52a0a42bc3e1675eccb123b56ea5e3c8 ● Etc... for gigs... and gigs... and gigs...
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce <<< – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information
Raw Bruteforce?
● Sooooo slooow – 4GHz 8-core CPU = ~74Million MD5s / sec – [a-zA-Z0-9] ^ 10 = 83,929,936,586,800,000 – ~36 years :( ● Duplicate work for every hash ● The power bill...
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries <<< ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information
Precomputed Lookup Lists?
● Requires storage space in the... – Terabytes? – Petabytes?? – Exabytes ??? – Forget it
● [10 chars] + [16 bytes] + [1 bytes '\n'] over [a-zA-Z0-9] ● (10 + 16 + 1) * (26 + 26 + 10)^10 = 2,266,108,287,840,000,000 bytes = ~ 2.266 exabytes :(
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables <<< – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information
Rainbow Tables
● Best of both worlds
● Time-memory trade off
● A few hundred gigs + a few hours of CPU – Same coverage as a hundred years of CPU – Same coverage as a petabyte of lookup lists
Or More Visually...
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions <<< – Table Generation – Hash Lookup – Potential Problems ● Related Information
Reduction Function
● “Opposite” or a hash function ● Input hash... output a plaintext – NOT THE SAME PLAINTEXT ● Fixed length in... arbitrary length out ● Ideally also pseudo-random
Example Reduction Function
● For charset=[0-9], length=1-10 – Find all digits in the hash – First digit represents length (0 = 10) – Other digits (up to length) represent plaintext – MYREDUCE(“52a0a42bc3e1675eccb123b56ea5e3c8”) = 20423 – MYREDUCE(“9dcf6acc37500e699f572645df6e87fc”) = 63700699 – MYREDUCE(“0b4e7a0e5fe84ad35fb5f95b9ceeac79”) = 4705843559
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation <<< – Hash Lookup – Potential Problems ● Related Information
We Can Chain Hash/Reduce!
● plaintext -hash()-> HASH -reduce()-> plaintext -hash()-> HASH ● Do this 1,000,000 times ● Only store the FIRST plaintext and LAST hash – 17829: ...lots of iterations... :52a0a42bc3e1675eccb123b56ea5e3c8 – 15186: ...lots of iterations... :9dcf6acc37500e699f572645df6e87fc – 123037: ...lots of iterations... :0b4e7a0e5fe84ad35fb5f95b9ceeac79 ● Each of these chains actually contains 1,000,000 password->hash transitions ● Now do THAT 1,000,000 times ● Congratulations! You stored 1,000,000,000,000 combinations in the space of 1,000,000
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup <<< – Potential Problems ● Related Information
Looking Up a Password
● Hash the password – If one of your chains ends in the hash, you know the password was the last iteration of that chain. ● Reduce and rehash – If one of your chains ends in the hash, you know the password was the second to last iteration of that chain. ● Reduce and rehash – If one of your chains ends in the hash,... third to last ● Etc... 1,000,000 times
● Once you find a matching hash, simply walk that chain by hashing and reducing until you hit your password
● Congratulations! You just covered 1,000,000,000,000 combinations with only 1,000,000 hash cycles INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems <<< ● Related Information
Problem #1: Collisions
● When you go from a small plaintext to a large hash, it's unlikely that two plaintexts will produce the same hash.
● When you go from a large hash to a small plaintext, it's quite likely that two hashes will generate the same plaintext.
● Our chains will be constantly colliding (merging)
Why Are Collisions Bad?
● LOTS of duplicate work ● Chains will have to be very small ● Hard to detect ● End up throwing out lots of hard work
Varied Reduction Functions
● Reduce1(), reduce2(), reduce3(), repeat...
● Use them in sequence
● Lowers the chances of collision
● And if you represent each function with a color...
● That's why they call it a rainbow table :)
Problem #2: Detecting Collisions
● Unfortunately, a few collisions WILL still occur ● We need to throw out those chains to achieve maximum efficiency ● How do we know if there has been a collision without wasting lots of CPU work?
Collision Detection Solution
● Stop each chain at a predefined point
● For example, when the first 8 characters are “0”
● Every time a chain ends, search the table for a match – If you find one, throw out the shorter chain ● Slightly more work = way better efficiency
● Remember: your chains are different lengths, so you need to search up to the
specific chain length Rainbow Table Drawback #1
● You must do extra work up front – You will inevitably hash more than the keyspace
● Collisions ● Duplicates – However, once the rainbow table has been generated, the work pays off
Rainbow Table Drawback #2
● You will never cover 100% of your keyspace – Good design allows in the >=99.9% range – You WILL miss passwords here and there – The exact numbers and probability can be estimated.
Rainbow Table Downside #3
● Doesn't work for salted hashes :( – Because, two of the same password will generate different hashes ● Breaks our predictable keyspace ● Breaks our chains ● Breaks everything ● PSA: If you store other people's passwords, please make sure they are hashed AND salted!
INDEX
● Hash Functions
● Cracking Hashes – CPU/GPU Bruteforce – Precomputed Dictionaries ● Rainbow Tables – Reduction Functions – Table Generation – Hash Lookup – Potential Problems ● Related Information <<<
Other Resources
● https://freerainbowtables.com/ – Distributed rainbow table generation
● Help out! Donate some CPU cycles! – rcracki_mt: Multi-threaded rainbow table lookup – Lots of tables available for free download
● MD5 / SHA1 / LM / NTLM ● Bittorrent downloads are lightning fast – Will ship you the same tables on HDD (for $)
● Total of about 12 TB ● $1,200
Other Great Information
● http://kestas.kuliukas.com/RainbowTables/
● http://stichintime.wordpress.com/2009/04/09/rai nbow-tables-part-1-introduction/
● https://www.freerainbowtables.com/en/faq/
EOF
● Thanks! ● Questions? ● Comments? ● Andrew Kramer – Dakota State University ● [email protected]