Hash-flooding DoS reloaded: Hash flooding begins? attacks and defenses July 1998 article Jean-Philippe Aumasson, “Designing and attacking Kudelski Security (NAGRA) port scan detection tools” by Solar Designer (Alexander D. J. Bernstein, Peslyak) in Phrack Magazine: University of Illinois at Chicago & Technische Universiteit Eindhoven “In scanlogd, I’m using a hash table to lookup source addresses. Martin Boßlet, This works very well for the Ruby Core Team typical case ::: average lookup time is better than that of a binary search. ::: Hash-flooding DoS reloaded: Hash flooding begins? However, an attacker can attacks and defenses choose her addresses (most July 1998 article likely spoofed) to cause hash Jean-Philippe Aumasson, “Designing and attacking collisions, effectively replacing the Kudelski Security (NAGRA) port scan detection tools” hash table lookup with a linear by Solar Designer (Alexander D. J. Bernstein, search. Depending on how many Peslyak) in Phrack Magazine: University of Illinois at Chicago & entries we keep, this might make Technische Universiteit Eindhoven “In scanlogd, I’m using a hash scanlogd not be able to pick table to lookup source addresses. ::: Martin Boßlet, new packets up in time. I’ve This works very well for the Ruby Core Team solved this problem by limiting typical case ::: average lookup the number of hash collisions, and time is better than that of a discarding the oldest entry with binary search. ::: the same hash value when the limit is reached. Hash-flooding DoS reloaded: Hash flooding begins? However, an attacker can attacks and defenses choose her addresses (most July 1998 article likely spoofed) to cause hash Jean-Philippe Aumasson, “Designing and attacking collisions, effectively replacing the Kudelski Security (NAGRA) port scan detection tools” hash table lookup with a linear by Solar Designer (Alexander D. J. Bernstein, search. Depending on how many Peslyak) in Phrack Magazine: University of Illinois at Chicago & entries we keep, this might make Technische Universiteit Eindhoven “In scanlogd, I’m using a hash scanlogd not be able to pick table to lookup source addresses. ::: Martin Boßlet, new packets up in time. I’ve This works very well for the Ruby Core Team solved this problem by limiting typical case ::: average lookup the number of hash collisions, and time is better than that of a discarding the oldest entry with binary search. ::: the same hash value when the limit is reached. Hash-flooding DoS reloaded: Hash flooding begins? However, an attacker can attacks and defenses choose her addresses (most July 1998 article likely spoofed) to cause hash Jean-Philippe Aumasson, “Designing and attacking collisions, effectively replacing the Kudelski Security (NAGRA) port scan detection tools” hash table lookup with a linear by Solar Designer (Alexander D. J. Bernstein, search. Depending on how many Peslyak) in Phrack Magazine: University of Illinois at Chicago & entries we keep, this might make Technische Universiteit Eindhoven “In scanlogd, I’m using a hash scanlogd not be able to pick table to lookup source addresses. ::: Martin Boßlet, new packets up in time. I’ve This works very well for the Ruby Core Team solved this problem by limiting typical case ::: average lookup the number of hash collisions, and time is better than that of a discarding the oldest entry with binary search. ::: the same hash value when the limit is reached. Hash flooding begins? However, an attacker can choose her addresses (most July 1998 article likely spoofed) to cause hash “Designing and attacking collisions, effectively replacing the port scan detection tools” hash table lookup with a linear by Solar Designer (Alexander search. Depending on how many Peslyak) in Phrack Magazine: entries we keep, this might make “In scanlogd, I’m using a hash scanlogd not be able to pick table to lookup source addresses. new packets up in time. ::: I’ve This works very well for the solved this problem by limiting typical case ::: average lookup the number of hash collisions, and time is better than that of a discarding the oldest entry with binary search. ::: the same hash value when the limit is reached. Hash flooding begins? However, an attacker can This is acceptable for port scans choose her addresses (most (remember, we can’t detect July 1998 article likely spoofed) to cause hash all scans anyway), but might “Designing and attacking collisions, effectively replacing the not be acceptable for detecting port scan detection tools” hash table lookup with a linear other attacks. ::: It is probably by Solar Designer (Alexander search. Depending on how many worth mentioning that similar Peslyak) in Phrack Magazine: entries we keep, this might make issues also apply to things like “In scanlogd, I’m using a hash scanlogd not be able to pick operating system kernels. For table to lookup source addresses. new packets up in time. ::: I’ve example, hash tables are widely This works very well for the solved this problem by limiting used there for looking up active typical case ::: average lookup the number of hash collisions, and connections, listening ports, etc. time is better than that of a discarding the oldest entry with There’re usually other limits binary search. ::: the same hash value when the which make these not really limit is reached. dangerous though, but more research might be needed.” Hash flooding begins? However, an attacker can This is acceptable for port scans choose her addresses (most (remember, we can’t detect July 1998 article likely spoofed) to cause hash all scans anyway), but might “Designing and attacking collisions, effectively replacing the not be acceptable for detecting port scan detection tools” hash table lookup with a linear other attacks. ::: It is probably by Solar Designer (Alexander search. Depending on how many worth mentioning that similar Peslyak) in Phrack Magazine: entries we keep, this might make issues also apply to things like “In scanlogd, I’m using a hash scanlogd not be able to pick operating system kernels. For table to lookup source addresses. new packets up in time. ::: I’ve example, hash tables are widely This works very well for the solved this problem by limiting used there for looking up active typical case ::: average lookup the number of hash collisions, and connections, listening ports, etc. time is better than that of a discarding the oldest entry with There’re usually other limits binary search. ::: the same hash value when the which make these not really limit is reached. dangerous though, but more research might be needed.” Hash flooding begins? However, an attacker can This is acceptable for port scans choose her addresses (most (remember, we can’t detect July 1998 article likely spoofed) to cause hash all scans anyway), but might “Designing and attacking collisions, effectively replacing the not be acceptable for detecting port scan detection tools” hash table lookup with a linear other attacks. ::: It is probably by Solar Designer (Alexander search. Depending on how many worth mentioning that similar Peslyak) in Phrack Magazine: entries we keep, this might make issues also apply to things like “In scanlogd, I’m using a hash scanlogd not be able to pick operating system kernels. For table to lookup source addresses. new packets up in time. ::: I’ve example, hash tables are widely This works very well for the solved this problem by limiting used there for looking up active typical case ::: average lookup the number of hash collisions, and connections, listening ports, etc. time is better than that of a discarding the oldest entry with There’re usually other limits binary search. ::: the same hash value when the which make these not really limit is reached. dangerous though, but more research might be needed.” However, an attacker can This is acceptable for port scans choose her addresses (most (remember, we can’t detect likely spoofed) to cause hash all scans anyway), but might collisions, effectively replacing the not be acceptable for detecting hash table lookup with a linear other attacks. ::: It is probably search. Depending on how many worth mentioning that similar entries we keep, this might make issues also apply to things like scanlogd not be able to pick operating system kernels. For new packets up in time. ::: I’ve example, hash tables are widely solved this problem by limiting used there for looking up active the number of hash collisions, and connections, listening ports, etc. discarding the oldest entry with There’re usually other limits the same hash value when the which make these not really limit is reached. dangerous though, but more research might be needed.” However, an attacker can This is acceptable for port scans Review of classic hash tables choose her addresses (most (remember, we can’t detect Choose ` 2 f1; 2; 4; 8; 16;:::g. likely spoofed) to cause hash all scans anyway), but might collisions, effectively replacing the not be acceptable for detecting Hash table: ` separate linked lists. hash table lookup with a linear other attacks. ::: It is probably Store string s in list #i search. Depending on how many worth mentioning that similar where i = H(s) mod `. entries we keep, this might make issues also apply to things like With n entries in table, scanlogd not be able to pick operating system kernels. For expect n=` entries ::: new packets up in time. I’ve example, hash tables are widely in each linked list. solved this problem by limiting used there for looking up active Choose ` n: the number of hash collisions, and connections, listening ports, etc. expect very short linked lists, discarding the oldest entry with There’re usually other limits so very fast list operations. the same hash value when the which make these not really n limit is reached. dangerous though, but (What if becomes too big? ` ` more research might be needed.” Rehash: replace by 2 .) However, an attacker can This is acceptable for port scans Review of classic hash tables choose her addresses (most (remember, we can’t detect Choose ` 2 f1; 2; 4; 8; 16;:::g. likely spoofed) to cause hash all scans anyway), but might collisions, effectively replacing the not be acceptable for detecting Hash table: ` separate linked lists. hash table lookup with a linear other attacks. ::: It is probably Store string s in list #i search. Depending on how many worth mentioning that similar where i = H(s) mod `. entries we keep, this might make issues also apply to things like With n entries in table, scanlogd not be able to pick operating system kernels. For expect n=` entries ::: new packets up in time. I’ve example, hash tables are widely in each linked list. solved this problem by limiting used there for looking up active Choose ` n: the number of hash collisions, and connections, listening ports, etc. expect very short linked lists, discarding the oldest entry with There’re usually other limits so very fast list operations. the same hash value when the which make these not really n limit is reached. dangerous though, but (What if becomes too big? ` ` more research might be needed.” Rehash: replace by 2 .) However, an attacker can This is acceptable for port scans Review of classic hash tables choose her addresses (most (remember, we can’t detect Choose ` 2 f1; 2; 4; 8; 16;:::g. likely spoofed) to cause hash all scans anyway), but might collisions, effectively replacing the not be acceptable for detecting Hash table: ` separate linked lists. hash table lookup with a linear other attacks. ::: It is probably Store string s in list #i search. Depending on how many worth mentioning that similar where i = H(s) mod `. entries we keep, this might make issues also apply to things like With n entries in table, scanlogd not be able to pick operating system kernels. For expect n=` entries ::: new packets up in time. I’ve example, hash tables are widely in each linked list. solved this problem by limiting used there for looking up active Choose ` n: the number of hash collisions, and connections, listening ports, etc. expect very short linked lists, discarding the oldest entry with There’re usually other limits so very fast list operations. the same hash value when the which make these not really n limit is reached. dangerous though, but (What if becomes too big? ` ` more research might be needed.” Rehash: replace by 2 .) This is acceptable for port scans Review of classic hash tables (remember, we can’t detect Choose ` 2 f1; 2; 4; 8; 16;:::g. all scans anyway), but might not be acceptable for detecting Hash table: ` separate linked lists. other attacks. ::: It is probably Store string s in list #i worth mentioning that similar where i = H(s) mod `. issues also apply to things like With n entries in table, operating system kernels. For expect n=` entries example, hash tables are widely in each linked list. used there for looking up active Choose ` n: connections, listening ports, etc. expect very short linked lists, There’re usually other limits so very fast list operations. which make these not really n dangerous though, but (What if becomes too big? ` ` more research might be needed.” Rehash: replace by 2 .) This is acceptable for port scans Review of classic hash tables e.g. strings one, two, ::: , ten; (remember, we can’t detect H(s) = first byte of s; ` = 256: Choose ` 2 f1; 2; 4; 8; 16;:::g. all scans anyway), but might . not be acceptable for detecting Hash table: ` separate linked lists. . ! other attacks. ::: It is probably Store string s in list #i e eight where i = H(s) mod `. f!four!five worth mentioning that similar . . issues also apply to things like With n entries in table, n!nine operating system kernels. For expect n=` entries o!one example, hash tables are widely in each linked list. p used there for looking up active Choose ` n: q connections, listening ports, etc. expect very short linked lists, r There’re usually other limits so very fast list operations. which make these not really s!six!seven n dangerous though, but (What if becomes too big? t!two!three!ten ` ` . more research might be needed.” Rehash: replace by 2 .) . This is acceptable for port scans Review of classic hash tables e.g. strings one, two, ::: , ten; (remember, we can’t detect H(s) = first byte of s; ` = 256: Choose ` 2 f1; 2; 4; 8; 16;:::g. all scans anyway), but might . not be acceptable for detecting Hash table: ` separate linked lists. . ! other attacks. ::: It is probably Store string s in list #i e eight where i = H(s) mod `. f!four!five worth mentioning that similar . . issues also apply to things like With n entries in table, n!nine operating system kernels. For expect n=` entries o!one example, hash tables are widely in each linked list. p used there for looking up active Choose ` n: q connections, listening ports, etc. expect very short linked lists, r There’re usually other limits so very fast list operations. which make these not really s!six!seven n dangerous though, but (What if becomes too big? t!two!three!ten ` ` . more research might be needed.” Rehash: replace by 2 .) . This is acceptable for port scans Review of classic hash tables e.g. strings one, two, ::: , ten; (remember, we can’t detect H(s) = first byte of s; ` = 256: Choose ` 2 f1; 2; 4; 8; 16;:::g. all scans anyway), but might . not be acceptable for detecting Hash table: ` separate linked lists. . ! other attacks. ::: It is probably Store string s in list #i e eight where i = H(s) mod `. f!four!five worth mentioning that similar . . issues also apply to things like With n entries in table, n!nine operating system kernels. For expect n=` entries o!one example, hash tables are widely in each linked list. p used there for looking up active Choose ` n: q connections, listening ports, etc. expect very short linked lists, r There’re usually other limits so very fast list operations. which make these not really s!six!seven n dangerous though, but (What if becomes too big? t!two!three!ten ` ` . more research might be needed.” Rehash: replace by 2 .) . Review of classic hash tables e.g. strings one, two, ::: , ten; H(s) = first byte of s; ` = 256: Choose ` 2 f1; 2; 4; 8; 16;:::g. . Hash table: ` separate linked lists. . Store string s in list #i e!eight where i = H(s) mod `. f!four!five . . With n entries in table, n!nine expect n=` entries o!one in each linked list. p Choose ` n: q expect very short linked lists, r so very fast list operations. s!six!seven (What if n becomes too big? t!two!three!ten ` ` . Rehash: replace by 2 .) . Review of classic hash tables e.g. strings one, two, ::: , ten; H(s) = first byte of s H(s) = first byte of s; ` = 256: is not a good hash function! Choose ` 2 f1; 2; 4; 8; 16;:::g. . Typical strings often start with t. Hash table: ` separate linked lists. . Very long t list; very slow. Store string s in list #i e!eight where i = H(s) mod `. f!four!five In some applications, most strings . . start with the same letter. With n entries in table, n!nine expect n=` entries o!one in each linked list. p Choose ` n: q expect very short linked lists, r so very fast list operations. s!six!seven (What if n becomes too big? t!two!three!ten ` ` . Rehash: replace by 2 .) . Review of classic hash tables e.g. strings one, two, ::: , ten; H(s) = first byte of s H(s) = first byte of s; ` = 256: is not a good hash function! Choose ` 2 f1; 2; 4; 8; 16;:::g. . Typical strings often start with t. Hash table: ` separate linked lists. . Very long t list; very slow. Store string s in list #i e!eight where i = H(s) mod `. f!four!five In some applications, most strings . . start with the same letter. With n entries in table, n!nine expect n=` entries o!one in each linked list. p Choose ` n: q expect very short linked lists, r so very fast list operations. s!six!seven (What if n becomes too big? t!two!three!ten ` ` . Rehash: replace by 2 .) . Review of classic hash tables e.g. strings one, two, ::: , ten; H(s) = first byte of s H(s) = first byte of s; ` = 256: is not a good hash function! Choose ` 2 f1; 2; 4; 8; 16;:::g. . Typical strings often start with t. Hash table: ` separate linked lists. . Very long t list; very slow. Store string s in list #i e!eight where i = H(s) mod `. f!four!five In some applications, most strings . . start with the same letter. With n entries in table, n!nine expect n=` entries o!one in each linked list. p Choose ` n: q expect very short linked lists, r so very fast list operations. s!six!seven (What if n becomes too big? t!two!three!ten ` ` . Rehash: replace by 2 .) . e.g. strings one, two, ::: , ten; H(s) = first byte of s H(s) = first byte of s; ` = 256: is not a good hash function! . . Typical strings often start with t. e!eight Very long t list; very slow. f!four!five In some applications, most strings . . start with the same letter. n!nine o!one p q r s!six!seven t!two!three!ten . . e.g. strings one, two, ::: , ten; H(s) = first byte of s H(s) = first byte of s; ` = 256: is not a good hash function! . . Typical strings often start with t. e!eight Very long t list; very slow. f!four!five In some applications, most strings . . start with the same letter. n!nine So we use fast hash functions o!one that look at the whole string s. p q 60 years of programmers exploring r hash functions for hash tables s!six!seven ) good speed for typical strings. t!two!three!ten . . e.g. strings one, two, ::: , ten; H(s) = first byte of s H(s) = first byte of s; ` = 256: is not a good hash function! . . Typical strings often start with t. e!eight Very long t list; very slow. f!four!five In some applications, most strings . . start with the same letter. n!nine So we use fast hash functions o!one that look at the whole string s. p q 60 years of programmers exploring r hash functions for hash tables s!six!seven ) good speed for typical strings. t!two!three!ten . What if the strings aren’t typical? . e.g. strings one, two, ::: , ten; H(s) = first byte of s Hashing malicious strings H(s) = first byte of s; ` = 256: is not a good hash function! Attacker provides strings . s ; : : : ; s H s ` . Typical strings often start with t. 1 n with ( 1) mod = ¡¡¡ H s ` e!eight Very long t list; very slow. = ( n) mod . f!four!five In some applications, most strings Then all strings are stored . . start with the same letter. in the same linked list; n!nine linked list becomes very slow. So we use fast hash functions o!one that look at the whole string s. p q 60 years of programmers exploring r hash functions for hash tables s!six!seven ) good speed for typical strings. t!two!three!ten . What if the strings aren’t typical? . e.g. strings one, two, ::: , ten; H(s) = first byte of s Hashing malicious strings H(s) = first byte of s; ` = 256: is not a good hash function! Attacker provides strings . s ; : : : ; s H s ` . Typical strings often start with t. 1 n with ( 1) mod = ¡¡¡ H s ` e!eight Very long t list; very slow. = ( n) mod . f!four!five In some applications, most strings Then all strings are stored . . start with the same letter. in the same linked list; n!nine linked list becomes very slow. So we use fast hash functions o!one that look at the whole string s. p q 60 years of programmers exploring r hash functions for hash tables s!six!seven ) good speed for typical strings. t!two!three!ten . What if the strings aren’t typical? . e.g. strings one, two, ::: , ten; H(s) = first byte of s Hashing malicious strings H(s) = first byte of s; ` = 256: is not a good hash function! Attacker provides strings . s ; : : : ; s H s ` . Typical strings often start with t. 1 n with ( 1) mod = ¡¡¡ H s ` e!eight Very long t list; very slow. = ( n) mod . f!four!five In some applications, most strings Then all strings are stored . . start with the same letter. in the same linked list; n!nine linked list becomes very slow. So we use fast hash functions o!one that look at the whole string s. p q 60 years of programmers exploring r hash functions for hash tables s!six!seven ) good speed for typical strings. t!two!three!ten . What if the strings aren’t typical? . H(s) = first byte of s Hashing malicious strings is not a good hash function! Attacker provides strings Typical strings often start with t. s1; : : : ; sn with H(s1) mod ` = Very long t list; very slow. ¡¡¡ = H(sn) mod `. In some applications, most strings Then all strings are stored start with the same letter. in the same linked list; linked list becomes very slow. So we use fast hash functions that look at the whole string s. 60 years of programmers exploring hash functions for hash tables ) good speed for typical strings. What if the strings aren’t typical? H(s) = first byte of s Hashing malicious strings is not a good hash function! Attacker provides strings Typical strings often start with t. s1; : : : ; sn with H(s1) mod ` = Very long t list; very slow. ¡¡¡ = H(sn) mod `. In some applications, most strings Then all strings are stored start with the same letter. in the same linked list; linked list becomes very slow. So we use fast hash functions that look at the whole string s. Solution: Replace linked list by a safe tree structure, 60 years of programmers exploring at least if list is big. hash functions for hash tables ) good speed for typical strings. What if the strings aren’t typical? H(s) = first byte of s Hashing malicious strings is not a good hash function! Attacker provides strings Typical strings often start with t. s1; : : : ; sn with H(s1) mod ` = Very long t list; very slow. ¡¡¡ = H(sn) mod `. In some applications, most strings Then all strings are stored start with the same letter. in the same linked list; linked list becomes very slow. So we use fast hash functions that look at the whole string s. Solution: Replace linked list by a safe tree structure, 60 years of programmers exploring at least if list is big. hash functions for hash tables ) good speed for typical strings. But implementors are unhappy: this solution throws away the What if the strings aren’t typical? simplicity of hash tables. H(s) = first byte of s Hashing malicious strings December 1999, Bernstein, is not a good hash function! dnscache software (OpenDNS: Attacker provides strings >41000000000000 DNS requests Typical strings often start with t. s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): Very long t list; very slow. ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; In some applications, most strings Then all strings are stored /* to protect against start with the same letter. in the same linked list; hash flooding */ linked list becomes very slow. So we use fast hash functions Discarding cache entries that look at the whole string s. Solution: Replace linked list trivially maintains performance by a safe tree structure, 60 years of programmers exploring if attacker floods hash table. at least if list is big. hash functions for hash tables But what about hash tables in ) good speed for typical strings. But implementors are unhappy: general-purpose programming this solution throws away the What if the strings aren’t typical? languages and libraries? simplicity of hash tables. Can’t throw entries away! H(s) = first byte of s Hashing malicious strings December 1999, Bernstein, is not a good hash function! dnscache software (OpenDNS: Attacker provides strings >41000000000000 DNS requests Typical strings often start with t. s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): Very long t list; very slow. ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; In some applications, most strings Then all strings are stored /* to protect against start with the same letter. in the same linked list; hash flooding */ linked list becomes very slow. So we use fast hash functions Discarding cache entries that look at the whole string s. Solution: Replace linked list trivially maintains performance by a safe tree structure, 60 years of programmers exploring if attacker floods hash table. at least if list is big. hash functions for hash tables But what about hash tables in ) good speed for typical strings. But implementors are unhappy: general-purpose programming this solution throws away the What if the strings aren’t typical? languages and libraries? simplicity of hash tables. Can’t throw entries away! H(s) = first byte of s Hashing malicious strings December 1999, Bernstein, is not a good hash function! dnscache software (OpenDNS: Attacker provides strings >41000000000000 DNS requests Typical strings often start with t. s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): Very long t list; very slow. ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; In some applications, most strings Then all strings are stored /* to protect against start with the same letter. in the same linked list; hash flooding */ linked list becomes very slow. So we use fast hash functions Discarding cache entries that look at the whole string s. Solution: Replace linked list trivially maintains performance by a safe tree structure, 60 years of programmers exploring if attacker floods hash table. at least if list is big. hash functions for hash tables But what about hash tables in ) good speed for typical strings. But implementors are unhappy: general-purpose programming this solution throws away the What if the strings aren’t typical? languages and libraries? simplicity of hash tables. Can’t throw entries away! Hashing malicious strings December 1999, Bernstein, dnscache software (OpenDNS: Attacker provides strings >41000000000000 DNS requests s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; Then all strings are stored /* to protect against in the same linked list; hash flooding */ linked list becomes very slow. Discarding cache entries Solution: Replace linked list trivially maintains performance by a safe tree structure, if attacker floods hash table. at least if list is big. But what about hash tables in But implementors are unhappy: general-purpose programming this solution throws away the languages and libraries? simplicity of hash tables. Can’t throw entries away! Hashing malicious strings December 1999, Bernstein, Bad solution: dnscache software (OpenDNS: Use SHA-3 for H. Attacker provides strings >41000000000000 DNS requests SHA-3 is collision-resistant! s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; Then all strings are stored /* to protect against in the same linked list; hash flooding */ linked list becomes very slow. Discarding cache entries Solution: Replace linked list trivially maintains performance by a safe tree structure, if attacker floods hash table. at least if list is big. But what about hash tables in But implementors are unhappy: general-purpose programming this solution throws away the languages and libraries? simplicity of hash tables. Can’t throw entries away! Hashing malicious strings December 1999, Bernstein, Bad solution: dnscache software (OpenDNS: Use SHA-3 for H. Attacker provides strings >41000000000000 DNS requests SHA-3 is collision-resistant! s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; Then all strings are stored /* to protect against in the same linked list; hash flooding */ linked list becomes very slow. Discarding cache entries Solution: Replace linked list trivially maintains performance by a safe tree structure, if attacker floods hash table. at least if list is big. But what about hash tables in But implementors are unhappy: general-purpose programming this solution throws away the languages and libraries? simplicity of hash tables. Can’t throw entries away! Hashing malicious strings December 1999, Bernstein, Bad solution: dnscache software (OpenDNS: Use SHA-3 for H. Attacker provides strings >41000000000000 DNS requests SHA-3 is collision-resistant! s1; : : : ; sn with H(s1) mod ` = from 50 million Internet users): ¡¡¡ = H(sn) mod `. if (++loop > 100) return 0; Then all strings are stored /* to protect against in the same linked list; hash flooding */ linked list becomes very slow. Discarding cache entries Solution: Replace linked list trivially maintains performance by a safe tree structure, if attacker floods hash table. at least if list is big. But what about hash tables in But implementors are unhappy: general-purpose programming this solution throws away the languages and libraries? simplicity of hash tables. Can’t throw entries away! December 1999, Bernstein, Bad solution: dnscache software (OpenDNS: Use SHA-3 for H. >41000000000000 DNS requests SHA-3 is collision-resistant! from 50 million Internet users):
if (++loop > 100) return 0; /* to protect against hash flooding */
Discarding cache entries trivially maintains performance if attacker floods hash table.
But what about hash tables in general-purpose programming languages and libraries? Can’t throw entries away! December 1999, Bernstein, Bad solution: dnscache software (OpenDNS: Use SHA-3 for H. >41000000000000 DNS requests SHA-3 is collision-resistant! from 50 million Internet users): Two reasons this is bad: if (++loop > 100) return 0; 1. It’s very slow. /* to protect against 2. It doesn’t solve the problem. hash flooding */
Discarding cache entries trivially maintains performance if attacker floods hash table.
But what about hash tables in general-purpose programming languages and libraries? Can’t throw entries away! December 1999, Bernstein, Bad solution: dnscache software (OpenDNS: Use SHA-3 for H. >41000000000000 DNS requests SHA-3 is collision-resistant! from 50 million Internet users): Two reasons this is bad: if (++loop > 100) return 0; 1. It’s very slow. /* to protect against 2. It doesn’t solve the problem. hash flooding */ H(s) mod ` Discarding cache entries is not collision-resistant. trivially maintains performance ` is small: e.g., ` = 220. if attacker floods hash table. No matter how strong H is, But what about hash tables in attacker can easily compute general-purpose programming H(s) mod 220 for many s languages and libraries? to find multicollisions. Can’t throw entries away! December 1999, Bernstein, Bad solution: 2003 USENIX Security dnscache software (OpenDNS: Use SHA-3 for H. Symposium, Crosby–Wallach, >41000000000000 DNS requests SHA-3 is collision-resistant! “Denial of service via from 50 million Internet users): algorithmic complexity attacks”: Two reasons this is bad: if (++loop > 100) return 0; 1. It’s very slow. “We present a new class of /* to protect against 2. It doesn’t solve the problem. low-bandwidth denial of service hash flooding */ attacks ::: if each element H(s) mod ` hashes to the same bucket, Discarding cache entries is not collision-resistant. the hash table will also trivially maintains performance 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” if attacker floods hash table. No matter how strong H is, Attack examples: But what about hash tables in attacker can easily compute Perl programming language, general-purpose programming H(s) mod 220 for many s Squid web cache, etc. languages and libraries? to find multicollisions. Can’t throw entries away! December 1999, Bernstein, Bad solution: 2003 USENIX Security dnscache software (OpenDNS: Use SHA-3 for H. Symposium, Crosby–Wallach, >41000000000000 DNS requests SHA-3 is collision-resistant! “Denial of service via from 50 million Internet users): algorithmic complexity attacks”: Two reasons this is bad: if (++loop > 100) return 0; 1. It’s very slow. “We present a new class of /* to protect against 2. It doesn’t solve the problem. low-bandwidth denial of service hash flooding */ attacks ::: if each element H(s) mod ` hashes to the same bucket, Discarding cache entries is not collision-resistant. the hash table will also trivially maintains performance 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” if attacker floods hash table. No matter how strong H is, Attack examples: But what about hash tables in attacker can easily compute Perl programming language, general-purpose programming H(s) mod 220 for many s Squid web cache, etc. languages and libraries? to find multicollisions. Can’t throw entries away! December 1999, Bernstein, Bad solution: 2003 USENIX Security dnscache software (OpenDNS: Use SHA-3 for H. Symposium, Crosby–Wallach, >41000000000000 DNS requests SHA-3 is collision-resistant! “Denial of service via from 50 million Internet users): algorithmic complexity attacks”: Two reasons this is bad: if (++loop > 100) return 0; 1. It’s very slow. “We present a new class of /* to protect against 2. It doesn’t solve the problem. low-bandwidth denial of service hash flooding */ attacks ::: if each element H(s) mod ` hashes to the same bucket, Discarding cache entries is not collision-resistant. the hash table will also trivially maintains performance 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” if attacker floods hash table. No matter how strong H is, Attack examples: But what about hash tables in attacker can easily compute Perl programming language, general-purpose programming H(s) mod 220 for many s Squid web cache, etc. languages and libraries? to find multicollisions. Can’t throw entries away! Bad solution: 2003 USENIX Security Use SHA-3 for H. Symposium, Crosby–Wallach, SHA-3 is collision-resistant! “Denial of service via algorithmic complexity attacks”: Two reasons this is bad: 1. It’s very slow. “We present a new class of 2. It doesn’t solve the problem. low-bandwidth denial of service attacks ::: if each element H(s) mod ` hashes to the same bucket, is not collision-resistant. the hash table will also 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” No matter how strong H is, Attack examples: attacker can easily compute Perl programming language, H(s) mod 220 for many s Squid web cache, etc. to find multicollisions. Bad solution: 2003 USENIX Security 2011 (28C3), Klink–W¨alde, Use SHA-3 for H. Symposium, Crosby–Wallach, “Efficient denial of service attacks SHA-3 is collision-resistant! “Denial of service via on web application platforms”: algorithmic complexity attacks”: Two reasons this is bad: Java, JRuby, PHP 4, PHP 5, 1. It’s very slow. “We present a new class of Python 2, Python 3, Rubinius, 2. It doesn’t solve the problem. low-bandwidth denial of service Ruby, Apache Geronimo, Apache attacks ::: if each element Tomcat, Oracle Glassfish, Jetty, H(s) mod ` hashes to the same bucket, Plone, Rack, V8 Javascript is not collision-resistant. the hash table will also Engine. 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” oCERT advisory 2011–003. No matter how strong H is, Attack examples: attacker can easily compute Application response: Perl programming language, H(s) mod 220 for many s use secret key to randomize H. Squid web cache, etc. to find multicollisions. ::: but is this secure? Bad solution: 2003 USENIX Security 2011 (28C3), Klink–W¨alde, Use SHA-3 for H. Symposium, Crosby–Wallach, “Efficient denial of service attacks SHA-3 is collision-resistant! “Denial of service via on web application platforms”: algorithmic complexity attacks”: Two reasons this is bad: Java, JRuby, PHP 4, PHP 5, 1. It’s very slow. “We present a new class of Python 2, Python 3, Rubinius, 2. It doesn’t solve the problem. low-bandwidth denial of service Ruby, Apache Geronimo, Apache attacks ::: if each element Tomcat, Oracle Glassfish, Jetty, H(s) mod ` hashes to the same bucket, Plone, Rack, V8 Javascript is not collision-resistant. the hash table will also Engine. 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” oCERT advisory 2011–003. No matter how strong H is, Attack examples: attacker can easily compute Application response: Perl programming language, H(s) mod 220 for many s use secret key to randomize H. Squid web cache, etc. to find multicollisions. ::: but is this secure? Bad solution: 2003 USENIX Security 2011 (28C3), Klink–W¨alde, Use SHA-3 for H. Symposium, Crosby–Wallach, “Efficient denial of service attacks SHA-3 is collision-resistant! “Denial of service via on web application platforms”: algorithmic complexity attacks”: Two reasons this is bad: Java, JRuby, PHP 4, PHP 5, 1. It’s very slow. “We present a new class of Python 2, Python 3, Rubinius, 2. It doesn’t solve the problem. low-bandwidth denial of service Ruby, Apache Geronimo, Apache attacks ::: if each element Tomcat, Oracle Glassfish, Jetty, H(s) mod ` hashes to the same bucket, Plone, Rack, V8 Javascript is not collision-resistant. the hash table will also Engine. 20 ` is small: e.g., ` = 2 . degenerate to a linked list.” oCERT advisory 2011–003. No matter how strong H is, Attack examples: attacker can easily compute Application response: Perl programming language, H(s) mod 220 for many s use secret key to randomize H. Squid web cache, etc. to find multicollisions. ::: but is this secure? 2003 USENIX Security 2011 (28C3), Klink–W¨alde, Symposium, Crosby–Wallach, “Efficient denial of service attacks “Denial of service via on web application platforms”: algorithmic complexity attacks”: Java, JRuby, PHP 4, PHP 5, “We present a new class of Python 2, Python 3, Rubinius, low-bandwidth denial of service Ruby, Apache Geronimo, Apache attacks ::: if each element Tomcat, Oracle Glassfish, Jetty, hashes to the same bucket, Plone, Rack, V8 Javascript the hash table will also Engine. degenerate to a linked list.” oCERT advisory 2011–003. Attack examples: Application response: Perl programming language, use secret key to randomize H. Squid web cache, etc. ::: but is this secure?
“used in code by Google, Microsoft, Yahoo, and many others”
“successor to MurmurHash2”
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) {
uint64_t k = *(uint64_t*)data;
k *= m; k ^= k >> 24; k *= m;
h *= m; h ^= k;
data += 8; len -= 8; }
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) {
uint64_t k = *(uint64_t*)data;
k *= m; k ^= k >> 24; k *= m;
h *= m; h ^= k;
data += 8; len -= 8; }
… /* finalization */ switch (len) {
case 7: h ^= data[6] << 48; case 6: h ^= data[5] << 40; case 5: h ^= data[4] << 32; case 4: h ^= data[3] << 24; case 3: h ^= data[2] << 16; case 2: h ^= data[1] << 8; case 1: h ^= data[0]; h *= m;
};
…
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) { /* first block */
uint64_t k = *(uint64_t*)data;
k *= m; /* inject difference D1 */ k ^= k >> 24; k *= m;
h *= m; h ^= k;
data += 8; len -= 8; }
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) { /* first block */
uint64_t k = *(uint64_t*)data;
k *= m; /* inject difference D1 */ k ^= k >> 24; k *= m; /* diff in k: 0x8000000000000000 */
h *= m; h ^= k;
data += 8; len -= 8; }
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) { /* first block */
uint64_t k = *(uint64_t*)data;
k *= m; /* inject difference D1 */ k ^= k >> 24; k *= m; /* diff in k: 0x8000000000000000 */
h *= m; h ^= k; /* diff in h: 0x8000000000000000 */
data += 8; len -= 8; }
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) { /* second block */
uint64_t k = *(uint64_t*)data;
k *= m; /* inject difference D2 */ k ^= k >> 24; k *= m; /* diff in k: 0x8000000000000000 */
h *= m; h ^= k;
data += 8; len -= 8; }
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) { /* second block */
uint64_t k = *(uint64_t*)data;
k *= m; /* inject difference D2 */ k ^= k >> 24; k *= m; /* diff in k: 0x8000000000000000 */
h *= m; /* diff in h still: 0x8000000000000000 */ h ^= k;
data += 8; len -= 8; }
const uint64_t m = (0xc6a4a793 << 32) | 0x5bd1e995; uint64_t h = seed ^ len;
while (len >= 8) { /* second block */
uint64_t k = *(uint64_t*)data;
k *= m; /* inject difference D2 */ k ^= k >> 24; k *= m; /* diff in k: 0x8000000000000000 */
h *= m; /* diff in h still: 0x8000000000000000 */ h ^= k; /* COLLISION !!! (0x80... ^ 0x80... = 0) */ data += 8; len -= 8; }
M1 M2 M3 M4 M5 M6
collision collision collision
http://crypto.junod.info/2012/12/13/hash-dos-and-btrfs/
def parse_nested_query(qs, d = nil)
params = KeySpaceConstrainedParams.new
(qs || '').split(d ? /[#{d}] */n : DEFAULT_SEP).each do |p|
k, v = p.split('=', 2).map { |s| unescape(s) } normalize_params(params, k, v)
end
return params.to_params_hash end def unescape(s, encoding = Encoding::UTF_8)
URI.decode_www_form_component(s, encoding) end def self.decode_www_form_component(str, enc=Encoding::UTF_8)
raise ArgumentError, "invalid %-encoding (#{str})" unless /\A[^%]*(?:%\h\h[^%]*)*\z/ =~ str
str.gsub(/\+|%\h\h/, TBLDECWWWCOMP_).force_encoding(enc) end
def parse_nested_query(qs, d = nil)
params = KeySpaceConstrainedParams.new
(qs || '').split(d ? /[#{d}] */n : DEFAULT_SEP).each do |p|
k, v = p.split('=', 2).map { |s| unescape(s) } normalize_params(params, k, v)
end
return params.to_params_hash end def normalize_params(params, name, v = nil)
name =~ %r(\A[\[\]]*([^\[\]]+)\]*)
k = $1 || ''
… end
def parse_nested_query(qs, d = nil)
params = KeySpaceConstrainedParams.new
(qs || '').split(d ? /[#{d}] */n : DEFAULT_SEP).each do |p|
k, v = p.split('=', 2).map { |s| unescape(s) } normalize_params(params, k, v)
end
return params.to_params_hash end class KeySpaceConstrainedParams
def []=(key, value)
@size += key.size if key && [email protected]?(key)
raise RangeError, 'exceeded available parameter key space‘ if @size > @limit
@params[key] = value
end end
Fix it
at the root
TM
public String(byte bytes[], int offset, int length, Charset charset) { …
char[] v = StringCoding.decode(charset, bytes, offset, length);
…
} problem, byte[] ?
public String(char value[]) {
int size = value.length; this.offset = 0; this.count = size; this.value = Arrays.copyOf(value, size);
}
http://www.ruby-lang.org/en/news/2012/11/09/ruby19-hashdos-cve-2012-5371/
http://jruby.org/2012/12/03/jruby-1-7-1.html https://github.com/rubinius/rubinius/commit/a9a40fc6a1256bcf6382631b710430105c5dd868
Don’t use MurmurHash CityHash? “Inside Google, where CityHash was developed starting in 2010, we use variants of CityHash64() mainly in hash tables such as hash_map
CityHash is weaker than MurmurHash
CityHash64( 0Y|L&:$;+[&HASH!, 16 ) = b553de6f34e878f CityHash64( JkMR_ 0\7](HASH!, 16 ) = b553de6f34e878f CityHash64(
Python’s hash()? $ python -V Python 2.7.3 $ time -p python -R poc.py 64 candidate solutions Verified solutions for _Py_HashSecret: 145cc9aade7d2453 275daf6070a41b99 945cc9aade7d2453 a75daf6070a41b99 real 0.32 user 0.17 sys 0.02 Python 2.x and 3.x • Randomization of hash() optional (-R) • Instantaneous key recovery • Multicollisions with TMTO
.NET’s Marvin32?
Something designed to be secure? SipHash: a fast short-input PRF
New keyed hash to fix hash-flooding: • Rigorous security requirements and analysis • Speed competitive with that of weak hashes • Can serve as MAC or PRF
Peer-reviewed research paper (A., Bernstein). published at DIAC 2012, INDOCRYPT 2012 SipHash initialization
256-bit state v0 v1 v2 v3 128-bit key k0 k1 v0 = k0 ⊕ 0x736f6d6570736575 v1 = k1 ⊕ 0x646f72616e646f6d v2 = k0 ⊕ 0x6c7967656e657261 v3 = k1 ⊕ 0x7465646279746573
SipHash initialization
256-bit state v0 v1 v2 v3 128-bit key k0 k1 v0 = k0 ⊕ “somepseu” v1 = k1 ⊕ “dorandom” v2 = k0 ⊕ “lygenera” v3 = k1 ⊕ “tedbytes”
SipHash compression
Message parsed as 64-bit words m0, m1, … v3 ⊕= m0 c iterations of SipRound v0 ⊕= m0 SipHash compression
Message parsed as 64-bit words m0, m1, … v3 ⊕= m1 c iterations of SipRound v0 ⊕= m1 SipHash compression
Message parsed as 64-bit words m0, m1, … v3 ⊕= m2 c iterations of SipRound v0 ⊕= m2 SipHash compression
Message parsed as 64-bit words m0, m1, …
Etc. SipRound
SipHash finalization
v2 ⊕= 255 d iterations of SipRound Return v0 ⊕ v1 ⊕ v2 ⊕ v3
SipHash-2-4 hashing 15 bytes Family SipHash-c-d Fast proposal: SipHash-2-4 Conservative proposal: SipHash-4-8
Weaker versions for cryptanalysis: SipHash-1-0, SipHash-2-0, etc. SipHash-1-1, SipHash-2-1, etc. Etc. Security claims
≈ 2128 key recovery ≈ 2192 state recovery ≈ 2128 internal-collision forgery ≈ 2s forgery with probab. 2s-64 Fast diffusion of differences, thanks to optimized rotation counts
Combination of ADD and XOR ensures a high nonlinearity (e.g. against cube attacks) How fast is SipHash-2-4?
On an old AMD Athlon II Neo (@1.6GHz)
Bytes 8 16 32 64 Cycles 123 134 158 204 (per byte) (15.38) (8.38) (4.25) (3.19) MiBps 99 182 359 478
Long messages: 1.44 cycles/byte (1 GiBps)
Proof of simplicity
June 20: paper published online June 28: 18 third-party implementations
C (Floodyberry, Boßlet, Neves); C# (Haynes) Cryptol (Lazar); Erlang, Javascript, PHP (Denis) Go (Chestnykh); Haskell (Hanquez) Java, Ruby (Boßlet); Lisp (Brown); Perl6 (Julin)
Who is using SipHash?
Rust
Perl 5 CRuby
Rubinius Soon: Take home message
Hash-flooding DoS works by enforcing worst case in data structure operations through large multicollisions in the hash function
Java and Rubies found vulnerable, due to their use of MurmurHash v2 or v3 CityHash and Python’s hash are weak too…
SipHash offers both security and performance
SipHash paper, code, etc. available on https://131002.net/siphash
Attacks paper coming soon…