DOCSLIB.ORG

Practical Survey on Hash Tables

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Practical Survey on Hash Tables Practical Survey on Hash Tables Aurelian Țuțuianu In memoriam Mihai Pătraşcu (17 July 1982 – 5 June 2012) “I have no intention to ever teach computer science. I want to teach the love for computer science, and let the learning happen.” Teaching Statement (http://people.csail.mit.edu/mip/docs/job-application07/statements.pdf) Abstract • Hash table definition • Collision resolving schemas: – Chained hashing – Linear and quadratic probing – Cuckoo hashing • Some hash function theory • Simple tabulation hashing Omnipresence of hash tables • Symbol tables in compilers • Cache implementations • Database storages • Manage memory pages in Linux • Route tables • Large number of documents Hash Tables Considering a set of elements 푆 from a finite and much larger universe 푈. A hash table consists of: ℎ푎푠ℎ 푓푢푛푐푡표푛 ℎ: 푈 → {0, . , 푚 − 1} 푣푒푐푡표푟 푣 표푓 푠푧푒 푚 Collisions same hash for two different 26.17.41.60 keys 126.15.12.154 What to do? 202.223.224.33 • Ignore them 7.239.203.66 • Chain colliding values • Skip and try again 176.136.103.233 • Hash and displace • Find a perfect hash function War Story: cache with hash tables Problem: An application which application gets some data from an expensive repository. hash table Solution: Hash table with collision replacement. Key point: a big chunk of users “watched” a lot of common data. Data source Collision Resolution Schemas • Chained hashing • Open hash: linear and quadratic probing • Cuckoo hashing And many many others: perfect hashing, coalesced hashing, Robin Hood hashing, hopscotch hashing, etc. Chained Hashing Each slot contains a linked list. 0 푛 푂( ) = 푂(1) for all operations. 1 푚 푛 2 y Load factor: <1. 푚 3 4 x w • easy to implement 5 z • works with weak hash functions 6 • consumes significant memory • default implementation Linear and quadratic probing All records are stored in the bucket array itself. 0 • Probe – a try to find an empty place. h(x,i) = 4 + i 1 • Linear probing w 2 3 ℎ 푥, = ℎ0(푥) + y 4 • Quadratic probing z 5 + ∗ ℎ 푥, = ℎ0(푥) + x 6 2 War Story: Linear probing trick Min. 1st Qu. Median Mean 3rd Qu. Max. 1 1947 3861 3925 5867 8070 – linear probing 1 8983 18370 21150 35600 50920 – chained hashing War Story: Let it be quadratic! Replace library implementation with a “home-made” hash table 4 hours of work Cuckoo hashing T1 T2 Two hash tables, T1, T2, of size m, and two hash functions h1, h2 : U -> {0, . , m −1}. x Value x stored in cell h1(x) of T1 or in cell h2(x) of T2. y Hash and displace. h1(x) z Lookup is constant in worst case! h2(x) Updates in constant amortized time. w What about hash functions? Any hash function is “good”? What does a “good” hash function mean? Can I have my own? The beginning of time Introduced by Alfred Dumey in 1956 for the symbol table in a compiler. He used a “crazy”, “chaotic”, “random” function h:U->{0..m-1}. h(x)=(x mod p) mod m, with p a big prime number. Is seems to work, but why? First station: rigorous analysis Consider that h really is a random function! Knuth established a way to make a complete analysis, but based on a false assumption. No matter how long you stare at h(x)=(x mod p) mod m, it will not morph into a random function! Next station: universality and k-independence Wegman and Carter (1978) • A family of hash functions • No need of perfect random hash function, but universal : 1 ∀ x ,x ∈ S | x ≠ x , Pr[h(x )=h(x )] ≤ 1 2 1 2 1 2 푁 In generalized form the k-independence model uses statistics to measure “how much random” can a family of hash functions produce! How it works? Random data x formula h(x) • Universal multiplicative shift: 표푢푡 ℎ푎 푥 = 푎 ∗ 푥 ≫ 푙 − 푙 • 2-independent multiplicative shift: 표푢푡 ℎ푎,푏 푥 = 푎 ∗ 푥 + 푏 ≫ 2푙 − 푙 • k-independent polynomial hashing: 푘−1 푖 푙표푢푡 ℎ 푥 = 푖=0 푎푖푥 푚표푑 푝 푚표푑 2 Facts on k-independence Chained hashing 1978 - Wegman, Carter: requires only universal hashing Linear probing 1990 – Siegel, Schmidt: O(logn)-independece is enough 2007 – Pagh – 5-independence suffices 2010 – Patrascu,Thorup – 4-independence is not enough Cuckoo hashing 2001 – Pagh: O(logn)-independence is enough 2005 – Cohen, Kane: 5-independence is not enough 2006 – Cohen, Kane: 6-independence is enough Simple tabulation hashing Simple tabulation is the fastest 3-independent family of hash functions known. • Key 푥 of length 푙푒푛 (required bit width to store values) is divided into 푐 chars 푥1, 푥2, . , 푥푐 • We create c tables 푅1, 푅2, .., 푅푐, filled with independent random values • Hash value is created with function ℎ 푥 = 푅1 푥1 ⊗ 푅2 푥2 ⊗ ⋯ ⊗ 푅푐 푥푐 푅 푥 ⊗ 푅 푥 ⊗ 푥 1 1 2 2 푅3 푥3 ⊗ 푅4 푥4 4 lookup tables with random 8-bit ℎ(푥) values The power of simple tabulation! “The power of simple tabulation hashing” – Mihai Pătrașcu, Mikkel Thorup – December 6, 2011 According to this paper, even if is only 3-independent, we have: • Constant time for linear probing • Constant time for static cuckoo hashing => There are also other probabilistic properties which can be exploited, other than ones captured in k-independence theory Summary • Easy ways to implement optimal hash tables • Simple scheme to generate a hash function family • Theory produces practical results and is still alive! There are a lot of occasions to apply these ideas, so: Work hard, have fun and make history! Questions? .
Load more

Recommended publications
  • Using Tabulation to Implement the Power of Hashing Using Tabulation to Implement the Power of Hashing Talk Surveys Results from I M
    Talk surveys results from I M. Patras¸cuˇ and M. Thorup: The power of simple tabulation hashing. STOC’11 and J. ACM’12. I M. Patras¸cuˇ and M. Thorup: Twisted Tabulation Hashing. SODA’13 I M. Thorup: Simple Tabulation, Fast Expanders, Double Tabulation, and High Independence. FOCS’13. I S. Dahlgaard and M. Thorup: Approximately Minwise Independence with Twisted Tabulation. SWAT’14. I T. Christiani, R. Pagh, and M. Thorup: From Independence to Expansion and Back Again. STOC’15. I S. Dahlgaard, M.B.T. Knudsen and E. Rotenberg, and M. Thorup: Hashing for Statistics over K-Partitions. FOCS’15. Using Tabulation to Implement The Power of Hashing Using Tabulation to Implement The Power of Hashing Talk surveys results from I M. Patras¸cuˇ and M. Thorup: The power of simple tabulation hashing. STOC’11 and J. ACM’12. I M. Patras¸cuˇ and M. Thorup: Twisted Tabulation Hashing. SODA’13 I M. Thorup: Simple Tabulation, Fast Expanders, Double Tabulation, and High Independence. FOCS’13. I S. Dahlgaard and M. Thorup: Approximately Minwise Independence with Twisted Tabulation. SWAT’14. I T. Christiani, R. Pagh, and M. Thorup: From Independence to Expansion and Back Again. STOC’15. I S. Dahlgaard, M.B.T. Knudsen and E. Rotenberg, and M. Thorup: Hashing for Statistics over K-Partitions. FOCS’15. I Providing algorithmically important probabilisitic guarantees akin to those of truly random hashing, yet easy to implement. I Bridging theory (assuming truly random hashing) with practice (needing something implementable). I Many randomized algorithms are very simple and popular in practice, but often implemented with too simple hash functions, so guarantees only for sufficiently random input.
    [Show full text]
  • The Power of Tabulation Hashing
    Thank you for inviting me to China Theory Week. Joint work with Mihai Patras¸cuˇ . Some of it found in Proc. STOC’11. The Power of Tabulation Hashing Mikkel Thorup University of Copenhagen AT&T Joint work with Mihai Patras¸cuˇ . Some of it found in Proc. STOC’11. The Power of Tabulation Hashing Mikkel Thorup University of Copenhagen AT&T Thank you for inviting me to China Theory Week. The Power of Tabulation Hashing Mikkel Thorup University of Copenhagen AT&T Thank you for inviting me to China Theory Week. Joint work with Mihai Patras¸cuˇ . Some of it found in Proc. STOC’11. I Providing algorithmically important probabilisitic guarantees akin to those of truly random hashing, yet easy to implement. I Bridging theory (assuming truly random hashing) with practice (needing something implementable). Target I Simple and reliable pseudo-random hashing. I Bridging theory (assuming truly random hashing) with practice (needing something implementable). Target I Simple and reliable pseudo-random hashing. I Providing algorithmically important probabilisitic guarantees akin to those of truly random hashing, yet easy to implement. Target I Simple and reliable pseudo-random hashing. I Providing algorithmically important probabilisitic guarantees akin to those of truly random hashing, yet easy to implement. I Bridging theory (assuming truly random hashing) with practice (needing something implementable). Applications of Hashing Hash tables (n keys and 2n hashes: expect 1/2 keys per hash) I chaining: follow pointers • x • ! a ! t • • ! v • • ! f ! s
    [Show full text]
  • SAHA: a String Adaptive Hash Table for Analytical Databases
    applied sciences Article SAHA: A String Adaptive Hash Table for Analytical Databases Tianqi Zheng 1,2,* , Zhibin Zhang 1 and Xueqi Cheng 1,2 1 CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; [email protected] (Z.Z.); [email protected] (X.C.) 2 University of Chinese Academy of Sciences, Beijing 100049, China * Correspondence: [email protected] Received: 3 February 2020; Accepted: 9 March 2020; Published: 11 March 2020 Abstract: Hash tables are the fundamental data structure for analytical database workloads, such as aggregation, joining, set filtering and records deduplication. The performance aspects of hash tables differ drastically with respect to what kind of data are being processed or how many inserts, lookups and deletes are constructed. In this paper, we address some common use cases of hash tables: aggregating and joining over arbitrary string data. We designed a new hash table, SAHA, which is tightly integrated with modern analytical databases and optimized for string data with the following advantages: (1) it inlines short strings and saves hash values for long strings only; (2) it uses special memory loading techniques to do quick dispatching and hashing computations; and (3) it utilizes vectorized processing to batch hashing operations. Our evaluation results reveal that SAHA outperforms state-of-the-art hash tables by one to five times in analytical workloads, including Google’s SwissTable and Facebook’s F14Table. It has been merged into the ClickHouse database and shows promising results in production. Keywords: hash table; analytical database; string data 1.
    [Show full text]
  • Chapter 5 Hashing
    Introduction hashing performs basic operations, such as insertion, Chapter 5 deletion, and finds in average time Hashing 2 Hashing Hashing Functions a hash table is merely an of some fixed size let be the set of search keys hashing converts into locations in a hash hash functions map into the set of in the table hash table searching on the key becomes something like array lookup ideally, distributes over the slots of the hash table, to minimize collisions hashing is typically a many-to-one map: multiple keys are if we are hashing items, we want the number of items mapped to the same array index hashed to each location to be close to mapping multiple keys to the same position results in a example: Library of Congress Classification System ____________ that must be resolved hash function if we look at the first part of the call numbers two parts to hashing: (e.g., E470, PN1995) a hash function, which transforms keys into array indices collision resolution involves going to the stacks and a collision resolution procedure looking through the books almost all of CS is hashed to QA75 and QA76 (BAD) 3 4 Hashing Functions Hashing Functions suppose we are storing a set of nonnegative integers we can also use the hash function below for floating point given , we can obtain hash values between 0 and 1 with the numbers if we interpret the bits as an ______________ hash function two ways to do this in C, assuming long int and double when is divided by types have the same length fast operation, but we need to be careful when choosing first method uses
    [Show full text]
  • Non-Empty Bins with Simple Tabulation Hashing
    Non-Empty Bins with Simple Tabulation Hashing Anders Aamand and Mikkel Thorup November 1, 2018 Abstract We consider the hashing of a set X U with X = m using a simple tabulation hash function h : U [n] = 0,...,n 1 and⊆ analyse| the| number of non-empty bins, that is, the size of h(X→). We show{ that the− expected} size of h(X) matches that with fully random hashing to within low-order terms. We also provide concentration bounds. The number of non-empty bins is a fundamental measure in the balls and bins paradigm, and it is critical in applications such as Bloom filters and Filter hashing. For example, normally Bloom filters are proportioned for a desired low false-positive probability assuming fully random hashing (see en.wikipedia.org/wiki/Bloom_filter). Our results imply that if we implement the hashing with simple tabulation, we obtain the same low false-positive probability for any possible input. arXiv:1810.13187v1 [cs.DS] 31 Oct 2018 1 Introduction We consider the balls and bins paradigm where a set X U of X = m balls are distributed ⊆ | | into a set of n bins according to a hash function h : U [n]. We are interested in questions → relating to the distribution of h(X) , for example: What is the expected number of non-empty | | bins? How well is h(X) concentrated around its mean? And what is the probability that a | | query ball lands in an empty bin? These questions are critical in applications such as Bloom filters [3] and Filter hashing [7].
    [Show full text]
  • 6.1 Systems of Hash Functions
    — 6 Hashing 6 Hashing 6.1 Systems of hash functions Notation: • The universe U. Usually, the universe is a set of integers f0;:::;U − 1g, which will be denoted by [U]. • The set of buckets B = [m]. • The set X ⊂ U of n items stored in the data structure. • Hash function h : U!B. Definition: Let H be a family of functions from U to [m]. We say that the family is c-universal for some c > 0 if for every pair x; y of distinct elements of U we have c Pr [h(x) = h(y)] ≤ : h2H m In other words, if we pick a hash function h uniformly at random from H, the probability that x and y collide is at most c-times more than for a completely random function h. Occasionally, we are not interested in the specific value of c, so we simply say that the family is universal. Note: We will assume that a function can be chosen from the family uniformly at random in constant time. Once chosen, it can be evaluated for any x in constant time. Typically, we define a single parametrized function ha(x) and let the family H consist of all ha for all possible choices of the parameter a. Picking a random h 2 H is therefore implemented by picking a random a. Technically, this makes H a multi-set, since different values of a may lead to the same function h. Theorem: Let H be a c-universal family of functions from U to [m], X = fx1; : : : ; xng ⊆ U a set of items stored in the data structure, and y 2 U n X another item not stored in the data structure.
    [Show full text]
  • Linear Probing with 5-Independent Hashing
    Lecture Notes on Linear Probing with 5-Independent Hashing Mikkel Thorup May 12, 2017 Abstract These lecture notes show that linear probing takes expected constant time if the hash function is 5-independent. This result was first proved by Pagh et al. [STOC’07,SICOMP’09]. The simple proof here is essentially taken from [Pˇatra¸scu and Thorup ICALP’10]. We will also consider a smaller space version of linear probing that may have false positives like Bloom filters. These lecture notes illustrate the use of higher moments in data structures, and could be used in a course on randomized algorithms. 1 k-independence The concept of k-independence was introduced by Wegman and Carter [21] in FOCS’79 and has been the cornerstone of our understanding of hash functions ever since. A hash function is a random function h : [u] [t] mapping keys to hash values. Here [s]= 0,...,s 1 . We can also → { − } think of a h as a random variable distributed over [t][u]. We say that h is k-independent if for any distinct keys x ,...,x [u] and (possibly non-distinct) hash values y ,...,y [t], we have 0 k−1 ∈ 0 k−1 ∈ Pr[h(x )= y h(x )= y ] = 1/tk. Equivalently, we can define k-independence via two 0 0 ∧···∧ k−1 k−1 separate conditions; namely, (a) for any distinct keys x ,...,x [u], the hash values h(x ),...,h(x ) are independent 0 k−1 ∈ 0 k−1 random variables, that is, for any (possibly non-distinct) hash values y ,...,y [t] and 0 k−1 ∈ i [k], Pr[h(x )= y ]=Pr h(x )= y h(x )= y , and ∈ i i i i | j∈[k]\{i} j j arXiv:1509.04549v3 [cs.DS] 11 May 2017 h i (b) for any x [u], h(x) is uniformly distributedV in [t].
    [Show full text]
  • The Power of Simple Tabulation Hashing
    The Power of Simple Tabulation Hashing Mihai Pˇatra¸scu Mikkel Thorup AT&T Labs AT&T Labs May 10, 2011 Abstract Randomized algorithms are often enjoyed for their simplicity, but the hash functions used to yield the desired theoretical guarantees are often neither simple nor practical. Here we show that the simplest possible tabulation hashing provides unexpectedly strong guarantees. The scheme itself dates back to Carter and Wegman (STOC'77). Keys are viewed as consist- ing of c characters. We initialize c tables T1;:::;Tc mapping characters to random hash codes. A key x = (x1; : : : ; xc) is hashed to T1[x1] ⊕ · · · ⊕ Tc[xc], where ⊕ denotes xor. While this scheme is not even 4-independent, we show that it provides many of the guarantees that are normally obtained via higher independence, e.g., Chernoff-type concentration, min-wise hashing for estimating set intersection, and cuckoo hashing. Contents 1 Introduction 2 2 Concentration Bounds 7 3 Linear Probing and the Concentration in Arbitrary Intervals 13 4 Cuckoo Hashing 19 5 Minwise Independence 27 6 Fourth Moment Bounds 32 arXiv:1011.5200v2 [cs.DS] 7 May 2011 A Experimental Evaluation 39 B Chernoff Bounds with Fixed Means 46 1 1 Introduction An important target of the analysis of algorithms is to determine whether there exist practical schemes, which enjoy mathematical guarantees on performance. Hashing and hash tables are one of the most common inner loops in real-world computation, and are even built-in \unit cost" operations in high level programming languages that offer associative arrays. Often, these inner loops dominate the overall computation time.
    [Show full text]
  • Fundamental Data Structures Contents
    Fundamental Data Structures Contents 1 Introduction 1 1.1 Abstract data type ........................................... 1 1.1.1 Examples ........................................... 1 1.1.2 Introduction .......................................... 2 1.1.3 Defining an abstract data type ................................. 2 1.1.4 Advantages of abstract data typing .............................. 4 1.1.5 Typical operations ...................................... 4 1.1.6 Examples ........................................... 5 1.1.7 Implementation ........................................ 5 1.1.8 See also ............................................ 6 1.1.9 Notes ............................................. 6 1.1.10 References .......................................... 6 1.1.11 Further ............................................ 7 1.1.12 External links ......................................... 7 1.2 Data structure ............................................. 7 1.2.1 Overview ........................................... 7 1.2.2 Examples ........................................... 7 1.2.3 Language support ....................................... 8 1.2.4 See also ............................................ 8 1.2.5 References .......................................... 8 1.2.6 Further reading ........................................ 8 1.2.7 External links ......................................... 9 1.3 Analysis of algorithms ......................................... 9 1.3.1 Cost models ......................................... 9 1.3.2 Run-time analysis
    [Show full text]
  • Fast and Powerful Hashing Using Tabulation
    Fast and Powerful Hashing using Tabulation Mikkel Thorup∗ University of Copenhagen January 4, 2017 Abstract Randomized algorithms are often enjoyed for their simplicity, but the hash functions employed to yield the desired probabilistic guaranteesare often too complicated to be practical. Here we survey recent results on how simple hashing schemes based on tabulation provide unexpectedly strong guarantees. Simple tabulation hashing dates back to Zobrist [1970]. Keys are viewed as consisting of c characters and we have precomputed character tables h1, ..., hc mapping characters to random hash values. A key x = (x1, ..., xc) is hashed to h1[x1] h2[x2]..... hc[xc]. This schemes is very fast with character tables in cache. While simple tabulation is⊕ not even 4-independent⊕ ,it does provide many of the guarantees that are normally obtained via higher independence, e.g., linear probing and Cuckoo hashing. Next we consider twisted tabulation where one input character is ”twisted” in a simple way. The resulting hash function has powerful distributional properties: Chernoff-style tail bounds and a very small bias for min-wise hashing. This also yields an extremely fast pseudo-random number generator that is provably good for many classic randomized algorithms and data-structures. Finally, we consider double tabulation where we compose two simple tabulation functions, applying one to the output of the other, and show that this yields very high independence in the classic framework of Carter and Wegman [1977]. In fact, w.h.p., for a given set of size proportionalto that of the space con- sumed, double tabulation gives fully-random hashing. We also mention some more elaborate tabulation schemes getting near-optimal independence for given time and space.
    [Show full text]
  • Fast Hashing with Strong Concentration Bounds
    Fast hashing with strong concentration bounds Aamand, Anders; Knudsen, Jakob Bæk Tejs; Knudsen, Mathias Bæk Tejs; Rasmussen, Peter Michael Reichstein; Thorup, Mikkel Published in: STOC 2020 - Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing DOI: 10.1145/3357713.3384259 Publication date: 2020 Document version Publisher's PDF, also known as Version of record Document license: Unspecified Citation for published version (APA): Aamand, A., Knudsen, J. B. T., Knudsen, M. B. T., Rasmussen, P. M. R., & Thorup, M. (2020). Fast hashing with strong concentration bounds. In K. Makarychev, Y. Makarychev, M. Tulsiani, G. Kamath, & J. Chuzhoy (Eds.), STOC 2020 - Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing (pp. 1265-1278). Association for Computing Machinery. Proceedings of the Annual ACM Symposium on Theory of Computing https://doi.org/10.1145/3357713.3384259 Download date: 25. sep.. 2021 Fast Hashing with Strong Concentration Bounds Anders Aamand Jakob Bæk Tejs Knudsen Mathias Bæk Tejs Knudsen [email protected] [email protected] [email protected] BARC, University of Copenhagen BARC, University of Copenhagen SupWiz Denmark Denmark Denmark Peter Michael Reichstein Mikkel Thorup Rasmussen [email protected] [email protected] BARC, University of Copenhagen BARC, University of Copenhagen Denmark Denmark ABSTRACT KEYWORDS Previous work on tabulation hashing by Patraşcuˇ and Thorup from hashing, Chernoff bounds, concentration bounds, streaming algo- STOC’11 on simple tabulation and from SODA’13 on twisted tabu- rithms, sampling lation offered Chernoff-style concentration bounds on hash based ACM Reference Format: sums, e.g., the number of balls/keys hashing to a given bin, but Anders Aamand, Jakob Bæk Tejs Knudsen, Mathias Bæk Tejs Knudsen, Peter under some quite severe restrictions on the expected values of Michael Reichstein Rasmussen, and Mikkel Thorup.
    [Show full text]
  • Double Hashing
    CSE 332: Data Structures & Parallelism Lecture 11:More Hashing Ruth Anderson Winter 2018 Today • Dictionaries – Hashing 1/31/2018 2 Hash Tables: Review • Aim for constant-time (i.e., O(1)) find, insert, and delete – “On average” under some reasonable assumptions • A hash table is an array of some fixed size hash table – But growable as we’ll see 0 client hash table library collision? collision E int table-index … resolution TableSize –1 1/31/2018 3 Hashing Choices 1. Choose a Hash function 2. Choose TableSize 3. Choose a Collision Resolution Strategy from these: – Separate Chaining – Open Addressing • Linear Probing • Quadratic Probing • Double Hashing • Other issues to consider: – Deletion? – What to do when the hash table gets “too full”? 1/31/2018 4 Open Addressing: Linear Probing • Why not use up the empty space in the table? 0 • Store directly in the array cell (no linked list) 1 • How to deal with collisions? 2 • If h(key) is already full, 3 – try (h(key) + 1) % TableSize. If full, 4 – try (h(key) + 2) % TableSize. If full, 5 – try (h(key) + 3) % TableSize. If full… 6 7 • Example: insert 38, 19, 8, 109, 10 8 38 9 1/31/2018 5 Open Addressing: Linear Probing • Another simple idea: If h(key) is already full, 0 / – try (h(key) + 1) % TableSize. If full, 1 / – try (h(key) + 2) % TableSize. If full, 2 / – try (h(key) + 3) % TableSize. If full… 3 / 4 / • Example: insert 38, 19, 8, 109, 10 5 / 6 / 7 / 8 38 9 19 1/31/2018 6 Open Addressing: Linear Probing • Another simple idea: If h(key) is already full, 0 8 – try (h(key) + 1) % TableSize.
    [Show full text]