DOCSLIB.ORG

Near-Optimal Space Perfect Hashing Algorithm

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Near-Optimal Space Perfect Hashing Algorithm Near-Optimal Space Perfect Hashing Algorithm Nivio Ziviani Fabiano C. Botelho Department of Computer Science Federal University of Minas Gerais, Brazil IR Course CS Department, UFMG, February 19th, 2014 LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 1 Objective of the Presentation Present a perfect hashing algorithm that uses the idea of partitioning the input key set into small buckets: Key set fits in the internal memory Internal Random Access Memory algorithm ( RAM ) Key set larger than the internal memory External Memory (cache-aware) algorithm ( EM ) LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 2 Objective of the Presentation Present a perfect hashing algorithm that uses the idea of partitioning the input key set into small buckets: Key set fits in the internal memory Internal Random Access Memory algorithm ( RAM ) Key set larger than the internal memory External Memory (cache-aware) algorithm ( EM ) Theoretically well-founded, time efficient, higly scalable and near space-optimal LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 3 Perfect Hash Function Static key set S of size n 0 1 ... n -1 Hash Table Perfect Hash Function 0 1 ... m -1 S ⊆ U, where |U| = u LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 4 Minimal Perfect Hash Function Static key set S of size n 0 1 ... n -1 Minimal Perfect Hash Function Hash Table 0 1 ... n -1 S ⊆ U, where |U| = u LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 5 Where to use a PHF or a MPHF? Access items based on the value of a key is ubiquitous in Computer Science Work with huge static key sets: In data warehousing applications: On -Line Analytical Processing (OLAP) applications In Web search engines: Large vocabularies Map long URLs in smaller integer numbers that are used as IDs LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 6 Crawling Pages Web Crawling Pages Parser URLs To Be Crawled Visited New URLs URLs LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 7 Crawling Pages Web Crawling Pages Parser URLs To Be Crawled Visited New URLs URLs LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 8 Representing Visited URLS MPHFs is the most compact way of representing the set of visited URLs Enable to keep much more URLs in main memory of each machine When the set of new URLs becomes large, a new MPHF is generated for the whole set of URLs LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 9 Indexing Collection of Vocabulary Inverted List documents Term 1 Doc 1 Doc 5 ... Doc 1 Term 2 Doc 1 Doc 2 ... Doc 2 Term 3 Doc 3 Doc 4 ... Doc 3 Term 4 Doc 7 Doc 9 ... Indexing Doc 4 Term 5 Doc 6 Doc 10 ... Doc 5 Term 6 Doc 1 Doc 5 ... ... Term 7 Doc n Term 8 ... Term t Doc 9 Doc 11 ... LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 10 Representing the Vocabulary Vocabulary Inverted List Term 1 Doc 1 Doc 5 ... Term 2 Doc 1 Doc 2 ... Term 3 Doc 3 Doc 4 ... Term 4 Doc 7 Doc 9 ... Term 5 Doc 6 Doc 10 ... Term 6 Doc 1 Doc 5 ... Term 7 Term 8 ... Term t Doc 9 Doc 11 ... LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 11 Mapping URLs to Web Graph Vertices 0 Web Graph 1 Vertices URL 1 URLS URL 2 2 URL 3 URL 4 3 URL 5 4 URL 6 URL 7 5 ... URL n 6 .. n-1 LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 12 Mapping URLs to Web Graph Vertices 0 Web Graph 1 Vertices URL 1 URLS URL 2 2 URL 3 M 3 URL 4 P URL 5 H 4 URL 6 F URL 7 5 ... URL n 6 .. n-1 LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 13 Information Theoretical Lower Bounds for Storage Space n2 PHFs (m ≈ n): Storage Space ≥ log e m MPHFs (m = n ): Storage Space ≥ nlog e m < 3n log e ≈ .1 4427 LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 14 Uniform Hashing Versus Universal Hashing Key universe Hash function Range M of size m U of size u LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 15 Uniform Hashing Versus Universal Hashing Key universe Hash function Range M of size m U of size u Uniform hashing # of functions from U to M? m u # of bits to encode each fucntion u log m Independent functions with values uniformly distributed LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 16 Uniform Hashing Versus Universal Hashing Key universe Hash function Range M of size m U of size u Uniform hashing Universal hashing # of functions from U to M? A family of hash functions HHH m u is universal if: for any pair of distinct # of bits to encode each fucntion keys (x 1, x 2) from U and u log m a hash function h chosen Independent functions with uniformly from HHH then: values uniformly distributed 1 Pr( h(x ) = h(x )) ≤ 1 2 m LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 17 Intuition Behind Universal Hashing We often lose relatively little compared to using a completely random map (uniform hashing) If S of size n is hashed to n2 buckets, with probability more than ½, no collisions occur Even with complete randomness, we do not expect little o( n2) buckets to suffice (the birthday paradox) So nothing is lost by using a universal family instead! LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 18 Related Work Theoretical Results (use uniform hashing) Practical Results (assume uniform hashing for free) Heuristics LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 19 Theoretical Results Work Gen. Time Eval. Time Size (bits) Mehlhorn (1984) Expon. Expon. O(n) Schmidt and Not analyzed O(1) O(n) Siegel (1990) Hagerup and O(n+log log u) O(1) O(n) Tholey (2001) LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 20 Theoretical Results – Use Uniform Hashing Work Gen. Time Eval. Time Size (bits) Mehlhorn (1984) Expon. Expon. O(n) Schmidt & Not analyzed O(1) O(n) Siegel (1990) Hagerup & O(n+log log u) O(1) O(n) Tholey (2001) Theoretic EM O(n) O(1) O(n) (CIKM 2007) F.C. Botelho, R. Pagh N. Ziviani. “Practical Perfect Hashing Algorithm in Nearly Optimal Space”, Information Systems 38(1), 2013, 108-131. LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 21 Practical Results – Assume Uniform Hashing For Free Work Gen. Time Eval. Time Size (bits) Czech, Havas & Majewski (1992) O(n) O(1) O(n log n) Majewski, Wormald, O(n) O(1) O(n log n) Havas & Czech (1996) Pagh (1999) O(n) O(1) O(n log n) Botelho, Kohayakawa, O(n) O(1) O(n log n) & Ziviani (2005) LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 22 Practical Results – Assume Uniform Hashing For Free Work Gen. Time Eval. Time Size (bits) Czech, Havas & Majewski (1992) O(n) O(1) O(n log n) Majewski, Wormald, O(n) O(1) O(n log n) Havas & Czech (1996) Pagh (1999) O(n) O(1) O(n log n) Botelho, Kohayakawa, O(n) O(1) O(n log n) & Ziviani (2005) RAM O(n) O(1) O(n) (WADS 2007) LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 23 Practical Results – Assume Uniform Hashing For Free Work Gen. Time Eval. Time Size (bits) Czech, Havas & Majewski (1992) O(n) O(1) O(n log n) Majewski, Wormald, O(n) O(1) O(n log n) Havas & Czech (1996) Pagh (1999) O(n) O(1) O(n log n) Botelho, Kohayakawa, O(n) O(1) O(n log n) & Ziviani (2005) RAM O(n) O(1) O(n) (WADS 2007) Heuristic EM O(n) O(1) O(n) (CIKM 2007) LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 24 Empirical Results Gen. Eval. Size Work Application Time Time (bits) Fox, Chen & Heath Index data in Exp. O(1) O(n) (1992) CD-ROM Lefebvre & Hoppe Sparse O(n) O(1) O(n) (2006) spatial data Chang, Lin & Chou Not Data mining O(n) O(1) (2005, 2006) analyzed LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 25 Internal Random Access and External Memory Algorithms Near-optimal space Evaluation in constant time Function generation in linear time Simple to describe and implement Known algorithms with near -optimal space either: Require exponential time for construction and evaluation, or Use near-optimal space only asymptotically, for large n Acyclic random hypergraphs Used before by Majewski et all (1996): O(n log n) bits We proceed differently: O(n) bits (we changed space complexity, close to theoretical lower bound) LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 26 Random Hypergraphs (r-graphs) 3-graph: 0 1 2 3 4 5 3-graph is induced by three uniform hash functions LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 27 Random Hypergraphs (r-graphs) 3-graph: h0(jan) = 1 h 1(jan) = 3 h 2(jan) = 5 0 1 2 3 4 5 3-graph is induced by three uniform hash functions LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 28 Random Hypergraphs (r-graphs) 3-graph: h0(jan) = 1 h 1(jan) = 3 h 2(jan) = 5 0 1 h0(feb) = 1 h 1(feb) = 2 h 2(feb) = 5 2 3 4 5 3-graph is induced by three uniform hash functions LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 29 Random Hypergraphs (r-graphs) 3-graph: h0(jan) = 1 h 1(jan) = 3 h 2(jan) = 5 0 1 h0(feb) = 1 h 1(feb) = 2 h 2(feb) = 5 2 3 h0(mar) = 0 h 1(mar) = 3 h 2(mar) = 4 4 5 3-graph is induced by three uniform hash functions Our best result uses 3-graphs LATIN - LAboratory for Treating INformation (www.dcc.ufmg.br/latin) 30 The MPHF Proposed by Czech, Havas e Majewski ..
Load more

Recommended publications
  • Algorithmic Improvements for Fast Concurrent Cuckoo Hashing Presentation by Nir David
    Algorithmic Improvements for Fast Concurrent Cuckoo Hashing Presentation by Nir David Xiaozhou Li , David G. Andersen , Michael Kaminsky , Michael J. Freedman Overview Background and Related Work Hash Tables Concurrency Control Mechanisms Naive use of concurrency control fails Principles to Improve Concurrency Concurrent Cuckoo Hashing Cuckoo Hashing Prior Work in Concurrent Cuckoo Algorithmic Optimizations Fine-grained Locking Optimizing for Intel TSX Evaluation Concurrent hash table Provides: Lookup, Insert and Delete operations. On Lookup, a value is returned for the given key, or “does not exist” if the key cannot be found. On Insert, the hash table returns success, or an error code to indicate whether the hash table is full or the key is already exists. Delete simply removes the key’s entry from the hash table. Several definitions before we go further Open Addressing: a method for handling collisions. A collision is resolved by probing, or searching through alternate locations in the array until either the target record is found, or an unused array slot is found. Linear probing - in which the interval between probes is fixed — often at 1 Quadratic probing - in which the interval between probes increases linearly Linear Probing: f(i) = i Insert(k,x) // assume unique keys 1. index = hash(key) % table_size; 2. if (table[index]== NULL) table[index]= new key_value_pair(key, x); 3. Else { • index++; • index = index % table_size; • goto 2; } 89 mod 10 = 9 Linear Probing Example 18 mod 10 = 8 Insert 89, 18, 49, 58, 69 58 mod 10 = 8 49 mod 10 = 9 Several definitions before we go further Chaining: another possible way to resolve collisions.
    [Show full text]
  • Cuckoo Hashing
    Cuckoo Hashing Outline for Today ● Towards Perfect Hashing ● Reducing worst-case bounds ● Cuckoo Hashing ● Hashing with worst-case O(1) lookups. ● The Cuckoo Graph ● A framework for analyzing cuckoo hashing. ● Analysis of Cuckoo Hashing ● Just how fast is cuckoo hashing? Perfect Hashing Collision Resolution ● Last time, we mentioned three general strategies for resolving hash collisions: ● Closed addressing: Store all colliding elements in an auxiliary data structure like a linked list or BST. ● Open addressing: Allow elements to overflow out of their target bucket and into other spaces. ● Perfect hashing: Choose a hash function with no collisions. ● We have not spoken on this last topic yet. Why Perfect Hashing is Hard ● The expected cost of a lookup in a chained hash table is O(1 + α) for any load factor α. ● For any fixed load factor α, the expected cost of a lookup in linear probing is O(1), where the constant depends on α. ● However, the expected cost of a lookup in these tables is not the same as the expected worst-case cost of a lookup in these tables. Expected Worst-Case Bounds ● Theorem: Assuming truly random hash functions, the expected worst-case cost of a lookup in a linear probing hash table is Ω(log n). ● Theorem: Assuming truly random hash functions, the expected worst-case cost of a lookup in a chained hash table is Θ(log n / log log n). ● Proofs: Exercise 11-1 and 11-2 from CLRS. ☺ Perfect Hashing ● A perfect hash table is one where lookups take worst-case time O(1). ● There's a pretty sizable gap between the expected worst-case bounds from chaining and linear probing – and that's on expected worst-case, not worst-case.
    [Show full text]
  • Cuckoo Hashing
    Cuckoo Hashing Rasmus Pagh* BRICSy, Department of Computer Science, Aarhus University Ny Munkegade Bldg. 540, DK{8000 A˚rhus C, Denmark. E-mail: pagh@daimi.au.dk and Flemming Friche Rodlerz ON-AIR A/S, Digtervejen 9, 9200 Aalborg SV, Denmark. E-mail: ffr@onair-dk.com We present a simple dictionary with worst case constant lookup time, equal- ing the theoretical performance of the classic dynamic perfect hashing scheme of Dietzfelbinger et al. (Dynamic perfect hashing: Upper and lower bounds. SIAM J. Comput., 23(4):738{761, 1994). The space usage is similar to that of binary search trees, i.e., three words per key on average. Besides being conceptually much simpler than previous dynamic dictionaries with worst case constant lookup time, our data structure is interesting in that it does not use perfect hashing, but rather a variant of open addressing where keys can be moved back in their probe sequences. An implementation inspired by our algorithm, but using weaker hash func- tions, is found to be quite practical. It is competitive with the best known dictionaries having an average case (but no nontrivial worst case) guarantee. Key Words: data structures, dictionaries, information retrieval, searching, hash- ing, experiments * Partially supported by the Future and Emerging Technologies programme of the EU under contract number IST-1999-14186 (ALCOM-FT). This work was initiated while visiting Stanford University. y Basic Research in Computer Science (www.brics.dk), funded by the Danish National Research Foundation. z This work was done while at Aarhus University. 1 2 PAGH AND RODLER 1. INTRODUCTION The dictionary data structure is ubiquitous in computer science.
    [Show full text]
  • Incremental Edge Orientation in Forests Michael A
    Incremental Edge Orientation in Forests Michael A. Bender Stony Brook University bender@cs.stonybrook.edu Tsvi Kopelowitz Bar-Ilan University kopelot@gmail.com William Kuszmaul Massachusetts Institute of Technology kuszmaul@mit.edu Ely Porat Bar-Ilan University porately@cs.biu.ac.il Clifford Stein Columbia University cliff@ieor.columbia.edu Abstract For any forest G = (V, E) it is possible to orient the edges E so that no vertex in V has out-degree greater than 1. This paper considers the incremental edge-orientation problem, in which the edges E arrive over time and the algorithm must maintain a low-out-degree edge orientation at all times. We give an algorithm that maintains a maximum out-degree of 3 while flipping at most O(log log n) edge orientations per edge insertion, with high probability in n. The algorithm requires worst-case time O(log n log log n) per insertion, and takes amortized time O(1). The previous state of the art required up to O(log n/ log log n) edge flips per insertion. We then apply our edge-orientation results to the problem of dynamic Cuckoo hashing. The problem of designing simple families H of hash functions that are compatible with Cuckoo hashing has received extensive attention. These families H are known to satisfy static guarantees, but do not come typically with dynamic guarantees for the running time of inserts and deletes. We show how to transform static guarantees (for 1-associativity) into near-state-of-the-art dynamic guarantees (for O(1)-associativity) in a black-box fashion.
    [Show full text]
  • Vivid Cuckoo Hash: Fast Cuckoo Table Building in SIMD
    ViViD Cuckoo Hash: Fast Cuckoo Table Building in SIMD Flaviene Scheidt de Cristo, Eduardo Cunha de Almeida, Marco Antonio Zanata Alves 1Informatics Department – Federal Univeristy of Parana´ (UFPR) – Curitiba – PR – Brazil ffscristo,eduardo,mazalvesg@inf.ufpr.br Abstract. Hash Tables play a lead role in modern databases systems during the execution of joins, grouping, indexing, removal of duplicates, and accelerating point queries. In this paper, we focus on Cuckoo Hash, a technique to deal with collisions guaranteeing that data is retrieved with at most two memory ac- cess in the worst case. However, building the Cuckoo Table with the current scalar methods is inefficient when treating the eviction of the colliding keys. We propose a Vertically Vectorized data-dependent method to build Cuckoo Ta- bles - ViViD Cuckoo Hash. Our method exploits data parallelism with AVX-512 SIMD instructions and transforms control dependencies into data dependencies to make the build process faster with an overall reduction in response time by 90% compared to the scalar Cuckoo Hash. 1. Introduction The usage of hash tables in the execution of joins, grouping, indexing, and removal of duplicates is a widespread technique on modern database systems. In the particular case of joins, hash tables do not require nested loops and sorting, dismissing the need to execute multiple full scans over the same relation (table). However, a hash table is as good as its strategy to avoid or deal with collisions. Cuckoo Hash [Pagh and Rodler 2004] stands among the most efficient ways of dealing with collisions. Using open addressing, it does not use additional structures and pointers - as Chained Hash [Cormen et al.
    [Show full text]
  • Cuckoo Hashing and Cuckoo Filters
    Cuckoo Hashing and Cuckoo Filters Noah Fleming May 17, 2018 1 Preliminaries A dictionary is an abstract data type that stores a collection of elements, located by their key. It supports operations: insert, delete and lookup. Implementations of dictionaries vary by the ways their keys are stored and retrieved. A hash table is a particular implementation of a dictionary that allows for expected constant-time operations. Indeed, in many cases, hash tables turn out to be on average more efficient than other dictionary implementations, such as search trees, both in terms of space and runtime. A hash table is comprised of a table T whose entries are called buckets. To insert a (key, value) pair, whose key is x, into the table, a hash function is used to select which bucket to store x. That is, a hash function is a map h : U! [T ] on a universe U mapping keys to locations in T . We will be particularly interested in the following strong family of universal hash functions. k-Universal Hash Functions: A family H of hash functions h : U! [T ] is k-universal if for any k distinct elements x1; : : : ; xk 2 U, and any outcome y1; : : : ; yk 2 [T ] k Pr [h(x1) = y1; : : : ; h(xk) = yk] ≤ 1=jT j ; h2H where the probability is taken as uniform over h 2 H. Typically, the universe U to which our keys belong is much larger than the size of our table T , and so collisions in the hash table are often unavoidable. Therefore, the hash func- tion attempts to minimize the number of collisions because collisions lead to longer insertion time, or even insertions failing.
    [Show full text]
  • Mitigating Asymmetric Read and Write Costs in Cuckoo Hashing for Storage Systems
    Mitigating Asymmetric Read and Write Costs in Cuckoo Hashing for Storage Systems Yuanyuan Sun, Yu Hua, Zhangyu Chen, and Yuncheng Guo, Huazhong University of Science and Technology https://www.usenix.org/conference/atc19/presentation/sun This paper is included in the Proceedings of the 2019 USENIX Annual Technical Conference. July 10–12, 2019 • Renton, WA, USA ISBN 978-1-939133-03-8 Open access to the Proceedings of the 2019 USENIX Annual Technical Conference is sponsored by USENIX. Mitigating Asymmetric Read and Write Costs in Cuckoo Hashing for Storage Systems Yuanyuan Sun, Yu Hua*, Zhangyu Chen, Yuncheng Guo Wuhan National Laboratory for Optoelectronics, School of Computer Huazhong University of Science and Technology *Corresponding Author: Yu Hua (csyhua@hust.edu.cn) Abstract The explosion of data volume leads to nontrivial challenge on storage systems, especially on the support for query ser- In storage systems, cuckoo hash tables have been widely vices [8, 12, 49]. Moreover, write-heavy workloads further used to support fast query services. For a read, the cuckoo exacerbate the storage performance. Much attention has hashing delivers real-time access with O(1) lookup com- been paid to alleviate the pressure on storage systems, which plexity via open-addressing approach. For a write, most demands the support of low-latency and high-throughput concurrent cuckoo hash tables fail to efficiently address the queries, such as top-k query processing [30, 33], optimizing problem of endless loops during item insertion due to the big data queries via automated program reasoning [42], of- essential property of hash collisions. The asymmetric fea- fering practical private queries on public data [47], and opti- ture of cuckoo hashing exhibits fast-read-slow-write perfor- mizing search performance within memory hierarchy [9].
    [Show full text]
  • An Overview of Cuckoo Hashing Charles Chen
    An Overview of Cuckoo Hashing Charles Chen 1 Abstract Cuckoo Hashing is a technique for resolving collisions in hash tables that produces a dic- tionary with constant-time worst-case lookup and deletion operations as well as amortized constant-time insertion operations. First introduced by Pagh in 2001 [3] as an extension of a previous static dictionary data structure, Cuckoo Hashing was the first such hash table with practically small constant factors [4]. Here, we first outline existing hash table collision poli- cies and go on to analyze the Cuckoo Hashing scheme in detail. Next, we give an overview of (c; k)-universal hash families, and finally, we summarize some selected experimental results comparing the varied collision policies. 2 Introduction The dictionary is a fundamental data structure in computer science, allowing users to store key-value pairs and look up by key the corresponding value in the data structure. The hash table provides one of the most common and one of the fastest implementations of the dictionary data structure, allowing for amortized constant-time lookup, insertion and deletion operations.1 In its archetypal form, a hash function h computed on the space of keys is used to map each key to a location in a set of bucket locations fb0; ··· ; br−1g. The key, as well as, potentially, some auxiliary data (termed the value), is stored at the appropriate bucket location upon insertion. Upon deletion, the key with its value is removed from the appropriate bucket, and in lookup, such a key-value pair, if it exists, is retrieved by looking at the bucket location corresponding to the hashed key value.
    [Show full text]
  • High-Performance Hash Tables for Networking Applications
    Cuckoo++ Hash Tables: High-Performance Hash Tables for Networking Applications Nicolas Le Scouarnec Technicolor ABSTRACT identifying the connection using its 5-tuple (protocol, source Hash tables are an essential data-structure for numerous address, destination address, source port and destination networking applications (e.g., connection tracking, rewalls, port) and mapping an application-specic value to this 5- network address translators). Among these, cuckoo hash tuple. As each connection has its own state, the number of tables provide excellent performance by allowing lookups entries in the system grows with the number of ows. As to be processed with very few memory accesses (2 to 3 per each route sees many concurrent ows going over it, the lookup). Yet, for large tables, cuckoo hash tables remain mem- scale is much higher: when routing and forwarding require ory bound and each memory access impacts performance. In to consider tens of thousands of routes (L3 information), this paper, we propose algorithmic improvements to cuckoo connection tracking requires to track millions of ows. To hash tables allowing to eliminate some unnecessary mem- deal with these requirements, the association is often stored ory accesses; these changes are conducted without altering in a hash table to access it eciently. the properties of the original cuckoo hash table so that all High-performance hash tables often rely on bucketized existing theoretical analysis remain applicable. On a single cuckoo hash-table [5, 8, 9, 14, 16, 19, 29] for they feature core, our hash table achieves 37M lookups per second for excellent read performance by guaranteeing that the state positive lookups (i.e., when the key looked up is present in associated to some connection can be found in less than the table), and 60M lookups per second for negative lookups, three memory accesses.
    [Show full text]
  • Module 5: Hashing
    Module 5: Hashing CS 240 - Data Structures and Data Management Reza Dorrigiv, Daniel Roche School of Computer Science, University of Waterloo Winter 2010 Reza Dorrigiv, Daniel Roche (CS, UW) CS240 - Module 5 Winter 2010 1 / 27 Theorem: In the comparison model (on the keys), Ω(log n) comparisons are required to search a size-n dictionary. Proof: Similar to lower bound for sorting. Any algorithm defines a binary decision tree with comparisons at the nodes and actions at the leaves. There are at least n + 1 different actions (return an item, or \not found"). So there are Ω(n) leaves, and therefore the height is Ω(log n). Lower bound for search The fastest implementations of the dictionary ADT require Θ(log n) time to search a dictionary containing n items. Is this the best possible? Reza Dorrigiv, Daniel Roche (CS, UW) CS240 - Module 5 Winter 2010 2 / 27 Proof: Similar to lower bound for sorting. Any algorithm defines a binary decision tree with comparisons at the nodes and actions at the leaves. There are at least n + 1 different actions (return an item, or \not found"). So there are Ω(n) leaves, and therefore the height is Ω(log n). Lower bound for search The fastest implementations of the dictionary ADT require Θ(log n) time to search a dictionary containing n items. Is this the best possible? Theorem: In the comparison model (on the keys), Ω(log n) comparisons are required to search a size-n dictionary. Reza Dorrigiv, Daniel Roche (CS, UW) CS240 - Module 5 Winter 2010 2 / 27 Lower bound for search The fastest implementations of the dictionary ADT require Θ(log n) time to search a dictionary containing n items.
    [Show full text]
  • History-Independent Cuckoo Hashing
    History-Independent Cuckoo Hashing Moni Naor¤z Gil Segevyz Udi Wiederx Abstract Cuckoo hashing is an e±cient and practical dynamic dictionary. It provides expected amor- tized constant update time, worst case constant lookup time, and good memory utilization. Various experiments demonstrated that cuckoo hashing is highly suitable for modern computer architectures and distributed settings, and o®ers signi¯cant improvements compared to other schemes. In this work we construct a practical history-independent dynamic dictionary based on cuckoo hashing. In a history-independent data structure, the memory representation at any point in time yields no information on the speci¯c sequence of insertions and deletions that led to its current content, other than the content itself. Such a property is signi¯cant when preventing unintended leakage of information, and was also found useful in several algorithmic settings. Our construction enjoys most of the attractive properties of cuckoo hashing. In particular, no dynamic memory allocation is required, updates are performed in expected amortized constant time, and membership queries are performed in worst case constant time. Moreover, with high probability, the lookup procedure queries only two memory entries which are independent and can be queried in parallel. The approach underlying our construction is to enforce a canonical memory representation on cuckoo hashing. That is, up to the initial randomness, each set of elements has a unique memory representation. A preliminary version of this work appeared in Proceedings of the 35th International Colloquium on Automata, Languages and Programming (ICALP), pages 631-642, 2008. ¤Incumbent of the Judith Kleeman Professorial Chair, Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot 76100, Israel.
    [Show full text]
  • A Cuckoo Filter Modification Inspired by Bloom Filter
    AUT Journal of Electrical Engineering AUT J. Elec. Eng., 51(2) (2019) 187-200 DOI: 10.22060/eej.2019.16370.5287 A Cuckoo Filter Modification Inspired by Bloom Filter Ha. Sasaniyan Asl1,*, B. Mozaffari Tazehkand2, M.J. Museviniya2 1 MSc,Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran 2 PhD,Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran ABSTRACT: Probabilistic data structures are so popular in membership queries, network applications, Review History: and databases and so on. Bloom Filter and Cuckoo Filter are two popular space efficient models Received: 2019-05-20 that incorporate in set membership checking part of many important protocols. They are compact Revised: 2019-09-17 representations of data that use hash functions to randomize a set of items. Being able to store more Accepted: 2019-09-17 elements while keeping a reasonable false positive probability is a key factor of design. A new algorithm Available Online: 2019-12-01 is proposed to improve some of the performance properties of Cuckoo Filter such as false positive rate and insertion performance and solve some drawbacks of the Cuckoo algorithm such as endless loop. Keywords: Main characteristic of the Bloom Filter is used to improve Cuckoo Filter, so we have a smart Cuckoo Bloom Filter Filter which is modified by Bloom Filter so called as SCFMBF. SCFMBF uses the same table of buckets Cuckoo Filter as Cuckoo Filter but instead of storing constant Fingerprints, it stores Bloom Filters. Bloom Filters can be accumulated in the table’s buckets which leads to higher insertion feasibility.
    [Show full text]