DOCSLIB.ORG

1 Hash Table Example B+ Tree Index by Example Clustered

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
1 Hash Table Example B+ Tree Index by Example Clustered Student ID fName lName Hash table example 10 Tom Hanks Introduction to Database Systems 20 Amy Hanks InDex Student_ID on Student.ID Data File Student … CSE 414 10 Tom Hanks 10 20 20 Amy Hanks 50 50 … … 200 200 … 220 240 220 Lecture 26: More Indexes and 420 240 800 Operator Costs … … 420 … … 800 Index File Data file (preferably (on disk) in memory) CSE 414 - Spring 2018 1 Hash table inDexesCSE are414 -gooDSpring 2018for point queries 2 B+ Tree Index by Example Clustered vs Unclustered Recall binary trees from CSE 143! d = 2 Find the key 40 80 Index File B+ Tree 40 <= 80 (preferably B+ Tree in memory) Index entries 20 60 100 120 140 Index entries (Index File) (Data file) 20 < 40 <= 60 10 15 18 20 30 40 50 60 65 80 85 90 Data Records Data Records CLUSTERED UNCLUSTERED 30 < 40 <= 40 Every table can have only one clustered and many unclustered indexes Data file Why? 10 15 18 20 30 40 50 60 65 80 85 90 (on disk) CSE 414 - Spring 2018 3 CSE 414 - Spring 2018 4 B+ indexes are good for range queries Student(ID, fname, lname) SELECT * Student FROM Student x, Takes y Takes(studentID, courseID) WHERE x.ID=y.studentID AND y.courseID > 300 ID fName lName Example Which Indexes? 10 Tom Hanks 20 Amy Hanks … for y in Takes • How many indexes could we create? if courseID > 300 then Assume the database has indexes on these attributes: for x in Student • Takes_courseID = index on Takes.courseID if x.ID=y.studentID • Student_ID = index on Student.ID output * • Which indexes should we create? Index selection Index join ⋈studentID=ID for y’ in Takes_courseID where y’.courseID > 300 y = fetch the Takes record pointed to by y’ for x’ in Student_ID where x’.ID = y.studentID σ x = fetch the Student record pointed to by x’ courseID>300 In general this is a very hard problem Index selection output * Takes Student CSE 414 - Spring 2018 5 CSE 414 - Spring 2018 6 1 Student ID fName lName Which Indexes? 10 Tom Hanks Index Selection: Which Search Key 20 Amy Hanks • The index selection problem … • Make some attribute K a search key if the – Given a table, and a “workload” (big Java WHERE clause contains: application with lots of SQL queries), decide which – An exact match on K indexes to create (and which ones NOT to create!) – A range predicate on K • Who does index selection: – The database administrator DBA – Semi-automatically, using a database administration tool CSE 414 - Spring 2018 7 CSE 414 - Spring 2018 8 The Index Selection Problem 1 The Index Selection Problem 1 V(M, N, P); V(M, N, P); Your workload is this Your workload is this 100000 queries: 100 queries: 100000 queries: 100 queries: SELECT * SELECT * SELECT * SELECT * FROM V FROM V FROM V FROM V WHERE N=? WHERE P=? WHERE N=? WHERE P=? What indexes ? CSE 414 - Spring 2018 9 CSE 414 - Spring 2018 10 The Index Selection Problem 1 The Index Selection Problem 2 V(M, N, P); V(M, N, P); Your workload is this Your workload is this 100000 queries: 100 queries: 100000 queries: 100 queries: 100000 queries: SELECT * SELECT * SELECT * SELECT * INSERT INTO V FROM V FROM V FROM V FROM V VALUES (?, ?, ?) WHERE N=? WHERE P=? WHERE N>? and N<? WHERE P=? A: V(N) and V(P) (hash tables or B-trees) What indexes ? CSE 414 - Spring 2018 11 CSE 414 - Spring 2018 12 2 The Index Selection Problem 2 The Index Selection Problem 3 V(M, N, P); V(M, N, P); Your workload is this Your workload is this 100000 queries: 100 queries: 100000 queries: 100000 queries: 1000000 queries: 100000 queries: SELECT * SELECT * INSERT INTO V SELECT * SELECT * INSERT INTO V FROM V FROM V VALUES (?, ?, ?) FROM V FROM V VALUES (?, ?, ?) WHERE N>? and N<? WHERE P=? WHERE N=? WHERE N=? and P>? A: definitely V(N) (must B-tree); unsure about V(P) What indexes ? CSE 414 - Spring 2018 13 CSE 414 - Spring 2018 14 The Index Selection Problem 3 The Index Selection Problem 4 V(M, N, P); V(M, N, P); Your workload is this Your workload is this 100000 queries: 1000000 queries: 100000 queries: 1000 queries: 100000 queries: SELECT * SELECT * INSERT INTO V SELECT * SELECT * FROM V FROM V VALUES (?, ?, ?) FROM V FROM V WHERE N=? WHERE N=? and P>? WHERE N>? and N<? WHERE P>? and P<? A: V(N, P) How does this index differ from: What indexes ? 1. Two indexes V(N) and V(P)? CSE 414 -2.SpringAn 2018 index V(P, N)? 15 CSE 414 - Spring 2018 16 The Index Selection Problem 4 Two typical kinds of queries V(M, N, P); • Point queries SELECT * • What data structure FROM Movie Your workload is this WHERE year = ? should be used for 1000 queries: 100000 queries: index? SELECT * SELECT * FROM V FROM V SELECT * • Range queries WHERE N>? and N<? WHERE P>? and P<? FROM Movie • What data structure WHERE year >= ? AND should be used for A: V(N) secondary, V(P) primary index year <= ? index? CSE 414 - Spring 2018 17 CSE 414 - Spring 2018 18 3 SELECT * Basic Index Selection Guidelines FROM R WHERE R.K>? and R.K<? • Consider queries in workload in order of importance Cost • Consider relations accessed by query – No point indexing other relations • Look at WHERE clause for possible search key • Try to choose indexes that speed-up multiple queries • Range queries benefit mostly from clustering 0 100 Percentage tuples retrieved CSE 414 - Spring 2018 19 CSE 414 - Spring 2018 20 SELECT * SELECT * FROM R FROM R WHERE R.K>? and R.K<? WHERE R.K>? and R.K<? Cost Sequential scan Cost Sequential scan Clustered index 0 100 0 100 Percentage tuples retrieved Percentage tuples retrieved CSE 414 - Spring 2018 21 CSE 414 - Spring 2018 22 SELECT * FROM R Choosing Index is Not Enough WHERE R.K>? and R.K<? • To estimate the cost of a query plan, we still Cost Unclustered index Sequential scan need to consider other factors: – How each operator is implemented Clustered index – The cost of each operator – Let’s start with the basics 0 100 Percentage tuples retrieved CSE 414 - Spring 2018 23 CSE 414 - Spring 2018 24 4 Cost Parameters • Cost = I/O + CPU + Network BW – We will focus on I/O in this class • Parameters (a.k.a. statistics): Cost of Reading – B(R) = # of blocks (i.e., paGes) for relation R – T(R) = # of tuples in relation R Data From Disk – V(R, a) = # of distinct values of attribute a When a is a key, V(R,a) = T(R) When a is not a key, V(R,a) can be anythinG <= T(R) • DBMS collects statistics about base tables must infer them for intermediate results CSE 414 - Spring 2018 25 26 Selectivity Factors for Conditions Cost of Reading Data From Disk • A = c /* σA=c(R) */ • Sequential scan for relation R costs B(R) – Selectivity = 1/V(R,A) • A < c /* σA<c(R)*/ – Selectivity = (c - min(R, A))/(max(R,A) - min(R,A)) • c1 < A < c2 /* σc1<A<c2(R)*/ – Selectivity = (c2 – c1)/(max(R,A) - min(R,A)) CSE 414 - Spring 2018 27 CSE 414 - Spring 2018 28 Index Based Selection Index Based Selection B(R) = 2000 B(R) = 2000 cost of cost of • Example: T(R) = 100,000 sa=v(R) = ? • Example: T(R) = 100,000 sa=v(R) = ? V(R, a) = 20 V(R, a) = 20 • Table scan: • Table scan: B(R) = 2,000 I/Os • Index based selection: • Index based selection: CSE 414 - Spring 2018 29 CSE 414 - Spring 2018 30 5 Index Based Selection Index Based Selection B(R) = 2000 B(R) = 2000 cost of cost of • Example: T(R) = 100,000 sa=v(R) = ? • Example: T(R) = 100,000 sa=v(R) = ? V(R, a) = 20 V(R, a) = 20 • Table scan: B(R) = 2,000 I/Os • Table scan: B(R) = 2,000 I/Os • Index based selection: • Index based selection: – If index is clustered: – If index is clustered: B(R) * 1/V(R,a) = 100 I/Os – If index is unclustered: – If index is unclustered: CSE 414 - Spring 2018 31 CSE 414 - Spring 2018 32 Index Based Selection Index Based Selection B(R) = 2000 B(R) = 2000 cost of cost of • Example: T(R) = 100,000 sa=v(R) = ? • Example: T(R) = 100,000 sa=v(R) = ? V(R, a) = 20 V(R, a) = 20 • Table scan: B(R) = 2,000 I/Os • Table scan: B(R) = 2,000 I/Os • Index based selection: • Index based selection: – If index is clustered: B(R) * 1/V(R,a) = 100 I/Os – If index is clustered: B(R) * 1/V(R,a) = 100 I/Os – If index is unclustered: T(R) * 1/V(R,a) = 5,000 I/Os – If index is unclustered: T(R) * 1/V(R,a) = 5,000 I/Os Note: we ignore I/O cost for index pages Lesson: Don’t build unclustered indexes when V(R,a) is small ! CSE 414 - Spring 2018 33 CSE 414 - Spring 2018 34 Outline • Join operator algorithms – One-pass algorithms (Sec. 15.2 and 15.3) Cost of Executing Operators – Index-based algorithms (Sec 15.6) (Focus on Single Node Joins) • Note about readings: – In class, we discuss only algorithms for joins – Other operators are easier: read the book CSE 414 - Spring 2018 35 CSE 414 - Spring 2018 36 6 Join Algorithms Nested Loop Joins • Tuple-based nested loop R ⋈ S • Nested loop join • R is the outer relation, S is the inner relation for each tuple t in R do • Hash join 1 for each tuple t2 in S do if t1 and t2 join then output (t1,t2) • Sort-merge join What is the Cost? CSE 414 - Spring 2018 37 CSE 414 - Spring 2018 38 Nested Loop Joins Page-at-a-time Refinement • Tuple-based nested loop R ⋈ S • R is the outer relation, S is the inner relation for each page of tuples r in R do for each page of tuples s in S do for each tuple t1 in R do for all pairs of tuples t1 in r, t2 in s for each tuple t2 in S do if t1 and t2 join then output (t1,t2) if t1 and t2 join then output (t1,t2) What is the Cost? What is the Cost? • Cost: B(R) + T(R) B(S) • Cost: B(R) + B(R)B(S) • Multiple-pass since S is read many times CSE 414 - Spring 2018 39 CSE 414 - Spring 2018 40 Page-at-a-time Refinement Page-at-a-time Refinement 1 2 Input buffer for Patient 11 2 Input buffer for Patient Input buffer for Insurance Input buffer for Insurance Disk 2 4 Disk 4 3 Patient Insurance 2 2 Patient Insurance 1 2 2 4 6 6 Output buffer 1 2 2 4 6 6 Output buffer 3 4 4 3 1 3 3 4 4 3 1 3 9 6 2 8 9 6 2 8 8 5 8 9 41 8 5 8 9 42 7 Page-at-a-time Refinement 11 2 Input buffer for Patient Input buffer for Insurance Disk 2 8 Patient Insurance Keep going until read all of Insurance 2 2 1 2 Output buffer 2 4 6 6 Then repeat for next 3 4 4 3 1 3 page of Patient… until end of Patient 9 6 2 8 Cost: B(R) + B(R)B(S) 8 5 8 9 43 8.
Load more

Recommended publications
  • 4 Hash Tables and Associative Arrays
    4 FREE Hash Tables and Associative Arrays If you want to get a book from the central library of the University of Karlsruhe, you have to order the book in advance. The library personnel fetch the book from the stacks and deliver it to a room with 100 shelves. You find your book on a shelf numbered with the last two digits of your library card. Why the last digits and not the leading digits? Probably because this distributes the books more evenly among the shelves. The library cards are numbered consecutively as students sign up, and the University of Karlsruhe was founded in 1825. Therefore, the students enrolled at the same time are likely to have the same leading digits in their card number, and only a few shelves would be in use if the leadingCOPY digits were used. The subject of this chapter is the robust and efficient implementation of the above “delivery shelf data structure”. In computer science, this data structure is known as a hash1 table. Hash tables are one implementation of associative arrays, or dictio- naries. The other implementation is the tree data structures which we shall study in Chap. 7. An associative array is an array with a potentially infinite or at least very large index set, out of which only a small number of indices are actually in use. For example, the potential indices may be all strings, and the indices in use may be all identifiers used in a particular C++ program.Or the potential indices may be all ways of placing chess pieces on a chess board, and the indices in use may be the place- ments required in the analysis of a particular game.
    [Show full text]
  • Prefix Hash Tree an Indexing Data Structure Over Distributed Hash
    Prefix Hash Tree An Indexing Data Structure over Distributed Hash Tables Sriram Ramabhadran ∗ Sylvia Ratnasamy University of California, San Diego Intel Research, Berkeley Joseph M. Hellerstein Scott Shenker University of California, Berkeley International Comp. Science Institute, Berkeley and and Intel Research, Berkeley University of California, Berkeley ABSTRACT this lookup interface has allowed a wide variety of Distributed Hash Tables are scalable, robust, and system to be built on top DHTs, including file sys- self-organizing peer-to-peer systems that support tems [9, 27], indirection services [30], event notifi- exact match lookups. This paper describes the de- cation [6], content distribution networks [10] and sign and implementation of a Prefix Hash Tree - many others. a distributed data structure that enables more so- phisticated queries over a DHT. The Prefix Hash DHTs were designed in the Internet style: scala- Tree uses the lookup interface of a DHT to con- bility and ease of deployment triumph over strict struct a trie-based structure that is both efficient semantics. In particular, DHTs are self-organizing, (updates are doubly logarithmic in the size of the requiring no centralized authority or manual con- domain being indexed), and resilient (the failure figuration. They are robust against node failures of any given node in the Prefix Hash Tree does and easily accommodate new nodes. Most impor- not affect the availability of data stored at other tantly, they are scalable in the sense that both la- nodes). tency (in terms of the number of hops per lookup) and the local state required typically grow loga- Categories and Subject Descriptors rithmically in the number of nodes; this is crucial since many of the envisioned scenarios for DHTs C.2.4 [Comp.
    [Show full text]
  • Hash Tables & Searching Algorithms
    Search Algorithms and Tables Chapter 11 Tables • A table, or dictionary, is an abstract data type whose data items are stored and retrieved according to a key value. • The items are called records. • Each record can have a number of data fields. • The data is ordered based on one of the fields, named the key field. • The record we are searching for has a key value that is called the target. • The table may be implemented using a variety of data structures: array, tree, heap, etc. Sequential Search public static int search(int[] a, int target) { int i = 0; boolean found = false; while ((i < a.length) && ! found) { if (a[i] == target) found = true; else i++; } if (found) return i; else return –1; } Sequential Search on Tables public static int search(someClass[] a, int target) { int i = 0; boolean found = false; while ((i < a.length) && !found){ if (a[i].getKey() == target) found = true; else i++; } if (found) return i; else return –1; } Sequential Search on N elements • Best Case Number of comparisons: 1 = O(1) • Average Case Number of comparisons: (1 + 2 + ... + N)/N = (N+1)/2 = O(N) • Worst Case Number of comparisons: N = O(N) Binary Search • Can be applied to any random-access data structure where the data elements are sorted. • Additional parameters: first – index of the first element to examine size – number of elements to search starting from the first element above Binary Search • Precondition: If size > 0, then the data structure must have size elements starting with the element denoted as the first element. In addition, these elements are sorted.
    [Show full text]
  • Introduction to Hash Table and Hash Function
    Introduction to Hash Table and Hash Function This is a short introduction to Hashing mechanism Introduction Is it possible to design a search of O(1)– that is, one that has a constant search time, no matter where the element is located in the list? Let’s look at an example. We have a list of employees of a fairly small company. Each of 100 employees has an ID number in the range 0 – 99. If we store the elements (employee records) in the array, then each employee’s ID number will be an index to the array element where this employee’s record will be stored. In this case once we know the ID number of the employee, we can directly access his record through the array index. There is a one-to-one correspondence between the element’s key and the array index. However, in practice, this perfect relationship is not easy to establish or maintain. For example: the same company might use employee’s five-digit ID number as the primary key. In this case, key values run from 00000 to 99999. If we want to use the same technique as above, we need to set up an array of size 100,000, of which only 100 elements will be used: Obviously it is very impractical to waste that much storage. But what if we keep the array size down to the size that we will actually be using (100 elements) and use just the last two digits of key to identify each employee? For example, the employee with the key number 54876 will be stored in the element of the array with index 76.
    [Show full text]
  • Balanced Search Trees and Hashing Balanced Search Trees
    Advanced Implementations of Tables: Balanced Search Trees and Hashing Balanced Search Trees Binary search tree operations such as insert, delete, retrieve, etc. depend on the length of the path to the desired node The path length can vary from log2(n+1) to O(n) depending on how balanced or unbalanced the tree is The shape of the tree is determined by the values of the items and the order in which they were inserted CMPS 12B, UC Santa Cruz Binary searching & introduction to trees 2 Examples 10 40 20 20 60 30 40 10 30 50 70 50 Can you get the same tree with 60 different insertion orders? 70 CMPS 12B, UC Santa Cruz Binary searching & introduction to trees 3 2-3 Trees Each internal node has two or three children All leaves are at the same level 2 children = 2-node, 3 children = 3-node CMPS 12B, UC Santa Cruz Binary searching & introduction to trees 4 2-3 Trees (continued) 2-3 trees are not binary trees (duh) A 2-3 tree of height h always has at least 2h-1 nodes i.e. greater than or equal to a binary tree of height h A 2-3 tree with n nodes has height less than or equal to log2(n+1) i.e less than or equal to the height of a full binary tree with n nodes CMPS 12B, UC Santa Cruz Binary searching & introduction to trees 5 Definition of 2-3 Trees T is a 2-3 tree of height h if T is empty (height 0), OR T is of the form r TL TR Where r is a node that contains one data item and TL and TR are 2-3 trees, each of height h-1, and the search key of r is greater than any in TL and less than any in TR, OR CMPS 12B, UC Santa Cruz Binary searching & introduction to trees 6 Definition of 2-3 Trees (continued) T is of the form r TL TM TR Where r is a node that contains two data items and TL, TM, and TR are 2-3 trees, each of height h-1, and the smaller search key of r is greater than any in TL and less than any in TM and the larger search key in r is greater than any in TM and smaller than any in TR.
    [Show full text]
  • Hash Tables Hash Tables a "Faster" Implementation for Map Adts
    Hash Tables Hash Tables A "faster" implementation for Map ADTs Outline What is hash function? translation of a string key into an integer Consider a few strategies for implementing a hash table linear probing quadratic probing separate chaining hashing Big Ohs using different data structures for a Map ADT? Data Structure put get remove Unsorted Array Sorted Array Unsorted Linked List Sorted Linked List Binary Search Tree A BST was used in OrderedMap<K,V> Hash Tables Hash table: another data structure Provides virtually direct access to objects based on a key (a unique String or Integer) key could be your SID, your telephone number, social security number, account number, … Must have unique keys Each key is associated with–mapped to–a value Hashing Must convert keys such as "555-1234" into an integer index from 0 to some reasonable size Elements can be found, inserted, and removed using the integer index as an array index Insert (called put), find (get), and remove must use the same "address calculator" which we call the Hash function Hashing Can make String or Integer keys into integer indexes by "hashing" Need to take hashCode % array size Turn “S12345678” into an int 0..students.length Ideally, every key has a unique hash value Then the hash value could be used as an array index, however, Ideal is impossible Some keys will "hash" to the same integer index Known as a collision Need a way to handle collisions! "abc" may hash to the same integer array index as "cba" Hash Tables: Runtime Efficient Lookup time does not grow when n increases A hash table supports fast insertion O(1) fast retrieval O(1) fast removal O(1) Could use String keys each ASCII character equals some unique integer "able" = 97 + 98 + 108 + 101 == 404 Hash method works something like… Convert a String key into an integer that will be in the range of 0 through the maximum capacity-1 Assume the array capacity is 9997 hash(key) AAAAAAAA 8482 zzzzzzzz 1273 hash(key) A string of 8 chars Range: 0 ..
    [Show full text]
  • SCS1201 Advanced Data Structures Unit V Table Data Structures Table Is a Data Structure Which Plays a Significant Role in Information Retrieval
    SCS1201 Advanced Data Structures Unit V Table Data Structures Table is a data structure which plays a significant role in information retrieval. A set of n distinct records with keys K1, K2, …., Kn are stored in a file. If we want to find a record with a given key value, K, simply access the index given by its key k. The table lookup has a running time of O(1). The searching time required is directly proportional to the number of number of records in the file. This searching time can be reduced, even can be made independent of the number of records, if we use a table called Access Table. Some of possible kinds of tables are given below: • Rectangular table • Jagged table • Inverted table. • Hash tables Rectangular Tables Tables are very often in rectangular form with rows and columns. Most programming languages accommodate rectangular tables as 2-D arrays. Rectangular tables are also known as matrices. Almost all programming languages provide the implementation procedures for these tables as they are used in many applications. Fig. Rectangular tables in memory Logically, a matrix appears as two-dimensional, but physically it is stored in a linear fashion. In order to map from the logical view to physical structure, an indexing formula is used. The compiler must be able to convert the index (i, j) of a rectangular table to the correct position in the sequential array in memory. For an m x n array (m rows, n columns): 1 Each row is indexed from 0 to m-1 Each column is indexed from 0 to n – 1 Item at (i, j) is at sequential position i * n + j Row major order: Assume that the base address is the first location of the memory, so the th Address a ij =storing all the elements in the first(i-1) rows + the number of elements in the i th row up to the j th coloumn = (i-1)*n+j Column major order: Address of a ij = storing all the elements in the first(j-1)th column + The number of elements in the j th column up to the i th rows.
    [Show full text]
  • Snickerdoodle> Saveenv Saving Environment to SPI Flash... SF
    snickerdoodle> saveenv Saving Environment to SPI Flash... SF: Detected N25Q128A with page size 256 Bytes, erase size 64 KiB, total 16 MiB Erasing SPI flash...Writing to SPI flash...done snickerdoodle> boot Device: sdhci@e0100000 Manufacturer ID: 28 OEM: 4245 Name: SDU32 Tran Speed: 50000000 Rd Block Len: 512 SD version 3.0 High Capacity: Yes Capacity: 29.8 GiB Bus Width: 4-bit Erase Group Size: 512 Bytes reading uEnv.txt 1046 bytes read in 13 ms (78.1 KiB/s) Loaded environment from uEnv.txt Importing environment from SD ... Running uenvcmd ... reading config.txt 1 bytes read in 13 ms (0 Bytes/s) Importing configuration environment... Config set to Device: sdhci@e0100000 Manufacturer ID: 28 OEM: 4245 Name: SDU32 Tran Speed: 50000000 Rd Block Len: 512 SD version 3.0 High Capacity: Yes Capacity: 29.8 GiB Bus Width: 4-bit Erase Group Size: 512 Bytes Copying Linux system from microSD to RAM... reading uImage 4153304 bytes read in 246 ms (16.1 MiB/s) reading devicetree.dtb 10287 bytes read in 18 ms (557.6 KiB/s) ## Booting kernel from Legacy Image at 03000000 ... Image Name: Linux-4.4.0-snickerdoodle-34505- Image Type: ARM Linux Kernel Image (uncompressed) Data Size: 4153240 Bytes = 4 MiB Load Address: 00008000 Entry Point: 00008000 Verifying Checksum ... OK ## Flattened Device Tree blob at 02ff0000 Booting using the fdt blob at 0x2ff0000 Loading Kernel Image ... OK Loading Device Tree to 1fffa000, end 1ffff82e ... OK Starting kernel ... [ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 4.4.0-snickerdoodle-34505-gcd2401c (russellbush@ub6 [ 0.000000] CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instructie [ 0.000000] Machine model: snickerdoodle [ 0.000000] cma: Reserved 16 MiB at 0x3f000000 [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] PERCPU: Embedded 12 pages/cpu @ef7d4000 s19072 r8192 d21888 u4912 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on.
    [Show full text]
  • Data Structures and Algorithm Analysis (CSC317) Hash Tables
    Data Structures and Algorithm Analysis (CSC317) Hash tables Hash table • We have elements with key and satellite data • Operations performed: Insert, Delete, Search/lookup • We don’t maintain order information • We’ll see that all operations on average O(1) • But worse case can be O(n) 254 Chapter 11 Hash Tables 11.1 Direct-address tables Direct addressing is a simple technique that works well when the universe U of keys is reasonably small. Suppose that an application needs a dynamic set in which each element has a key drawn from the universe U 0; 1; : : : ; m 1 ,wherem D f ! g is not too large. We shall assume that no two elements have the same key. To represent the dynamic set, we use an array, or direct-address table,denoted by TŒ0::m 1,inwhicheachposition,orslot,correspondstoakeyintheuni- ! verse U .Figure11.1illustratestheapproach;slotk points to an element in the set with key k.Ifthesetcontainsnoelementwithkeyk,thenTŒk NIL. D The dictionary operations are trivial to implement: DIRECT-ADDRESS-SEARCH.T; k/ 1 return TŒk Hash table DIRECT-ADDRESS-INSERT.T; x/ 1 TŒx:key x • SimpleD implementation: If universe of keys DIRECT-ADDRESS-DELETE.T; x/ 1 comesTŒx:key fromNIL a small set of integers [0..9], we canD store directly in array using the Each of these operations takes only O.1/ time. keys as indices into the slots. T 0 key satellite data U 1 (universe of keys) 2 0 6 2 9 3 4 7 3 4 1 5 K 2 5 (actual 3 6 5 keys) 8 7 8 8 9 Figure 11.1 How to implement a dynamic set by a direct-address table T .Eachkeyintheuniverse U 0; 1; : : : ; 9 corresponds to an index in the table.
    [Show full text]
  • Asarray() Provides One- and Multi-Dimensional Associative Arrays, Also Known As Containers, Maps, Dictionaries, Indices, and Hash Tables
    Title stata.com asarray( ) — Associative arrays Description Syntax Remarks and examples Conformability Diagnostics Also see Description asarray() provides one- and multi-dimensional associative arrays, also known as containers, maps, dictionaries, indices, and hash tables. Also see[ M-5] AssociativeArray( ) for a class-based interface into the functions documented here. Syntax A = asarray create( keytype declare A , keydim , minsize , minratio , maxratio ) asarray(A, key, a) A[key] = a a = asarray(A, key) a = A[key] or a = notfound asarray remove(A, key) delete A[key] if it exists bool = asarray contains(A, key) A[key] exists? N = asarray elements(A) # of elements in A keys = asarray keys(A) all keys in A loc = asarray first(A) location of first element or NULL loc = asarray next(A, loc) location of next element or NULL key = asarray key(A, loc) key at loc a = asarray contents(A, loc) contents a at loc asarray notfound(A, notfound) set notfound value notfound = asarray notfound(A) query notfound value 1 2 asarray( ) — Associative arrays where A: Associative array A[key]. Created by asarray create() and passed to the other functions. If A is declared, it is declared transmorphic. keytype: Element type of keys; "string", "real", "complex", or "pointer". Optional; default "string". keydim: Dimension of key; 1 ≤ keydim ≤ 50. Optional; default 1. minsize: Initial size of hash table used to speed locating keys in A; real scalar; 5 ≤ minsize ≤ 1,431,655,764. Optional; default 100. minratio: Fraction filled at which hash table is automatically downsized; real scalar; 0 ≤ minratio ≤ 1. Optional; default 0.5. maxratio: Fraction filled at which hash table is automatically upsized; real scalar; 1 < maxratio ≤ .
    [Show full text]
  • EECS 281, Week 8: Wednesday
    EECS 281, Week 8: Wednesday Hash Tables 1. Describe hashing as a two-step process. We first compute the hash code by converting the key that we use in the hash table to an integer. Then we compress the hash code to range [0, m), where m is the number of slots in the array used by the hash table. 2. Consider the hash function for a string below: int hash(const string &s) { return toupper(s[0])) - 'A'; } Critique this hash function, identifying and explaining its downsides. Since the hash function bases its output only on the first letter of its input, it will tend to output some values more frequently than others if we use it on English words (some letters are more common at the start of English words than others). In addition, this hash function not distinguish between lowercase and uppercase. 3. What is a collision? A collision occurs when two keys map to the same index in the array representing the hash table. 4. When can collisions occur? At the step of creating the hash code, so that two different keys map to the same hash code. Then, no matter which compression function we use, the hash code will compress to the same slot in the array. The compression method can cause clusters if the keys that we insert have patters and the size of the hash table is not a prime number. 5. How are collisions different from clusters? Clustering is the tendency for a collision resolution scheme to create long runs of filled slots.
    [Show full text]
  • Hash Function: Int Hash (Key K) // Return Value 0…M-1
    HashTable CISC4080, Computer Algorithms CIS, Fordham Univ. ! Instructor: X. Zhang Spring 2018 Acknowledgement • The set of slides have used materials from the following resources • Slides for textbook by Dr. Y. Chen from Shanghai Jiaotong Univ. • Slides from Dr. M. Nicolescu from UNR • Slides sets by Dr. K. Wayne from Princeton • which in turn have borrowed materials from other resources • Other online resources 2 Support for Dictionary • Dictionary ADT: a dynamic set of elements supporting INSERT, DELETE, SEARCH operations • elements have distinct key fields • DELETE, SEARCH by key • Different ways to implement Dictionary • unsorted array • insert O(1), delete O(n), search O(n) • sorted array • insert O(n), delete O(n), search O(log n) • binary search tree • insert O(log n), delete O(log n), search O(log n) • linked list … • Can we have “almost” constant time insert/delete/ search? 3 Towards constant time • Direct address table: use key as index into the array • T[i] stores the element whose key is i 0 ! T Insert ( element(2,Alice)) ! 1 T[2]=element(2, Alice); ! 2, Alice Delete (element(4)) 2 ! T[4]=NULL; NULL NULL Search! (element(5)) 4, Bob U: the set of all possible key values ! return T[5]; 5, Ed ! …. K: actual set of keys ! in your data • How big is the table? • big enough to have one slot for every possible key 4 Case studies • A web server: maintains all active clients’ info, using IP addr. as key ! U: the set of all ! possible key values ! K: actual ! set of keys ! in your data • Universe of keys: the set of all possible IPv4 addr., |U|=232 • much bigger than total # of active clients • Too big to use direct access table: • a table with 232 entries, if each entry is 32bytes, then 128GB is needed! • How to have constant accessing time, while not requiring huge memory usage? 5 Hash Table • Hash Table: use a (hash) function to map key to index of the table (array) • Element x is stored in T[h(x.key)] • hash function: int hash (Key k) // return value 0…m-1 Collision: when two different keys are mapped to same index.
    [Show full text]