Hash Functions
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Horner's Method: a Fast Method of Evaluating a Polynomial String Hash
Olympiads in Informatics, 2013, Vol. 7, 90–100 90 2013 Vilnius University Where to Use and Ho not to Use Polynomial String Hashing Jaku' !(CHOCKI, $ak&' RADOSZEWSKI Faculty of Mathematics, Informatics and Mechanics, University of Warsaw Banacha 2, 02-097 Warsaw, oland e-mail: {pachoc$i,jrad}@mimuw.edu.pl Abstract. We discuss the usefulness of polynomial string hashing in programming competition tasks. We sho why se/eral common choices of parameters of a hash function can easily lead to a large number of collisions. We particularly concentrate on the case of hashing modulo the size of the integer type &sed for computation of fingerprints, that is, modulo a power of t o. We also gi/e examples of tasks in hich strin# hashing yields a solution much simpler than the solutions obtained using other known algorithms and data structures for string processing. Key words: programming contests, hashing on strings, task e/aluation. 1. Introduction Hash functions are &sed to map large data sets of elements of an arbitrary length 3the $eys4 to smaller data sets of elements of a 12ed length 3the fingerprints). The basic appli6 cation of hashing is efficient testin# of equality of %eys by comparin# their 1ngerprints. ( collision happens when two different %eys ha/e the same 1ngerprint. The ay in which collisions are handled is crucial in most applications of hashing. Hashing is particularly useful in construction of efficient practical algorithms. Here e focus on the case of the %eys 'ein# strings o/er an integer alphabetΣ= 0,1,...,A 1 . 5he elements ofΣ are called symbols. -
Why Simple Hash Functions Work: Exploiting the Entropy in a Data Stream∗
Why Simple Hash Functions Work: Exploiting the Entropy in a Data Stream¤ Michael Mitzenmachery Salil Vadhanz School of Engineering & Applied Sciences Harvard University Cambridge, MA 02138 fmichaelm,[email protected] http://eecs.harvard.edu/»fmichaelm,salilg October 12, 2007 Abstract Hashing is fundamental to many algorithms and data structures widely used in practice. For theoretical analysis of hashing, there have been two main approaches. First, one can assume that the hash function is truly random, mapping each data item independently and uniformly to the range. This idealized model is unrealistic because a truly random hash function requires an exponential number of bits to describe. Alternatively, one can provide rigorous bounds on performance when explicit families of hash functions are used, such as 2-universal or O(1)-wise independent families. For such families, performance guarantees are often noticeably weaker than for ideal hashing. In practice, however, it is commonly observed that simple hash functions, including 2- universal hash functions, perform as predicted by the idealized analysis for truly random hash functions. In this paper, we try to explain this phenomenon. We demonstrate that the strong performance of universal hash functions in practice can arise naturally from a combination of the randomness of the hash function and the data. Speci¯cally, following the large body of literature on random sources and randomness extraction, we model the data as coming from a \block source," whereby each new data item has some \entropy" given the previous ones. As long as the (Renyi) entropy per data item is su±ciently large, it turns out that the performance when choosing a hash function from a 2-universal family is essentially the same as for a truly random hash function. -
SELECTION of CYCLIC REDUNDANCY CODE and CHECKSUM March 2015 ALGORITHMS to ENSURE CRITICAL DATA INTEGRITY 6
DOT/FAA/TC-14/49 Selection of Federal Aviation Administration William J. Hughes Technical Center Cyclic Redundancy Code and Aviation Research Division Atlantic City International Airport New Jersey 08405 Checksum Algorithms to Ensure Critical Data Integrity March 2015 Final Report This document is available to the U.S. public through the National Technical Information Services (NTIS), Springfield, Virginia 22161. U.S. Department of Transportation Federal Aviation Administration NOTICE This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest of information exchange. The U.S. Government assumes no liability for the contents or use thereof. The U.S. Government does not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the objective of this report. The findings and conclusions in this report are those of the author(s) and do not necessarily represent the views of the funding agency. This document does not constitute FAA policy. Consult the FAA sponsoring organization listed on the Technical Documentation page as to its use. This report is available at the Federal Aviation Administration William J. Hughes Technical Center’s Full-Text Technical Reports page: actlibrary.tc.faa.gov in Adobe Acrobat portable document format (PDF). Technical Report Documentation Page 1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. DOT/FAA/TC-14/49 4. Title and Subitle 5. Report Date SELECTION OF CYCLIC REDUNDANCY CODE AND CHECKSUM March 2015 ALGORITHMS TO ENSURE CRITICAL DATA INTEGRITY 6. Performing Organization Code 220410 7. Author(s) 8. Performing Organization Report No. -
CSC 344 – Algorithms and Complexity Why Search?
CSC 344 – Algorithms and Complexity Lecture #5 – Searching Why Search? • Everyday life -We are always looking for something – in the yellow pages, universities, hairdressers • Computers can search for us • World wide web provides different searching mechanisms such as yahoo.com, bing.com, google.com • Spreadsheet – list of names – searching mechanism to find a name • Databases – use to search for a record • Searching thousands of records takes time the large number of comparisons slows the system Sequential Search • Best case? • Worst case? • Average case? Sequential Search int linearsearch(int x[], int n, int key) { int i; for (i = 0; i < n; i++) if (x[i] == key) return(i); return(-1); } Improved Sequential Search int linearsearch(int x[], int n, int key) { int i; //This assumes an ordered array for (i = 0; i < n && x[i] <= key; i++) if (x[i] == key) return(i); return(-1); } Binary Search (A Decrease and Conquer Algorithm) • Very efficient algorithm for searching in sorted array: – K vs A[0] . A[m] . A[n-1] • If K = A[m], stop (successful search); otherwise, continue searching by the same method in: – A[0..m-1] if K < A[m] – A[m+1..n-1] if K > A[m] Binary Search (A Decrease and Conquer Algorithm) l ← 0; r ← n-1 while l ≤ r do m ← (l+r)/2 if K = A[m] return m else if K < A[m] r ← m-1 else l ← m+1 return -1 Analysis of Binary Search • Time efficiency • Worst-case recurrence: – Cw (n) = 1 + Cw( n/2 ), Cw (1) = 1 solution: Cw(n) = log 2(n+1) 6 – This is VERY fast: e.g., Cw(10 ) = 20 • Optimal for searching a sorted array • Limitations: must be a sorted array (not linked list) binarySearch int binarySearch(int x[], int n, int key) { int low, high, mid; low = 0; high = n -1; while (low <= high) { mid = (low + high) / 2; if (x[mid] == key) return(mid); if (x[mid] > key) high = mid - 1; else low = mid + 1; } return(-1); } Searching Problem Problem: Given a (multi)set S of keys and a search key K, find an occurrence of K in S, if any. -
4 Hash Tables and Associative Arrays
4 FREE Hash Tables and Associative Arrays If you want to get a book from the central library of the University of Karlsruhe, you have to order the book in advance. The library personnel fetch the book from the stacks and deliver it to a room with 100 shelves. You find your book on a shelf numbered with the last two digits of your library card. Why the last digits and not the leading digits? Probably because this distributes the books more evenly among the shelves. The library cards are numbered consecutively as students sign up, and the University of Karlsruhe was founded in 1825. Therefore, the students enrolled at the same time are likely to have the same leading digits in their card number, and only a few shelves would be in use if the leadingCOPY digits were used. The subject of this chapter is the robust and efficient implementation of the above “delivery shelf data structure”. In computer science, this data structure is known as a hash1 table. Hash tables are one implementation of associative arrays, or dictio- naries. The other implementation is the tree data structures which we shall study in Chap. 7. An associative array is an array with a potentially infinite or at least very large index set, out of which only a small number of indices are actually in use. For example, the potential indices may be all strings, and the indices in use may be all identifiers used in a particular C++ program.Or the potential indices may be all ways of placing chess pieces on a chess board, and the indices in use may be the place- ments required in the analysis of a particular game. -
SHA-3 and the Hash Function Keccak
Christof Paar Jan Pelzl SHA-3 and The Hash Function Keccak An extension chapter for “Understanding Cryptography — A Textbook for Students and Practitioners” www.crypto-textbook.com Springer 2 Table of Contents 1 The Hash Function Keccak and the Upcoming SHA-3 Standard . 1 1.1 Brief History of the SHA Family of Hash Functions. .2 1.2 High-level Description of Keccak . .3 1.3 Input Padding and Generating of Output . .6 1.4 The Function Keccak- f (or the Keccak- f Permutation) . .7 1.4.1 Theta (q)Step.......................................9 1.4.2 Steps Rho (r) and Pi (p).............................. 10 1.4.3 Chi (c)Step ........................................ 10 1.4.4 Iota (i)Step......................................... 11 1.5 Implementation in Software and Hardware . 11 1.6 Discussion and Further Reading . 12 1.7 Lessons Learned . 14 Problems . 15 References ......................................................... 17 v Chapter 1 The Hash Function Keccak and the Upcoming SHA-3 Standard This document1 is a stand-alone description of the Keccak hash function which is the basis of the upcoming SHA-3 standard. The description is consistent with the approach used in our book Understanding Cryptography — A Textbook for Students and Practioners [11]. If you own the book, this document can be considered “Chap- ter 11b”. However, the book is most certainly not necessary for using the SHA-3 description in this document. You may want to check the companion web site of Understanding Cryptography for more information on Keccak: www.crypto-textbook.com. In this chapter you will learn: A brief history of the SHA-3 selection process A high-level description of SHA-3 The internal structure of SHA-3 A discussion of the software and hardware implementation of SHA-3 A problem set and recommended further readings 1 We would like to thank the Keccak designers as well as Pawel Swierczynski and Christian Zenger for their extremely helpful input to this document. -
Choosing Best Hashing Strategies and Hash Functions
2009 IEEE International Advance ComDutint! Conference (IACC 2009) Patiala, India, 4 Choosing Best Hashing Strategies and Hash Functions Mahima Singh, Deepak Garg Computer science and Engineering Department, Thapar University ,Patiala. Abstract The paper gives the guideline to choose a best To quickly locate a data record Hash functions suitable hashing method hash function for a are used with its given search key used in hash particularproblem. tables. After studying the various problem we find A hash table is a data structure that associates some criteria has been found to predict the keys with values. The basic operation is to best hash method and hash function for that supports efficiently (student name) and find the problem. We present six suitable various corresponding value (e.g. that students classes of hash functions in which most of the enrolment no.). It is done by transforming the problems canfind their solution. key using the hash function into a hash, a Paper discusses about hashing and its various number that is used as an index in an array to components which are involved in hashing and locate the desired location (called bucket) states the need ofusing hashing forfaster data where the values should be. retrieval. Hashing methods were used in many Hash table is a data structure which divides all different applications of computer science elements into equal-sized buckets to allow discipline. These applications are spreadfrom quick access to the elements. The hash spell checker, database management function determines which bucket an element applications, symbol tables generated by belongs in. loaders, assembler, and compilers. -
DBMS Hashing
DDBBMMSS -- HHAASSHHIINNGG http://www.tutorialspoint.com/dbms/dbms_hashing.htm Copyright © tutorialspoint.com For a huge database structure, it can be almost next to impossible to search all the index values through all its level and then reach the destination data block to retrieve the desired data. Hashing is an effective technique to calculate the direct location of a data record on the disk without using index structure. Hashing uses hash functions with search keys as parameters to generate the address of a data record. Hash Organization Bucket − A hash file stores data in bucket format. Bucket is considered a unit of storage. A bucket typically stores one complete disk block, which in turn can store one or more records. Hash Function − A hash function, h, is a mapping function that maps all the set of search- keys K to the address where actual records are placed. It is a function from search keys to bucket addresses. Static Hashing In static hashing, when a search-key value is provided, the hash function always computes the same address. For example, if mod-4 hash function is used, then it shall generate only 5 values. The output address shall always be same for that function. The number of buckets provided remains unchanged at all times. Operation Insertion − When a record is required to be entered using static hash, the hash function h computes the bucket address for search key K, where the record will be stored. Bucket address = hK Search − When a record needs to be retrieved, the same hash function can be used to retrieve the address of the bucket where the data is stored. -
Linear-XOR and Additive Checksums Don't Protect Damgård-Merkle
Linear-XOR and Additive Checksums Don’t Protect Damg˚ard-Merkle Hashes from Generic Attacks Praveen Gauravaram1! and John Kelsey2 1 Technical University of Denmark (DTU), Denmark Queensland University of Technology (QUT), Australia. [email protected] 2 National Institute of Standards and Technology (NIST), USA [email protected] Abstract. We consider the security of Damg˚ard-Merkle variants which compute linear-XOR or additive checksums over message blocks, inter- mediate hash values, or both, and process these checksums in computing the final hash value. We show that these Damg˚ard-Merkle variants gain almost no security against generic attacks such as the long-message sec- ond preimage attacks of [10,21] and the herding attack of [9]. 1 Introduction The Damg˚ard-Merkle construction [3, 14] (DM construction in the rest of this article) provides a blueprint for building a cryptographic hash function, given a fixed-length input compression function; this blueprint is followed for nearly all widely-used hash functions. However, the past few years have seen two kinds of surprising results on hash functions, that have led to a flurry of research: 1. Generic attacks apply to the DM construction directly, and make few or no assumptions about the compression function. These attacks involve attacking a t-bit hash function with more than 2t/2 work, in order to violate some property other than collision resistance. Exam- ples of generic attacks are Joux multicollision [8], long-message second preimage attacks [10,21] and herding attack [9]. 2. Cryptanalytic attacks apply to the compression function of the hash function. -
Hash (Checksum)
Hash Functions CSC 482/582: Computer Security Topics 1. Hash Functions 2. Applications of Hash Functions 3. Secure Hash Functions 4. Collision Attacks 5. Pre-Image Attacks 6. Current Hash Security 7. SHA-3 Competition 8. HMAC: Keyed Hash Functions CSC 482/582: Computer Security Hash (Checksum) Functions Hash Function h: M MD Input M: variable length message M Output MD: fixed length “Message Digest” of input Many inputs produce same output. Limited number of outputs; infinite number of inputs Avalanche effect: small input change -> big output change Example Hash Function Sum 32-bit words of message mod 232 M MD=h(M) h CSC 482/582: Computer Security 1 Hash Function: ASCII Parity ASCII parity bit is a 1-bit hash function ASCII has 7 bits; 8th bit is for “parity” Even parity: even number of 1 bits Odd parity: odd number of 1 bits Bob receives “10111101” as bits. Sender is using even parity; 6 1 bits, so character was received correctly Note: could have been garbled, but 2 bits would need to have been changed to preserve parity Sender is using odd parity; even number of 1 bits, so character was not received correctly CSC 482/582: Computer Security Applications of Hash Functions Verifying file integrity How do you know that a file you downloaded was not corrupted during download? Storing passwords To avoid compromise of all passwords by an attacker who has gained admin access, store hash of passwords Digital signatures Cryptographic verification that data was downloaded from the intended source and not modified. Used for operating system patches and packages. -
Cryptographic Checksum
Cryptographic checksum • A function that produces a checksum (smaller than message) • A digest function using a key only known to sender and recipient • Ensure both integrity and authenticity • Used for code-tamper protection and message-integrity protection in transit • The choices of hashing algorithms are Message Digest (MD) by Ron Rivest of RSA Labs or Secure Hash Algorithm (SHA) led by US government • MD: MD4, MD5; SHA: SHA1, SHA2, SHA3 k k message m tag Alice Bob Generate tag: Verify tag: ? tag S(k, m) V(k, m, tag) = `yes’ 4/19/2019 Zhou Li 1 Constructing cryptographic checksum • HMAC (Hash Message Authentication Code) • Problem to solve: how to merge key with hash H: hash function; k: key; m: message; example of H: SHA-2-256 ; output is 256 bits • Building a MAC out of a hash function: Paddings to make fixed-size block https://en.wikipedia.org/wiki/HMAC 4/19/2019 Zhou Li 2 Cryptoanalysis on crypto checksums • MD5 broken by Xiaoyun Input Output Wang et al. at 2004 • Collisions of full MD5 less than 1 hour on IBM p690 cluster • SHA-1 theoretically broken by Xiaoyun Wang et al. at 2005 • 269 steps (<< 280 rounds) to find a collision • SHA-1 practically broken by Google at 2017 • First collision found with 6500 CPU years and 100 GPU years Current Secure Hash Standard Properties https://shattered.io/static/shattered.pdf 4/19/2019 Zhou Li 3 Elliptic Curve Cryptography • RSA algorithm is patented • Alternative asymmetric cryptography: Elliptic Curve Cryptography (ECC) • The general algorithm is in the public domain • ECC can provide similar -
Hash Tables & Searching Algorithms
Search Algorithms and Tables Chapter 11 Tables • A table, or dictionary, is an abstract data type whose data items are stored and retrieved according to a key value. • The items are called records. • Each record can have a number of data fields. • The data is ordered based on one of the fields, named the key field. • The record we are searching for has a key value that is called the target. • The table may be implemented using a variety of data structures: array, tree, heap, etc. Sequential Search public static int search(int[] a, int target) { int i = 0; boolean found = false; while ((i < a.length) && ! found) { if (a[i] == target) found = true; else i++; } if (found) return i; else return –1; } Sequential Search on Tables public static int search(someClass[] a, int target) { int i = 0; boolean found = false; while ((i < a.length) && !found){ if (a[i].getKey() == target) found = true; else i++; } if (found) return i; else return –1; } Sequential Search on N elements • Best Case Number of comparisons: 1 = O(1) • Average Case Number of comparisons: (1 + 2 + ... + N)/N = (N+1)/2 = O(N) • Worst Case Number of comparisons: N = O(N) Binary Search • Can be applied to any random-access data structure where the data elements are sorted. • Additional parameters: first – index of the first element to examine size – number of elements to search starting from the first element above Binary Search • Precondition: If size > 0, then the data structure must have size elements starting with the element denoted as the first element. In addition, these elements are sorted.