Consistent Hashing and Distributed Hash Tables Marco Serafini COMPSCI 590S Lecture 12 Motivation 2 Peer-to-Peer Networks • Decentralized distributed systems • No/minimal central authority • Main functionality: file sharing • Napster • BitTorrent • Problems • No central index: how to find file? • No central master: how to know participant? • Churn: nodes come and go all the time • Q: Do these problems arise in Big Data systems? 33 Content Delivery Networks • Cache services closer to the clients • Initially: mainly static web contents, images etc. • Lower latency due to proximity • Data replication offloads source servers • Example: Akamai • Similar problems as P2P networks 44 Consistent Hashing • Consistent Hashing • Assigns keys to values (files) • Membership information is distributed • Designed to balance load and deal with churn • Distributed Hash Table (DHT) • Used to find the node responsible for a key quickly • Distributed index: each node keeps a subset of the index 55 Consistent Hashing 6 Consistent Hashing • Nodes are assigned a key • Values are assigned a key • Values are stored by the first successor node nodes NOTE: In this example: • n = 8 • Ids are uniformly distributed, but this is a simplification 7 7 Membership Changes • Node joins: take data from successor • Node leaves: give data to successor • Local changes, no global reconfiguration à good for churn 8 8 Theoretical Results • Q: What do these results tell us? • ! is arbitrarily small with "(log ') virtual nodes • Virtual nodes: multiple keys associated to same virtual node 99 Finding Data with Consistent Hashing 10 Finger Tables • Each node keeps a finger table • Start of finger table element !: " + 2%&' ()* 2+ • Each element defines interval of size 2%&' until next element • Each interval is associated to the first node in it Typo: should • Constants be “node” • Number of bits in key: ( • Number of nodes in system: " 11 11 Properties • More info on closer keys • Subsequent intervals double their size • No info for successors of all keys • Only successors for intervals 1212 Lookups for Key k • Find node j immediately preceding k in local finger table • Ask j for node immediately preceding k in remote finger table Typo: should • Iterate be “node” • Every step halves distance • Q: Why? • After !"# $ steps go sequential 13 13 Number of Lookups With high probability, the number of nodes that must be contacted to find a success in a !-nodes network is "($%& !) Proof • Each step from node n to node p halves the distance • After log ! steps, distance ≤ 2-/! • Interval contains 1 node on average, " log ! w.h.p. 1414 Dealing with Node Joins • Single node joins • New node initializes finger table (with lookups) • Existing nodes update fingers • Transfer keys to the new successor • In practice • Concurrent node joins • Failures • No strong guarantees: lookups can fail • Chord tries to stabilize eventually (best effort) 1515 Stabilization • Having correct successor pointers is sufficient for correctness (with joins) • When joining • Find successor and point to it • Get part of the successor’s keys • Periodically • Replace successor’s predecessor with self if needed • Check random local fingers and see if still updated by doing lookup n p s 1616 Fault Tolerance • Each node keeps list of r-hop successors • If stabilize notices that successor is failed, replace it • Application can replicate to r-hop successors 1717.