Examples of “Overlay” Networks • Overlay routing infrastructure • Distributed name look up 15-744: Computer Networking • Content delivery networks • Peer-to-peer content sharing • Distributed hash tables • Disruption tolerant networks L-12 DHT and DTN 2 Overview Routing: Structured Approaches • Routed Lookups – Chord • Goal: make sure that an item (file) identified is always found in a reasonable # of steps • Abstraction: a distributed hash-table (DHT) data structure • Comparison of DHTs • insert(id, item); • item = query(id); • Note: item can be anything: a data object, document, file, pointer •I3 to a file… • Many proposals • CAN (ICIR/Berkeley) •DTNs • Chord (MIT/Berkeley) • Pastry (Rice) • Tapestry (Berkeley) •… 3 4 1 Routing: Chord Aside: Hashing • Associate to each node and item a unique id • Advantages in an uni-dimensional space • Let nodes be numbered 1..m • Properties • Client uses a good hash function to map a URL to 1..m • Say hash (url) = x, so, client fetches content from node • Routing table size O(log(N)) , where N is the x total number of nodes • One hop access • Guarantees that a file is found in O(log(N)) • Any problems? steps • What happens if a node goes down? • What happens if a node comes back up? • What if different nodes have different views? 5 6 Robust hashing Consistent Hash • Assume 90 documents, node 1..9, and node 10 • “view” = subset of all hash buckets that are which was dead is alive again visible • % of documents in the wrong node? • Desired features • 10, 19-20, 28-30, 37-40, 46-50, 55-60, 64-70, 73-80, 82-90 • Balanced – in any one view, load is equal • Disruption coefficient = ½ across buckets • Unacceptable, use consistent hashing – idea behind • Smoothness – little impact on hash bucket Akamai! contents when buckets are added/removed • Spread – small set of hash buckets that may hold an object regardless of views • Load – across all views # of objects assigned to hash bucket is small 7 8 2 Consistent Hash – Example Routing: Chord Basic Lookup • Construction 0 N120 • Assign each of C hash buckets to 14 random points on mod 2n circle, N10 “Where is key 80?” where, hash key size = n. Bucket N105 12 4 • Map object to random position on circle • Hash of object = closest 8 clockwise bucket “N90 has K80” N32 • Smoothness addition of bucket does not cause much movement between existing buckets K80 N90 • Spread & Load small set of buckets that lie near object • Balance no bucket is responsible for large number of objects N60 9 10 Routing: Finger table - Faster Lookups Routing: Chord Summary • Assume identifier space is 0…2m • Each node maintains • Finger table ¼ ½ •Entry i in the finger table of n is the first node that succeeds or equals n + 2i 1/8 • Predecessor node • An item identified by id is stored on the 1/16 1/32 1/64 successor node of id 1/128 N80 11 12 3 Routing: Chord Example Routing: Chord Example • Assume an Succ. Table • Node n2:(3) joins 0 i id+2i succ 0 2 1 identifier space 1 Succ. Table 7 1 3 1 i 0..8 2 5 1 0 i id+2 succ 1 0 2 2 • Node n1:(1) 7 1 3 1 joinsall entries 6 2 2 5 1 in its finger table are initialized to 6 2 itself 5 3 4 Succ. Table i id+2i succ 5 3 4 0 3 1 1 4 1 2 6 1 13 14 Routing: Chord Example Routing: Chord Examples Succ. Table Succ. Table Items i id+2i succ i id+2i succ 7 0 1 1 0 1 1 1 2 2 1 2 2 • Nodes n3:(0), n4:(6) 2 4 0 • Nodes: n1:(1), n2(3), 2 4 0 join Succ. Table i n3(0), n4(6) 0 i id+2 succ 0 Succ. Table Items 0 2 2 i 1 • Items: f1:(7), f2:(1) 1 i id+2 succ 1 7 1 3 6 7 0 2 2 2 5 6 Succ. Table 1 3 6 2 5 6 i id+2i succ 0 7 0 6 2 Succ. Table 6 2 1 0 0 i id+2i succ 2 2 2 0 7 0 Succ. Table 1 0 0 Succ. Table i id+2i succ 2 2 2 i id+2i succ 5 3 5 3 4 0 3 6 4 0 3 6 1 4 6 1 4 6 2 6 6 2 6 6 15 16 4 Routing: Query What can DHTs do for us? • Upon receiving a • Distributed object lookup query for item id, a Succ. Table Items node: i id+2i succ 7 • Based on object ID 0 1 1 • Check whether stores 1 2 2 the item locally 2 4 0 • De-centralized file systems • If not, forwards the • CFS, PAST, Ivy query to the largest 0 Succ. Table Items 7 1 i id+2i succ 1 node in its successor 0 2 2 query(7) • Application Layer Multicast table that does not 1 3 6 exceed id 2 5 6 • Scribe, Bayeux, Splitstream Succ. Table 6 2 i id+2i succ • Databases 0 7 0 1 0 0 Succ. Table •PIER 2 2 2 i id+2i succ 5 3 4 0 3 6 1 4 6 2 6 6 17 18 Overview Comparison • Routed Lookups – Chord • Many proposals for DHTs • Tapestry (UCB) -- Symphony (Stanford) -- 1hop (MIT) • Pastry (MSR, Rice) -- Tangle (UCB) -- conChord (MIT) • Comparison of DHTs • Chord (MIT, UCB) -- SkipNet (MSR,UW) -- Apocrypha (Stanford) • CAN (UCB, ICSI) -- Bamboo (UCB) -- LAND (Hebrew Univ.) • Viceroy (Technion) -- Hieras (U.Cinn) -- ODRI (TexasA&M) •I3 • Kademlia (NYU) -- Sprout (Stanford) • Kelips (Cornell) -- Calot (Rochester) • Koorde (MIT) -- JXTA’s (Sun) •DTNs • What are the right design choices? Effect on performance? 19 20 5 Deconstructing DHTs DHT Routing Geometries Two observations: • Geometries: 1. Common approach • Tree (Plaxton, Tapestry) • N nodes; each labeled with a virtual identifier (128 bits) • Ring (Chord) • define “distance” function on the identifiers • Hypercube (CAN) • routing works to reduce the distance to the destination • XOR (Kademlia) • Hybrid (Pastry) 2. DHTs differ primarily in their definition of “distance” • typically derived from (loose) notion of a routing geometry • What is the impact of geometry on routing? 21 22 Tree (Plaxton, Tapestry) Hypercube (CAN) 110 111 100 101 010 011 000 001 000 001 010 011 100 101 110 111 Geometry Geometry • nodes are the corners of a hypercube • nodes are leaves in a binary tree • distance = #matching bits in the IDs of two nodes • distance = height of the smallest common subtree • logN neighbors per node; each at distance=1 away • logN neighbors in subtrees at distance 1,2,…,logN 23 24 6 Ring (Chord) Hybrid (Pastry) 000 111 001 110 010 101 011 Geometry: 100 Geometry • combination of a tree and ring • nodes are points on a ring • two distance metrics • distance = numeric distance between two node IDs • default routing uses tree; fallback to ring under failures • logN neighbors exponentially spaced over 0…N • neighbors picked as on the tree 25 26 XOR (Kademlia) Geometry’s Impact on Routing • Routing • Neighbor selection: how a node picks its routing entries • Route selection: how a node picks the next hop 00 0110 11 • Proposed metric: flexibility 0100 11 10 • amount of freedom to choose neighbors and next-hop paths • FNS: flexibility in neighbor selection Geometry: • FRS: flexibility in route selection • distance(A,B) = A XOR B • logN neighbors per node spaced exponentially • intuition: captures ability to “tune” DHT performance • not a ring because there is no single consistent • single predictor metric dependent only on routing issues ordering of all the nodes 27 28 7 FNS for Ring Geometry FRS for Ring Geometry 000 000 110 111 001 111 001 110 010 110 010 101 011 101 011 100 100 • Chord algorithm picks ith neighbor at 2i distance • Chord algorithm picks neighbor closest to destination • A different algorithm picks ith neighbor from [2i , 2i+1) • A different algorithm picks the best of alternate paths 29 30 Flexibility: at a Glance Geometry Flexibility Performance? Validate over three performance metrics: Flexibility Ordering of Geometries 1. resilience 2. path latency Neighbors Hypercube << Tree, XOR, Ring, Hybrid 3. path convergence (FNS) (1) (2i-1) Metrics address two typical concerns: Routes Tree << XOR, Hybrid < Hypercube < Ring • ability to handle node failure (FRS) (1) (logN/2) (logN/2) (logN) • ability to incorporate proximity into overlay routing 31 33 8 Analysis of Static Resilience Does flexibility affect static resilience? Two aspects of robust routing 100 • Dynamic Recovery : how quickly routing state is 80 recovered after failures Tree XOR 60 Hypercube • Static Resilience : how well the network routes before Hybrid recovery finishes 40 • captures how quickly recovery algorithms need to work % Failed Paths Ring • depends on FRS 20 0 Evaluation: 0 102030405060708090 % Failed Nodes • Fail a fraction of nodes, without recovering any state • Metric: % Paths Failed Tree << XOR ≈ Hybrid < Hypercube < Ring Flexibility in Route Selection matters for Static Resilience 34 35 Which is more effective, FNS or FRS? Overview 100 FNS + FRS Ring • Routed Lookups – Chord 80 FNS Ring FRS Ring 60 Plain Ring • Comparison of DHTs CDF 40 20 • I3 0 • Slides Yuan; following slides FYI 0 400 800 1200 1600 2000 Latency (msec) •DTNs Plain << FRS << FNS ≈ FNS+FRS Neighbor Selection is much better than Route Selection 36 39 9 Multicast Mobility Sender S1 S2 RRPR R HA FA RP: Rendezvous 5.0.0.1 12.0.0.4 Mobile Point R R Node Home Network 5.0.0.3 C1 C2 Network 5 40 41 i3: Motivation The i3 solution • Today’s Internet based on point-to-point • Solution: abstraction • Add an indirection layer on top of IP • Implement using overlay networks • Applications need more: • Multicast • Solution Components: • Naming using “identifiers” • Mobility • Subscriptions using “triggers” • Anycast So, what’s the problem? A different solution for each service • DHT as the gluing substrate • Existing solutions: Only primitive needed • Change IP layer •Overlays Every problem in CS … Indirection 42 43 10 i3: Rendezvous Communication i3: Service Model • Packets addressed to identifiers (“names”) • API • Trigger=(Identifier, IP address): inserted by • sendPacket(id, p); receiver • insertTrigger(id, addr); • removeTrigger(id, addr); // send(R, data) optional send(ID, data) Sendertrigger Receiver (R) • Best-effort