L13 P2P Overlay Networks: Theory by T.S.R.K. Prasad EA C451 Internetworking Technologies References / Acknowledgements Ch 6: Peer-to-Peer Content Networks, [Hoffman] Sec 2.6: Peer-to-Peer Applications, [Kurose] [Theotokis] Stephanos Androutsellis-Theotokis, A Survey of Peer-to-Peer File Sharing Technologies References EA C451 INET TECH References / Acknowledgements [Ross-P2P] Keith W. Ross, and Dan Rubenstein, P2P Systems, Infocomm Tutorial [Kurose-P2P] Ch2: Applications, [Kurose] [Zhang-P2P] Prof. Zhi-Li Zhang, P2P, CSci5221: Foundations of Advanced Networking, Spring 2011. (http://www-users.cselabs.umn.edu/classes/Spring- 2011/csci5221/index.php) [Rexford-P2P] Jennifer Rexford, Peer-to-Peer File Sharing, COS 461: Computer Networks (http://www.cs.princeton.edu/courses/archive/spr12/cos461/) References EA C451 INET TECH References / Acknowledgements [Stoica-P2P] Ion Stoica, P2P Networks, EE122: Computer Networking, Dept of EECS, UCB, Fall 2002. (www-inst.eecs.berkeley.edu/~ee122/fa02) References EA C451 INET TECH Optional Readings [Andersen] David G. Andersen, Hari Balakrishnan, M. Frans Kaashoek, and Robert Morris, Resilient Overlay Networks, SOPS-2001. [Lua] Eng Keong Lua, Jon Crowcroft, Marcelo Pias, Ravi Sharma and Steven Lim, A Survey and Comparison of Peer-to-Peer Overlay Network Schemes, IEEE Communications Survey and Tutorial, March 2004. [Rodrigues] Rodrigo Rodrigues, and Peter Druschel, Peer-to- Peer Systems Optional Reading EA C451 INET TECH Presentation Overview Structured P2P Networks Unstructured P2P Networks Overlay Networks File Distribution Example Introduction Lecture Outline EA C451 INET TECH Presentation Overview Structured P2P Networks Unstructured P2P Networks Overlay Networks File Distribution Example Introduction Lecture Outline EA C451 INET TECH Client-Server Limitations • Scalability is hard to achieve • Presents a single point of failure • Requires administration • Unused resources at the network edge P2P systems try to address these limitations Introduction C/S Limitations EA C451 INET TECH Defintion of P2P 1) Significant autonomy from central servers 2) Exploits resources at the edges of the Internet – storage and content – CPU cycles – human presence 3) Resources at edge have intermittent connectivity, being added & removed Introduction Definition of P2P EA C451 INET TECH It’s a broad definition: • • P2P file sharing DHTs & their apps – Chord, CAN, – Napster, Gnutella, KaZaA, Pastry, Tapestry etc • P2P communication • P2P apps built over – Instant messaging emerging overlays – • P2P computation PlanetLab – seti@home Introduction Definition of P2P EA C451 INET TECH Pure P2P architecture • no always-on server • arbitrary end systems directly communicate • peers are intermittently connected and change IP addresses Introduction Pure P2P Architecture EA C451 INET TECH Characteristics of P2P Networks • Clients are also servers and routers – Nodes contribute content, storage, memory, CPU • Nodes are autonomous (no administrative • authority) • Network is dynamic: nodes enter and leave the network “frequently” • Nodes collaborate directly with each other (not through well-known servers) • Nodes have widely varying capabilities Introduction Characteristics of P2P Networks EA C451 INET TECH Benefits of P2P Networks • Efficient use of resources – Unused bandwidth, storage, processing power at the edge of the network • Scalability – Consumers of resources also donate resources – Aggregate resources grow naturally with utilization Introduction Benefits of P2P Networks EA C451 INET TECH Benefits of P2P Networks • Reliability – Replicas – Geographic distribution – No single point of failure • Ease of administration – Nodes self organize – No need to deploy servers to satisfy demand (c.f. scalability) – Built-in fault tolerance, replication, and load balancing Introduction Benefits of P2P Networks EA C451 INET TECH Key Issues in P2P Networks • Join/leave – How do nodes join/leave? Who is allowed? • Search and retrieval – How to find content? – How are metadata indexes built, stored, distributed? • Content Distribution – Where is content stored? How is it downloaded and retrieved? Introduction Key Issues P2P Networks EA C451 INET TECH Four Key Primitives (APIs) • Join – How to enter/leave the P2P system? • Publish – How to advertise a file? • Search – how to find a file? • Fetch – how to download a file? Introduction Four Key Primitives EA C451 INET TECH Presentation Overview Structured P2P Networks Unstructured P2P Networks Overlay Networks File Distribution Example Introduction Lecture Outline EA C451 INET TECH File Distribution: Client-Server vs Peer-to-Peer Question: how much time to distribute file (size F) from one server to N peers? – peer upload/download capacity is limited resource F bits Internet File Distribution Example The Problem EA C451 INET TECH Server Distributing a Large File d F bits 4 upload rate us Internet d3 d1 d2 upload rates ui download rates di File Distribution Example Server Distributing a Large File EA C451 INET TECH Server Distributing a Large File • Sending an F-bit file to N receivers – Transmitting NF bits at rate us – … takes at least NF/us time • Receiving the data at the slowest receiver – Slowest receiver has download rate dmin= mini{di} – … takes at least F/dmin time • Download time: max{NF/us , F/dmin} File Distribution Example Server Distributing a Large File EA C451 INET TECH Speeding Up the File Distribution • Increase the server upload rate – Higher link bandwidth at the server – Multiple servers, each with their own link • Alternative: have the receivers help – Receivers get a copy of the data – … and redistribute to other receivers – To reduce the burden on the server File Distribution Example Speeding up the File Distribution EA C451 INET TECH Peers Help Distributing a Large File F bits d4 upload rate u s u4 Internet d3 d1 u3 u2 u1 d2 upload rates ui download22 rates di File Distribution Example Peer Helpout EA C451 INET TECH Peers Help Distributing a Large File • Components of distribution latency – Server must send each bit: min time F/us – Slowest peer must receive each bit: min time F/dmin • Upload time using all upload resources – Total number of bits: NF – Total upload bandwidth us + sumi(ui) • Total: max{F/us , F/dmin , NF/(us+sumi(ui))} 23 File Distribution Example Peer Helpout EA C451 INET TECH Peer-to-Peer is Self-Scaling • Download time grows slowly with N – Client-server: max{NF/u s, F/dmin} – Peer-to-peer: max{F/us , F/dmin , NF/(us+sumi(ui))} • But… – Peers may come and go – Peers need to find each other – Peers need to be willing to help each other 24 File Distribution Example P2P is Self-Scaling EA C451 INET TECH Client-server vs. P2P: example client upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us 3.5 P2P 3 Client-Server 2.5 2 1.5 1 0.5 Minimum Distribution Time 0 0 5 10 15 20 25 30 35 N File Distribution Example Comparison EA C451 INET TECH P2P file distribution: BitTorrent • file divided into 256Kb chunks • peers in torrent send/receive file chunks tracker: tracks peers torrent: group of peers participating in torrent exchanging chunks of a file Alice arrives … … obtains list of peers from tracker … and begins exchanging file chunks with peers in torrent File Distribution Example BitTorrent EA C451 INET TECH Presentation Overview Structured P2P Networks Unstructured P2P Networks Overlay Networks File Distribution Example Introduction Lecture Outline EA C451 INET TECH Overlay networks overlay edge Overlay Networks EA C451 INET TECH Overlay graph Virtual edge • TCP connection • or simply a pointer to an IP address Overlay maintenance • Periodically ping to make sure neighbor is still alive • Or verify liveness while messaging • If neighbor goes down, may want to establish new edge • New node needs to bootstrap Overlay Networks Overlay Graph EA C451 INET TECH Overlays: all in the application layer application Tremendous design transport network data link flexibility physical – Topology, maintenance – Message types – Protocol – Messaging over TCP or UDP application application transport Underlying physical net is transport network network data link data link physical transparent to developer physical – But some overlays exploit proximity Overlay Networks Overlays @ Application Layer EA C451 INET TECH Overlay Networks – Normal View Overlay Networks Normal View EA C451 INET TECH Overlay Networks Focus at the application level Overlay Networks Application Level View EA C451 INET TECH Examples of overlays • DNS • BGP routers and their peering relationships • Content distribution networks (CDNs) • Application-level multicast – economical way around barriers to IP multicast • And P2P apps ! Overlay Networks Examples of Overlays EA C451 INET TECH More about overlays Unstructured overlays • e.g., new node randomly chooses three existing nodes as neighbors Structured overlays • e.g., edges arranged in restrictive structure Proximity • Not necessarily taken into account Overlay Networks More about Overlays EA C451 INET TECH Presentation Overview Structured P2P Networks Unstructured P2P Networks Overlay Networks File Distribution Example Introduction Lecture Outline EA C451 INET TECH Unstructured P2P Networks • Central Server / Directory (Napster) • Flooding (Gnutella) • Super Peers / Super Nodes (KaZaA) • Swarming (BitTorrent) Unstructured P2P Networks Types EA C451 INET TECH Central Server / Directory centralized directory Join and Publish File list and IP address is C uploaded 1 A 1 1 B Unstructured P2P Networks Central Server EA C451 INET TECH