Distributed Computing Systems Definition of Peer-To-Peer (P2P

2/26/2016

Definition of Peer-to-Peer (P2P)

• Significant autonomy from central servers • Exploits resources at edges of Internet Distributed Computing Systems – Storage and content – Multicast routing – CPU cycles Peer-to-Peer – Human knowledge (e.g., recommendations, classification) • Resources at edge may have intermittent connectivity

P2P Includes Introduction Outline • P2P file sharing – Napster, gnutella, KaZaA, eDonkey … • Definition (done ) • P2P communication • Overlay Networks (next ) – Instant messaging – Voice-over-IP (e.g. Skype) • P2P Applications • P2P multicast routing – Mbone, Yoid, Scattercast • P2P computation – seti@home • P2P apps built on overlays – PlanetLab

Overlay Networks Overlay Network – Overview • Virtual edges • Network on top of another network – e.g., a TCP connection – Simpler: an IP address (connection method not specified) • Creation – May be structured (e.g., tree) or unstructured (nodes randomly choose neighbors) – May or may not take into account proximity of nodes Overlay • Overlay maintenance edge – Periodically “ping” to make sure neighbor alive – Or, verify liveness while sending messages – If neighbor goes down, may establish new edge – New node needs to “bootstrap” in

1 2/26/2016

Overlay Networks – Examples of Overlays at Application Layer • Tremendous design flexibility • Domain Name Service (DNS) – Topology, maintenance • Border Gateway Protocol (BGP) routers (with – Message types their peering relationships) – Protocol • Content Distribution Networks (CDNs) – Underlying protocol (TCP/UDP) Application-level multicast (e.g., Mbone) • Underlying network • transparent – But some may exploit proximity • And P2P applications! (e.g., peers in same ISP)

Introduction Outline P2P File Sharing – General • Alice runs P2P client on her • Registers her content in • Definition (done ) laptop P2P system • Intermittently connected • Asks for “Hey Jude” • Overlay Networks (done ) – Gets new IP address each • Application displays other time • P2P Applications (next ) peers with copy • Alice choses one, Bob • File is copied from Bob’s computer to Alice’s ‰ P2P • While Alice downloads, others upload

P2P File Sharing Capabilities Copyright Issues (1 of 2)

• Allows Alice to show directory in her file Direct Infringement Indirect infringement system • End users who download or • An individual accountable for actions of others upload copyright works – Anyone can retrieve file from it • Contributory – Like Web server – Knew of direct infringement – And caused, induced or • Allows Alice to copy files from other’s materially contributed to direct infringement – Like Web client • Vicarious – Able to control direct infringers • Allows users to search nodes for content (e.g. terminate accounts) – And derived direct financial based on keyword matches benefit – Like search engine (e.g., Google) (Knowledge not necessary)

2 2/26/2016

Betamax VCR Defense Guidelines for P2P developers • Napster – Disruptive, proof of concept • Manufacturer not liable for • Total control so that can be contributory infringement sure there is no direct • Gnutella infringement • “Capable of substantial, – Open source, non-centralized search non-infringing use” Or • KaZaA/FastTrack • No control – no remote kill • But in Napster case, court – Surpassed the Web (in terms of bytes) switch, no customer support found defense does not • eDonkey/Overnet apply to all vicarious liability • Actively promote non- infringing use of products – Distributed Hash Table (distributed content) • Disaggregate functions • Is success due to massive number of servers (i.e., – Indexing, search transfer P2P aspect) or simply because content is “free”?

P2P Distributed Computing: P2P Communication seti@home • Alice runs IM client on PC • Alice initiates direct TCP • Search for extra • Peer sets up TCP terrestrial (ET) connection to central • Intermittently connects to connection with Bob intelligence computer Internet ‰P2P • Central site collects radio • Downloads chunk – Gets new IP address each • Alice and Bob chat telescope data time • Peer does FFT on chunk, • Data divided into work uploads results, gets new Registers herself with • chunks (300 Kbytes) chunk “system” • Can also be voice, video • User obtains client • Learns from “system” that and text (e.g., Skype ) – Runs in background (when • Not Peer-to-peer, but Bob (in her buddy list) is screensaver on) exploits resource at active network edge

Outline P2P Common Primitives

• Introduction (done ) • Join : how to I begin participating? • P2P file sharing techniques (next ) • Publish : how do I advertise my file? • Uses of P2P • Challenges • Search : how to I find a file? • Fetch : how to I retrieve a file?

Top 2, relatively easy Bottom 2, more of challenge

3 2/26/2016

Example: Searching P2P Challenges

1000’s of nodes N2 • Challenge 1 – Search Set of nodes may change N1 N3 – Human’s goals ‰ find file

– Given keywords or human description, find a Key=“title” Internet Value=MP3 data… ? specific file Client Publisher • Challenge 2 – Fetch Lookup(“title”) N4 N6 – One file determined ‰ get bits N5 – Computer goal, obtain content • Needles versus Haystacks Searching for top 40 pop song? Or obscure punk track ‘81 nobody’s heard of? • Search expressiveness Whole word? Regular expressions? File names? Attributes? Whole-text search?

What’s out there? Next Topic...

Central Flood Super- Route • Centralized Database node flood – Napster Whole File Napster Gnutella Freenet • Query Flooding – Gnutella Chunk BitTorrent KaZaA (DHTs) • Intelligent Query Flooding – KaZaA Based (bytes, not eDonkey2k chunks) • Swarming New BT – BitTorrent • Structured Overlay Routing – Distributed Hash Tables

Napster Legacy Napster: History

• May ‘99 : Shawn Fanning • Paradigm shift (freshman at Northeastern) • Not the first (probably Eternity, from Ross founds Napster • Dec ‘99 : first lawsuit Anderson in Cambridge) • Mar ‘00 : 25% UWisc traffic – http://www.cl.cam.ac.uk/~rja14/eternity/eternity.html Napster • Apr ‘01 : US Circuit Court of • But instructive for what it got right Appeals: “Napster knew users • And wrong… violating copyright laws” • Jul ‘01 : # simultaneous online • Also had an economic message… users – Napster: 160K, Gnutella: 40K, • And legal… Morpheus (KaZaA): 300K

4 2/26/2016

Napster: History Napster: Overiew

• Jul ’01 : Napster • Application-level, client-server protocol over shuts down TCP – Judge orders • Centralized Database: Napster to pull – Join : on startup, client contacts central server plug… – Publish : reports list of files to central server • But other file – Search : query the server ‰ return someone that sharing apps take stores the requested file over! • Selected as best based on “pings” – Fetch : get the file directly from peer

Napster: Publish Napster: Search

123.2.0.18

Centralized Centralized insert (X, 123.2.21.23 ) search(A) (napster.com ) (napster.com ) ‰ returns 123.2.0.18 ... Fetch ‰ returns 163.2.1.0 …

Publish Query Reply

I have X, Y, and Z! Where is file A? 123.2.21.23 Client “pings” each host, picks closest

Napster: Discussion Next Topic...

• Pros: • Centralized Database (Searching) – Simple – Napster • Query Flooding (Searching) – Search scope is O(1) – Gnutella – Controllable (pro or con?) • Supernode Query Flooding (Searching) Cons: – KaZaA • • Swarming (Fetching) – Server maintains O(N) state – BitTorrent – Server does all processing • Structured Overlay Routing (Both, but mostly search) – Single point of failure – Distributed Hash Tables (DHT) – (Napster’s server farm had difficult time keeping up with traffic)

5 2/26/2016

Query Flooding Overview Query Flooding

• Decentralized method • Join : on startup, client I have file A. of searching contacts a few other nodes – Central server directory – these become its “neighbors” I have file A. no longer bottleneck • Publish : no need – More difficult to “pull • Search : ask neighbors the plug” (Gnutella uses 7 ), who ask Reply • Each peer: their neighbors, and so on... – Stores selected files when/if found, reply to – Routes queries to and sender. from neighbors – TTL limits propagation ( Gnutella – Responds to queries if uses 10 ) files stored locally – Reverse path forwards Query – Serves files responses (not files) • Fetch : get the file directly Where is file A? from peer

Flooding Discussion Gnutella: History

• Pros: • Mar ’00 : J. Frankel and T. Pepper from – Fully de-centralized released Gnutella (AOL) – Search cost distributed – Processing @ each node permits powerful search semantics – Immediately withdrawn, but Slashdot-ted • Cons: • Became open source – Search scope is O( N) – Good, since initial design was poor – Search time is O(???) – depends upon “height” of tree – Nodes leave often, network unstable • Soon many other clients: Bearshare, • TTL-limited search works well for haystacks Morpheus, LimeWire, etc. – For scalability, does NOT search every node. May have to re- issue query later • ’01 : many protocol enhancements including • Example: Gnutella “ultrapeers” http://www.computing2010.com/expansions.php?id=07

Gnutella: Discussion Next Topic...

• Researchers like it because it is open source • Centralized Database (Searching) – But is it representative for research? – Napster • Users’ anecdotal comments: “It stinks” • Query Flooding (Searching) – Tough to find anything – Gnutella • Supernode Query Flooding (Searching) – Downloads don’t complete – KaZaA • Fixes • Swarming (Fetching) – Hierarchy of nodes – BitTorrent – Queue management • Structured Overlay Routing (Both, but mostly search) Distributed Hash Tables (DHT) – Parallel downloads – – …

6 2/26/2016

Flooding with Supernodes Stability and Superpeers

• Some nodes better • Join : on startup, client • Why superpeers? connected, longer contacts a “supernode” connected than others – may at some point – Query consolidation – Use them more heavily become one itself • Many connected nodes may have only a few files • Architecture • Publish : send list of files • Propagating query to sub-node would take more b/w than to supernode answering it yourself – Hierarchical • Search : send query to – Caching effect – Cross between Napster supernode, supernodes and Gnutella flood query amongst • Requires network stability • “Smart” Query Flooding themselves. • Superpeer selection is time-based • Fetch : get the file directly from peer(s) – How long you’ve been on good predictor of how long you’ll be around – can fetch simultaneously from multiple peers

Supernodes Network Design Supernodes: Publish

“Super Nodes”

insert(X, 123.2.21.23) ...

Publish

I have X!

123.2.21.23

Supernodes: Fetch Supernodes: Search (And use of hashes…) search(A) • More than one node may have requested file... --> 123.2.22.50 • How to tell? – Must be able to distinguish identical files – Not necessarily same filename – Same filename not necessarily same file... search(A) • Use Hash of file 123.2.22.50 --> KaZaA uses UUHash: fast, but not secure 123.2.0.18 – Query Replies – Alternatives: MD5, SHA-1 • How to fetch? Where is file A? – Get bytes [0..1000] from A, [1001...2000] from B

123.2.0.18

7 2/26/2016

Supernode Flooding Discussion KaZaA: History

• Pros : • In 2001, KaZaA created by Dutch company Kazaa BV – Tries to take into account node heterogeneity • Single network called FastTrack used by other clients as well • Bandwidth – Morpheus, giFT, etc. • Host Computational Resources • Eventually protocol changed so other clients could no longer Host Availability (?) • talk to it – Rumored to take into account network locality • Most popular file sharing network in 2005 with >10 million – Scales better users, sharing over 10,000 terabytes of content (numbers • Cons : vary) – Still no real guarantees on search scope or search time – More popular than Napster • Similar behavior to plain flooding, but better – Over 50% of Internet traffic at one time • Example: KaZaA • MP3s & entire albums, videos, games

KaZaA: The Service KaZaA: Technology

• Optional parallel downloading of files • Software – User can configure max down and max up – Proprietary • Automatically switches to new download server when Control data encrypted current server unavailable – Everything in HTTP request and response messages • Provides estimated download times – • Queue management at server and client • Each peer a supernode or assigned to a – Frequent uploaders can get priority supernode • Keyword search – Each SN has about 100-150 children • Responses to keyword queries come in waves: stops when x – Each SN has TCP connections with 30-50 other responses are found supernodes • From user’s perspective, resembles Google, but provides links to mp3s and videos rather than Web (Measurement study next slide)

KaZaA Measurement Study KaZaA Measurement Study

• KaZaA is more than one workload! – Many files < 10MB (e.g., Audio Files) – Many files > 100MB (e.g., Movies)

from Gummadi et al ., SOSP 2003

8 2/26/2016

KaZaA: Architecture KaZaA: Overlay Maintenance

• Nodes with more connection bandwidth and • List of potential supernodes when download more availability ‰ supernodes (SN) software • Each supernode acts as a min-Napster hub • New peer goes through list until finds – Tracking content of children (ordinary nodes, ON) – Maintaining IP addresses of children operational supernode • When ON connects to SN, upload metadata – Connects, contains up-to-date list (200 entries) – File name – Nodes in list are “close” to ON – File size – Node pings 5 nodes, connects to one – Content hash – used for fetch request, rather than name • If supernode goes down, node obtains – File descriptions – used for keyword matches updated list and choses new supernode

KaZaA: Queries KaZaA-lite

• Node first sends query to supernode • Hacked version of KaZaA client – Supernode responds with matches • No spyware, no pop-up windows – If x matches found, done • Everyone is rated as a priority user • Otherwise, supernode forwards query to • Supernode hopping subset of supernodes – After receiving replies from SN, ON often connects – If total of x matches found, done to a new SN an re-sends query • Otherwise, query further forwarded – SN does not cache hopped-out ON’s metadata – By original supernode

KaZaA: Downloading & Recovery KaZaA: Summary

• If file is found in multiple nodes, user can • KaZaA provides powerful file search and transfer service select parallel downloading without server infrastructure • Exploit heterogeneity – Identical copies identified by ContentHash • Provide automatic recovery for interrupted downloads • HTTP byte-range header used to request • Powerful, intuitive user interface different portions of the file from different • Copyright infringement nodes – International cat-and-mouse game – With distributed, serverless architecture, can the plug be • Automatic recovery when server peer stops pulled? sending file – Prosecute users? – ContentHash – Launch DoS attack on supernodes? – Pollute? (Copyright holder intentionally puts bad copies out)

9 2/26/2016

Searching & Fetching Next Topic...

• Query flooding finds: • Centralized Database (Searching) – An object (filename or hash) – Napster • Query Flooding (Searching) – A host that serves that object – Gnutella • In Query flooding systems, download from • Supernode Query Flooding (Searching) host that answered query – KaZaA • Swarming (Fetching) • Generally uses only one source – BitTorrent • Structured Overlay Routing (Both, but mostly search) • Can we do better? – Distributed Hash Tables (DHT)

Fetching in Parallel and Swarming BitTorrent: Swarming

• When you have an object ID, • 2001, BramCohen debuted BitTorrent • Get list of peers serving that ID – Was open source, now not (µTorrent) • Key Motivation: – Easier than the keyword lookup – Popularity exhibits temporal locality (Flash Crowds) – Queries are structured – E.g., Slashdot effect, CNN on 9/11, new movie/game release • Download in parallel from multiple peers • Focused on Efficient Fetching , not Searching – Distribute the same file to all peers • “Swarming” – Single publisher, multiple downloaders – Download from others downloading same object • Has some “real” publishers: at same time – Blizzard Entertainment has used it to distribute beta of games • Example: BitTorrent

BitTorrent: Overview BitTorrent: Overview

tracks peers • Swarming: participating in torrent – Join : contact centralized “tracker” server, get a list of peers – Publish : Run a tracker server – Search : Out-of-band • E.g., use Google to find a tracker for the file you want. – Fetch : Download chunks of the file from your peers. Upload chunks you have to them. • Big differences from Napster: – Chunk based downloading – “Few large files” focus – Anti-freeloading mechanisms

10 2/26/2016

BitTorrent: Publish/Join BitTorrent: Fetch

Tracker

BitTorrent: Sharing Strategy BitTorrent: Summary

• File is broken into pieces • Pros – Typically 256 Kbytes – Works reasonably well in practice – Upload pieces while download – Gives peers incentive to share resources; avoids • Piece selection freeloaders – Select rarest piece for request • Cons – Except at beginning, select random pieces • Employ “Tit-for-tat” sharing strategy – Central tracker server needed to bootstrap swarm – A is downloading from some other people – Tracker is a design choice, not a requirement • A will let the fastest N of those download from him • Newer BT variants use a “distributed tracker” - a Distributed Hash – Be optimistic: occasionally let freeloaders download Table • Otherwise no one would ever start! • Also allows you to discover better peers to download from when they reciprocate

Next Topic... Directed Searches

• Centralized Database (Searching) • Idea: – Napster – Assign particular nodes to hold particular content (or • Query Flooding (Searching) pointers to it) – Gnutella – When node wants that content, go to node that is • Supernode Query Flooding (Searching) supposed to have or know about it – KaZaA • Challenges: • Swarming (Fetching) – Distributed: want to distribute responsibilities among – BitTorrent existing nodes in overlay • Structured Overlay Routing (Both, but mostly search) – Distributed Hash Tables (DHT) – Adaptive: nodes join and leave P2P overlay • Distribute knowledge responsibility to joining nodes • Redistribute responsibility knowledge from leaving nodes

11 2/26/2016

Distributed Hash Table (DHT): DHT: Step 1 – The Hash Overview • Introduce a hash function to map the object being searched for to a unique identifier: • Abstraction : a distributed “hash-table” data – e.g., h(“Led Zepplin”) → 8045 structure: • Distribute range of hash function among all nodes in – put(id, item); network – item = get(id); • Implementation : nodes in system form distributed data structure – Can be Ring, Tree, Hypercube, …

• Each node must “know about” at least one copy of each object that hashes within its range (when one exists)

DHT: “Knowing About Objects” DHT: Step 2 – Routing • Two alternatives • For each object, node(s) whose range(s) cover – Node can cache each (existing) object that hashes that object must be reachable via “short” path within its range – by querier node (assumed can be chosen arbitrarily) Pointer-based: level of indirection – node caches – – by nodes that have copies of object (when pointer- pointer to location(s) of object based approach is used) • Different approaches (CAN, Chord, Pastry, Tapestry) differ fundamentally only in routing approach – Any “good” random hash function will suffice

Example: Circular DHT (1) Example: Circle DHT (2) 1 O(N) messages 0001 Who’s resp on avg to resolve 3 for key 1110 ? 15 query, when there I am are N peers 0011 4 1111

12 1110 5 1110 0100 10 1110 8 1100

1110 0101 • Each peer only aware of immediate 1110 Define closest 1110 successor and predecessor. as closest 1010 • “Overlay network” successor 1000

12 2/26/2016

Example: Circular DHT with Shortcuts Example: Peer Churn 1 1 Who’s resp for key 1110? •To handle peer churn, require 3 3 each peer to know IP address 15 15 of its two successors. • Each peer periodically pings its 4 4 two successors to see if still alive

12 12 5 5

10 10 8 8 • Each peer keeps track of IP addresses of predecessor, • Peer 5 abruptly leaves successor, short cuts. • Peer 4 detects; makes 8 its immediate successor; • Reduced from 6 to 2 messages. asks 8 who its immediate successor is; makes 8’s • Possible to design shortcuts so O(log N) neighbors, O(log N) messages in query immediate successor its second successor. • What if peer 13 wants to join?

DHT: Operations DHT: Discussion

• Join : On startup, contact “bootstrap” node and integrate into • Pros: distributed data structure ‰ get a node id – Guaranteed lookup Publish : Route publication for file id toward close node id • O(log N) per node state and search scope along data structure (e.g. next in ring) – • Search : Route query for file id toward a close node id • Cons: • Fetch : Two options: – Not used – Publication contains actual file ‰ fetch from where query stops • Academically popular since 2k, but in practice, not so – Publication says “I have file X” ‰ query says 128.2.1.3 has X, use much IP routing to get X from 128.2.1.3 – Supporting non-exact match search is hard – Churn hard

Peers as Relays When P2P / DHTs useful?

• Problem when both Alice • Caching and “soft-state” data and Bob are behind “NATs”. – NAT prevents outside peer – Works well! BitTorrent, KaZaA, etc., all use peers from initiating call to insider as caches for hot data peer (e.g. can’t be server) • Solution: • Finding read-only data – Relay is chosen (based on connectivity to both) – Limited flooding finds “hay” – Each peer initiates session DHTs find “needles” with relay – – Peers can now communicate through NATs via relay • (Used by Skype)

13 2/26/2016

A Peer-to-peer Google? Writable, Persistent P2P

• Complex intersection queries (“the” + “who”) • Do you trust your data to 100,000 monkeys? – Billions of hits for each term alone • Node availability hurts • Sophisticated ranking – Ex: Store 5 copies of data on different nodes – When someone goes away, you must replicate data they – Must compare many results before returning subset to held user – Hard drives are *huge*, but upload bandwidths relatively • Very, very hard for DHT / P2P system tiny – Need high inter-node bandwidth • Takes many hours to upload contents of hard drive • Very expensive leave/replication situation! – (This is exactly what Google does - massive clusters)

P2P: Summary

• Many different styles – Centralized, flooding, swarming, unstructured and structured routing • Lessons learned: – Single points of failure are bad – Flooding messages to everyone is bad – Underlying network topology is important – Not all nodes are equal – Need incentives to discourage freeloading