Server Distribu ng a Large File
F bits
d4 Peer-to-Peer File Sharing upload rate u s Internet Mike Freedman d d3 COS 461: Computer Networks 1 d h p://www.cs.princeton.edu/courses/archive/spr14/cos461/ 2 download rates di
2
Server Distribu ng a Large File Speeding Up the File Distribu on • Sending an F-bit file to N receivers • Increase the server upload rate
– Transmi ng NF bits at rate us – Higher link bandwidth at the server
– … takes at least NF/us me – Mul ple servers, each with their own link • Receiving the data at the slowest receiver • Alterna ve: have the receivers help
– Slowest receiver has download rate dmin= mini{di} – Receivers get a copy of the data
– … takes at least F/dmin me – … and redistribute to other receivers – To reduce the burden on the server • Download me: max{NF/us , F/dmin}
3 4
1 Peers Help Distribu ng a Large File Peers Help Distribu ng a Large File • Components of distribu on latency
F bits – Server must send each bit: min me F/us d4 – Slowest peer must receive each bit: min me F/dmin
u4 • Upload me using all upload resources upload rate us Internet – Total number of bits: NF d – Total upload bandwidth us + sumi(ui) d1 3 u • Total: max{F/u , F/d , NF/(u +sum (u ))} u1 u2 3 s min s i i d2
upload rates ui download rates d 5 i 6
Peer-to-Peer is Self-Scaling Loca ng the Relevant Peers • Download me grows slowly with N • Three main approaches
– Client-server: max{NF/u s, F/dmin} – Central directory (Napster)
– Peer-to-peer: max{F/us , F/dmin , NF/(us+sumi(ui))} – Query flooding (Gnutella) – Hierarchical overlay (Kazaa, modern Gnutella) • But… – Peers may come and go • Design goals – Peers need to find each other – Scalability – Peers need to be willing to help each other – Simplicity – Robustness – Plausible deniability
7 8
2 Peer-to-Peer Networks: Napster Napster Directory Service
• Napster history: the rise • Napster history: the fall • Client contacts Napster (via TCP) – 1/99: Napster version 1.0 – Mid 2001: out of – Provides a list of music files it will share – 5/99: company founded business due to lawsuits – … and Napster’s central server updates the directory – Mid 2001: dozens of – 12/99: first lawsuits • Client searches on a tle or performer – 2000: 80 million users decentralized P2P alterna ves – Napster iden fies online clients with the file – 2003: growth of pay – … and provides their IP addresses services like iTunes • Client requests the file from the chosen supplier – Supplier transmits the file to the client Shawn Fanning, – Both client and supplier report status to Napster Northeastern freshman 9 10
Napster Proper es Napster: Limita ons of Directory • Server’s directory con nually updated • File transfer is decentralized, but loca ng – Always know what music is currently available content is highly centralized – Point of vulnerability for legal ac on – Single point of failure – Performance bo leneck • Peer-to-peer file transfer – Copyright infringement – No load on the server – Plausible deniability for legal ac on (but not enough) • So, later P2P systems were more distributed • Bandwidth – Gnutella went to the other extreme… – Suppliers ranked by apparent bandwidth and response me
11 12
3 14 Peer-to-Peer Networks: Gnutella Gnutella: Search by Flooding • Gnutella history • Query flooding – 2000: J. Frankel & – Join: contact a few nodes T. Pepper released to become neighbors Gnutella – Publish: no need! – Soon a er: many other – Search: ask neighbors, who search clients (e.g., Morpheus, ask their neighbors xyz.mp3 Limewire, Bearshare) – Fetch: get file directly from xyz.mp3 ? – 2001: protocol another node enhancements, e.g., Flooding “ultrapeers”
13
15 16 Gnutella: Search by Flooding Gnutella: Search by Flooding
xyz.mp3 xyz.mp3 ? transfer Flooding search
4 18 Gnutella: Pros and Cons Lessons and Limita ons
• Advantages • Client-Server performs well – Fully decentralized – But not always feasible: Performance not o en key issue! – Search cost distributed For the following, you should choose a system that’s – Processing per node permits powerful search (A) Flood-based (B) DHT-based (C) Either (D) Neither seman cs – Organic scaling • Disadvantages – Decentraliza on of visibility and liability – Finding popular stuff – Search scope may be quite large – Finding unpopular stuff – Search me may be quite long – Fancy local queries – High overhead, and nodes come and go o en – Fancy distributed queries – Prevent data poisoning – Performance guarantees 17
19 20 Lessons and Limita ons Peer-to-Peer Networks: KaAzA
• Client-Server performs well • KaZaA history • Super-node hierarchy – But not always feasible: Performance not o en key issue! – 2001: created by Dutch – Join: on start, the client For the following, you should choose a system that’s company (Kazaa BV) contacts a super-node (A) Flood-based (B) DHT-based (C) Both (D) Neither – Single network called – Publish: client sends list – Organic scaling: C FastTrack used by other of files to its super-node – Decentraliza on of visibility and liability: C clients as well – Search: queries flooded – Finding popular stuff: A (maybe C) – Eventually protocol among super-nodes – Finding unpopular stuff: B changed so others – – Fetch: get file directly Fancy local queries: A could no longer use it – Fancy distributed queries: D from one or more peers – Prevent data poisoning: B (depends on query interface) – Performance guarantees: B 20
5 21 “Ultra/super peers” in KaZaA: Mo va on for Super-Nodes KaZaA and later Gnutella • Query consolida on – Many connected nodes may have only a few files – Propaga ng query to a sub-node may take more me than for the super-node to answer itself • Stability – Super-node selec on favors nodes with high up- me – How long you’ve been on is a good predictor of how long you’ll be around in the future
22
Peer-to-Peer Networks: BitTorrent BitTorrent: Simultaneous Downloads • BitTorrent history • Divide file into many chunks (e.g., 256 KB) – 2002: B. Cohen debuted BitTorrent – Replicate different chunks on different peers – Peers can trade chunks with other peers • Emphasis on efficient fetching, not searching – Peer can (hopefully) assemble the en re file – Distribute same file to many peers – Single publisher, many downloaders • Allows simultaneous downloading – Retrieving different chunks from different peers • Preven ng free-loading – And uploading chunks to peers – Incen ves for peers to contribute – Important for very large files
23 24
6 BitTorrent: Tracker BitTorrent: Overall Architecture
• Infrastructure node Web Server Tracker
Web page – Keeps track of peers par cipa ng in the torrent with link to .torrent – Peers registers with the tracker when it arrives • Tracker selects peers for downloading
.torrent – Returns a random set of peer IP addresses – So the new peer knows who to contact for data C A • Can have “trackerless” system Peer Peer [Seed] B – Using distributed hash tables (DHTs) [Leech] Downloader Peer
25 26 “US” [Leech]
BitTorrent: Overall Architecture BitTorrent: Overall Architecture
Web Server Tracker Web Server Tracker
Web page Web page with link with link to .torrent to .torrent
Get-announce
Response-peer list C C A A Peer Peer Peer [Seed] Peer [Seed] B B [Leech] [Leech] Downloader Peer Downloader Peer
27 “US” [Leech] 28 “US” [Leech]
7 BitTorrent: Overall Architecture BitTorrent: Overall Architecture
Web Server Tracker Web Server Tracker
Web page Web page with link with link to .torrent to .torrent
Shake-hand pieces C C A A pieces Shake-hand Peer Peer Peer [Seed] Peer [Seed] B B [Leech] [Leech] Downloader Peer Downloader Peer
29 “US” [Leech] 30 “US” [Leech]
BitTorrent: Overall Architecture BitTorrent: Overall Architecture
Web Server Tracker Web Server Tracker
Web page Web page with link with link to .torrent to .torrent
Get-announce
pieces Response-peer listpieces C C A pieces A pieces Peer Peer pieces pieces Peer [Seed] Peer [Seed] B B [Leech] [Leech] Downloader Peer Downloader Peer
31 “US” [Leech] 32 “US” [Leech]
8 BitTorrent: Chunk Request Order BitTorrent: Rarest Chunk First • Which chunks to request? • Which chunks to request first? – Could download in order – Chunk with fewest available copies (i.e., rarest chunk) – Like an HTTP client does • Benefits to the peer • Problem: many peers have the early chunks – Avoid starva on when some peers depart – Peers have li le to share with each other – Limi ng the scalability of the system • Benefits to the system • Problem: eventually nobody has rare chunks – Avoid starva on across all peers wan ng a file – E.g., the chunks need the end of the file – Balance load by equalizing # of copies of chunks – Limi ng the ability to complete a download • Solu ons: random selec on and rarest first
33 34
Free-Riding in P2P Networks Bit-Torrent: Preven ng Free-Riding • Vast majority of users are free-riders • Peer has limited upload bandwidth – Most share no files and answer no queries – And must share it among mul ple peers – Others limit # of connec ons or upload speed – Tit-for-tat: favor neighbors uploading at highest rate • • A few “peers” essen ally act as servers Rewarding the top four neighbors – Measure download bit rates from each neighbor – A few individuals contribu ng to the public good – Reciprocate by sending to the top four peers – Making them hubs that basically act as a server • Op mis c unchoking • BitTorrent prevent free riding – Randomly try a new neighbor every 30 seconds – Allow the fastest peers to download from you – So new neighbor has a chance to be a be er partner – Occasionally let some free loaders download
35 36
9 BitTyrant: Gaming BitTorrent Conclusions • BitTorrent can be gamed, too • Finding the appropriate peers – Peer uploads to top N peers at rate 1/N – Centralized directory (Napster) – E.g., if N=4 and peers upload at 15, 12, 10, 9, 8, 3 – Query flooding (Gnutella) – … peer uploading at rate 9 gets treated quite well – Super-nodes (KaZaA) • Best to be the Nth peer in the list, rather than 1st • BitTorrent – Offer just a bit more bandwidth than low-rate peers – Distributed download of large files – And you’ll s ll be treated well by others – An -free-riding techniques
• BitTyrant so ware h p://bi yrant.cs.washington.edu/ • Great example of how change can happen so – Uploads at higher rates to higher-bandwidth peers quickly in applica on-level protocols
37 38
10