P2P Content Distribution BitTorrent and Spotify Amir H. Payberah [email protected] Amirkabir University of Technology (Tehran Polytechnic) Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 1 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 2 / 62 Possible Solutions for Content Distribution Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 3 / 62 Client-Server Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 4 / 62 Client-Server Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 5 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 6 / 62 I Single Point of failure? The Client-Server Model Problems I Scalability? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62 The Client-Server Model Problems I Scalability? I Single Point of failure? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62 The Client-Server Model Problems I Scalability? I Single Point of failure? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62 Client-Server Systems Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 8 / 62 The Client-Server Model Problem Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 9 / 62 Scalable and Fault-Tolerant Client-Server Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 10 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 11 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 12 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 13 / 62 Peer-to-Peer Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 14 / 62 Peer-to-Peer (P2P) Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 15 / 62 P2P Challenges I Churn in the system I Free-riding problem I Bottleneck in the overlay network I Connectivity problem, e.g., NAT Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 16 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 17 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 18 / 62 How To Discover Data? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 19 / 62 Possible Solutions - First Generation I Central directory Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 20 / 62 Possible Solutions - Second Generation I Flooding Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 21 / 62 Possible Solutions - Third Generation I Distributed Hash Table (DHT) Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 22 / 62 P2P Content Distribution Applications - File Sharing Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 23 / 62 P2P Content Distribution Applications - Media Streaming Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 24 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 25 / 62 BitTorrent I BitTorrent is a system for efficient and scalable replication of large amounts of static data. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 26 / 62 BitTorrent Players Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 27 / 62 Files I Files are broken into pieces of size between 64KB and 1MB. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 28 / 62 .torrent Files I Metadata I Contains: • URL of tracker • Information about the file, e.g., filename, length, ... Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 29 / 62 I It, then, connects to the tracker. I The tracker tells the peers from which other peers to download the pieces of the file. I Peers use this information to communicate with each other. I The peers send information about the file and themselves to tracker. The Core Idea I A peer obtains .torrent file. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62 I The tracker tells the peers from which other peers to download the pieces of the file. I Peers use this information to communicate with each other. I The peers send information about the file and themselves to tracker. The Core Idea I A peer obtains .torrent file. I It, then, connects to the tracker. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62 I Peers use this information to communicate with each other. I The peers send information about the file and themselves to tracker. The Core Idea I A peer obtains .torrent file. I It, then, connects to the tracker. I The tracker tells the peers from which other peers to download the pieces of the file. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62 I The peers send information about the file and themselves to tracker. The Core Idea I A peer obtains .torrent file. I It, then, connects to the tracker. I The tracker tells the peers from which other peers to download the pieces of the file. I Peers use this information to communicate with each other. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62 The Core Idea I A peer obtains .torrent file. I It, then, connects to the tracker. I The tracker tells the peers from which other peers to download the pieces of the file. I Peers use this information to communicate with each other. I The peers send information about the file and themselves to tracker. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 31 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 32 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 33 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 34 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 35 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 36 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 37 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 38 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 39 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 40 / 62 What About Free Riders? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 41 / 62 Tit-For-Tat Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 42 / 62 Question I From which peers download the pieces? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 43 / 62 Peer Selection I Use choking algorithm to choose peer to download pieces. I Decision to choke/unchoke based on tit-for-tat. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 44 / 62 Discover More Cooperating Peers I Optimistic unchoking I Allocate an upload slot to a randomly chosen uncooperative peer Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 45 / 62 Snubbed Peers I If all its peers choke it. I Increase the number of optimistic unchokes. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 46 / 62 Question I Which piece? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 47 / 62 I Random first piece: start-up need to get a complete piece I Endgame mode: broadcast for all remaining blocks Piece Selection I Rarest first: common parts left for later Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62 I Endgame mode: broadcast for all remaining blocks Piece Selection I Rarest first: common parts left for later I Random first piece: start-up need to get a complete piece Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62 Piece Selection I Rarest first: common parts left for later I Random first piece: start-up need to get a complete piece I Endgame mode: broadcast for all remaining blocks Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62 BitTorrent Extension I Distributed tracker I Peer-exchange Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 49 / 62 Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 50 / 62 Spotify I Active users: over 50 million I Number of songs: over 20 million I Number of songs added per day: over 20000 I Number of playlists: over 1.5 billion created so far I Available in 58 countries I Legal Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 51 / 62 I Meanwhile, search P2P network for remainder. I Switch back and forth between Spotify servers and peers as needed. I Towards end of a track, start prefetching the next one. The Core Idea I Request first piece from Spotify servers. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62 I Switch back and forth between Spotify servers and peers as needed. I Towards end of a track, start prefetching the next one. The Core Idea I Request first piece from Spotify servers. I Meanwhile, search P2P network for remainder. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62 I Towards end of a track, start prefetching the next one. The Core Idea I Request first piece from Spotify servers. I Meanwhile, search P2P network for remainder. I Switch back and forth between Spotify servers and peers as needed.