Protocols and System Design, Reliability, and Energy E±ciency in Peer-to-Peer Communication Systems Salman Abdul Baset Submitted in partial ful¯llment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2011 °c 2011 Salman Abdul Baset All Rights Reserved ABSTRACT Protocols and System Design, Reliability, and Energy E±ciency in Peer-to-Peer Communication Systems Salman Abdul Baset Modern Voice-over-IP (VoIP) communication systems provide a bundle of services to their users. These services range from the most basic voice-based services such as voice calls and voicemail to more advanced ones such as conferencing, voicemail-to-text, and online address books. Besides voice, modern VoIP systems provide video calls and video conferencing, presence, instant messaging (IM), and even desktop sharing services. These systems also let their users establish a voice, video, or a text session with devices in cellular, public switched telephone network (PSTN), or other VoIP networks. The peer-to-peer (p2p) paradigm for building VoIP systems involves minimal or no use of managed servers and is therefore attractive from an administrative and economic per- spective. However, the bene¯ts of using p2p paradigm in VoIP systems are not without their challenges. First, p2p communication (VoIP) systems can be deployed in environ- ments with varying requirements of scalability, connectivity, security, interoperability, and performance. These requirements bring forth the question of designing open and standard- ized protocols for diverse deployments. Second, the presence of restrictive network address translators (NATs) and ¯rewalls prevents machines from directly exchanging packets and is problematic from the perspective of establishing direct media sessions. The p2p com- munication systems address this problem by using an intermediate peer with unrestricted connectivity to relay the session or by preferring the use of TCP. This technique for address- ing connectivity problems raises questions about the reliability and session quality of p2p communication systems compared with the traditional client-server VoIP systems. Third, while administrative overheads are likely to be lower in running p2p communication sys- tems as compared to client-server, can the same be said about the energy e±ciency? Fourth, what type of techniques can be used to gain insights into the performance of a deployed p2p VoIP system like Skype? The thesis addresses the challenges in designing, building, and analyzing peer-to-peer communication systems. The thesis presents Peer-to-Peer Protocol (P2PP), an open proto- col for building p2p communication systems with varying operational requirements. P2PP is now part of the IETF's P2PSIP protocol and is on track to become an RFC. The thesis de- scribes the design and implementation of OpenVoIP, a proof-of-concept p2p communication system to demonstrate the feasibility of P2PP and to explore issues in building p2p commu- nication systems. The thesis introduces a simple and novel analytical model for analyzing the reliability of peer-to-peer communication systems and analyzes the feasibility of TCP for sending real-time tra±c. The thesis then analyzes the energy e±ciency of peer-to-peer and client-server VoIP systems and shows that p2p VoIP systems are less energy e±cient than client-server even if the peers consume a small amount of energy for running the p2p network. Finally, the thesis presents an analysis of the Skype protocol which indicates that Skype is free-riding on the network bandwidth of universities. Table of Contents 1 Introduction 1 1.1 Problems and Challenges ............................ 4 1.2 Overview and Contributions of the Thesis ................... 5 1.3 Related Work ................................... 7 2 De¯nitions and Background 8 2.1 De¯nitions ..................................... 8 2.2 Network Address Translators (NATs) ...................... 9 2.2.1 Mapping Behavior of NATs ....................... 10 2.2.2 Filtering Behavior of NATs ....................... 11 2.2.3 Typical NAT Behavior .......................... 11 2.2.4 Protocols for Traversing NATs ..................... 12 2.3 Percentage of VoIP Calls in the Internet Requiring a Relay . 12 2.4 Overlay Algorithms ................................ 13 2.4.1 Distributed Hash Tables (DHTs) .................... 13 I Protocol and System Design 15 3 Protocols for Building Peer-to-Peer Communication Systems 16 3.1 Introduction .................................... 16 3.2 Requirements ................................... 18 3.3 Peer-to-Peer Protocol (P2PP) .......................... 22 3.4 Design Overview of P2PP ............................ 22 i 3.4.1 Node Model ................................ 22 3.4.1.1 P2PP Node Stack ....................... 24 3.4.2 Data Model ................................ 27 3.4.3 Message Model .............................. 28 3.4.4 Reliability Model ............................. 29 3.4.5 Security Model .............................. 30 3.4.6 NAT and Firewall Traversal ....................... 30 3.5 Key Components of a P2PP Message ...................... 31 3.5.1 Common Header ............................. 31 3.5.2 General Object Format ......................... 33 3.5.3 Node-Info ................................. 33 3.5.4 Resource-Object ............................. 34 3.6 Overview of P2PP Methods ........................... 35 3.6.1 Enrollment and Bootstrap ........................ 35 3.6.1.1 Enroll .............................. 35 3.6.1.2 Bootstrap ............................ 37 3.6.2 Overlay Maintenance ........................... 38 3.6.2.1 Join ............................... 39 3.6.2.2 Leave .............................. 42 3.6.2.3 ExchangeTable ......................... 42 3.6.2.4 LookupPeer ........................... 43 3.6.2.5 KeepAlive ........................... 44 3.6.2.6 Invite .............................. 44 3.6.3 Data Storage ............................... 45 3.6.3.1 PublishObject ......................... 45 3.6.3.2 LookupObject ......................... 46 3.6.3.3 TransferObject ......................... 47 3.6.3.4 ReplicateObject ........................ 47 3.6.4 Connection Management ........................ 48 3.6.4.1 Tunnel ............................. 48 ii 3.6.5 Monitoring and Bandwidth Measurement . 48 3.6.5.1 GetDiagnostics ......................... 48 3.6.5.2 MeasureBandwidth ...................... 49 3.6.6 Response and Error Codes ........................ 50 3.6.6.1 2xx (Successful) Responses . 50 3.6.6.2 3xx (Redirect) Responses ................... 50 3.6.6.3 4xx (Failure) Responses .................... 50 3.7 Related Work ................................... 51 3.8 Conclusion .................................... 53 4 OpenVoIP { A Peer-to-Peer Communication System 54 4.1 Introduction .................................... 54 4.2 System Architecture ............................... 55 4.2.1 Resource-Objects for Operation ..................... 56 4.2.1.1 SIP-CONTACT ........................ 57 4.2.1.2 STUN-TURN ......................... 58 4.3 System Functionality ............................... 58 4.3.1 Bootstrap ................................. 58 4.3.2 Joining the Overlay ........................... 60 4.3.2.1 Peer Join ............................ 60 4.3.2.2 Client Join ........................... 61 4.3.3 Call Establishment ............................ 62 4.3.4 Distributed Relay Search ........................ 62 4.3.4.1 Link Capacity Measurements . 63 4.3.5 Monitoring and Diagnostics ....................... 64 4.4 Lessons Learned ................................. 65 4.5 Related Work ................................... 66 4.6 Conclusion .................................... 66 5 Peer-to-Peer Multiparty Video Conferencing 68 5.1 Introduction .................................... 68 iii 5.2 Helper Bandwidth Usage in Skype and ooVoo . 70 5.3 Practical Issues in Application Layer Multicast Tree (ALMC) Construction 71 5.3.1 How Many Helpers? ........................... 72 5.3.2 Reliability ................................. 74 5.3.3 Helper and Participant State, and Latency of a Video Stream . 76 5.3.4 Always Minimize Helpers? ........................ 77 5.3.5 Select Helper Close to the Source or Recipient? . 77 5.3.6 Transcoding a Video Stream? ...................... 77 5.3.7 Sharing Helpers Across ALMC Trees? . 78 5.3.8 Participants as Helpers? ......................... 78 5.3.9 Managed Helpers ............................. 79 5.4 Putting it All Together .............................. 79 5.4.1 Use Split Approach with Multiple Description Coding . 80 5.4.2 Select Helpers Close to the Video Source . 80 5.4.3 Prioritize Voice over Video ....................... 80 5.4.4 Limit Bandwidth Usage at Helpers ................... 80 5.4.5 Mitigate Helper Churn .......................... 81 5.4.6 Use IP Multicast Where Possible .................... 81 5.5 Related Work ................................... 81 5.6 Conclusion .................................... 83 II Analysis 84 6 Reliability and Relay Selection 85 6.1 Introduction .................................... 85 6.2 Problem Setting ................................. 87 6.3 Reliability of a P2P Communication System . 88 6.4 Modeling the Reliability of Relayed Calls ................... 89 6.4.1 Number of Relays ............................ 89 6.4.1.1 Pareto Node Lifetimes ..................... 92 iv 6.4.2 No-replacement Scheme ......................... 93 6.4.3 With-replacement Scheme .......................