SECOND QUARTER 2005, VOLUME 7, NO. 2 IEEE C OMMUNICATIONS SURVEYS T he Electronic Magazine of O riginal Peer-Reviewed Survey Articles www.comsoc.org/pubs/surveys A SURVEY AND COMPARISON OF PEER-TO-PEER OVERLAY NETWORK SCHEMES ENG KEONG LUA, JON CROWCROFT, AND MARCELO PIAS, UNIVERSITY OF CAMBRIDGE RAVI SHARMA, NANYANG TECHNOLOGICAL UNIVERSITY STEVEN LIM, MICROSOFT ASIA ABSTRACT Over the Internet today, computing and communications environments are significantly more complex and chaotic than classical distributed systems, lacking any centralized organization or hierarchical control. There has been much interest in emerging Peer-to-Peer (P2P) network overlays because they provide a good substrate for creating large-scale data sharing, content distribution, and application-level multicast applications. These P2P overlay networks attempt to provide a long list of features, such as: selection of nearby peers, redundant storage, efficient search/loca- tion of data items, data permanence or guarantees, hierarchical naming, trust and authentication, and anonymity. P2P networks potentially offer an efficient routing architecture that is self-organizing, massively scalable, and robust in the wide-area, combining fault tolerance, load balancing, and explicit notion of locality. In this article we present a survey and comparison of various Structured and Unstructured P2P overlay networks. We categorize the various schemes into these two groups in the design spectrum, and discuss the application-level network performance of each group. eer-to-peer (P2P) overlay networks are distributed sys- The Network Communications layer describes the network tems in nature, without any hierarchical organization characteristics of desktop machines connected over the Inter- or centralized control. Peers form self-organizing over- net or small wireless or sensor-based devices that are connect- Play networks that are overlayed on the Internet Protocol (IP) ed in an ad-hoc manner. The dynamic nature of peers poses networks, offering a mix of various features such as robust challenges in the communication paradigm. The Overlay wide-area routing architecture, efficient search of data items, Nodes Management layer covers the management of peers, selection of nearby peers, redundant storage, permanence, which include discovery of peers and routing algorithms for hierarchical naming, trust and authentication, anonymity, mas- optimization. The Features Management layer deals with the sive scalability, and fault tolerance. Peer-to-peer overlay sys- security, reliability, fault resiliency, and aggregated resource tems go beyond services offered by client-server systems by availability aspects of maintaining the robustness of P2P sys- having symmetry in roles where a client may also be a server. tems. The Services Specific layer supports the underlying P2P It allows access to its resources by other systems and supports infrastructure and the application-specific components resource sharing, which requires fault-tolerance, self-organiza- through scheduling of parallel and computation-intensive tion, and massive scalability properties. Unlike Grid systems, tasks, content and file management. Meta-data describes the P2P overlay networks do not arise from the collaboration content stored across the P2P peers and the location informa- between established and connected groups of systems and tion. The Application-level layer is concerned with tools, without a more reliable set of resources to share. applications, and services that are implemented with specific We can view P2P overlay network models spanning a wide functionalities on top of the underlying P2P overlay infra- spectrum of the communication framework, which specifies a structure. Thus, there are two classes of P2P overlay networks: fully-distributed, cooperative network design with peers build- Structured and Unstructured. ing a self-organizing system. Figure 1 shows an abstract P2P The technical meaning of structured is that the P2P overlay overlay architecture, illustrating the components in the over- network topology is tightly controlled and content is placed lay not at random peers but at specified locations that will make 72 1553-877X IEEE Communications Surveys & Tutorials • Second Quarter 2005 Applications ToolsServices Application-level layer In 1999 Napster [8] pioneered the idea of a peer-to-peer file sharing system supporting a Services centralized file search facility. It was the first sys- management Meta-data Services messaging Services-specific layer tem to recognize that requests for popular con- Services scheduling tent need not be sent to a central server but Security Resource instead could be handled by many peers that management management Features have the requested content. Such P2P file-shar- Reliability and managment layer fault resiliency ing systems are self-scaling in that as more peers Routing and join the system, they add to the aggregate down- location lookup Overlay nodes load capability. Napster achieved this self-scaling Resources management layer behavior by using a centralized search facility discovery based on file lists provided by each peer; thus, it Network Network does not require much bandwidth for the central- communications ized search. Such a system has the issue of a sin- layer gle point of failure due to the centralized search mechanism. However, a lawsuit filed by the n Figure 1. An abstract P2P overlay network architecture. Recording Industry Association of America (RIAA) attempted to force Napster to shut down the free-for-all file-sharing service for copyright- subsequent queries more efficient. Such structured P2P sys- ed digital music — literally, its killer application. However, this tems use the Distributed Hash Table (DHT) as a substrate, in paradigm caught the imagination of platform providers and which data object (or value) location information is placed users alike. Gnutella [9–11] is a decentralized system that dis- deterministically, at the peers with identifiers corresponding tributes both the search and download capabilities, establish- to the data object’s unique key. DHT-based systems have a ing an overlay network of peers. It is the first system that property that consistently assigned uniform random NodeIDs makes use of an unstructured P2P overlay network. An to the set of peers into a large space of identifiers. Data unstructured P2P system is composed of peers joining the net- objects are assigned unique identifiers called keys, chosen work with some loose rules, without any prior knowledge of from the same identifier space. Keys are mapped by the over- the topology. The network uses flooding as the mechanism to lay network protocol to a unique live peer in the overlay net- send queries across the overlay with a limited scope. When a work. The P2P overlay networks support the scalable storage peer receives the flood query, it sends a list of all content and retrieval of {key,value} pairs on the overlay network, as matching the query to the originating peer. While flooding- illustrated in Fig. 2. Given a key, a store operation based techniques are effective for locating highly replicated (put(key,value)) lookup retrieval operation (value=get(key)) items and are resilient to peers joining and leaving the system, can be invoked to store and retrieve the data object corre- they are poorly suited for locating rare items. Clearly this sponding to the key, which involves routing requests to the approach is not scalable as the load on each peer grows lin- peer corresponding to the key. early with the total number of queries and the system size. Each peer maintains a small routing table consisting of its Thus, unstructured P2P networks face one basic problem: neighboring peers’ NodeIDs and IP addresses. Lookup peers readily become overloaded, and thus the system does queries or message routing are forwarded across overlay paths not scale when handling a high rate of aggregate queries and to peers in a progressive manner, with the NodeIDs that are sudden increases in system size. closer to the key in the identifier space. Different DHT-based Although structured P2P networks can efficiently locate systems will have different organization schemes for the data rare items since the key-based routing is scalable, they incur objects and its key space and routing strategies. In theory, significantly higher overheads than unstructured P2P networks DHT-based systems can guarantee that any data object can be for popular content. Consequently, over the Internet today located in small O(logN) overlay hops on average, where N is the decentralized unstructured P2P overlay networks are more the number of peers in the system. The underly- ing network path between two peers can be sig- nificantly different from the path on the DHT-based overlay network. Therefore, the Distributed structured P2P overlay application lookup latency in DHT-based P2P overlay net- works can be quite high and could adversely affect the performance of the applications run- ning over it. Plaxton et al. [1] provide an elegant API interface: API interface: API interface: Value algorithm that achieves nearly optimal latency on Put(Key, Value) Remove(Key) Value=Get(Key) graphs that exhibit power-law expansion [2], at the same time preserving the scalable routing properties of the DHT-based system. However, Distributed hash table this algorithm requires pair-wise probing between peers to determine latencies, and it is unlikely to scale to a large number of peers in the overlay. DHT-based systems