Scalable Supernode Selection in Peer-To-Peer Overlay Networks∗

Scalable Supernode Selection in Peer-to-Peer Overlay Networks∗

Virginia Lo, Dayi Zhou, Yuhong Liu, Chris GauthierDickey, and Jun Li {lo|dayizhou|liuyh|chrisg|lijun} @cs.uoregon.edu Network Research Group – University of Oregon

Abstract ically fulfill additional requirements such as load balance, resources, access, and fault tolerance. We define a problem called the supernode selection The supernode selection problem is highly challeng- problem which has emerged across a variety of peer-to- ing because in the peer-to-peer environment, a large peer applications. Supernode selection involves selec- number of supernodes must be selected from a huge tion of a subset of the peers to serve a special role. The and dynamically changing network in which neither the supernodes must be well-dispersed throughout the peer- node characteristics nor the network topology are known to-peer overlay network, and must fulfill additional re- a priori. Thus, simple strategies such as random selec- quirements such as load balance, resource needs, adapt- tion don’t work. Supernode selection is more complex ability to churn, and heterogeneity. While similar to than classic dominating set and p-centers from graph dominating set and p-centers problems, the supernode theory, known to be NP-hard problems, because it must selection problem must meet the additional challenge respond to dynamic joins and leaves (churn), operate of operating within a huge, unknown and dynamically among potentially malicious nodes, and function in an changing network. We describe three generic super- environment that is highly heterogeneous. node selection protocols we have developed for peer-to- Supernode selection shows up in many peer-to-peer peer environments: a label-based scheme for structured and networking applications. For example, in peer- overlay networks, a distributed protocol for coordinate- to-peer file sharing systems, such as Kazaa [11] and based overlay networks, and a negotiation protocol for Gnutella [6], protocols were developed for the designa- unstructured overlays. We believe an integrated ap- tion of qualified supernodes (ultrapeers) to serve the or- proach to the supernode selection problem can benefit dinary peers for scalable content discovery. Peer-to-peer the peer-to-peer community through cross-fertilization infrastructure services require methods for the judicious of ideas and sharing of protocols. placement of monitors/landmarks/beacons throughout the physical network for purposes of measurement, po- sitioning, or routing [5, 16, 14]. In ad-hoc wireless net- 1. Introduction works, connectivity under highly dynamic conditions is achieved by identifying a subset of the nodes to serve as bridging nodes. This subset is formed using a dis- We have identified a fundamental problem, which tributed dominating set protocol such as in [22, 2, 23] so we call the supernode selection problem, which occurs that every node is within broadcast range of a bridging across a spectrum of peer-to-peer applications, includ- node. Within sensor networks, supernodes are selected ing file sharing systems, distributed hash tables (DHTs), for the purpose of data aggregation under the conditions publish/subscribe architectures, and peer-to-peer cycle that they are well-distributed among the sensors and also sharing systems. The supernode selection problem also have sufficient remaining battery life [3]. shows up in the fields of sensor networks, ad-hoc wireless networks, and peer-based Grid computing. Super- In this paper, we demonstrate that many peer-to-peer node selection involves the selection of a subset of the and networking applications are seeking solutions to peers in a large scale, dynamic network to serve a dis- variations on the same fundamental problem. We first tinguished role. The specially selected peers must be define a general model for the supernode selection prob- well-dispersed throughout the network, and must typ- lem, describing its key requirements and challenges. We show how the supernode selection problem is related ∗Supported in part by NSF 9977524 and an NSF Graduate Re- to dominating set and p-centers problems, and describe search Fellowship how the supernode selection problem is instantiated in several well-known peer-to-peer applications. We then Supernode Distribution Criteria. The supernodes present three general purpose supernode selection proto- must be distributed throughout the peer-to-peer overlay cols we have developed for the three basic types of over- network in a topologically sensitive way to meet one or lay networks used by peer-to-peer applications: struc- more of the distribution criteria listed below. tured overlay networks, coordinate-based overlay net- Access: non-supernodes must have low latency ac- works, and unstructured overlay networks. cess to one or more supernodes. Access can be measured SOLE: a label-based supernode selection protocol in hop counts or delay. for applications built on a structured overlay network Dispersal: supernodes must be evenly distributed such as CAN [16], Chord [20] or Pastry [18]. SOLE throughout the overlay network; they should not be clus- exploits the regular node labeling schemes used within tered within only a few subregions of the overlay. structured overlays to distribute supernodes evenly Proportion: a pre-specified global ratio of super- throughout the overlay, to ensure fast access from non- nodes to non-supernodes must be maintained to meet supernodes to supernodes, and to maintain a fixed sup- application-specific performance requirements. ernode to non-supernode ratio as the overlay grows and Load balance: supernodes should not serve more shrinks dynamically. than k non-supernodes, where k can be configured lo- PoPCorn: a protocol for applications that use a cally based on the resource capability of each supernode. coordinate-based overlay network using systems such as Note that these criteria are inter-related in that speci- Vivaldi [1] or GNP [14]. This protocol distributes a fixed fication of requirements for one may impact another, or number of supernodes evenly with respect to the topol- a given application may require multiple criteria. ogy of the overlay network using a repulsion model. Peer-to-Peer Factors. The design of supernode selec- H20: an advertisement-based protocol deployable in tion protocols within a peer-to-peer environment is chal- an unstructured overlay network. This protocol can be lenging because in addition to fulfilling the distribution used to find qualified supernodes from among the peers requirements outlined above, they must also deal with such that every ordinary peer is within k hops of C sup- factors arising from the underlying nature of large scale, ernodes (multiple distance domination). highly dynamic systems. Heterogeneity: Current large-scale peer-based sys- 2. Problem Definition and Background tems consist of a large number of heterogeneous nodes with differing hardware and software resources. A node We first describe the unique requirements and chal- might not be eligible to be a supernode unless it meets lenges of the supernode selection problem in the context certain minimum qualifications. These qualifications in- of peer-to-peer systems. We then describe dominating clude resources, such as CPU power, disk or memory set and p-centers problems from graph theory that form space, or battery life; stability, such as uptime or fault the theoretical underpinnings of the supernode selection tolerance; communication, such as bandwidth or fan- problem. We conclude by surveying examples of appli- out; and safety, such as trust or security. cations from peer-to-peer computing and other network- Adaptability to churn: Peer-to-peer environments are ing domains that call for scalable supernode selection extremely dynamic. Supernode selection protocols must protocols. be able to handle churn and respond quickly, especially when supernodes leave the system. Supernode selection 2.1. The Supernode Selection Problem protocols must also be adaptive to dynamic changes in network traffic and overlay topology. We broadly define the supernode selection problem Resilience and fault tolerance: When a given super- as that of selecting some subset of the peers in a large node dies, other supernodes should quickly take over its scale peer-to-peer overlay network to take a special role, functions or a new supernode should be quickly selected. with the designated supernodes providing service to Security: Supernodes may be vulnerable to denial the non-supernodes. A potentially large and undefined of service attacks, malicious supernodes can disrupt the number of supernodes must be selected from an un- system by failing to forward messages or by giving out known, large scale, and dynamically changing overlay wrong information. network. First, we describe key topological distribution criteria that the supernodes must fulfill relative to the 2.2. Dominating sets, p-centers, and leader elec- non-supernodes. Second, we enumerate characteristics tion of peer-to-peer networks that make supernode selection difficult. A wealth of research in graph theory, location theory, and distributed computing provides a formal foundation for the supernode selection problem. These algorithms were not designed for large scale peer- The basic dominating set problem is the problem of to-peer networks that exhibit a high degree of churn and finding a minimal subset of the vertices in graph G, that are dynamically heterogeneous. called the dominator set, such that every node is either a dominator or adjacent to a dominator. Dominating 2.3. Ultrapeers, landmarks, and rendezvous set problems and algorithms are described thoroughly points in [8]. Most versions are NP-hard. Distance domination seeks to find a minimum size d- The best known example of supernode selection in a dominating set such that the distance from an arbitrary peer-to-peer application is the gnutella protocol [6] for ≤ node to a dominator is d. Multiple (c,d)-domination selection of ultrapeers—peers with sufficient bandwidth requires that every peer be within distance d of c domi- and processing power to serve as proxies for other peers. nators. Colored domination presumes that each node in The use of ultrapeers reduces network traffic and speeds graph G has an associated color from the set c1,c2, ...cn. up content discovery. A dominating set of color ci is one in which the domina- Any peer can select itself as an ultrapeer if it meets tors are all of that color. Colored domination can be used the following criteria: it has been up for at least 5 min- to model heterogeneous networks in which only certain utes, has high bandwidth, sufficient processing power, nodes are qualified to be dominators. runs an OS that can handle a large number of simul- A secure dominating set is a subset of vertices S such taneous TCP connections, and is not firewalled/NATed. that for any vertex v not in S there exists a neighbor u The ultrapeer selection protocol dynamically adjusts the of v in S such that, if we add v to S and remove u from number of supernodes as follows: if a leaf cannot find S, we get another secure dominating set S.Aglobal de- an ultrapeer with free slots, it can promote itself to be an fensive alliance is a variant of dominating set where the ultrapeer. Ultrapeers also downgrade themselves when set S is such that every vertex v not in S has a neighbor they no longer serve any leaf nodes, or through negotia- in S and every vertex v in S has a majority of its neigh- tion with nearby peers. Gnutella’s supernode selection bors in S.Aglobal offensive alliance is such that every (ultrapeer selection) protocol loosely meets the distri- vertex not in S has a majority of its neighbors in S.A bution criteria defined above and adapts dynamically to k-defensive dominating set is a set of vertices that can users’ needs in a best effort fashion. One of gnutella’s counter an attack on any k vertices where an attack on a goals is to achieve a certain ratio of ultrapeers to leaf vertex must be countered by itself or by a neighbor. nodes, but currently there is no way to control this ratio. The p-center problem is applicable when placing a Security is not addressed in gnutella. fixed number of supernodes in a network. Algorithms A canonical supernode selection problem that has and variations on this NP-hard discrete location prob- shown up as a bootstrapping step in a number of promi- lem are found in [7]. The p-center problem is the prob- nent peer-to-peer systems is the designation of landmark lem of finding a subset of p vertices in a graph G, called nodes in the Internet. The CAN [17] structured over- centers, to minimize the maximum (or total) distance be- lay is constructed using a binning technique in which tween a non-center node and its nearest center. Colored carefully placed landmarks are used to construct a topo- p-centers can be used in a colored graph for the problem logically sensitive overlay. Also, in GNP [14], a co- of finding a subset of pi vertices of color ci to minimize ordinate system is built using landmark nodes. It has the above distance criteria. been shown that the accuracy of the GNP coordinates is The classic leader election problem from distributed highly sensitive to the location of the landmark nodes. computing differs from supernode selection in that the Choosing landmark nodes boils down to supernode se- former assumes all nodes vote (directly or indirectly) lection for a fixed, small number of supernodes that are on the choice of each supernode. Leader election al- well-dispersed in the physical network. The notion of gorithms are not scalable because they require broad- evenly distributed or well-dispersed can be captured by casting or passing a token to all nodes. The best known metrics based on pairwise distances, N-medians and N- leader election protocols electing a leader (typically the cluster medians [14], or distances from a centroid. Cur- node with highest ID number) under various fault toler- rent approaches use manual placement of landmarks or ant scenarios, such as Ring, Bully, etc. [13]. centralized dominating set algorithms [21]. Heuristic algorithms developed for these classic Examples from outside peer-to-peer computing illus- problems have been utilized in the field of networking, trate the wide scope of the supernode selection problem. but their applicability is usually limited to smaller scale, In ad-hoc wireless networks, a distributed dominating static networks. For the most part, they involve cen- set protocol is used to select nodes to serve as interme- tralized algorithms or high message passing overhead. diary routing nodes to bridge the distance between wireless nodes that are out of broadcast range of each other not be qualify to be supernodes and may be poorly dis- [22, 2]. The IDMaps [5] distance map of the Internet persed. Flooding-based search for a large number of used a centralized p-centers algorithm to place Tracer supernodes is not scalable and does not adapt well to monitors and Boxes throughout a known network topol- dynamic changes in the network. ogy. Sensor networks utilize specially placed monitor For these reasons, we introduce three new supernode nodes as rendezvous points to collect and aggregate sen- selection protocols for the three basic types of over- sor data [3] under access and battery life constraints. lay networks: structured, coordinate-based, and unstruc- Table 1 lists a few examples of the supernode selec- tured. These protocols take advantage of properties of tion problem for a range of applications, showing the the respective overlay types to better address supernode commonality and diversity in supernode qualifications selection. Performance evaluation of these protocols is and supernode distribution requirements. part of our ongoing research agenda, but outside scope of this paper. Projects/papers Supernode Distribution Qualifications Criteria 3.1. SOLE: Supernode selection in structured Distance Estimation overlay networks Inter-landmark CAN[17], distance,N- GNP[14], Minimal medians,N-cluster- SOLE is designed to select a group of supernodes in IDMaps median, Fixed # of a structured overlay network, with a goal of keeping the (Tracer/Box)[5] landmarks supernode to non-supernode ratio stable as peers join Content Location and leave the overlay. SOLE also maintains low access Operating from non-supernodes to supernodes and provides load system, Maintain # of ultra- Gnutella[6], balancing for each supernode. In the last part of this Bandwidth, peer in proportion to Kazaa[11] Uptime, leaf nodes section, we discuss how SOLE addresses resource het- Memory/CPU erogeneity, churn, and fault tolerance. Routing A DHT (Distributed Hash Table) built on a struc- Highly Close to network ac- tured overlay network such as CAN [16], Chord [20], Expressway[10], available, cess points, k hops to and Pastry [18] makes use of a symmetric, regular node Sequoia Fan-out, c supernodes, Disjoint label space, in which each physical node owns a virtual (router)[9] Bandwidth, paths subspace in the overlay. In these structured overlay net- Trust Data Aggregation works, a compact node label expression can encode a Sensor Battery Within signal radius (large) collection of virtual nodes. networks[3] life SOLE exploits this notation and uses a node label ex- Rendezvous Point pression to designate a subset of the virtual node label Bandwidth, Easily accessible, Sequoia space as supernodes. The supernode label expression is Uptime, Distribution fits (RP)[9], Memory/CPU, density of non- stored in the DHT for fast and easy lookup. The number Ferreira[4] Trust supernodes of supernodes can be expanded simply by changing the Monitors (point-of-presence) node label expression (see examples below). Because a IDMaps Fixed # of monitors, structured overlay network maps physical nodes to vir- [15], Minimal Sensitive to network tual subspaces in a manner that is sensitive to both den- NETI@home[19] topology sity and topology, the supernodes are evenly distributed among physical non-supernode nodes and every non- Table 1. Supernode Selection Examples supernode has one or more nearby supernodes. Initiation of supernode selection. A node initiates the supernode selection procedure for some service by hash- ing information about the service into the DHT: the pub- 3. Three New Supernode Selection Protocols lic key of the initiator and the supernode selection policy. This policy contains the supernode label expression, Heuristic algorithms for classical problems such as the minimum criteria for a node to be a supernode, and dominating set and p-center cannot be applied to large maximum lifetime of a supernode. A non-supernode can scale, dynamic peer-to-peer environments. Standard discover the identity of supernodes for this service by ac- networking techniques such as random selection of cessing the DHT using the service related key to lookup peers, or flooding-based search for suitable supernodes the supernode label expression. do not work. Nodes selected by a random strategy may Example 1: CAN structured overlay. Assume the CAN space is a d-dimensional space and the length of then figure out the nearest supernode in the virtual the ith dimension is di. To select k supernodes, the ser- overlay according to the label-based routing protocol. vice initiator first factors k into k1,k2,k3,...,kd where The structured overlay network provides bounded d k = i=1 ki.Theith dimension of the supernode label virtual routing with path length proportional to the should obey the formula (bi + n ∗ di/ki)%di, where bi distance between labels in the label space. In CAN the is a random number chosen in the range of [0,di] and n non-supernode uses the Cartesian distance between its is an integer. The randomness prevents different service own label and the supernode label to estimate distance. initiators from choosing the same group of supernodes. In Pastry, the non-supernode uses the bit-wise XOR Figure 1 illustrates this process for 16 supernodes in a operation to compute the label distance from which it two-dimensional CAN space. estimates the physical distance.

(0.05, 0.95) (0.3, 0.95) (0.55, 0.95) (0.8, 0.95) Virtual Supernode Label Supernode Resilience and fault tolerance. SOLE must function in Non−supernode situations in which supernodes depart gracefully or die (0.05, 0.7) (0.3, 0.7) (0.55, 0.7) (0.8, 0.7) suddenly. A supernode can replicate supernode-related state on the neighbor nodes that will take over the subspace covered by its label if it fails. Supernode fail- (0.05, 0.45) (0.3, 0.45) (0.55, 0.45) (0.8, 0.45) ure will be detected by the underlying overlay network maintenance protocol. The most capable neighbor of the failed supernode can then take over the supernode role.

(0.05, 0.2)(0.3, 0.2) (0.55, 0.2) (0.8, 0.2) Heterogeneity. Supernodes should be those nodes with better capability with respect to CPU speed, network connections, and other resources. If a new node joins Figure 1. Supernode selection in CAN. towards an existing supernode coordinate, the new node (Supernode label expression is (0.05+0.25∗ and the existing physical node serving as a supernode n)%1, (0.20 + 0.25 ∗ m)%1 can negotiate to see which one is more capable to take the supernode role. Finally, a nearby non-supernode can offer to take over a nearby supernode’s role if it is more Example 2: Pastry structured overlay. To select capable by swapping virtual subspaces. k supernodes, the service initiator needs to generate a node label with log2k don’t-care bits as the highest order bits (or the don’t-care bits can be distributed Adaptability. A non-supernode can switch to another randomly within the label); the remaining bits in the supernode when it is not satisfied with the performance node label are randomly set to 0 or 1. For example, if of its current supernode. It can determine the distance the service initiator needs 1024 supernodes it will use to other supernodes by comparing its own node label ××× × 1001 1 ... ... as the supernode label expres- expression with the supernode’s node label expression, or it can query the DHT to see if new supernodes have 10 bits 118 bits sion. Any node whose label matches the 118 instanti- been selected by the initiator. When more supernodes ated bits is a potential supernode. When a message are are needed the initiator simply expands the supernode sent towards the supernode label, the physical node with label expression to cover more virtual node labels. the closest node label will receive it. Supernode takes charge of the service. The initiator Many peer-to-peer applications built on structured can send a message towards the supernode labels to in- overlay networks can benefit from SOLE. For example, form the chosen supernodes. The notification is done via SOLE can be used to dynamically select supernodes to multicast in the overlay network. Each physical node act as rendezvous points for applications such as cycle that owns a node whose label matches the supernode la- sharing, publish/subscribe, or storage sharing. Research bel expression will receive the message and become a in resource discovery in peer-to-peer cycle sharing sys- supernode. Alternatively, a supernode takes charge upon tems has shown that if rendezvous points are used to receiving the first request from a non-supernode. collect resource information and to match queries from Non-supernode joins service. If a non-supernode clients, the performance will be dramatically improved wants to join a service, it looks up the supernode compared with probing-based or advertisement-based label expression in the DHT. The non-supernode can resource discovery methods [24]. 3.2. PoPCorn: Supernode Selection on a y Coordinate-based Overlay Network

4 Adjust Token PoPCorn assumes an n-dimensional Euclidean coor- Repulsion Combined Repulsion dinate space using an Internet coordinate system such as Potential Supernode Non Supernode GNP [14] or Vivaldi [1]. PopCorn is suited for appli- 3 B(2,3) F(3,3) cations that wish to select a fixed set of k supernodes and distribute them evenly throughout the overlay, to perform a service such as security monitoring, protocol 2 E(1,2) A(2,2) C(3,2) RF testing, or data repositories. PoPCorn’s primary distri- ∠ RAD bution criteria, dispersal, is achieved by maximizing the RB sum of inter-node distances between all pairs of super- 1 R D(2,1) nodes. PoPCorn also achieves good access from non- supernodes to supernode, and can be easily extended to address heterogeneity, adaptability, and fault tolerance. 0 123 4 x The PoPCorn protocol selects k supernodes by dis- persing k tokens through the overlay coordinate space Figure 2. PoPCorn Repulsion Protocol using a repulsion model among the tokens, analogous to forces among charged particles. Each token represents one of the supernodes which moves through the overlay participate in token migration. After the PoPCorn to- based on the forces exerted on it by other tokens. When kens have stabilized, each node holding a token reports equilibrium is reached, each node holding a token is se- its position to the PoP initiator who will eventually ship lected as a supernode. the code for the PoP task as well as the location of the Initial Token Placement. The Initiator sends out k to- other tokens. kens to random peers in the overlay. Each peer can receive at most one token. Any peer which receives a to- Heterogeneity The threshold function TR for a node can ken becomes a potential supernode. The initiator can be changed according to the qualifications of that node distribute the tokens itself or it can ask its neighbors to to be a supernode. Nodes that are well qualified to help distribute tokens. serve as supernodes have higher threshold values, while Token Adjustment. The repulsion model is used to ad- poorly qualified nodes have lower thresholds. An un- just the location of the tokens. After a node receives a qualified node can set its threshold to zero. token and becomes a potential supernode, it will start a Adaptiveness and Resilience. Each token has a unique scoped gossiping session with its neighbors to tell them ID and nodes which hold the tokens gossip the IDs of the the coordinates of the token it holds. Whenever a gos- tokens. Loss of a token caused by node leave or failure siping message arrives, a potential supernode will re- will be noticed by missing heartbeat messages from that calculate the combined force vector of repulsions from token. The neighbor tokens can decide to replace this nearby tokens. If the magnitude of the combined repul- missing token by generating a token with an ID identi- sions exceeds a threshold TR, this potential supernode cal to the missing token. They will place the new token will pass its token to the neighbor whose position is clos- in the last reported location of the missing token, and est to the direction of the combined repulsion vector. then let the PoPCorn protocol adjust the location of that Figure 2 illustrates the repulsion model with a sim- new token based on repulsion forces. Alternatively, the ple example in which the nodes are evenly distributed node which discovers the loss of a token can report to in a 2-dimensional Euclidean space. There are three po- the initiator and let it decide whether it needs to inject a tential supernodes: A, B, and F . Node A receives two new token or not. repulsions RF and RB, from nodes F and B, respec- Load Balance. The threshold function in the PoPCorn tively. The combination of the two repulsions is vector protocol can be used in other creative ways, for exam- R. Among A’s neighbors, D has the smallest angle with ple, to meet load balancing criteria. In order to make R (RAD is the smallest). If the magnitude of R ex- more tokens stay in high density regions of the over- ceeds A’s threshold, A will pass its token to node D, lay, a potential supernode can query its neighbors and which will became a new potential supernode. gather local information about the number of nearby Finalization. If the time that a token stays on one nodes. Based on this information, if a potential super- potential supernode exceeds some time limit T ,the node detects that it is in a high density region, it can set node will mark its status as stable and it will no longer a higher threshold which must be overcome for the token to move. Similarly, potential supernodes in lower den- cache, it queries its immediate neighbors. If a contacted sity regions will set a lower threshold and will be more neighbor is a supernode, it replies to the requestor with likely to pass tokens on to other nodes. its qualifications. If a contacted neighbor is not a sup- PoPCorn was designed as part of the CCOF (Clus- ernode, it replies with entries from its local cache and ter Computing on the Fly) [12] project for peer-to-peer the requestor will cache these new responses. The re- cycle sharing (harnessing idle cycles throughout the In- questor then chooses the best candidate(s) according to ternet). PoPCorn places tasks that collectively form a its own criteria, and applies to those candidate super- distributed point-of-presence (PoP) application into a nodes. The contacted supernodes can confirm or reject cycle sharing overlay network. PoP applications typi- such requests. If the requestor does not hear from any- cally have low CPU and moderate communication re- one within a given time interval or is not satisfied with quirements. Examples include security monitors, Inter- the current supernodes, it has several choices: if it is net measurement monitors, and distributed protocol test- qualified, it can declare itself a supernode and begin the ing. Compared to volunteer systems, like NETI@home, advertisement protocol; it can join the overlay at another PoPCorn can better satisfy the distribution criteria and node, or it wait for a random amount of time and try place tasks evenly throughout the overlay. again. H20 is currently used to create a 2-level hierarchy 3.3. H20: Supernode selection protocol for un- for communication among security monitors within the structured overlay networks Sequoia collaborative security monitoring system [9]. The supernodes form themselves into an overlay network which is utilized as a backbone for fast and secure The H20 (Hierarchical 2-level Overlay) protocol for information dissemination. Only nodes that hold a se- supernode selection is a distributed negotiation proto- curity certificate (obtained from a central authority) are col for unstructured overlay networks. It is essentially eligible to be supernodes. A non-supernode can check a scalable protocol for multiple (c, d) colored domina- the trust level information in the certificate presented by tion that addresses the following supernode selection re- a supernode. A non-supernode can also choose a super- quirements: access, load balance, and fault tolerance in node based on trust-based routing criteria by evaluating a dynamic and heterogeneous environment, and some the trust level of nodes along the path to the potential security issues. supernode. For fault tolerance, a given non-supernode H20 uses a classic advertisement-based protocol, can connect to several supernodes, verifying that its con- in which supernodes advertise supernode information, nections to those supernodes traverse disjoint paths. La- and non-supernodes cache these advertisements. Non- tency criteria can also be considered. supernodes can then choose to join the best supernode(s) H20 can also be used to support threshold cryp- using locally cached information. This protocol gives tographic techniques (secret sharing) to certify public full autonomy to both supernodes and non-supernodes, keys. An (n, t) threshold scheme distributes partial sig- allowing each to negotiate using its own local policy. natures to each of n parties in such a way that any t ≤ n H20 is similar in many ways to the gnutella protocol, parties can authenticate, but it is infeasible for any sub- but allows for finer-grained control over the supernode set of t − 1 parties to do so. H20 selects a subset of selection process e.g., it can consider trust, secure paths, the nodes in the network to act as certificate authorities and routing performance. (CAs) such that any ordinary node has one-hop access Supernode Advertisement. A node capable of serving to at least t CAs, a colored (t, 1) domination problem. as a supernode advertises itself to its neighbors within a certain scope. The advertisement includes information about its qualifications to be a supernode (such as trust 4 Discussion and Conclusions level, uptime, bandwidth, and neighborhood size). Each advertisement is propagated a certain number of hops The challenge of the supernode selection problem for (set by each individual supernode) and carries with it the peer-to-peer applications lies in its need to fulfill disper- route travelled, i.e. an ordered list of the overlay nodes sal and access criteria in a way that is scalable, highly visited. This information is used by non-supernodes as adaptive, fault tolerant, and that respects local node au- part of the selection criteria. A node that receives an tonomy. Furthermore, the supernode selection protocols advertisement message caches the advertisement about must cope with an unknown, large scale, dynamically the potential supernode in its local cache. changing, and heterogeneous network. Supernode Search. If a new node want to find sup- We have described three supernode selection proto- ernodes, it first consults its local cache for supernode cols that deal with some of these challenges in unique candidates. If there are no suitable candidates in the ways. SOLE and any protocol based on structured overlays capitalizes on the use the supernode label expres- [7] G. Handler and P. Mirchandani. Location on Networks: sion to efficiently encode the identity of all the super- Theory and Algorithms. The MIT Press, 1979. nodes as well as the distances to supernodes. Asyn- [8] T. Haynes, S. Hedetniemi, and P. Slater. Fundamentals of Domination in Graphs. 1998. chronous distributed protocols such as PoPCorn and [9] X. Kang, D. Zhou, D. Rao, J. Li, and V. Lo. Sequoia: 0 H2 are maximally flexible because they provide lo- A robust communication architecture for collaborative cal control over supernode selection criteria. Thus, they security monitoring systems. In Poster session, SIG- can be easily tailored to a wide variety of applications. COMM’04. However, they must be carefully designed to minimize [10] M. Karlsson, M. Mahalingam, and Z. Xu. Turning het- message-passing overhead and converge to a (reason- erogeneity into an advantage in overlay routing. In IN- ably) stable set of supernodes. FOCOM’02. [11] Kazaa http://www.kazaa.com. There are many open problems for the supernode se- [12] V. Lo, D. Zappala, D. Zhou, Y. Liu, and S. Zhao. Cluster lection problem. (1) Algorithm and protocol develop- Computing on the Fly: P2P scheduling of idle cycles in ment. There is a need for fully distributed algorithms the Internet. In IPTPS’04. for the underlying graph theoretic dominating set and [13] A. Lynch. Distributed Algorithms. Morgan Kaufmann, San Francisco, 1997. p-centers that operating under dynamic conditions. In [14] T. Ng and H. Zhang. Predicting Internet network distance addition, protocols that emphasize specific distribution with coordinates-based approaches. In INFOCOM’02. criteria, or that are tailored to classes of applications [15] V. Paxson. End-to-end routing behavior in the Internet. need to be developed. Bridging the gap between ab- In SIGCOMM Conference on Applications, Technolo- stract algorithms and practical protocols is challenging. gies, Architectures, and Protocols for Computer Commu- (2) Performance analysis. Measuring the performance nications, 1996. [16] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and of a supernode selection protocol is a hard problem in S. Shenker. A scalable content addressable network. In a large dynamic network. Metrics, workloads, bench- SIGCOMM’01. marks from peer-to-peer computing need to be assem- [17] S. Ratnasamy, M. Handley, R. Karp, and S. Shenker. bled that can be used to evaluate and compare supernode Topologically-aware overlay construction and server se- selection protocols in dynamic large scale networks. It lection. In INFOCOM’02, 2002. is not clear yet what metrics best capture dispersal, ac- [18] A. Rowstron and P. Druschel. Pastry: scalable, decen- cess, proportion, and load balance. (3) Security. Clearly tralized object location, and routing for large-scale peer- to-peer systems. In Proc. 18th IFIP/ACM Int’l Conf. on a wide range of questions related to secure supernode se- Distributed Systems Platforms, Nov. 2001. lection remain open. These include dealing with denial [19] C. R. Simpson, Jr., and G. F. Riley. Neti@home: A dis- of service attacks on the supernodes, as well as how to tributed approach to collecting end-to-end network per- detect and neutralize malicious supernodes. formance measurements. In 5th Passive and Active Mea- We believe an integrated approach to the supernode surement Workshop, Apr. 2004. selection problem, built on strong graph theoretic foun- [20] I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Bal- dations and guided by realistic applications, can yield akrishnan. Chord: a scalable peer-to-peer lookup service for internet applications. In Proc. ACM SIGCOMM,San benefits for the field of peer-to-peer computing and be- Diago, CA, Aug. 2001. yond. [21] V. Vazirani. Approximation Methods. Springer-Verlag, 1999. References [22] P. Wan, K. Alzoubi, and O. Frieder. Distributed construction of connected dominating set in wireless ad hoc networks. Mob. Netw. Appl., 9(2), 2004. [1] F. Dabek, R. Cox, F. Kaashoek, and R. Morris. Vi- [23] J. Wu, F. Dai, M. Gao, and I. Stojmenovic. On calculat- valdi: A decentralized network coordinate system. In ing power-aware connected dominating sets for efficient SIGCOMM ’04. routing in ad hoc wireless networks. Journal of Commu- [2] B. Das and V. Bharghavan. Routing in ad-hoc networks nications and Networks, 4(1), March 2002. using minimum connected dominating sets. In ICC (1), [24] D. Zhou and V. Lo. Cluster Computing on the Fly: in a pages 376–380, 1997. [3] D. Estrin, R. Govindan, J. Heidemann, and S. Kumar. cycle sharing peer-to-peer system. In Proceedings of the Next century challenges: scalable coordination in sensor 4th International Workshop on Global and P2P Comput- networks. In MobiCom’99. ing (GP2PC’04), 2004. [4] R. A. Ferreira, S. Jagannathan, and A. Grama. Enhancing locality in structured peer-to-peer networks. In ICPADS, 2004. [5] P. Francis, S. Jamin, V. Paxson, L. Zhang, D. Gryniewicz, and Y. Jin. An architecture for a global Internet host distance estimation service. In INFOCOM’99. [6] Gnutella Protocol Development. http://www. the-gdf.org, 2005.