Overlay and P2P Networks

Unstructured networks

Prof. Sasu Tarkoma 17.1.2013 Contents

• Terminology and overlays continued

• Unstructured networks • Today: • Napster • Skype • Gnutella • Next week: • Freenet • BitTorrent Terminology

• Peer-to-peer (P2P) – Different from client-server model – Each peer has both client/server features • Overlay networks – Routing systems that run on top of another network, such as the Internet. • Distributed Hash Tables (DHT) – An algorithm for creating efficient distributed hash tables (lookup structures) – Used to implement overlay networks • Typical features of P2P / overlays – Scalability, resilience, high availability, and they tolerate frequent peer connections and disconnections Peer-to-peer in more detail

• A P2P system is distributed – No centralized control – Nodes are symmetric in functionality

• Large faction of nodes are unreliable – Nodes come and go

• P2P enabled by evolution in data communications and technology

• Current challenges: – Security (zombie networks, trojans), IPR issues

• P2P systems are decentralized overlays Characteristics of P2P systems

P2P can be seen as an organizational principle Applied in many different application domains

Characteristics Self-organization Lack of central coordination Resource sharing Based on collaboration between peers Peers are typically equal Large number of peers Resilient to certain kinds of attacks (but vulnerable to others)

P2P Volume

Estimates range from 20-40% of Internet Traffic

Latest estimates from Cisco suggest that video delivery is the growing and the share of P2P file exchange traffic is becoming smaller

P2P can be used for video delivery as well

.. And voice (Skype, P2PSIP)

Hundreds of millions of people use P2P technology today

Evolution of P2P systems

• ARPAnet had P2P like qualities – End-to-end communication, FTP, USENET,.. – Today’s BGP is P2P • Started from centralized servers – Napster • Centralized directory • Single point of failure • Second generation used flooding (Gnutella v0.4) – Local directory for each peer – High cost, worst-case O(N) messages for lookup – Third generation use some structure (Gnutella v0.7) • Research systems use DHTs – Chord, Tapestry, CAN, .. – Decentralization, scalability • Some recent CDNs and content delivery systems exhibit P2P features (P2P assisted CDN) Unstructured networks

Unstructured networks are typically based on random graphs following flat or hierarchical organization

Unstructured networks utilize flooding and similar opportunistic techniques, such as random walks, expanding-ring, Time-to-Live (TTL) search, in order to locate peers that have interesting data items.

Many P2P systems: Gnutella, Freenet, BitTorrent, …

Napster

Napster was a centralized P2P music sharing service (mp3s)

Lauched in 1999 and made P2P popular and dubious from the legal viewpoint Lawsuits from 1999, close-down in 2001, Chapter 7 in 2002, rebirth as a music store in 2003

Utilized a centralized index (server farm) for searching, transfers were peer-to-peer

User installing the software Download the client program Register name, password, local directory, etc. centralized Bob directory server 1. Client contacts Napster (via TCP) 1 Provides a list of music files it will share peers 1 … and Napster’s central server updates the directory 3 1 2. Client searches on a title or performer Napster identifies online clients with the 2 1 file … and provides IP addresses 3. Client requests the file from the chosen

Alice supplier Supplier transmits the file to the client Both client and supplier report status to Napster

Napster Summary

Centralized server allows Consistent view of the P2P network Search guaranteed to find all files in the network

Limitations of this design are Centralized server is the weakest point of the system Attacks, network partitions, … Limited scalability

Skype I

Skype is a well-known Internet telephony service Calls between peers Interface to traditional telephony services

Skype architecture is similar to KaZaa and Gnutella Supernodes and regular nodes (covered soon!) Developed by makers of Kazaa, now owned by Microsoft

A proprietary protocol, protocol uses encryption

A centralized server for logging and billing Skype II

Supernodes and regular nodes maintain a distributed directory of online peers

Supernodes forward calls and call traffic (mostly for firewalled/natted peers)

A number of built-in techniques for traversing firewalls and NAT boxes, STUN-like behaviour

Security support: authentication with public key crypto and certificates, symmetric session keys

What is NAT

Expand IP address space by deploying private address and translating them into publicly registered addresses Private address space (RFC 1918, first in RFC 1631) 10.0.0.0 - 10.255.255.255 (10.0.0.0/8) 172.16.0.0 - 172.31.255.255 (172.16.0.0/12) 192.168.0.0 - 192.168.255.255 (192.168.0.0/16) Technique of rewriting IP addresses in headers and application data streams according to a defined policy

Source: Tanenbaum 4th NATs and Firewalls

Firewalls Security main concern Demilitarized zone Increasingly complex rules (what is filtered, how) NATs Lightweight security devices Topology hiding and firewalling Increasing number in deployment Solves some of the address space problems of IPv4 (Port Translation, NAPT) IPv6 solves the addressing problem so NATs are not needed for this Skype

Skype clients (SC) • Skype is P2P • Proprietary application-layer protocol • Hierarchical overlay with Skype super nodes login server

• Index maps usernames to IP Skype addresses; distributed over Super Node super nodes • Peers with connectivity issues use NAT traversal or Skype Skype communicate via super node relays

• Security: RSA, AES, SHA-1, authentication with Skype Servers Skype peers as relays

Problem when both Alice and Bob are

behind “NATs”.

NAT prevents an outside peer from

initiating a call to insider peer Solution: Skype Using Alice’s and Bob’s SNs, Relay is

chosen

Each peer initiates session with relay. Skype Skype Peers can now communicate through

NATs via relay Login

1. Login routed through a super node. Find super nodes by sending UDP packets to bootstrap super nodes (defaults) and wait for responses 2. Establish TCP connections with selected super nodes based on responses 3. Acquire the address of a login server and authenticate user 4. Send UDP packets to a preset number of nodes to advertise presence (a backup connectivity list).

Host Cache (HC) is a list of super node IP address and port pairs that Skype Client maintains.

Login algorithm (HC is host cache, Login server not shown)

Reference: An Analysis of the Skype Peer-to-Peer Internet Telephony Protocol, Infocom 2006. 19 Skype and NATs

Comparison of three network setups Exp A: both Skype users with public IP address Exp B: Skype caller behind port-restricted NAT (incoming port must be the one that sent the packet, more difficult to punch a hole) Exp C: Both Skype users behind port-restricted NAT and UDP- restricted firewall

Message flows for first time login process Exp A and Exp B are similar Exp C only exchange data over TCP

Total data Login process exchanged time

Exp A Approx 9 KB 3-7 secs Exp B Approx 10 KB 3-7 secs

Exp C Approx 8.5 KB Approx 34 secs

Reference: An Analysis of the Skype Peer-to-Peer Internet Telephony Protocol, Infocom 2006. Establishing a Call: Three cases

Case 1: Public IP addresses. Caller establishes TCP connection with callee Skype client.

Case 2: Caller is behind port-restricted NAT, callee has public IP. Caller uses online Skype node to forward packets over TCP/UDP.

Case 3: Both caller and callee behind port-restricted NAT and UDP restricted firewall. Exchange info with a Skype node using TCP. Caller sends media over TCP to an online node which forwards to callee via TCP.

Security

1. Skype client has a built in list of Skype servers and their public keys (Ks+) 2. Users first register username and a hash of password (H(pwd)) at the server (encrypt with server public key) 3. On each login session, Skype client generates a session key K 4. Skype client also generates a 1024-bit private/public RSA key pair (KA+, KA-). 5. Skype client sends Ks+ (K), K (KA+, Username, H(pwd)) to server and obtains a certificate for the Username, public key pair (only if password is valid) 6. Certificate is disseminated to Super Nodes 7. Skype clients can then authenticate by exchanging certificates and verifying that a Skype server has signed the certificates 8. Final step is to derive a session key that is used to encrypt all communications Blocking skype

Firewall rules

Skype traffic detection

http://www.tml.tkk.fi/Publications/C/23/papers/ Santolalla_final.pdf User Search

Skype uses a global index to search for a user

UDP/TCP between Skype nodes and/or super nodes

Skype claims that search is distributed and is guaranteed to find a user if it exists and has logged in during last 72 hours

Search results are observed to be cached at intermediate nodes Supernode map (Infocom 2006 article) Conclusion on Skype

Successful overlay technology

Super node design

Call forwarding with self-organizing network of nodes

Distributed index of users (phonebook)

NAT traversal techniques

Built-in Security

Gnutella

Gnutella addresses some of Napster’s limitations

A decentralized P2P system based flooding the queries Queries are flooded and responses are sent on the reverse path Downloads directly between peers

Open protocol specification, originally developed by Nullsoft (bought by AOL)

Differs between versions 0.4 is the original version (simple flooding) 0.7 is more advanced (similar to KaZaa) More structure (hierarchy is good for scalability!)

Gnutella v0.4 protocol messages I

• A peer joining the network needs to discover the adress of a peer who is already a member of the network • New peer sends GNUTELLA CONNECT message

• A peer then uses PING messages to discover peers and receives PONG messages.

• PONGs include data regarding peers and follow the reverse path of PINGs. Gnutella v0.4 protocol messages II

• A peer uses the QUERY message to find files, and receives QUERYHIT messages as replies (again on reverse path) • Peers forward QUERY messages (flooding)

• The QUERYHIT contains the IP address of the node that can then be used for the file transfer (HTTP)

• PUSH request message can be used to circumvent firewalls (servent sends file to the requesting node after receiving request)

• HTTP Push proxy: proxy sends the push request Gnutella messages

Payload Descriptor:

Ping : does not contain any payload. 23 B

Pong: Port IP Number of shared files Size of shared 14 B

Query: Minimum Speed Search Criteria n+1 B

Query_hit: Number of hits Port IP Speed Result Set Node ID n+16 B

Source: www3.in.tum.de/teaching/ss09/DBSeminar/P2P.ppt 5 2 Ping/Pong Message propagation 1 4 6 3

8 7 P7 P3 P1 P5 P2 P4 P6 P8

Gnu-Con Gnu-Con OK OK Gnu-Con OK Ping Ping Ping Ping Ping Ping Ping Ping Ping Ping Ping ` Pong Pong Pong Pong Pong Pong Pong Pong Pong

Pong Pong

Pong

Source: www3.in.tum.de/teaching/ss09/DBSeminar/P2P.ppt The Gnutella Protocol: Query

Query message sent File transfer: over existing TCP HTTP connections Query Peers forward Query message QueryHit QueryHit sent over

reverse path

Scalability: Query limited scope flooding QueryHit Traffic breakdown

Can be more PONGs than PINGs (see previous diagram)

From “A Quantitative Analysis of the Gnutella Network Traffic” From “A Quantitative Analysis of the Gnutella Network Traffic” Trees vs graphs

Tree N nodes, N-1 links

Network with N hosts and M connections, M >= N-1 then (M – (N -1)) loop connections

These make the network more robust, but increase communications overhead

Loops result in infinite message loops (unless specific loop prevention measures are implemented) Looping and message processing

Gnutella network is based on a cyclic graph

Loops are problematic

Two key solutions: 1. TTL (Time-To-Live): reduces flooding (7 by default) 2. Duplicate detection with unique request identifier

Gnutella uses both (v0.7 is not using flooding anymore so the problem is alleviated)

Even with duplicate detection cannot prevent receiving the same message many times (but can prevent propagation)

Request messages

Each peer keeps track of all messages it has seen

Can forget about that after some time period

Remember who first sent you a message

If a second copy or subsequent copy of a message arrives, ignore it

Response messages

Use the same GUID as the message they are in response to

Each peer routes a response msg to the peer from which it first received the original msg

Drop message if did not see original

Problems in original Gnutella reverse path

Peers come and go à routes break

Reverse path requires that the traversed route is used This means that reverse path may not work

The implementation requires state in the nodes

Solutions 1. introduce more stable ultra nodes 2. send message toward known content sources à reduce overhead 3. Contact nodes directly! The Gnutella v0.7 Architecture

Ultra node layer

Ultra node

Flooding (Bloom filters) Ultra node Ultra node

Leaf Data transfer Leaf Leaf Leaf Gnutella v0.7 routing

Since version 0.6, Gnutella has been a composite network consisting of leaf nodes and ultra nodes. The leaf nodes have a small number of connections to ultra nodes, typically three The ultra nodes are hubs of connectivity, each being connected to more than 32 other ultra nodes. When a node with enough processing power joins the network, it becomes an ultra peer and establishes connections with other ultra nodes This network between the ultra nodes is flat and unstructured. Then the ultra node must establish a minimum number of connections with client nodes in order to continue acting as an ultra node. These changes attempt to make the Gnutella network reflect the power- law distributions found in many natural systems. Query Routing Protocol I

In Gnutella terminology, the leaf nodes and ultra nodes use the Query Routing Protocol to update routing tables, called Query Routing Table (QRT)

The QRT consists of a table hashed keywords that is sent by a leaf node to its ultra nodes

Ultra nodes merge the available QRT structures that they have received from the leaf nodes, and exchange these merged tables with their neighbouring ultra nodes Query Routing Protocol II

Query routing is performed by hashing the search words and then testing whether or not the resulting hash value is present in the QRT of the present node

Ultrapeer forwards query to top-level connections and waits for responses

Query is flooded outward until the TTL expires Query Routing Protocol III

The ultrapeer then waits for the results, and determines how rare matches are (the ratio between the number of results and the estimated number of visited peers)

If matches are rare, the query is sent through more connections with a relatively high TTL

If matches are more common but not sufficient, the query is sent down a few more connections with a low TTL

The classical Gnutella protocol used reverse path routing to send a message back to this origin peer. Later incarnations of the protocol use UDP to directly contact the origin peer

L7 L6

S3 Message Propagation L5 S2 L4 S1 L1

L2 L3 L2 L3 L1 S1 S3 S2 L7 L6 L5 L4

Gnu-Con OK Ping R_T_U Ping ping Ping Ping pong pong pong pong pong

Query Query

Query Query Query

Query Query Query Query Query Query Query Query Query Query Query

Source: www3.in.tum.de/teaching/ss09/DBSeminar/P2P.ppt Bloom filters in Gnutella v0.7

Bloom filters are probabilistic structures used to store dictionaries A bit-vector that supports constant time querying of keywords

Decrease Increase

Number of hash functions (k) Less computation More computation Higher false positive rate Lower false positive rate Size of filter (m) Smaller space requirements More space is needed Higher false positive rate Lower false positive rate

Number of elements in the Lower false positive rate Higher false positive rate inserted set (n) Example Bloom filter

x y z

0 1 0 1 1 1 0 0 0 0 0 1 0 1 0 0 1 0

TARKOMA et al.:THEORYANDPRACTICEOFBLOOMFILTERSFORDISTRIBUTEDSYSTEMS 133

hash functions by Henke et al. [4]. Later in Section II-C we False positive rate of Bloom filters discuss relevant hashing techniques further. 1 ABloomfilter constructed based on S requires space O(n) 0.1

and can answer membership queries in O(1) time. Given x 0.01 ∈ S,theBloomfilter will always report that x belongs to S,but 0.001 given y S the Bloom filter may report that y S. "∈ ∈ 0.0001 Table I examines the behaviour of three key parameters when their value is either decreased or increased. Increasing 1e-005 or decreasing the number of hash functions towards kopt can 1e-006 False positive probability (p) lower false positive ratio while increasing computation in 1e-007 m=64 insertions and lookups. The cost is directly proportional to the m=512 1e-008 m=1024 m=2048 number of hash functions. The size of the filter can be used to m=4096 1e-009 tune the space requirements and the false positive rate (fpr). 1 10 100 1000 10000 100000 Alargerfilter will result in fewer false positives. Finally, the Number of inserted elements (n) size of the set that is inserted into the filter determines the false positive rate. We note that although no false negatives Fig. 3. False positive probability rate for Bloom filters. (fn)occurwithregularBFs,somevariantswillbepresented later in the article that may result in false negatives. with respect to k.Thisisaccomplishedbytakingthederivative and equaling to zero, which gives the optimal value of k A. False Positive Probability m 9m k = ln 2 . (6) We now derive the false positive probability rate of a Bloom opt n ≈ 13n filter and the optimal number of hash functions for a given This results in the false positive probability of false positive probability rate. We start with the assumption 1 k that a hash function selects each array position with equal 0.6185m/n. (7) probability. Let m denote the number of bits in the Bloom 2 ≈ TARKOMA et al.:THEORYANDPRACTICEOFBLOOMFILTERSFORDISTRIBUTEDSYSTEMS 133 ! " filter. When inserting an element into the filter, the probability Using the optimal number of hashes kopt,thefalsepositive that a certain bit is not set to one by a hash function is probability can be rewritten and bounded hash functions by Henke et al. [4]. Later in Section II-C we False positive rate of Bloom filters discuss relevant hashing techniques further. 1 1 m 1 1 . (1) . (8) ABloomfilter constructed based on S requires space O(n) 0.1 − m n ≥ ln 2 and can answer membership queries in O(1) time. Given x 0.01 Now, there are k hash functions, and the probability of any This means that in order to maintain a fixed false positive S,theBloomfilter will always report that x belongs to S,but∈ 0.001 of them not having set a specificbittooneisgivenby given y S the Bloom filter may report that y S. probability, the length of a Bloom filter must grow linearly "∈ ∈ 0.0001 k with the number of elements inserted in the filter. The number Table I examines the behaviour of three key parameters 1 when their value is either decreased or increased. Increasing 1e-005 1 . (2) of bits m for the desired number of elements n and false − m orTARKOMA decreasinget al.:THEORYANDPRACTICEOFBLOOMFILTERSFORDISTRIBUTEDSYSTEMS the number of hash functions towards kopt can 1e-006 ! " 133 positive rate p,isgivenby False positive probability (p) lower false positive ratio while increasing computation in 1e-007 After inserting n elements to the filter, the probability that m=64 n ln p insertions and lookups. The cost is directly proportional to the m=512 1e-008 agivenbitisstillzerois m=1024 m = 2 . (9) hash functions by Henke et al. [4]. Later in Section II-C we False positive rate of Bloom filtersm=2048 −(ln 2) number of hash functions. The size of the filter can be used to m=4096 discuss relevant hashing techniques further. 1e-009 1 kn tune the space requirements and the false positive rate (fpr). 1 10 100 1000 110000 100000 Figure 3 presents the false positive probability rate p as a ABloomfilter constructed based on S requires space O(n) 0.1 Number of inserted elements1 (n) . (3) Alargerfilter will result in fewer false positives. Finally, the − m function of the number of elements n in the filter and the filter sizeand of can the answer set that membership is inserted queries into the in fiOlter(1) determinestime. Given thex 0.01 ! " ∈ size m.Anoptimalnumberofhashfunctionsk =(m/n)ln2 falseS,theBloom positive rate.filter We will note always that report although that nox belongs false negatives to S,butFig. 3. False 0.001 positiveAnd probability consequently rate for Bloom the probabilityfilters. that the bit is one is (fngiven)occurwithregularBFs,somevariantswillbepresentedy S the Bloom filter may report that y S. has been assumed. "∈ ∈ 0.0001 kn Table I examines the behaviour of three key parameters 1 There is a factor of log e 1.44 between the amount of later in the article that may result in false negatives. 1 1 . (4) 2 ≈ when their value is either decreased or increased. Increasing with respect 1e-005 to k.Thisisaccomplishedbytakingthederivative− − m space used by a Bloom filter and the optimal amount of space and equaling to zero, which gives the optimal! value" of k or decreasing the number of hash functions towards kopt can 1e-006 that can be used. There are other data structures that use space

A. False Positive Probability False positive probability (p) For an element membership test, if all of the k array lower false positive ratio while increasing computation in 1e-007 m 9m closer to the lower bound, but they are more complicated (cf. kopt = ln 2 . m=64 (6) insertions and lookups. The cost is directly proportional to the positions in the filter computed bym=512 the hash functions are set We now derive the false positive probability rate of a Bloom 1e-008 n ≈ 13n m=1024 [5], [6], [7]). to one, the Bloom filter claims thatm=2048 the element belongs to the finumberlter and of the hash optimal functions. number The of size hash of functions the filter can for be a given used to This results in the false positive probability ofm=4096 1e-009 Recently, Bose et al. [8] have shown that the false positive falsetune positive the space probability requirements rate. and We the start false with positive the assumption rate (fpr). set. 1 The probability 10 100 of this 1000 happening 10000 when 100000 the element is not BF Falsek positive probability is given by: analysis originally given by Bloom and repeated in many sub- thatAlarger a hashfilter function will result selects in each fewer array false position positives. with Finally, equal the 1 Number of insertedm/n elements (n) part of the set is0 given.6185 by. (7) probability.size of the Let setm thatdenote is inserted the number into the offi bitslter in determines the Bloom the 2 ≈ sequent articles is optimistic and only a good approximation ! " kn k for large Bloom filters. The revisited analysis proves that the fifalselter. When positive inserting rate. We an element note that into although the filter, no the false probability negatives Fig.Using 3. the False optimal positive number probability of rate hashes for1 Bloomk ,thefalsepositivefilters. k 1 1 opt 1 e kn/m . (5) that(fn a)occurwithregularBFs,somevariantswillbepresented certain bit is not set to one by a hash function is probability can be rewritten and bounded − commonly used estimate (Eq. 5) is actually a lower bound and # − − m $ ≈ − later in the article that may result1 in false negatives. ! " % & the real false positive rate is larger than expected by theory, m 1 1 . (1) with respect to k.Thisisaccomplishedbytakingthederivativekn/m. (8) especially for small values of m. − m We noteOptimal thatn ≥e number−ln 2 ofis hash a very functions close k: approximation of (1 and equaling1 to zero, which gives the optimal value of k − Now, there are k hash functions, and the probability of any This means)kn that[1]. in order The to false maintain positive a fixed probability false positive decreases as the size A. False Positive Probability m m 9m of them not having set a specificbittooneisgivenby probability,of the the length Bloomk of a=fi Bloomlter,lnm 2fi,lter increases. must. grow The linearly probability(6) increases B. Operations We now derive the false positive probability rate of a Bloom opt n ≈ 13n k with the number of elements inserted in the filter. The number filter and the optimal number1 of hash functions for a given with n as more elements are added. Now, we want to minimize Standard Bloom filters do not support the removal of 1 . (2) of bitsThism resultsfor the in desired the false number positive of probability elements n ofand false kn/m k false positive probability rate.− m We start with the assumption the probabilitySize of of filter false given positives, optimal number by minimizing of hash functions:(1 e− ) elements. Removal of an element can be implemented by positive rate p,isgivenby k − ! " 1 thatAfter a hash inserting functionn elements selects to each the fi arraylter, the position probability with that equal 0.6185m/n. (7) 2 ≈n ln p agivenbitisstillzeroisprobability. Let m denote the number of bits in the Bloom !m =" . (9) filter. When inserting an element into the filter, the probability −(ln 2)2 kn Using the optimal number of hashes kopt,thefalsepositive 1 Details in the survey paper available on course page. that a certain bit is not set1 to one by. a hash function is (3) probabilityFigure 3 presents can be the rewritten false positive and bounded probability rate p as a − m 1 function of the number of elements n in the filter and the filter ! 1 " . (1) m 1 And consequently the probability− m that the bit is one is size m.Anoptimalnumberofhashfunctions. k =(m/n)ln2 (8) has been assumed. n ≥ ln 2 kn Now, there are k hash functions,1 and the probability of any ThereThis is means a factor that of inlog ordere to1. maintain44 between a fixed the amount false positive of 1 1 . (4) 2 ≈ of them not having set− a speci− mficbittooneisgivenby spaceprobability, used by a the Bloom lengthfilter of and a Bloom the optimalfilter amount must grow of space linearly ! " k thatwith can the be numberused. There of elements are other inserted data structures in the fi thatlter. use The space number For an element membership1 test, if all of the k array 1 . (2) closerof bits to them lowerfor the bound, desired but numberthey are moreof elements complicatedn and (cf. false positions in the filter computed− m by the hash functions are set ! " [5],positive [6], [7]). rate p,isgivenby to one, the Bloom filter claims that the element belongs to the After inserting n elements to the filter, the probability that Recently, Bose et al. [8] have shown that the false positive set. The probability of this happening when the element is not n ln p agivenbitisstillzerois analysis originally given bym Bloom= and repeated. in many sub- (9) part of the set is given by −(ln 2)2 kn sequent articles is optimistic and only a good approximation k 1 Figure 3 presents the false positive probability rate p as a 1 kn1 . k (3) for large Bloom filters. The revisited analysis proves that the − m kn/m function of the number of elements n in the filter and the filter 1 1 ! " 1 e− . (5) commonly used estimate (Eq. 5) is actually a lower bound and # − − m $ ≈ − size m.Anoptimalnumberofhashfunctionsk =(m/n)ln2 And consequently! the" probability% that the bit& is one is the real false positive rate is larger than expected by theory, kn/m especiallyhas been for assumed. small values of m. We note that e− is a very closekn approximation of (1 1 − There is a factor of log e 1.44 between the amount of 1 )kn [1]. The false positive1 1 probability decreases. as the size(4) 2 ≈ m − − m space used by a Bloom filter and the optimal amount of space of the Bloom filter, m, increases.! The" probability increases B. Operations that can be used. There are other data structures that use space withForn as an more element elements membership are added. Now, test, ifwe all want of to the minimizek array Standard Bloom filters do not support the removal of kn/m k closer to the lower bound, but they are more complicated (cf. thepositions probability in the of falsefilter positives, computed by by minimizing the hash functions(1 e− are) set elements. Removal of an element can be implemented by − [5], [6], [7]). to one, the Bloom filter claims that the element belongs to the Recently, Bose et al. [8] have shown that the false positive set. The probability of this happening when the element is not analysis originally given by Bloom and repeated in many sub- part of the set is given by sequent articles is optimistic and only a good approximation k kn k for large Bloom filters. The revisited analysis proves that the 1 kn/m 1 1 1 e− . (5) commonly used estimate (Eq. 5) is actually a lower bound and # − − m $ ≈ − ! " % & the real false positive rate is larger than expected by theory, kn/m especially for small values of m. We note that e− is a very close approximation of (1 1 kn − m ) [1]. The false positive probability decreases as the size of the Bloom filter, m, increases. The probability increases B. Operations with n as more elements are added. Now, we want to minimize Standard Bloom filters do not support the removal of kn/m k the probability of false positives, by minimizing (1 e− ) elements. Removal of an element can be implemented by − If false positive rate is fixed, the filter size grows linearly with inserted elements. Mapping the Gnutella Network

A F A F

B D E G B D E G

C H C H

Perfect mapping for message from A. Inefficient mapping that results in link D-E Link D-E is traversed only once. being traversed six times

Overlay networks can result in really bad application layer routing configurations unless the underlay is taken into account! Gnutella v0.4 Gnutella v0.7

Decentralization Flat topology (random Random graph with two graph), equal peers tiers. Two kinds of nodes, regular and ulta nodes. Ultra nodes are connectivity hubs Foundation Flooding mechanism Selective flooding using the super nodes

Routing function Flooding mechanism Selective flooding mechanism

Routing performance Search until Time-To-Live Search until Time-To-Live expires, no guarantee to expires, second tier locate data improves efficiency, no guarantee to locate data

Routing state Constant (reverse path Constant (regular to ultra, state, max rate and TTL ultra to ultra). Ultra nodes determine max state) have to manage leaf node state. Performance degrades Performance degrades when Reliability when the number of peer the number of peer grows. grows. No central point. Hubs are central points that can be taken out.