DOCSLIB.ORG

Rethinking the Building Block: a Profiling Methodology for UDP Flows

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Rethinking the Building Block: a Profiling Methodology for UDP Flows Rethinking The Building Block: A Profiling Methodology for UDP Flows 123Jing Cai 13Zhibin Zhang 13Peng Zhang 13Xinbo Song 1Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 2Graduate University of Chinese Academy of Sciences, Beijing, China 3National Engineering Laboratory for Information Security, Beijing, China [email protected] Abstract—With the increase of network bandwidth, more and analysis on the 24-hour data we collected from a backbone more new applications such as audio, video and online games router, we found that the proportion of the UDP packets is have become the main body in network traffic. Based on real- around 40%. It could sometimes reach 80% at most. From the time considerations, these new applications mostly use UDP as transport layer protocol, which directly increase UDP traffic. view of the proportion of UDP bytes, we also found that the However, traditional studies believe that TCP dominates the proportion of the UDP bytes fluctuated greatly, but generally Internet traffic and previous traffic measurements were generally exceeded 40%, and sometimes was even above 80%. based on it while UDP was ignored. Since the increase of UDP traffic, more and more people In view of this, we mainly discuss the profiling methodology of have started to pay attention to the traffic of UDP. However, UDP flows in this paper. First, from the view of the network layer, we present a profiling methodology on the basis of the Claffy’s compared with the TCP, we found there at least exist two big parameter flow model. Next, due to the significant differences differences. among the different applications, we found that using a unified Firstly, TCP is a connection-oriented protocol, it has con- methodology and ignoring the information of application layer is trolling flags such as FIN and RST to explicitly identify the not appropriate for UDP flows. At last, we get the conclusion end of flow. But for UDP, it is a connectionless protocol. In that for different applications, such as dns, qq[16], we must use corresponding profiling methodology based on their specific essence, there is no concept of ”flow” in udp. At present, characteristics. people commonly refer to UDP flows as a series of packets which are required to complete a data transfer from the view I. INTRODUCTION of the application layer. Therefore, Claffy’s flow model is still The main purpose of the network traffic measurement is to applicable for UDP. But in this case, the parameters setting is enhance people’s awareness about traffic characteristics. The not the same as before. traffic measurement that works based on the network layer The second, compared with TCP, the composition of UDP started from the 1980s. Earlier studies took the packet as the is more complicated. In the past, in measurement of TCP Building Block. But due to its small granularity, it could not flows and from the view of the network layer, people do not meet the needs in many ways. According to the locality of care about the specific application services, so the parameters network traffic in time and space, Claffy et al.[1][2] firstly of flow model are generally identical. However in UDP, proposed a parameters flow model. Since then, the traffic the characteristics of different applications often demonstrate measurement based on flow had gradually become a hot issue significant differences. Using uniformed parameters to profile in the field of network measurement. flow without considering the assumption on application layer However, in the past, during the process of network traffic is feasible for TCP flows, but will bring many uncertainties to measurement, people generally believed that TCP traffic oc- the connectionless UDP flows. cupied the main body of the network traffic, and UDP traffic Due to these two great differences, earlier network measure- is negligible, and therefore ignored the measurements of the ment based on flows mostly focus the TCP flows, while UDP UDP flow. However, the situation has undergone tremendous flows was ignored. The study on the UDP flows is nearly changes at present. With the increase of network bandwidth, in the blank stage. In view of these, we mainly discuss the the traditional networking services based on images and text profiling methodology of UDP flow in this paper. To the best could no longer satisfy people’s needs. More and more audio, of our knowledge, we are the first to do so. There are two video, and online games, have gradually become the main main contributions in our paper. body of the network traffic. These applications mostly use ∙ Through our comprehensive considerations on each eval- UDP as their transport layer protocol[3], which directly results uation criteria, we built the appropriate profiling method- in the increase of UDP traffic. The organization of CAIDA[4] ology on the basis of Claffy’s parameter flow model from analyzed the trace collected in the period 2002-2009 on several the view of the network layer. backbone links located in the US and Sweden, found the ratio ∙ Taking the complexity of UDP into account, due to the between the UDP and TCP in packets, bytes, and flows have significant differences between the application protocols, increased greatly. In China, as shown in Fig.1, after statistical we want to know whether it is enough to use a unified The proportion of the udp’s packets and bytes TABLE I: The basic information of the trace 1 UDP−pkt / IP−pkt 0.9 UDP−byte / IP−byte Id Begin time End time Bytes Packets I 2009.5.5,14:57 2009.5.6,00:30 275G 2805(million) 0.8 0.7 0.6 However, earlier studies believe that TCP dominates the 0.5 Internet traffic, thus their study generally based on TCP flows 0.4 while UDP flows was ignored. Therefore, we will present our 0.3 profiling methodology of UDP flows on the basis of Claffy’s the ratio between udp and ip 0.2 parameter flow model in this paper. 0.1 III. PROFILING METHODOLOGY FROM THE VIEW OF THE 0 0:05:57 4:25:57 11:15:57 14:35:58 18:05:58 21:35:58 NETWORK LAYER time A. Date set Fig. 1: The proportion of the UDP’s packets and bytes We collected the data trace from a backbone router in China. The basic information of this trace is in Table I. The reason for not using the data set of CAIDA is that its payload information profiling methodology to measure the characteristics of has been encrypted. And in this paper, we must use the payload different applications. After our statistics and analysis, we information to classify the different applications. found there indeed exist significant differences between the profiling methodologies of different applications. We B. profiling methodology were unable to use a unified methodology to process UDP During the measurement based on flows, how to decide the flow as a whole. In contrast, we must use corresponding boundary of flow is critical. There exist two main approaches. profiling methodology based on the characteristics of The first, controlling flags(etc., FIN and RST) are used to applications. terminate the flow[7]. The second is the timeout strategy. If a The remainder of this paper is organized as follows: Section flow become inactive beyond a given timeout, it is deemed to 2 presents some related work on the profiling methodologies be ended and should be removed from memory. The timeout- of flows. In section 3, we discuss the profiling methodology based method does not rely on the explicit protocol labels in from the view of the network layer. In section 4, from the packet header, thus it can deal with the TCP flows and UDP view of the application layer, considering the great differences flows. In this case, although UDP is a connectionless protocol, between the different applications, we want to know whether Claffy’s parameter flow model is still applicable. What we exists a unified profiling methodology to process UDP flows. need to do is to find out the most appropriate parameters. At last, we conclude this paper and give some suggestions in We formally define UDP flows as a series of continuous Section 5. arriving UDP packets which have the same five-tuple(source address, destination address, source port, destination port, II. RELATED WORKS transport protocol), and their packets inter-arrival do not The studies of earlier traffic measurement took the packets exceed a specified fixed timeout. In this paper, we assume as the Building Block. But as D.Clark[5]pointed out that, our scenario is located in a backbone router. the traffic measurements based on packets can not reflect the Among the above six parameters, the selection of timeout relation between the packets and the higher-level information. is most difficult and important. The timeout determines the Thus, it can not meet the need for understanding network boundary of the flow, so it directly impact on the accuracy traffic in many ways. of flow measurement. If a smaller timeout value is selected, Jain[6] proposed a model named Packet Train. In this model, it can cause more long flows to be segmented into multiple a series of packets which have the same source and destination short flows. If a larger timeout is selected, it may result in more address will be identified as a packet train. If the interval unnecessary measurement overhead. For TCP, because it has between two packets exceeds a specified fixed timeout value, controlling flags which can make up the impact of setting then these packets are said to belong to separate train. a inappropriate timeout. But for UDP, the selection of the Claffy et al.[1][2] first propose a parameters flow model, in timeout is critical.
Load more

Recommended publications
  • Uila Supported Apps
    Uila Supported Applications and Protocols updated Oct 2020 Application/Protocol Name Full Description 01net.com 01net website, a French high-tech news site. 050 plus is a Japanese embedded smartphone application dedicated to 050 plus audio-conferencing. 0zz0.com 0zz0 is an online solution to store, send and share files 10050.net China Railcom group web portal. This protocol plug-in classifies the http traffic to the host 10086.cn. It also 10086.cn classifies the ssl traffic to the Common Name 10086.cn. 104.com Web site dedicated to job research. 1111.com.tw Website dedicated to job research in Taiwan. 114la.com Chinese web portal operated by YLMF Computer Technology Co. Chinese cloud storing system of the 115 website. It is operated by YLMF 115.com Computer Technology Co. 118114.cn Chinese booking and reservation portal. 11st.co.kr Korean shopping website 11st. It is operated by SK Planet Co. 1337x.org Bittorrent tracker search engine 139mail 139mail is a chinese webmail powered by China Mobile. 15min.lt Lithuanian news portal Chinese web portal 163. It is operated by NetEase, a company which 163.com pioneered the development of Internet in China. 17173.com Website distributing Chinese games. 17u.com Chinese online travel booking website. 20 minutes is a free, daily newspaper available in France, Spain and 20minutes Switzerland. This plugin classifies websites. 24h.com.vn Vietnamese news portal 24ora.com Aruban news portal 24sata.hr Croatian news portal 24SevenOffice 24SevenOffice is a web-based Enterprise resource planning (ERP) systems. 24ur.com Slovenian news portal 2ch.net Japanese adult videos web site 2Shared 2shared is an online space for sharing and storage.
    [Show full text]
  • What Is Peer-To-Peer File Transfer? Bandwidth It Can Use
    sharing, with no cap on the amount of commonly used to trade copyrighted music What is Peer-to-Peer file transfer? bandwidth it can use. Thus, a single NSF PC and software. connected to NSF’s LAN with a standard The Recording Industry Association of A peer-to-peer, or “P2P,” file transfer 100Mbps network card could, with KaZaA’s America tracks users of this software and has service allows the user to share computer files default settings, conceivably saturate NSF’s begun initiating lawsuits against individuals through the Internet. Examples of P2P T3 (45Mbps) internet connection. who use P2P systems to steal copyrighted services include KaZaA, Grokster, Gnutella, The KaZaA software assesses the quality of material or to provide copyrighted software to Morpheus, and BearShare. the PC’s internet connection and designates others to download freely. These services are set up to allow users to computers with high-speed connections as search for and download files to their “Supernodes,” meaning that they provide a How does use of these services computers, and to enable users to make files hub between various users, a source of available for others to download from their information about files available on other create security issues at NSF? computers. users’ PCs. This uses much more of the When configuring these services, it is computer’s resources, including bandwidth possible to designate as “shared” not only the and processing capability. How do these services function? one folder KaZaA sets up by default, but also The free version of KaZaA is supported by the entire contents of the user’s computer as Peer to peer file transfer services are highly advertising, which appears on the user well as any NSF network drives to which the decentralized, creating a network of linked interface of the program and also causes pop- user has access, to be searchable and users.
    [Show full text]
  • The Effects of Digital Music Distribution" (2012)
    Southern Illinois University Carbondale OpenSIUC Research Papers Graduate School Spring 4-5-2012 The ffecE ts of Digital Music Distribution Rama A. Dechsakda [email protected] Follow this and additional works at: http://opensiuc.lib.siu.edu/gs_rp The er search paper was a study of how digital music distribution has affected the music industry by researching different views and aspects. I believe this topic was vital to research because it give us insight on were the music industry is headed in the future. Two main research questions proposed were; “How is digital music distribution affecting the music industry?” and “In what way does the piracy industry affect the digital music industry?” The methodology used for this research was performing case studies, researching prospective and retrospective data, and analyzing sales figures and graphs. Case studies were performed on one independent artist and two major artists whom changed the digital music industry in different ways. Another pair of case studies were performed on an independent label and a major label on how changes of the digital music industry effected their business model and how piracy effected those new business models as well. I analyzed sales figures and graphs of digital music sales and physical sales to show the differences in the formats. I researched prospective data on how consumers adjusted to the digital music advancements and how piracy industry has affected them. Last I concluded all the data found during this research to show that digital music distribution is growing and could possibly be the dominant format for obtaining music, and the battle with piracy will be an ongoing process that will be hard to end anytime soon.
    [Show full text]
  • Piratez Are Just Disgruntled Consumers Reach Global Theaters That They Overlap the Domestic USA Blu-Ray Release
    Moviegoers - or perhaps more accurately, lovers of cinema - are frustrated. Their frustrations begin with the discrepancies in film release strategies and timing. For example, audiences that saw Quentin Tarantino’s1 2 Django Unchained in the United States enjoyed its opening on Christmas day 2012; however, in Europe and other markets, viewers could not pay to see the movie until after the 17th of January 2013. Three weeks may not seem like a lot, but some movies can take months to reach an international audience. Some take so long to Piratez Are Just Disgruntled Consumers reach global theaters that they overlap the domestic USA Blu-Ray release. This delay can seem like an eternity for ultiscreen is at the top of the entertainment a desperate fan. This frustrated enthusiasm, combined industry’s agenda for delivering digital video. This with a lack of timely availability, leads to the feeling of M is discussed in the context of four main screens: being treated as a second class citizen - and may lead TVs, PCs, tablets and mobile phones. The premise being the over-anxious fan to engage in piracy. that multiscreen enables portability, usability and flexibility for consumers. But, there is a fifth screen which There has been some evolution in this practice, with is often overlooked – the cornerstone of the certain films being released simultaneously to a domestic and global audience. For example, Avatar3 was released entertainment industry - cinema. This digital video th th ecosystem is not complete without including cinema, and in theaters on the 10 and 17 of December in most it certainly should be part of the multiscreen discussion.
    [Show full text]
  • Forescout Counteract® Endpoint Support Compatibility Matrix Updated: October 2018
    ForeScout CounterACT® Endpoint Support Compatibility Matrix Updated: October 2018 ForeScout CounterACT Endpoint Support Compatibility Matrix 2 Table of Contents About Endpoint Support Compatibility ......................................................... 3 Operating Systems ....................................................................................... 3 Microsoft Windows (32 & 64 BIT Versions) ...................................................... 3 MAC OS X / MACOS ...................................................................................... 5 Linux .......................................................................................................... 6 Web Browsers .............................................................................................. 8 Microsoft Windows Applications ...................................................................... 9 Antivirus ................................................................................................. 9 Peer-to-Peer .......................................................................................... 25 Instant Messaging .................................................................................. 31 Anti-Spyware ......................................................................................... 34 Personal Firewall .................................................................................... 36 Hard Drive Encryption ............................................................................. 38 Cloud Sync ...........................................................................................
    [Show full text]
  • In Re Xunlei Limited Securities Litigation 18-CV-00467-Amended
    Case 1:18-cv-00467-RJS Document 27 Filed 06/04/18 Page 1 of 35 UNITED STATES DISTRICT COURT SOUTHERN DISTRICT OF NEW YORK ) ) ) Case No. 18-cv-467 (RJS) ) In re XUNLEI LIMITED SECURITIES ) LITIGATION ) CLASS ACTION COMPLAINT ) ) ) JURY TRIAL DEMANDED ) AMENDED CLASS ACTION COMPLAINT Lead Plaintiffs Tongyan Wang and Yuyan Jia and additional named plaintiff Peng Li (collectively, “Plaintiffs”), individually and on behalf of all other persons similarly situated, by Plaintiffs’ undersigned attorneys, for Plaintiffs’ complaint against Defendants, allege the following based upon personal knowledge as to Plaintiffs and Plaintiffs’ own acts, and information and belief as to all other matters, based upon, inter alia, the investigation conducted by and through Plaintiffs’ attorneys, which included, among other things, a review of the Defendants’ public documents, conference calls and announcements made by Defendants, United States Securities and Exchange Commission (“SEC”) filings, news articles and press releases published by and regarding Xunlei Limited (“Xunlei” or the “Company”) and its cryptocurrency program, analysts’ reports and advisories about the Company, and information readily obtainable on the Internet. Plaintiffs believe that substantial evidentiary support will exist for the allegations set forth herein after a reasonable opportunity for discovery. NATURE OF THE ACTION 1. This is a federal securities class action on behalf of a class consisting of persons other than defendants who purchased or otherwise acquired Xunlei’s American Depositary Shares (“ADSs”) between October 10, 2017, and January 11, 2018, both dates inclusive (the “Class 1 Case 1:18-cv-00467-RJS Document 27 Filed 06/04/18 Page 2 of 35 Period”), seeking to recover damages caused by defendants’ violations of the federal securities laws and to pursue remedies under Sections 10(b) and 20(a) of the Securities Exchange Act of 1934 (the “Exchange Act”) and Rule 10b-5 promulgated thereunder, against the Company and its Chief Executive Officer (“CEO”), Lei Chen.
    [Show full text]
  • The Robustness of Content-Based Search in Hierarchical Peer to Peer Networks
    The Robustness of Content-Based Search in Hierarchical Peer to Peer Networks ∗ M. Elena Renda Jamie Callan I.S.T.I. – C.N.R. Language Technologies Institute and School of Computer Science Scuola Superiore Sant’Anna Carnegie Mellon University I-56100 Pisa, Italy Pittsburgh, PA 15213, US [email protected] [email protected] ABSTRACT Keywords Hierarchical peer to peer networks with multiple directory services Peer to Peer, Hierarchical, Search, Retrieval, Content-based, Ro- are an important architecture for large-scale file sharing due to their bustness effectiveness and efficiency. Recent research argues that they are also an effective method of providing large-scale content-based federated search of text-based digital libraries. In both cases the 1. INTRODUCTION directory services are critical resources that are subject to attack Peer to peer (P2P) networks have emerged in recent years as a or failure, but the latter architecture may be particularly vulnera- popular method of sharing information, primarily popular music, ble because content is less likely to be replicated throughout the movies, and software, among large numbers of networked com- network. puters. Computers (‘nodes’) in P2P networks can be information This paper studies the robustness, effectiveness and efficiency providers and information consumers, and may provide network of content-based federated search in hierarchical peer to peer net- services such as message routing and regional directory services. works when directory services fail unexpectedly. Several recovery A single node may play several roles at once, for example, posting methods are studied using simulations with varying failure rates. queries, sharing files, and relaying messages.
    [Show full text]
  • A Study of Peer-To-Peer Systems
    A Study of Peer-to-Peer Systems JIA, Lu A Thesis Submitted in Partial Fulfilment of the Requirements for the Degree of Master of Philosophy in Information Engineering The Chinese University of Hong Kong August 2009 Abstract of thesis entitled: A Study of Peer-to-Peer Systems Submitted by JIA, Lu for the degree of Master of Philosophy at The Chinese University of Hong Kong in June 2009 Peer-to-peer (P2P) systems have evolved rapidly and become immensely popular in Internet. Users in P2P systems can share resources with each other and in this way the server loading is reduced. P2P systems' good performance and scalability attract a lot of interest in the research community as well as in industry. Yet, P2P systems are very complicated systems. Building a P2P system requires carefully and repeatedly thinking and ex- amining architectural design issues. Instead of setting foot in all aspects of designing a P2P system, this thesis focuses on two things: analyzing reliability and performance of different tracker designs and studying a large-scale P2P file sharing system, Xun- lei. The "tracker" of a P2P system is used to lookup which peers hold (or partially hold) a given object. There are various designs for the tracker function, from a single-server tracker, to DHT- based (distributed hash table) serverless systems. In the first part of this thesis, we classify the different tracker designs, dis- cuss the different considerations for these designs, and provide simple models to evaluate the reliability of these designs. Xunlei is a new proprietary P2P file sharing protocol that has become very popular in China.
    [Show full text]
  • Demystifying Commercial Content Delivery Networks in China
    CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. (2015) Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/spe.3464 SPECIAL ISSUE PAPER Demystifying commercial content delivery networks in China Hao Yin1,*,†,BoQiao1,YanLuo2,ChenTian3 and Y. Richard Yang4 1Dept.t of CS, Tsinghua University, Beijing, China 2Dept. of ECE, University of Massachusetts Lowell, Lowell, MA, USA 3Dept. of EIE, Huazhong Univ. of Sci. and Tech., Wuhan, China 4Dept. of CS, Yale University, New Haven, CT, USA SUMMARY Over the past decade, content delivery networks (CDNs) have attracted substantial Internet traffic and improved quality of experience for Internet users. However, the evolution of the Internet ecosystem, which is driven by underlying economic incentives and ever emerging technologies, posts great challenges to the existing commercial CDNs (CCDNs). Thoroughly understanding the CDN industry from different aspects including market choice, technology, performance, tendency and infrastructure is indispensable to future Internet. In this paper, we conduct the first comprehensive study of China’s CDNs using continuous, at-scale, content-driven measurements. Based on the massive amount of measurement data with multidimensional properties, we demystify the CCDNs in China and answer two important questions: (1) what is the develop- ment trend of CCDNs in China and (2) what are their unique characteristics. The answers to these questions have significant implications on CDN providers and users. Copyright © 2015 John Wiley & Sons, Ltd. Received 25 September 2014; Accepted 24 November 2014 KEY WORDS: CDN 1. INTRODUCTION 1.1. Motivation To bridge the gap between network applications and best effort Internet provider (IP) networks, the technique of content delivery networks (CDNs) has emerged and been developed over the last decade.
    [Show full text]
  • Online Piracy
    Online Piracy Christian Helmers 16 April 2015 1 / 31 Online piracy • Digital technologies: • Create, transform, copy data • Digital technologies: transmission, display and analysis of data through copying • Enabled large-scale online piracy • Piracy: illegal copying of copyrighted material online • Music • Movies • Ebooks/audiobooks • Software • Note: [Piracy] is not an offense of timeless character, universally definable by a priori criteria. It is far richer and trickier than that. (Johns, 2009:4) • Beware: often very ideological debate 2 / 31 P2P: Napster 3 / 31 P2P: The Pirate Bay 4 / 31 P2P: LimeWire Pirate Edition (\Not an equivalent, not a replacement, but the exact same thing") 5 / 31 P2P: More BitTorrent sites 6 / 31 Direct downloads: Megaupload 7 / 31 Direct downloads: many anonymous options 8 / 31 Cracking devices: Amazon's MOBI 9 / 31 Online piracy • Digital technologies: • P2P music sharing software: Napster, KaZaa, Morpheus, Grokster, BitTorrent sites, etc. • Direct downloads, file repositories • Cracking software • Enabled large-scale online piracy • Importance of digital platforms for sharing • File sharing market organized in private groups with tightly controlled access (non-profit) • Filehosters often for-profit 10 / 31 The economics of piracy (Klein et al., 2002) • Negative effect on copyright owner: • Pirated copies substitutes for legal downloads (but lower quality files may limit substitution) • Inability to price-discriminate • Inability to control prices • Positive effect on copyright owner: • Sampling (learning) • Space
    [Show full text]
  • Title: P2P Networks for Content Sharing
    Title: P2P Networks for Content Sharing Authors: Choon Hoong Ding, Sarana Nutanong, and Rajkumar Buyya Grid Computing and Distributed Systems Laboratory, Department of Computer Science and Software Engineering, The University of Melbourne, Australia (chd, sarana, raj)@cs.mu.oz.au ABSTRACT Peer-to-peer (P2P) technologies have been widely used for content sharing, popularly called “file-swapping” networks. This chapter gives a broad overview of content sharing P2P technologies. It starts with the fundamental concept of P2P computing followed by the analysis of network topologies used in peer-to-peer systems. Next, three milestone peer-to-peer technologies: Napster, Gnutella, and Fasttrack are explored in details, and they are finally concluded with the comparison table in the last section. 1. INTRODUCTION Peer-to-peer (P2P) content sharing has been an astonishingly successful P2P application on the Internet. P2P has gained tremendous public attention from Napster, the system supporting music sharing on the Web. It is a new emerging, interesting research technology and a promising product base. Intel P2P working group gave the definition of P2P as "The sharing of computer resources and services by direct exchange between systems". This thus gives P2P systems two main key characteristics: • Scalability: there is no algorithmic, or technical limitation of the size of the system, e.g. the complexity of the system should be somewhat constant regardless of number of nodes in the system. • Reliability: The malfunction on any given node will not effect the whole system (or maybe even any other nodes). File sharing network like Gnutella is a good example of scalability and reliability.
    [Show full text]
  • Scaling Content Delivery Outline P2P System
    3/17/2019 Problem: Scaling Content Delivery • Millions of clients server and network meltdown 15-441/641: Content Delivery Peer-to-Peer 15-441 Spring 2019 Profs Peter Steenkiste & Justine Sherry Fall 2019 https://computer-networks.github.io/sp19/ Outline P2P System • Peer-to-peer • Overlays: naming, addressing, and routing • CDNs • Leverage the resources of client machines (peers) • Computation, storage, bandwidth 3 4 1 3/17/2019 P2P Definition Why p2p? • Harness lots of spare capacity Distributed systems consisting of interconnected • 1 Big Fast Server: $10k/month++ versus 1000s .. 1000000s clients: $ ?? nodes able to self-organize into network topologies with the purpose of sharing resources such as • Capacity grows with the number of users! content, CPU cycles, storage and bandwidth, • Build very large-scale, self-managing systems capable of adapting to failures and accommodating • Same techniques useful for companies, transient populations of nodes while maintaining • E.g. Akamai’s 14,000+ nodes, Google’s 100,000+ nodes acceptable connectivity and performance, without requiring the intermediation or support of a global • But: servers vs. arbitrary nodes, hard vs. soft state (backups vs caches), …. centralized server or authority. • Also: security, fairness, freeloading, .. • No single point of failure – A Survey of Peer-To-Peer Content Distribution Technologies, • Some nodes go down – others take over Androutsellis-Theotokis and Spinellis • … government shuts down nodes – peers in other countries are available 6 P2P Construction Key Idea: Network Overlay P2P Overlay Network • A network overlay is a network that is layered on top of the Internet • Simplified picture: overlays use IP as their datalink layer • Overlays need the equivalent of all the functions IP networks need: Clients • Naming and addressing Servers • Routing SPRINT • Bootstrapping Verizon • Security, error recovery, etc.
    [Show full text]