Broadband Internet Performance: a View from the Gateway
Total Page：16
File Type：pdf, Size：1020Kb
Broadband Internet Performance: A View From the Gateway Srikanth Sundaresan Walter de Donato Nick Feamster Georgia Tech University of Napoli Federico II Georgia Tech Atlanta, USA Napoli, Italy Atlanta, USA [email protected] [email protected] [email protected] Renata Teixeira Sam Crawford Antonio Pescapè CNRS/UPMC Sorbonne Univ. SamKnows University of Napoli Federico II Paris, France London, UK Napoli, Italy [email protected] [email protected] [email protected] ABSTRACT developing performance-testing metrics for access providers [5,15, We present the first study of network access link performance mea- 35]. Policymakers, home users, and Internet Service Providers sured directly from home gateway devices. Policymakers, ISPs, (ISPs) are in search for better ways to benchmark home broadband and users are increasingly interested in studying the performance Internet performance. of Internet access links. Because of many confounding factors in a Benchmarking home Internet performance, however, is not as home network or on end hosts, however, thoroughly understanding simple as running one-time “speed tests”. There exist countless access network performance requires deploying measurement in- tools to measure Internet performance [7,14,29,32]. Previous work frastructure in users’ homes as gateway devices. In conjunction has studied the typical download and upload rates of home access with the Federal Communication Commission’s study of broad- networks [12, 24]; others have found that modems often have large band Internet access in the United States, we study the throughput buffers [24], and that DSL links often have high latency [26]. These and latency of network access links using longitudinal measure- studies have shed some light on access-link performance, but they ments from nearly 4,000 gateway devices across 8 ISPs from a de- have typically run one-time measurements either from an end-host ployment of over 4,200 devices. We study the performance users inside the home (from the “inside out”) or from a server on the achieve and how various factors ranging from the user’s choice of wide-area Internet (from the “outside in”). Because these tools run modem to the ISP’s traffic shaping policies can affect performance. from end-hosts, they cannot analyze the effects of confounding fac- Our study yields many important findings about the characteristics tors such as home network cross-traffic, the wireless network, or of existing access networks. Our findings also provide insights into end-host configuration. Also, many of these tools run as one-time the ways that access network performance should be measured and measurements. Without continual measurements of the same ac- presented to users, which can help inform ongoing broader efforts cess link, these tools cannot establish a baseline performance level to benchmark the performance of access networks. or observe how performance varies over time. This paper measures and characterizes broadband Internet per- formance from home gateways. The home gateway connects the Categories and Subject Descriptors home network to the user’s modem; taking measurements from this C.2.3 [Computer-Communication Networks]: Network Opera- vantage point allows us to control the effects of many confounding tions—Network Management; C.2.3 [Computer-Communication factors, such as the home wireless network and load on the mea- Networks]: Network Operations—Network Operations surement host (Section 4). The home gateway is always on; it can conduct unobstructed measurements of the ISP’s network and ac- General Terms count for confounding factors in the home network. The drawback Management, Measurement, Performance to measuring access performance from the gateway, of course, is Keywords that deploying gateways in many homes is incredibly difficult and expensive. Fortunately, we were able to take advantage of the on- Access Networks, Broadband Networks, BISMark, Benchmarking going FCC broadband study to have such a unique deployment. We perform our measurements using two complementary de- 1. INTRODUCTION ployments; the first is a large FCC-sponsored study, operated by Of nearly two billion Internet users worldwide, about 500 mil- SamKnows, that has installed gateways in over 4,200 homes across lion are residential broadband subscribers [19]. Broadband pene- the United States, across many different ISPs. The second, BIS- tration is likely to increase further, with people relying on home Mark, is deployed in 16 homes across three ISPs in Atlanta. The connectivity for day-to-day and even critical activities. Accord- SamKnows deployment provides a large user base, as well as di- ingly, the Federal Communication Commission (FCC) is actively versity in ISPs, service plans, and geographical locations. We de- signed BISMark to allow us to access the gateway remotely and run repeated experiments to investigate the effect of factors that we could not study in a larger “production” deployment. For ex- Permission to make digital or hard copies of all or part of this work for ample, to study the effect of modem choice on performance, we personal or classroom use is granted without fee provided that copies are were able to install different modems in the same home and con- not made or distributed for profit or commercial advantage and that copies duct experiments in a controlled setting. Both deployments run a bear this notice and the full citation on the first page. To copy otherwise, to comprehensive suite of measurement tools that periodically mea- republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. sure throughput, latency, packet loss, and jitter. SIGCOMM’11, August 15–19, 2011, Toronto, Ontario, Canada. We characterize access network throughput (Section 5) and la- Copyright 2011 ACM 978-1-4503-0797-0/11/08 ...$10.00. 134 tency (Section 6) from the SamKnows and BISMark deployments. We explain how our throughput measurements differ from com- mon “speed tests” and also propose several different latency met- rics. When our measurements cannot fully explain the observed behavior, we model the access link and verify our hypotheses using controlled experiments. We find that the most significant sources of throughput variability are the access technology, ISPs’ traffic shap- ing policies, and congestion during peak hours. On the other hand, (a) DSL. latency is mostly affected by the quality of the access link, modem buffering, and cross-traffic within the home. This study offers many insights into both access network perfor- mance and the appropriate measurement methods for benchmark- ing home broadband performance. Our study has three high-level lessons, which we expand on in Section 7: • ISPs use different policies and traffic shaping behavior that can (b) Cable. make it difficult to compare measurements across ISPs. • There is no “best” ISP for all users. Different users may prefer Figure 1: Access network architectures. different ISPs depending on their usage profiles and how those ISPs perform along performance dimensions that matter to them. • A user’s home network equipment and infrastructure can signif- also cannot isolate confounding factors, such as whether the user’s icantly affect performance. own traffic is affecting the access-link performance. From inside home networks. The Grenouille project in France [1] As the first in-depth analysis of home access network perfor- measures the performance of access links using a monitoring mance, our study offers insights for users, ISPs, and policymakers. agent that runs from a user’s machine inside the home network. Users and ISPs can better understand the performance of the ac- Neti@Home [23] and BSense [4] also use this approach, although cess link, as measured directly from the gateway; ultimately, such these projects have fewer users than Grenouille. PeerMetric [25] a deployment could help an ISP differentiate performance prob- measured P2P performance from about 25 end hosts. Installing lems within the home from those on the access link. Our study also software at the end-host measures the access network from the informs policy by illustrating that a diverse set of network metrics user’s perspective and can also gather continuous measurements of ultimately affect the performance that a user experiences. The need the same access link. Han et al. [18] measured access network per- for a benchmark is clear, and the results from this study can serve formance from a laptop that searched for open wireless networks. as a principled foundation for such an effort. This approach is convenient because it does not require user inter- vention, but it does not scale to a large number of access networks, 2. RELATED WORK cannot collect continuous measurements, and offers no insights into This section presents related work; where appropriate, we com- the specifics of the home network configuration. pare our results to these previous studies in Sections 5 and 6. Other studies have performed “one-time” measurements of access-link performance. These studies typically help users trou- From access ISPs. Previous work characterizes access networks bleshoot performance problems by asking the users to run tests using passive traffic measurements from DSL provider networks in from a Web site and running analysis based on these tests. Net- Japan [8], France [33], and Europe [26]. These studies mostly fo- alyzr [29] measures the performance of commonly used protocols cus on traffic patterns and application usage, but they also infer the using a Java applet that is launched from the client’s browser. Net- round-trip time and throughput of residential users. Without active work Diagnostic Tool (NDT) [7] and Network Path and Applica- measurements or a vantage point within the home network, how- tion Diagnostics (NPAD) [14] send active probes to detect issues ever, it is not possible to measure the actual performance that users with client performance. Glasnost performs active measurements receive from their ISPs, because user traffic does not always satu- to determine whether the user’s ISP is actively blocking BitTorrent rate the user’s access network connection.