Comparing Broadband ISP Performance Using Big Data from M-Lab
1 Comparing Broadband ISP Performance using Big Data from M-Lab Xiaohong Deng, Yun Feng, Thanchanok Sutjarittham, Hassan Habibi Gharakheili, Blanca Gallego, and Vijay Sivaraman Abstract—Comparing ISPs on broadband speed is challeng- remote areas served by low-capacity (wired or wireless) infras- ing, since measurements can vary due to subscriber attributes tructure will compare poorly to an ISP-B whose subscribers such as operation system and test conditions such as access are predominantly city dwellers connected by fiber; yet, it capacity, server distance, TCP window size, time-of-day, and network segment size. In this paper, we draw inspiration from could well be that ISP-A can provide higher speeds than ISP- observational studies in medicine, which face a similar challenge B to every subscriber covered by ISP-B! The comparison bias in comparing the effect of treatments on patients with diverse illustrated above arising from disparity in access capacity is characteristics, and have successfully tackled this using “causal but one example of many potential confounding factors, such inference” techniques for post facto analysis of medical records. as latency to content servers, host and server TCP window size Our first contribution is to develop a tool to pre-process and visualize the millions of data points in M-Lab at various time- settings, maximum segment size in the network, and time-of- and space-granularities to get preliminary insights on factors day, that directly bias measurement test results. Observational affecting broadband performance. Next, we analyze 24 months studies therefore need to understand and correct for such biases of data pertaining to twelve ISPs across three countries, and to ensure that the comparisons are fair.
[Show full text]