CheesePi: Measuring Home Network Performance Using Dedicated Hardware Devices BINIAM GEBREGERGS GUULAY Master’s Thesis at KTH Laboratory of Communication Networks Supervisors at SICS: Dr. Liam McNamara and Dr. Ian Marsh Examiner: Prof. Gunnar Karlsson TRITA-EE 2015:114 Abstract Internet users may not get the service quality promised by their providers, and also may not know what service they can receive. When users experience poor Internet connection performance, it is not easy to identify the source of the problem. We develop CheesePi, a distributed measurement system that measures the Internet connection experience of home users based on some net- work performance attributes (e.g. latency, packet loss rate, and WiFi signal quality). The CheesePi runs on a Raspberry Pi (a credit card sized computer) connected to the user’s home network as a measurement agent. It is important to measure the network performance from the user’s side since it is difficult to measure each individual’s link from the operator (provider) side. Each measurement agent conducts measurement periodically without disturbing the user’s Internet quality. Measurements are con- ducted during popular media events from SICS (Swedish Insti- tute of Computer Science) and student accommodations. The measurement results show customers with an Ethernet connection experienced significantly better latency and packet loss compared to WiFi users. In most of the measurements users at SICS per- ceived better latency and packet loss compared to the users at the student accommodation. We also quantify how customers experi- enced lower performance when streaming from websites which do not use CDN technology compared to the websites which do use CDN, particularly during popular media events. Sammanfattning Internetanvändare får kanske inte den kvalitet på deras inter- netuppkoppling som leverantörerna låvar, och vet kanske inte hel- ler hur bra uppkoppling de potentiellt kan få. När en användare upplever dålig prestanda på internetuppkopplingen så är det svårt att identifiera vad som är källan till problemet. Vi har utvecklat CheesePi, ett distribuerat mätningssystem som mäter hemmaan- vändarens upplevelse av internetuppkopplingen baserat på nät- verksegenskaper (t.ex. latens, andel förlorade paket, styrkan på WiFi signalen). CheesePi kör på en Raspberry Pi (en dator stor som ett kreditkort) som är kopplad till användarens hemnätverk som en mätningsagent. Det är viktigt att mäta nätverksprestanda från användarens sida eftersom det är svårt att mäta varje indi- vids länk från leverantörens sida. Varje mätningsagent genomför mätningar periodiskt utan att störa användarens internetkvalitet. Mätningar genomförs under stora media-event från SICS (Swe- dish Institute of Computer Science) och studentlägenheter. Mät- ningsresultaten visar att användare som använder Ethernet för sin uppkoppling upplever en märkbart bättre latens och färre för- lorade paket jämfört med WiFi-användare. I de flesta mätningar upplevdes bättre latens och färre förlorade paket hos använda- re i SICS jämfört med i studentlägenheter. Vi kvantifierar också hur användare upplevde sämre prestanda när de strömmade in- nehåll från webbplatser som inte använder sig av CDN teknologi jämfört med webbplatser som använder CDN, speciellt vid stora media-event. Acknowledgements First and foremost thanks to the almighty God for everything. I am so grateful to my supervisors Dr. Liam McNamara and Dr. Ian Marsh for giving me the opportunity to work on this master thesis, and the continuous support, guidance, and constructive suggestions throughout the whole thesis. I would like to express my gratitude to Prof. Gunnar Karlsson for being supportive and patient as my examiner for this master thesis. I want to extend my sincere thanks and appreciation to my family and friends for their unconditional support and love. This master program would have never been possible without your support and encouragement. I also would like to thank SICS for providing me an office and all necessary materials to accomplish this project. And many thanks goes to other master thesis students Urban Peterson and Gustaf Lindstedt for all the fruitful dis- cussions, especially Gustaf for translating the abstract part of this report to Swedish. Contents List of Tables List of Figures 1 Introduction 1 1.1 Motivation . 1 1.2 Goals . 2 1.3 Organization of the report . 2 2 Background 3 2.1 Internet Protocol (IP) Networks . 3 2.2 Access technologies . 5 2.2.1 Ethernet . 5 2.2.2 WiFi . 6 2.2.3 Mobile/cellular connection . 6 2.3 Network performance measurements . 6 2.4 Network performance measurement tools . 7 2.4.1 Ping . 7 2.4.2 Httping . 8 2.4.3 Traceroute . 8 2.4.4 Measurement device . 8 2.4.5 Storage . 9 2.5 Content delivery approaches . 9 2.5.1 Single server . 9 2.5.2 Multiple servers . 9 2.5.3 Content delivery network (CDN) . 10 3 State of the art within network monitoring 11 3.1 RIPE Atlas . 11 3.2 ARK . 11 3.3 Samknows . 12 3.4 NetBeez . 12 3.5 Software based network monitoring systems . 13 3.5.1 SpeedTest . 13 3.5.2 ICSI Netalyzr . 14 3.5.3 Fathom . 14 3.5.4 PingER . 14 4 Methodology: CheesePi development and experiment con- ducting approaches 15 4.1 CheesePi architecture . 15 4.2 ICMP-based measurements . 16 4.3 HTTP-based measurements . 17 4.4 Traceroute measurement . 18 4.5 Database design . 18 4.5.1 Measurement agent design I . 18 4.5.2 Measurement agent design II . 20 4.5.3 The Central Server Design . 21 4.6 Sampling of network packets . 22 4.7 Measurement frequency scheduling . 23 4.8 Data analysis tools . 24 4.9 Measurement agent’s location . 25 5 Network performance evaluation 27 5.1 Measurement motivation . 27 5.2 Popular media events . 28 5.2.1 Boxing - fight of the century . 29 5.2.2 Eurovision 2015 . 35 5.2.2.1 Eurovision 2015 semi final one . 36 5.2.2.2 Eurovision semi final two . 42 5.2.2.3 Eurovision 2015 final . 44 5.2.2.4 One week after the Eurovision 2015 final . 48 5.2.3 2015 UEFA Champions League final . 51 6 Discussion 63 7 Conclusions 67 8 Future work 69 9 Lessons learned 71 Bibliography 73 A SICS open house 2015 77 List of Tables 4.1 Network measurement from an MA in SICS to www.bbc.com in different time intervals. 24 4.2 First few probes of the first five minutes network measurement from an MA in SICS to www.bbc.com. 24 4.3 Measurement agent’s location and Internet connection type. 25 5.1 HTTP latency to HBO from different MAs during boxing match between Mayweather and Pacquiao. 32 5.2 HTTP latency to HBO from SICS Ethernet one week later after the boxing match between Mayweather and Pacquiao. 33 5.3 Network and HTTP latency to Philstar from SICS Ethernet during and one week after the boxing match between Mayweather and Pacquiao. 34 5.4 HTTP latency comparison among different MAs during boxing match between Mayweather and Pacquiao. 35 5.5 Comparison of HTTP latency from SICS Ethernet to different tar- gets before, during, and after the program of Eurovision semi-final one. 38 List of Figures 2.1 Internet building blocks. 4 2.2 The TCP/IP Model. 5 4.1 CheesePi Architecture. 16 4.2 Ping and httping database design. 19 4.3 Traceroute database design. 20 4.4 The CheesePi dashboard. 21 4.5 Central server database design. 22 5.1 HTTP latency to different targets from SICS Ethernet during the boxing match between Mayweather and Pacquiao. 31 5.2 HTTP latency variability to HBO from SICS Ethernet during and one week later after the boxing match between Mayweather and Pacquiao. 33 5.3 Network and HTTP latency to Philstar from SICS Ethernet during the boxing match between Mayweather and Pacquiao. 34 5.4 HTTP latency to different targets from an MA at SICS (Ethernet) during Eurovision 2015 semi-final one. 37 5.5 HTTP latency from a MA at SICS (Ethernet) to EurovisionTV during Eurovision 2015 semi-final one. 39 5.6 Comparison of network and HTTP delays from a MA at SICS (Ethernet) to EurovisionTV during Eurovision 2015 semi-final one. 39 5.7 HTTP latency from a MA at SICS (Ethernet) to ORF during Eu- rovision 2015 semi-final one. 40 5.8 Comparison of HTTP latency to EurovisionTV from different MAs during Eurovision 2015 semi-final one. 41 5.9 Comparison of HTTP latency to SVT among different MAs during Eurovision Semi-final two. 43 5.10 Comparison of HTTP latency to EurovisionTV among different MAs during Eurovision Semi-final two. 44 5.11 Network and HTTP latency of different targets form SICS Ethernet during Eurovision 2015 final. 45 5.12 Network and HTTP latency of different targets form SICS Ethernet during Eurovision 2015 final. 46 5.13 Comparison of HTTP delay to EurovisionTV from different MAs during Eurovision 2015 final. 47 5.14 Comparison of HTTP delay to BBC iPlayer from different MAs during Eurovision 2015 final. 48 5.15 Comparison of HTTP delay from SICS Ethernet to different targets during Eurovision 2015 final and one week after the final. 49 5.16 Comparison of HTTP latency to EuroVisionTV from student apart- ment WiFi and SICS Ethernet between the measurements during Eurovision final and one week after the final. 50 5.17 HTTP latency from student apartment to EurovisionTV during different measurements. 51 5.18 Network, HTTP, and network packet loss relation of measurement from SICS Ethernet to ATDHE during UEFA 2015 Champions League final. 53 5.19 HTTP latency of different MAs to ATDHE during UEFA 2015 Champions League final. 55 5.20 Network latency of different MAs to ATDHE during UEFA 2015 Champions League final.