Behaviour of WebRTC in Non-optimal Networks Simon Johansson Computer Science and Engineering, master's level 2018 Luleå University of Technology Department of Computer Science, Electrical and Space Engineering Behaviour of WebRTC in Non-optimal Networks Simon Johansson Lule˚aUniversity of Technology Dept. of Computer Science, Electrical and Space Engineering 22nd February 2018 ABSTRACT The behaviour of WebRTC when the real-time communication with audio is done over a non-optimal network is investigated in this thesis. Different methods for collecting and analyzing data from an online survey are considered. A test environment was developed from which two online surveys would be conducted, where the outgoing packets had various interferences applied to them by the server. This was made in order to be able to simulate a non-optimal network e g WiFi. The partic- ipants are told to listen to various audio sequences and are asked to rate the quality as they perceive it. Although considered, video was not used in the surveys, as it would have increased the complexity of the surveys and increasing the risk of having the participants rejecting the surveys. Two independent surveys were conducted. The first survey utilized WebRTC for sending the audio, this was compared to the second survey which instead used Icecast. The result showed that WebRTC behaves well when there was only one type of in- terference added. Compared to Icecast it had lower performance. However, this could be contributed to the fact that two independent groups were used and the surveys had low participation rates. The surveys proved the feasibility of conducting online surveys for measuring perceived quality, although the participation rate was extremely low (2.8%), something that has to be considered when performing online surveys. iii PREFACE Firstly, I would like to thank Prof. Peter Parnes at Lule˚aUniversity of Technology for his help during this master thesis. Secondly, I want to thank Neava for helping me with my master thesis and to Staffan Johansson for all the ideas when designing the survey. And lastly, I want to thank my family and friends that have supported me during this project. Simon Johansson v CONTENTS Chapter 1 { Introduction 1 1.1 Introduction . 1 1.2 Goal . 2 1.3 Delimitations . 2 1.4 Thesis structure . 3 Chapter 2 { Background 5 2.1 Before WebRTC . 5 2.2 WebRTC . 5 2.2.1 Mediastream . 6 2.2.2 RTCPeerConnection . 6 2.2.3 RTCDataChannel . 6 2.2.4 Signaling . 6 2.3 Peerjs . 6 2.4 Icecast . 7 2.5 TC....................................... 7 2.6 Opus . 7 2.7 Related work . 9 Chapter 3 { Theory 11 3.1 Overview . 11 3.2 Mean opinion score . 11 3.3 Perceived Quality Survey . 12 3.4 Interference . 14 Chapter 4 { Methodology 15 4.1 Overview . 15 4.2 Pre-study . 16 4.3 Hardware and Software . 16 4.3.1 Streaming server . 17 4.3.2 Interference . 18 4.4 Pre-test . 19 4.5 Survey . 19 Chapter 5 { Evaluation 21 5.1 Pre-test . 21 5.2 Test . 23 5.3 Result . 23 Chapter 6 { Discussion 29 6.1 Goal . 30 6.2 Future work . 30 6.2.1 Tests . 31 6.2.2 Interference . 31 6.2.3 Participants . 31 6.2.4 Audio . 32 6.2.5 Video . 32 6.2.6 Test environment . 32 6.3 Conclusions . 33 Appendix A{ 35 A.1 Requirement specification . 35 A.2 Tables . 36 A.3 TC Commands . 37 1.3.1 Pre-test commands . 37 1.3.2 Test commands . 38 viii CHAPTER 1 Introduction 1.1 Introduction With its rise in popularity in the early 90s, the Internet has significantly changed the way we consume and distribute media. It allowed real-time communication between comput- ers and with the introduction of the web browser, it simplified the process for sending and receiving information. This set a great demand for new ways of communication between computers. Direct communication between two web browsers could be achieved with plugins[1], third- party programs, which allowed the browser to communicate directly with another browser without the need for a server to relay the information. However, plugins introduce secu- rity issues and were mostly incompatible with different browsers. Another problem is the prevalence of firewalls and NAT that limits the access of Internet traffic to computers in networks where such systems are deployed. This created pressure for the development of new standards for real-time communication, one of them being WebRTC (Web Real-Time Communication) which is being investigat- ing in this thesis. WebRTC is a collection of protocols and APIs first developed at Google and then subsequently released as an open source project back in 2011. It gives a set of standardized methods for sending audio, visuals and arbitrary data between computers in real-time. This removes the need for plugins and thus both simplifies the communica- tion and development of web applications. 1 2 Introduction The success of a new standard is determined by its ability to simplify development and improve user experience. To achieve this researchers have to perform rigorous testing to assure the quality. There exist many different ways of testing, ranging from automated tests using algorithms to determine the quality objectively, to manual participation of end users. In this thesis I focused on allowing university students to participate in a test method called Perceived Quality Survey (PQS). It gathers information about the end users experience by letting them be subjects either to a number of listening tests or watching a number of sequences being played, followed up by telling them to vote on the quality as they perceived it. 1.2 Goal The goal of this thesis is to investigate what kind of behaviour WebRTC has when the real-time communication is done over a network with non-optimal performance e g wireless networks. This is done by gathering students and letting them participate in two independent online Perceived Quality Surveys where the test is done with respect to induced packet loss and delay. 1.3 Delimitations A big part of the project is the gathering of raw data from the participants. This puts a lot of emphasis on how the survey is to be constructed. It is crucial to keep the survey as simple as possible to encourage the students that are participating to finish the survey. With that in mind, even though WebRTC is fully capable of both sending audio and vi- sual information between the peers, it was decided early on that in this thesis only audio would be considered for evaluation. The reason being that video is evaluated differently than audio and would increase the complexity of the surveys. Although having a large range of different kinds of sounds, ranging from pop to rock music, would contribute more to the overall picture, only one music sample was used in the thesis. Due to the implementation the only audio format that was available for testing was the .Wav-format, which has a low bit-rate, that could potentially have an effect on the participants regarding their perceived quality. In order to perform the surveys, there is a need to be able to produce a variety of different interferences to simulate, for instance, a wireless connection. Writing the in- terferences from the bottom up would be preferred, as it would yield better control and be tailored for the survey, this was deemed out of scope. Instead it was decided to use already existing software. 1.4. Thesis structure 3 Only two different types of interference were used in the surveys. The reason being that there is a limitation to what the software for creating the interferences is able to do, but also to reduce the complexity of the test-cases. It was decided to use packet loss and packet delay, since they are able to create gaps[2] in the audio stream which will affect the perceived quality. As the thesis focuses on the behaviour of WebRTC, no investigation is made on how WebRTC handles the added interference. The participants in the test groups were mainly university students. When perform- ing a perceived quality survey it is important to have a wide audience to be able to get a complete picture, but in this thesis only students are considered as it is easier to contact and to organize them. The two surveys were conducted by using two independent groups of students. This was done because it was unknown who participated in the first survey. 1.4 Thesis structure Chapter 2 gives details about the background for the surveys. Chapter 3 describes the theories that are being used in this thesis. Chapter 4 shows the general steps taken, from the planning all the way up to the implementation of the system and how everything is connected. Chapter 5 contains both the final test and result, including a pre-test that was made. Chapter 6 talks about potential future work and a final conclusion of the result in this thesis. CHAPTER 2 Background This chapter describes the background for this thesis. It starts off with a brief explanation of how real-time communication was achieved before and after WebRTC was introduced. Followed up is a short description of some of the technologies that are being used in this thesis. Lastly, there is a presentation over related works that have already been done. 2.1 Before WebRTC If you wanted to be able to send real-time media over the Internet via a web browser before WebRTC was developed, it would take considerably more development time as most of the things that WebRTC provides had to be implemented.