MASTER'S THESIS

OTT Video Measurements

Viktor Lindgren 2014

Master of Science in Engineering Technology Computer Science and Engineering

Luleå University of Technology Department of Computer Science, Electrical and Space Engineering OTT Video Measurements

Viktor Lindgren1

Supervisor: Fredrik Kers2 and Peter Parnes3

Luleå University of Technology

Master thesis in Computer Science

[email protected] [email protected] [email protected] ABSTRACT

Video streaming has become more and more popular in recent years. New protocols have emerged to improve the availability and simplify usage. The purpose of this thesis work is to measure video player's performance and make a model of the player's behavior. The intent of the model is to aid understanding of the problems with video playback, such as low video quality or interruption, in order to ease troubleshooting. To create a model video players were observed and measurement data, called metrics, was collected when they were playing video during dierent conditions. The metrics were found in the source code of the video player and with the help from the ocial documentation. Tests were performed with the ocial video player for three popular protocols: HTTP Live Streaming created by Apple, HTTP Dynamic Streaming created by Adobe and Microsoft Smooth Streaming. Characteristic and pattern were seen in the test results. The three examined protocols use adaptive bitrate to adjust the video quality according to the circumstances. Examples of circumstances can be an unstable network connection or when a too slow processor is being used. In the tests, parameters such as network bandwidth, video size and processor usage were tested in the web browser Safari. This thesis work suggests a simplied and general model for the three examined proto- cols aimed for testing the network connection when streaming video. PREFACE

This thesis was carried out at a company called Netrounds, during the rst half year of 2013. It was the nal part to nish my studies to a Master degree in Computer Science and Engineering at Luleå University of Technology(LTU). I would like to thank Netrounds for the opportunity to do this thesis and the support from my supervisors Fredrik Kers at Netrounds and Peter Parnes at LTU. I would also like to thank Envivo for the provided video les that was used in the tests.

Viktor Lindgren

May 2013 CONTENTS

Chapter 1 Introduction 1 1.1 Background...... 1 1.2 Problem description...... 4 1.3 Goals...... 5 1.4 Methodology and delimitations...... 5 1.4.1 Tools...... 5 1.5 Related work...... 6

Chapter 2 Theory 7 2.1 HTTP adaptive bitrate streaming...... 7 2.2 Adaptive algorithm...... 8 2.3 Protocols...... 8 2.3.1 HTTP Live Streaming...... 8 2.3.2 HTTP Dynamic Streaming...... 10 2.3.3 Microsoft Smooth Streaming...... 11

Chapter 3 Implementation of video players 12 3.1 HTTP Live Streaming...... 12 3.2 HTTP Dynamic Streaming...... 13 3.3 Microsoft Smooth Streaming...... 13 3.4 HTTP Dynamic Streaming algorithm...... 14 3.4.1 Target buer...... 14 3.4.2 Switching to higher or lower bitrate level...... 14 3.4.3 Emergency rules...... 16

Chapter 4 Tests 18 4.1 Test environment...... 19 4.1.1 Video player settings...... 19 4.1.2 Video les...... 20

i CONTENTS

4.1.3 Graphs...... 20 4.2 Visibility...... 21 4.2.1 HLS...... 21 4.2.2 HDS...... 22 4.2.3 MSS...... 22 4.3 Sound test...... 23 4.4 Max video buer length...... 23 4.5 CPU load...... 24 4.6 Minimum bandwidth demanded for a bitrate level...... 24 4.7 An observed behavior for HLS...... 25 4.8 Bandwidth limit scenarios...... 25 4.8.1 HLS...... 27 4.8.2 HDS...... 29 4.8.3 MSS...... 32

Chapter 5 Evaluation 34 5.1 General model...... 34 5.2 Future work...... 35

Chapter 6 Discussion 36 6.1 Issues...... 36 6.2 Conclusion...... 37

References 42

ii Glossary

Progressive download Downloading a video and having the ability to play before the complete movie le is downloaded.

VOD Video On Demand. Start watching a chosen video when the user wants.

AHS Adaptive HTTP streaming. Created by 3GPP.

HAS HTTP Adaptive Steaming. Created by Open IPTV Forum.

HLS HTTP Live Streaming. Created by Apple.

HDS HTTP Dynamic Streaming. Created by Adobe.

MSS Microsoft Smooth Streaming.

DASH Dynamic Adaptive Streaming over HTTP. International standard from ITU.

OSMF Open Source Media Framework is a framework for developing video streaming in ash.

SMP Strobe Media Playback is a graphical interface on top of OSMF.

Video bitrate The amount of data used to describe the video, given in bits per second. It is a common measurement for video quality.

Bitrate level A xed target level when encoding the video. The actual bitrate can dier slightly.

Video player metrics Measured data from the video player describing the state of the video player and how it performs.

Video segment A video splitted into multiple parts contained in multiple les. Those les are called segments.

Video fragment A video splitted into multiple parts contained in a single le. A fragment is identied by its byte range within the le.

Media codec A specic format to encode the video in for storage and transport.

Media encoding The process to convert the video into a specic format.

Media decoding The process to convert an encoded video le into a viewable video stream. Normally done in the video player. CHAPTER 1

INTRODUCTION

Video streaming is a convenient way to watch video. The user is able to almost instantly watch a selected movie over a network connection. That is called Video On Demand(VOD). The user can use dierent kinds of devices and platforms to stream video. It can be a PC, smartphone or tablet. The screen size on these devices varies and thereby dierent video resolution is needed to maintain the best possible quality. To give the user the best possible experience; the video quality should be high and the video should play smoothly without interruptions. It is not always this can be achieved. A possible problem is that the network connection is unable to handle high video quality as high resolution requires higher network capacity. The user will notice the problem when the video playback stops, and the video player waits for more parts of the movie to nish downloading. New technologies have emerged to address these issues. They can adjust the video quality to the network connection's capacity and performance. The performance may vary due to many reasons, for example the device uses a wireless connection, or the network connection is shared with other users. This thesis work aimed to help understand the problems with the video playback in the new protocols.

1.1 Background

Over-The-Top Content(OTT) can be described as delivering video through an Internet connection to the end-users were the producer and the Internet Service Provider(ISP) is not the same company. Examples of producers are SVT Play, Netix and Hulu. The user can for example access the website of the OTT producer and directly stream video through the Internet. As the producer does not have full control of the Internet connection, they can not guarantee delivery of the video to the end-user. Therefore, it would be good if a video streaming protocol can handle those situations and adapt the video quality.

Data transmission protocols On network level when you want to send data from one computer to another computer, there is a chance that the data will not reach the other computer or that the data has

1 1. Introduction been corrupted. There is a protocol called Transmission Control Protocol(TCP) that is designed to handle those situations. TCP uses a checksum to verify that the data is not corrupted and in that case sends an acknowledgment message back to the sender that say everything was received. If the initial sender does not recive the acknowledgment message after a certain time, the initial sender will resend the data. If the initial sender wants to send more data, the process will be done again. An option to TCP is the User Datagram Protocol(UDP), were there is no resending of data if something goes wrong. This is useful when old data is no longer relevant as the newer data has an updated value. It can, for example, be a position of someone location or when talking on the phone with someone. The resending of the data and the acknowledgment messages for TCP will cause some extra delay. Thus, UDP and TCP are suitable for dierent situations. Both UDP and TCP are widley used on the Internet.

History of video streaming protocols In the past, the Real-Time Transport Protocol(RTP) with Real-Time Streaming Proto- col(RTSP) have been used for streaming video. RTP is used for sending the video data with UDP and RTSP for controlling the data ow with TCP. RTSP controls video bitrate adjustments while playing a movie. The bitrate can be seen as the quality of the video. Later came Adobe's solution called Real Time Messaging Protocol(RTMP) that use TCP. RTMP provides the same functionality as RTP and RTSP. To put the protocols in time perspective, the proposed standard[1] for RTSP is dated back to 1998. The support for RTMP was introduced in Flash Player 6, released in 2002. The RTMP specications[2] was not publicly published until July 2009. The specications are needed to allow others to develop compatible video players. Newer video streaming protocols are based upon HTTP, that use TCP. HTTP stands for Hypertext Transfer Protocol and is used on the web. It is the protocol that the web browsers, such as Firefox, use to communicate with web servers. On a computer that runs a web server application, there can be more than one application server running. Therefore, in each connection to the server, a port number must be given in order to pass through the data to the right application. The port number for HTTP is usually 80. The newer protocols based upon HTTP will be called HTTP adaptive bitrate streaming in this report. The protocols make use of already existing ecosystem around HTTP with web server software and Content Delivery Network(CDN). The CDNs are used for caching content, for example video les for a movie, on servers closer to the end-users in order to enable higher transfer speed. Many OTT producers use CDNs.

Reachability All the mentioned video streaming protocols dier slightly how they reach the end-user over a network connection. Both RTMP and RTP together with RTSP face diculties to reach the end-users as there are rewalls and Network Address Translation(NAT) pre- venting the network trac to pass through. Firewalls are used to lter network trac as a security mechanism. Firewalls and NATs are common and can be found in home

2 1. Introduction routers connected to Internet. NAT is used for sharing one Internet Protocol(IP) address to multiple computers. The Internet provider may not give as many IP addresses as there are computers in the home to give them separate IP addresses, and therefore the NAT is needed. The home router needs to be congured to allow video streaming with RTP and RTSP. HTTP is normally already allowed to pass through rewalls and does not need to care about NAT on the client side. In some networks only HTTP trac is allowed to pass through the rewalls. As RTMP uses a dierent port and is not identied as HTTP trac, it is disallowed to pass. However, for RTMP there is a variant of it, Real Time Messaging Protocol Tunnel(RTMPT), that tunnels the data trac by encapsulating the RTMP data into HTTP and thereby allows the video to reach the end-users. RTMP does not require the client side to have the NAT congured. HTTP adaptive bitrate streaming have none of these problems.

HTTP streaming The basic idea of HTTP adaptive bitrate streaming is to divide the video into small parts and encode each part in multiple video qualities. The video qualities can have dierent resolution and video bitrate. The video bitrate is the amount of data that is used to describe the video each second. A video can be encoded to keep one xed bitrate, one quality level. In this report, it is also called a bitrate level.

Figure 1.1: The picture shows the timeline for a movie of ten minutes. The user is currently looking into the third minute and have a buer to the fth minute. The video buer's length is two minutes.

The video is usually divided into small parts by time intervals. For example, the rst part consist of the rst ten seconds of the movie and the next part consist of the succeeding ten seconds. The video player will download video parts from the web server during playback. The video player has the ability to choose what quality, video bitrate, it wants to request during playback. The choice can be made based upon historical events, like detected bandwidth on the network connection, the current state of the video buer and others reasons the player considers. The video player's buer holds the video that has been downloaded from the web server, but not yet played, as those parts lie ahead of the current position in the movie. This is illustrated in Figure 1.1. If the buer is

3 1. Introduction emptied the video player has to stop playback and wait for more parts of the video to nish downloading. This it is called that the video player is buering. An alternative to HTTP adaptive bitrate streaming is to use HTTP progressive down- load. Progressive download is when a single video le is directly streamed, downloading, from the server to the client, and the user can begin playback before the entire le has been completely downloaded. This is supported natively in the web browser. The key dif- ference, when comparing with HTTP adaptive bitrate streaming, is that only one bitrate level available and thus no shifting to another quality can occur.

Protocols based on HTTP HTTP adaptive bitrate streaming exist in various protocols. The main popular ones are:

ˆ HTTP Dynamic Streaming(HDS) created by Adobe for Flash[3].

ˆ HTTP Live Streaming(HLS) created by Apple, for Apple's devices and software.

ˆ Microsoft Smooth Streaming(MSS) developed for Silverlight[4].

Flash and Silverlight are web browser plugins that needs to be installed separately in order to use the video player. The web browser Chrome have Flash pre-installed. One more protocol has recently, in 2012, been standardized by the International Telecom- munication Union(ITU). The protocol is called Dynamic Adaptive Streaming over HTTP (DASH). The utilization, today, is low but as support in the web browser increases this might change. Both Firefox[5] and Chrome[6] has begun to implement support. For now, there is no royalty for DASH but the question is not yet determined, there might be a fee in the future[7]. HDS is used by companies like SVT, ABC, Discovery, ESPN, FOX and Warner Brothers for their Internet based video streaming services[8]. Netix and ViaPlay uses MSS. HLS is used on Youtube for their live event streaming service. Of the four mentioned protocols, Microsoft released the rst HTTP adaptive bitrate streaming protocol, MSS, in October 2008. In mid 2009 Apple introduced HLS for iOS 3.0. Adobe released HDS late 2009. The Moving Picture Experts Group(MPEG) decided start work on DASH in April 2009 by requesting proposals to consider[9] as a standard. In April 2012 the DASH standard[10] was nished. Some of the protocols do not have a nished standard. For HLS, there is still only drafts available, and codec support for H.264 in DASH is coming in DASH-264[11]. H.264 is a widely used standard in the media industry. The standard describes how the video is encoded and decoded by the video player. HLS, HDS and MSS, have support for H.264.

1.2 Problem description

When a user watch a video online they may experience that the video player is buering or that the video quality is bad. One reason for that can be slow network performance

4 1. Introduction or a problem at the computer. Such behavior is not wanted by most users and therefore it should be avoided. The user may contact the OTT producer, for example SVT, for troubleshooting. This thesis wants to help the OTT producer so that they can identify and solve the problem; What should they look for and whether is it possible to extract any useful information from the video player that can help.

1.3 Goals

A set of goals for this thesis was set up and is described as follows:

1. Monitor the video player performance. To be able to verify that a problem exists and to see if actions taken had any eect.

2. Identify parameters that aect the video quality and the playback, so that useful improvements can be made.

3. Create a model to predict the future of the video players behavior. The model should help explaining the video players behavior.

1.4 Methodology and delimitations

The rst goal in Section 1.3 was intended to be achieved by investigating what metrics is available in the video player. A metric is data on the video player's state. The second goal was planned to be achieved by running tests and varying parameters, for example avaliable bandwidth. The third goal was planned to be achieved by studying the test results and nd connections. The focus was on the three popular protocols HDS, HLS and MSS using their ocial supported video player for PC.

1.4.1 Tools To be able to extract metrics from the players and test them they needed to be set up for respective protocol. The video player used for HDS was the Open Source Me- dia Framework(OSMF)[12], which is the base for streaming media in Flash. OSMF was combined with Strobe Media Playback(SMP)[13] for showing the graphical interface. For HLS, there is native support in the web browser Safari by using HTML and JavaScript, both are programming languages for building web pages. For MSS, a project, Health Monitor[14], was used. It provides more metrics than the base, Microsoft Media Platform: Player Framework(MMPPF)[15]. The Health Monitor project is written in the program- ming language C#. For programming, the Adobe's Flash Builder[16] 4.7 and Microsoft Visual Studio[17] 2010 with Silverlight 5 SDK[18] tools were used.

5 1. Introduction

1.5 Related work

There already exist some testing on the three protocols. Akhshabi, Begen, Ali and Dovrolis[19] made an article about thier tests with the MSS ocial player and the Net- ix player. They did also test the performance of two competing MSS video players that stream video at the same time, using a shared network connection. Mueller, Lederer and Timmerer[20] made an article about thier tests with a DASH video player against one for HLS, HDS and MSS in a special scenario. The scenario was constructed by monitoring the network performance during a car drive with a phone connected to Internet using 3G. Their test result was that the MSS video player performs best when considering the average video quality, the number of switches in qualities and how long time the video player was buering. In their tests from all the previous mentioned articles, they use older versions of the video players than currently available. In newer versions of the video players things have changed. This might invalidate their ndings when using new versions of players. There also exist some research on how the optimal switching quality algorithm for HTTP adaptive bitrate streaming could be constructed. One attempt is made by Miller, Quacchio, Gennari and Wolisz[21], where they test their algorithm with a DASH imple- mentation.

6 CHAPTER 2

THEORY

This chapter describes how HTTP adaptive bitrate streaming works and specic details about the three protocols HLS, HDS and MSS is described.

2.1 HTTP adaptive bitrate streaming

The web server hosting the video, has a set of les for each video. The video itself and metadata les describing video settings and options. The video is encoded into multiple bitrate levels making the video available in dierent qualities. The video is then divided into multiple parts, by time or data intervals. Each part can be stored either as a separate le, segments, or as a single le using a byte-range as delimiter. In the latter case, they are called fragments. When a user start a movie, the video player acts as a client and start by requesting the metadata les from the web server. The metadata les contains information about the media, what video bitrates levels are available, required supported codec, subtitles, security settings and other options. The included options is up to the respective protocol to decide. The client makes a decision about which bitrate level to start with and downloads the rst video parts of the choosen bitrate level. One option is to start with the lowest possible bitrate level available and then increase to higher if there is enough bandwidth. Which bitrate it starts downloading depends on the algorithm implemented in the video player. The algorithm thus has a large impact on the playback experience. The video player can have settings such as how much the video buer must be lled before it starts to play the video, and how fast it can adapt and change bitrate level. There is no negotiation between the client and the server about what bitrate level the video player should download. Thereby can the algorithm make real-time decisions. One parameter for that bitrate level selection can the CPU usage. The CPU usage can indicate that the processor is overloaded and is not able to handle higher bitrate level as it needs more process usage for that.

7 2. Theory

2.2 Adaptive algorithm

How algorithms to adjust the bitrate level should be designed is not trivial. Dierent approaches are benecial in dierent scenarios. The greedy approach that only looks at current available bandwidth will make it sensitive to fast variations on network connection. That will cause fast switches but also high utilization of the available bandwidth. If it instead takes historical events into consideration and build up an average value of the bandwidth, the result would be more stable. That can then be combined with a limit on how many steps in bitrate level it can maximum shift to get even more stable. The best would of course be to know exactly how the future would look like for better optimization of the algorithm.

2.3 Protocols

The three protocols HLS, HDS and MSS, are described with deeper insight here. There are more details for HLS as there is much information available[22]. The others protocols work in a similar way.

2.3.1 HTTP Live Streaming The client support for HLS is very diverse. As mentioned in Section 1.1, there is still only a draft available even though HLS have existed for many years. The best support can be found in Apple's own products, iOS and OS X, in the media player QuickTime[23] and the web browser Safari[24]. It is supported in Android 3[25] and later versions. A small test was done with Android 4.0 and the ocial example video stream[26]. The result was the video did not play well. Another popular video player called VLC[27] introduced support for HLS in version 2.0. It was also tested, with VLC 2.0.3, and the ocial example stream but only the sound worked. In version 2.0.4 and 2.0.5, neither the sound nor video worked. Also, there exists various HLS implementations for Flash such as Yospace HLS Streaming Player[28].

Metadata les The rst metadata le the HLS video player downloads is the variant playlist. It describes general information such as available video qualities and subtitles. An example variant playlist is illustrated in Figure 2.1. On each bitrate level in the variant playlist, the bandwidth attribute tells the maximum bitrate in the movie and an address to another metadata le, called the event playlist. In Safari, the client will choose the lowest bitrate level avaliable in the beginning. If resolution information is given for each bitrate level in the variant playlist, Safari will choose that level ts the monitor's resolution best. In the event playlist, see example in Figure 2.2, the movie is splitted by time intervals and a web address link to each segment is specied. Apple recommends using ten second

8 2. Theory

#EXTM3U

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=232370,CODECS="mp4a.40.2, avc1.4d4015" gear1/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=649879,CODECS="mp4a.40.2, avc1.4d401e" gear2/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=991714,CODECS="mp4a.40.2, avc1.4d401e" gear3/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1927833,CODECS="mp4a.40.2, avc1.4d401f" gear4/prog_index.m3u8

Figure 2.1: A simple variant playlist.

#EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:10, segment0.ts #EXTINF:10, segment1.ts #EXTINF:10, segment2.ts #EXT-X-ENDLIST

Figure 2.2: An example event playlist. Dividing video parts by segments of ten seconds. intervals[29]. The complete ow from the client's perspective of downloading the les can be seen in Figure 2.3. The variant playlist may provide a link to a decryption key for the video. The key make it possible to decrypt an encrypted video and thus only give authorized users ability to watch the video. HLS does not support the Digital Rights Management(DRM) security features but uses its own. DRM has become popular in the media industry as it is used for DVD and Blu-ray discs. When streaming live from an event, the event playlist behaves in a special way. It will only contain a few parts, segments or fragments, closest in time. The server is constantly updating the event playlist with new parts. The client will then, when it reaches the end of the event playlist request it again to see if there are any changes. It is desired to keep the event playlist le small in order to send less data as the client will frequently request

9 2. Theory

Figure 2.3: HLS stream ow. the le. Apple provides free of charge packaging tool for creating the metadata les and the necessary video les, called mediastreamsegmenter[30]. On the web server, there is no need for any extra modules as the protocol only uses standard HTTP features. In the HTTP standard, there is a Range[31] option to specic a byte-range, and it is used by HLS to request fragments.

Media encoding In the variant playlist, there is a codec attribute for specifying the media encoding. The protocol itself gives no restriction regarding encodings, and it depends on what the client's video player support. Apple's video player supports H.264 video with AAC or MP3 audio.

2.3.2 HTTP Dynamic Streaming Adobe's protocol, HDS, has support for both splitting the video into segments and fragments[32]. There is an extra metadata le only for the web server. The extra metadata le holds information about where each fragment in the segments is located. A request from a client for a video part will contain a segment and fragment number. The web server use the extra metadata le to map which bytes the fragment number corresponds to within the segment. This means a special module is needed for the web server that does this process. Adobe provides a module[33] for the web server software Apache[34]. To create HDS video les, one can free of charge use the f4fpackager[35] tool provided by Adobe. Adobe's Flash player supports VP6 with MP3 media encoding, and H.264 with

10 2. Theory

AAC audio. HDS has support for DRM protection.

2.3.3 Microsoft Smooth Streaming In the MSS protocol, there is a dierent approach than the other two by separating the video and audio into dierent les. The client then has two connections to the server[19] to fetching the media. As the audio and video are separated it allows for having the same video but with dierent audio tracks, which can be a director's commentary version of the movie. This makes it possible to only store the video once on the server and save storage. MSS for Silverlight supports H.264 video and AAC audio encoding, and VC-1 with WMA[36]. To create the video les for the web server one can use the Expression Encode Pro[37] tool from Microsoft. The Pro version is required and is not free of charge. Microsoft has at the end of this project released a plugin for Adobe's video player built on OSMF that makes it possible to play MSS[38] video. A small test shows that the video player does not play well by playing to fast and sometimes stops. On the server side, only Microsoft's own web server Internet Information Services(IIS)[39] is supported with the Media Services[40] module.

11 CHAPTER 3

IMPLEMENTATION OF VIDEO PLAYERS

The metrics avaliable from the video player can only be extracted by using an Application Programming Interface(API) that the publisher, or the developer, of the video player can provide. The API allows the programmer to see or change the behavior of an application. This chapter describes the development process for respective player and the extracted metrics. The adaptive algorithm for HDS was found in the source code and is described in Section 3.4. Some metrics was named as another metric in another video player. To be consistent, the follow metrics are dened as follows:

Throughput Transfer speed on the network connection between the server and the client.

Downloading bitrate The selected bitrate level from the metadata le the video player currently downloads and is inserted to the video buer.

Current viewing bitrate The selected bitrate level from the metadata le that is cur- rently playing and is taken from the video buer.

Playback bitrate The actual bitrate on the selected bitrate level that is playing. This value varies within a movie because some sequences are easier to compress than others. Thus lower bitrate can be used during that time. The playback bitrate is also known as Variable BitRate(VBR).

3.1 HTTP Live Streaming

Development began by creating a simple test web page starting from Apple's ocial exam- ples by streaming video directly from one of their servers and then testing functions from the Safari API[41]. As it turns out, not everything in the API worked as documented. For example the video buer size in seconds was not working at all, when playing HLS. With normal progressive download, were only one big single le is used, it works ne. A post

12 3. Implementation of video players on Apple Developer Forums[42] was made in order to get an answer to why the buer API was not working correctly. No response has been received at the end of the project. In the Safari API, there was no support for extracting the metrics mentioned in the introduction to this chapter. However, the API to show when the video player begins to buer and when it starts to play again worked, and thus the buering time could be measured. Overall, the API was very limiting. WHATWG has suggested adding new API:s specic for HTTP adaptive bitrate stream- ing to the web standard were, for example, the throughput, downloading bitrate, current viewing bitrate and playback bitrate are mentioned[43]. WHATWG stands for Web Hy- pertext Application Technology Working Group and is a community founded in 2004 by people at Apple, the Mozilla Foundation and Opera Software since World Wide Web Con- sortium(W3C) at that time did not focus on HTML but rather XHTML[44]. W3C is an international community who sets the web standards[45]. The mentioned companies be- hind WHATWG all have their own web browser. HTML is a markup language that is used to describe the layout of web pages. XHTML is an alternative to HTML.

3.2 HTTP Dynamic Streaming

The development with the OSMF player and the Graphical User Interface(GUI) called SMP started o with problems. The main problem was what version that was used of those two. Apparently the OSMF and SMP had merged into one project where a newer version of SMP was hidden, and this was not documented. The latest version of OSMF did not work correctly in Chrome version 24, and thus Firefox was used during development. OSMF provides a debug web page for streaming video. On the debug web page many metrics where found, and those were updated every second. The fragment throughput and the downloading bitrate was missing and were added by modifying the code. The buering times were calculated by adding the buering and play event API to JavaScript. Other interesting values on the debug web page was found, for example frames per seconds and number of dropped frames per second. Those two metrics shows if there are very small interruptions in the playback, which is not counted in the buering time. On the debug web page, information about the video buer, such as the size given in bytes or seconds, could be found.

3.3 Microsoft Smooth Streaming

There was no problem getting started with the Health Monitor project. From the player, the following metrics could be extracted:

ˆ Current viewing and downloading bitrate.

ˆ Average throughput.

13 3. Implementation of video players

ˆ Buering events, giving the possibility to calculate the buering times. Sadly the video buer size could not be found. In the source code for the project a setting was found for the size of the video buer at the beginning before it starts to play. It was set to two seconds even though the documentation on Microsoft's web page species it to ve seconds by default[46]. Others interesting metrics was found, for example system and process CPU usage, video and audio latency on the network connection. How these metrics was measured could not be found, neither on their website documenting the API nor in the source code.

3.4 HTTP Dynamic Streaming algorithm

An attempt to reach the goals was to investigate what the source code for the video player could tell about how the video players should behave. The OSMF source code gave a deep insight on the underlying algorithm in the HDS video player. For the other video players, this type of information is not avaliable. The source code for OSMF is pretty large and therefore was not all code studied. To nd what code should be studied the debugging tool in Flash Builder was used to understand the ow of the code under normal conditions. The algorithm described here has been simplied to make it easier to understand. A variable is indicated by using italic text on the name of the variable in this report.

3.4.1 Target buer The target buer is a setting in the video player. It is given in seconds and is a threshold value. It symbolizes when the video player should leave the buering state, which it does when it is lled to that amount of time in the video buer. For the rst time when loading a movie, the target buer is set to 0.1 seconds. This makes the time before the video player starts playing short as few video parts actually have to be downloaded before the playback begins. When the video is playing and suddenly runs out of video buer, the target buer is directly set to ve seconds. Then the video player is buering and waits until ve seconds of playback time have been downloaded and inserted into the video buer. At the moment playback start again, the target buer is set to four seconds. That means the target buer will go between four and ve seconds depending on if the video player is buering or not. During playback with good network connection, the video buer is kept around the target buer value.

3.4.2 Switching to higher or lower bitrate level An up or down switch in the bitrate level can only occur when a fragment just has been downloaded. The algorithm in the video player will remember the fragment's informa- tion, like size and throughput, by putting it into a history list. Then the average band- width is calculated for the last maxF ragments fragments from the history list. The

14 3. Implementation of video players

maxF ragments variable is by default set to ten. Calculating the average bandwidth is done by summing the measured bandwidth for each fragment and divide that by the amount of fragments. Each fragment's bandwidth is weighted. By default, there are two weights; the rst is seven and the second is three. The last downloaded fragment is more important than the one before it with a ratio of 7:3. An actualBitrate is calculated by taking maxF ragments fragments from the history and dividing the sum the of fragment size in bytes by the sum of fragment duration in playback time. In other words, the actualBitrate can be described as the average Variable BitRate(VBR) during the playback time of the fragments. If the actualBitrate is more than the average bandwidth and there are more fragments in the video buer than bufferF ragmentsT hreshold, then idealBitrate is set to the actualBitrate. If that is not the case, the idealBitrate is set to the average bandwidth value and then set according to equation 3.1. The bufferF ragmentsT hreshold is set to two by default. In the case the idealBitrate is set to the actualBitrate means that the video buer is large enough for the algorithm to think that the bitrate actualBitrate is worth a try. That can cause an up switch, but if something goes wrong there is a video buer to cover for the misstake for a short time.

idealBitrate = actualBitrate + climbfactor · (idealBitrate − actualBitrate) (3.1)

The climbfactor in equation 3.1 is set to 0.9 by default. The idealBitrate is used to get the highest possible bitrate level available that is under the idealBitrate, call it the newLevel. If the newLevel is an up switch, it will be counted down until it respects the maxUpSwitchLimit threshold and that the level is not marked as unreliable. If the newLevel instead is a down switch, the newLevel will be counted up until it respects the maxDownSwitchLimit threshold. By default, the maxUpSwitchLimit is set to one, and the maxDownSwitchLimit is set to two. If the level is reliable or not will be decided upon the level's reliability. It is determined by the formula in equation 3.2.

2 downSwitchesF rom (3.2) reliability = 1 − decisionHistoryLength switchesF rom · b 2 c In equation 3.2, the decisionHistoryLength represent the number of decisions that has been made by the algorithm, and it includes decisions not to change level. Every time a fragment is downloaded a new value is added into the decision history list. The variable decisionHistoryLength is limited by maxDecisionHistory, set to 30 decisions by default. The switchesF rom represents the amount of decisions in the decision history list that is on the level we are deciding the reliability for. The downSwitchesF rom represents the amount of down switches from the level to a lower level in the decision history. There exists some conditions that must hold in order to the equation 3.2 to be valid:

15 3. Implementation of video players

ˆ The level cannot have been marked as emergency before, then the reliability is set to zero. A level is marked as emergency by the emergency rules described in Section 3.4.3.

ˆ decisionHistoryLength must be at least ve, if not the reliability is set to one.

ˆ switchesF rom must be larger than zero. That means the algorithm must have selected the level previously, before telling anything about it, if not the reliability is set to one.

ˆ If downSwitchesF rom is zero, it is directly counted as reliable. That means that no switching has occurred to a lower level earlier.

The reliability can thus be between zero and one. There is a minReliablity set to 0.85 by default and if reliability is higher than minReliablity it is marked as a reliable level. Otherwise, it is unreliable.

3.4.3 Emergency rules The ow in Section 3.4.2 is the normal switching of bitrate level when nothing dramatically happens. This can be overridden by a set of emergency rules. There are three such rules called DroppedFramesRule, EmptyBuerRule and AfterUpSwitchBuerBandwidthRule. If any of those rules have a confidence of one then that specic rule's recommended bitrate will be used instead to get the maximum possible bitrate no matter how many steps will be taken, and that is also independent of the reliability of the level. If the newLevel is a down switch to a lower bitrate level then the current level is marked as emergency in the decision history list. The EmptyBuerRule occurs when the video buer is detected as empty. Then it will set the confidence to one and recommend a bitrate of scaleF actor times actualBitrate, there the scaleF actor is 0.4 by default. The DroppedFramesRule is triggered if there are too many dropped frames per sec- ond. This means that parts of the video is not displayed or skipped because the computer could not handle the load. A frame is a picture in the movie and a video consist of a lot of pictures that are changing fast. Normally there are 24 or 30 frames each second. The confidence is set to one if the droppedF P S, the number of dropped frames, to fps ratio is greater than a particular threshold, named maximumDroppedF P SRatio. The maximumDroppedF P SRatio is by default set to 0.1. Otherwise, the confidence is cal- culated by equation 3.3.

droppedF P S confidence = (3.3) fps · maximumDroppedF P SRatio The fps variable is the video stream's frames per second, and the droppedF P S is the amount of frames that was not displayed. The recommended bitrate from the rule is calculated by equation 3.4.

16 3. Implementation of video players

droppedF P S bitrate = actualBitrate · (1 − ) (3.4) fps The AfterUpSwitchBandwidthRule takes care of the case there the actualBitrate is higher than the avaliable bandwidth. That means if the video player continues on current bitrate level it will eventually need to stop for buering as the fragments requires more bandwidth than the bandwidth measured. It then calculates a bitrate the same way the idealBitrate was calculated in Section 3.4.2, except that the equation 3.1 is not used. The confidence is set to one if the ratio between the bitrate and the actualBitrate is smaller than minBandwidthT oBitrateRatio, that is 0.5 by default. Otherwise, the confidence is set to zero. The AfterUpSwitchBandwidthRule will only apply just after an up switch, that is the previous and the just downloaded fragment level is at least one step up. The recommend bitrate will be the bandwidth on the last fragment.

Simplied variant of the algorithm A simplied variant of the algorithm can be seen in Algorithm1.

if Any emergency rule triggered then newBitrate ← Rule's recommended bitrate; newLevel ← getLevelFromBitrate(newBitrate); else if average bandwidth < actualBitrate ∩ enoughBuer then idealBitrate ← actualBitrate; else idealBitrate ← average bandwidth; Use equation 3.1 end newLevel ← getLevelFromBitrateAndConsiderReliability(idealBitrate); end switchToDownload(newLevel) Algorithm 1: HDS algorithm for switching downloading bitrate level.

17 CHAPTER 4

TESTS

One initial goal was to identify the parameters that aect the video players by running tests. Before tests were performed and analyzed, some potential parameters were identied by thinking of what those could be in general for all video players. The following were identied:

Throughput The transfer speed of data sent from the server to the client.

Available bitrates The available bitrate levels provided by the metadata le.

VBR The actual bitrate that variate within a movie.

Window size The area available for the video to show in on the web page.

Video visibility How much of the video that can be seen by the user. For example, half of the Safari window may be outside of the screen, making only part of the video visible.

Volume The volume level. The mute and unmute functionality.

CPU load The load of the processor.

System memory The total memory usage on the client.

Behavior when pausing, seeking and replaying The video player may keep or clear its history when a user seeks to a new position in the movie.

Most of these parameters were tested. They were chosen based upon the complexity, relevance and available video les. The test environment and the test results are described in the rest of this chapter.

18 4. Tests

4.1 Test environment

The test environment consisted of a client connected to a trac controlling bridge which in turn was connected to the server, as seen in Figure 4.1. The server was congured with a web server in order to deliver the video les.

Figure 4.1: Test environment.

The client was a Mac Mini from late 2012, equipped with an Intel i5 processor at 2.5 GHz and 4GB of ram connected to a monitor with 1920x1080 pixels resolution. On the trac controlling bridge, a TC[47] script was used to control the trac ow. The server was running the operating system Windows Server 2008 R2 SP1 with the latest updates available installed. The web server software Apache of version 2.2.22 with Adobe's HDS module[33] was used to deliver the HDS and HLS les to the client. Apache was chosen because it is the most popular web server[48] and its ocial supported for HDS. For MSS, Microsoft's own web server, IIS, was installed with version 7.5 as IIS is the only ocially supported software. On the client, the web browser Safari of version 6.0.3 was used in all tests. Safari is the only web browser that have support for HLS. The other protocols can use any of the popular web browsers but to be consistent Safari was also used there. Adobe's Flash Player of version 11.6.602.180 was installed to play HDS. For MSS, Silverlight 5.1.10411.0 was installed. As Safari had few useful metrics for HLS, it made it dicult to get test results. Thus, the downloading bitrate level was concluded from the trac control bridge by analyzing the network trac. Each request for a video part contained information about the lename. The lename contained what bitrate level was used for that video part and thus the bitrate level was easy to extract.

4.1.1 Video player settings The general goal was to use the default settings as much as possible to simplify. An exception had to be made for HLS as it uses the video tag in HTML5. On the video tag there is a preload attribute that is by default set to auto. Auto tells the browser to prefetch as much of the video content as it wants when web page load. The preload value was set to metadata as it was easier to get reliable test results. That tells the browser to

19 4. Tests only download the metadata les when loading the web page. This behavior was tested in Safari for HLS, and it shows that the web browser also loads the second metadata le that it chose to begin with but not the other bitrate level's metadata les.

4.1.2 Video les A movie was needed for the video players to play. For the initial development, the example videos from the Internet were used. This is not appropriated to use for testing as the source and the network connection to the server is not reliable. Instead video les where encoded and hosted on a local server. The movie Big Buck Bunny[49] was chosen as it is free to use and available in many formats. It is about 10 minutes long, which is enough time to see the bitrate level changes. Envivo provided encoded video les for HLS and MSS. The bitrate levels can be seen with respective resolution in Table 4.1.

Resolution(pixels) Bitrate(kbps) 480x270 400 640x360 800 704x396 1400 1024x576 2000 1280x720 2700 1920x1080 6000

Table 4.1: Encoded video resolution and bitrate.

Video les for HDS were created from the existing HLS les. They were merged into one big le, by using the encoding tool FFmpeg[50], and given as input to the f4fpackager for splitting the movie and creating the metadata les. No segment or fragment length parameter was given to f4fpackager. The result was one segment and many fragments were created for each bitrate level. By analyzing the network trac, the last requested fragment seen while playing was of number 148. That gives an average fragment length of about four seconds.

4.1.3 Graphs In some tests, a graph was suitable to show the video players' performance. Those were generated with a JavaScript library called jqPlot[51]. To indicate in the graph when the video player is buering the current viewing bitrate is set to zero. For HLS, the current viewing bitrate metric was not available and a boolean value was used instead.

20 4. Tests

4.2 Visibility

The rst test was about visibility of the video to see if it had any impact on the chosen bitrate level to download. One part was to change the screen area on the web page with the steps of the video bitrate's resolution that was seen in Table 4.1. The video player could be smart by not downloading the highest bitrate level since the full resolution is not showed to the user. Another part of the test was to change how much the user can see of the movie when keeping the screen area constant. For example when the user minimize the browser window or partly show the video by dragging the web browser window so that only half of the video can be seen. The network bandwidth limit was set to 12000 kbps during the tests, twice the highest bitrate level on the available streams. A bandwidth limit was set as the aect of having very high speed connection was unknown.

4.2.1 HLS The screen area available for the video on the web page was changed in steps corresponding to the resolution that was set for each bitrate level. The results can be seen in Table 4.2. The downloading bitrate level is what the video player choose to download when having the specied amount of space available. The rst row in Table 4.2: The screen area for the video on the web page was set to 480x270 pixels, the same as the rst bitrate level available. The video player choose to download the second quality, the higher bitrate level of 800 kbps. The second quality have a resolution of 640x360 pixels and therefore the video player scaled down the video to t on the web page.

Resolution available on the web page Downloading bitrate level 1 (480x270) 2 (640x360) 2 (640x360) 3 (704x396) 3 (704x396) 3 (704x396) 4 (1024x576) 5 (1280x720) 5 (1280x720) 5 (1280x720) 6 (1920x1080) 5 (1280x720)

Table 4.2: The resolution level of the video available on the web page versus the bitrate level the video player chose to download.

The downloading bitrate level of four and six was never switched to in the test. Thus as a small extra test to reach level six the video player was set to fullscreen mode but that resulted in no change. The bandwidth limit was set to 13000 kbps, only 1000 kbps higher, and it reached level six without fullscreen. When changing the resolution it is possible to see when the switch of video bitrate occurred since the screen goes blank for about one second before it comes back again with

21 4. Tests the new bitrate level. Changing the visibility of the window by scrolling down making the video invisible, minimize the Safari window or selecting another tab was not found to have any impact at all.

4.2.2 HDS On the debug web page for HDS, the window size for the video was set to 640x480 by default. When starting to play the video player switched up until the highest bitrate level was reached. If the web page were scrolled down so that the video was no longer visible or selecting another tab, the video player switched down to the lowest bitrate level available. The limit for scrolling down and downgrade the bitrate level seem to be exactly when nothing of the video player could be seen. When scrolling back to full visible video or switching back from another tab and wait a minute, the player keeps playing the lowest bitrate level. That means only watching a full visible video after a down switch has no aect on the bitrate level it chooses to download, and at start it always tries to get the highest bitrate level if nothing prevent it. Dragging the Safari window outside of the screen or losing focus of the web browser had no aect. In the later case, another program was selected and thus marked as active. Changing the width and height of the video on the web page for the ash video had no aect either even though it detected the new width and height correctly. Entering fullscreen mode caused the video player to get the highest bitrate level. Minimizing the Safari window caused it to switching down to the lowest bitrate level.

4.2.3 MSS In the Health Monitoring project, there was a system to inspect logs and show graphs in the GUI. Those parts were hidden to give the video more space. The video control bar, located where the pause and forward button are, was left behind, and took space from the video. In Safari when playing HLS, the control bar were only an overlay with no space taken from the video. In Health Monitoring project, the address eld on the top to enter the metadata le location was inside the video player, and it was not removed. To compensate and give the video the actual height an extra 77 pixels were added to the height for each resolution. A picture of how the video player look like after the changes can be seen in Figure 4.2. The test results for the dierent resolutions can be seen in Table 4.3. When testing to minimize the Safari window, move the window out of the screen, scrolling down so that the video was no longer visible or selecting another tab, nothing seemed to have an impact on the downloading bitrate level.

22 4. Tests

Figure 4.2: MSS video player after the logging system and graphs was removed.

Resolution available on the web page Downloading bitrate level 1 (480x270) 1 (480x270) 2 (640x360) 3 (704x396) 3 (704x396) 3 (704x396) 4 (1024x576) 4 (1024x576) 5 (1280x720) 5 (1280x720) 6 (1920x1080) 6 (1920x1080)

Table 4.3: The resolution level of the video available on the web page versus the bitrate level the video player chose to download.

4.3 Sound test

A test for changing the sound by mute and unmute the volume of the video player was done. This was not found to have any impact on the downloading bitrate level for any of the video players. One reason for this could be that the video and audio where in the same le for HLS and HDS. In MSS, they are separated but only one audio bitrate level was available. The video players could therefore not choose to download a lower audio bitrate.

4.4 Max video buer length

The video players may have a limit for how big the video buer can be. Thus a test was done for this to see where the limit was, given in seconds. The limit might have varied within the movie so the result should not be seen as nal. The test was done by giving the video player 100 Mbit/s bandwidth limit for a while and then setting the bandwidth limit to one kbit/s to prevent the video player to ll the buer. The time from the bandwidth limit change until playback stops were measured with a stopwatch. The results can be seen in Table 4.4.

23 4. Tests

HLS HDS MSS Time(s) 55 7 30

Table 4.4: Tested max video buer length in seconds.

There was a dierence of one less second between what was reported by the video player for HDS and what were measured by the stopwatch. This could be due to the reaction time and the fact that the value was updated only once each second.

4.5 CPU load

When the processor is overloaded, the video player has less CPU power to play the video with. This could result in frames being dropped or playing frames for a longer time than intended. The video player could try to decrease this aect by decreasing the downloading bitrate level as less CPU usage would be needed. This behavior was tested with a utility called CPUTest, used to stress the processor. It was downloaded from MacUpdate[52]. The CPUTest program creates threads, which can be seen as small subprograms, that constantly do work to load the CPU. There was no bandwidth limit set in the test and therefore the bandwidth available to the server was 100 Mbit/s. The result was that MSS and HDS video player lowered the downloading bitrate level, and HLS did not. The MSS video player triggered down switches relatively easily compared to the others. When trying to get the same aect for HLS, the amount of threads had to be increased so that the whole system became unresponsive. This test became more a test of how the operating system distributes its processor resources among programs. The test was not intended to test that distribution. Video decoding can be done on the graphic card to o oad the CPU. If the video player does not have support for that it may impact the result. In the test, MSS was much more sensitive than the other players. A participation factor maybe the fact that it measures the process and system usage. The HDS video player measures the number of frames it drops and should adapt according to the source code. This test veries that it works as intended.

4.6 Minimum bandwidth demanded for a bitrate level

The available bandwidth between the server and the client is important when choosing bitrate level to download. Thus the idea of this test is to see how much bandwidth is demanded for each bitrate level. The result can be seen in Table 4.5. The rst bitrate level of 400 kbps was always chosen as long the bandwidth was under the threshold for the bitrate level above, the 800 kbps level. When the bandwidth was increased to 1000 kbps the video player for HDS chose to upgrade from the 400 kbps bitrate level to 800 kbps.

24 4. Tests

For the others, HLS demanded 1800 kbps and MSS 1200 kbps.

Bitrate level HLS HDS MSS 6000 12875 6400 7800 2700 5450 3300 3600 2000 4100 2500 2800 1400 3000 1700 2100 800 1800 1000 1200 400 - - -

Table 4.5: The minimum bandwidth, given in kbps, demanded by the video player to chose and stay on that bitrate level.

The thresholds for HLS were easy to determine as the video player easily could stabilize at a given bitrate level. For HDS and MSS this was harder. The HLS video player seem to demand a bit above twice the amount of bandwidth than the bitrate level it chose to use. The HDS and MSS video player are closer to one quarter extra bandwidth.

4.7 An observed behavior for HLS

A behavior for HLS video player was discovered during the tests. Therefore, to show it, a new scenario was constructed. The bandwidth limit was set and kept to 3000 kbps. In the scenario, the video player began to request the 6000 kbps bitrate level with the rst two segments. It then switched to download the 800 kbps bitrate level. When the switch occurred to the 800 kbps level, the video player requested the rst two segments once again and continued with the other segments until it switched up to the 1400 kbps bitrate level. That means it had both the good and the bad quality of the beginning of the movie. When watching the movie, the 6000 kbps bitrate was played at the beginning and then it switched to a lower quality without playing from the beginning again. This behavior can be seen in Figure 4.3, and the Apache log in Figure 4.4. From the Apache log it can be calculated that the time it should take to download an 800 kbps segment and conclude that the downloaded segments were sequentially done.

4.8 Bandwidth limit scenarios

As the available bandwidth has an impact on the video players performance, this was tested by varying the bandwidth during playback. The tests were carried out by starting a specic scenario where bandwidth limit was changed in dierent intervals. It was done by pressing the play button and simultaneously activating the scenario with minimum delay. The delay is within a second. The scenarios were constructed to see how the video player handle small and large spikes where the bandwidth limit is largely increased or decreased for a short time. An

25 4. Tests

Figure 4.3: HLS playing with 3000 kbps bandwidth limit.

11:22:32 "GET /[email protected] HTTP/1.1" 200 8550240 11:23:11 "GET /[email protected] HTTP/1.1" 200 7360200 11:23:26 "GET /[email protected] HTTP/1.1" 200 2907 11:23:26 "GET /[email protected] HTTP/1.1" 200 1293252 11:23:31 "GET /[email protected] HTTP/1.1" 200 1039828 11:23:34 "GET /[email protected] HTTP/1.1" 200 1317504 11:23:38 "GET /[email protected] HTTP/1.1" 200 1084384 11:23:41 "GET /[email protected] HTTP/1.1" 200 1145296 11:23:45 "GET /[email protected] HTTP/1.1" 200 1048288 11:23:48 "GET /[email protected] HTTP/1.1" 200 1232152 11:23:51 "GET /[email protected] HTTP/1.1" 200 3027 11:23:51 "GET /[email protected] HTTP/1.1" 200 1754604 11:23:56 "GET /[email protected] HTTP/1.1" 200 1969112 11:24:02 "GET /[email protected] HTTP/1.1" 200 1755168 11:24:07 "GET /[email protected] HTTP/1.1" 200 1913088 11:24:12 "GET /[email protected] HTTP/1.1" 200 2182680

Figure 4.4: HTTP GET requests seen in the Apache log where dierent bitrates levels was requested. up spike means temporary increased bandwidth for a short time, and a down spike means temporarily decreased bandwidth for a short time. The length of the spikes was set to two, four, seven and ten seconds. They were chosen from the time length of the segments or fragments for the dierent protocols that were on the video les used. The segment length was ten seconds for HLS. HDS had an average of four seconds and MSS two seconds. Another test was done with longer periods between bandwidth limit changes.

26 4. Tests

4.8.1 HLS

Figure 4.5: HLS played with up spikes in bandwidth of two, four, seven and ten seconds where the bandwidth limit was 3000 and 5000 kbps from 1000 kbps. The time started when the play button was pressed in the video player. The yellow line indicate that it is buering when it is high.

The video player started to request the highest bitrate level suitable for the monitor's resolution as that resolution information was given in the variant playlist. It took 150 sec- onds before playback started and then it switched to download the lowest bitrate available. By that time, it had downloaded two segments with 20 seconds of playback time but it only played ten seconds before it started to buer again. The video player reacted to the small up spikes of 3000 kbps bandwidth limit by in- creasing the downloading bitrate level one step. With the 5000 kbps up spike, there were two steps up even though it was buering just before. The test can be seen in Figure 4.5. In the down spike test, the video player ignored all spikes even when the bandwidth limit dropped from 3000 to 300 kbps during ten seconds. It could be due to it had large enough buer to handle it. The test can be seen in Figure 4.6. In the test with longer periods of bandwidth limit, the video player did not restore to previous downloading bitrate after the rst valley, from 5000 kbps to 2000 kbps and back again to 5000 kbps in bandwidth limit. Instead, it selected the bitrate level between those two levels. The test can be seen in Figure 4.7.

27 4. Tests

Figure 4.6: HLS played with down spikes in bandwidth of two, four, seven and ten seconds where bandwidth limit was 1000 and 300 kbps from 3000 kbps. The time started when the play button was pressed in the video player. The yellow line indicate that it is buering when it is high.

Figure 4.7: HLS played with 5000 kbps bandwidth limit and had 90 seconds long variations of 2000, 3000 and 1000 kbps. The time started when the play button was pressed in the video player. The yellow line indicate that it is buering when it is high.

28 4. Tests

4.8.2 HDS

Figure 4.8: HDS plays with up spikes in bandwidth of two, four, seven and ten seconds where the bandwidth limit was 3000 and 5000 kbps from 1000 kbps. At time 365 and 550, the video player is buering for a few seconds.

In the up spike test for HDS, the measured fragment throughput takes a jump up when the spikes come. There was one exception, which was the second spike of four seconds long, it instead barely increased. The downloading bitrate level increased at the spikes of two seconds length but not in the others. That could be because that higher bitrate level was not marked as reliable in the shifting bitrate level algorithm, and it takes some time before the video player forget that. The up spike test can be seen in Figure 4.8. There, the fragment throughput is what the video player has measured as the average throughput on each fragment downloaded. Recall that the buer time represent the amount of seconds the video player has of playback time ahead. The target buer is a setting in the video player and is measured in seconds. More details were described in Section 3.4.1. The

29 4. Tests playback bitrate is the variable bitrate that can variate within a movie as some sequences are easier to compress than others while still maintaining the same quality. For example when the next picture within the movie change very little, one can save only that parts of the picture that has changed in order to save the amount of data used to represent the movie.

Figure 4.9: HDS played with down spikes in bandwidth of two, four, seven and ten seconds where bandwidth limit was 1000 and 300 kbps from 3000 kbps. At time 250 and 460 where the video player begins to buer.

In the down spike test at the beginning, the video player had some problems to stabilize on one bitrate level even though the bandwidth limit was not changed. A reason to that could be the varying fragment throughput that was measured. On all spikes except one, which is the two seconds long down spike to 300 kbps, the video player reacted by decreasing the downloading bitrate level. The fragment throughput is at that point not lowered much compared to the others. The down spike test can be seen in Figure 4.9. At the times 250 and 460, where the video player is buering, one can see that the buer is not completely empty. The lowest amount of data in the buer are 0.46 and 0.29

30 4. Tests seconds. When the video player decides to go to the buering state was not found in the source code. The event when it happens exist but not when it is triggered in order to see the cause.

Figure 4.10: HDS played with 5000 kbps bandwidth limit and had 90 seconds long varia- tions of 2000, 3000 and 1000 kbps.

In the test with longer periods of bandwidth limit variation, the video player plays well without ever buering. It restores to previous downloading bitrate level after the rst valley. This test can be seen in Figure 4.10.

31 4. Tests

4.8.3 MSS

Figure 4.11: MSS plays with up spikes in bandwidth of two, four, seven and ten seconds where the bandwidth limit was 3000 and 5000 kbps from 1000 kbps.

In the up spike test for MSS, the video player increased the downloading bitrate level by one step except when the spikes length was two seconds long. The up spike test can be seen in Figure 4.11. The throughput was measured by the video player, how it is calculated could not be determined.

Figure 4.12: MSS playing with down spikes in bandwidth of two, four, seven and ten seconds where bandwidth limit was 1000 and 300 kbps from 3000 kbps.

In the down spike test, the measured throughput was going up when the bandwidth limit was going down. That does not seem to be reasonable. The test was done again with the same result. However, even though a higher throughput was reported the video player

32 4. Tests decreased the downloading bitrate level a little time later and avoided buering. The down spike test can be seen in Figure 4.12.

Figure 4.13: HDS played with 5000 kbps bandwidth limit and had 90 seconds long varia- tions of 2000, 3000 and 1000 kbps.

In the test with longer periods of bandwidth limit variation, the video player plays well without any buering. It took some time to switch to the new downloading bitrate level on the rst bandwidth change and show it to the user. That is because the buer was lled with a higher quality and it take some time to consume that playback time. The test can be seen in Figure 4.13.

33 CHAPTER 5

EVALUATION

After seeing the test results, a new attack angel on the base problem was made. The new attempt was to create a general model for the three protocols, focusing on the network bandwidth problem. The model is described in Section 5.1. In Section 5.2, there is a description how the thesis could have been extended.

5.1 General model

The network connection is a common problem when looking at OTT performance. In particular CDN providers want to test their in order to make sure they perform well enough for OTT. Therefore, a general model to cover all these protocols would be useful, and one is suggested here. The problem is that they all have dierent algorithms to shift bitrate level. Therefore is the model simplied and ignores shifting of bitrate level. It is completely driven by the video buer. Then, in order to test, one rst needs to decide the following parameters:

ˆ A maximum buer length when the video player should stop requesting further video parts. ˆ A minimum buer length when the video player can begin request video parts again after have been lled. ˆ An initial buer length when to play from the buer the rst time. ˆ A minimum buer length that must be exceeded before playing from the buer after the video player has emptied it.

Tests can then be performed by requesting corresponding video parts sequential with respect to above mentioned parameters. In Safari for both HLS and HDS, the video players were observed to download the parts sequentially. The model would be able to tell if a certain bitrate level is sustainable. If the highest bitrate level were sustainable, it would indicate that it also is able to play the lower qualities

34 5. Evaluation as they require less bandwidth. A minimum bandwidth should also be taken into account in the test. To calculate the thresholds for each bitrate level, the overhead percentage from Section 4.6 can be used. Future usage of the model in a testing utility would tell how close the model is to reality.

5.2 Future work

The thesis can be complemented with the following:

ˆ Test the remaining parameters that have been proposed.

ˆ Evaluate the proposed model.

ˆ Make evaluation of DASH.

35 CHAPTER 6

DISCUSSION

The project goals has been fullled to various degrees. All the video players can be mon- itored, but to what extent varies. They can all show when it is buering, but not what bitrate is playing or downloading. Parameters such as network bandwidth, window size, video visibility and CPU have been tested and found to impact the video players in some cases. Almost all proposed parameters in Chapter4 were tested. One exception was to see what happens with the video player when the user change current playback position in the movie. One of the goals was to create a model of how the video players behaves has only been fullled for HDS. That was thanks to the source code that could provide that information. This was not the case for the other two protocols, but during the tests some characteristics have been found. For example that the HLS video player can download multiple video qualities of the same playback time even though the rst downloaded quality was higher. This behavior was seen in Section 4.7. It would be interesting to hear the cause of that. The question has not been asked due to Apple did not respond to another question about why the video buer API was not working when HLS plays. The latter question was posted on Apple Developer Forums.

6.1 Issues

One issue was how to deal with the metrics for HLS in Safari. There was few metrics available, and many did not work or show the right value. For example, the bitrate level there was no API for, neither the current viewing bitrate level or the downloading bitrate level. The idea was then to use the API that tells the video resolution that is currently playing. As each bitrate level in the video all had a dierent resolution, then it would be easy to conclude the current viewing bitrate level. The problem with that was the API did not detect the resolution correctly. The highest video resolution was not detected with the correct value, and some as the same resolution. Another idea was to encode the bitrate level as a text string into the movie. The computer would then have to take pictures while playing and use those to extract it to text again. That idea was not done since it may take

36 6. Discussion extra CPU usage aecting the result. Instead, the network trac was analyzed to conclude the downloading bitrate level, which is related to the current viewing bitrate level. Another issue arose when the video players were tested: How should the bandwidth limit scenarios in Section 4.8 be designed to capture the shifting bitrate level algorithm? For HLS and the MSS video player, nothing was known about the shifting bitrate level algorithm as nothing was found neither in the code of the video player that was available or documented by the creator. The tests were very time consuming. Therefore, the scenarios were designed to test more general aspects of how the players handled disturbances.

6.2 Conclusion

The initial problem to ease troubleshooting is not yet completely solved. There is no complete model for each video player except the one for HDS. But the report has given a good base of what can aect the video players and an example of how an underlying algorithm might look like. The support for DASH in the web browser will come in the future and can potentially get popular. It is then important that there are metrics available to monitor the video player. The metrics available today is not sucient. WHATWG are a community focusing on the web. They have notice that metrics is missing in the web browser and has suggested many to be part of the web standard. Hopefully will some of these metrics be accepted, or the issues will be addressed in some way. Companies that have control of some part of an Internet connection may want to make sure that there is no disturbance in their network. That may be done by placing a server on one side of the network and then a testing device on the other side. This might help to locate problems inside the network. If one want specically test OTT, it would be benecial to use one of the ocial video players, as that is what the end-users use. The problem can then be the missing metrics. An alternative may be to use the suggested model or to implement an own shifting bitrate level algorithm. Other companies that use CDN networks for video streaming may also want to evaluate the performance to make sure the CDN company performs as expected. For the end-user, this means the troubleshooting time can be decreased and hopefully the problem never occurs as it has already been resolved.

37 REFERENCES

[1] Schulzrinne. Rao, Lanphier. (2013, Jan) Real Time Streaming Protocol (RTSP). [Online]. Available: http://tools.ietf.org/html/rfc2326

[2] Adobe. (2013, Jan) RTMP Specication Now Available. [Online]. Available: http://blogs.adobe.com/ams/2009/07/rtmp_specication_now_availab_1.html

[3] Adobe. (2013, March) Adobe Flash Player. [Online]. Available: http://www.adobe. com/products/ashplayer.html

[4] Microsoft. (2013, March) Silverlight. [Online]. Available: http://news.netcraft.com/ archives/2012/07/03/july-2012-web-server-survey.html

[5] Steve Workman. (2013, Jan) Bug 816726 - DASH: Add support for seeking in DASH-WebM streams. [Online]. Available: https://bugzilla.mozilla.org/show_bug. cgi?id=816726

[6] ITEC. (2013, Jan) DASH-JS  A JavaScript- and WebM-based DASH library for Google Chrome. [Online]. Available: http://www-itec.uni-klu.ac.at/dash/?page_id= 746

[7] Andy Salo at RGB Networks. (2012, May) MPEG DASH: A Technical Deep Dive and Look at What's Next . [Online]. Available: http://www.rgbnetworks.com/pdfs/ RGB-MPEG-DASH-Cable12.pdf

[8] Robinson, Jutras and Craciun, Subjective Video Quality Assessment of HTTP Adaptive Streaming Technologies. Bell Labs Technical Journal, vol. 16, no. 4, pp. 523, 2012. [Online]. Available: http://dblp.uni-trier.de/db/journals/bell/bell16. html#RobinsonJC12

[9] Andy Salo at RGB Networks. (2012, May) MPEG DASH: A Technical Deep Dive and Look at What's Next. The Cable Show 2012. [Online]. Available: http://www.rgbnetworks.com/pdfs/RGB-MPEG-DASH-Cable12.pdf

38 REFERENCES

[10] International TeleCommunication Union(ITU). (2013, jan) Information technol- ogy  Dynamic adaptive streaming over HTTP (DASH). [Online]. Avail- able: http://standards.iso.org/ittf/PubliclyAvailableStandards/c057623_ISO_IEC_ 23009-1_2012.zip

[11] DASH Industry Forum. (2013, Jan) First Live MPEG-DASH Large Scale Demonstration. [Online]. Available: http://dashif.org/ rst-live-mpeg-dash-large-scale-demonstration-2/

[12] Adobe. (2013, Feb) Open Source Media Framework. [Online]. Available: http: //www.osmf.org/

[13] Adobe. (2013, Feb) Strobe Media Playback. [Online]. Available: http://sourceforge. net/projects/smp.adobe/?source=navbar

[14] Christensen. (2013, March) Health Monitoring project. [Online]. Available: https: //bitbucket.org/tchristensen/mmp-player-framework/src/b3ce85aa60b4?at=default

[15] Microsoft. (2013, Jan) Microsoft Media Platform: Player Framework. [Online]. Available: http://smf.codeplex.com/

[16] Adobe. (2013, Feb) Flash Builder. [Online]. Available: http://www.adobe.com/ products/ash-builder.html

[17] Microsoft. (2013, Jan) Visual Studio. [Online]. Available: http://www.microsoft. com/visualstudio

[18] Microsoft. (2013, March) Microsoft® Silverlight® 5 SDK. [Online]. Available: http://www.microsoft.com/en-us/download/details.aspx?id=28359

[19] Akhshabi, Begen, Ali and Dovrolis, An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP, in Proceedings of the second annual ACM conference on Multimedia systems, ser. MMSys '11. New York, NY, USA: ACM, 2011, pp. 157168. [Online]. Available: http://doi.acm.org/10.1145/1943552.1943574

[20] Mueller, Lederer and Timmerer, An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments, in Proceedings of the Fourth Annual ACM SIGMM Workshop on Mobile Video (MoVid12). New York, NY, USA: ACM, feb 2012, pp. 3742.

[21] Miller, Quacchio, Gennari and Wolisz, Adaptation Algorithm for Adap- tive Streaming over HTTP, in Proc. of 19th International Packet Video Workshop, Munich, Germany, May 2012, ConferenceProceedings. [Online]. Avail- able: http://www.tkn.tu-berlin.de/leadmin/fg112/Papers/Adaptation_Algorithm_ for_Adaptive_Streaming_over_HTTP.pdf

39 REFERENCES

[22] Apple. (2013, Jan) HTTP Live Streaming. [Online]. Available: https://developer. apple.com/resources/http-streaming/ [23] Apple. (2013, Jan) QuickTime and beyond. [Online]. Available: http://www.apple. com/quicktime/extending/resources.html [24] Apple. (2013, Jan) Safari. [Online]. Available: http://www.apple.com/safari/ [25] Google. (2013, Jan) Android 3.0 highlights. [Online]. Available: http://developer. android.com/guide/appendix/media-formats.html [26] Apple. (2013, Jan) Apple HTTP Live Streaming example. [Online]. Available: https://developer.apple.com/resources/http-streaming/examples/basic-stream.html [27] VideoLAN Organiaztion. (2013, Jan) VLC media player. [Online]. Available: http://www.videolan.org/vlc/ [28] Yospace. (2013, Jan) HLS Streaming Player and SDK for Flash. [Online]. Available: http://www.yospace.com/index.php/hls-sdk-for-ash-overview.html [29] Apple. (2011, Jan) HTTP Live Streaming Overview: Frequently Asked Questions. [Online]. Available: http://developer.apple.com/library/ios/#documentation/ networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/ FrequentlyAskedQuestions.html [30] Apple. (2013, Jan) HTTP Live Streaming Overview: Using HTTP Live Streaming. [Online]. Available: http://developer.apple.com/library/ios/#documentation/ networkinginternet/conceptual/streamingmediaguide/UsingHTTPLiveStreaming/ UsingHTTPLiveStreaming.html [31] Mogul, Frystyk, Masinter, Leach and Berners-Lee. (1999, June) Hypertext Transfer Protocol  HTTP/1.1. [Online]. Available: http://tools.ietf.org/html/rfc2616# page-138 [32] Adobe. (2013, Jan) HTTP Dynamic Streaming on the Adobe® Flash® Platform. [Online]. Available: https://bugbase.adobe.com/index.cfm?event=le. view&id=2943064&seqNum=6&name=httpdynamicstreaming_wp_ue.pdf [33] Adobe. (2013, feb) Installing and conguring the HTTP Origin Module. [Online]. Available: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/ WS7b362c044b7dd076-735e76121260080a90e-8000.html [34] The Apache Software Foundation. (2013, Feb) Apache. [Online]. Available: http://www.apache.org/ [35] Adobe. (2013, March) File packager. [Online]. Available: http://help.adobe.com/en_ US/HTTPStreaming/1.0/Using/WS9463dbe8dbe45c4c-c126f3b1260533756d-7c. html

40 REFERENCES

[36] Zambelli. (2013, March) IIS Smooth Streaming Technical Overview. [Online]. Available: http://www.microsoft.com/en-us/download/details.aspx?id=17678

[37] Microsoft. (2013, March) Expression Encoder Pro. [Online]. Available: http://www.microsoftstore.com/store/msstore/en_US/buy/pageType. product/externalRefID.0BE96B98

[38] Microsoft. (2013, March) Smooth Streaming plugin for OSMF (RTW!!!). [Online]. Available: http://blogs.iis.net/cenkd/archive/2013/03/26/ announcing-smooth-streaming-plugin-for-osmf-rtw.aspx

[39] Microsoft. (2013, Feb) The Ocial Microsoft IIS Site. [Online]. Available: http://www.iis.net/

[40] Microsoft. (2013, Jan) IIS Media Services 4.1. [Online]. Available: http: //www.microsoft.com/en-us/download/details.aspx?id=27955

[41] Apple. (2013, Feb) HTMLMediaElement Class Reference. [On- line]. Available: https://developer.apple.com/library/safari/#documentation/ AudioVideo/Reference/HTMLMediaElementClassReference/HTMLMediaElement/ HTMLMediaElement.html

[42] Apple. (2013, Feb) Apple Developer Forums. [Online]. Available: https: //developer.apple.com/devforums/

[43] WHATWG community. (2013, Jan) Video Metrics. [Online]. Available: http: //wiki.whatwg.org/wiki/Video_Metrics

[44] WHATWG community. (2013, Jan) FAQ. [Online]. Available: http://wiki.whatwg. org/wiki/FAQ#What_is_the_WHATWG.3F

[45] World Wide Web Consortium (W3C). (2013, Feb) About W3C. [Online]. Available: http://www.w3.org/Consortium/

[46] Microsoft. (2013, March) MediaElement.BueringTime Property. [On- line]. Available: http://msdn.microsoft.com/en-us/library/system.windows.controls. mediaelement.bueringtime.aspx

[47] Kuznetsov. (2013, Feb) TC, Trac control. [Online]. Available: http://www. linuxmanpages.com/man8/tc.8.php

[48] Netcraft. (2013, March) July 2012 Web Server Survey. [Online]. Available: http://news.netcraft.com/archives/2012/07/03/july-2012-web-server-survey.html

[49] Blender Foundation. (2013, Feb) Big Buck Bunny. [Online]. Available: http: //www.bigbuckbunny.org/

41 REFERENCES

[50] FFmpeg team. (2013, March) FFmpeg multimedia framework. [Online]. Available: http://www.mpeg.org/

[51] Chris Leonello. (2013, Feb) jqPlot pure javascript plotting. [Online]. Available: www.jqplot.com

[52] MacUpdate. (2013, April) Download CPUTest for Mac. [Online]. Available: http://www.macupdate.com/app/mac/23935/cputest

42