Master Thesis Electrical Engineering October 2013

QoE of Video Streaming over LTE Network

Pradeep Uppu, Sushanth Kadimpati

School of Computing Blekinge Institute of Technology 37179 Karlskrona Sweden This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for the degree of Master of Science in Electrical Engineering. The thesis is equivalent to 20 weeks of full time studies.

This Master Thesis is typeset using LATEX

Contact Information

Author(s): Pradeep Uppu Address: Karlskrona, Sweden E-mail: [email protected] Sushanth Kadimpati Address: Karlskrona, Sweden E-mail: [email protected]

University advisor(s):

Dr. Adrian Popescu, Prof. COM/BTH

School of Computing Internet: www.bth.se/com Blekinge Institute of Technology Phone: +46 455 385000 371 79 KARLSKRONA SWEDEN SWEDEN Abstract

In recent years, the mobile Internet has increased dramatically with the development of 3G and 4G technologies. Especially the usage of mobile broadband internet on the devices like cellular mobiles, Tablets and Laptops has skyrocketed. Among the multimedia applications video streaming is the most popular mobile application. But, making these services available to users in a cost effective way without compromising quality is a big challenge. The development of Long Term Evolution (LTE) technology in the mobile world made this task achievable. The features of LTE technology provide effective services in multimedia applications with high data rates and low latency. In this paper, we study and analyze the Quality of Experience (QoE) at the end user for Video on Demand (VoD) over the LTE network. To achieve this, we streamed High Definition (HD) videos based on H.264/AVC and these videos are delivered from source to destination using Transport Control Protocol (TCP) and User Datagram Protocol (UDP). Specifically, our study is about QoE evaluation in terms of delay variation, packet loss metrics and provides performance evaluation to characterize the impact of transport layer protocol in video streaming over radio networks like LTE. In order to know the performance of video streaming over LTE network, we also evaluate the LTE performance in terms of one-way delay, packet loss and inter packet delay for the generated UDP and TCP packets.

Keywords: QoE, Video Streaming, H.264/AVC, LTE, One-way Delay, Packet loss.

i Acknowledgements

It gives us great immense joy in acknowledging Prof Adrian Popescu for his diligent support and extending the opportunity to pursue our master thesis under his immaculate supervision. We would like to thank Dr. Patrik Arlos for providing the experimental test bed and valuable suggestions in the evolution of this thesis. We would also like to thank Wowza Media Systems for providing the software. A word of thanks to Mr. Tahir Minhas Nawaz for his valuable suggestions and tips. We are indebted to our friends and family for their constant support and prayers that helped us complete the thesis. This thesis is dedicated to our parents who stood beside us through thick and thin in making this thesis substantial.

Pradeep Uppu Sushanth Kadimpati

ii Contents

Abstracti

Acknowledgements ii

Contents ii

List of Figures vi

List of Tables viii

Acronyms ix

1 Introduction2 1.1 Motivation...... 3 1.2 Related Works...... 4 1.3 Contribution...... 6 1.4 Aims and Objectives...... 6 1.5 Research Questions...... 7 1.6 Thesis Outline...... 7

2 Technical Background9 2.1 Quality of Experience...... 9 2.2 Video Streaming...... 9 2.3 Video Compression...... 10 2.4 Video Format...... 11 2.4.1 Video Codec...... 11 2.4.2 Video Container...... 11 2.4.3 H.264/AVC Codec...... 11 2.5 Types of Video Streaming...... 12 2.6 Supported Protocols for Video Streaming...... 12 2.7 Assessment of Videos...... 12

iii 2.8 Overview of 3GPP Releases...... 13 2.9 Technical Overview of LTE...... 15 2.10 Architecture of LTE...... 16 2.10.1 Core Network...... 16 2.10.2 Radio Access Network...... 17

3 Design and Implementation 20 3.1 LTE Network Performance Evaluation with Generated Traffic 20 3.1.1 Experimental Procedure...... 21 3.1.2 Gateway...... 22 3.1.3 Sender...... 22 3.1.4 Receiver...... 22 3.1.5 Measurement Point (MP)...... 22 3.1.6 Consumer...... 22 3.2 Measurements...... 23 3.2.1 One Way Delay (OWD)...... 23 3.2.2 Packet Loss (PL)...... 23 3.2.3 Inter Packet Delay (IPD)...... 23 3.3 Video Quality Assessment using Subjective Analysis...... 24 3.3.1 Experimental Setup and Procedure...... 24 3.3.2 WOWZA Media Server...... 25 3.3.3 Test Video Parameters...... 25 3.3.4 NetEm...... 26 3.3.5 Delay...... 27 3.3.6 Packet Loss...... 27 3.3.7 Assessment of Videos...... 27

4 Results and Analysis 30 4.1 Analysis of LTE Network Performance with Generated Traffic 30 4.1.1 One Way Delay for UDP packets in Uplink...... 30 4.1.1.1 Minimum One Way Delay...... 30 4.1.1.2 Maximum One Way Delay...... 31 4.1.1.3 Mean One Way Delay...... 31 4.1.2 Inter Packet Delay for UDP packets in Uplink..... 32 4.1.2.1 Minimum Inter Packet Delay...... 32 4.1.2.2 Maximum Inter Packet Delay...... 33 4.1.2.3 Mean Inter Packet Delay...... 34 4.1.3 Packet Loss for UDP in Uplink...... 34 4.1.4 One Way Delay for TCP packets in Uplink...... 35 4.1.4.1 Minimum One Way Delay...... 35 4.1.4.2 Maximum One Way Delay...... 35 4.1.4.3 Mean One Way Delay...... 36 4.1.5 Inter Packet Delay for TCP packets in Uplink..... 37

iv 4.1.5.1 Minimum Inter Packet Delay...... 37 4.1.5.2 Maximum Inter Packet Delay...... 38 4.1.5.3 Mean Inter Packet Delay...... 38 4.1.6 Packet Loss for TCP in Uplink...... 39 4.2 Gateway Evaluation...... 39 4.3 One Way Delay Comparison in TCP and UDP...... 40 4.4 One Way Delay for Video Streaming Over LTE...... 40 4.5 QoE Analysis of Video Streaming...... 41 4.5.1 Packet Delay Variation...... 42 4.5.1.1 Packet Delay Variation for TCP...... 42 4.5.1.2 Packet Delay Variation for UDP...... 44 4.5.1.3 Standard Deviation for Delay Variation... 45 4.5.1.4 Confidence Interval for Delay Variation... 45 4.5.2 Packet Loss...... 46 4.5.2.1 Packet Loss for TCP...... 46 4.5.2.2 Packet Loss for UDP...... 47 4.5.2.3 Standard Deviation for Packet Loss..... 48 4.5.2.4 Confidence Interval for Packet Loss..... 49

5 Conclusion and Future Work 51 5.1 Conclusion...... 51 5.2 Future Work...... 52

Bibliography 53

v List of Figures

2.1 Quality of Experience Measurement...... 10 2.2 Evolution of LTE...... 15 2.3 LTE Radio Access Network...... 18

3.1 Detailed Experimental Set up...... 21 3.2 Experimental Set up for Video Streaming...... 25 3.3 Screen Shot of QoE Evaluation Tool...... 27

4.1 Minimum One way Delay for Generated UDP Packets for Uplink...... 30 4.2 Maximum One way Delay for Generated UDP Packets for Uplink...... 31 4.3 Mean One way Delay for Generated UDP Packets for Uplink 32 4.4 Minimum Inter Packet Delay for Generated UDP Packets for Uplink...... 33 4.5 Maximum Inter Packet Delay for Generated UDP Packets for Uplink...... 33 4.6 Mean Inter Packet Delay for Generated UDP Packets for Uplink 34 4.7 Packet Loss for Generated UDP Packets for Uplink...... 34 4.8 Minimum One way Delay for Generated TCP Packets for Uplink 35 4.9 Maximum One way Delay for Generated TCP Packets for Uplink...... 36 4.10 Mean One way Delay for Generated TCP Packets for Uplink 36 4.11 Minimum Inter Packet Delay for Generated TCP Packets for Uplink...... 37 4.12 Maximum Inter Packet Delay for Generated TCP Packets for Uplink...... 38 4.13 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38 4.14 Packet Loss for Generated TCP Packets for Uplink...... 39 4.15 Minimum One Way Delay for Generated TCP and UDP pack- ets for Uplink...... 40 4.16 Minimum One Way Delay for Generated TCP and UDP packet size of 1500 bytes for Uplink...... 41 4.17 MOS for TCP videos subjected to Delay Variation...... 43

vi 4.18 MOS for UDP videos subjected to Delay Variation...... 44 4.19 Standard Deviation for Delay Variation...... 45 4.20 Confidence Interval for Delay Variation...... 46 4.21 MOS for TCP videos subjected to Packet loss...... 47 4.22 MOS for UDP videos subjected to Packet Loss...... 47 4.23 Standard Deviation for Packet Loss...... 48 4.24 Confidence Interval for Packet Loss...... 49

vii List of Tables

2.1 Five-level scale for rating overall quality of video...... 13

3.1 Test Video Parameters...... 26

4.1 MOS for Delay Variation...... 43 4.2 MOS for Packet Loss...... 46

viii Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska H¨ogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union, Radio Communication Sector

ITU-T International Telecommunication Union, Telecommunication Stan- dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii Introduction

1 Chapter 1

Introduction

For the last two decades radio access technologies are not just limited to provide voice communications alone, but also used for the video and data applications as well. Due to the rapid development of technology used in telecommunication systems and consumer electronics, network operators are now able to provide better Internet services over radio networks. After the development of 2G and 3G technologies, the Internet based services are available on mobile systems, namely mobile broadband. According to CISCO report, mobile video has been growing at a Com- pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and 2017, and it is the highest growth rate of any other mobile application [1]. Meeting this demand, maintaining the Quality of Service and user satis- faction has become a big challenge to network operators. To achieve this a radio interface is needed, which can provide the best Quality of Services with design parameters like Data rates, Delay and Capacity. One of the main reasons for LTE evolution is to provide the IP based services to people on mobile devices with better QoS [2]. Some services have been already provided by 3G networks, but providing HD video streaming, interactive video gaming and other multimedia services without degrading the Quality of Services is a major challenge. Among these multimedia ser- vices, video streaming over mobile Internet is the most popular one. In the burst growth of data rates and services, being aware of user experience is important to maintain the service quality and the application performance. The Quality of Experience is defined as “The process of understand- ing the actual performance of services, as delivered to the customer, for the purpose of ensuring those services meet customer expectations and re- quirements”. Understanding user experience is very critical for the network operators in managing the QoS of the network. Quality of Experience mea- surements are made at the point of delivery directly from the subscriber’s smart phone or PC. QoE measurements deals with how well applications (Video streaming, VOIP and Web browsing) work in the hands of subscriber

2 CHAPTER 1. INTRODUCTION 3

[3]. QoE measurements require an understanding of the Key Performance Indicators (KPI) that impact on the user perception. KPI’s vary with the service type and services like VoIP, Video streaming, On-line gaming, In- ternet browsing has unique performance indicators to measure [4]. QoE considers the individual subscriber experience with a service unlike network conditions in QoS. The knowledge of user actual experience is very impor- tant for the operators to know the customers satisfactory levels and then operators can concentrate on issues to prevent the churn. The mobile communication technologies are tracked back to various gen- erations 1G, 2G, 3G and 4G. 1G which started in 1980 stands for first gener- ation of wireless telecommunications popularly known as Cellular phones, in which analog radio signals are used. In 1991 2G (Second Generation) wire- less telephone technology started using digital mobile systems [2]. The sec- ond generation mobile technologies provide low bandwidth services, which are suitable for voice traffic. The packet data over cellular systems started with the introduction of GPRS (General Packet Radio Services) in GSM (Global System for Mobile Communications). With the introduction of 3G technology, network operators can provide better and more advanced ser- vices like video calls and mobile broadband services. In our thesis, we worked on user quality assessment for video streaming over LTE network. We report on user perception of video quality degrada- tion of selected videos, which are subjected to different delays and packet losses. This thesis is specifically aimed to understand the experience of High Definition videos with H.264/AVC. We report the user experience by con- ducting Subjective Assessment as per the International Telecommunications Union (ITU) [5]. In addition to this, we also studied the Uplink behaviour of the LTE network by calculating the One-way Delay, Inter Packet Delay and Packet losses with different payloads at different data rates. We analyzed the Qual- ity of Service of video packets over LTE network with OWD, IPD and Packet losses as QoS metrics.

1.1 Motivation

LTE is the latest technology in the telecommunications world using which network operators are able to provide advanced multimedia applications to users maintaining Quality of service [2]. By the introduction of LTE network, users are able to enjoy the broad- band quality Internet services [2] on the mobile devices, but providing the advanced multimedia services over radio network without degrading the Quality of Services is a big challenge. Video streaming is the most pop- ular application in the next generation mobile systems. Due to the rapid increase in data rates using LTE technology, users are able to watch High CHAPTER 1. INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobile systems are being manufactured to access advanced services. In this current day of huge resource demand applications, understanding the user experience is very critical for the network operators to manage the QoS of the network and hence our motivation to measure and analyze it. We choose High Definition videos with 1280 × 720p and H.264/AVC, since H.264 codec is the widely used codec for video streaming. As compared to the previous codec like MPEG-2 and MPEG-4 Visual, AVC provides good quality of video for wide range of services like broadcast multimedia streaming and Video on Demand. The QoS and QoE are so interdependent and they have to be studied and managed with common understanding. So, as a part of QoS evaluation we conducted experiments to measure the OWD, IPD and packet loss for gener- ated TCP, UDP packets over LTE network for Uplink. We conducted these experiments to know the behaviour of TCP and UDP packets for different data rates with different payloads. We chose uplink, since video sharing became popular with the emerging of websites like YouTube, Facebook and Vimeo. According to YouTube statistics 72 hours of video are uploaded to YouTube every minute [6].

1.2 Related Works

In paper [7], the authors proposed QoE assessment models for video stream- ing services like IPTV by using QoS parameters in network layer. This helps the network service provider in providing multimedia services with improved QoE by using the suggested QoE assessment process. This also helps the network service provider to prevent the unnecessary expenses for mainte- nance and repair of the network. In paper [8], the authors evaluated the QoE measurement in a live 3G network with automatic data capturing tools are used in the experiment. Evaluation is carried out by subjective methods relevant to a set of objective parameters. The author concluded that the QoE of mobile video streaming was influenced by the QoS and with the context also. In paper [9], the authors investigated the QoE evaluation method of video streaming service in 3G networks. The author suggested a non reference QoE model of video streaming services based on the gradient boosting machine. In paper [10], the authors analyzed the perception of users towards the videos encoded with H.264 baseline profile in laptops and mobile devices. The videos are streamed through an emulated network with packet losses and packet delay variations. The obtained results from both devices are compared using matched-sample-test. The conclusion infers that the device does not show any impact on user perception for videos of same resolution. In paper [11], the authors investigated on different types of QoE analysis CHAPTER 1. INTRODUCTION 5 and proposed a flexible framework well capable to correlate with both packet loss ratio and subjective quality degradation. The proposed metric and the framework provide help for performing easy and accurate QoE evaluations on streaming applications. In paper [12], the authors presented a new model for non-intrusive pre- diction of H.264 encoded video quality over UMTS networks. The test bed was evaluated on the NS2 based UMTS simulation network. The proposed model was based on combination of a set of objective parameters in the physical and network layers, in terms of Mean Opinion Score (MOS). In paper [13], the author proposed a conceptual model of QoE, cor- responding to the hourglass model of Internet architecture based on the streaming video quality of experience and its factors. This model of QoE has four layers, similar to the Internet hourglass model, and each layer has a role towards the user perceived quality. In paper [14], the author analyses the user perception towards the QoE of video, which is encoded with H.264 baseline profile and streamed through an emulated network with packet loss and packet delay variation. The ex- periment was conducted both on a laptop and a mobile device. In paper [15], the authors analyzed the effect of QoS parameters like end-to-end delay, packet loss and packet delay on the performance of video conferencing in the LTE network. The authors carried out their experiment on OPNET 16.0 [16] simulator. The results are evaluated by taking three network scenarios namely low load, medium load and high load. In paper [17], the authors analyzed the impact of payload size and data rate on one-way delay and packet loss in network on three different com- mercial 3G mobile operators available in Sweden. The measurement in the network are carried out by using Endace DAG [18] cards and the Endace DAG are synchronized with GPS to get an accurate measurement. In paper [19], the authors analyzed the one-way delay in wireless broad- band network based on traffic measurements. The experimental setup is designed in such a way to get perfect time synchronization and accurate results. The authors concluded that the one-way delay of uplink is much higher than the one-way delay of downlink. In paper [20], the authors investigated the operator services on one-way delay and jitter using packets of different protocols with random packet size and random IPD. They investigated the impact of constant IPD and reducing the interval of IPD on OWD in 3G networks. They also investigated the one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate (VBR) transmission patterns. CHAPTER 1. INTRODUCTION 6

1.3 Contribution

The experiments were conducted in two phases. In the first phase, we con- ducted the video quality assessment using subjective method. Videos are transmitted over LTE network and subjected to different delays and packet losses. There has been previous works [21][22] done on video streaming over 3G networks and other wireless networks. However most of the works are based on simulations and objective analysis using Peak Signal-to-Noise Ratio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) tools. PEVQ tool is recommended by ITU, but it has limitation of video length not more than 20 seconds [23]. We adopted subjective analysis method, which gives better user opinion than objective analysis. In wireless radio networks it is hard to predict the network behaviour with less than one minute video [24]. So we choose the videos, which are having a duration of four minutes. The Quality of end user experience affected by the technical factors of QoS (network quality and network coverage) and non technical Subjective factors (service content, ease of service setup and pricing). So, we attempted to know the performance of LTE network by measuring QoS metrics that are OWD, IPD and Packet loss for uplink. We measured these metrics on an experimental test bed where OWD is calculated with the packet traces collected at the link level. It gives the possible precise measurements up to 60 nano seconds [25]. With the collected traces we calculated the OWD, IPD and Packet losses using Perl program.

1.4 Aims and Objectives

This Thesis aims at studying the user experience on video quality with the parameters packet loss and delay/delay variance over LTE network and H.264 as video codec. Another aim is to know the LTE network behaviour in terms of OWD, IPD and Packet loss for generated traffic on UDP and TCP protocols. The objectives are as follows:

• To understand the user perception of video quality for high definition videos with packet losses and delay variations over LTE network, with H.264 as codec.

• To understand the user perception of video quality for different pro- tocols.

• To understand the LTE network performance in terms of OWD, IPD and Packet loss at different data rates for different payloads. CHAPTER 1. INTRODUCTION 7

1.5 Research Questions

1. What is the effect of TCP and UDP protocols on OWD, IPD and Packet loss in LTE network uplink with different payloads at different data rates ?

2. How is the user perception affected by the delay variation in video streaming over LTE network using TCP and UDP protocol ?

3. How is the user perception affected by the packet loss in video stream- ing over LTE network using TCP and UDP protocol ?

1.6 Thesis Outline

The rest of the document is as follows. Chapter 2 provides the technical background of Quality of Experience and LTE releases. Chapter 3 describes the experimental methodology, design and implementation. Chapter 4 illus- trates the results and its analysis. Chapter 5 comprises the conclusion and future work. Technical Background

8 Chapter 2

Technical Background

2.1 Quality of Experience

QoE is also defined [23] as “The overall acceptability of an application or service, as perceived subjectively by the end user”. It mainly deals with how an individual is satisfied with the provided service in terms of usability, accessibility, retain ability and integrity of the service. Quality of Experience considers the complete end-to-end system effects like, the effects of the client, network and infrastructure of the services. It also takes into consideration the end user’s mood, emotions and physical status, which encompasses the psychology of the end user. So, there is no particular defined statement for QoE, that is accepted universally [26]. QoE considers the individual subscriber experience with a service unlike network conditions in QoS. The knowledge of actual user experience is very important for the operators to meet the customer’s satisfactory levels. In doing so, operators can turn their attention on issues to prevent the churn.

2.2 Video Streaming

Today video sharing is the most effective way of entertainment and gaining knowledge. In order to share a video it must be stored and transmitted over a communication channel, but it is expensive to transmit a raw video over a communication channel because of the huge amount of data size. Even to store a raw video, it requires a lot of space on the data storing devices. Usually the video taken from camera footage contains lot of redundant data. So, there is a need to reduce the size of the redundant data in the raw video also considering the quality of the video. Here video compression comes into the picture, to reduce the redundant data by considering the quality of the video [27].

9 CHAPTER 2. TECHNICAL BACKGROUND 10

Figure 2.1: Quality of Experience Measurement

2.3 Video Compression

Video Compression is a technique where the raw video is compressed using mathematical models and algorithms to reduce the size of the video to lower bit rates. Video compression is an essential process for video applications like Internet video streaming, mobile TV, video conferencing, digital televi- sion and DVD-Video [28]. Video compression is mainly classified into two types lossless compression and lossy compression. In lossless compression no information is lost in the compression process. So it is possible to recover the original raw video from a compressed video. In practical cases, the amount of data reduced is less. Quality of the video is maintained well in lossless compression [29]. This type of video compression is not used for streaming videos because even though video is compressed, it still maintains a large data size. In lossy compression, as the name suggest information is lost in compres- sion process. The information once lost cannot be retrieved [30]. Obviously, the size of the video is reduced and the quality of video is degraded too. This technique is predominantly used for video streaming services. Even though the quality of the video is lost it is still perceivable. Video compres- CHAPTER 2. TECHNICAL BACKGROUND 11 sion should be done at an optimum level while simultaneously considering the video quality.

2.4 Video Format

The video format consists of two distinct technology concepts: Video Codec and Video Container.

2.4.1 Video Codec Video compression process involves a complementary pair of systems, a com- pressor (encoder), which converts the source video into compressed form before the transmission or storage and a decompressor (decoder) which con- verts the compressed data, back to represents the original data. So, this encoder-decoder pair is called as CODEC. Video codec is a software pro- gram that compresses a raw video into a compressed video of small data size in a way that the compressed video must play on the computer, since raw or uncompressed data requires a large bit rate, approximately 216 MB per second [28]. Video codec can only compress a video, similarly audio codec is used for audio file and played along with video files. Different types of codes are available to compress raw video and the selection of video codec depends on various factors like size of video, type of video streaming and type of application [31]. Nowadays there are many video codes available for video compression, some of them are MPEG-1, MPEG-2, MPEG-3, MPEG-4, H.264, and Vorbis. Among all available video codes H.264 is the most widely used codec. Main features of this codec are, providing better video quality at lower bit- rates, fast encoding speed and other advanced features make H.264/AVC better than its counterparts [28].

2.4.2 Video Container Video Container describes the structure of the video, in which way it has been compressed and stored [32]. Video codec compresses the raw video into a format, which is considered as the video container. Examples of the video containers are .avi, .mp4, .mov, and .asf.

2.4.3 H.264/AVC Codec H.264 is a method and format for video compression. It was developed based on the concepts of earlier standards such as MPEG-2 and MPEG-4 Visual and provides better compression efficiency [28]. It also provides features like better-quality compressed video and greater flexibility in transmitting and storing videos. Furthermore, it offers robust compression for a wide range of CHAPTER 2. TECHNICAL BACKGROUND 12 applications, from low bit rate mobile video applications to high definition broadcast services.

2.5 Types of Video Streaming

Nowa-days different types of video streaming applications are in use. Some of them are point-to-point communication, broadcast communication and multicast communication. All these video streaming applications are mostly depends on two video streaming types. One of them is live video streaming, which is a process of real time transmission of a video over Internet [33]. So the streamed video file can be viewed on smart phones, mobiles and personal computers. The video application is mainly used in live sports broadcasting. Another type of video streaming is Video-on-Demand, this video stream- ing process is quite contrary to the previous type. In this video streaming, the selected video file is played whenever the viewer wishes to watch the video. In this type of streaming, one-to-many viewers can view the same or different video [34]. This process is mainly observed in video streaming websites like YouTube, Hulu and DailyMotion.

2.6 Supported Protocols for Video Streaming

There are several protocols that support media streaming. Some of the ma- jor protocols are Hypertext Transfer Protocol (HTTP), Session Description Protocol (SDP), Real-time Streaming Protocol (RTSP), Real-time Trans- port Protocol (RTP), Real Data Transport (RDT) and Real-time Transport Control Protocol (RTCP) [35]. Each protocol is used depending on the type of application in that particular point and also depends upon the require- ment of service. For most of the video streaming protocols, TCP and UDP serves as the underlying protocol. UDP is a preferable to its contrary pro- tocol TCP in video streaming application because UDP send packets at a constant rate and it doesn’t care about the lost packets. But, TCP is also widely used in video streaming because of benefits of streaming with TCP, including its retransmission capabilities, congestion control and flow control [36]. On the same hand, TCP introduces delay due to the retransmission of data whenever data is lost. But in case of UDP, there is no point of retransmission.

2.7 Assessment of Videos

When it comes to the point of video quality assessment, there are mainly two types of methods. They are as follows.

• Objective Assessment. CHAPTER 2. TECHNICAL BACKGROUND 13

• Subjective Assessment.

The Objective video quality assessment method is based on Mathemati- cal models and fast algorithm that produces the results approximately equals to the subjective video quality assessment and it does not involve any hu- man grading. It is a software program designed to deliver results based on error signal ratio of the original and processed video. The most popular Objective methods are Mean Squared Error (MSE), Perceptual Evaluation of Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37] [38]. This method has few categories referred as Full-Reference (FR) and Reduced - Reference (RR) video quality metrics [39]. The other video assessment includes Subjective analysis, which is based on human perception. The subjective video quality assessment is considered as accurate way of measuring quality of video compared to objective assess- ment. For subjective video quality assessment, a set of videos are given to the subjects for rating the videos on a scale of five (5) and the grades for quality is given in Table 2.1. The rating given by the subjects are known as Mean opinion Score (MOS). The subjects include experts and non- expert observers.

MOS Rating User Opinion 5 Imperceptible 4 Perceptible, but not annoying 3 Slightly annoying 2 Annoying 1 Very annoying

Table 2.1: Five-level scale for rating overall quality of video

2.8 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobile networks for instant access to information. The user needs to have instant access to information thereby facilities a way for the creation of user ap- plications, which required low jitter and low latency. Usage of applications like banking, multiplayer on-line gaming, downloading music, watching live news and sports, IPTV and so on over the Internet is rapidly increasing. There is a great and tremendous need for high speed data networks required to meet the above user applications. On this behalf, 3rd Generation Part- nership Project (3GPP) developed LTE to have higher data rate. 3GPP is a collaboration agreement that was established in December 1998 and it was formed by six telecommunication standards that came together on an agreement from different countries known as the Organizational Partners. CHAPTER 2. TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologies are differentiated by releases. A brief overview of different 3GPP releases from GSM to LTE-Advanced is as follows: In Releases 99 [40], the GSM specifications of the Universal Terrestrial Radio Access Network (UTRAN) are developed. UMTS was first standard- ized by the European Telecommunications Standards Institute (ETSI) in January 1998. Mobile technologies like EDGE (Enhanced Data rates for GSM Evolution) provides high data rate than GPRS, which offers services like messaging, email, etc. Majority of technologies in usage are based on release 99. Release 4 [41], is provided with minor improvements in UMTS network. It mainly concentrates on Multimedia Messaging support, which includes development of the MMS Reference Architecture, MMS service features, streaming Support in MMS and a lot more. In release 5 [42], the 3GPP featured a new technology called HSPDA, which helps the wireless operators to provide the customer need of high speed wireless data services with improved spectral efficiency. In the same release it also introduced the IP Multimedia Subsystem (IMS) architecture. IMS enhances the end user experience for multimedia application and also facilitates the use of IP (Internet Protocol) for packet communications in all wireless networks [43]. Release 6 [44], is mainly concentrated on Quality of Service (QoS) for multimedia applications. Enhanced Multimedia Broadcast/Multicast Ser- vices (MBMS), which is a user service, enables data to deliver to a set of users using same radio resources within a service area like High Speed Uplink Packet Access (HSUPA) for high uplink speed. Release 7 [45], focuses on decreasing latency, improved QoS for the real- time applications such as gaming, VoIP. In this release the spectral efficiency of the HSPA is increased by the introduction of Multiple-Input Multiple- Output (MIMO) antenna systems. Enhancements to GSM with Evolved EDGE which increases usual throughput rates, reduces the latency by two and improves spectral efficiency. Release 8 [46], consists of evolution of HSPA features such as 64 QAM and simultaneous use of MIMO. Innovation of LTE technology, which is ex- pected to meet the high data rate requirements of the end user. Reduced user plane latency resulting highly improved user experience with full mo- bility. Using single-carrier frequency domain multiple access (SC-FDMA) for uplink and orthogonal frequency domain multiple access (OFDMA) for downlink. It introduces Evolved Packet System (EPS) to provide network architecture, which integrate common mobility, security and QoS mecha- nisms for fixed and mobile broadband accesses. Release 9 [47], provides much more enhancement to both HSPA and LTE by including HSPA dual-carrier operation in combination with MIMO. Evolution of the IMS architecture, introduces the concept of LTE Femto CHAPTER 2. TECHNICAL BACKGROUND 15 cells. It also initiated the study of Advanced LTE. In Release 10 [48], new concepts in LTE-Advanced are introduced like Carrier Aggregation (CA), multi-antenna enhancements and relays. It also includes Self Organizing Networks (SON), Heterogeneous Networks (Het- Nets), quad-carrier operation for HSPA+, etc. Enhanced uplink and down- link schemes for much more higher data rates than LTE-Advanced. In Release 11 [48], will build on the advancements and refinements of capabilities of technologies that are developed in release 10. Enhancements to relays, Carrier Aggregation and MIMO etc. Release 12 [48], includes the study of network optimization for Machine Type Communication (MTC), Nonvoice emergency services and Session Ini- tiation Protocol Uniform Resource Identifier (SIP URI) portability.

Figure 2.2: Evolution of LTE

2.9 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom- munications System (UMTS) radio access through the Evolved UTRAN (E- UTRAN). It is mainly accomplished by the evolution of core network known as System Architecture Evolution (SAE). The developed new architecture provides higher rate data delivering capacity to the LTE network. By this LTE is able to support different types of services including HD video stream- ing, VoIP, Multi user online gaming, Video on demand, Push-to talk and Push-to-view. The main features and capabilities of LTE [49][50][51]are: CHAPTER 2. TECHNICAL BACKGROUND 16

• The downlink speed is up to 100 Mbps and the technique used is Orthogonal frequency-division multiplexing (OFDM).

• The uplink speed is up to 50 Mbps and the technique used is Single- carrier frequency-division multiple access (SC-FDMA).

• It uses 4x4 antennas for downloading rates and it uses a single antenna for uploading rates.

• The data transmission in LTE is significantly low for application that use low latency such as VoIP, Online Multi player gaming etc.

• The user plane latency offered in LTE is less than 10 ms and the control plane latency is less than 100 ms.

• In the case of mobility, LTE supports up to 500 kmph but like other mobile technologies it will be optimized for lower speed from 0 to 15 km/h.

• LTE supports higher flexibility in carrier bandwidths, the set of band- widths actually supported are 1.25, 2.5, 5, 10, 15, and 20 MHz.

• LTE is capable of delivering optimum performance in a cell size up to 5 km but it still can deliver its effective performance up to 30 km. Whereas with its limited performance, it can deliver up to 100 km radius.

2.10 Architecture of LTE

The enhancement features like higher packet data rates and significantly lower-latency of LTE cannot be possible without the evolution of System Ar- chitecture Evolution (SAE). This includes the Evolved Packet Core (EPC) network. The Evolved Packet System (EPS) consists of LTE and SAE. It was decided to have a ”Flat Architecture”. EPS [52] is defined to support only packet-switched traffic. It uses the concept of EPS bearers to direct the IP traffic from a gateway in the Packet Data Network (PDN) to the User Equipment (UE). EPS bearer is a virtual connection provides transport ser- vice with specific QoS attributes between the gateway and the UE. Likewise the HSPA architecture the LTE architecture also divided into two networks as radio access network and a core network. However the ultimate goal of the LTE is to minimize the number of nodes. As a result of this, the Radio Access Network (RAN) contains only one node.

2.10.1 Core Network The nodes contained in the core network are explained below [53][54]. CHAPTER 2. TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF): PCRF provides the service management and control of the LTE services. It is responsible for policy control decision making besides regulating the flow based charging functionalities in the Control Enforcement Function (PCEF). It helps to have dynamic control over QoS in turn help operators, to provide customers with a variety of QoS and charging options when opting for a service. Home Subscriber Server (HSS): HSS is the master database, it con- tains the user subscription data to support call control and session manage- ment entities. This includes the subscribed QoS profile and restrictions to access, when the subscriber is in roaming. It provides the authentication and authorization of subscribers. It holds the information related to the location of subscriber and the information about the Packet Data Network (PDN) to which the subscriber is connected. HSS also supports multi-access, multi domain networks, which provides end to end traffic handling facilities for the subscribers moving between LTE and Wireless Local Area Network (WLAN). PDN Gateway (P-GW): P-GW is responsible for allocating the dy- namic IP addresses to the subscriber and routes the user plane packets. It provides the QoS enforcement and flow based charging as per the policies in the PCRF. PCRF gives the instructions on how to deal with a particu- lar service data flow by keeping in mind the QoS terms of priority as per the subscriber profile. It acts as an anchor for mobility between 3GPP and non-3GPPP technologies. Serving Gateway (S-GW): S-GW directs data packets to all sub- scribers, which acts as a local mobility anchor for data bearer that moves between eNB hand overs. It also acts as the anchor between LTE and 3GPP technologies. It gets the information about the bearer when the UE is in an idle state. S-GW terminates the Downlink data path and also triggers paging when DL data arrives for the UE. Mobility Management Entity (MME): MME is the key control node for LTE access network that processes the signaling between UE and the Core network. It is responsible for choosing the SWG for a UE at the initial registration process and also for intra-LTE handover including Core Network (CN) node relocation. The main functions that are carried by the MME are related to the bearer management and connection management. Bearer management includes maintaining, establishment and release of the bearers. And the connection management includes establishment of the connection as well as security between the network and UE.

2.10.2 Radio Access Network The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man- ages the radio communication between the User Equipment and the Evolved CHAPTER 2. TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB. Unlike normal NodeB, eNodeB does not have centralized controller system and hence the E-UTRAN is regarded as a flat architecture. eNodeB is responsible for Radio Resource Management, which includes radio bearer control, scheduling and dynamic allocation of links to UE for uplink and downlink. Also it compresses the IP packet header for efficient use of resources. Encrypted data is sent over the radio interface for better security. eNodeB’s are connected to EPC by means of S1 interface more specifi- cally to S-GW by the S1 User plane part and to MME by the means of S1 control plane part. ENodeB’s are interconnected by means of X2 interfaces [55].

Figure 2.3: LTE Radio Access Network Design and Implementation

19 Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we used for conducting the experiments. The section consists of explanation of two experimental setups, one for the evaluation of LTE network performance with OWD, IPD and Packet loss as QoS metrics, using generated traffic for uplink and the other for video quality assessment over LTE network using subjective analysis. Before proceeding to our aim of QoE evaluation of video streaming over LTE network, we measured the QoS KPI’s of live LTE network. Because QoE and QoS are integral part of each other and the evaluation of QoE would not be complete without addressing the QoS [56].

3.1 LTE Network Performance Evaluation with Gen- erated Traffic

Quality of Service is basically technical concept and it is expressed, measured and understood in networks and network elements, which usually has little concerned to user [56]. In this section we used Distributive Passive Measurement Infrastructure (DPMI) in our experimentation, which is a dedicated hardware to perform measurements at the link level, to evaluate the performance of LTE network in terms of OWD, IPD and Packet loss for Uplink. The traffic generating tool is used to generate the TCP and UDP packets from sender system (S) to receiver system (R). The test bed is configured in such a way that the sender is connected to LTE network using Huawei E398 LTE USB modem having business type subscription and the receiver is connected to the BTH Internet. We configured our setup in such a way that the sender is able to send the TCP and UDP packets over LTE network and the receiver is able to receive those packets via BTH Internet. In between sender and receiver, we placed Measurement Point (MP) to capture the traffic. This MP gives the accurate time stamp of packets when it leaves from sender and arrives

20 CHAPTER 3. DESIGN AND IMPLEMENTATION 21 at the receiver.

3.1.1 Experimental Procedure The detailed experimental setup is shown in Figure 3.1. The packet flow in this experiment starts from the sender system, which generates the UDP packets with the help of Traffic generator and it passes to Gateway. This Gateway sends packets to receiver through LTE network. The receiver sys- tem connected to BTH Internet receives the packets. The transmitted traffic at the sender and receiver end is fed to the MP through wiretaps, which are connected to it by monitoring ports. The time stamping of packets at MP does not effect the actual packet flow, since the wiretaps duplicates the packets without disturbing the original packets.

Figure 3.1: Detailed Experimental Set up

The traffic generator tool consists of two programs, one is client program and the other is server program. The client program is installed in sender system and the server program is installed in receiver system. The client program consists [20] of Experiment number (E), Run Id (R), Key Id (K), destination IP, destination port number, number of packets to be sent, length of packets and inter frame gap in micro seconds. The server program consists of Experiment number (E), Run Id (R), Key Id (K) [20]. The collected traces at the consumer are analyzed using Perl program, based on the sequence numbers along with above mentioned E, R, K values respectively. These values are used to recognize the packets at source and destination [20]. By analyzing the collected traces we calculate the minimum, maximum and mean of OWD, IPD and packet loss percentage for different payloads at different data rates. Based on the calculated values we plotted the graphs. The main components in this setup are Gateway, Sender, Receiver, Mea- surement Point and Consumer. CHAPTER 3. DESIGN AND IMPLEMENTATION 22

3.1.2 Gateway Gateway is a Linux based computer operating on Ubuntu 12.04 OS, consist- ing of AMD Athlon processor and 2 GB RAM. The gateway is connected between sender and receiver as shown in Figure 3.1. The gateway is di- rectly connected to the sender with Ethernet cable via Broadcom Ethernet card and the LTE USB modem is connected to the gateway. We used this gateway to access the LTE network, since the sender with this LTE USB modem cannot be connected to the Measurement Point. We generate the traffic using existing traffic generators [20] at the sender system. The gen- erated packets are sent to receiver through Internet via gateway and the receiver also communicates in the same way.

3.1.3 Sender Sender is a Linux based computer operating on Ubuntu 12.04 OS, consist- ing of AMD Athlon processor and 2 GB RAM. It is connected directly to the Gateway. The sender generates the UDP and TCP traffic using traffic generator tool and it sends to receiver through Internet via gateway.

3.1.4 Receiver Receiver is a Linux based computer operating with Ubuntu 12.04 OS, con- sisting of AMD Athlon processor and 2 GB RAM. It is connected to BTH network using Ethernet. It is used as a sink, to receive the UDP and TCP traffic transmitted from the sender system using traffic generator tool.

3.1.5 Measurement Point (MP) Measurement point (MP) is a Linux based system used for getting the times- tamps for the generated packets. It is equipped with Endace DAG 3.5E cards and these are enabled in such a way to capture the traffic without loss. It is synchronized to Network Time Protocol (NTP) server and GPS (Global Positioning System) to achieve the most possible accuracy in time stamping. By this we can achieve time stamp accuracy of 60 nanoseconds. The MP filters the packets according to the rules set by Marc [20]. The wiretaps connected at the sender and receiver ends are connected to the DAG cards.

3.1.6 Consumer Consumer is a Linux based computer operating with Ubuntu 10.04 OS con- sisting of AMD Athlon processor, 2 GB RAM. It is installed with LibCa- putils. It is used to get the link level packet traces, which are broadcasted by the MP. The collected traces are in .cap format, these are converted to human readable .txt format using the LibCaputils. CHAPTER 3. DESIGN AND IMPLEMENTATION 23

3.2 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD), Inter-packet delay (IPD) and Packet loss over LTE network.

3.2.1 One Way Delay (OWD) One-way delay is the time taken by the packet to travel from source to destination. In our case, OWD of packet having the same sequence number (S) is calculated by subtracting the time stamp at receiver dag interface (T(S, Dr)) with the time stamp at sender dag interface (T(S, Ds)).

OWD = T (S, Dr) − T (S, Ds)

Dr—Dag at receiver side. Ds—Dag at sender side.

In order to get fair one-way delay measurements, the clocks should be synchronized. Some of the available synchronization methods are Network Time Protocol (NTP) and Global Positioning System (GPS). By using NTP we can able to synchronize the clocks within [10 ms, 20 ms] for WAN scenar- ios and around 1 ms in LAN scenarios. In our case, for better synchroniza- tion we used DAG cards together with GPS which synchronizes the clock of MP’s in order of 60 ns. In practical case, it is difficult to get high accurate time stamp with two DAG cards (one at sender and other at receiver). In order to avoid this minor inaccuracy we used special wiring system, where we capture traffic on singe DAG card and get time stamp from same clock [25].

3.2.2 Packet Loss (PL) The percentage of packet loss is calculated as the ratio of the number of packets lost (L) to the total number of packets sent by the sender (N). The number of lost packets is the difference in the number of packets received by receiver and the total number of packets sent by sender with unmatched sequence numbers.

PL = L/N

3.2.3 Inter Packet Delay (IPD) It is defined that the time interval between two successive packets at sender or receiver. In our experiment we calculated at receiver end and it repre- sented by following equation.

IPD = T (i + 1)r − T (i)r CHAPTER 3. DESIGN AND IMPLEMENTATION 24

T—Time of arrival. i—ith packet. r—at receiver side.

We calculated the IPD in these experiments at the receiver end and considering the no delay variation at the source side, since we observed from our traces that delay variation at source end showed a constant rate in most cases. We can also calculate the IPD at the source side, but we need further study and moreover we are more concerned about the client side (receiver).

3.3 Video Quality Assessment using Subjective Analysis

We streamed the videos without any delays and packet loss over LTE net- work and from the obtained videos we did not find significant freezes, jerk- iness in them. This might be the experiment was conducted in a region of good signal coverage. Then arises the thought of conducting the experiment in the worst network performance conditions. To achieve this, traffic shaper is used to introduce different delays and packet loss in the video streaming. In this experiment, two types of videos are taken and they are streamed over a live LTE network by introducing different delay variations and packet losses. The streamed videos are saved for video quality assessment.

3.3.1 Experimental Setup and Procedure The experiment is carried out using the experimental setup shown in Figure 3.2. It contains the server, which is operating on Ubuntu 12.04, with AMD Athlon processor, 2 GB RAM and it is connected to the BTH network. The gateway is operating on Ubuntu 12.04, with AMD Athlon processor, 2 GB RAM and it is connected to LTE network via USB modem. The client runs on Ubuntu 12.04, with AMD Athlon processor and 2 GB RAM. It is connected to gateway via Ethernet full duplex link, bandwidth of 100 Mbps. The client system receives packets from server through LTE network via gateway. The Measurement point is connected in between the sender and the client. The MP consisting of two DAG cards, one of them is connected to capture the traffic at the server side and the other DAG card is used to capture the traffic at the client side. The traces are collected and saved in the consumer system. The Wowza Media Server software is installed in the server system and the videos are encoded to the required format and placed in the server. In this thesis, the video streaming used is Video on Demand. Before streaming the video, required delay and packet loss settings are made at traffic shaper which is installed on server system. Videos are streamed from server to client using TCP and UDP protocols as per the request of client. At client side CHAPTER 3. DESIGN AND IMPLEMENTATION 25 we used the VLC player to access the videos. The VLC media player has a feature to save different videos that are being streaming over network. With this option in VLC player we saved the streamed videos in the client system. These streamed videos are saved in the client system. The saved videos are further used to get the Mean Opinion Score from the users. We performed our experiments in months of January, February and March between 8 AM to 6 PM. We repeated the experiments to provide reliability in results and to reduce the chance errors.

Figure 3.2: Experimental Set up for Video Streaming

3.3.2 WOWZA Media Server Wowza Media Server is a software used to stream video and audio files over public and private networks. It can stream Live video streaming, Video on Demand and Video recording over Adobe Flash player, Microsoft Sil- verlight player, Apple iPhone, iPad, iPod touch, Quick Time player, Smart phones devices, tablets and IPTV set-top boxes. The supporting protocols of Wowza Media Server are Real Time Messaging Protocol (RTMP), Microsoft Smooth Streaming, Apple HTTP Live Streaming (HLS), Real Time Stream- ing Protocol (RTSP), Adobe HTTP Dynamic Streaming (HDS), Real-time Transport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS) [57]. It is an alternative to Adobe Media Server, Darwin Streaming Server, Microsoft IIS Media Services, and other media servers. In our thesis we used Wowza media server and installed on server system, the encoded videos are saved in the server machine with a specific name. These videos are accessed from client system using VLC player which is installed on it.

3.3.3 Test Video Parameters We selected two types of videos for our experimentation. One is a Rugby game video, which comes under the fast moving video and the other is Big CHAPTER 3. DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video, which comes under slow moving video. We covered the two types of videos suggested by ITU-T [58]. With the availability of high speed data networks, users are more at- tracted to view high definition videos. There is a scope in this direction, to test the high definition videos over high speed data networks. So, we used 720p High Definition videos with resolution 1280 × 720 pixels and is coded with a Main Profile encoder at Level 3.1 [28]. These two raw videos used for video quality testing are taken from test media website [59]. These two videos are encoded to the required format using FFmpeg encoder [60]. The duration of selected video sequence is about 4 minutes. In [24], performance of 3G data services was observed by taking packet data streams for 5 minutes. To observe the user QoE for video streaming over radio network, a minimum 4 minutes length of video is considered as appropriate. We chose the .mp4 as a container, since it is supported by most of the latest smart phones based on Android and iPhone [61][62]. We selected H.264/AVC as video codec, because it has more advantages than MPEG-2 and MPEG-4 Visual in better image quality at same com- pressed bit rate [28]. H.264 offers greater flexibility in terms of transmission support and compression options.

Video sequences Rugby , Big Bucks Bunny Codec Perceptible, H.264 Main Profile, Level 3.1 Resolution 720p, (1280 × 720) Frame Rate 25fps Container MP4

Table 3.1: Test Video Parameters

3.3.4 NetEm In our thesis we used NetEm traffic shaper for network emulation to vary the performance parameters like delay variation and packet loss. It is in- stalled on server system which is operating on Ubuntu 12.04 operating sys- tem shown in Figure 3.2. The main motivation to use NetEm is, it provides long distance network scenarios in the lab environment. In [63][64] authors showed that performance of NetEm is more reliable as compared to NIST Net and KauNet. Two traffic control facilities (packet loss and delay) are used in this thesis. The NetEm delivers each packet that flows through it with certain delay that should be in the delay range given to it. In the case of packet loss the amount of required packet loss must be in percentage form. It drops some packets randomly as per the given loss percentage before they are queued. CHAPTER 3. DESIGN AND IMPLEMENTATION 27

3.3.5 Delay To choose the packet delay variation values range, some set of laboratory tests were performed for final assessment. Constant delay values were intro- duced from 100 ms to 500 ms and we found significant changes at 150 ms of constant delay. In case of delay variations, we introduced ∆D from 0 to 50 ms with an increment of 5 ms and we could not find significant changes. We repeated the test with values from 0, 10, 25, 50 and 100. We observed significant changes in the range of constant delay 150 ± {0, 10, 25, 50, 100} and similar settings are used in [65].

3.3.6 Packet Loss One or more packets that originate from the source being transmitted across the network, fail to reach the destination. This shows major impact on the performance of the network and causes significant problems in applications like Video conference, VoIP and Video streaming. The packet loss is mea- sured in percentage of packets lost from the overall transmitted packets. To choose the packet loss variation values range, some set of laboratory tests were performed similarly for delay. Significant variations in video were observed for values 2%, 4%, 6% and 8% and similar settings are used in [65].

3.3.7 Assessment of Videos For the purpose of subjective video quality analysis of the users, we devel- oped an assessment tool. The tool is designed to have a graphical interface to the users to give a rating for each video based on its video quality. The tool contains front end developed in PHP and the back end in MySQL, which used to store the video ratings of the users. A screen-shot of the tool is given below.

Figure 3.3: Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendation. Before starting the experiment the users are given a questionnaire about CHAPTER 3. DESIGN AND IMPLEMENTATION 28 the details and expert level of the users. In the demo a brief explanation is given about how to use the assessment tool which we used for taking MOS rating. The users are asked to watch the videos with full screen mode for quality rating of the videos [58]. Results and Analysis

29 Chapter 4

Results and Analysis

4.1 Analysis of LTE Network Performance with Generated Traffic

We analyzed the traces got from the experimental procedure. Firstly we calculated the OWD, IPD and Packet loss using the Perl code.

4.1.1 One Way Delay for UDP packets in Uplink We calculated the One-way delay of the generated packets and we found the minimum OWD, maximum OWD and mean OWD.

4.1.1.1 Minimum One Way Delay The minimum OWD is the best possible performance provided by the net- work operator.

Figure 4.1: Minimum One way Delay for Generated UDP Packets for Uplink

30 CHAPTER 4. RESULTS AND ANALYSIS 31

The Figure 4.1 shows the minimum One-way Delay for the UDP packets for Uplink transmitted over LTE network. This minimum One-way Delay shows the best performance that can be expected for a particular size of packets at different payloads. The minimum OWD delay range for LTE network is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and data rates 64 kbps to 8 MB. While in 3G network the OWD in uplink varies from 80 ms to 250 ms by neglecting the rapid increase at the starting stage from 64 to 252 bytes, which occurred due to the technology shift in the network operator of WCDMA to HSDPA [11].

4.1.1.2 Maximum One Way Delay Maximum OWD is the worst performance of the network at the particu- lar instant. There may be different reasons for this network behaviour, to explain this we need further investigation.

Figure 4.2: Maximum One way Delay for Generated UDP Packets for Uplink

Most of the maximum OWD does not exceed 200 ms, the remaining values that occurred are due to the disturbances in the network at that instant. The worst OWD is 1060 ms. These maximum values effect the mean of OWD calculations.

4.1.1.3 Mean One Way Delay The mean OWD shown in the graph below gives the average of OWD for different payloads at different data rates. The mean OWD is the average of minimum OWD and maximum OWD. This mean of OWD is mainly influenced by the maximum values we got, which may be due to the external disturbances occurred at that instant. CHAPTER 4. RESULTS AND ANALYSIS 32

Figure 4.3: Mean One way Delay for Generated UDP Packets for Uplink

4.1.2 Inter Packet Delay for UDP packets in Uplink It is defined as the time interval between the two successive packets at sender or receiver side. In our experiment we calculated it at the receiver side. We calculated the IPD from the collected traces and found the minimum, maximum and mean IPD from it. The variation of Inter Packet Delay is known as Jitter. There is always a little amount of jitter present in the network. The jitter preferences are varied with type of traffic class and type of applications. The variation of Inter Packet Delay is known as Jitter. There is always a little amount of jitter present in the network. The jitter preferences are varied with type of traffic class and type of applications.

4.1.2.1 Minimum Inter Packet Delay Minimum IPD is the best possible performance that the network operator provides for that service. At lower data rates the IPD value increases linearly as the size of data increases. As the data rates increases the IPD values are very small. To explain this behaviour we need further investigation. For the small data rates 64 kbps, 128 kbps and 512 kbps the IPD values increase as the payload increases. CHAPTER 4. RESULTS AND ANALYSIS 33

Figure 4.4: Minimum Inter Packet Delay for Generated UDP Packets for Uplink

4.1.2.2 Maximum Inter Packet Delay In Quality of Service the weight of each KPI varies from application to application, so the variation in IPD ie., jitter preferences are also varies with different applications. The jitter has almost no impact on performance in case of file downloading, but it has significant impact on applications like streaming and video sharing.

Figure 4.5: Maximum Inter Packet Delay for Generated UDP Packets for Uplink

The maximum IPD values for UDP packets in Figure 4.5 shows uneven increase and drops, the reason for these behaviour need further investigation. CHAPTER 4. RESULTS AND ANALYSIS 34

4.1.2.3 Mean Inter Packet Delay

Figure 4.6: Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD values. In Figure 4.6 the mean IPD for UDP packets showed linear increase in IPD values with increase in packet size at lower data rates.

4.1.3 Packet Loss for UDP in Uplink

Figure 4.7: Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure 4.7. In our experiment, the packet loss effect for UDP packets at different data rates with varying payloads is not much significant. The few instances CHAPTER 4. RESULTS AND ANALYSIS 35 showing packet loss in the Figure 4.7 are not considerable loss compared to total number of packet sent and those are not even up to 1% of packet loss.

4.1.4 One Way Delay for TCP packets in Uplink We calculated the OWD, IPD and Packet loss for generated TCP packets from the collected traces. Then we found the minimum, maximum and mean for the One-way delay, Inter-packet delay and Packet loss percentage. We drew graphs regarding each case for clear understanding.

4.1.4.1 Minimum One Way Delay Minimum OWD is the possible best performance of network that the net- work operator can provide to users.

Figure 4.8: Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 4.8. Its OWD values vary from 20 ms to below 30 ms. For the UDP packets also it showed a similar behaviour. We did not find any significant difference in its behaviour for minimum OWD.

4.1.4.2 Maximum One Way Delay We found the maximum delay from calculated delays and we considered it as the bad performance of network at that time. CHAPTER 4. RESULTS AND ANALYSIS 36

Figure 4.9: Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under- standing. The maximum OWD value for TCP packets over LTE network is having a range of 50 ms to 500 ms for most of the values. It also have few extreme values like 1260 ms, 850 ms, 890 ms.

4.1.4.3 Mean One Way Delay From the calculated OWD’s we found mean OWD, which gave the overall behaviour of network by considering the best and worst OWD values.

Figure 4.10: Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 4.10 we represented the mean OWD for TCP packets over LTE network. It showed slight increase in the OWD as the payload and data rate increase. It showed some extreme value for 600 bytes payload at CHAPTER 4. RESULTS AND ANALYSIS 37

1.5 mbps data rate. It may be caused due to the worst network service at that particular time.

4.1.5 Inter Packet Delay for TCP packets in Uplink We calculated the IPD for TCP packets similar to the calculation of UDP packets. IPD is defined as the time interval between the two successive packets at sender or receiver. In our experiment we calculated it at the receiver side. We calculated the IPD from the collected traces and then we found the minimum, maximum and mean IPD from it.

4.1.5.1 Minimum Inter Packet Delay The network having low and constant IPD is the best network, it is also known as the Delay-Jitter. In several multimedia applications this jitter performance is very crucial.

Figure 4.11: Minimum Inter Packet Delay for Generated TCP Packets for Uplink

Minimum IPD is the best possible performance that the network opera- tor provides for that service. Similar to UDP, for TCP also the IPD values increase linearly with the increase in packet size at lower data rates in Figure 4.11. CHAPTER 4. RESULTS AND ANALYSIS 38

4.1.5.2 Maximum Inter Packet Delay

Figure 4.12: Maximum Inter Packet Delay for Generated TCP Packets for Uplink

Similar to the UDP, the maximum IPD values for TCP packets in Figure 4.12 also shows uneven increase and drops, the reason for these behaviour need further investigation.

4.1.5.3 Mean Inter Packet Delay

Figure 4.13: Mean Inter Packet Delay for Generated TCP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD values. Similar to UDP, in the Figure 4.13 TCP packets also showed linear increase CHAPTER 4. RESULTS AND ANALYSIS 39 in IPD values with increase in packet size at lower data rates. Along with that it also showed abnormal values at higher data rate 4 mb.

4.1.6 Packet Loss for TCP in Uplink

Figure 4.14: Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure 4.14. Similar to UDP, the packet loss effect for TCP packets at different data rates with varying payloads is much significant. The few instances showing packet loss in the Figure 4.14 are not considerable loss compared to total number of packet sent and those are not even up to 1% of packet loss.

4.2 Gateway Evaluation

In order to know the effect of gateway in the measurements, we conducted an experiment using the USB Ethernet device. The LTE USB modem is re- placed with the ASUS USB 2.0 to Fast Ethernet Adapter and it is directly connected to the wiretap of receiver system with one meter long Ethernet cable. This gives the same treatment to packets as we are using the USB Ethernet Adaptor. We generated the UDP traffic of different payloads at different IPD’s and sent from Sender to Receiver. We evaluated the mini- mum, maximum and mean of OWD by analyzing the captured traffic. From the obtained results of gateway evaluation, the maximum OWD which is in micro seconds (less than 1 ms) and the obtained packet loss percentage is very low, so we considered both as negligible. The similar case is observed in the paper [59] where packet loss and delay are neglected. CHAPTER 4. RESULTS AND ANALYSIS 40

4.3 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTE network, we calculate OWD from the collected traces for TCP and UDP packets. Taking the data rate as constant, which is 4 mbps and the size of packet is varied.

Figure 4.15: Minimum One Way Delay for Generated TCP and UDP pack- ets for Uplink

From the Figure 4.15 the UDP packets experiencing the minimum delay vary from 22 ms to 25 ms, and graph showing a behaviour of sawtooth wave form. The TCP packets experience more variations in delay pattern compared to UDP packets. The minimum OWD vary from 23 ms to 28 ms.

4.4 One Way Delay for Video Streaming Over LTE

The two phases of our work, firstly the measuring OWD of generated packets over LTE and secondly with streaming video over LTE network. By analyz- ing the traces of video packets, we observed that the data payloads MTU (Maximum Transmission Unit) are 1500 bytes. We analyzed the OWD of 1500 data size packets at different data rates, we presented the OWD for multimedia application packets. We plotted the graphs for the OWD for the packets of 1500 bytes at different data rates. We generated UDP packets using the traffic generator programs written C++. We also plotted the graphs for TCP and UDP protocols. Thus we can observe the behaviour of multimedia packets over LTE network. The following graph shows the OWD of packets having unit size of 1500 bytes, which resembles the multimedia packets. In the Figure 4.16 graph the OWD of 1500 bytes size packets shows CHAPTER 4. RESULTS AND ANALYSIS 41

Figure 4.16: Minimum One Way Delay for Generated TCP and UDP packet size of 1500 bytes for Uplink a standard delay up to 1.5 mb and after it was subjected to some linear decrease in delay for higher data rates. This behaviour is followed for all sets of payloads. We predict that this behaviour is due to Scalable Bandwidth, which is one of the significant feature of LTE technology. Scalable Bandwidth is one of the excellent advantage of LTE network over 3G systems. LTE system provides the scalable bandwidth up to 20 MHz, covering 1.4, 3, 5, 10, 15 and 20 MHz.

4.5 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks, the facts provoked us to find the QoE of video streaming over mobile network like LTE [1]. The main idea behind this part of the thesis is, to present QoE of video streaming over the LTE network. In this chapter the analysis of the results obtained from the subjective video quality assessment are presented. The mean, standard deviation and confidence interval for the MOS collected from different subjects were calcu- lated and the results of the video quality assessment were plotted. In detail, this experiment has 30 subjects who were asked to give MOS for a sequence of 40 videos. These videos include the videos captured at different delay variations and different packet losses for both TCP and UDP. In turn for each protocol we chose two different videos, one was fast moving video and the other slow moving video was used. For these accessed videos subjec- tive video quality assessment was conducted using our own assessment tool and the collected MOS are stored in database. From these collected MOS, the mean, standard deviation and confidence interval were calculated and presented in the graphs. The mean is defined as the average and it is computed as the sum of all the observed outcomes from the collected samples divided by the total number of events. CHAPTER 4. RESULTS AND ANALYSIS 42

The Mean is defined as:

N 1 X U¯ = U¯ (4.1) jk N ijk i=1 The Standard Deviation (SD) is defined as: v u N ¯ ¯ 2 uX (Ujk − Uijk) § =t (4.2) jk N − 1 i=1

th Where: N is the number of observers, Uijk is the score of i observer for test condition j, video sequence k.

Confidence Interval (CI): As recommended by the International Telecommunications Union- Ra- diocommunications Sector (ITU-R) in the recommendation BT-500 we used 95% confidence interval. This means “With a probability of 95%, the ab- solute value of the difference between the experimental mean score and the ‘true’ mean score (for a very high number of observers) is smaller than the 95% confidence interval” [66]. The confidence interval consists of an upper and a lower limit for the mean U¯. (U¯jk − δjk, U¯jk + δjk) (4.3) where Sjk δjk = 1.96√ N

Sjk is Standard Deviation calculated from the equation 4.2

4.5.1 Packet Delay Variation 4.5.1.1 Packet Delay Variation for TCP The MOS obtained from the subjective video quality assessment for delay variations are plotted for fast and slow videos of TCP and UDP protocol. The graph is plotted by considering the MOS on Y- axis from one to five. The delay variations are considered on the X- axis, the constant delay (D) is maintained at 150 ms and the ∆D varying delay is taken as mentioned in the Table 4.1. The units for both D and ∆D are in milliseconds. The graph of Delay variation verses MOS of TCP protocol for both fast and slow moving videos are plotted. Total five different values are taken for ∆D from 0 to 100 along with a single value for constant delay. Only with constant delay 150 ms and with no ∆D i.e., 150 ms±0 ms the MOS obtained for both fast and slow moving videos are more than perceptible and are not annoying. This suggests that videos are more than good and the user does CHAPTER 4. RESULTS AND ANALYSIS 43

Figure 4.17: MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny 150±0 4.15 4.62 4 4.04 150±10 3.41 4.19 4.04 3.92 150±25 3.31 4.19 3.85 3.84 150±75 2.92 4.08 3.65 3.31 150±100 1.96 2.5 3.31 3.08

Table 4.1: MOS for Delay Variation not have any problem with the videos. For the next increased level at 10 ms the slow moving video is maintained with a value slightly below the previous MOS rating than the ∆D 0 ms and still maintains a user perception level more than good. But for the fast moving video the MOS drops below the perceptible level. For the ∆D at 25 ms the MOS obtained for both the videos show that they are nearly the same when compared to the previous values, this specifies that, in spite of the increased delay by 15 ms of the previous value does not affect the quality of the video. By further increase of ∆D to 75 ms shows little drop in MOS for slow video, still maintains a MOS of perceptible level. In case of fast video the MOS Drops little more amount when compared to the previous level and maintains below the slightly annoying level. By further increase of ∆D to 100 ms, at this value the delay shows significant impact on the video quality of both the fast and slow moving videos. From the graph, it shows a sudden drop in the MOS readings and the slow moving video maintains a little bit above the annoying level. But for the fast moving video the MOS drops to an annoying level. By considering the total behaviour of the MOS readings for both fast and slow moving videos of TCP protocol, it is observed that the slow moving video maintained better video quality than the fast moving video. Because the fast moving video contains bit rate higher than the slow moving video. On the same basis, by taking account of TCP protocol in this scenario, the CHAPTER 4. RESULTS AND ANALYSIS 44

TCP protocol resends the data, if ever the data is lost. So this in turn, introduces delays and decreases the video quality. Hence the MOS of a slow moving video performed well than fast moving video.

4.5.1.2 Packet Delay Variation for UDP

Figure 4.18: MOS for UDP videos subjected to Delay Variation

The Figure 4.18 shows the delay variations of fast and slow moving videos of UDP protocol. Similar to the previous graph same five values are maintained in this case also. For the constant delay 150 ms and ∆D at 0 ms both the fast and slow moving videos got same MOS of perceptible range and have good video quality. The ∆D at 10 ms both got almost equal MOS and are still maintain perceptible video quality and are good. By further increasing the ∆D to 25 ms the MOS still got nearly same for both fast and slow moving videos. MOS of those videos drops little less than the previous value and maintains slightly below the perceptible and are slightly annoying. For the ∆D value at 75 ms the MOS for both the videos falls to the region slightly above annoying. From this point the MOS of slow moving video starts degrading more when compared to the fast moving video. At ∆D is 100 ms the MOS for both the videos degrades further and the MOS for slow moving video degrades more than the fast moving video and tends to catch the slightly annoying level of video quality. From the graphs it is observed that the MOS for both the videos started at same MOS rating and maintains the same till the ∆D is equal to 25 ms. Because the UDP protocol sends the data packets irrespective of, whether the data packets reaching the destination or not. For this reason ∆D up to 25 ms shows the same effects on both the fast and slow moving videos. Beyond the ∆D 25 ms the MOS rating for both fast and slow moving videos showed almost the same. There is not much difference in the MOS ratings CHAPTER 4. RESULTS AND ANALYSIS 45 for fast and slow moving videos when compared to the TCP protocol.

4.5.1.3 Standard Deviation for Delay Variation

Figure 4.19: Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variations are calculated and the graph is plotted. As shown Figure 4.19, this is the standard deviation graph plotted for both types of videos and for both the protocols too. The graph was plotted by taking standard deviation on Y- axis and packet delay variation on X-axis. From the graph it was observed that low standard deviation was noted in the case of fast moving video of TCP protocol. The standard deviations obtained from the MOS where ranging between 0.491 and 0.935. It was also observed that, as the packet delay variation increases the overall standard deviation also increases. This is because of the video quality degrades, the users are in a state of ambiguity to rate the quality of a video.

4.5.1.4 Confidence Interval for Delay Variation The 95% confidence interval for the packet delay variations are calculated and the graph is plotted. As shown in Figure 4.20, is the 95% confidence interval graph plotted for both types of videos and for both the protocols too. The graph was plotted by taking MOS on Y-axis and packet delay variation of the video sequences on X-axis. From the graph it is observed that 95% confidence interval is high for videos of TCP in all packet delay variations except the packet delay variation at 100 ms. CHAPTER 4. RESULTS AND ANALYSIS 46

Figure 4.20: Confidence Interval for Delay Variation

4.5.2 Packet Loss The MOS rating of packet loss is taken for five successively increasing values. The MOS rating are for fast and slow moving videos in both the TCP and UDP protocol as shown in the Table 4.2 below.

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny 0% 3.96 3.88 3.904 3.97 2% 2.08 3.54 2 1.73 4% 1.62 2.42 1.2 1.68 6% 1.42 1.92 1.2 1.11 8% 1.35 2.5 1.58 1

Table 4.2: MOS for Packet Loss

4.5.2.1 Packet Loss for TCP Figure 4.21 is the graph of packet loss for TCP protocol and on X-axis packet loss it is taken for five different values and on Y-axis is for MOS. From the graph, it is observed that at zero percent packet loss the MOS rating obtained is just below the perceptible level and the video quality of both videos are good. If the packet loss is increased to 2% the slow moving video drops a little amount and stands in slightly annoying level and the user considers it as a fair video. But in the case of fast moving video a sudden drop in MOS is observed and it stands just above the annoying level. By further increase in packet loss to 4% MOS drop is observed in slow moving video and the MOS stands above the annoying level. But in the case of fast CHAPTER 4. RESULTS AND ANALYSIS 47

Figure 4.21: MOS for TCP videos subjected to Packet loss moving video, drop is observed and the MOS falls into very annoying level. At packet loss 6% and 8% a slight drop is observed in both videos compared to the packet loss at 4%. From the graph it is observed that the video quality of fast moving video is degraded more when compared to slow moving video. This may be in fast moving video the bit rate is higher that the slow moving video. TCP retransmits the packets, when the packets were lost. Hence this leads to congestion of packets in fast moving video and degrades the video quality of fast moving video.

4.5.2.2 Packet Loss for UDP

Figure 4.22: MOS for UDP videos subjected to Packet Loss

The Figure 4.22 is the packet loss graph plotted for fast and slow moving videos of UDP protocol. In this graph, packet loss percentage is taken on CHAPTER 4. RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis. Under no loss i.e., packet loss is 0%, for both fast and slow moving videos the MOS rating is in perceptible level and user feels both the videos are good. On increasing packet loss, at 2% the video quality degrades and can observed from graph a sudden fall in MOS rating. By further increase in packet loss, at packet loss 4% the MOS for slow moving video is slightly increased when compared to the packet loss at 2%. This may be due to delays in the network, as the experiment is conducted on a real time LTE network. There is a scope for getting network delays. But in case of fast moving video the video quality is further degrades and the MOS drops slightly above to the very annoying level. When the video quality of fast moving video degrades to bad quality and there is almost no scope for further degradation of video quality at packet loss of 6% and 8%. But in the case of slow moving video it is different, packet loss at 6% and 8% the video quality totally degraded and it is even difficult to play. The MOS rating obtained from this video is bad. From the graph it is observed that even when the packet loss is 2% there is a sudden drop in the MOS rating and this is because, there is no retransmission of lost packets in the UDP protocol. So the video quality is observed as very annoying. A slight increase of MOS rating is observed in slow moving video even with higher amount of packet loss and this may be due to the varying delays in the live LTE network. High definition videos are used for the experiment because of this, much degradation of video quality was observed.

4.5.2.3 Standard Deviation for Packet Loss

Figure 4.23: Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentage are calculated and graph is plotted. The standard deviation graph plotted CHAPTER 4. RESULTS AND ANALYSIS 49 for both types of videos and for the both the protocols too. The graph was plotted by taking standard deviation on Y-axis and packet loss percentage on X-axis. The standard deviations obtained ranged between 0.277 and 0.857. From Figure 4.23 it is observed that as the packet loss percentage increases the overall standard deviation decreases. This is because, as video quality degrades, the certainness for grading same for video increases and the user regards the video as bad.

4.5.2.4 Confidence Interval for Packet Loss

Figure 4.24: Confidence Interval for Packet Loss

The 95% confidence interval for the packet loss percentage was calculated and the graph plotted. As shown Figure 4.24, this is the 95% confidence interval graph plotted for both types of videos and for both the protocols too. The graph was plotted by taking MOS on Y-axis and packet loss percentage of the video sequences on X-axis. From the graph, it is observed that as the packet loss percentage increases, the 95% confidence interval drops close to the bad video quality level. Conclusion and Future Work

50 Chapter 5

Conclusion and Future Work

5.1 Conclusion

This thesis work is based on the performance of video streaming over live LTE network. The whole thesis work is divided into two phases based on the aimed experiment and expected outcomes. The first part of the thesis work deals with measurement of one way delay, jitter and packet loss in the uplink of the LTE network. For this work we generated an artificial traffic with UDP and TCP packets of different payload sizes. This artificial traffic is streamed from sender to receiver with different payload sizes and at random IPD. The minimum, maximum and mean one-way delay were calculated for the streamed payload sizes at different data rates. The graphs are plotted for minimum, maximum and mean one-way delay of different payloads and data rates. Like one-way delay, same calculations were also done for IPD and graphs were plotted. In case of packet loss, we calculated the number of packets lost at different data rates for varying payloads. The whole set of analysis is done for both TCP and UDP of uplink in LTE network. From our analysis with collected data we observed that, there is no significant effect of protocol on OWD, IPD and Packet loss. So, this concludes the first research question. We also tried to map the data obtained from with artificial traffic with video traffic by analysing the traces of video, we observed that maximum amount of data transmission is done around payload sizes of 1500 bytes. So we select the traces of 1500 bytes payload at different data rates and plot the graph for TCP and UDP packets. We observed a decline of one-way delay from 1.5 Mbps data rate. Reasons for this decline were discussed in the analysis part. The experimental setup is modified as it is suitable to stream video from server to client with emulated packet loss and packet delay in the setup. The raw videos are encoded into the H.264 Mainline profile with the help of FFmpeg encoder. The encoded videos are streamed through emulated packet delay variation in NetEm traffic shaper from server to client, on

51 CHAPTER 5. CONCLUSION AND FUTURE WORK 52 client request. Like packet delay variations same process is repeated for packet loss too and the videos are saved for further subjective analysis of the users. Statistics like mean, Standard Deviation (SD) and 95% Confidence Interval (CI) is calculated for both fast and slow moving videos of TCP and UDP protocols. The delay variation in the videos of TCP was observed and a slight fall in both fast moving and slow moving videos observed a change at delay variation 10 ms when compared to the no delay variation case. The video quality is maintained well even as the delay variation is increased up to 75 ms At 100 ms a drastic fall in the video quality is observed and the user considered slow videos as annoying and fast moving videos as very annoying. But in the case of videos of UDP almost same video quality is maintained for both fast and slow moving videos and slight degradation of video quality is observed up to 25 ms. As further increase of packet delay variation there is no significant degradation of video quality observed for both the videos up to 100 ms. The user considered both videos as slightly annoying at 100 ms delay variation. So this concludes the second research question. From the results of packet loss percentage variations it was observed that a drastic fall in the quality of fast moving video of TCP protocol even at 2% packet loss and user considered this video as annoying. But the slow moving video of TCP protocols is better sustained compared to the fast moving video. Both videos are considered as very annoying from 6% packet loss. The video quality is even more degraded for both the videos in UDP protocol. The quality of the video is degraded drastically from slightly annoying to very annoying at 2% packet loss. The quality went even worse and it was hard to play as further increase in packet loss percentage. So this concludes the third research question.

5.2 Future Work

In this thesis, the videos used for streaming is H.264/AVC and there is lot of scope to use video that is encoded to H.264/SVC scalable videos (SVC). Scalable video coding allows video conferencing devices to send and receive multi layered video streams. In those multi layered streams, a small base layer is present along with some optional layers and that help to improve resolution frame rate and in turn quality. In this thesis the Quality of Service (QOS) evaluation is limited to uplink only, so this work can be extend for downlink. The reasons for abnormal values in OWD, IPD and Packet losses are not fully investigated and the patterns for IPD at lower data rates are yet to be investigated. There is a need to improve the mapping of Quality of Service values with Quality of Experience. Bibliography

[1] T. Cisco, “ Cisco Visual Networking Index : Global Mobile Data Traffic Forecast Update,” CISCO, Research Report, 2012.

[2] S. P. Dahlman, Erik and J. Skold, 4G: LTE/LTE-Advanced for Mobile Broadband. Oxford, OX5 1GB, UK: Academic Press, 2011.

[3] Epitiro, “Quality of Experience Measurement,” Epitiro Ltd., White Paper, 2011.

[4] IXIA, “Quality of Service ( QoS ) and Policy Management in Mobile Data Networks,” IXIA, Research Report, 2011.

[5] (2013) Vocabulary for Performance and Quality of Service. http://www.itu.int. [Online]. Available: http://www.itu.int/rec/ T-REC-P.10

[6] (2013) Statistics. youtube.com. [Online]. Available: http://www. youtube.com/yt/press/statistics.html

[7] H.-J. Kim, D.-G. Yun, H.-S. Kim, K.-S. Cho, and S.-G. Choi, “QoE assessment model for video streaming service using QoS parameters in wired-wireless network,” in Advanced Communication Technology (ICACT), 2012 14th International Conference on, 2012, pp. 459–464.

[8] I. Ketyko, K. De Moor, W. Joseph, L. Martens, and L. De Marez, “Per- forming QoE-measurements in an actual 3G network,” in Broadband Multimedia Systems and Broadcasting (BMSB), 2010 IEEE Interna- tional Symposium on, 2010, pp. 1–6.

[9] X. Yu, H. Chen, W. Zhao, and L. Xie, “No-Reference QoE Prediction Model for Video Streaming Service in 3G Networks,” in Wireless Com- munications, Networking and Mobile Computing (WiCOM), 2012 8th International Conference on, 2012, pp. 1–4.

[10] T. N. Minhas, “Network Impact on Quality of Experience of Mobile Video,” Ph.D. dissertation, Blekinge Institute of Technology, 2012.

53 BIBLIOGRAPHY 54

[11] A. Alvarez, S. Cabrero, X. Paneda, R. Garcia, D. Melendi, and R. Orea, “A flexible QoE framework for video streaming services,” in GLOBE- COM Workshops (GC Wkshps), 2011 IEEE, 2011, pp. 1226–1230.

[12] A. Khan, L. Sun, E. Ifeachor, J. O. Fajardo, and F. Liberal, “Video Quality Prediction Model for H.26 video over umts networks and their application in mobile video streaming,” in Communications (ICC), 2010 IEEE International Conference on.

[13] T. N. Minhas and M. Fiedler, “Quality of Experience Hourglass Model,” in Computing, Management and Telecommunications (ComManTel), 2013 International Conference on, 2013, pp. 87–92.

[14] T. Minhas, O. Gonzalez Lagunas, P. Arlos, and M. Fiedler, “Mobile video sensitivity to packet loss and packet delay variation in terms of QoE,” in Packet Video Workshop (PV), 2012 19th International, 2012, pp. 83–88.

[15] S. Hossen and N. Islam, “QoS Performance Evaluation of Video Con- ferencing over LTE ,” Master’s thesis, Blekinge Institute of Technology, Sweden, 2012.

[16] (2013) Homepage. opnet.com. [Online]. Available: http://www.endace. com

[17] P. Arlos, R. Kommalapati, and M. Fiedler, “Evaluation of protocol treatment in 3G networks,” in Local Computer Networks (LCN), 2011 IEEE 36th Conference on, 2011, pp. 688–696.

[18] (2013) Homepage. Endace.com. [Online]. Available: http://www. endace.com

[19] D. Kim, H. Cai, and S. Choi, “Measurement and Analysis of One-Way Delays over IEEE 802.16e/WiBro Network,” in Vehicular Technology Conference Fall (VTC 2009-Fall), 2009 IEEE 70th, 2009, pp. 1–5.

[20] K. Vamsi Krishna and D. Praveen, “Impact of Transmission Patterns on one-way Delay in 3G Networks of Sweden ,” Master’s thesis, Blekinge Institute of Technology, Sweden, 2011.

[21] K. Gatimu, T. Johnson, M. Sinky, J. Zhao, B. Lee, M. Kim, H.-S. Kim, C. gone Kim, and J.-S. Baek, “Evaluation of wireless high definition video transmission using H.264 over WLANs,” in Consumer Commu- nications and Networking Conference (CCNC), 2012 IEEE, 2012, pp. 204–208.

[22] A. Khan, L. Sun, E. Ifeachor, J. O. Fajardo, and F. Liberal, “Video Quality Prediction Model for H.264 Video over UMTS Networks and BIBLIOGRAPHY 55

Their Application in Mobile Video Streaming,” in Communications (ICC), 2010 IEEE International Conference on, 2010, pp. 1–5.

[23] (2013) Perceptual Evaluation Video Quality. www.pevq.org. [Online]. Available: http://www.pevq.org/perceptual-evaluation-video-quality. html

[24] R. Kommalapati, “Performance of 3G data Services Over Mobile Net- work in Sweden,” Master’s thesis, Blekinge Institute of Technology, Sweden, 2010.

[25] P. Arlos, “On the Quality of Computer Network Measurements,” Ph.D. dissertation, Blekinge Institute of Technology, 2005.

[26] M. Ries, P. Froehlich, and R. Schatz, “QoE evaluation of high-definition IPTV services,” in Radioelektronika (RADIOELEKTRONIKA), 2011 21st International Conference, 2011, pp. 1–5.

[27] W. Lu, A. Varna, and M. Wu, “Secure video processing: Problems and challenges,” in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, 2011, pp. 5856–5859.

[28] S. G. F. Breu and J. Wollmann, The H.264 Advanced Video Compres- sion Standard. West Sussex, UK: John Wiley and Sons Ltd, 2008.

[29] J. Ohm and G. Sullivan, “High efficiency video coding: the next fron- tier in video compression [Standards in a Nutshell],” Signal Processing Magazine, IEEE, vol. 30, no. 1, pp. 152–158, 2013.

[30] A. Gupte, B. Amrutur, M. Mehendale, A. Rao, and M. Budagavi, “Memory Bandwidth and Power Reduction Using Lossy Reference Frame Compression in Video Encoding,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 21, no. 2, pp. 225–230, 2011.

[31] S. Issa and O. Khalifa, “Performance analysis of Dirac video codec with H.264/AVC,” in Computer and Communication Engineering (ICCCE), 2010 International Conference on, 2010, pp. 1–6.

[32] S. R. Subramanya, “Video containers: a system for the on-demand stor- age, delivery, and management of television programs,” in Multimedia and Expo, 2000. ICME 2000. 2000 IEEE International Conference on, vol. 3, 2000, pp. 1245–1249 vol.3.

[33] C.-R. Lan, H.-H. Lu, C.-W. Yi, and C.-C. Tseng, “A P2P HD Live Video Streaming system,” in Multimedia Technology (ICMT), 2011 In- ternational Conference on, 2011, pp. 475–478. BIBLIOGRAPHY 56

[34] C. Zheng and Z. Ming, “An efficient video similarity search strategy for video-on-demand systems,” in Broadband Network Multimedia Tech- nology, 2009. IC-BNMT ’09. 2nd IEEE International Conference on, 2009, pp. 174–178.

[35] A. Begen, T. Akgul, and M. Baugher, “Watching Video over the Web: Part 1: Streaming Protocols,” Internet Computing, IEEE, vol. 15, no. 2, pp. 54–63, 2011.

[36] D. Wu, Y. Hou, W. Zhu, Y.-Q. Zhang, and J. Peha, “Streaming video over the Internet: approaches and directions,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 11, no. 3, pp. 282–300, 2001.

[37] I. G. Vasos Vassiliou, Pavlos Antoniou and A. Pitsillides, “Require- ments for the Transmission of Streaming Video in Mobile Wireless Net- works,” University of Cyprus, Research Report, 2006.

[38] H. R. S. Zhou Wang and A. C. Bovik, “Objective Video Quality As- sessment,” The University of Texas, Research Report, 2003.

[39] (2013) Terminals and subjective and objective assessment methods. www.itu.int. [Online]. Available: http://www.itu.int/rec/T-REC-P

[40] C.-S. Chiu and C.-C. Lin, “Comparative downlink shared channel eval- uation of WCDMA release 99 and HSDPA,” in Networking, Sensing and Control, 2004 IEEE International Conference on, vol. 2, 2004, pp. 1165–1170 Vol.2.

[41] (2013) Release-4. www.3gpp.org. [Online]. Available: http://www. 3gpp.org/specifications/Releases/article/release-4

[42] H. Holma and J. Reunanen, “3GPP Release 5 HSDPA Measurements,” in Personal, Indoor and Mobile Radio Communications, 2006 IEEE 17th International Symposium on, 2006, pp. 1–5.

[43] (2013) Release-5 . www.3gpp.org. [Online]. Available: http://www. 3gpp.org/specifications/Releases/article/release-5

[44] X. Li, Y. Zaki, T. Weerawardane, A. Timm-Giel, and C. Goerg, “HSUPA backhaul bandwidth dimensioning,” in Personal, Indoor and Mobile Radio Communications, 2008. PIMRC 2008. IEEE 19th Inter- national Symposium on, 2008, pp. 1–6.

[45] H. Holma, A. Toskala, K. Ranta-aho, and J. Pirskanen, “High-Speed Packet Access Evolution in 3GPP Release 7 [Topics in Radio Communi- cations],” Communications Magazine, IEEE, vol. 45, no. 12, pp. 29–35, 2007. BIBLIOGRAPHY 57

[46] M. F. L. Abdullah and A. Z. Yonis, “Performance of LTE Release 8 and Release 10 in wireless communications,” in Cyber Security, Cyber War- fare and Digital Forensic (CyberSec), 2012 International Conference on, 2012, pp. 236–241.

[47] C. Cox, An Introduction to LTE: LTE, LTE-advanced, SAE and 4G mobile communications. West Sussex, UK: 1st ed., John Wiley and Sons Ltd, 2012, ch. Enhancements in Release 9.

[48] C. Christopher, An Introduction to LTE: LTE, LTE-advanced, SAE and 4G mobile communications. West Sussex, UK: 1st ed., John Wiley and Sons Ltd, 2012, ch. Enhancements in Release 10.

[49] MOTOROLA, “ Long Term Evolution (LTE): A Technical Overview,” MOTOROLA, White Paper, 2007.

[50] ERICSSON, “ Long Term Evolution (LTE): An Introduction ,” ERIC- SSON, White Paper, 2007.

[51] Alcatel-Lucent, “The LTE Network Architecture,” Alcatel-Lucent, White Paper, 2009.

[52] (2013) LTE. www.3gpp.org. [Online]. Available: http://www.3gpp. org/LTE

[53] H. Holm and A. Toskala, LTE for UMTS OFDMA and SC-FDMA Based Radio Access. West Sussex, UK: John Wiley and Sons Ltd, 2009.

[54] (2013) The Evolved Packet Core. www.3gpp.org. [Online]. Available: http://www.3gpp.org/The-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core. www. rcrwireless.com. [Online]. Available: http://www.rcrwireless.com/lte/

[56] M. L. David Soldani and R. Cuny, QoS and QoE Management in UMTS Cellular Systems. West Sussex, UK: John Wiley and Sons Ltd, 2006.

[57] (2013) Wowza Media Server. www. wowza.com. [Online]. Avail- able: http://www.wowza.com/uploads/images/WowzaMediaServer3 Overview (1).pdf

[58] ITU-T, “Subjective Video Quality Assessment Methods for Multimedia Applications,” ITU-T, Research Report, 1999.

[59] (2013) Videos. www.media.xiph.org. [Online]. Available: http: //media.xiph.org/video/derf/ BIBLIOGRAPHY 58

[60] (2013) Home Page. www. ffmpeg.org. [Online]. Available: http: //ffmpeg.org/

[61] (2013) Specifications. www. apple.com. [Online]. Available: http: //www.apple.com/iphone/specs.html

[62] Media Formats . www. android.com. [Online]. Available: http: //developer.android.com/guide/appendix/media-formats.html

[63] J. Shaikh, T. Minhas, P. Arlos, and M. Fiedler, “Evaluation of delay performance of traffic shapers,” in Security and Communication Net- works (IWSCN), 2010 2nd International Workshop on, 2010, pp. 1–8.

[64] T. Minhas, M. Fiedler, J. Shaikh, and P. Arlos, “Evaluation of through- put performance of traffic shapers,” in Wireless Communications and Mobile Computing Conference (IWCMC), 2011 7th International, 2011, pp. 1596–1600.

[65] O. A. G. Lagunas, “Analysis of H.264 Sensitivity to Packet Loss and De- lay Variation,” Master’s thesis, Blekinge Institute of Technology, Swe- den, 2010.

[66] M. F. L. Abdullah and A. Z. Yonis, “Performance of LTE Release 8 and Release 10 in wireless communications,” in Cyber Security, Cyber War- fare and Digital Forensic (CyberSec), 2012 International Conference on, 2012.