Joint Buffering and Rate Control for Video Streaming over Heterogeneous Wireless Networks

by

Lei Hua

A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto

Copyright c 2010 by Lei Hua Abstract

Joint Buffering and Rate Control for Video Streaming over Heterogeneous Wireless

Networks

Lei Hua

Master of Applied Science

Graduate Department of Electrical and Computer Engineering

University of Toronto

2010

The integration of heterogeneous access networks is becoming a possible feature of 4G wireless networks. It is challenging to deliver the multimedia services over such integrated networks because of the discrepancy in the bandwidth of different networks. This thesis presents an adaptive approach that combines source rate adaptation and buffering to achieve high quality VBR video streaming with less quality variation over an integrated two-tier network. Statistical information of the residence time in each network or local- ization information are utilized to anticipate the handoff occurrence. The performance of this approach is analyzed under the CBR case using a Markov reward model. Simulation under the CBR and VBR cases is conducted for different types of network models. The results are compared with a dynamic programming algorithm as well as other naive or intuitive algorithms, and proved to be promising.

ii Acknowledgements

I would like to express my sincerest gratitude to my supervisor, Professor Ben Liang,

for this exciting opportunity to work under his supervision at this prestigious institu-

tion. During the whole process he provided me with invaluable guidance, inspiration and

support, without which I couldn’t have completed this work.

I am thankful to the members of my thesis committee, Prof. Elvino S. Sousa, Prof.

Raviraj Adve, and Prof. Jason H. Anderson for the time spent in reviewing my thesis,

and for their helpful feedback and comments on improving its content.

I thank all my current and former colleagues in my research group for their useful

inputs and suggestions on the research work itself and also the presentation of the work.

Special thanks to all of my friends at University of Toronto, Colin Jiang, Eric Yuan,

Junqi Yu, Lilin Zhang, Weiwei Li, Yuan Feng, Yunfeng Lin and others, whose company,

care and encouragement made the two years of Master’s studies much more enjoyable.

Last but never the least, I dedicate this thesis to my family, who are always there for me in my life.

iii Contents

1 Introduction 1

1.1 Overview...... 1

1.1.1 VideoStreaming ...... 1

1.1.2 Heterogeneous Wireless Networks ...... 2

1.1.3 Buffering...... 3

1.1.4 RateAdaptation ...... 4

1.1.5 ContributionoftheThesis ...... 4

1.2 ThesisOutline...... 6

2 Literature Review 7

2.1 VideoRateAdaptationTechniques ...... 7

2.1.1 Transcoding...... 8

2.1.2 Joint Source/Channel Coding ...... 8

2.1.3 ScalableVideoCoding ...... 9

2.1.4 Content-Aware Coding Techniques ...... 10

2.2 Rate Control in Heterogeneous Wireless Networks ...... 11

2.3 Buffering in Heterogeneous Wireless Networks ...... 12

3 Problem Statement 14

3.1 ApplicationScenario ...... 14

3.2 ModelsandAssumptions...... 16

iv 3.2.1 Rate Adaptation and Playback ...... 16

3.2.2 Residence Time and Rate Estimation ...... 18

3.2.3 Feedback Control Mechanism ...... 19

3.3 ProblemFormulation...... 19

4 Generic Network Model 22

4.1 ControlAlgorithms ...... 22

4.1.1 Adaptive Control Algorithm ...... 22

4.1.2 SimpleAlgorithm...... 25

4.1.3 Mean Residual Life Based Algorithm ...... 26

4.1.4 SimpleShapingAlgorithm ...... 27

4.2 Analytical Framework and Analytical Results ...... 29

4.2.1 Analytical Results for Generic Model ...... 30

5 Markov Chain Network Model 36

5.1 MarkovDecisionProcessModel ...... 36

5.2 DynamicProgrammingAlgorithm...... 38

5.3 SimulationResults ...... 39

5.4 MoreRealistic3-ZoneNetworkModel...... 43

5.5 PH-FittingofResidenceTimes ...... 45

5.6 Estimation in Adaptive Control Algorithm ...... 47

5.6.1 Utilizing Statistical Information ...... 47

5.6.2 Utilizing Localization Information ...... 48

5.6.3 Simulation Results for 3-Zone Model ...... 49

5.7 Simulating with VBR Network and VBR Video Stream ...... 52

6 Conclusion 57

Bibliography 59

v List of Tables

3.1 Notationsinsystemmodel...... 20

4.1 Analysisparameters-1...... 31

4.2 Analysisparameters-2...... 31

5.1 Simulation parameters for 2-zone Markov model ...... 40

5.2 Simulation parameters for 3-zone model ...... 49

5.3 Simulation parameters for VBR network and VBR video ...... 52

vi List of Figures

3.1 Integrated two-tier network ...... 15

3.2 Relationship between the transmission sequence in time and the playback

sequenceintime ...... 18

4.1 Illustration of proportional feedback controller ...... 24

4.2 Distributionsofresidencetimes ...... 32

4.3 Analysis vs simulation results: generic model, Gamma distribution - 1 . . 34

4.4 Analysis vs simulation results: generic model, Gamma distribution - 2 . . 35

5.1 DP: variation and utilization vs. α ...... 41

5.2 Adaptive algorithm: variation and utilization vs. β ...... 41

5.3 Comparison between algorithms: utilization vs. variation ...... 42

5.4 Integrated two-tier network with 2-zone T2N ...... 44

5.5 An example of the generated user’s moving trace ...... 44

5.6 CDF’s of residence times in different zones ...... 46

5.7 PH-fittedMarkovchainnetworkmodel ...... 47

5.8 DP on 3-zone model: variation and utilization vs. α ...... 50

5.9 Adaptive algorithm on 3-zone model: variation and utilization vs. β . . . 51

5.10 Comparison DP and AA: utilization vs. variation ...... 51

5.11 VBR simulation: adaptive algorithm with statistical information . . . . . 53

5.12 VBR simulation: adaptive algorithm with localization information . . . . 53

vii 5.13 VBR simulation: simple adaptive algorithm ...... 54

5.14 Simulating VBR case - variation vs. β ...... 55

5.15 Simulating VBR case - utilization vs. β ...... 55

5.16 Simulating VBR case - utilization vs. variation ...... 56

viii Chapter 1

Introduction

1.1 Overview

1.1.1 Video Streaming

Online video has become a mainstream medium and the single most influential factor driving the need for increased mobile network capacity [8]. It would take 28 years to watch the video uploaded to YouTube in the week of April 29th, 2010 [20]; HD(high defi- nition) movies and television programs are widely available online with the help of CDNs

(Content Distribution Networks) and P2P (Peer-to-Peer) networks; video conferencing and video phones are not the exclusive rights of large companies any more, but can be enjoyed by individuals and families. It is then important and interesting to research on improving video streaming techniques.

There are two types of video streaming applications: live streaming, which captures real-time events and provides the video to users, and on-demand streaming, which offers stored video contents. Application scenarios of live streaming include video conference, video phone and live event broadcasting, which have stringent delay requirement. In this thesis we consider the transmission of pre-encoded video, which is used for delivering all kinds of published video contents and user generated contents online and is expected to

1 Chapter 1. Introduction 2

account for sixty-six percent of the world’s mobile data traffic by 2014 [21].

In comparison to other traffic flows such as Web browsing and E-mail, video streaming

has its unique characteristics and therefore may impose certain requirements on the

network. Video streaming traffic is inelastic. Unlike web browsing or file downloading, where data can be transmitted at any rate, video streaming requires certain amount of data to be delivered and decoded before the playback deadline. Hence it is sensitive to variations in both bandwidth and transmission delay. Video streaming applications are loss-tolerant. Robust coding techniques allow video to be decoded with certain loss of

data. However, this does not mean any level of loss can be tolerated. In high error-

rate networks, it is challenging to develop loss-prevention techniques for robust video

transmission.

1.1.2 Heterogeneous Wireless Networks

With the rapid growth of mobile communication technology, various wireless networking

technologies have evolved and become widely deployed all over the world, allowing people

to access the Internet with all kinds of mobile computing devices, at all times and all

places. The popular access technologies include IEEE802.11 wireless local area networks

(WLAN), WiMAX, GPRS, UMTS, and CDMA2000, etc. These technologies are hetero-

geneous in certain attributes, such as coverage area, protocol, signaling mechanism, data

rate, error rate, etc. However, it is common for the personal mobile devices (laptops,

smart phones, PDAs, digital media players) to support more than one wireless access

technologies simultaneously.

With the coexistence of heterogeneous wireless networks and the devices supporting

multiple access technologies, the integration of heterogeneous wireless networks is be-

coming a trend and is part of the 4G network design [30]. This feature allows user to

seamlessly switch among different wireless network interfaces and enjoy greatly enlarged

coverage and more reliable wireless access on a single device. Chapter 1. Introduction 3

However, there are many challenges in deploying such an inter-technology roaming environment. Active research topics on heterogeneous wireless networks involve admis- sion control, hand-off mechanism, mobility management, traffic flow assignment, etc.

The heterogeneity of wireless access technologies also imposes great challenges on video streaming applications running on a mobile device in the integrated network.

In heterogeneous wireless networks, handoffs inside one technology and between tech- nologies, can cause extra delays, which exaggerates the challenge on the delay require- ments of video streaming applications. A more substantial problem in the heterogeneous wireless networks is that, different access networks offer different ranges of bandwidth, which greatly exacerbates the variations in streamed video quality if we simply match the video source rate to the available transmission rate. Hence, in this thesis we mainly focus on reducing the variation in streamed video quality while maintain high average quality.

1.1.3 Buffering

Two types of video streaming techniques are commonly applied in both wired and wireless networks to combat the varying network bandwidth and delays: buffering and video rate adaptation.

Buffering sustains the video playback when available bit rate (ABR) is low, by prefetching and storing a certain amount of data ahead of (playback) time. With a

finite buffer size, two types of event will happen and may cause detrimental effects to the streaming process: buffer underflow and buffer overflow. Underflow may happen when the playback rate (data consumption rate) is higher than the transmission rate, which leads to playback jitters (stops). Overflow may happen when the transmission rate is higher than the playback rate, while at the same time the buffer size is small. Buffer overflow may lead to loss of data and then playback jitters.

Another factor to consider in buffering is the initial buffering delay, i.e. the waiting Chapter 1. Introduction 4

time between starting the buffering and starting the playback. There is a trade-off

between the initial buffering delay and the buffer size when we aim to provide satisfiable

video streaming service[19].

In this thesis, we consider the longer-term variations in the transmission rate in heterogeneous wireless networks, hence we assume an infinite buffer size. Also we set the initial buffering delay to be minimal. We are primarily interested in how much data to buffer for the future in every time slot and at what quality should we buffer it.

1.1.4 Rate Adaptation

Rate adaptation techniques match the video source rate to the network transmission rate, when the transmission rate is low, at the cost of lowering the perceived quality of decoded video. Various video rate adaptation techniques have been proposed over time, such as transcoding, joint source/channel coding, multiple file/rate switching, scalable video coding, and content-aware coding techniques [2]. We present some of them in

Chapter 2.

While theoretically rate adaptation can ensure continuous playback as long as the

ABR is higher than the minimum required rate of the specific adaptation technique, it introduces fluctuations in the perceived quality of the video, which can be annoying to users. This problem is exaggerated in heterogeneous wireless networks since the variation of ABR there is much higher than in homogeneous wireless networks.

1.1.5 Contribution of the Thesis

In this thesis, we consider the problem of streaming pre-encoded video on a moving mobile terminal (MT) in heterogeneous wireless networks. The video source is stored in a remote server and transmitted through the backbone network to the local access points

(AP) or base stations (BS) and then to the MT through different wireless networks. The bottleneck of the connections always lies in the last hop (i.e. the wireless hop.) There is Chapter 1. Introduction 5

a receiving buffer on the device, which is used for storing prefetched video contents.

We focus on coping with the variable ABR in heterogeneous wireless networks. The effects of other network characteristics, such as varying end-to-end delay and high error rate, are assumed to be resolved using any available technique. Our objectives are con- tinuous playback, high image quality, and low variation in the perceived image quality

(or constant-quality playback).

To achieve these objectives, we propose to combine buffering and rate adaptation

techniques with prediction of certain attributes of the network. Our scheme predicts

the residence times at each individual network, then dynamically allocates the ABR to

each unit of video sequences being transmitted, hence controlling the buffer and rate

adaptation at the same time, under the constraint of fully utilized network resources.

In order to determine the optimal way to allocate the ABR, we propose an adaptive

video rate control scheme using a linear feedback control technique on a generic network

model for a two-zone network. In designing the scheme, we divide the whole streaming

process within the heterogeneous wireless networks into cycles and try to achieve local

optimality within each cycle.

To show the applicability of our scheme to any arbitrary distribution, the performance

of our proposed scheme in a simplified constant-bit-rate (CBR) network scenario with

CBR video source is evaluated in an analytical framework based on Markov chains, where

the state space is dimensioned by the normalized quality levels and the buffered lengths

of video at the end of each cycle. We associate with each state a cost being the quality

variations within the cycle and calculate the average cost per cycle. Other naive and

intuitive algorithms are also studied within the analytical framework in order to show

the advantage of our adaptive scheme.

Then, for the special case of exponential network residence times, we formulate the

streaming process into a finite-horizon controlled Markov Decision Process (MDP), and

solve the optimization problem using a dynamic programming based optimal control Chapter 1. Introduction 6 algorithm. Although this method is assumed to provide the theoretical optimality, it involves a large amount of computation, cannot deal with increasing dimensionality, and is not applicable to more generalized residence time distributions. On the other hand, the aforementioned adaptive algorithm is much simpler than the dynamic programming algorithm in terms of the amount of computation involved, works with more generalized distributions, and requires less knowledge of the network. Through simulations we show that this scheme provides near optimal performance.

Furthermore, we develop a more realistic network model by modeling the movement of the MT and fitting the actual residence times using Phase-Type distributions. We also increase the number of network zones to three. We show through simulation that our scheme also gives near-optimal performance under the new model. Furthermore, we simulated our scheme with variable-bit-rate (VBR) networks and VBR video sources, and explored the effects of utilizing different estimations of residence times, i.e. the statistical information extracted from history mobility traces, and the geographical information provided by localization service. Our algorithm proves to provide significantly improved performance with VBR networks and VBR video source than the naive algorithms.

1.2 Thesis Outline

The thesis is organized as follows. The next chapter reviews work in the related areas of video rate control and buffering techniques in both homogeneous and heterogeneous wireless networks. The system setup and the problem statement are presented in Chapter

3. Chapter 4 is focused on a generic network model and the design of our adaptive control algorithm. We also present the analytical performance evaluation framework. In

Chapter 5, we introduce the Markov Chain network model and the dynamic programming algorithm. We also extend the system models to a 3-zone model with PH-fitted residence times and simulate the VBR case in Chapter 5. Finally, Chapter 6 concludes the thesis. Chapter 2

Literature Review

This chapter briefly reviews the existing research progress on video streaming technologies on both homogeneous and heterogeneous wireless networks and the current challenges, which motivated our research work. We first discuss some research works on video rate adaptation in general variable-bit-rate (VBR) networks. Then we present some related research on buffering and rate control techniques in video streaming over heterogeneous wireless networks.

2.1 Video Rate Adaptation Techniques

Various video rate control or adaptation techniques have been proposed to combat short- term variations in homogeneous VBR wireless channels when performing video streaming.

Specifically for streaming pre-encoded video, research focus has been put on transcod- ing [22, 3, 11], joint source/channel coding [12, 14, 5], scalable video coding [9], and other techniques such as content-aware or motion-aware coding [18, 7, 25, 27]. While in commercial systems, the multiple file/rate switching techniques are widely implemented

[2].

These rate adaptation proposals work well in homogeneous wireless network where average ABR doesn’t vary over time. However, they could not provide satisfactory per-

7 Chapter 2. Literature Review 8 formance in terms of quality variation in heterogeneous wireless networks, since purely adapting the source bit rate to the channel bit rate will lead to a large variation of video quality over different sub-networks.

2.1.1 Transcoding

Transcoding is a technique to adapt the video source rate through recompression. [22,

3, 11] are three examples of research studies on video rate adaptation with transcoding.

These techniques dynamically choose the quantizer used in encoding each frame or block, and try to minimize the total distortion while matching the video source rate with the network rate. Their focus is mainly on analyzing the specific encoding technique and extracting the rate-distortion models. The heavy computation involved in transcoding is its main disadvantage.

2.1.2 Joint Source/Channel Coding

The authors of [12] show that the perceptual source distortion decreases exponentially with the increasing MPEG-2 source rate, and the perceptual distortion due to data loss is directly proportional to the number of lost macro blocks. Hence they propose to use Joint Source/Channel Coding (JSCC) technique, specifically adding FEC (forward error correction) bits, to protect the data from loss. The optimal channel coding FEC parameters can be selected according to the aforementioned relationships and the total rate of received video stream can be controlled to minimize the total distortion.

Similarly, [14] and [5] both consider the Joint Source/Channel Coding problem with

FEC channel coding and focus on how to choose the channel coding parameters. In [14], the authors translate the Quality of Service (QoS) requirements of the video streaming applications into a threshold of occupancy of playback buffer. By adapting the JSCC parameters their scheme tries to maintain a certain level of buffer occupancy to sustain continuous playback. Chapter 2. Literature Review 9

In [5], a probabilistic QoS requirement, i.e. the buffer starvation probability has been proposed. The authors use cycle-based rate control with cycles being successively alternating between good(non-fading) and bad(fading) period, while guaranteeing an upper bound on the probability of starvation at the playback buffer. The cycle-based idea inspired us to divide the streaming process in heterogeneous networks into cycles, but our “cycle” have a completely different definition from theirs in that our cycle contains intervals of the MT residing in different sub-networks.

While the Joint Source /Channel Coding technique can adapt the source rate within

certain range, it usually involves cross-layer design with information flows across PHY

/MAC /Network layers, which might be applicable in homogeneous networks but could

become extremely complex in terms of implementation and computation in heterogeneous

networks.

2.1.3 Scalable Video Coding

The layered or scalable video coding techniques are said to be suitable for adapting

to longer-term bandwidth fluctuations [2]. There has been a substantial body of re-

search works developing efficient scalable compression techniques. A scalable extension

of H.264/AVC [31], Scalable Video Coding (SVC) [24] has been standardized, which pro-

vides scalability of temporal, spatial, quality resolution, or a combination of scalability on

these three dimensions, of a decoded video signal through adaptation of the bit stream.

Fine Granularity Scalability (FGS) coding [17] and Fine Granularity Scalability Tempo-

ral (FGST) coding [29] have also been adopted as amendments to the MPEG-4 standard.

Multiple Description Coding (MDC) [13] is another type of scalable video coding where

each description (substream) of the video stream is of equal weight and independent of

each other in contrast to the Base Layer/Enhancement Layer structure of SVC.

The authors of [10] developed a heuristic rate control algorithm for 2-layer FGS coded

video over TCP-friendly “connection”, which can achieve the same level of smoothness Chapter 2. Literature Review 10 over both TCP and TCP-friendly protocols. Their algorithm works with CBR coded video and the loss model is simple. The authors of [16] proposed a stochastic dynamic programming algorithm for VBR scalable coded video with a more realistic loss model.

The authors of [9] studied the problem of minimizing the average distortion of FGS

video under a limited transmission rate. The authors provided a framework which jointly

considers the effects of packet scheduling at the sender and the error concealment at the

receiver.

The authors of [33] explicitly considered the effect of fading in wireless channel and

develops cross-layer rate adaptation algorithm for layered video in fading channels. The

complexity in cross-layer design makes it difficult to implement even in homogeneous

wireless network.

The authors of [23] introduced a novel streaming strategy to improve the probability

of successfully stream a scalable coded video sequence by adaptively selecting the number

of layers according to mobility information in Ad-Hoc wireless networks. This is relevant

to our work in that our proposed algorithm can also utilize the mobility and location

information to predict the MT’s movement and channel status, as described in Chapter

5.

In our control scheme, we can use either the transcoding or scalable coding techniques

to perform rate adaptation. However, we assume a generic rate-quality relationship in

our model and that one cannot change the quality of a video sequence that is already

transmitted.

2.1.4 Content-Aware Coding Techniques

There exists many other video rate adaptation algorithms which try to achieve different

QoS objectives. Content-aware encoding/playout has been an interesting and contro-

versial topic for video rate adaptation, as there exists no generally accepted standard

for perceived quality of motion pictures when we consider the presentation of the actual Chapter 2. Literature Review 11 content, instead of quantifiable metrics such as resolution, frame rate and PSNR (Peak

Signal-to-Noise Ratio). Representative works of content-aware rate adaptation/playout control include [18, 7, 25, 27], etc. These works try to analyze the amount of motion or interested objects in each frame, and allocate the available bit rates unfairly among different objects / frames to achieve best perceived quality when the network rate is not high enough to present the full pictures.

2.2 Rate Control in Heterogeneous Wireless Net-

works

Video streaming in heterogeneous wireless networks has been a relatively new topic. Most of the available works address the issues in architecture design and hand-off handling.

The authors of [26] analyzed the effects of handoffs on rate control and proposed a cross- layer solution to anticipate the handoff occurrence and to adjust the data rate. They use transport-layer dummy packets to probe the channel in their solution, while in this thesis we propose to utilize the statistical information of the residence times and ABR in each sub-network.

Some researchers consider video streaming in a multiple stream environment with

heterogeneous access technologies and focus on fairness or priority among all users/flows.

In [38] the authors study the rate allocation problem in streaming over wireless networks

with heterogeneous link speeds. The focus of their work is on how to allocate the rate

between multiple video streaming sessions on heterogeneous links to maximize the aver-

age quality among all users, while the quality enjoyed by a single user is not explicitly

considered.

The authors of [1] addressed the problem of flow rate control for different types of

traffic flows and heterogeneous wireless links, and employs an H-infinity optimal rate

controller to achieve efficient utilization of all channels while taking the requirements of Chapter 2. Literature Review 12 different flow types in to account. Both of them assume that all the access networks in the integrated network are available all the time, while our assumption is that the user is moving and the trajectory is not always covered by both sub-networks.

The authors of [35, 36, 37] considered the video streaming problem in an integrated

3G/WLAN network from a monetary cost point of view. Their system setting of the heterogeneous networks is the most similar one to ours, yet the objective and control actions are completely different. Different streaming strategies are proposed to decide how much data to be streamed (i.e. the transmission rate) in each individual network as well as when to hand off to the other network, so that the monetary cost of streaming the data is minimized. While in our system model we also consider the cost effects of each network, we assume that it is always good to fully utilize the ABR and that the MT will switch to the higher-rate, lower-cost network whenever it is available, as this strategy is simple and already implemented in commercial systems such as the iPhone.

It is worth mentioning that, some of the techniques mentioned above, such as [33] and

[5] have similar objectives as ours, i.e. minimizing the variations in adapted video rate caused by variations in network transmission rate. However, in homogeneous wireless networks, the variation caused by fading and other short-term effects are quite different from the variation caused by handoff between different access technologies in both time scale and magnitude. Hence these techniques cannot be directly applied to our problem.

Furthermore, because the time scale of variation in our problem is longer, we may have more ways to predict the variation, such as utilizing geographical information.

2.3 Buffering in Heterogeneous Wireless Networks

For streaming pre-encoded video, buffering is another technique to overcome the mis- match between video source rate and channel bit rate. By caching enough data in the client buffer ahead of time, continuous high-rate playback can be sustained when the Chapter 2. Literature Review 13 channel throughput is low. Buffering schemes for streaming VBR video over heteroge- neous wireless networks are studied in [15]. These schemes include fixed/jointly optimized schemes based on buffering delay, buffered playout data, and playout time. Analysis on both the jitter frequency and the buffering delay are conducted for these schemes.

However, without rate adaptation buffer underflow happens frequently when the av- erage channel throughput is lower than the average video source rate, leading to playback jitters. Hence we propose to combine rate adaptation with buffering for long-term varia- tions to smooth out the streaming process in heterogeneous networks. We choose only the playout time based buffering scheme, as when we introduce rate adaptation the buffering delay become meaningless, and the buffered playout data become highly variable with the changing video rate.

To the best of our knowledge, this thesis is the first work to propose the combination of rate adaptation and buffering to address the problem of smooth video playback in heterogeneous networks. Nevertheless, our work is inspired by and based on the related works listed here in that the scheme shall utilize a generic rate adaptation techniques mentioned above to perform the control actions. Chapter 3

Problem Statement

In this chapter, we explain in details the problem we study in this thesis. We first present the application scenario, then introduce our way modeling of the system based on this scenario with some practical assumptions. A mathematical formulation of our problem is then presented based on these common assumptions.

3.1 Application Scenario

We study video streaming over the heterogeneous wireless networks with the overlapping of two networks (or two zones), as shown in Figure 3.1. The Tier-1 Network (T1N) is assumed to provide universal coverage with low bit rate, while Tier-2 Network (T2N) covers limited areas around the Access Points (APs), with high bit rate. In reality, T1N is usually more costly than T2N. A proper example of T1N can be the 3G cellular network, while T2N can be Wireless LAN. A user prefers to access the Internet through T2N due to its high bandwidth and low cost, thus whenever he/she enters a T2N covered area, the mobile device switches to T2N for transmission. While our scheme is independent of lower layer (e.g. PHY/MAC layer) implementations and can actually handle the simultaneous transmission over both sub-networks, we maintain the assumption of using only one sub-network at the same time since it is more practical to do so in reality.

14 Chapter 3. Problem Statement 15

T1N T2N

AP

BS T2N

Figure 3.1: Integrated two-tier network

Two types of handoffs take place in this network: intra-technology handoff or Hori- zontal Handoff (HHO) in which the mobile terminal (MT) switches between two Access

Points (AP) or Base Stations (BS) using the same access technology, e.g. from one

WLAN AP to another, and inter-technology handoff or Vertical Handoff (VHO), which occurs when the MT roams between different access technologies, e.g. switching from 3G to WLAN when entering a WLAN covered area. VHO affects different system perfor- mance metrics, such as the signaling load, resource utilization and user perceived QoS. In particular, the available bit rate (ABR) in our model may vary by one order of magnitude after any VHO. Both types of handoffs may cause extra delays in the transmission, but we do not consider the extra delays here and assume that there is a seamless handoff handling scheme which can eliminate the delays caused by both types of handoffs (which may be achieved at the cost of ABR). In practice, a handoff handling scheme such as the one presented in [6] can be employed to satisfy this assumption.

A video streaming session is running on the device while the MT traverses through this integrated two-tier network. The streaming server lies outside this wireless network, but the bottleneck of the connection is the last hop - the wireless link between the

MT and the BS/AP. The MT keeps sending control messages to request the server to adjust the video source rate and transmission rate. The server then makes adjustments accordingly and transmits the data to the MT. The MT has a buffer, which stores the Chapter 3. Problem Statement 16 received data before they are used for playback. Our goal is then to develop a control scheme to determine how to choose the video source rate and how much to buffer ahead of time given some statistical and observed channel information. The streaming session is assumed to be very long and we analyze the performance of our scheme on a time-average basis.

3.2 Models and Assumptions

3.2.1 Rate Adaptation and Playback

We model the variation in the wireless channel, the error control scheme, and the handoff handling effects all into the random ABR of the network, denoted by R(i), which is always positive. In reality, ABR is the amount of error-free video data received by the MT at each time slot.

We divide the whole streaming session, which consists of several “in-T1N” and “in-

T2N” intervals, into cycles, and denote each cycle by its sequence number in the whole streaming session, j, where j = 1 represents the first cycle in this session. Each cycle

starts at a T1N-to-T2N VHO and contains one “in-T2N” interval followed by one “in-

T1N” interval, the lengths of which are denoted as T2 and T1 respectively. (Note that, here we assume the streaming session always starts in T2N, i.e. the high rate network. This is

reasonable since there is not much we can optimize before the first VHO if it starts within

T1N, and we are considering the average performance in the whole streaming process,

so edge effect at the beginning can be ignored.) We further model the video streaming

process as time-discrete with time slots of equal length, and denote each time slot by its

sequence number in the current cycle, i, where i = 1 represents the first time slot in

the current cycle. The control actions are decided and performed at the beginning of

each time slot.

We assume that we have original video streams with very high quality. The average Chapter 3. Problem Statement 17 rate of video can be higher than the highest network rate, and we can adjust the source encoding rate to any level at any granularity up to the original rate. This assumption of rate adaptation at any granularity can be accommodated by quantization in practice. By making this assumption, we eliminate the possibility of playback stop, since transmission rate R(i) is assumed to be always positive.

The original source rate of the whole video sequence as well as the rate-quality rela- tionship of the video are transmitted to the MT before the streaming starts, thus the MT can utilize this information to adjust the rate of “future” parts of the video which are being streamed. Here we do not define the specific technique used to adapt the source rate. The transcoding or SVC technique mentioned in Chapter 2 may be employed in reality to perform rate adaptation.

It is worth noting that, although control actions are performed at the beginning of each time slot, it is not necessarily effective for only one time slot in playback time.

This is because control decisions are made at every time slot of transmission time, in which period more than one time slot of video data in the playback sequence may be transmitted. We illustrate the relationship between transmission time and playback time in Figure 3.2.

We use a “quality level” defined as q(k) = f(r(k), r0(k)) to control the source rate adaptation at time slot k, where f(x, y) is monotonously increasing with x, r0(k) is the original video rate at time slot k, and r(k) is the adapted video rate at time slot k. Hence for each time slot, the perceived image quality increases with q(k). While there are several methods to characterize the quality of the picture, e.g. PSNR, distortion, etc, we do not choose a specific metric, but assume a generic form of the rate-quality relationship. For simplicity, we assume the original video stream is constant-quality encoded (whether it is CBR or VBR), thus the perceived quality only depends on the adjusted bit rate and the original bit rate, and q(k)= f(r(k), r0(k)) = f(r(k)).

0

2 4 1 … 3

5 m Transmission Rate 2 m T T m m m + … 2 2 + + m + + + 4 + 2 1 1 2 2 T1 T1

TT2+ 1 Transmission Time T2 Original Rate

Playback Rate … … m m 1 2 3 4 5 + T2 1 +

T1

Playback Time

Figure 3.2: Relationship between the transmission sequence in time and the playback

sequence in time

performance of our algorithm, where T can be a fixed length of time or the length of one cycle, and g() is a monotonously increasing function. This metric reflects the average amount of total quality variation of the rate-adapted video stream over time.

3.2.2 Residence Time and Rate Estimation

We assume that statistical information such as mean and variance of the residence time within each sub-network and also the average transmission rates R2 and R1 throughout the two intervals can be obtained form the AP. This information can be extracted from previous data collected by the AP. The MT requests the information at the beginning of each cycle (i = 0). Another possibility is to utilize the geographical information and

MT’s moving speed/direction to estimate the residence time in each network, which will Chapter 3. Problem Statement 19 be briefly discussed in Chapter 5.

3.2.3 Feedback Control Mechanism

We assume that the ABR of current time slot, R(i), as well as the current connected network can be estimated with good accuracy and fed back by the MT to the server at the beginning of this time slot. The server then transmits data at the estimated rate.

This mechanism guarantees that our algorithm runs under the following constraint.

Constraint: the ABR of the network is 100% utilized.

However, this doesn’t mean that the transmitted data are 100% utilized. We introduce another metric called “data utilization”, calculated as the total data used for playback up to T divided by the total data transmitted up to T . The difference between these

two values is the amount of data left in the buffer at time slot T . From the design of our

algorithms we will see it is not possible to achieve 100% data utilization, but we still try

to maintain the data utilization above an acceptable level within our observation horizon.

The notations for the system models in this thesis are summarized in Table 3.1.

3.3 Problem Formulation

We can translate the aforementioned objectives, assumptions and constraints into the

following optimization problem: Chapter 3. Problem Statement 20

Notation Description

i Index of time slots (in transmission time).

R(i) ABR in ith slot of current cycle.

k Index of time slots (in playback time).

r0(k) Video source rate (to be played) in kth slot. r(k) Adapted video rate (to be played) in kth slot.

Residence time in T1N (random variable), its mean and T1, T1, T1 the estimation value of it used in the rate control scheme. Residence time in T2N (random variable), its mean and T2, T2, T2 the estimation value of it used in the rate control scheme.

R1, R2 Average ABR in T1N and T2N.

R1, R2 ABR in T1N and T2N in CBR network. q(k) Quality of adapted video (to be played) in kth time slot of current cycle.

j Index of cycles.

T1(j) ,T2(j) Residence time in T1N and T2N in jth cycle.

q0(j) Quality of adapted video in the last time slot of the (j − 1)th cycle.

Table 3.1: Notations in system model Chapter 3. Problem Statement 21

g(|q(k + 1) − q(k)|) min V=E( 1<k

where q(k) = f(r(k), r0(k))

s.t. 0 < r(k) < r0(k) (3.1)

r(k) ≤ R(i), 1 < t < T 1<k

Given the randomness of the R(i), this optimization problem is not directly solvable.

In the generic model in Chapter 4, T = T1 + T2 in the above formulation. In the

Markov Chain based channel model in Chapter 5 , we assume T1 and T2 are exponentially distributed, and T is set to be a fixed length of time instead of the length of each cycle. Chapter 4

Generic Network Model

Three algorithms are developed for the generic network model with one simple algorithm as comparison. All of them can work with any type of residence time distributions.

4.1 Control Algorithms

4.1.1 Adaptive Control Algorithm

We first propose an adaptive control algorithm which can use statistical information or localization information to estimate the residence times, and adaptively adjusts the quality level based on the estimation at each time slot. The estimation horizon is within the current cycle. Let Te(i) denote the estimation of residence-time-to-go at time slot i in the current cycle. If the MT is in T2N at time slot i, Te(i) includes the remain- ing residence time in T2N and the estimated length of the following “in-T1N” interval:

Te(i) = [T2(i) T1(i)]. Otherwise, T2(i) = 0 and the estimation gives us the remaining residence time in T1N, i.e. Te(i) = [0 T1(i)]. In one approach, we can use the average residence times, T2 and T1, as the estimations of residence times. At every time slot i in cycle j, the estimation Te(i) is calculated as follows:

22 Chapter 4. Generic Network Model 23

If the MT is in T2N at time slot i,

T2(i) = max{T2 − i, 1} (4.1) T1(i)= T1 (4.2) If the MT is in T1N at time slot i,

T1(i) = max{T1 − (i − T2(j)), 1} (4.3) T2(i) = 0 (4.4) where T2(j) is the actual residence time the MT spent in T2N in the current cycle. We can also use other forms of estimations in our algorithms. In Chapter 5, we describe a case where our algorithm can work with estimations generated by localization service.

Our adaptive control algorithm is as follows: At the beginning of each cycle (time slot 0), we have the original playback curve (i.e. the cumulated video source rate curve) starting from the current time slot, denoted as P (t). We may already have some data buffered from the previous cycle, denoted as TB(0) seconds or B(0) bits in the buffer. Also, we want to buffer some data for the next cycle, i.e. at the end of this cycle, we should have TBh seconds of video in the buffer. TBh is set to be 0 in the CBR cases but to a very small value in the later VBR cases to accommodate short-term variations in

VBR networks.

To simplify the calculation here, we choose a simple form of “quality level”, q(i) = f(r(i), r0(i)) = r(i)/r0(i), i.e. quality level is proportional to the adjusted video rate given a fixed original rate. But for any other f(x, y) monotonously increasing with x, similar calculations can be applied here for all the algorithms mentioned below.

Then we calculate the initial quality level q(0), which is, the average quality level we can sustain throughout the current cycle under the assumption of perfect estimation:

R T (0) + R T (0) q(0) = 2 2 1 1 (4.5) P (T2(0) +T1(0) + TBh − TB(0)) Chapter 4. Generic Network Model 24

Desired State Error Signal Control Signal β q - Estimation Feedback Signal System

Figure 4.1: Illustration of proportional feedback controller

At each subsequent time slot i, the MT keeps estimating Te(i). Also, we have the amount of buffered data at this time is B(i) bits. Then if we keep using the quality level decided at the previous time slot, q(i − 1), the estimated amount of buffered data (in bits) at the end of this cycle would be:

Be(i)= B(i)+ R2T2(i)+ R1T1(i) − q(i − 1)[P (i + T2(i)+ T1(i)) − P (i)] (4.6)

The estimated length of buffered video is TBe(i), which is the solution of the following equation:

q(i − 1)[P (i + T2(i)+ T1(i)+ TBe(i)) − P (i + T2(i)+ T1(i))] = Be(i) (4.7) Then, we decide q(i) as follows:

q(i) − q(i − 1) = β (TBe(i) − TBh) (4.8)

Here, we actually construct a linear negative-feedback controller with a proportional gain

β (See Figure 4.1 for an illustration). The input of our system is q(i − 1), the output

is TBe(i), and our setpoint is TBh. The controller attempts to minimize the “error” between the given setpoint and the output by adjusting the control inputs according to

the proportional negative feedback law. If the q(i) we get from the above controller is

higher than 1, which means that the new rate will be higher than the original rate (which

is impossible under our system assumptions), we set q(i) = 1. Also, if the new rate cannot

be sustained by the current network transmission rate, i.e. there is not enough buffer

(TB(i) < 1) and the transmission rate is low, we set q(i) to be the highest level that can Chapter 4. Generic Network Model 25 be supported for the current time slot:

R q(i)= 1 P (1 − TB(i))

Hence,

max{1,q(i − 1) + β (TBe(i) − TBh)}, if TB(i) ≥ 1 q(i)=  (4.9)  min{ R1 ,q(i − 1) + β (T (i) − T )}, if T (i) < 1 P (1−TB (i)) Be Bh B   Other forms of feedback controllers, such as PID (Proportional-Integral-Derivative)

controller [32], can also be applied here, but since our system is non-linear and stochastic

in general, it is difficult to provide theoretical guidance on choosing the parameters.

Hence we keep the controller simple with only one adjustable parameter, β. Even for the only parameter, we cannot provide a method to choose the proper β which guarantees

optimality and system stability (which is the case for most PID controllers). Generally,

there might be some empirical rules for choosing parameters in PID control systems,

but the effectiveness of these rules depends highly on the system structure. However,

we discuss the choice of the parameter for the specific set of system parameters in the

analysis of our simulation results in Chapter 5 and provide some intuitions on tuning the

parameters.

During each time slot, the server sets the quality levels of video sequences to be

transmitted according to the adaptive algorithm and transmits them to the MT.

4.1.2 Simple Algorithm

To compare our algorithm with other bandwidth smoothing techniques without long-term

prediction or estimation for the VHOs, we design another simple adaptive algorithm. In

this algorithm, we don not use any information about the network distribution but decide

q(i) only based on the current buffered length, i.e. at every time slot i we decide q(i) as Chapter 4. Generic Network Model 26 follows:

max{1,q(i − 1) + β (TB(i) − TBh)}, if TB(i) ≥ 1 q(i)=  (4.10)  min{ R1 ,q(i − 1) + β (T (i) − T )}, if T (i) < 1 P (1−TB (i)) B Bh B   Where TB(i) is the buffered length of video in seconds at time slot i, which is the solution for the following equation:

q(i − 1)[P (i + TB(i)) − P (i)] = B(i) (4.11)

We are employing the same negative feedback control technique here, except that our

system does not include the estimation part any more. Instead, the algorithm uses the

observation of the buffer level as the input.

4.1.3 Mean Residual Life Based Algorithm

Assume we know the exact distribution of the residence times. One intuitive algorithm

we can come up with is to calculate the expected remaining residence time (or Mean

Residual Life, MRL) at every time slot given the elapsed residence time the MT spent in

current sub-network, then allocate the expected network resources evenly according to

the MRL.

To be more specific, assume we have the Probability Density Functions (PDFs) for

T1, T2 to be f1(t), f2(t) respectively. At each time slot i in T2N, the algorithm calculates the MRL in T2N as follows:

MRLˆ 2(i)= E(T2 − i|T2 >i) (4.12) 1 ∞ =(− tf (t)dt) − i ∞ 2 i f2(t)dt i ˆ MRL2(i)= max{MRL2(i), 1} (4.13)

Then, it calculates the quality level to be used according to the MRL (also assuming the

proportional rate-quality relationship here): R MRL (i)+ R T q(i)= max{1, 2 2 1 1 } (4.14) P (MRL2(i)+ T1 − TB(i)) Chapter 4. Generic Network Model 27

where TB(i) is the length of video the MT has buffered at time slot i.

If at time slot i the MT is in T1N, the calculation for the MRL is similar, but the q(i) is calculated as follows:

max{1, R1MRL1(i) }, if T (i) ≥ 1  P (MRL1(i)−TB (i)) B q(i)=  (4.15)  min{ R1 , R1MRL1(i) }, if T (i) < 1 P (1−TB(i)) P (MRL1(i)−TB (i)) B   For a relatively stable network distribution, the MRLs given different elapsed residence times can be pre-calculated and stored within a table, hence this algorithm does not involve large amount of calculation. However, since the transfer function from the resi- dence times to the time-averaged variation is not linear, the MRL is not necessarily the optimal estimation for the remaining residence time if we want to achieve the minimum expected variation.

Furthermore, the performance of the MRL based algorithm depends heavily on the distributions of residence times. If the residence times are exponentially distributed, then the MRL is always the mean of the exponential distribution regardless of how long the MT has stayed in the current sub-network. Thus the MRL based algorithm will keep increasing the quality level from the beginning of the cycle, until it reaches the end of T2N, and keep decreasing the quality level from the beginning of the following

T1N interval until the end of the cycle. Such curve introduces quality variations in itself instead of eliminating them. With certain other light-tail distributions, the MRL based algorithm gives better performance.

4.1.4 Simple Shaping Algorithm

Another intuitive solution is to explicitly consider the form of our optimization objective: the time averaged variation. The algorithm then designs a “best shaped” quality level curve according to the specific form of this metric. Again we consider the quadratic form 2 0

At every time slot, we have an estimation of how much data to receive from current time slot to the end of current cycle. We also have the record of the quality level set by the algorithm in the previous time slot. Now assume the estimation is accurate, the best way to allocate the deliverable data to each time slot of video is to perform a quadratic programming over the remaining time of the cycle within the constraint of using up all the deliverable data at the end of the cycle, and the objective of minimum variation

(which is in a quadratic form). We use the expected values of residence times, T1 and T2 as the estimations.

However, if the estimation is inaccurate, which is almost always the case, the output of quadratic programming is never the optimal result. The problem is not substantial when the estimation of T2 is inaccurate, as at the end of the T 2N period, we always end up with plenty of buffer to compensate for the low-rate in T 1N, which can be used to guarantee a smooth degrading curve afterwards if the MT enters T 1N earlier than

expected. The only case when there is a problem is when T1 > T1. Because when this is the case, after the (T2 + T1)th time slot, there is nothing in the buffer, and the quality

level of time slot (T2 + T1 + 1) will be forced to set to R1/r0, which will usually cause a huge difference between q(T2 + T1) and q(T2 + T1 + 1). In order to avoid this, we add a further constraint to the quadratic programming, that at time slot (T2 + T1 + 1), the

quality level is set to be R1/r0.

So the quadratic programming part in this algorithm becomes:

2 2 2 min (q(i + 1) − q(i)) +(q(T2) − q(T2 + 1)) +(R1/r − q(T2 + T1)) T2+2≤i≤T2+T1

s.t. q(i)= TB1 R1/r + T1 R1/r (4.16) T2+1≤i≤T2+T1

R1/r ≤ q(i) ≤ 1, T2 + 1 ≤ i ≤ T2 + T1

When the MT is in the following T 1N network, the algorithm keeps using the “best shaped” quality curve determined by the quadratic programming until it reaches the end Chapter 4. Generic Network Model 29

of this cycle. If T1 > T1, the quality levels for each time slot between [T1, T1] will be set

to R1/r.

4.2 Analytical Framework and Analytical Results

In order to evaluate the performance of these algorithms, we further introduce an ana-

lytical framework based on Markov chains, which can work with any kind of residence

time distribution.

Following the generic network model, we model the streaming process as a discrete-

time Markov chain, with states {TB,q0}, where TB is the playback length of buffered video

(in seconds) at the beginning of each cycle, and q0 is the quality level set at the last time

slot in the previous cycle. We assume T2(j) and T1(j) are i.i.d. random variables (note that this is an assumption in the analytical framework, we do not make this assumption

when we design the rate control scheme), where j is the index of cycles. Then the values

of {TB(j),q0(j)} depend only on the previous state, {TB(j − 1),q0(j − 1)}. To make the

state space finite, we adopt a uniform quantization scheme on the range of TB and q0 values.

We can see that, since we are always trying to achieve an expected amount of buffered

data of 0 at the end of each cycle, (i.e. high data utilization) the playback length of

buffered video should never exceed the estimated value of T1, T1 (since in T2N period, the algorithm buffers for no longer than T1(R2 − R1)/R2 to achieve a zero buffer at the end of the cycle, while in the T1N period, the algorithm uses up the previously buffered

data). Hence this Markov chain is a finite-state Markov chain, with state TB being

{0, 1, 2, ...T1(R2 − R1)/R2}, and q0 being {ql,ql + qs,ql + 2qs, ..., qh}. Where ql is the lowest quality level allowed by the algorithm, and qh is the highest quality level allowed by the algorithm.

Between any two states, there is a transition to each other, making this Markov chain a Chapter 4. Generic Network Model 30 fully connected graph. To analyze the overall performance of our algorithm, we associate

g(|q(i+1)−q(i)|)+g(|q0−q(1)|) a cost J (T ,q ) = E( 1

P (TB(j +1) = TBm,q0(j +1) = q0n|TB(j)= TBl,q0(j)= q0k)

= P (TBm,q0n|TBl,q0k)

= P (TBm,q0n|TBl,q0k, T2(j)= T2, T1(j)= T1) P (T2(j)= T2) P (T1(j)= T1) T2,T1 (4.17)

Now we can compose the transition matrix PA for our defined Markov chain. Solving

this Markov chain, we can obtain the steady state distribution, π, which satisfies π = πPA. Then we can calculate the expected cost per cycle:

E(Je)= πTB ,q0 Je(TB,q0) (4.18) TB ,q0 Note that, in this analytical framework we do not consider the data utilization metric

(while in later studies of the Markov chain network model and dynamic programming algorithms, we explicitly consider it). This is because, in this model we are observing a relatively short time horizon (one cycle), the utilization in this time horizon cannot reflect the long-term utilization. Instead, we set the objective of using up all the transmitted data at the end of each cycle in the design of our algorithms, which ensures that the long-term data utilization is near 100%.

4.2.1 Analytical Results for Generic Model

Under the established framework, we analyze the performance of our adaptive control al- gorithm, the MRL based algorithm and the simple shaping algorithm with the quadratic Chapter 4. Generic Network Model 31

Parameter Value Parameter Value

Time slot length 1 s Time slot length 1 s

T1, T2 20 s, 20 s T1, T2 20 s, 20 s

R1 0.5:0.25:2.75 Mbps R1 0.5:0.25:2.75 Mbps

R2 6:-0.25:3.75 Mbps R2 6:-0.25:3.75 Mbps

r0 6 Mbps r0 6 Mbps

a1, b1 6.67, 3 a1, b1 13.33, 1.5

a2, b2 6.67, 3 a2, b2 13.33, 1.5

Table 4.1: Analysis parameters - 1 Table 4.2: Analysis parameters - 2

variation metric. The simple adaptive algorithm is also evaluated as a comparison. To

avoid the cases when MRL based algorithm doesn’t perform intuitively (like the expo-

nential distribution case), we choose light-tail distributions of residence times to analyze

the performance. An example of the analytical results with Gamma-distributed residence

times is shown in Figure 4.3 and 4.4 along with the residence time distributions (the same

for T1 and T2) in Figure 4.2. We also simulate the algorithms in the same settings to val- idate the analytical framework. The parameter β in the adaptive and simple algorithms are selected based on simulations first to ensure the long-term utilization is above 90%.

Parameters are listed in Tables 4.1 and 4.2 (ai and bi are the two parameters in the Gamma distribution). Proper values of these parameters (especially the average

residence times) depend on the density and coverage of T2Ns, the MT’s moving speed

and moving pattern. In Section 5.6 when we discuss the network model generated by

mobility modeling, we use practical values for moving speed and T2N coverage, and a

realistic distribution of the T2Ns. The mean residence times we get in that setting are

T1 = 35sandT2 = 25s. So we think the mean residence time of 20s here is also practical in real scenarios.

As we can see, the adaptive control algorithm gives the best performance all the time, Chapter 4. Generic Network Model 32

0.06 0.08

0.07 0.05

0.06

0.04 0.05 ) ) 1 1 0.03 0.04 Pdf(T Pdf(T

0.03 0.02

0.02

0.01 0.01

0 0 0 10 20 30 40 50 0 10 20 30 40 T 1 T 1 Gamma distribution of residence time -1 Gamma distribution of residence time -2

Figure 4.2: Distributions of residence times

even though we do not minimize the variation metric in the adaptive algorithm explic-

itly. In contrast, the MRL based algorithm tries to optimize through selecting the best

estimation, yet it fails because of the non-linear relationship between the residence times

and the variation. The simple shaping algorithm is not optimal either, because although

it tries to minimize the variation by shaping the quality curve under the assumption of

perfect estimation, it fails to deal with the uncertainty of the residence times.

Furthermore, the adaptive algorithm has more advantage if we compare the complex- ity of the algorithms and the ability to deal with VBR cases. While the adaptive control algorithm is designed for VBR cases and performs well in VBR cases as we show later, the MRL based algorithm and the simple shaping algorithm are designed based on CBR assumptions and would simply fail to deal with more uncertainty in the VBR cases.

We can also see that the simulation results and the analytical results are reason- ably close to each other and exhibits the same trends, which validates our analytical framework. However, there is a consistent bias in the analysis shown in both figures in comparison to the simulation. Through experiments we found that by using finer quan- Chapter 4. Generic Network Model 33

tization for TB and q0 we can reduce the gap between analytical results and simulation results. Hence we believe the bias is caused by quantization errors in the analysis.

On the other hand, comparing between the results with differently shaped distribu- tions (the first one more dispersed and the second one more centralized), we can also notice some characteristics about the MRL algorithm: the MRL based algorithm works better (nearer to the adaptive one) when the distribution of the algorithm is more cen- tralized, because when the distribution is more centralized, the MRL will be nearer to the real remaining residence time. The simple shaping algorithm has its own character- istics as well. However, as they show poor performance in comparison to our adaptive algorithm, we do not look further into the details of these algorithms. Chapter 4. Generic Network Model 34

0.014 Adaptive(A) MRL(A) Shaping(A) Simple(A) 0.012 Adaptive(S) MRL(S) Shaping(S) Simple(S) 0.01

0.008 V

0.006

0.004

0.002

0 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 R 1

Figure 4.3: Analysis vs simulation results: generic model, Gamma distribution - 1 Chapter 4. Generic Network Model 35

0.012 Adaptive(A) MRL(A) Shaping(A) Simple(A) Adaptive(S) 0.01 MRL(S) Shaping(S) Simple(S)

0.008 V 0.006

0.004

0.002

0 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 R 1

Figure 4.4: Analysis vs simulation results: generic model, Gamma distribution - 2 Chapter 5

Markov Chain Network Model

We can also model the heterogeneous wireless network channel as a Markov chain where

we assume exponential network residence time in each sub-network and memoryless tran-

sitions between sub-networks. Based on this Markov chain, we make decisions on video

source rate adaptation, turning the streaming process into a controlled Markov Decision

Process (MDP). We can then apply dynamic programming algorithm on this MDP, which

is assumed to achieve optimal performance.

5.1 Markov Decision Process Model

A common assumption on residence times in separated homogeneous wireless networks,

such as cellular and WLAN networks, is that they are exponentially distributed. Follow-

ing this assumption, we can assume the transitions between the two sub-networks to be

memoryless. The ABR in the channel is then modeled as a Markov chain with two states

{R1,R2} and transition probabilities

 p11 p12  P =    p21 p22    Hence a part of the streaming session can be characterized as a Markov Decision

Process (S, T , Φ), where S is the set of possible states in the streaming session, T is

36 Chapter 5. Markov Chain Network Model 37 the transition probability matrix between states, and Φ is the set of possible decisions

(quality levels) we can choose at each state.

We define the system state as s = {R, A, L}, where R is the ABR at the current time slot, A is the control action (quality level) chosen by the algorithm at the previous time slot, and L is the buffered length of video stream (in seconds) at the beginning of the current time slot. Thus the transitions take place as follows:

P (R(i +1) = Rl|R(i)= Rk)= plk,l,k = 1, 2

A(i +1) = φ(i) (5.1)

L(i +1) = L(i)+ R(i)/f −1(φ(i)) − 1

Note that the state variables A and L are completely determined given R and the current control action.

For an arbitrary admissible control action φ at time slot i, we associate a cost with the transition from i-th slot to (i + 1)th (from state s = {R(i),A(i),L(i)} to state s′ = {R(i + 1),A(i + 1),L(i + 1)}):

φ −1 2 Vi (s)= α (L(i)+ R(i)/f (φ) − 1) + g(φ(i) − A(i)), if i = N (5.2)

φ Vi (s)= g(φ − A(i)), i = 1, 2, ..., N − 1 (5.3)

where the g(φ − A(i)) part reflects the variation in the adapted video quality de- termined by the algorithm, and the α (L(i)+ R(i)/f −1(φ) − 1)2 part reflects another objective of our algorithms: high utilization of transmitted data, and α is a weight as- sociated with this part. N is the total number of time slots in the part of the streaming session we are observing. f −1(φ) is the inverse mapping from control action φ to the adapted video source rate. The objective of the algorithm is to minimize the expected cost over all transitions:

N Ψ Ψ Ji = E[ Vl(s )] (5.4) l=1 Chapter 5. Markov Chain Network Model 38

Thus the system becomes a finite-horizon controlled MDP. The process is not homoge- neous, because its transition matrix vary from time to time. As a result, there exists no optimal static control policy.

Now define the cost-to-go at time slot i for an admissible policy Ψ = (φ1,φ2, ...φN ): N Ψ Ψ Ji = E[ Vl(s )] (5.5) l=i According to Bellman’s Principle of Optimality, the optimal control policy is given by

the following equation:

∗ φi (s) = argmin{Vi(s)+ E[Ji(s)]} (5.6)

5.2 Dynamic Programming Algorithm

Based on the system model described by Equations (5.1) ∼ (5.6), we use a backward

induction based algorithm to solve for the optimal control action at every state:

∗ Algorithm 1 Find the optimal control policy Φ =(φ1,φ2, ...φN ). Require: T ≥ 1

i ⇐ N

for all states s do

φN (s) = argmin{VN (s)]} end for

while i ≥ 1 do

for all states s do

φi(s) = argmin{Vi(s)+ E[Ji(s)]} end for

i ⇐ i − 1

end while

After obtaining the optimal policy Φ∗, we can store it into a look-up table at the MT.

At the very beginning, we let the system start within T2N, from state s0 = {R2,q0, 0}, Chapter 5. Markov Chain Network Model 39

where R2 denotes the ABR in T2N, q0 denotes the quality level corresponding to the average ABR in the network: q = f −1( R¯2T¯2+R¯1T¯1 ). 0 T¯2+T¯1 Then at the beginning of each following time slot, the MT chooses the optimal action

according to current system state. Throughout this time slot, the server sets the quality

levels of video sequences to be transmitted as the optimal one and transmits them to the

MT.

A major difference between the dynamic programming algorithm and the adaptive

control algorithm lies in their estimation horizons. The dynamic programming algorithm

has the information about the network throughout the whole length of N, and utilizes

this information in order to achieve global optimality during [0,N]. Yet the adaptive algorithm only utilizes the information in the current cycle (though we assume the dis- tributions of residence times don’t vary from cycle to cycle in the model in simulation and analysis, it is not an assumption in the algorithm, and is not used by the algorithm.)

Hence the adaptive algorithm only tries to achieve local optimality within the cycle.

5.3 Simulation Results

The dynamic programming algorithm is supposed to provide optimal performance on the

Markov chain network model. While our adaptive algorithm can work with all kinds of residence time distributions, it also works on the Markov chain model. It is then of our interest to see how far away is the adaptive algorithm from optimum.

In the simulation, we generate realizations of T2’s and T1’s for a fixed length of time, within each residence interval the transmission rate is constant. We run our al- gorithm with different parameters on the generated network rate trace and the (CBR) video trace, then calculate the expected variation in a fixed length of time, i.e. V = 2 0

Parameter Value

Time slot length 1 s

T 150 s

R1 1 Mbps

R2 6 Mbps

r0 6 Mbps

p11, p22 0.1

T1, T2 10 s

NA 8

NB 17

Table 5.1: Simulation parameters for 2-zone Markov model

objectives is to achieve high data utilization over time.

Some parameters we used in the simulation are listed in Table 5.1. NA denotes the quantization level of control actions (number of control actions), and NB denotes the quantization level of buffered video length. Notice that, NB has to be increasing with

NA, otherwise, two control actions on one state may lead to a transition to states with the same buffered length of video, but different variations, so the control action (quality level) with the smaller variation will always be selected, and the other one (usually the largest one) will never be selected by the algorithm.

To ensure a fair comparison, the outputs of the adaptive algorithm and the simple algorithm are quantized using the same quantization level.

The performance of dynamic programming algorithm as a function of α is shown in

Figure 5.1. As we can see, since α represents the weight of the residual buffer part in the total cost, as α increases, the data utilization also increases, and the expected variation increases slightly.

The performance of adaptive algorithm and simple algorithm as a function of β is Chapter 5. Markov Chain Network Model 41

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1 0.032

0.995 0.03

0.028 0.99

0.026 0.985 V — 0.024 Utilization − − 0.98 0.022

0.975 0.02

0.018 0.97 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 α

Figure 5.1: DP: variation and utilization vs. α

0.03 0.032 0.034 0.036 0.038 0.04 0.042 0.14 0.98

0.96 0.12 0.94 0.1 0.92

0.08 0.9

V — 0.06 0.88 0.86 Utilization − − 0.04 0.84 Adaptive 0.02 Simple 0.82

0 0.8 0.03 0.032 0.034 0.036 0.038 0.04 0.042 β

Figure 5.2: Adaptive algorithm: variation and utilization vs. β shown in Figure 5.2. β is the proportional gain between the input and the “error” in the negative feedback system, hence adjusts the amount of variation at each step. From the

figure we can see that, there is an optimal operating point of β for a specific set of network model parameters, when β is too small, the variation at each step is too small. Although it offers a small total variation, the utilization can also be low because the cumulative adjusted video rate cannot track the cumulative transmission rate quite closely. Chapter 5. Markov Chain Network Model 42

1 DP 0.99 Adaptive Simple 0.98

0.97

0.96

Utilization 0.95

0.94

0.93

0.92 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 V

Figure 5.3: Comparison between algorithms: utilization vs. variation

While when β is too large, the variation at each step is also too large, thus may cause a certain amount of oscillation in the adjusted video rate and significantly deteriorates the performance in terms of both variation and data utilization. The data utilization becomes lower because if the control output (q(i)) is too high due to the oscillation, and the network transmission rate cannot sustain that quality level, only the highest sustainable quality will be chosen, hence the cumulative adjusted video rate cannot track the cumulative transmission rate closely. Generally, the choice of β is determined by a

lot of parameters, such as the distribution of the network, the MT’s moving speed and

moving pattern, the average video source rate and network transmission rates at different

zones, etc. Hence it is empirical and can be determined through experiments. Since the

distribution of networks is relatively stable, the proper choices of β under different MT

parameter sets can be stored in a look-up table in the coordinating BS or AP which is

providing other network-wise information to the MT and can be regularly updated.

To compare both types of algorithms, we draw the relationship between the time-

averaged variation and the data utilization in Figure 5.3.

Since our objectives are high utilization and low variation, the more upper-left the

curve is, the better the algorithm’s performance is. Dynamic programming algorithm Chapter 5. Markov Chain Network Model 43 achieves very high utilization and lowest variation at the same utilization level. The performance of adaptive control algorithm depends heavily on the choice of parameter, yet the optimal operating point provide near-optimal variation performance at some acceptable utilization levels. Also, the performance of adaptive algorithm is much better than that of the simple algorithm.

5.4 More Realistic 3-Zone Network Model

In the previously used 2-zone Markov chain network model, we assume constant bit-rate in each sub-network, and assume exponentially distributed residence times. While for

T1N which covers large area, we may make the assumption of constant bit-rate in the current moving area of the MT, we can hardly make the same assumption for the T2N network. This is because the T2N we are considering only covers local area, and the supported transmission rates may vary dramatically with the distance between MT and

AP. Thus we consider a more realistic network model in this chapter, where the T2N is assumed to support multiple transmission rates at different ranges (zones). We adopt a simple 2-zone T2N model here, as shown in Figure 5.4. We denote the two zones within

T2N as T 2No for outer zone, and T 2Ni for inner zone. Hence our network model becomes a 3-zone model.

Also, it is not generally true that the residence times are exponentially distributed (or

with other commonly used distributions) in a two-tier integrated wireless network such

as the one we are considering. Hence it is of our interest to explore how our proposed

algorithms perform under more realistic residence time distributions. We generate the

actual moving traces for MT in the network with a simple mobility model, and then

obtain the samples of the residence times within each zone along the traces.

One example of the generated map and MT’s moving trace is shown in Figure 5.5.

The map is repetitive so that the MT will reach the other end and continue moving if Chapter 5. Markov Chain Network Model 44

Low Rate

T1N r2

r1

BS

T2N High Rate

Figure 5.4: Integrated two-tier network with 2-zone T2N

300

250

200

150

100

50

0 0 50 100 150 200 250 300

Figure 5.5: An example of the generated user’s moving trace it passes the border of the map. The moving trace is generated by a simple mobility model: the MT picks a random point to start first, then at every step, it picks a random direction and random speed (with in a reasonable range [Vmin, Vmax]), and moves toward that direction for an exponentially distributed length of time (with mean T¯s). Chapter 5. Markov Chain Network Model 45

By simulating enough moving traces of the MT on each map, the distributions of the samples of residence times approximate to the real distribution. As we can see in

Figure 5.6, the distribution for T1 is near exponential, while the distributions for T2i and

T2o are far from exponential. While the distributions of residence times are generally affected by the mobility models and the coverage map, it is out of our scope to discuss about the realistic types of distributions in this thesis. So we rely on the simplified model for generating more realistic residence time distributions than exponential distribution.

Again, we want to claim that our adaptive algorithm does not rely on the structure of the specific distribution hence can work on any type of distribution in the real world.

5.5 PH-Fitting of Residence Times

While the adaptive algorithm only requires the mean values of residence times and hence can work directly on the generated distribution, the dynamic programming algorithm requires a structure of Markov chain and more parameters such as the transition prob- abilities between any two zones. We adopt Phase-Type (PH) distribution, as suggested and evaluated by [34], as a modeling tool to convert the generated distributions into a

Markov chain.

In general, a PH distribution is the distribution of the time until absorption in a

Markov chain. The states in this Markov chain are called “phases”, which generally have no physical meaning. PH distributions are highly versatile and can be used to approximate any distribution of non-negative random variables. A PH distribution can be described by (τ, Q), where τ is the initial distribution vector, and Q is the infinitesimal generator. We use a fitting tool called EMPHT [4] to fit the PH distributions. An example of the original and PH-fitted CDF’s for T1, T2o (residence time in outer region of T2N) and T2i (residence time in inner region of T2N) are shown in Figure 5.6. Each variable is fitted using a 2-phase PH distribution. As we can see, the CDF for T1 is quite Chapter 5. Markov Chain Network Model 46

Distribution function Distribution function Distribution function 1 1 1

0.9 0.9 0.9

0.8 0.8 0.8

0.7 0.7 0.7

0.6 0.6 0.6

0.5 0.5 0.5

0.4 0.4 0.4

0.3 0.3 0.3

0.2 0.2 0.2

0.1 0.1 0.1

0 0 0 0 50 100 150 0 10 20 0 10 20 30 40 − input, − − fitted PH − input, − − fitted PH − input, − − fitted PH

Figure 5.6: CDF’s of residence times in different zones

well approximated. Yet the fitted distributions of T2i and T2o are not that accurate. While increasing the numbers of phases will increase the accuracy of the model, it will also increase the state space and the complexity of the dynamic programming algorithm.

On the other hand, our experiments show that there will be no significant improvement of accuracy in terms of the value of the log-likelihood function in the EMPHT algorithm, when we increase the number of phases to 12 or more, in comparison to the 2-phase model. Hence we keep using 2 phases to fit the residence times in each zone.

After fitting the residence time distributions individually, we need to combine them together to obtain the Markov chain model of the integrated network. Figure 5.7 gives an illustration of this Markov chain. Denote the transition probability matrix (between different phases) within each zone and the initial distribution for each zone as {T1, τ1},

{T2o, τ2o} and {T2i, τ2i}, then the transition probability matrix for the Markov chain of the integrated network would be:

 T1 p12to τo 0    T =  p21t1 τ1 To poitiτi         0 pioto τo Ti   

where p12 = pio = 1, and the value of p21 and poi can be calculated as p21 = v21 ,p = voi , with v and v representing the numbers of transitions from T2N v21+voi oi v21+voi 21 oi Chapter 5. Markov Chain Network Model 47

71 71R 71L poi

P11 P2o ,1 P2i ,1

P12 P2o ,2 P2i ,2

p21

Figure 5.7: PH-fitted Markov chain network model

to T1N and from T2No to T2Ni in our collected traces, respectively.

Now we have finished constructing the Markov chain on which we can run the dynamic programming algorithm.

5.6 Estimation in Adaptive Control Algorithm

The dynamic programming algorithm can directly run on the 3-zone model. However, since the adaptive algorithm is designed for the 2-zone model, when it runs on the 3-zone model we need some modifications on the residence time and transmission rate estimation part. There are two types of information we can utilize to estimate the residence times.

5.6.1 Utilizing Statistical Information

In one approach, we can use the statistical information, i.e. the average residence times, as estimations. To extend the adaptive algorithm to the new model, we still treat the

T2N as a homogeneous zone, but change the way to calculate the average residence time and the average transmission rate in T2N. Since now we have two zones within T2N, the estimation of residence time in T2N for the adaptive algorithm is not directly given as the average residence time. Instead, we may have the average residence times with in each zone T , T , as well as the transition probabilities between zones p = voi and 2i 2o oi v21+voi Chapter 5. Markov Chain Network Model 48

pio = 1. Thus we can calculate the T2(0) in each cycle as follows: ∞ j T2(0) = T2 = [T2o + j (T2i + T2o)] poi p21 j=0 1 poi = p21[T2o +(T2i + T2o) 2 ] 1 − poi (1 − poi) poi 1 = p21[T2i 2 + T2o 2 ] (5.7) (1 − poi) (1 − poi)

The estimations at the following time slots is similar to what we described in Chapter 4.

Similarly, we calculate the average transmission rate inside the entire T2N and use it as the estimation of transmission rate in the adaptive algorithm.

5.6.2 Utilizing Localization Information

With the network model generated by actual mobility trace, we are able to obtain localiza- tion information of the MT at every time slot, hence we can also utilize the localization information to estimate the residence time. We make following assumptions when we simulate the case of utilizing the localization information:

1. The MT’s moving direction and speed, as well as its position at current time slot

are known.

2. The position of each T2N AP and their coverage areas (including both T 2No and

T 2Ni) are known.

3. The MT moves in straight line most of the time.

When estimating the residence time in each zone, the MT is expected to move toward

current direction with current speed until it performs the next T1N-to-T2N VHO. Thus

we can calculate the residence time in each zone by dividing the distances in each zone

by the moving speed. Chapter 5. Markov Chain Network Model 49

Parameter Value

Time slot length 1 s

T 150 s

R1 1 Mbps

R2i, R2o 3 Mbps, 6 Mbps

r0 6 Mbps

Vmin, Vmax 3.5 m/s, 5 m/s

r1, r2 30 m, 20 m

T¯s 20 s

NA 8

NB 15

Table 5.2: Simulation parameters for 3-zone model

5.6.3 Simulation Results for 3-Zone Model

The parameters we used in 3-zone network model simulation are listed in Table 5.2. The transition probability matrix we obtain from the above mentioned PH-fitting method with 2 phases in each zone is as follows:

 0.8350 0.1506 0 0.0144 0 0     0 0.8350 0 0.1650 0 0         0.0877 0 0.7662 0 0.0651 0.0810    T =      0.0112 0 0.2041 0.7662 0.0083 0.0102         0 0 00.0592 0.9073 0.0335         0 0 00.0115 0.0313 0.9572    The performance of each algorithm on the 3-zone model is shown in Figures 5.8- 5.10.

In Figure 5.9, we also draw the performance curve of the adaptive and simple algorithm running on the actual network model instead of the PH-fitted one. We can see that the Chapter 5. Markov Chain Network Model 50

0 0.1 0.2 0.3 0.4 0.5 0.03 1

0.028 0.99 0.026 0.98 0.024 0.97 0.022

V — 0.96 0.02

0.95 Utilization − − 0.018 0.94 0.016

0.014 0.93

0.012 0.92 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 α

Figure 5.8: DP on 3-zone model: variation and utilization vs. α

two curves for the same algorithm are reasonably near each other and exhibit the same

trend, which shows that the PH-fitting process does not incur a significant amount of

error when comparing the performance of the algorithms. Furthermore, it shows that

the adaptive algorithm performs better on actual model than on the PH-fitted one, thus

it is advantageous in the realistic problem we are considering.

Similar trends exhibited for the 3-zone model as for the 2-zone model. The data

utilization of the adaptive algorithm is significantly lower. This is because we use the ex-

pected values of residence time and transmission rate over the whole T2N as estimations,

the estimations are more inaccurate.

For the dynamic programming algorithm running on the 3-zone model, it was difficult

for us to generate more results when V < 0.013, which is primarily because of the quan- tization level we were using. The simulation was conducted on a Dell PowerEdge 1950 server with dual dual-core Intel Xeon 3.0 GHz processors and 2GB memory, and further increasing the dimensionality in this simulation would give us a “OUT OF MEMORY” error. However, judging from the trend of the curve we can still believe that the dynamic programming generates less variation than the adaptive and simple algorithms at lower Chapter 5. Markov Chain Network Model 51

0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.2 0.98

0.18 0.96

0.16 0.94 0.14

0.92 0.12

0.1 0.9 V—,-- Ut iliz ation – – 0.08 0.88 Adaptive(A) 0.06 Adaptive(M) Simple(A) 0.86 Simple(M) 0.04

0.84 0.02

0 0.82 0.015 0.02 0.025 0.03 0.035 0.04 0.045 β

Figure 5.9: Adaptive algorithm on 3-zone model: variation and utilization vs. β

1 Adaptive 0.99 Simple DP

0.98

0.97

0.96 Utilization

0.95

0.94

0.93 0 0.02 0.04 0.06 0.08 0.1 0.12 V

Figure 5.10: Comparison DP and AA: utilization vs. variation

data utilization area. Chapter 5. Markov Chain Network Model 52

Parameter Value

Time slot length 1 s

T 500 s

R1 0.453 Mbps

R2i, R2o 1.520 Mbps, 3.333 Mbps

Table 5.3: Simulation parameters for VBR network and VBR video

5.7 Simulating with VBR Network and VBR Video

Stream

As we mentioned before, our adaptive algorithm can work with VBR network and VBR video as well. While it is theoretically possible for the dynamic programming based algorithm to work with VBR cases (as long as we use Markov process to describe the network ABR and the video source rate), it would be either extremely complex in terms of computation, if the models are accurate and complex, or far from optimal, if the models are simple and inaccurate. Hence we only simulate and compare with the simple adaptive algorithm to show the effect of different types of long-term estimation. We simulate the VBR cases on the 3-zone network model, where T2N supports two ranges of transmission rates.

In every simulation, we use a random part of the “Tokyo Olympics” video trace provided by the authors of [28] to evaluate our algorithm. The average bit rate of this video trace is 3.71Mbps. The channel rates are generated randomly within each zone, with different average rates. Parameters used in the simulation are listed in Table 5.3.

Figure 5.11 shows one instance out of many simulations when we run adaptive al- gorithm and use statistical information to estimate the residence times. In this set of simulation, β = 0.01.

Figure 5.12 shows the same simulation instance running the algorithm with local- Chapter 5. Markov Chain Network Model 53

4 Network Rate (Mbps) Video Rate (Mbps) 3.5 Quality Level

3

2.5

2

1.5

1

0.5

0 0 50 100 150 200 250 300 350 400 450 500 t

Figure 5.11: VBR simulation: adaptive algorithm with statistical information

4 Network Rate (Mbps) Video Rate (Mbps) 3.5 Quality Level

3

2.5

2

1.5

1

0.5

0 0 50 100 150 200 250 300 350 400 450 500 t

Figure 5.12: VBR simulation: adaptive algorithm with localization information ization information. We use similar map as the one in Figure 5.5 to generate the MT’s moving traces and the localization information. In this set of simulation, we use the same parameters as listed above, β = 0.01.

Figure 5.13 shows the result of simple algorithm (without estimation of the residence Chapter 5. Markov Chain Network Model 54

6 Network Rate (Mbps) Video Rate (Mbps) Quality Level 5

4

3

2

1

0 0 50 100 150 200 250 300 350 400 450 500 t

Figure 5.13: VBR simulation: simple adaptive algorithm

times) with same parameter β = 0.01.

Comparing the three results, we can see that the adjusted video source rate curves are much smoother when we run the adaptive algorithm with either kind of estimation.

Averaging 1500 runs of experiments, we have the average performance with both types of estimation under different parameters, shown in Figures 5.14 - 5.16. We can see that, estimation with statistical information (i.e. mean values) leads to lower variations most of the time, but also with lower data utilization. This is because the amount of buffer left in the end depends on the accuracy of the estimation. Generally the localization information provides more accurate estimation, hence utilizing localization information leads to better data utilization but it also causes large variation when the MT changes the direction abruptly.

Besides, the results indicate that either kind of information will be helpful in estimat- ing the residence time and smoothing the video quality to overcome long-term variations in video adaptation. In our specific setting, the localization information might be more accurate on an average basis. Yet when the MT’s moving trace is more unpredictable, utilizing localization information may not have so much advantage over using statistical Chapter 5. Markov Chain Network Model 55

−3 x 10 9 Localization Simple 8 Statistical

7

6

V 5

4

3

2

1 0 0.005 0.01 0.015 0.02 0.025 β

Figure 5.14: Simulating VBR case - variation vs. β

0.96 Localization Simple 0.95 Statistical

0.94

0.93

0.92 Utilization 0.91

0.9

0.89

0.88 0 0.005 0.01 0.015 0.02 0.025 β

Figure 5.15: Simulating VBR case - utilization vs. β information. With a proper choice of parameter, the adaptive control algorithm can reduce the variation in quadratic form by 80% or more. Chapter 5. Markov Chain Network Model 56

0.96

0.95

0.94

0.93

0.92 Utilization 0.91

0.9 Localization Simple 0.89 Statistical

0.88 0 0.005 0.01 0.015 0.02 0.025 V

Figure 5.16: Simulating VBR case - utilization vs. variation Chapter 6

Conclusion

In this thesis, we have proposed an adaptive control algorithm for video streaming over

heterogeneous wireless networks with dramatically different available channel bit rates.

The proposed approach combines source rate adaptation with buffering and employs

certain information of the network, such as the expected residence time in each network

and the location of mobile device and access points.

By modeling the streaming process over an integrated two-tier network using a frame- work based on Markov chain with associated rewards (costs), the theoretical analysis for

CBR video, CBR channel case shows a significant reduction in video quality variation in comparison to some intuitive and complex algorithms.

Modeling the network using a Markov chain allows us to run a dynamic programming based algorithm to adjust the video source rate, the result of which is supposed to be optimal under our assumptions. In order to model the more complicated and realistic heterogeneous wireless network, we further introduced Phase-Type fitting of the resi- dence times and extended the algorithms to a 3-zone network model. Simulations with both 2-zone and 3-zone models indicate that our adaptive algorithm gives near-optimal performance with acceptable data utilization. At the same time, it has the advantage of low computation and requires less information of the network.

57 Chapter 6. Conclusion 58

Further in the simulations of the VBR cases, the result shows that by utilizing statisti- cal or localization information we are able to reduce the quality variation of the streamed

VBR video by 80% or more under the quadratic form metric.

From these results, we believe that although the proposed approach is simple and intuitive, it can improve the quality of streamed video significantly in terms of quality variation, and is promising in achieving constant-quality high-rate video streaming over heterogeneous wireless networks. Bibliography

[1] T. Alpcan, J. P. Singh, and T. Ba¸sar. Robust rate control for heterogeneous net-

work access in multihomed environments. IEEE Transactions on Mobile Computing,

8(1):41–51, 2009.

[2] J. G. Apostolopoulos, W.-T. Tan, and S. J. Wee. Video streaming: Concepts,

algorithms, and systems. Technical report, HP Laboratories, 2002, 2002.

[3] S. Aramvith, I.-M. Pao, and M.-T. Sun. A rate-control scheme for video trans-

port over wireless channels. IEEE Transactions on Circuits and Systems for Video

Technology, 11(5):569–580, May 2001.

[4] S. Asmussen, O. Nerman, and M. Olsson. Fitting phase-type distribution via the

EM algorithm. Scand.J.Statist., 23:419–441, 1996.

[5] L. Atzori, M. Krunz, and M. Hassan. Cycle-based rate control for one-way and

interactive video communications over wireless channels. IEEE Transactions on

Multimedia, 9(1):176–184, Jan. 2007.

[6] M. Bernaschi, F. Cacace, R. Clementelli, and L. Vollero. Adaptive streaming on

heterogeneous networks. In WMuNeP ’05: Proceedings of the 1st ACM workshop on

Wireless multimedia networking and performance modeling, pages 16–23, New York,

NY, USA, 2005. ACM.

59 Bibliography 60

[7] H.-C. Chuang, C. Huang, and T. Chiang. Content-aware adaptive media playout

controls for wireless video streaming. Multimedia, IEEE Transactions on, 9(6):1273

–1283, oct. 2007.

[8] Allot Communications. The allot mobiletrends report. http://www.allot.com/

mobiletrends.html, 2009.

[9] P. de Cuetos and K. W. Ross. Adaptive rate control for streaming stored fine-grained

scalable video. In NOSSDAV ’02: Proceedings of the 12th international workshop

on Network and operating systems support for digital audio and video, pages 3–12,

New York, NY, USA, 2002. ACM.

[10] P. de Cuetos and K. W. Ross. Optimal streaming of layered video: joint scheduling

and error concealment. In MULTIMEDIA ’03: Proceedings of the eleventh ACM

international conference on Multimedia, pages 55–64, New York, NY, USA, 2003.

ACM.

[11] M.U. Demircin, P. van Beek, and Y. Altunbasak. Delay-constrained and r-d opti-

mized transrating for high-definition video streaming over wlans. Multimedia, IEEE

Transactions on, 10(6):1155 –1168, oct. 2008.

[12] P. Frossard and O. Verscheure. Joint source/fec rate selection for quality-optimal

mpeg-2 video delivery. Image Processing, IEEE Transactions on, 10(12):1815 –1825,

dec 2001.

[13] V.K. Goyal. Multiple description coding: compression meets the network. Signal

Processing Magazine, IEEE, 18(5):74 –93, sep 2001.

[14] M. Hassan and M. Krunz. Video streaming over wireless packet networks: An

occupancy-based rate adaptation perspective. IEEE Transactions on Circuits and

Systems for Video Technology, 17(8):1017–1027, Aug. 2007. Bibliography 61

[15] G. Ji and B. Liang. Buffer schemes for vbr video streaming over heterogeneous

wireless networks. In IEEE International Conference on Communications (ICC),

Dresden, Germany, June 2009.

[16] G. Ji and B. Liang. Stochastic rate control for scalable vbr video streaming over

wireless networks. In Global Telecommunications Conference, 2009. GLOBECOM

2009. IEEE, pages 1 –6, nov. 2009.

[17] W. Li. Overview of fine granularity scalability in mpeg-4 video standard. IEEE

Transactions on Circuits and Systems for Video Technology, 11(3):301–317, March

2001.

[18] Y. Li, A. Markopoulou, J. Apostolopoulos, and N. Bambos. Content-aware playout

and packet scheduling for video streaming over wireless links. Multimedia, IEEE

Transactions on, 10(5):885 –895, aug. 2008.

[19] G. Liang and B. Liang. Effect of delay and buffering on jitter-free streaming over

random vbr channels. Multimedia, IEEE Transactions on, 10(6):1128 –1141, oct.

2008.

[20] K. McArthur. The future is here: Next is now. http://redboard.rogers.com/

2010/the-future-is-here-next-is-now/, 2010.

[21] Cisco VNI Mobile. Cisco visual networking index: Global mobile data traffic fore-

cast update, 2009-2014. https://www.cisco.com/en/US/solutions/collateral/

ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html, 2010.

[22] A. Ortega and M. Khansari. Rate control for video coding over variable bit rate

channels with applications to wireless transmission. In ICIP ’95: Proceedings of the

1995 International Conference on Image Processing (Vol. 3)-Volume 3, page 3388,

Washington, DC, USA, 1995. IEEE Computer Society. Bibliography 62

[23] M. Qin and R. Zimmermann. Improving mobile ad-hoc streaming performance

through adaptive layer selection with scalable video coding. In MULTIMEDIA ’07:

Proceedings of the 15th international conference on Multimedia, pages 717–726, New

York, NY, USA, 2007. ACM.

[24] H. Schwarz, D. Marpe, and T. Wiegand. Overview of the scalable video coding

extension of the h.264/avc standard. IEEE Transactions on Circuits and Systems

for Video Technology, 17(9):1103–1120, September 2007.

[25] Y. Sun, I. Ahmad, D. Li, and Y.-Q. Zhang. Region-based rate control and bit

allocation for wireless video transmission. Multimedia, IEEE Transactions on, 8(1):1

– 10, feb. 2006.

[26] T. Taleb, K. Kashibuchi, A. Leonardi, S. Palazzo, K. Hashimoto, N. Kato, and

Y. Nemoto. A cross-layer approach for an efficient delivery of tcp/rtp-based mul-

timedia applications in heterogeneous wireless networks. IEEE Transactions on

Vehicular Technology, 57(6):3801–3814, Nov. 2008.

[27] Y.-H. Tseng, E.H.-K. Wu, and G.-H. Chen. Scene-change aware dynamic bandwidth

allocation for real-time vbr video transmission over ieee 802.15.3 wireless home net-

works. Multimedia, IEEE Transactions on, 9(3):642 –654, april 2007.

[28] G. Van der Auwera, P. T. David, M. Reisslein, and L. J. Karam. Traffic and quality

characterization of the h.264/avc scalable video coding extension. Adv. MultiMedia,

2008(2):1–27, 2008.

[29] M. van der Schaar and H. Radha. A hybrid temporal-snr fine-granular scalability

for internet video. IEEE Trans. on CSVT, 11:318–331, 2001.

[30] P. Vidales, J. Baliosian, J. Serrat, G. Mapp, F. Stajano, and A. Hopper. Autonomic

system for mobility support in 4g networks. Selected Areas in Communications,

IEEE Journal on, 23(12):2288 – 2304, dec. 2005. Bibliography 63

[31] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the

h.264/avc video coding standard. IEEE Transactions on Circuits and Systems for

Video Technology, 13(7):560–576, July 2003.

[32] Wikipedia. PID controller. http://en.wikipedia.org/wiki/PID_controller#

PID_controller_theory.

[33] J. Xu, X. Shen, J.W. Mark, and J. Cai. Adaptive transmission of multi-layered

video over wireless fading channels. Wireless Communications, IEEE Transactions

on, 6(6):2305 –2314, june 2007.

[34] A. H. Zahran and B. Liang. Zone residence time mobility modeling and performance

evaluation framework for heterogeneous multi-tier wireless networks.

[35] A.H. Zahran and C.J. Sreenan. Cost efficient media streaming algorithms for rate-

dependent pricing strategies in heterogeneous wireless networks. In Next Generation

Mobile Applications, Services and Technologies, 2008. NGMAST ’08. The Second

International Conference on, pages 485 –491, 16-19 2008.

[36] A.H. Zahran and C.J. Sreenan. Pgms: Pseudo-optimal greedy media streaming al-

gorithm for heterogeneous wireless networks. In Networking and Communications,

2008. WIMOB ’08. IEEE International Conference on Wireless and Mobile Com-

puting,, pages 390 –396, 12-14 2008.

[37] A.H. Zahran and C.J. Sreenan. Threshold-based media streaming optimization

for heterogeneous wireless networks. Mobile Computing, IEEE Transactions on,

9(6):753 –764, june 2010.

[38] X. Zhu and B. Girod. Distributed rate allocation for video streaming over wireless

networks with heterogeneous link speeds. In IWCMC ’07: Proceedings of the 2007

international conference on Wireless communications and mobile computing, pages

296–301, New York, NY, USA, 2007. ACM.