Mobile Broadband - Service

Examensarbete LITH-ITN-KTS-EX--07/020--SE

Experience and Evaluation

Pontus Sandberg

2007-09-24

Department of Science and Technology Institutionen för teknik och naturvetenskap Linköpings universitet Linköpings universitet SE-601 74 Norrköping, Sweden 601 74 Norrköping LITH-ITN-KTS-EX--07/020--SE

Mobile Broadband - Service Experience and Evaluation Examensarbete utfört i kommunikations- och transportsystem vid Linköpings Tekniska Högskola, Campus Norrköping Pontus Sandberg

Handledare David Gundlegård Examinator Di Yuan

Norrköping 2007-09-24 Datum Avdelning, Institution Date Division, Department

Institutionen för teknik och naturvetenskap 2007-09-24

Department of Science and Technology

Språk Rapporttyp ISBN Language Report category ______Svenska/Swedish Examensarbete ISRN LITH-ITN-KTS-EX--07/020--SE x Engelska/English B-uppsats ______C-uppsats Serietitel och serienummer ISSN x D-uppsats Title of series, numbering ______

______

URL för elektronisk version

Titel Title Mobile Broadband - Service Experience and Evaluation

Författare Author Pontus Sandberg

Sammanfattning Abstract Mobile broadband is becoming an alternative to fixed broadband connections for residential areas. EGDE manages Internet Radio streams up to 96kbit/s while WCDMA R99 delivers satisfactory quality for media streams up to 320 kbit/s. Internet Radio is not possible to use over GPRS with satisfactory quality. The quality assessment for Internet Radio is highly dependent on the stream bit rate and a stream of 128 kbit/s delivers high user satisfaction. For online gaming other parameters than the well known latency, jitter and packet loss affects the session quality. Shorter Counter-Strike sessions are playable over HSPA where it outperforms WCMDA R99 in both network performance and user quality assessment. World of Warcraft has more limited network requirements and is playable over both WCDMA R99 and HSPA but suffers from latency impairments when played over GPRS/EDGE. The developed objective quality prediction model for First Person Shooter (FPS) games predicts gaming quality with an RMSE value of 0.609, prediction accuracy of 91.9% and correlation between predicted and subjective MOS of 0.8803.

Nyckelord Keyword QoE, GPRS/EDGE, WCDMA, HSPA, Gaming, User Perception, Quality Model Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extra- ordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/

The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/

i ”‘If we knew what it was we were doing, it would not be called research, would it?”’

- Albert Einstein

Publication year 2007

Supervisor J¨orgenGustafsson, Ericsson AB

Examiner Associate Professor Di Yuan Supervisor David Gundleg˚ard Link¨opingsUniversitet Mobile Communications, ITN

ii Abstract

EGDE manages Internet Radio streams up to 96kbit/s while WCDMA R99 delivers satisfactory quality for media streams up to 320 kbit/s. Internet Radio is not possible to use over GPRS with satisfactory quality. The quality assessment for Internet Radio is highly dependent on the stream bit rate and a stream of 128 kbit/s delivers high user satisfaction.

For online gaming other parameters than the well known latency, jitter and packet loss aﬀects the session quality. High start up or map load times will cause premature game termination and poor quality grading. Shorter Counter- Strike sessions are playable over HSPA where it outperforms WCMDA R99 in both network performance and user quality assessment. World of Warcraft has more limited network requirements and is playable over both WCDMA R99 and HSPA but suﬀers from latency impairments when played over GPRS/EDGE.

The developed objective quality prediction model for First Person Shooter (FPS) games, like CounterStrike, shows pretty good performance, despite a small number of test persons. The model predicts online gaming quality with an RMSE value of 0.609, prediction accuracy of 91.9% and correlation between predicted and subjective MOS of 0.8803.

Keywords: QoE, GPRS/EDGE, WCDMA, HSPA, Gaming, User Perception, Quality Model

iii Abstract in Swedish

EDGE klarar att leverera Internet Radio-flödenupp till bittakter av 96 kbit/s medan WCDMA R99 klarar bittakter upp till 320 kbit/s med tillfredställande kvalitet. GPRS kan inte användasförInternet Radio sessioner med tillfredsställande kvalitet. Kvalitetensbedömningenav ett Internet Radio flödeärstrikt beroende av flödetsbithastighet och Internet Radio-flödenp˚a128 kbit/s och högrelever- erar högkvalitet och tillfredställsebland användare.

Kvalitetsbedömningenav online spel p˚averkars av andra parametrar ände typiskt välkändalatency, jitter och paketförluster. Högastart- och kart- laddningstider leder till l˚agtbetyg p˚asessionen och ett tidigt avslutande av spelet. HSPA klarar kortare spel sessioner av CounterStrike och levererar av- sevärtmycket bättreprestanda och kvalitetsbedömningänförWCDMA R99. World of Warcraft har betydligt mindre krav p˚aaccessnätverket och ärspel- bart över b˚adeWCDMA R99 och HSPA men lider av högalatencyniv˚aerför GPRS/EDGE.

Den framtagna kvalitetsprediktionsmodellen förFirst Person Shooter (FPS) spel som CounterStrike visar p˚abra prestanda, trots baserad p˚aett begränsat antal testpersoner. Modellen predikterar kvalitet p˚aonline spel med ett RMSE värdep˚a0.609, en prediktionssäkerhet p˚a91.9% och en korrelation p˚a0.8803 mellan predikterat och subjektivt MOS.

iv Acknowledgments

First of all I would like to thank my supervisor J¨orgenGustavsson at Ericsson Research for insightful comments and guidance during my work. I would also like to thank Gunnar Hiekkil¨afor your competent comments and honest opin- ions especially regarding the model chapter. I also appreciate the participants in my online gaming survey.

Further, thanks to Jim Olander and Jonas Nilsson for welcoming and helping me with the testing activities. You both greatly contribute to the ﬁnal result of this thesis.

Finally, I would like to say thank you to my invaluable family and my lovely Helena. I am grateful for your love and the way you are supporting me in every possible way.

Pontus Sandberg, Lule˚a,2007

v Abbreviations

2G 2nd generation of mobile networks 3G 3rd generation of mobile networks 3GPP 3rd generation partnership project ACK Acknowledgment ARQ Automatic Repeat-reQuest BER Bit Error Rate BLER BLock Error Rate BSC Base Station Controller BSS Base Station System CDMA Code Division Multiple Access CS Circuit-Switched DL DownLink DSL Digital Subscriber Line EDGE Enhanced Data rates for GSM Evolution EGPRS Enhanced GPRS ETSI European Telecommunications Standards Institute EUL Enhanced Uplink FDMA Frequency Division Multiple Access FPS First Person Shooter GGSN Gateway GPRS Support Node GPRS General Packet Radio Service GSM Global System for Mobile communications HSDPA High Speed Downlink Packet Access HSPA High Speed Packet Access HSUPA High Speed Uplink Packet Access IP Internet Protocol ISDN Integrated Services Digital Network ITU International Telecommunication Union kbit/s Kilobit per second kHz kilo Hertz KPI Key Performance Indicator LAN Local Area Network LTE Long Term Evolution MB MegaByte Mbit/s Megabit per second Mcps Mega chips per second

vi MHz Mega Hertz MIMO Multiple Input Multiple Output MMORPG Massive Multiplayer Online Role Playing Game MS Mobile Station NSS Network Switching System OSS Operation and Support Subsystem PLMN Public Land Mobile Network PSDN Public Switched Data Network PSTN Public Switched Telephone Network QAM Quadrature Amplitude Modulation QoE Quality of Experience QoS Quality of Service RAN Radio Access Network RNC Radio Network Controller RPG Role Playing Game RTS Real Time Strategy RTT Round Trip Time SF Spreading Factor SGSN Serving GPRS Support Node SIM Subscriber Identity Module SIR Signal to Interference Ratio TCP Transmission Control Protocol TDMA Time Division Multiple Access TP ThroughPut TS TimeSlot TTI Transmission Time Interval UDP User Datagram Protocol UE User Equipment UL UpLink UMTS Universal Mobile Telecommunication System WCDMA Wide Code Division Multiple Access VoIP Voice over IP

vii Contents

1 Introduction 1 1.1 Related Work ...... 1 1.2 Objective ...... 1 1.3 Work Description ...... 2 1.4 Chapter outline ...... 2

2 Cellular Radio Networks 3 2.1 Technical Approach ...... 3 2.1.1 Cellular Networks ...... 4 2.1.2 Handovers ...... 4 2.1.3 Multi-path Fading ...... 5 2.1.4 Path Loss ...... 5 2.2 Multiple Access Schemes ...... 5 2.2.1 Time Division Multiple Access ...... 5 2.2.2 Frequency Division Multiple Access ...... 6 2.2.3 Code Division Multiple Access ...... 6 2.2.4 Othogonal Frequency Division Multiplexing ...... 7 2.3 GSM ...... 7 2.3.1 GSM System Architechture ...... 7 2.4 GPRS ...... 9 2.4.1 EDGE ...... 10 2.5 UMTS ...... 11 2.5.1 UMTS QoS Classes ...... 11 2.6 WCDMA R99 ...... 12 2.6.1 Power Control ...... 12 2.7 HSPA ...... 13 2.7.1 HSDPA ...... 14 2.7.2 EUL ...... 14 2.8 LTE ...... 14

3 Quality of Experience Parameters 15 3.1 QoS vs. QoE ...... 15 3.2 User-Driven Performance Evaluation ...... 16 3.3 Key Performance Indicators ...... 17 3.3.1 Network delay ...... 19 3.3.2 Latency ...... 19 3.3.3 Jitter ...... 19 3.3.4 Packet Loss ...... 19

viii CONTENTS CONTENTS

3.3.5 Throughput ...... 19 3.4 Service Performance Requirements ...... 20 3.4.1 Voice ...... 20 3.4.2 Videophone ...... 20 3.4.3 Online Gaming ...... 21

4 Service Evaluation Cases 22 4.1 Internet Radio ...... 22 4.1.1 Audio Coding ...... 24 4.2 Online Gaming ...... 25 4.2.1 FPS ...... 25 4.2.2 RTS ...... 26 4.2.3 MMORPG ...... 26 4.2.4 Game Phases ...... 27 4.2.5 Traﬃc types ...... 27 4.3 Test setup ...... 28 4.3.1 Hardware ...... 29 4.3.2 Network load ...... 29 4.3.3 Software ...... 30 4.4 Data Post-Processing ...... 31 4.4.1 CounterStrike Gaming Experience ...... 31

5 Test Results and Analysis 32 5.1 Internet Radio ...... 32 5.1.1 Service Functionality ...... 34 5.2 Online Gaming, FPS: CounterStrike ...... 36 5.2.1 Service Functionality ...... 37 5.3 Online Gaming, MMORPG: World of Warcraft ...... 39 5.3.1 Service Functionality ...... 40

6 Online Gaming Quality Model: FPS 42 6.1 Principal Components Analysis ...... 42 6.2 Quality Model ...... 46 6.2.1 Performance Calculation Methods ...... 48 6.2.2 Model Performance ...... 50 6.3 Extended Quality Model ...... 52

7 Conclusions 54

ix Chapter 1

Introduction

Mobile broadband is becoming an alternative to fixed broadband connections for residential areas. High speed radio access, such as HSPA with bit rates up to 14 Mbit/s, enables the mobile terminals and systems to be used for Internet services traditionally accessed over DSL or other fixed access. However, it is not obvious that all services normally used with fixed broadband can be used over mobile broadband with equal quality.

The service characteristics put different requirements on the Internet connection. For example, delay (or latency) is most important for online gaming, while throughput (or high bit rate) is most important for file sharing. The combination of service requirements and a certain radio access will result in a perceived quality of the service for a user. Can mobile broadband substitute a broadband connection with no perceived diffrence or does it exist service specific limitations?

1.1 Related Work

Previously performed studies in this subject have been done at Swinburne Uni- versity [1, 2] where focus have been on game traﬃc analysis and service assessment for ﬁxed network impairments. Two Master Thesis’s [3, 4] have been performed at Ericsson where characteristics and functionality of online games have been evaluated in a test radio environment. Previously only limited online gaming service evaluation in an operators network have been done, as this thesis contains.

1.2 Objective

This master thesis evaluates how well broadband services work in different radio networks and environments, for example GPRS/EDGE, WCDMA R99 and HSPA. For each service this thesis describes, identification of key parameters that influences service quality and their limitations. One important issue is to evaluate how a degradation in the radio environment affects’s the user perception. The services that has been tested and evaluated are: Internet Radio and two network based online games. The two games are CounterStrike and World

1 1.3. WORK DESCRIPTION of Warcraft, and originates from two separate game genres. Finally, this thesis describes an objective quality prediction model for online gaming.

1.3 Work Description

The work have been be divided into the following steps: 1. A literature survey have been conducted to gain insight to cellular radio networks and service evaluation 2. The testing activities have been deﬁned together with an introduction to the testing equipment 3. Testing execution for chosen services together with a smaller group of test persons in a subjective test.

4. Data analysis of test results, i.e. service evaluation over various radio accesses and identiﬁcation of the most important service performance parameters 5. The analysis work serves as input to a quality prediction model for online gaming 6. Finally, analysis and conclusions of service experience over mobile networks and online gaming quality model performance

1.4 Chapter outline

The structure of this thesis is laid out as follows: Chapter 2 gives an introduction to Cellular Radio Networks and a technical description of multiple access schemes and techniques. Chapter 3 describes Quality of Experience, Key Performance Indicators and Service Evaluation. This is followed by chapter 4 which outlines the services who are tested and the testing equipment. Chapter 5 provide results and some initial analysis from the testing activities. Section 6 describes an online gaming quality model for estimating the quality of an online gaming session and a proposed future extension of this model. Finally, chapter 7 contains conclusions from the test results chapter and future work in the ﬁeld of study.

2 Chapter 2

Cellular Radio Networks

Technologies as the Universal Mobile Telecommunications Systems (UMTS) and High Speed Packet Access (HSPA) are cornerstones in todays and future communications. They are the continuation of the first digital communication system, the Global System for Mobile Communications (GSM). The first step toward Third Generation networks (3G) were done by introducing data capabilities to GSM through the General Packet Radio Service (GPRS). UMTS and HSPA provides the contingency for new services as video telephony, web browsing, file downloads, video streaming and other multimedia services over an air-interface. The mobile wireless systems are currently evolving and there is a steady demand for higher data rates and improved Quality of Service (QoS). This chapter will introduce the reader to Cellular Radio Networks and also take a glimpse at future wireless networks like Long Term Evolution (LTE). Further additional reading on mobile communications systems can be found in [5] and [6].

2.1 Technical Approach

In radio network communication there are two kinds of radio bearers, Circuit- Switched (CS) and Packet-Switched (PS). When a CS connection is established network capacity is reserved during the entire session, even though sometimes no data is transmitted. Existing networks as PSTN and ISDN are typical CS networks. Billing is here based on the session time.

The other network is the PS network. No capacity is reserved and data is routed through the network in small pieces, called packets or frames. The Inter- net is a typical PS network and billing is based on received/transmitted data. Although both CS and PS exist in todays telecommunication there is an obvious convergence toward an all-IP packed-based network. Higher data-rate demands and lower maintenace costs are the two main reasons for this IP convergence. [5]

Deﬁnition: A mobile station (MS) is the user equipment used to access the cellular radio network, for example; a mobile phone, a PDA or a Laptop.

3 2.1. TECHNICAL APPROACH

2.1.1 Cellular Networks Cellular radio is a technique to increase the capacity of mobile radio services. The use of multiple low-power transmitters in a cellular network puts limits on the transmission range. The geographical area is then divided into smaller areas, called cells, each served by one or more antennas located on a base station. The location of a base station is called a site. A base station who uses an antenna that radiates in in all directions is called an omnidirectional. By introducing directional antennas cells can be further divided. This method is called sectori- sation. Figure 2.1 shows three sites (left) and on directional antenna (right).

Definition: Uplink transmission is defined as traffic from the mobile station to the network and downlink transmission as the opposite.

Adjacent cells in GSM are assigned different frequencies to avoid crosstalk or interference. In WCDMA systems this is done by using diffrent othogonal codes. However, cells that are sufficiently spatial separated can reuse the same frequencies or codes to increase the system capability. A cellular radio network is often visualised by a simplified model using a hexagonal pattern. In practice, due to variations in signal propagations conditions and local hot-spots a precise hexagonal pattern is not used.

Figure 2.1: Three omnidirectional cell sites (left) where one antenna covers one whole cell. To the right a sectorized cell with three directed antennas.

2.1.2 Handovers Cellular systems are based on mobility. Mobile stations are able to move while connected to the network. When a user crosses a cell border it will result in a handover between the two cells. The connection will be switched from the subchannel in the ﬁrst cell to an available subchannel in the new cell. In case of no free channels the connection will be dropped. By measuring the broadcast channel of nearby base stations, a handover is made when the neighboring broadcast channel is higher than the current one. There are three types of handovers:

• Hard handover; A mobile station is switched between two base stations, at any time instance the MS isconnected to only one of them. • Soft handover; A mobile station is connected to two, or more cells at the same time.

4 2.2. MULTIPLE ACCESS SCHEMES

• Softer handover; Like soft handover, buth all cells belog to the same base station.

2.1.3 Multi-path Fading A fundamental characteristic of many radio channels is that the strength of the received signal ﬂuctuates in a random way. This phenomenon is called fading. A mobile station is often not in the line of sight from a base station, meaning several bouncing rays of the signal will reach the receiver. Each ray causes a phase shift and because of movement of obstacles, transmitter and receiver signal power will vary. Multi-path fading is often referred to as fast fading and is often modeled as a Rayleigh distribution. Another fading phenomenon is shadow fading. It is caused by series of obstacles that causes signal attenuation.

2.1.4 Path Loss Due to the distance between the transmitter and the receiver the signal suﬀers from attenuation, referred to as path loss. The detailed components are Pr receiver transmit power, Ps sender transmit power, Gs sender antenna gain and Gr is the receiver antenna gain and are signal dependent. A general path loss model is formulated as:

Pr = PsGsGrGpath (2.1) where d is the distance between sender and receiver, d0 reference distance, λ0 carrier frequency and α the path loss exponent (normally in the range from 2 to 5 according to measurements). Depending on radio environment, a variety of wireless propagation models can be used. The most known ones are the Okumura-Hata and the Walﬁsch-Ikegami propagation models. They take into account: frequency, base- and mobile station antenna height, area type and distance and are derived from a combination of empirical and deterministic research. [7]

2.2 Multiple Access Schemes

In a cellular radio network multiple users are able to communicate at the same time, using the same physical resource (i.e. the air interface) to receive and transmit information. In wireless communication there are several diﬀerent multiple access schemes to divide the air interface between users.

2.2.1 Time Division Multiple Access

Time Division Multiple Access (TDMA) divides a time period Tf into a number of non overlapping subintervals, each with the duration Tf /N. Each subinterval, or timeslot can be assigned to a user. To increase the capacity several timeslots can be assigned to one user, called multi-slot operation. Synchronization and guard-bands in the time domain is needed to prevent interference between users.

5 2.2. MULTIPLE ACCESS SCHEMES

Figure 2.2: Multiple access schemes.

2.2.2 Frequency Division Multiple Access Another approach is Frequency Division Multiple Access (FDMA) where the available frequency spectrum is used to create a number, N, of non-overlapping subchannels. These channels are separated by guard-bands. A combination between TDMA and FDMA schemes is possible and is referred to as F/TDMA. It can be used with ﬁxed channel assignment or by changing channel with a speciﬁc pattern, called frequency hopping number.

In FDMA and TDMA systems a channel is assigned to a user for the duration of a session. When the traﬃc is of bursty nature, as for speech with periods of silence, these silent periods carries no information. A part of the system capacity is being wasted. Systems limited by frequencies or in the latter case timeslots is called hard capacity systems.

2.2.3 Code Division Multiple Access Code Division Multiple Access (CDMA) overcomes the problem of capacity waste by allowing users to communicate in the same time- and frequency domain but separated by codes. CDMA systems are interference limited, meaning every new user contributes to the interference level. By multiplying the signal with a code, the signal will be spread over a wide frequency, i.e. the power per Hertz will be low. At the receiving side the same code will be applied to all incoming signals and the wanted signal will be despread and decoded while the other signal components will appear as noise. When the interference level is so high that the receiver barley can detect the wanted signal, no more users is allowed to communicate in the cell. [8]

In CDMA systems there are many users who communicates in the same frequency- and time domain. Mobiles that are on diﬀerent distances from the

6 2.3. GSM base station transmits with a certain individual transmission power. If a mobile near the base station transmits with too high power it will make other signals with lower power undistinguishable. On the other hand, if a station far from the base station transmits with to little power, the signal may never reach the base station. This problem is called the near-far eﬀect and is dealt with by power control, where the channel is estimated before transmitting. This approach is used in current 3G systems such as UMTS where frequency bands are 5MHz wide. The access method is called Wideband CDMA (WCDMA). Figure 2.2 shows TDMA, FDMA and CDMA schemes. [9], [10]

2.2.4 Othogonal Frequency Division Multiplexing Orthogonal Frequency Division Multiplexing (OFDM) uses a large number of closely-spaced orthogonal sub-carriers. The idea is to split a high-rate data stream into a number of low-rate streams to be transmitted in parallel. Multi path delay and interference is decreased when a lower data rate is transmitted on each sub-carrier. OFDM can be viewed as many simultaneously slowly transmitted narrow band signals instead of just one fast wideband signal. OFDM is chosen by the 3GPP for its Long Term Evolution Project (LTE) described in section 2.8.

2.3 GSM

The digital GSM-system replaced the old analog cellular phone system and started a new era in telecommunications. GSM represents the second generation networks, often referred to as 2G. GSM speciﬁcations were settled by the European Telecommunications Standards Institute (ETSI). Although GSM was standardized in Europe; it became, and still is, world-wide utilized. GSM uses TDMA to divide the available bandwidth and to serve as many users as possible. The spectrum is divided into uplink and downlink bands and the channels are separated with a 200 kHz guard band, which gives 125 carrier frequencies in the 900 MHz band.

In Europe GMS frequencies are also available at 450, 1800 and 1900 MHz. Each carrier utilizes one TDMA-frame and this frame consists of eight timeslots (TS). The duration of one frame is 4,615 ms, giving each TS 0,577 ms. The Guard Period (GP) is an empty space separating the carriers during transmission. Like other cellular networks GSM allows roaming services, meaning that users can use their phones outside of their home network.

2.3.1 GSM System Architechture The GSM network consists of the Base Station System (BSS) and the Network Switching System (NSS). The BSS carries out all the radio functions and consists of the following units: • Base Transceiver Station (BTS) communicates radio signals with the user equipment or the Mobile Station (MS). • Base Station Controller (BSC) which manages and controls the radio function.

7 2.3. GSM

4.615 ms 0 1 2 3 4 5 6 7

TB Data Training Data TB GP

3 57 1 26 1 57 3 8.25 156.25 bits

0.577 ms

Figure 2.3: A basic TDMA frame with eight timeslots. The tail-bits (TB) are used as start/stop sequence for a TS and the training sequence is used by the equalizer to create a channel model.

The MS can be a hand held device or installed in a vehicle. A MS has to be personalized, i.e. associated with a given subscription. In the GSM system this is done with the Subscriber Identity Module, or SIM. Without a SIM, a user can’t access the network, except for emergency traﬃc. The NSS includes the following functional units: • Mobile Services Switching Centre (MSC), responsible for switching and su- pervision functions. The MSC also serves as a gateway from the PSTN and other networks and serves a number of Base Stations Controllers (BSC). • The Home Location Register (HLR) contains user information and authentication parameters. • The Visitor Location Register (VLR) contains user information relating where the user currently is located. This means when a user is called, only a particular location area have to be paged, not the entire network. • In the Authentication Centre (AuC) HLR is provided with authentication and ciphering keys for security reasons.

• The last unit in the NSS is the Equipment Identity Register (EIR). It is connected with the MSC via a signaling link and checks the validity of the used equipment. Network management is realized by Operation and Support Subsystem (OSS) and is connected to both NSS and BSS. OSS realizes functions as performance management, network administration and traﬃc measurements. [11], [12]

In PSTN speech is digitalized at a rate of 64kbit/s. The GSM uses another, highly optimized approach to make more eﬃcient use of the available spectrum/bandwidth. Speech is coded at 12.2 kbit/s that uses a full rate channel.

8 2.4. GPRS

PSDN ISDN IP PSTN PLMN

NSS

VLR GGSN HLR MSC OSS AuC SGSN EIR

BSS BSC

BTS

Figure 2.4: GSM and GPRS architechture.

Speech quality is ensured by additional bits for error detection and correction. Coding sequences at 12.2 kbit/s involves speech blocks of 20 ms and consists of 260 bits. [13]

2.4 GPRS

General Packet Radio Service (GPRS) is a packet-based IP connectivity solution and was the start of Mobile Internet with applications as WAP, iMode, MMS and so forth. GPRS is an extension to the GSM architecture and not an entirely new system, additional GPRS nodes are presented in figure 2.4. GPRS is, together with EDGE, described as 2,5G. By allowing timeslots previously used for circuit-switched telephony to be filled with data packets enabled new network capabilities with slightly modified network architecture. The network can adjust capacity between voice and data functions to allow more data traffic when voice traffic is low and vice versa. The GPRS elements to be added in the GSM architechture are: • Serving GPRS Support Node (SGSN) directs/receives packed data to/from the BSC and authenticates and tracks the location of the MS. • The Gateway GPRS Support Node (GGSN) acts as a proxy toward external IP-networks and manages dynamic IP addresses to the MS. GPRS offers a connection between voice and data services, meaning while in a data session; users can accept an incoming voice call. GPRS uses four different coding schemes, CS1 to CS4 depending on the radio environment. Close to the BTS a less robust and thereby faster coding scheme can be used. Throughput

9 2.4. GPRS

GPRS TP/TS (kbit/s) EDGE TP/TS (kbit/s) CS1 8 MCS1 8,8 CS2 12 MCS2 11,2 CS3 14,4 MCS3 14,8 CS4 20 MCS4 17,6 MCS5 22,4 MCS6 29,6 MCS7 44,8 MCS8 54,4 MCS9 59,2

Table 2.1: Modulation and coding schemes for GPRS and EDGE per timeslot varies from 8 to 20 kbit/s depending on coding scheme. If additional slots are available they can be assigned to a user creating a multi slot transmission and boost the data rate, and typical GPRS rates is around 40 - 50 kbit/s. An assignment of 3-4 TS per MS is most often used for data traﬃc. [5]

2.4.1 EDGE Enhanced Data rates for GSM Evolution (EDGE) is an enhancement to the packet switched GPRS. EDGE is a method to increase the data rates on a GSM radio link by adding new modulation and channel coding techniques. Through the use of existing GPRS standards EDGE can deliver signiﬁcant improvements, this implies that EDGE is an add-on to existing GPRS networks and cannot work alone. To achieve higher data rates EDGE has the contingency to utilize more modulation methods (MCS 1-8, Table 2.1), each optimized to the existing radio environment. Theoretically EDGE can provide 59,2 kbit/s in each if the eight timeslots, adding up to a peak network rate of 473,6 kbit/s if eight timeslots are used. Todays devices can occupy up to four timeslots which results in a peak data rate of about 200 kbit/s. [5], [14]

Before radio blocks can be sent over a radio interface, the transmitter has to address the packets with a unique identification number. In GPRS the packets are numbered from 1 to 128, with a sliding window of 64 packets. After the transmission of a packet sequence the transmitter receives information if some packets were defected and to be resent. This erroneously decoded packet may have the same identification number as a new packet in the queue. Because the addressing window only have 64 addresses the protocol between the terminal and the network might stall and the whole frame have to be retransmitted. In EDGE the identification numbers have been increased to 2048 and the addressing window to 1024. This minimizes the risk for stalling and prevents unwanted throughput decrease.

EDGE uses link adaptation which is a technique that measures the quality of the radio link and based on this knowledge it chooses among the available modulations schemes. Measurements are done on each burst instead of every 240ms in the GPRS system. This results in highly accurate measurements and enables to a quick reaction when the radio environment changes. [14]

10 2.5. UMTS

2.5 UMTS

UMTS is one of the third-generation-mobile technologies (3G) and employs Wideband CDMA as radio-access technology (WCDMA), described in chapter 2.6. The primary benefits for UMTS are higher user bit rates, better spectral efficiency and simultaneous transmission of voice and data, lower infrastructure costs and a migration towards an all-packet-based network. Compared to other mobile networks, like GSM, UMTS provides different QoS classes for different kinds of traffic described below. The UMTS network achitechture consists of three parts namely; the mobile station, the radio access network UTRAN and the core network seen in figure 2.5.

Core Network

UTRAN RNC Iur RNC

Iub Node B

Node B Node B UE Uu

Figure 2.5: UMTS network architechture

2.5.1 UMTS QoS Classes UMTS employs four fundamental Quality of Service (QoS) traﬃc classes for packet-switched data. This QoS architecture divides applications and services depending on, for example, how delay-sensitive the traﬃc is. These four classes are:

• Conversional - real time data as voice, video telephony and online and interactive games. Very delay sensitive. • Streaming - Streaming of multimedia, moderately delay sensitive. • Interactive - Web browsing. Preserve data integrity, some delay accepted.

• Background - Downloads of email or ﬁles. Preserve data integrity, delay may be seconds or even minutes. A conversional service as real-time conversation is restricted by human perception of maximum end-to-end delay. Evaluations have shown that an unacceptable quality is reached when the end-to-end delay exceeds 400 ms.

11 2.6. WCDMA R99

2.6 WCDMA R99

WCDMA R99 technology has emerged as the most widely adopted third generation interface and its standardization is done by the 3GPP (the 3rd Generation Partnership Project). In release 99 theoretical downlink data rate is just over 2Mbit/s and uplink pratical 384 kbit/s. WCDMA R99 uses a chip rate of 3,84 Mcps which lead to a carrier bandwidth of approximately 5 MHz. WCDMA R99 spreads user information over a wide bandwidth by multiplying it with spreading codes, this keeps interference low and supports very high bit rates. It also supports Bandwidth on Demand (BoD) where a highly variable user data rate is supported. Each user allocates frames of 10 ms duration and the data capacity among the users can change from frame to frame. Finally WCDMA R99 is backwards compatible with existing GSM networks; therefore handovers between the two networks is possible.

2.6.1 Power Control A fast closed loop is the solution to power control in WCDMA R99. The base station and the MS performs frequent estimations (1500 times per second) of the received Signal-to-Interference Ratio (SIR), both in the up- and downlink. This SIR is then compared with the target SIR and the mobile station transmit power is then regulated. Power control regulates the target SIR according to the needs of the individual radio link and aims for a constant quality. This quality is often deﬁned as a speciﬁc bit error rate (BER) or block error rate (BLER). The fact that battery lifetime of mobile stations is limited, saving power resources is crucial. [13], [9]

12 2.7. HSPA

2.7 HSPA

High Speed Packet Access is a WCDMA upgrade and with theoretical data rates of up to 14 Mbit/s in the downlink and up to 5 Mbit/s in the uplink. HSPA boosts system performance and improves the user experience by providing a reduced latency and increased data rates. HSPA makes mobile broadband a reality by improving service experience for web access, FTP, VoIP and streaming services. By the following means can HSPA increase the system capacity: • High speed shared channel transmission, shared both in code and time domain.

• A shorter Transmission Time Interval (TTI), reduces round-trip time and improves link adaptation. • Fast scheduling, prioritizes users with good channel conditions. • Fast retransmission and higher modulation schemes.

• Fast hybrid Automatic Repeat reQuest (ARQ). A mobile station can request for a rapid retransmission, improves system performance and en- hances robustness.

4000 HSPA TP downlink HSPA TP uplink 3500

3000

2500

2000 kbit/s

1500

1000

500

0 0 50 100 150 200 250 time [s]

Figure 2.6: FTP upload and download over HSPA with EUL, HSPA reference speedtest performed at UTN

HSPA is a combination between High Speed Downlink Packet Access (HS- DPA) and High Speed Uplink Packet Access (HSUPA), also known as Enhanced Uplink (EUL). Below are basic principles of HSDPA and EUL described.

13 2.8. LTE

2.7.1 HSDPA HSDPA is based on a shared channel transmission, meaning the channel codes and the transmission power in a cell is seen as a common resource. This results in more eﬃcient use of available resources. In 3GPP Rel 5, HSDPA adds a new transport channel to WCDMA, the High Speed Shared Channel (HS-DSCH). In future HSPA evolved, additional downlink 2x2 MIMO (Multiple Input Multiple Output) is supported. By using two antennas and 64 QAM modulation scheme the peak data rate can be doubled, enabling a data rate of 42 Mbit/s. Operators are currently deploying HSDPA networks with downlink data rates of up to 7.2 Mbit/s.

2.7.2 EUL Enhanced Uplink (EUL) is deﬁned in 3GPP Rel 6 as a new transport channel, called Enhanced Channel (E-DCH) and is dedicated to only one user during a session. EUL improves uplink performance by introducing features as multi code transmission, fast hybrid ARQ, improved scheduling and shorter TTI, as short as 2 ms compared to previous 10 ms. When EUL together with HSDPA is fully deployed operators have the freedom to move traditional voice traﬃc over to an IP-based platform. Hence new innovative voice and data applications are enabled in the packet domain. [15], [16]

2.8 LTE

3GPP:s Long Term Evolution (LTE) is the continued improvement of the UMTS network to ensure future competitiveness among other emerging radio technologies. LTE aims to be an extensive evolution of today’s 3G and a precursor to upcoming 4G networks. The overall intent of LTE is to provide an extremely high-performance radio access with downlink data rates of 100 Mbit/s and 50 Mbit/s in the uplink. Planned to be released in 2009, a number of speciﬁc goals for the LTE project have already been set up.

LTE is going to be an ”All-IP” network, based on the Internet IP core protocol. LTE will also support high user mobility and increased QoS for packet services; this means for example traditional circuit-switched telephony will be replaced by Voice over IP (VoIP). LTE also facilitates usage of advanced antenna technologies, such as MIMO concepts to boost system performance even further. [5], [17]

14 Chapter 3

Quality of Experience Parameters

A wide range of applications as e-mail, file downloads, instant messaging, video and voice streaming and on-line gaming are widely used over a traditional fixed Internet accesses. With the introduction of mobile broadband raises the oppor- tunity of using these applications over a radio interface. But there are uncertainties of how well traditional broadband services works when connected over a radio access network and which parameters that influence the service quality the most. This chapter will introduce the reader to Quality of Experience (QoE) and evaluation of important service parameters. Additional reading can be found in [18] and [19].

3.1 QoS vs. QoE

Mobile data services are growing very rapidly. Improved terminals, new applications and networks with higher data rates contribute to this higher demand. Technology centered measurements like data rate, delay and bit error rate re- veals little of the true user experience of a session. This type of measurements are often called the QoS of the network. With the growth of mobile services it has become important for operators and developers to measure the QoE both of the network and the services provided. QoE could be deﬁned as ”‘the overall acceptability of an application or service, as perceived subjectively by the end- user”’.

The QoS approach is technical and often described as a hard measurement. QoE on the other hand, delivers a framework of how a user perceives the usabil- ity and quality of a service, i.e. user satisfaction [19]. As human experience is open minded and dynamic is has become important to recognize user perception quality.

15 3.2. USER-DRIVEN PERFORMANCE EVALUATION

Quality of Experience

Network coverage, Objective factors End-to-end service offers and as user network QoS expectations or level of support requirements

Underlying network infrastructure

Figure 3.1: The user can’t see the underlying network infrastructure, but ”feels” the overall service quality which highly equals to Quality of Experience [19]

3.2 User-Driven Performance Evaluation

A major challenge for wireless IP-based networks is to provide adequate QoS for different types of services. This requires a detailed knowledge of the performance requirements for the particular services and applications. The starting point for deriving these performance requirements must be the user. The measuring of hard QoS parameters is not enough for knowing user satisfaction. High throughput is meaningless when the end-user is unsatisfied with the service quality. Figure 3.1 presents a typical sketch of network quality, from a users’ point of view. It is possible to gain a greater use of new networks if subjective service evaluation can be performed when designing the networks. This gain includes a larger scope of available services that can be supported and accessed through the network with sufficient user satisfaction.

A typical user is not concerned with how a particular service is implemented. However, the user is interested in comparing the service over a wireless access with the ﬁxed broadband access. User driven performance evaluation should be expressed by parameters that: • Take into account all aspects of the service from the user’s point of view • Focus on user-perceivable eﬀects, rather than their causes within the network

• Independent of the speciﬁc network architecture or technology • Can be objectively or subjectively measured at the service access point • Can be easily related to network performance parameters, for data analysis [18]

16 3.3. KEY PERFORMANCE INDICATORS

Figure 3.2: Collection of network KPI:s can be made in all of the network architechture nodes or in the user equipment

3.3 Key Performance Indicators

QoS figures from a network can be collected either at system level or in the user equipment. At system level the data is collected from log files, sniffing tools or counters in network nodes or base stations. The other side is at the user equipment where network measurement software tools as protocol- or link sniffing are used during fixed- or mobile networks tests. Assembled data from user equipment or station logs serves as input to Key Performance Indicator (KPI) computations, where typical KPI:s are presented in table 3.3 and 3.3. A Network Management System (NMS) at a higher network level can also be used for KPI collection.

To be able to rate QoE of the users the relationship between QoS KPI and QoE KPIs has to be identiﬁed. KPI measurements are possible for both PS and CS services, although this section only considers PS based services. [18]

17 3.3. KEY PERFORMANCE INDICATORS

Quality of Experience is often based on human perception expressed with words like excellent, good, fair, poor and bad, instead of just in metrics. A combination of these two is also possible. There is a methodology where QoS performance parameters are mapped onto user perceptible QoE targets. This is a very challenging task and hard to do with acceptable accuracy.

User expectation comes in place when identifying QoE KPI:s and can be grouped into two main categories, namely reliability and comfort. Another classiﬁcation of QoE KPI:s are: accessibility, retainability and integrity. Table 3.1 and 3.2 outlines reliability and comfort when perceived user experience is evaluated.

Service availability Local and global coverage - Seamless usage Service accessibility Success rate of user connections Service access time Delay in setup time - Experienced response time by user Continuity of service The retainability of a service connection and its performance over time - Duration of a session or a call - Blocking and downtime of the system

Table 3.1: Reliability, availability, accessibility and maintenance KPI:s

Quality of Session Application layer packet loss Ratio of achieved and demanded bit rate Bearer stability, bit rate variation Average throughput (kbit/s) Average latency (ms) Average delay variation (ms) Ease of use Ease of service accessibility Level of support Quick user support

Table 3.2: Service comfort KPI:s [19]

18 3.3. KEY PERFORMANCE INDICATORS

Figure 3.3: Latency measurement from the user equipment to the server where the latency equals T1 + T2

3.3.1 Network delay Delay is a very important measurement and can be described in many ways, for example the time it takes to set up a service or to receive information during a service session. There are three types of delay, namely delay introduced from the user equipment, end node delay (i.e. server delay) and network delay, which further can be divided into core network delay and access network delay.

3.3.2 Latency Latency is measured from the time a request (e.g. a single packet) leaves the user to the time the response (e.g. an ACK) arrives back at the user from the serving entity, ﬁgure 3.3. It can also be measured in a one-way fashion, e.g. the time from the sending node to the receiving node. Latency is often refered to as round-trip time (RRT). [3]

3.3.3 Jitter Jitter or delay variation, is an important parameter in packet switched networks as the Internet. The packet inter-arrival times can diﬀer due to queuing delays in nodes along the route from the source to thedestination. Jitter at the application layer is removed by using buﬀers at the receiving end. In UDP based services delayed packets will be discarded causing periods with no data at the application level.

3.3.4 Packet Loss Another service parameter is packet loss, sometimes referred to as information loss and is caused by lost packets. It is obvious that packet loss has a direct impact on the perceived quality for the end user. If TCP (Transmission Control Protocol) is being used, lost packets are resent, but this is not the case when UDP (User Datagram Protocol) is used. [3]

3.3.5 Throughput Throughput, or data rate in the up- or downlink is the traﬃc to or from the user equipment. The data rate between two end points during a session is important when evaluating the service. To fully utilize a service, the data rate has to reach the minimum level of the service requirements, and with limited variations. A smaller variation in data rate is handled by input buﬀers in the application at

19 3.4. SERVICE PERFORMANCE REQUIREMENTS

Internet Radio Online Gaming Audio Codec Game start-up time Stream bit rate Map load time Buffering time Mean latency Re-buffering time Kills/Deaths Inter-buffer times Jitter Packet loss

Table 3.3: Service speciﬁc parameters for Internet Radio and online gaming the user equipment. Packet loss, jitter, latency and throughput are the four main parameters that inﬂuences users’ service perception the most.

3.4 Service Performance Requirements

3GPP together with the International Telecommunications Union (ITU) have stated requirements between communication entities for diﬀerent types of multimedia services. These parameters represent commonly accepted values for voice, video and data services as real-time games and are presented in table 3.4, for Internet Radio and online gaming is service parameters presented in table 3.3.5. Both QoS parameters QoE for a service are necessary when evaluating the service quality.

3.4.1 Voice The human ear is sensitive to quality variations and conversational voice requirements are heavily inﬂuenced by one-way delay and jitter. In the span between 150 and 400 ms a voice conversation can proceed with acceptable quality, although a value below 150 ms is preferable. Emerging IP-based services as VoIP will have to meet the quality and performance levels of the established PSTN, presumably with even better performance to reach market and user satisfaction. A typical ﬁgure for VoIP communications is packet loss is below 1%. Compared to CS networks where frame loss is typical below 2% with an end-to-end delay below 250 ms.

3.4.2 Videophone Videophone means a full-duplex communication where the traﬃc consists of both voice and video. In fact, the same requirements for voice will apply and the added video service must be within certain limits to provide suﬃcient quality. Due to human eye perception the videphone packet loss limit is around 1%.

20 3.4. SERVICE PERFORMANCE REQUIREMENTS

3.4.3 Online Gaming Online games represent the gaming genre when the game is played over a network connection usually over the Internet or just over a Local Access Network (LAN). With the growt of broadband accesses the demand on online games have been increasing and previously single player games have implemented network support. The market produces a variety of network games and they all have diﬀerent network requirements for their functionality.

Medium Application Data rate Latency Audio Voice service e.g. 4-25 kbit/s < 150 ms preferred Skype < 400 ms limit Video Videophone/Webcam 32-384 kbit/s < 150 ms preferred < 400 ms limit < 100 ms lip-synch Online gaming FPS e.g. < 60 kbit/s < 50 ms preferred CounterStrike < 150 limit RTS e.g. < 80 kbit/s < 150 ms preferred World of Warcraft < 450 limit

Table 3.4: 3GPP framework requirements for network based on voice and multimedia services. [20]

21 Chapter 4

Service Evaluation Cases

This chapter will introduce the services which have been evaluated in this thesis. A user connected to the Internet over a ﬁxed broadband access is nowadays using a range of network based services. Wireless radio access networks as GPRS/EDGE, WCDMA R99 or HSPA opens the possibility to extend the use of Internet services. This chapter describes the services and their speciﬁc characteristics, and chapter 5 contains the test evaluation.

Measurements of radio network performance include methods to supervise system and service quality and to ﬁnd reasons for low performance. Tradition- ally focus has been on speech and video quality for services like normal voice calls, video streaming and video telephony. Previously performed multimedia evaluations has got limited focus on online gaming, although it employs mil- lions of simultaneous users every day. Therefore this service evaluation puts more weight in the online gaming area.

Evaluation of Internet Radio was done by one user, namely the author himself, and the online gaming evaluation was made both by the author and a testing group of nine persons.

4.1 Internet Radio

Internet Radio is nowadays a popular service where listeners can tune in their favorite radio channel, located anywhere in the world with Internet broadcast- ing. There are channels dedicated to (almost) all existing music styles and some concentrated on news coverage and stand-up comedy. Internet Radio is not synonymous with ﬁle downloading because the media is streamed from a server and not stored at the user terminal.

Radio stations are chosen from a portal and a variety of bit rates are available, which gives the listener a hint of expected audio quality. One popular portal that hosts a range of Internet Radio stations is www.shoutcast.com with thousands of simultaneous listeners. An Internet Radio session is initialized when the user tunes-in a station, followed by a music or information ﬂow. [21], [22]

22 4.1. INTERNET RADIO

Figure 4.1: An Internet Radio stream modeled as a queuing system.

Since Internet Radio uses a buffer for the incoming data stream and is TCP- based, playback will continue until the buffer is empty and no packets are lost. Quality impairment is therefore gaps causing silence periods when the application is rebuffering. A condition with an empty buffer is the result of insufficient bandwidth or large amount of TCP retransmissions. Retransmissions can be triggered by bad radio environment interference by nearby cells, or bad propagation environment.

The buffer can be modeled as a queuing system with a traffic arrival rate, λ, buffer size, n and traffic departure rate, µ (play back stream rate). Serving discipline of the queue applies first come first served, i.e. in order of arrival. In the stationary case when the buffer is full, traffic arrival rate equals traffic departure rate (i.e. audio playback rate) regardless if the network can deliver higher data rates.

Inter arrival mean equals 1/λ and service mean is 1/µ. The parameter ρ is defined as traffic intensity and it is required that µ ρ = < 1 (4.1) λ since, otherwise, there will be an buffer overflow. [23]

Depending on TCP-connections and due to the nature of Internet Radio, quality degradation effects are silent periods when the application buffer runs empty. The duration, frequency and time between these gaps effect the service perception together with codec and stream bit rate of the audio session. Additional measurements as tune-in times and tune-in failure rates can also be monitored and can be seen as service quality parameters. The distance between the traffic arrival and playback curve corresponds to the amount of data in the client buffer and if the playback curve reaches the traffic arrival curve the media stream will be interrupted, figure 4.2. [24]

23 4.1. INTERNET RADIO

Traffic arrival

n io at er en Network delay Buffer g fic

Sequence number Sequence f ra T k ac yb la P

Time

Figure 4.2: Playback of an Internet Radio stream.

4.1.1 Audio Coding Audio coding-decoding system called codec constitutes the most critical element in the quality of a digital radio system. Due to the nature of the human audi- tory system and the psycho acoustical model, some audio components will not be perceived by the ear. Therefore codecs aim to code information that the ear will hear while maintaining high audio quality.

Some of todays’ available codecs are the MP2, MP3, AAC, AAC+ and HDC codecs. The MP2 and MP3 codecs were the first ones to compress audio bit rates to useful levels. The AAC/AAC+ codecs provide a far more efficient compression and outperforms both MP2 and MP3. Different encoding algorithms do have ”‘sweet spots”’ where they work best. At bit-rates much larger than this target bit-rate the audio quality improves only very slowly with bit-rate, at much lower bit-rates the quality decreases very fast. MP3 at 128 kbit/s provides equivalent AAC quality at 96 kbit/s where 192 kbit/s is needed for the MP2 codec. For most people an MP3 codec at 128 kbit/s constitute does sufficient quality with minor or no audio impairment. Internet Radio stations coded with the MP3 codec been utilized in all the test cases. Table 4.1 outlines a comparision of audio bit rates fot the MP3 codec. [25]

24 4.2. ONLINE GAMING

32 kbit/s Similar to analog AM quality 96 kbit/s Similar to analog FM quality 128 - 160 kbit/s Decent quality, diﬀerence can sometimes be obvious 128 - 192 kbit/s Often used for MP3 music ﬁles 192 kbit/s Very good quality 224 - 320 kbit/s High, near transparent, very good quality 1411 kbit/s WAV sound format of Compact Disc Digital Audio

Table 4.1: Comparsion of audio bit rates for the MP3 codec. [26]

4.2 Online Gaming

The growth and penetration of broadband access networks to the home has fueled the growth of online games played over the Internet. Gaming servers reports that the number of online players is steadily growing. There are four main network parameters that aﬀect a user’s perception of a multiplayer online game: latency, jitter, packet loss and throughput.

Game genres are divided by how the user interacts with the game and how the game world is being viewed. Two factors, interaction and perspective, helps the classiﬁcation that determines latency impacts on online games. Gamers can also be classiﬁed. Those who spend a lot of their leisure time and money into gaming, are often called Hardcore gamers. They have higher demands on their network access and user equipment. Leisure gamers are people who spend less time and money into their hobby, and their demands are not as high as the hardcore gamers.

The online gaming part of this thesis considers two games, CounterStrike and World of Warcraft.

4.2.1 FPS The ﬁrst person shooter (FPS) genre, with games as CounterStrike (CS), Quake and Unreal has become extremely popular in recent years. The game is viewed through the eyes of an avatar in the game itself, creating a ﬁrst person perspective. An avatar is the player’s physical representation as a three-dimensional model. FPS games are highly interactive and require quick hand-eye coordi- nation in moving the cross-hair towards an opponent, forcing players to make split-second decisions.

CS is played online and consists of many users in a highly dynamic environment. Players are grouped together in teams (called clans) and the goal is to complete a speciﬁc task, for example to defeat the opposite team. Communi- cation between the team members is done via an Internet Relay Chat (IRC), a many-to-many communication protocol or a voice service implemented inside the program. FPS relies on fast user responses and is therefore very sensitive for delays. It is today’s most network demanding game regarding latency, jitter and packet loss. The amount of data being transmitted depends on number of players active in each session.

25 4.2. ONLINE GAMING

It is obvious that the game requirements are diﬀering depending on the game and player expectations. A very experienced player may experience gaming impairments more annoying than a less experienced player. Nevertheless, performance limits for diﬀerent games can also vary during game play. In FPS games periods of close combat puts high requirements in low delay while in other periods, for example during movement, the same delay impairment is not as crucial. [27]

4.2.2 RTS Real Time Strategy (RTS) requires slightly lower user response time and the game play is less reaction based. RTS perspective is often from an overview position, giving the player freedom to supervise large areas at the same time. Players interact with a handful of other players during a game session and their contribution to the game is often greater than in a FPS game. Players manages their resources and struggles to harvest goods, build buildings and train armys. Finally players challenge their opponents in battle. RTS games results in longer playing sessions and leads to an unwillingness to leave during the game session. Example of RTS games are Command and Conquer, Star Craft and the precursor game Dune II.

4.2.3 MMORPG Massive Multiplayer Online Role Playing Games (MMORPG) like World of Warcraft (WoW) and Guild Wars, is a multi-user online game where players interact with one another in a virtual world. A subscription is needed to play and the game consists of highly variable environments with thousands of players online at the same time. An avatar is created when ﬁrst entering the game and the character can belong to diﬀerent character classes and races. In the game the player receives quests from computer-controlled players (also called Non Player Characters) and receives experience, money and items as rewards for completed quests. [28]

Players can join groups and complete quests together. Conversation is done via a built in chat or through an external voice program. Gained experience increases the players skills and opens up new levels with increased diﬃculty. MMORPG games have compared to RTS games both tighter delay restrictions and higher data rates. This is needed due to the higher number of concurrently online players. But the MMORPG is played at a slower pace compared to a FPS game. [20], [29]

26 4.2. ONLINE GAMING

4.2.4 Game Phases Most multiplayer online games utilizes a client-server architechture where game- state information is shared among connected users. Game state information is supposed to be as close to real-time as possible and player updates are sent in regular fashion to the server. Player updates are actions as walking or shooting. Generated network traﬃc alters between roughly four phases during game-play. These phases aﬀects player interaction, although the duration and frequency of these phases varies. Depending on the game played most online games consist of:

• Setup - In this phase, the game hosts is waiting for other player to join the game. Players are also choosing game parameters as team, maps, character outfit and other game specific options. A typical player often makes only a few game setting, resulting in limited network traffic from the client to the game server. This phase is not latency sensitive but depending on high throughput. • Synchronization - The synchronization phase is just before the actually game starts. Here the choices made are by the players in the setup phase synchronized exchanged between game host and active players. Game specific information like as downloads of maps, are also done in this phase. Users don’t interact with the game and this phase is often characterized by high bit rates to get the game to started as soon as possible. Latency is not an issue here. • Game play - During this phase the game is played and player interactions are communicated to the game server. A FPS combat game communicates the movement of an avatar and firing of a weapon towards other avatars. The game play phase has generally moderately bit rates but exchanges small packets of information in a frequent manner aiming to keep latency at a minimum. During this phase latency, jitter and packet loss are major performance degrading parameters and effects player actions and the perceived gaming experience. • Transition - When a user moves between for example two worlds new information as maps or attributes have to be loaded. During this phase network traffic is limited and the user is not active and due to most of the information is extracted from a local disk. [27], [2]

4.2.5 Traffic types The network traffic varies depending on which phase (as described above) the player currently is in. During the setup and synchronization phases where information loss is crucial TCP is used for most of the traffic. During game play when player updates is sent to the server most traffic flows over UDP, where information loss is resulting in sudden game freezing or other impairments which reduces the game experience.

Most games are designed with low bit rate requirements, instead sending frequent but small update packets to the server. It reduces the eﬀects of sudden

27 4.3. TEST SETUP

CounterStrike − Downlink throughput − WCDMA R99 400

SYNCHRONIZATION 350

300

250

SETUP TRANSITION GAMEPLAY 200 kbit/s

150

100

0 11:45:45 11:46:19 11:46:52 11:47:26 11:48:00 11:48:33 11:49:07 11:49:40 time [HH:MM:SS]

Figure 4.3: Throughput downlink for the gamephases in CounterStrike measured with TEMS Investigation. packet loss and can in some cases be handled by repair techniques. One popular loss repair technique is called ”Dead reckoning”. It is a process that estimates one’s current position based upon previously determined positions. More detailed information can be founf in [3]. Impairments during game play are often referred to as ”lags” and some common problems are listed below. [4].

• Game freezing, the screen freezes for a random time because of a major packet loss and is often followed by a sudden update, called warping. • Warping, an update from the client to the server, or vice versa, is lost. This loss results in a hasty re-location of the user to the last known client position. • Delayed responses, due to high network latency, a user can suﬀer from a delayed response. A user action will not take place as the user intended.

4.3 Test setup

The coming sections presents an outline of the test setup and how the testing activities were performed. The evaluation cases for each service have been performed using a test protocol with a set of variables for each test case. The perceived user experiences were graded with an quality scale ranging from ”bad” to ”excellent”. A Mean Opinion Score (MOS) was the calculated. MOS provides a numerical indication of the perceived quality of received media after compression and/or transmission. The MOS is expressed as a single number in the range 1 to 5, where 1 is lowest perceived quality (bad), and 5 is the highest perceived quality (excellent). Table 4.2 presents the MOS scale. Finally

28 4.3. TEST SETUP the perceived user experiences were commented in more vivid terms to evaluate session.

MOS Quality Impairment 5 Excellent Imperceptible 4 Good Perceptible but not annoying 3 Fair Slightly annoying 2 Poor Annoying 1 Bad Very annoying

Table 4.2: Mean Opinion Score.

4.3.1 Hardware Testing activities were made in both an operators network (Telia) and at Erics- son’s test networks in Stockholm (STN) and Ursviken (UTN). A Sierra Wireless AIR-Card 850 with possibilities to connect to GPRS/EDGE, WCDMA R99 and HSDPA networks was used for the operator network tests. HSPA tests were done by using a EUL capable Qualcomm TM7200 mobile at the test networks in STN and UTN. The GPRS/EDGE and WCDMA R99 tests were performed at a variety of places in the area around Porsön,Lule˚a.To evoke varying (and sometimes bad) radio conditions drive tests were performed for GPRS/EDGE, WCDMA and HSPA networks. Broadband reference tests were performed at the office over a fixed broadband connection.

Table 4.3 lists the speciﬁcations of the laptop being used. Due to latency and throughput limitations in GPRS/EGDE access networks, these two radio access techniques are only used for Internet Radio evaluation and not considered when evaluating FPS online games.

Equipment Speciﬁcation Brand Dell Latitude D610 CPU Intel Pentium M 1,6GHz Video card Intel Graphics Media Accelerator 128MB RAM memory 1GB RAM Hard drive 60GB - 5400 rpm Operating system Windows XP Professional SP2 Screen 14.1 inch TFT active matrix 1400 x 1050

Table 4.3: Speciﬁcations of the user terminal.

4.3.2 Network load The network load in WCDMA R99 networks in an around Lule˚ais at a consis- tently low level and the tests have not been affected by concurrent traffic. At STN and UTN experiments performed were with no other, or limited concurrent network traffic.

29 4.3. TEST SETUP

4.3.3 Software Logging software running in parallel with the service being evaluated enabled intense RLC- and IP-level statistics for test analysis. These statistics served as input for later performance analysis of the measured parameters. In particular Ericsson’s optimization tool TEMS Investigation was used for logging of radio network statistics in GPRS/EDGE, WCMDA R99 and HSPA networks. Further, Wireshark (former Ethereal) packet sniffer was used for logging of IP traffic statistics. For measuring latency times PingPlotter, a simple ping tool was used. PingPlotter uses ICMP (Internet Control Message Protocol) packets for measuring latency times in the network, which generates some limitied extra network traffic. Figure 4.4 outlines data collection and analysis.

Core Network

User Equipment Application

Wireshark TEMS PingPlotter

logs

PERL

Matlab

Figure 4.4: Sequence picture for data collection and analysis

30 4.4. DATA POST-PROCESSING

4.4 Data Post-Processing

Three different log files were created from Wireshark, TEMS Investigation and the PingPlotter software. Log files were parsed with perl script and .m files were created for later analysis in Matlab. One important issue was to analyze graphs in system time to be able to corrolate time events between different log files. Figure 4.4 outlines information extraction flow.

4.4.1 CounterStrike Gaming Experience Nine players with diﬀerent gaming background and skill level played the game CounterStrike. The players ﬁrst estimated their playing skill and amount of playing hours and were then divided into three separate groups: ”‘novice”’, ”‘intermediate” and ”‘skilled”’. Each playing session lasted 10 minutes. Due to changing playing conditions on game servers the goal was to play as similar sessions as possible for all the players and test cases. The two utilized servers are listed in table 4.4.

Game map is the visual appearence where the playing session was performed. At game servers there are a a range of maps available. The map was ﬁrst played over a broadband access as a reference; afterwards the same map was played over a WCDMA R99 connection. Each session lasted 10 minutes. The goal was to kill as many of the other players as possible, and at the same time avoid to get killed by the other players. If the player itself got killed, he could re- enter the game after a short while and continue playing. The quality grading scale in table 4.2 were explained to facilitate the service session grading before session started. Additional grading was with a the Stay/Leave scale. The scale faciliates the willingness to stay or leave the game due to the perceived quality, where grade 1 means that the user rather quits playing than to continue the session. Each playing session ended with an interview to evaluate the players gaming experience. The players the answered a questionnaire on the perceived service experience where the questions are listed in Appendix I together with raw data from the tests.

Server IP-addr Map SWE-BiH #1 213.114.234.192 de dust2 cz CZ1.2 #2 211.98.162.100 de dust2 cz

Table 4.4: Utilized CounterStrike servers and gaming map where test sessions were performed.

31 Chapter 5

Test Results and Analysis

This chapter presents test results from the evaluated services Internet Radio and online gaming (CounterStrike and World of Warcraft). Each test section is concluded with a short service evaluation. Additional conclutions are found in chapter 7.

5.1 Internet Radio

1400

4 1200

1000

3 800 MOS score Latency [ms]

600 2

400

200 1 16:54:54 16:56:12 16:57:30 16:58:48 17:00:05 17:01:23 48 64 96 128 196 320 time [HH:MM:SS] Bitrate (kbit/s)

Figure 5.1: Latency for a 128 kbit/s radio stream over a WCDMA R99 bearer (left) and mean MOS for bitrates from 48 to 320 kbit/s for GRPS/EGDE, WCDMA R99 and HSPA tests (right). For MP3 bitrates below 64 kbit/s is played in mono, while higher bitrates is played in stereo.

Tests made by the author himself have shown that Internet Radio is strictly depending on the access network throughput limitation. The service is based on TCP and jitter from the streaming server is handled by input buffers. This makes the service insensitive to high latency times. Application buffer times are set to WinAmp default. Figure 5.1 shows latency times during a 128 kbit/s In- ternet Radio session over WCDMA R99 and MOS for different stream bit rates. High latency variations did not cause any degradation of the session quality.

32 5.1. INTERNET RADIO

20 TP DL (kbit/s) 10

0 19:28:53 19:29:08 19:29:23 19:29:38 19:29:53 19:30:08 19:30:23 time[HH:MM:SS]

−50 rx−lev neigh−rx−lev(1) −60 neigh−rx−lev(2)

−70 dB

−80

−90

19:28:52 19:29:06 19:29:19 19:29:33 19:29:46 19:30:00 19:30:13 19:30:27 time[HH:MM:SS]

Figure 5.2: Decreased throughput in the downlink (above) caused by a severe radio environment impairment (below) in a GRRS/EDGE network.

Playback interruptions occur when the playback buffer is empty. This hap- pens when the radio bearer can’t provide sufficient throughput to cover the stream bit rate. Interruptions can also occur during packet retransmissions or when packets suffer major delay from the streaming server. Reduced throughput does not influence the playback stream as long as the playback buffer is not empty. Packet distribution for different radio access networks are shown in figure 5.3.

The time to start a Internet Radio session includes the time to browse web- page’s for available channels. Limited throughput increases the start up time rapidly, degrading the user experience as a consequence. As described in chapter 4 most channels are available at webpages (www.shoutcast.com) or via the application GUI.

When encountering bad radio environment with decreased signal strength and increased interference a consequence is reduced throughput as shown in ﬁg- ure 5.2. But a prolonged period with bad radio environment causes interruptions and a degradation of service experience.

33 5.1. INTERNET RADIO

GPRS < 48 kbit/s EGDE < 96 kbit/s WCDMA R99 < 320 kbit/s HSPA 1 < 320 kbit/s

Table 5.1: Maximal bitrates per radio access network.

5.1.1 Service Functionality A good experience can be provided over WCDMA R99 in urban environments but suﬀer from restricted coverage in sparsely populated areas. GPRS/EDGE provides better and more wide-spread coverage but has on the other hand restrictions maximal stream bit rate, which inﬂuences the service- and audio quality. The bitrate limitations showed in table 5.1 outlines bitrates for maximal user experience per radio access.

Table 5.2 consists of correlation reference values between two independent variables. Not surprising, the stream bit rate is strongly correlated with the subjective MOS as shown in table 5.3. An Internet Radio stream at 48 kbit/s appre- hends unpleasant and suﬀers from clear digital codec impairments. Therefore it is unusual with media streams at this rate.

Negative Positive Small -0.29 to -0.10 0.10 to 0.29 Medium -0.49 to -0.30 0.30 to 0.49 Large -1.00 to -0.50 0.50 to 1.00

Table 5.2: Correlation reference matrix for two independent variables. Used as a framework for correlation between measured variables.

MOS Bit rate MOS 1.000 0.728 Bit rate 0.728 1.000

Table 5.3: Correlation between MOS and stream bit rate from the Internet Radio tests (grading were done by one person).

In good radio conditions and when the user is standing still GPRS/EDGE delivers a good experience over a 64 kbit/s stream. If the user is moving he suffers from clear quality degradation when using GPRS/EDGE for a 64 kbit/s stream. EDGE provides at 96 kbit/s sufficient throughput for a good experience without interruptions and rebufferings, but a handover to GPRS will spoil the session.

For WCDMA R99 and HSPA are bitrates up to 320 kbit/s tested and throughout good quality has been delivered. Audio quality is very good at bitrates as high as 320 kbit/s.

1Streams over 320 kbit/s is unusual although HSPA manages higher bit rates.

34 5.1. INTERNET RADIO

12000 1

10000 0.8 8000 0.6 6000 distr 0.4

# packets 4000 2000 0.2 0 0 0 500 1000 1500 0 500 1000 1500 GPRS/EDGE GPRS/EDGE

12000 1

10000 0.8 8000 0.6 6000 distr 0.4

# packets 4000 2000 0.2 0 0 0 500 1000 1500 0 500 1000 1500 WCDMA R99 WCDMA R99

12000 1

10000 0.8 8000 0.6 6000 distr 0.4

# packets 4000 2000 0.2 0 0 0 500 1000 1500 0 500 1000 1500 HSPA HSPA

12000 1

10000 0.8 8000 0.6 6000 distr 0.4

# packets 4000 2000 0.2 0 0 0 500 1000 1500 0 500 1000 1500 Broadband Broadband

Figure 5.3: Internet Radio - Packet sizes histogram and packet distribution for GRPS/EGDE, WCDMA R99, HSPA and Broadband. The diﬀerence in the GPRS/EDGE packet distribution is due to lower stream bitrate.

35 5.2. ONLINE GAMING, FPS: COUNTERSTRIKE

5.2 Online Gaming, FPS: CounterStrike

CounterStrike has been evaluated over WCDMA R99, HSPA and fixed broadband. As described in the previous chapter, a test session with 9 players took place where the game was evaluated over WCDMA R99 and fixed broadband. The tests have secured the fact that CounterStrike is a very network demanding game. Clear game phases have been identified, seen in figure 4.3. For each game phase different parameters are important. During the start up phase the most important is high throughput. During gameplay low levels of latency, jitter and packet loss are most important. There is no (or limited) user interaction during the start up phase and the network traffic flows over TCP to ensure no information loss.

60 Throughput downlink Throughput uplink

30 kbit/s

0 13:12:13 13:12:26 13:12:39 13:12:52 13:13:05 13:13:19 13:13:32 time [HH:MM:SS]

Figure 5.4: CounterStrike throughput during gameplay

Figure 5.5 shows start up times for broadband, HSPA and WCDMA R99. A radio access network gives a much higher variation in start up times depending on both limited throughput and radio environment (see ﬁgure 5.4. Data for map load times has been collected by time measurements during the test sessions. Figure 5.5 shows high variance in map load times and almost 4 times higher mean for WCDMA R99 than for a normal broadband session. Table 5.4 lists mean throughput and packet sizes during a CS session.

Uplink Downlink Throughput 18.25 kbit/s 72.45 kbit/s Packet size 51.04 bytes 788.71 bytes

Table 5.4: Mean throughput and packet sizes for a CounterStrike session over HSPA

36 5.2. ONLINE GAMING, FPS: COUNTERSTRIKE

250

200

150

Map load [s] 100

0 Broadband HSPA WCDMA R99

Figure 5.5: Map load times and 95% conﬁdence interval for CounterStrike over three evaluated radio access networks.

5.2.1 Service Functionality The tests have been performed at public CounterStrike game servers on the In- ternet (see table 4.4 in chapter 4). Most of these servers don’t allow players to participate if their Internet connection can’t deliver a steady low latency, typical below 100 ms with low or no jitter. This causes problem when connected over a radio access network as WCDMA R99 or HSPA. Both latency and the level of

350 5

300

4 250

200

3 MOS 150 Latency [ms]

100 2

0 1 Broadband HSPA WCDMA R99 Broadband HSPA WCDMA Access network Access network

Figure 5.6: 95% conﬁdence interval´s for Latency (left) and Subjective MOS (right) for three access networks. HSPA tests were performed with fewer tests (5) than the other two. jitter are clearly higher when connected over a radio bearer than over a broadband connection. Jitter for WCDMA R99 reaches +/- 150 ms and for HSPA +/- 40 ms. This implicates that to complete a test session the game server

37 5.2. ONLINE GAMING, FPS: COUNTERSTRIKE have to allow players with high or varying latency levels. Figure 5.6 points out latency and MOS for three access networks and 95% conﬁdence intervals.

Figure 5.4 shows logged throughput uplink and downlink during a Coun- terStrike session. The graph represents typical throughput measurements when connected to a game server with 10-15 active players. More active players in a session aﬀect the throughput, although values above 120 kbit/s in the downlink are unusual. With these quite moderate bit rates a radio access as WCDMA R99 delivers enough throughput. But as stated before, the latency requirements can’t be fulﬁlled.

A positive experience is when playing over HSPA where a gaming session could be performed with no or limited service impairments. It is worth noting that HSPA test networks are equipped with EUL which greatly contribute to the overall quality. But for ”‘hardcore”’ gamer this network alternative still falls out of hand due to high latency and jitter levels.

38 5.3. ONLINE GAMING, MMORPG: WORLD OF WARCRAFT

5.3 Online Gaming, MMORPG: World of War- craft

280 900 5

260 800 4.5

240 700 4 220

600 3.5 200

180 500 3 MOS Latency [ms]

160 Latency [ms] 400 2.5

140 300 2 120

200 1.5 100

80 100 1 15:56:09 15:57:10 15:58:10 15:59:11 16:00:11 16:01:12 16:02:12 16:03:12 16:04:13 16:05:13 16:06:14 HSPA WCDMA R99 GPRS/EDGE time [HH:MM:SS]

Figure 5.7: Latency log during a World of Warcraft session over HSPA (left) and subjective MOS and mean latency per access network.

World of Warcraft is less network sensible than CS but puts on the other hand higher demands on the terminal hardware. This game has, compared with CounterStrike, moderate latency and bit rate requirements during gameplay. Table 5.5 presents acceptable latency intervals and ﬁgure 5.7 typical latency variation when connected over HSPA, which clearly lies in the acceptable span.

No clear game phases can be distinguished and ﬁgure 5.8 represents throughput during a normal game session. Compared with CounterStrike where game information have to be downloaded before the game session could start, all information is here locally stored at the user equipment (i.e. computer). During game play only player updates are sent to and from the game server, making the mean bit rate as low as described in table 5.6.

Game state updates are encapsulated in small packets, and the stream is fairly symmetrical in the up- and downlink. This behavior is unusual for a network game where often more traﬃc generated in the downlink than in the uplink. Figure 5.7 plots latency, with a mean of 100ms during a World of Warcraft session over HSPA. During the game session no quality impairments were noticed, although latency spikes arise at some points. Table 3.4 from section 3.4, states the 3GPP framework for multimedia services. Their data rate requirements for RTS and/or MMORPG games are clearly higher than results produced from this and other [3] World of Warcraft evaluations. Figure 5.9 shows packet distribution for HSPA and WCDMA R99 test sessions.

39 5.3. ONLINE GAMING, MMORPG: WORLD OF WARCRAFT

Latency [ms] 0-400 Acceptable 401-600 Fair > 601 Bad

Table 5.5: Acceptable latency intervals in World of Warcraft. [30]

Uplink Downlink Throughput 17.68 kbit/s 18.43 kbit/s Packet size 70.06 bytes 85.92 bytes

Table 5.6: Mean throughput and packet sizes for a World of Warcraft session over HSPA.

5.3.1 Service Functionality World of Warcraft quality assessment was done by only one person. Tests have shown that HSPA delivers quality in resemblance with a fixed broadband connection. Before the game can be started updates or patches have to be downloaded. Game patching is done continuously to fix bugs and to add new features to the game. These patches vary in size, but it is normal with file sizes from 200 to 600 MB. When connected over a radio access, such as GPRS/EDGE or WCDMA R99, this patching process should be very time consuming and contribute to great quality degradation. Tests have shown that shorter World of Warcraft playing session is viable over WCDMA R99, but in a longer horizon WCDMA R99 is not an acceptable alternative. The use of GRPS/EDGE as access network is not preferable in any way due to clear service impairments.

40 5.3. ONLINE GAMING, MMORPG: WORLD OF WARCRAFT

Throughput downlink 60 Throughput uplink

kbit/s 30

0 15:56:31 15:57:10 15:57:48 15:58:27 15:59:06 15:59:45 16:00:24 time [HH:MM:SS]

Figure 5.8: Throughput during a World of Warcraft session.

1 1200 0.9

0.8 1000 0.7

800 0.6

0.5 600 distr # packets 0.4

400 0.3

0.2 200 0.1

0 0 0 200 400 600 800 1000 1200 1400 1600 0 500 1000 1500 WCDMA R99 WCDMA R99

1600 0.9

0.8 1400 0.7 1200 0.6 1000 0.5 800 distr # packets 0.4 600 0.3 400 0.2

200 0.1

0 0 0 200 400 600 800 1000 1200 1400 1600 0 500 1.000 1.500 HSPA HSPA

Figure 5.9: Packet sizes and cumulative distribution function for WCDMA R99 and HSPA access networks

41 Chapter 6

Online Gaming Quality Model: FPS

The development of objective perceptual quality models provide the telecommunication industry with valuable test and measurement tools for audio and multimedia services. Quality models removes the need to perform expensive and time- consuming subjective tests. Focus for perception quality modeling have previously been on multimedia services as audio and video quality. This chapter presents two quality models for the FPS game CounterStrike, one linear and one non- linear. At the end of the chapter an extended quality model with more input variables is presented. [31]

The study has shown that most online games have utmost limited throughput restrictions throughout the game. Radio access networks as WCDMA R99 and HSPA both fulﬁll the throughput requirements, thus are, the limitation factors latency, jitter and packet loss. The model focus’s on FPS games when they are clearly the most demanding on the above mentioned parameters. For FPS online games have also most subjective data been collected for model design and veriﬁcation.

6.1 Principal Components Analysis

For each test case a number of variables have been produced. Fortunately, in data sets with many variables, groups of variables often move together. One reason is that more than one variable might be measuring the same driving prin- ciple for the behavior of the observed system. To narrow the dimensions down to just a few variables, Principal Components Analysis (PCA) where made in Matlab and additional variable analysis in SIMCA. PCA analysis simpliﬁes the data set by creating new variables, called principal components. The principal components as a whole form an orthogonal basis for the data set. [32].

The variable analysis is intended to sort out the variables that inﬂuences the subjective MOS the most. Packet loss and jitter greatly inﬂuences the service quality, but weren’t measurable in this thesis due to the available testing equipment. An extended model is presented at the end of this section where

42 6.1. PRINCIPAL COMPONENTS ANALYSIS

Figure 6.1: A quality model sketch for online gaming. additional input parameters as jitter and packet loss have been introduced to the model.

• Latency [ms] - During gameplay this variable, together with jitter, causes the most obvious impairments and steers the quality perception more than other measurable variables. • Map load time [s] - The map load time is closely associated with throughput of the Internet connection. An ability to deliver suﬃcient throughput to download information and start the game session after the initial synchronization phase. The rest of the session has game dependent throughput limitations, for CounterStrike < 120 kbit/s and World of Warcraft < 45 kbit/s. • Packet loss [%] - If packet loss reaches or exceeds 3 % the service quality is notably decreased, and no connection is established if it exceeds 10 %.

• Jitter [ms] - The impact of jitter (delay variation) is game dependent. Jitter causes, according to [33] signiﬁcant perception impairment for Coun- terStrike when exceeding 150 ms.

Input parameters from the gaming tests where ﬁrst reduced to these variables who serve as input to the latter PCA analysis: Modelvariables = Map Load, Latency, MOS, Stay/Leave

43 6.1. PRINCIPAL COMPONENTS ANALYSIS

Map Load Latency MOS Stay/Leave Map Load 1.0000 0.3373 -0.4721 -0.4923 Latency 0.3373 1.0000 -0.6653 -0.7068 MOS -0.4721 -0.6653 1.0000 0.8204 Stay/Leave -0.4923 -0.7068 0.8204 1.0000

Table 6.1: Correlation matrix for the Modelvariables

As expected, the correlation between MOS and Stay/Leave is high seen in table 6.1. Table 5.2 from chapter 5 presents reference correlation values. High correlation can be seen between Latency and MOS (and Stay/Leave). The Map Load variable has a moderate correlation to the other parameters, except to Latency where the correlation is quite low.

90 90%

80 80%

70 70%

60 60%

50 50%

40 40%

Variance Explained (%) 30 30%

20 20%

10 10%

0 0% 1 2 Principal Component

Figure 6.2: Principal Components Analysis from FPS game CounterStrike

44 6.1. PRINCIPAL COMPONENTS ANALYSIS

5 5

4 4

3 3 MOS MOS

2 2

1 1 0 100 200 300 400 500 0 50 100 150 200 250 Latency [ms] Map load [s]

Figure 6.3: MOS for Latency (left) and Map Load (right) in 95 % conﬁdence intervals plotted at the mean value of each span.

PCA1 PCA2 PCA3 PCA4 Map Load 0.0903 0.9958 -0.0111 -0.0007 Latency 0.9959 -0.0904 -0.0059 -0.0005 MOS -0.0047 -0.0074 -0.7436 0.6685 Stay/Leave -0.0050 -0.0076 -0.6685 -0.7437

Table 6.2: Coordinates for the four principal components.

Figure 6.3 shows MOS grouped together in five spans, [0-50,51-100,101- 150,151-250,>250] for latency and [0-20,21-40,41-60, 61-100,>101] for map load. This opens up the possibility to calculate confidence intervals for MOS depending on latency and map load times. Otherwise no confidence interval could be calculated due to scattered values in the data set.

Each column in table 6.2 represent four principal components, where the coordinates in the four dimensional orthogonal space spans by the principal components. Principal component 1 puts high weight on latency (0.9959), and principal component 2 is considerable build up by map load (0.9958). Figure 6.2 shows the level of signiﬁcance, where principal component 1 and 2 satisfy almost 100% of the analysis signiﬁcance and components 3 and 4 don’t declare any considerable contribution.

45 6.2. QUALITY MODEL

6.2 Quality Model

Figure 6.1 from the beginning of the chapter outlines important input variables to a Gaming Quality Index (GQI) model. This chapter presents two GQIMOS models, one a linear and one more complex non-linear model. The GQI quality models estimates a predicted MOS from the input function whom depend on the variables xlatency and ymapload. The parameter GQIBASE (which is set to 3.9) steers the model initial value, i.e. maximal model output if no degradation from xlatency and ymapload occurs. GQIMIN adapts output to MOS, where minimal MOS value is 1. The Q-functions does only exist in the span [0, 1]. Due to absence of variables jitter and packet loss the linear GQIMOS,1 model is formulated as: GQIMOS,1 ∝ f(xlatency, ymapload) =

GQIMIN + GQIBASE ∗ (Qxlatency ) ∗ (Qymapload ) (6.1)

where

Qxlatency = −0.0053 ∗ xlatency + 1.2 (6.2)

Qymapload = −0.0075 ∗ ymapload + 1.4 (6.3)

A linear model as GQIMOS,1 in ﬁgure 6.4 demands boundary conditions to

prevent MOS predictions outside the scale. For Qxlatency are values below 40

and above 225 ms are set to model max respective min value. Qymapload have boundary conditions of below 55 and over 180 seconds to ensure an authorized MOS value.

5 5

4 4

3 3 MOS MOS

2 2

1 1 0 50 100 150 200 250 300 20 40 60 80 100 120 140 160 180 200 Latency [ms] Map Load [s]

Figure 6.4: Linear regression curves for Latency (left) and Map Load (right) with boundary conditions.

46 6.2. QUALITY MODEL

Figure 6.5: Illustration of non-linear GQIMOS,2 prediction outputs.

The non-linear GQIMOS,2 model in ﬁgure 6.5 is formulated as:

GQIMOS,2 ∝ f(xlatency, ymapload) =

GQIMIN + GQIBASE ∗ (Qxlatency ) ∗ (Qymapload ) (6.4) where 1 Qxlatency = ( 3 ) (6.5) 1 + (α ∗ xlatency)

−1 Qymapload = 0.6 − β ∗ atan(γ ∗ (ymapload − TH)) (6.6)

Where constants α, β and γ are 110, 2.3 and 165 respectively. The two Q degradation functions exists only within the span [0,1], and no bounding conditions are needed. This model has a more smooth behavior than the linear model and is more suited to the data set. Due to measured patience from the user’s when entering the game, the threshold parameter TH equals 220 and steers the quality degradation in the Qymapload function together with β and γ.

47 6.2. QUALITY MODEL

6.2.1 Performance Calculation Methods This section will give an insight to mathematical calculation methods used when evaluating model performance. The Student’s t-distribution render possible for an statistical approach when the sample sizes are as small as this study has produced. By using the Student’s t-distribution can conﬁdence intervals for the data samples be calculated. Suppose the sample set has X1, ..., Xn independent random variables, who are normally distributed with expected mean µ and variance σ2. The sample mean of this set is calculated:

X¯n = (X1 + ... + Xn)/n (6.7) and the sample variance

n 1 X S 2 = (X − X¯ )2 (6.8) n (n − 1) i n i=1

William Gosset formulated 1908 [34] the related quantity

X¯ − µ T = n √ (6.9) σ/ n where the probability density function for the t-distribution can be stated

Γ((ν + 1)/2) −(ν+1)/2 f(t) = √ (1 + t2/ν) (6.10) νπΓ(ν/2)

Here α controls the accuracy of the conﬁdence interval and T has a t-distribution with n-1 degrees of freedom

X¯ − µ P r(−A < T < A) = α ⇔ P r(−A < n √ < A) = α (6.11) Sn/ n Finally, with appropriate α value the conﬁdence interval endpoints can be calculated S S [X¯ − A √n , X¯ + A √n ] (6.12) n n n n

The root mean square error (RMSE) is a quality measure of the error between all the model prediction values and the subjective MOS. q q ˆ ˆ ˆ 2 RMSE(θ1, θ2) = MSE(θ1, θ2) = E((θ1 − θ2) ) (6.13)

ˆ where θ1 is the subjective MOS and θ2 is the predicted MOS from the quality model.     x1,1 xˆ2,1 x1,2  xˆ2,2  θ =   , θˆ =   (6.14) 1  .  2  .   .   .  x1,n xˆ2,n

48 6.2. QUALITY MODEL

Then the RMSE error is calculated: s Pi=1(x − x )2 RMSE(θ , θˆ ) = n 1,i 2,i (6.15) 1 2 n

Residuals are each error between subjective MOS and the predicted value for ˆ each point in the data set, θR = θ1 - θ2. Together with RMSE the residuals of the regression is an important quality measure. In the telecommunication industry, an acceptable RMSE value for quality prediction models is in the range 0.2 - 0.3.

When sample sizes of residuals are small (<50) a histogram is not the best choice for judging the distribution of residuals. A more sensitive graph is the normal probability plot. By ﬁrst sorting the residuals in ascending order and plot versus the cumulative probability, who is calculated as: P(i-th residual) = i(N + 1), where P denotes the cumulative probability of a point, i is the value in the residual list and N is the number of residuals. The normal probability plot should produce an approximately straight line if the residuals originate from a normal distribution.

In probability theory and statistics, correlation, also known as correlation coeﬃcient, indicates the strength and direction of a linear relationship between two random variables. In general statistical usage, correlation or co-relation refers to the departure of two variables from independence. The correlation ρ between two random variables X and Y with expected mean µX and µY and standard deviations σX and σY is calculated as

cov(X,Y) E((X − σX)(Y − σY)) ρX,Y = = (6.16) σXσY σXσY Where E is the expected value and cov stands for covariance, calculated as 2 2 2 stated in the numerator. Since µX = E(X) and µX = E(X )- E (X) and same for Y, ρX,Y can be stated

E(X,Y ) − E(X)E(Y ) ρX,Y = (6.17) pE(X2) − E2(X)pE(Y 2) − E2(Y )

Table 5.2 in chapter 5 contains reference correlation values. Correlation near 1 means a linear increasing relationship and -1 means a linear decreasing relationship. Strong correlation between two variables X and Y means values near +/- 1. [35]

49 6.2. QUALITY MODEL

6.2.2 Model Performance

The linear GQIMOS,1 model performance metrics presents an RMSE value of 0.677 and a corresponding correlation between subjective MOS and the GQIMOS,1 of 0.8678. Figure 6.6 points out 81.1% prediction accuracy within 95% MOS conﬁdence interval. Corresponding ﬁgures for the non-linear GQIMOS,2 model are: RMSE value of 0.609, correlation of 0.8803 and a prediction accuracy of 91.9%. Both models produces a high correlation value between predicted and subjective MOS.

5 5

4 4

3 3 MOS MOS

2 2

1 1 0 100 200 300 400 500 0 100 200 300 400 500 Latency spans [ms] Latency spans [ms]

Figure 6.6: Model prediction outputs (♦) for the linear model (left) and the non-linear model (right) plotted in 95% conﬁdence interval.

5 5

4 4

3 3 Subjective MOS Subjective MOS

2 2

1 1 1 2 3 4 5 1 2 3 4 5 GQI GQI MOS,1 MOS,2

Figure 6.7: GQI model prediction versus Subjective MOS for both models. Linear model to the left and the non-linear model to the right.

The somewhat straggling GQI prediction can be explained by small sample sizes and changing circumstances at gaming servers during tests which decreases the models accuracy. It is hard to perform several tests with exact the same game conditions. The normal probability plot and the residuals histogram indicated a competent GQI prediction for a broad variable span. If the residuals in

50 6.2. QUALITY MODEL a normal probability plot lies on a straight line indicates a normal distribution. Another important metric is the high proportion of GQI predictions within 95% MOS conﬁdence interval for both models, although the conﬁdence intervals are fairly wide.

10 0.99 0.98 0.95 8 0.90

0.75 6

0.50 Probability Frequenzy 4 0.25

0.10 2 0.05 0.02 0.01 0 −1 −0.5 0 0.5 1 1.5 −2 −1 0 1 2 Data Residuals

Figure 6.8: Normal probability plot and histogram with a Gaussian regression curve for the linear GQIMOS,1 model.

10 0.99 0.98 9 0.95 8 0.90 7 0.75 6

0.50 5 Probability Frequency 4 0.25 3 0.10 2 0.05

0.02 1 0.01 0 −1 −0.5 0 0.5 1 1.5 −2 −1 0 1 2 Data Residuals

Figure 6.9: Normal probability plot and histogram with a Gaussian regression curve for the GQIMOS,2 non-linear model.

51 6.3. EXTENDED QUALITY MODEL

6.3 Extended Quality Model

5 5

4 4

3 3 MOS MOS

2 2

1 1 0 100 200 300 400 500 0 1 2 3 4 5 Jitter [ms] Packet Loss [%]

Figure 6.10: Linear regression curves for jitter (left) and packet loss (right) with boundary conditions.

A model extension can be made by adding the two extra input variables jitter and packet loss. Input data to this model extension have been collected from [33] and [1] where data for jitter and packet loss have been measured in a similar fashion for an FPS game. Regression analysis in Matlab discharge into quality estimation equations for jitter and packet loss. Figure 6.10 presents the regression curves for input functions to the extended quality model.

GQIBASE steers the initial model value and GQIMIN does the adaptation to MOS, as for the ﬁrst models. An extended Gaming Quality Index quality model GQIMOS,EX , where the parameters jitter and packet loss have been introduced can be formulated as:

GQIMOS,EX ∝ f(xlatency, ymapload, zjitter, wpacketloss) =

GQIMIN + GQIBASE ∗ f(xlatency, ymapload, zjitter, wpacketloss) (6.18)

Where f(xlatency, ymapload, zjitter, wpacketloss)

(Qxlatency ) ∗ (Qymapload ) ∗ (Qzjitter ) ∗ (Qwpacketloss ) (6.19)

where Qxlatency and Qymapload are the same as for the linear GQIMOS,1 model.

Qzjitter = −0.0022 ∗ zjitter + 1 (6.20)

Qwpacketloss = −0.24 ∗ wpacketloss + 1 (6.21) Maximum boundary for jitter is set to 450 ms and for packet loss it is min set to values above 4 %. Validation of the extended model can unfortunately not be

52 6.3. EXTENDED QUALITY MODEL done when no complete data set is possible. Therefore additional test have to be done when these variables are measured and the GQIMOS,EX quality model can be validated and further reﬁned with additional training data set.

53 Chapter 7

Conclusions

This chapter presents conclusions from chapter 5, made up from the Internet Radio, online gaming tests and the quality model. At the end future work in the ﬁeld of study is presented.

Internet Radio

Internet Radio evaluation have indicated that short and frequent rebufferings are experienced as more annoying than longer sessions of media followed by a longer rebuffering time. When the bit rate of the media stream is just above what the radio bearer can manage, short rebufferings have to be done to keep up with the stream. Therefore it is advantageous with a media application implemented with a large buffer to reduce the risk of playback interruptions due to delayed or retransmitted packets.

When starting the Internet Radio session the over all service quality grading must include the browsing phase. Quality grading during the media playback together with the start up phase both contribute to the service quality grading. GRPS/EDGE reaches the stream bit rate limit at 96 kbit/s, and even lower if the user is moving and exposed to bad radio environment. For WCDMA R99 and HSPA high quality streams up to 320 kbit/s can be delivered in urban areas.

CounterStrike

When players are exposed to start up/map load times above 120 seconds the willingness to leave the game increases rapidly. Thus, a clear service degradation occur when the start up phase in the game is too long. Map load times exceeding 300 seconds are clearly unacceptable and leads to premature game termination. Focus in earlier game evaluations have been during game play and not under start up and synchronization phases.

Due to high player interaction during game play it is complicated to do countinous service evaluation. Evaluation is therefore done in shorter sessions and is concluded with questions and session quality grading. The testing activities have identiﬁed two player types, namely ”‘optimistic”’ and ”‘complainers”’

54 where their patience and quality assessment diﬀerentiate from each other.

Latency has, as concluded a great impact in the subjective users’ perception of the gaming experience. But it has also shown that players tolerate higher latency levels, as in ﬁgure 6.3 where MOS scores of 3 is reached with latency levels of up to 150 ms. This means that the game is still playable and players tend to overestimate latency’s impact on the gaming experience. This phenomenon has also been discovered in [33] which pleads for more focus on other quality disparaging variables as jitter and packet loss in parallel with latency.

Chapter 6 demonstrates a quality model for FPS games as CounterStrike. From the measured parameters two models could be build, one linear and one non-linear model. The performance metrics have shown that the non-linear model perform slightly better than the linear model at all points. The linear models RMSE value is 0.677 and for the non-linear is 0.609. Both models produces a high correlation value between predicted and subjective MOS. Pre- diction accuracy for the non-linear model is as high as 91.9%. But, due to uncertainties in the data set the linear model produces acceptable predictions and gains conﬁdence through it’s simplicity. More measurements in a more controlled environment can reﬁne and improve the GQIMOS models even further.

World of Warcraft

Due to long patch/update times World of Warcraft have proven not to be playable over a GPRS/EDGE connection. Whereas shorter gaming session over WCDMA R99 is possible and provides an acceptable, but far from good gaming experience. As for GPRS/EDGE does WCDMA R99 also suﬀer from exces- sive patching times and high jitter. Far better experience is monitored over an HSPA connection. The tests indicate that HSPA in every point can provide an excellent gaming experience with no visible session impairments. One major issue for World of Warcraft, although not covered in this thesis, is the hardware requirement’s as graphical card and RAM memory. Suitable requirement for mobile terminals tend to be very expensive.

One reason for the huge success World of Warcraft has encountered is the cause of addiction that strike’s many players who joins the game. As mentioned in chapter 4 gamers have to pay for subscription to access the game. Mobile broadband access networks as HSPA opens a gaming possibility for a more agile use of World of Warcraft gaming accounts, for people on the run.

The testing activities have been performed with utmost limited concurring network traﬃc. Increasing load in the networks will inﬂuate network performance and the gaming quality for all types of services.

Future Work

Changing Internet behavior and increased use of network based games opens the possibility for more intense use of services on the ﬂy. Users aren’t bound to

55 their ﬁxed broadband access at home. Operators and network developers have to make intensive research of what type of applications and their requirements when designing new wireless network technologies. One example is just World of Warcraft, which is played by more than 8 million gamers around the world and has more than 2 million players in North America, more than 1.5 million players in Europe, and more than 3.5 million players in China. In addition to World of Warcraft there are a range of similar online games with an increasing user community. [36], [28]

To further enhance the quality model more subjective testing have to be done. Measurements of parameters such as packet loss and jitter will contribute to a better and more accurate model. This gives higher signiﬁcance and lower uncertainties of the quality model output and the model will beneﬁt to greater and more accurate predictions on FPS gaming quality.

An interesting progress for online gaming services will be the future employ- ment of LTE cellular networks. LTE will precede the HSPA networks which provided a great leap forward in network capabilities compared with GPRS/EDGE and WCDMA R99.

56 Bibliography

[1] T. Lang, “User Experience while playing Halo with network delay or loss,” Centre for Advanced Internet Architectures, CAIA, Swinburne University of Technology, Tech. Rep. 031205A, December 2003. [Online]. Available: http://www.caia.swinburne.edu.au/ [2] P. Branch and G. Armitage, “Measuring the auto-correlation of server to client traffic in First Person Shooter games,” Swinburne University, December 2003. [Online]. Available: http://www.caia.swinburne.edu.au/ [3] M. Akervik,˚ “Network Gaming - Performance and Traffic Modeling,” Mas- ter’s thesis, Royal Institute of Technology, Stockholm (KTH), December 2006. [4] C. Westermark, “Mobile Multiplayer Gaming,” Master’s thesis, Royal In- stitute of Technology, Stockholm (KTH), May 2007. [5] P. Rysavy, “Mobile Broadband, EGDE, HSPA and LTE,” 3G Americas, September 2006. [Online]. Available: http://www.rysavy.com [6] ——, “Data Capabilities: GPRS to HSDPA,” 3G Americas, Rysavy Research, vol. 1-541-386-7475, September 2004. [Online]. Available: http://www.rysavy.com [7] C. Hartmann, “Resource Management in Wireless Networks, Lehrstuhle für Kommunikations Netzen, Techniche Universität München, TUM,” 2006. [Online]. Available: http://www.lkn.ei.tum.de [8] Apis Technical Training GSM/GPRS/UMTS System Intensive. Apis Technical Training AB, 2006. [Online]. Available: http://www.apis.se [9] H. Holma and A. Toskala, WDCMA for UMTS, Radio Access For Third Generation Mobile Communications. John Wiley and Sons, Ltd, 2001. [10] R. F. Mikael Olofsson, Tomas Ericson and Ulf Henriksson, Basic Telecom- munication. Bokakademin, 2006. [11] J. Scourias. (1997, October) Overview of the Global System for Mobile Communications. [Online]. Available: https://styx.uwaterloo.ca/ js- couria/GSM/gsmreport.html) [12] CME20, System Survey, Training Document. Ericsson, 1992. [13] CME20, System Survey, Students Workbook. Ericsson, 1994.

57 BIBLIOGRAPHY

[14] “White Paper - EDGE, Introduction of high-speed data in GSM/GPRS networks,” 2003.

[15] “White Paper - Basic concepts of HSPA,” Ericsson AB, Tech. Rep. 284 23-3087 Uen Rev A, 2007. [16] “Mobile Broadband: The Global Evolution of UMTS/HSPA,” 3G Ameri- cas, July 2006. [17] (2006, October) Long Term Evolution of the 3GPP Radio Technology. [Online]. Available: http://www.3gpp.org/Highlights/LTE/LTE.htm [18] “End-user multimedia QoS categories,” in ITU-T G.1010, November 2001, pp. 814–818. [19] “White Paper - QoE can it be measured?” Nokia, Tech. Rep. 11212 - 1004, 2004. [20] “ETSI TS 125 215,” ETSI, Tech. Rep. v 7.1.0, September 2006. [Online]. Available: http://www.etsi.org [21] (2007, August) Shoutcast.com. [Online]. Available: http://shoutcast.com/

[22] (2007, August) Winamp.com. [Online]. Available: http://www.winamp.com/ [23] I. Adan and J. Resing, “Queueing Theory,” Department of Mathematics and Computing Science, Eindhoven University of Technology, Tech. Rep., February 2001. [Online]. Available: http://mia.ece.uic.edu/ papers/WWW/Flexi-Tunes/tarballs/queue.pdf [24] Dr. Wolfgang Balzer, “QoS parameters for Web Radio,” vol. V.1.4.1, no. 250-2 034, May 2007, pp. 814–818. [Online]. Available: www.etsi.org [25] Perry Priestley, “HD Radio How it compares to other Digital Radio Tech- nologies,” iBiquity Digital Corporation, Tech. Rep. 11212 - 1004.

[26] (2007, August) Wikipedia.org. [Online]. Available: http://en.wikipedia.org/wiki/Bit rate [27] M. Claypool and K. Claypool, “On Latency and Player Actions in Onlie Games,” July 2006. [Online]. Available: http://web.cs.wpi.edu/ claypool/

[28] (2007, August) World of Warcraft. [Online]. Available: http://www.worldofwarcraft.com/ [29] “QoS Performance Requirements for UMTS, TSG S1 3(99) 362,” 3GPP, Tech. Rep. v 6.3.3, April 1999. [Online]. Available: http://www.etsi.org [30] (2007, August) World of Warcraft, Community Forum. [Online]. Available: http://forums.worldofwarcraft.com/ [31] D. S. Hands, “A Basic Multimedia Quality Model,” IEEE Transactions on Multimedia, December 2004. [Online]. Available: http://www.ieee.org

58 BIBLIOGRAPHY

[32] “Matlab, The Mathworks, Inc,” 1994-2006. [Online]. Available: http://www.mathworks.com/access/helpdesk/help/techdoc/matlab.shtml

[33] O. W. Mattias Dick and L. Wolf, “Analysis of Factors Affecting Players’ Performance and Perception in Multiplayer Games,” Institut fürBetrieb- systeme und Rechenverbund, Techniche UniversitätBraunschweig, Tech. Rep. v 7.1.0. [34] E. W. Weisstein. (2004, August) Student’s t-Distribution, From MathWorld–A Wolfram Web Resource. [Online]. Available: http://mathworld.wolfram.com/Studentst-Distribution.html [35] Gunnar Blom, Sannolikhetsteori och statistikteori med tillämpningar, 5th ed. Studentlitteratur AB, 2005. [36] (2007, January) World of Warcraft Surpasses 8 Million Subscribers World- wide. [Online]. Available: http://www.blizzard.com/press/070111.shtml

59 List of Figures

2.1 Three omnidirectional cell sites (left) where one antenna covers one whole cell. To the right a sectorized cell with three directed antennas...... 4 2.2 Multiple access schemes...... 6 2.3 A basic TDMA frame with eight timeslots. The tail-bits (TB) are used as start/stop sequence for a TS and the training sequence is used by the equalizer to create a channel model...... 8 2.4 GSM and GPRS architechture...... 9 2.5 UMTS network architechture ...... 11 2.6 FTP upload and download over HSPA with EUL, HSPA reference speedtest performed at UTN ...... 13

3.1 The user can’t see the underlying network infrastructure, but ”feels” the overall service quality which highly equals to Quality of Experience [19] ...... 16 3.2 Collection of network KPI:s can be made in all of the network architechture nodes or in the user equipment ...... 17 3.3 Latency measurement from the user equipment to the server where the latency equals T1 + T2 ...... 19

4.1 An Internet Radio stream modeled as a queuing system...... 23 4.2 Playback of an Internet Radio stream...... 24 4.3 Throughput downlink for the gamephases in CounterStrike measured with TEMS Investigation...... 28 4.4 Sequence picture for data collection and analysis ...... 30

5.1 Latency for a 128 kbit/s radio stream over a WCDMA R99 bearer (left) and mean MOS for bitrates from 48 to 320 kbit/s for GRPS/EGDE, WCDMA R99 and HSPA tests (right). For MP3 bitrates below 64 kbit/s is played in mono, while higher bitrates is played in stereo...... 32 5.2 Decreased throughput in the downlink (above) caused by a severe radio environment impairment (below) in a GRRS/EDGE network. 33 5.3 Internet Radio - Packet sizes histogram and packet distribution for GRPS/EGDE, WCDMA R99, HSPA and Broadband. The diﬀerence in the GPRS/EDGE packet distribution is due to lower stream bitrate...... 35 5.4 CounterStrike throughput during gameplay ...... 36

60 LIST OF FIGURES

5.5 Map load times and 95% conﬁdence interval for CounterStrike over three evaluated radio access networks...... 37 5.6 95% conﬁdence interval´s for Latency (left) and Subjective MOS (right) for three access networks. HSPA tests were performed with fewer tests (5) than the other two...... 37 5.7 Latency log during a World of Warcraft session over HSPA (left) and subjective MOS and mean latency per access network. . . . 39 5.8 Throughput during a World of Warcraft session...... 41 5.9 Packet sizes and cumulative distribution function for WCDMA R99 and HSPA access networks ...... 41

6.1 A quality model sketch for online gaming...... 43 6.2 Principal Components Analysis from FPS game CounterStrike . 44 6.3 MOS for Latency (left) and Map Load (right) in 95 % conﬁdence intervals plotted at the mean value of each span...... 45 6.4 Linear regression curves for Latency (left) and Map Load (right) with boundary conditions...... 46 6.5 Illustration of non-linear GQIMOS,2 prediction outputs...... 47 6.6 Model prediction outputs (♦) for the linear model (left) and the non-linear model (right) plotted in 95% conﬁdence interval. . . . 50 6.7 GQI model prediction versus Subjective MOS for both models. Linear model to the left and the non-linear model to the right. . 50 6.8 Normal probability plot and histogram with a Gaussian regression curve for the linear GQIMOS,1 model...... 51 6.9 Normal probability plot and histogram with a Gaussian regression curve for the GQIMOS,2 non-linear model...... 51 6.10 Linear regression curves for jitter (left) and packet loss (right) with boundary conditions...... 52

61 List of Tables

2.1 Modulation and coding schemes for GPRS and EDGE ...... 10

3.1 Reliability, availability, accessibility and maintenance KPI:s . . . 18 3.2 Service comfort KPI:s [19] ...... 18 3.3 Service speciﬁc parameters for Internet Radio and online gaming 20 3.4 3GPP framework requirements for network based on voice and multimedia services. [20] ...... 21

4.1 Comparsion of audio bit rates for the MP3 codec. [26] ...... 25 4.2 Mean Opinion Score...... 29 4.3 Speciﬁcations of the user terminal...... 29 4.4 Utilized CounterStrike servers and gaming map where test sessions were performed...... 31

5.1 Maximal bitrates per radio access network...... 34 5.2 Correlation reference matrix for two independent variables. Used as a framework for correlation between measured variables. . . . 34 5.3 Correlation between MOS and stream bit rate from the Internet Radio tests (grading were done by one person)...... 34 5.4 Mean throughput and packet sizes for a CounterStrike session over HSPA ...... 36 5.5 Acceptable latency intervals in World of Warcraft. [30] ...... 40 5.6 Mean throughput and packet sizes for a World of Warcraft session over HSPA...... 40

6.1 Correlation matrix for the Modelvariables ...... 44 6.2 Coordinates for the four principal components...... 45

7.1 Figures in the table are presented as Broadband/WCDMA R99. 63

62 Appendix I

Questionnare 1. Have you played this game before? If yes, how many hours approximately?

2. Is the game playable over a WCDMA R99 radio network? What where your expectaions? 3. Are you satisﬁed by the game experience? 4. Are you willing to leave the game?

5. What was the most obvious impairment during gameplay? 6. Over all service experience in addition to the grade?

Test 1 2 3 4 Map Load 9/3.38 25/23 14/45 11/32 Kills 0/0 3/6 1/2 2/4 Deaths 5/3 5/5 6/5 7/9 Latency 85/200 65/200 36/580 40/210 MOS 5/2 4/2 4/3 5/2 Stay/Leave 5/4 5/3 4/2 5/2 Exp (hrs) 2 >500 >500 <50 Test 5 6 7 8 9 Map Load 23/32 11/43 12/25 17/1.18 23/33 Kills 2/0 0/0 3/2 1/2 0/3 Deaths 5/4 7/5 7/8 5/6 7/6 Latency 40/200 35/230 35/220 30/680 40/340 MOS 5/3 5/2 4/2 5/2 5/2 Stay/Leave 3/2 5/3 5/2 5/2 5/3 Exp (hrs) ∼ 20 ∼ 100 > 250 ∼ 100 ∼ 10

Table 7.1: Figures in the table are presented as Broadband/WCDMA R99.

63 Copyright Svenska Detta dokument h˚allstillgängligtp˚aInternet - eller dess framtida ersättare- under en längretid fr˚anpubliceringsdatum under förutsättning att inga extra- ordinäraomständigheteruppst˚ar.

Tillg˚angtill dokumentet innebärtillst˚andförvar och en att läsa,ladda ner, skriva ut enstaka kopior förenskilt bruk och att användadet oförändratförick- ekommersiell forskning och förundervisning. Overföringav¨ upphovsrättenvid en senare tidpunkt kan inte upphäva detta tillst˚and.All annan användningav dokumentet kräver upphovsmannens medgivande. Föratt garantera äktheten, säkerheten och tillgänglighetenfinns det lösningarav teknisk och administrativ art.

Upphovsmannens ideella rätt innefattar rättatt bli nämndsom upphovsman i den omfattning som god sed kräver vid användningav dokumentet p˚aovan beskrivna sättsamt skydd mot att dokumentet ändraseller presenteras i s˚adanform eller i s˚adant sammanhang som ärkränkande förupphovs- mannens litteräraeller konstnärligaanseende eller egenart. Förytterligare information om LinköpingUniversity Electronic Press se förlagetshemsida: http://www.ep.liu.se/

English The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances.

The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.

According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Link¨opingUniver- sity Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/

Pontus Sandberg Link¨oping,