4th International Workshop on Perceptual Quality of Systems (PQS 2013) 2-4 September 2013, Vienna, Austria Need for Speed? On Quality of Experience for Cloud-based File Storage Services Philipp Amrehn1, Karel Vandenbroucke2, Tobias Hoßfeld1, Katrien De Moor2,3, Matthias Hirth1, Raimund Schatz4, Pedro Casas4 1University of Wurzburg¨ (Germany) 2iMinds-MICT-Ghent University (Belgium) 3ITEM-NTNU (Norway) 4Telecommunications Research Center Vienna FTW (Austria) [email protected], [email protected] Abstract (i.e. performance) and the possibility to utilize a service at all (i.e. availability of services) [2]. Yet, their relative impact on Cloud computing is receiving growing attention by researchers QoE and the role of other influencing factors, are to date poorly from a variety of disciplines. However, so far only a few studies understood. It has also been argued that in order to estimate exist that investigate the Quality of Experience (QoE) of cloud QoE for cloud storage services and applications, it is necessary services, including the category of personal file storage services to monitor network environment and conditions, terminal capa- like Dropbox or Livedrive, from an end user perspective. bilities, SLAs and service and application-specific information The contribution of this paper is a methodology and the re- [3]. However, not only such purely technical and QoS-related sults of four different user studies towards a situational QoE parameters influence QoE. It has been shown that human and model for file storage services. In order to obtain insights in context-related factors may also play a major role. In the con- existing usage practices and to detect possible QoE influenc- text of Web QoE for instance, memory effects, i.e., the psycho- ing factors and relevant features, we conducted an online sur- logical influence of past experiences [4] bear a strong influence. vey amongst users of personal cloud storage services in general Furthermore, a recent study on adoption of cloud services indi- (N=349) and one specifically targeted at users of one of the ma- cated that security and privacy issues impose strong barriers to jor popular cloud storage services, namely Dropbox (N=49). A user adoption [5]. Finally, personal cloud storage services and third study (N=13) on mobile Dropbox further investigated QoE applications are increasingly used in different context and ac- and specific use cases in a mobile context, via smartphones and cessed from multiple devices, with distinct characteristics. This tablets. Based on these results, typical usage situations as well evolution does not only amplify availability problems and other as short-term and long-term QoE influence factors were derived technical challenges, it potentially also has crucial implications which included user profile, context and situation, as well as for QoE, which need to be considered. system level influences. In a fourth lab study (N= 52) the impact As a result, in order to identify the major QoE influencing of waiting times, i.e. short-term QoE influences on system level factors, there is a need to go beyond technical aspects and to during the regular usage of Dropbox, is investigated for differ- gain a better understanding of the experiences and use practices ent situations and use cases are obtained from the first three themselves. As a first step, we therefore conducted two user sur- studies. As an outcome of our studies, we formulate research veys, one about file storage services in general and one specif- questions driving the agenda towards measuring and modeling ically about Dropbox. The aim was to understand the actual QoE for cloud-based file storage services. usage of file storage services as well as to detect possible QoE : Quality of Experience, survey research, file stor- Index Terms influencing factors and relevant features to address the follow- age services and applications, user perspective ing questions. Is it possible to define different user profiles of file storage services? What are main influence factors on QoE 1. Introduction for file storage services? How do user profiles differ in QoE in- In the last couple of years, the emergence of file storage services fluence factors? Is the performance of file storage services (i.e. facilitated by cloud computing technology represented one of speed of data transfer) a major item influencing the QoE? So, the trends that received particular attention in several research does a need for speed exist? Do service providers solely have domains. From an end-user and market point-of-view, it can- to focus on networking speed to improve the users’ QoE or is a not be ignored that the use of personal cloud storage services secure, reliable service also or even more important? such as Dropbox, Google Drive and iCloud has boomed enor- Secondly, since Dropbox is - like personal cloud storages mously. According to [1], the most important application which services in general - also increasingly being accessed via smart- is responsible for this evolution, is Dropbox. From a service phones and tablets, a dedicated mobile Dropbox study was set providers’ perspective, measuring users’ Quality of Experience up. The aim here was to investigate QoE in different mobile (QoE) is critical since it is strongly tied to revenue and market use cases and considering different devices (smartphones and success. So far however, the literature on the QoE of file storage tablets). Information was gathered through on-device Experi- services is very limited. As a result, it is currently still unclear ence Sampling Monitoring (ESM) questionnaires and mobile what the major QoE influence factors are. Factors that have sensor logs. already been identified in the literature include waiting times Finally, we also discuss approaches on modeling cloud stor- 174 10.21437/PQS.2013-33 sonal cloud storage services and applications participated. 87% of them indicated to use Dropbox. Study 2: Dropbox Service. In the second online survey, we therefore zoomed in more concretely on the use of Dropbox storage capacities and tried to detect factors influencing Drop- box QoE. For this survey (N=49 Dropbox users), a dedicated application was installed on the participants’ Dropbox account in order to gather information on available and used storage ca- pacity. To recruit respondents, we addressed a paid crowdsourc- ing website, as well as volunteers, in particular members of the Qualinet-panel [6] and respondents of the first survey. In 2.2 we discuss additional methodological consideration related to Figure 1: Methdology and conducted user studies towards a sit- the use of crowdsourcing in this study. Depending on users’ uational QoE model for file storage services goals and specific purposes for using Dropbox, their personal characteristics and the usage situation, the impact of influence factors on Dropbox QoE may differ. Therefore, the information age QoE depending on the actual usage context and situation. collected in this second survey is used to define user profiles and Therefore, a QoE study on different usage scenarios of cloud groups of QoE influence factors by using the Expectation Max- storage users was conducted in a laboratory environment. The imization (EM) cluster algorithm. For modeling Dropbox QoE network conditions were varied in such a way that different depending on the actual usage context and situation, we analyze waiting times occurred, in order to answer the following ques- the connection between these clusters to map user groups to sets tions: Do cloud storage users perceive waiting times differently of QoE influence factors. according to the actual usage or task (e.g. synchronization or Study 3: Mobile Dropbox. In order to gain more insight collaboration) and context (e.g. mobile device)? in the use practices and to explore the possible role of contex- The remainder of this paper is structured as follows. Sec- tual factors, an exploratory, follop-up study (N=13) considering tion 2 first gives an overview of our methodology and the con- real usage of Dropbox in a natural context, was set up. More ducted studies. Thereupon, we discuss whether crowdsourc- concretely, this study focused on the use of Mobile Dropbox on ing can be used for our Dropbox user survey and share a num- smartphones and tablets. QoE-relevant usage information was ber of important considerations. Section 3 discusses the results collected in users’ natural enviroment by means of contextual of the personal cloud storage survey and the Dropbox survey. data logging and an on-device Experience Sampling Monitoring In particular, the importance of cloud storage features is eval- (ESM) questionnaire [7], facilitated by the AWARE Framework uated, before typical user profiles of Dropbox users are de- [8]. During two weeks, 156 ESM questionnaires were filled in rived through clustering. Section 4 approaches the Dropbox by 7 smartphone and 6 tablet users: after the use of Dropbox on QoE model by analyzing the relevance of influence factors and the participant’s device, a questionnaire was triggered. During by correlating contextual parameters with ratings from mobile the usage session, additional device- and context- related in- Dropbox users, before the influence of waiting times is quan- formation (such as gps-location, network type, signal strength, tified. We propose a generic QoE model for Dropbox which duration) was collected. is open for dicussions and formulate research questions driving Study 4: Dropbox Waiting Times Lab Study. As the the agenda for file storage QoE. Finally, Section 5 summarizes above studies indicated and as is discussed in Section 4, fast our main insights and conclusions . synchronization and short startup times matter to end users. For this reason, a subjective lab study (N=52) was conducted at the premises of FTW in order to quantify the impact of waiting 2. Methodology times on QoE for cloud storage and file synchronization ser- Four individual studies were conducted to derive typical usage vices like Dropbox.