Sede Amministrativa: Università degli Studi di Padova Dipartimento di Matematica ___________________________________________________________________ SCUOLA DI DOTTORATO DI RICERCA IN: SCIENZE MATEMATICHE INDIRIZZO: INFORMATICA CICLO: XXVIII SMARTPHONE AS UBIQUITOUS DEVICES FOR BEHAVIOR ANALYSIS AND BETTER LIFESTYLE PROMOTION Direttore della Scuola: Ch.mo Prof. Pierpaolo Soravia Coordinatore d’indirizzo: Ch.mo Prof. Francesca Rossi Supervisore: Ch.mo Prof. Ombretta Gaggi Dottorando: Matteo Ciman Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion PhD student: Matteo Ciman Supervisor: Ombretta Gaggi Doctoral School in Mathematical Sciences Computer Science area, XXVIII course University of Padua, Italy Contents Abstract......................................................7 I Introduction 1 Introduction.................................................. 13 1.1 Ubiquitous computing for health 13 1.2 Problem Statement 16 1.2.1 Cross-platform application development.......................... 16 1.2.2 Ubiquitous smartphone computing for behavior analysis............. 17 1.2.3 Power of serious games on people life............................ 18 1.3 Research Contributions 20 1.4 List of Publications 22 1.5 Structure of the Thesis 25 II Cross-platform frameworks 2 Cross-platform frameworks overview.......................... 29 2.1 Cross-platform frameworks classification 30 3 Development Tools analysis................................... 35 3.1 Related works 36 3.2 Case Study 37 3.3 Analysis guidelines 38 3.3.1 Licenses and costs............................................. 38 3.3.2 API........................................................... 38 3.3.3 Tutorials and Community........................................ 39 3.3.4 Programming complexity....................................... 40 3.3.5 IDE........................................................... 40 3.3.6 Supported mobile devices...................................... 40 3.3.7 Native user interface........................................... 40 3.3.8 Basic knowledge............................................... 41 3.4 Frameworks 41 3.4.1 MoSync (Cross-Compiled Approach)............................. 41 3.4.2 Titanium (Interpreted Approach)................................. 44 3.4.3 jQuery & jQuery mobile (Web Approach)......................... 47 3.4.4 Phonegap (Hybrid Approach)................................... 48 3.4.5 Final remarks.................................................. 50 4 Energy consumption issues analysis........................... 51 4.1 Related Works 52 4.2 Methodology 53 4.2.1 Hardware setup............................................... 53 4.2.2 Software setup................................................ 55 4.3 Experimental Results 58 4.4 Final remarks 68 III Emotion assessment using smartphones 5 Smartphone interaction for stress assessment.................. 73 5.1 Related works 74 5.2 Protocol design 77 5.2.1 Human-Smartphone Interaction.................................. 77 5.2.2 Test Tasks...................................................... 77 5.2.3 Stressor Task................................................... 78 5.2.4 Stress Measurement............................................ 80 5.2.5 Protocol Overview............................................. 80 5.2.6 Participants................................................... 81 5.3 Data collection 82 5.3.1 Object and action............................................. 82 5.3.2 Feature Extraction.............................................. 83 5.4 Results 85 5.4.1 Stress induction analysis......................................... 86 5.4.2 Features analysis............................................... 87 5.5 Final remarks 89 IV Serious games for medical purposes 6 Serious games for behavior change........................... 93 6.1 Serious games overview 94 6.1.1 Exergames.................................................... 95 6.1.2 Serious games in the medical field................................ 96 6.1.3 Improving accessibility through serious games..................... 97 6.2 Persuasive Technologies 98 6.2.1 The Fogg Behavior Model (FBM)................................. 100 6.2.2 Behavior Change Support System (BCSS)......................... 104 6.2.3 Gamification................................................. 105 7 ClimbTheWorld to promote physical activity.................. 109 7.1 Related Works 111 7.2 Game Design 115 7.3 Pipeline overview 117 7.4 Data smoothing 118 7.5 Data standardization 120 7.6 Segmentation with data-dependent window 122 7.7 Representation and features standardization 124 7.8 Experiments and results 125 7.8.1 Classification setting and results................................. 125 7.8.2 Energy consumption results..................................... 128 7.8.3 Triggers design................................................ 130 7.8.4 Users Tests.................................................... 132 8 PlayWithEyes for visual acuity assessment..................... 137 8.1 Related Works 139 8.1.1 Existing Software.............................................. 140 8.2 Description of the game PlayWithEyes 141 8.2.1 Visual acuity tests............................................. 142 8.2.2 Test for color blindness......................................... 143 8.2.3 Interaction methods........................................... 143 8.3 System Requirements and Architecture 145 8.4 Resizing Optotypes 150 8.4.1 Theoretical basis.............................................. 150 8.4.2 Implementation.............................................. 153 8.5 Tests 155 8.5.1 The serious part of the system................................... 155 8.5.2 The game part of the system................................... 158 8.5.3 Tests execution time........................................... 159 V Conclusions and Future Works 9 Conclusions and future works................................ 163 9.1 Conclusions 163 9.2 Future Works 166 9.3 Final remarks 169 Bibliography................................................ 170 Abstract Ubiquitous Computing, the third computing era, has taken into people’ life a huge amount of invisible and pervasive devices, i.e., smartphones, tablets, sensors, etc. Considering their pervasiveness and their presence in people’ everyday life, in the last few years research focused its attention on how it is possible to use smartphones to improve people’ life. In particular, nowadays smartphones are equipped with an ample set of different sensors, i.e., accelerometer, barometer, light sensor, etc., that can constantly acquire data from the surrounding environment of the user, or about his/her activity and behavior during the day. This data acquisition can be done in a non-intrusive way, since smartphones are always in people’ pocket or bag and are already part of individuals’ life. Thanks to these sensors, a smartphone is able to acquire a huge amount of data, which can be used for different purposes. For example, the use of ubiquitous computing for healthcare and medical purposes is extremely interesting. In particular, it can improve people’ life, help patients to accept their therapies and reduce the drop-out-from-therapy phenomena, and improve medical practice, thus decreasing medical costs both for patients and for governments. Actually, ubiquitous computing for healthcare is facing and addressing several prob- lems that will be discussed in this thesis. First of all, since there is not a single smartphone vendor (which is an advantage from the end-user point of view), when considering smart- phone applications for healthcare it is necessary to consider market fragmentation. This kind of applications aims to reach the highest number of possible users, since increasing the number of possible users increases the number of possible patients that could benefit 8 from the usage of a particular application. Clearly, the objective of reaching the highest number of users has to consider also the problem of containing expenses. For this par- ticular problem, we will analyze cross-platform frameworks for smartphone applications development, also called “write once, run everywhere” framework”. We will highlight pros and cons of them, considering both the developer and the end user point of view and power consumption, providing a deep analysis of their main characteristics and evaluating them with respect to the native mobile application development approach. Afterward, we will focus our attention to the possibility of using smartphones to infer individuals’ moods and feelings. In particular, since smartphones are always present in everybody life, it is possible to use them to transparently and pervasively acquire data and information about how the smartphone is used by the user to understand his/her behavior. In this way, we explored the possibility to use smartphone data, without the usage of any external sensor, thus reducing the entry cost and increasing user acceptance, to assess their stress state. In particular, we will see how the way people interact with their smartphone changes depending on their stress condition, opening the doors for the possibility to pervasively monitor stress of people all the day long without interfering with their life and providing the possibility to doctors to monitor their patients with objective data. Finally, since ubiquitous computing can be used to improve people’ life, we explore the usage of serious games with mobile applications to improve people lifestyle. We applied this idea to the possibility to