What to Put on the User: Sensing Technologies for Studies and Physiology Aware Systems Katrin Hänsel Romina Poguntke Hamed Haddadi Queen Mary University of University of Stuttgart Imperial College, London London [email protected] [email protected] [email protected] stuttgart.de Akram Alomainy Albrecht Schmidt Queen Mary University of Ludwig Maximilian London University, Munich [email protected] [email protected]fi.lmu.de ABSTRACT consumer wearables flooding the market1 every year, wire- Fitness trackers not just provide easy means to acquire physi- free and independent tools for assessing physiological data ological data in real world environments due to affordable are becoming an increasingly valuable for researchers and sensing technologies, they further offer opportunities for scientists alike. With the feasibility to measure signals (e.g. physiology-aware applications and studies in HCI; however, heart rate response) ’on the fly’ and in daily life situations, their performance is not well understood. In this paper, we there come along immense opportunities. report findings on the quality of 3 sensing technologies: PPG- Rosalind Picard was among the first to emphasize the impor- based wrist trackers (Apple Watch, Microsoft Band 2), an tance of sensing wearables. In the article "Affective Wear- ECG-belt (Polar H7) and reference device with stick-on ECG ables" [48], she discussed application scenarios and presented electrodes (Nexus 10). We collected physiological (heart rate, a prototype for recording physiological data, like blood vol- electrodermal activity, skin temperature) and subjective data ume pressure, Galvanic Skin Response and respiration. In the from 21 participants performing combinations of physical ac- past years, various systems exploiting the feasibility to access tivity and stressful tasks. Our empirical research indicates that physiological data have been published approaching more and wrist devices provide a good sensing performance in station- more user-adaptive interfaces [7] and systems [56]. Emerging ary settings. However, they lack accuracy when participants fields in the HCI community like calm-computing [39] and are mobile or if tasks require physical activity. Based on our the avoidance of technostress [74] can benefit from ubiquitous findings, we suggest a Design Space for Wearables in Re- and wearable affect sensing technology and adaptive systems. search Settings and reflected on the appropriateness of the investigated technologies in research contexts. However, much of the understanding of physiological sensing is based on high accuracy lab equipment and it remains un- ACM Classification Keywords clear how well consumer devices are suited for this purpose. H.5.m. Information Interfaces and Presentation (e.g. HCI): Khusainov et al. [37] discuss in their survey paper that wear- Miscellaneous able sensors often lack accuracy and appropriate sampling rates. Furthermore, not every wearable system delivers raw 2 Author Keywords data; e.g. the Fitbit and Jawbone device families do not allow Wearable Technology, Validation, Stress, Affective users to assess their raw physiological data. Consequently, Computing there is a need to evaluate the reliability of wearable consumer technology with regards to their accuracy and suitability for INTRODUCTION physiological and psychological research applications. Recent advances in consumer wearables allow the ubiquitous In this paper, we perform a comparison between two wrist- collection of health data, such as physical activity or sleep worn devices with optical heart rate sensors (Apple Watch, in everyday life. Enabled through the variety of affordable Microsoft Band 23) against a heart rate chest strap (Polar H74) Permission to make digital or hard copies of all or part of this work for personal or and a laboratory measurement instrument (Nexus 10 kit) under classroom use is granted without fee provided that copies are not made or distributed different physical and stressful conditions. To evaluate the for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or 1 republish, to post on servers or to redistribute to lists, requires prior specific permission The market value for wearable devices is forecasted to be almost 6 and/or a fee. Request permissions from [email protected]. billion USD by 2018 [33]. 2 CHI 2018, April 21–26, 2018, Montreal, QC, Canada www.fitbit.com and www.jawbone.com 3 © 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. www.apple.com/watch and www.microsoft.com/microsoft-band 4 ISBN 978-1-4503-5620-6/18/04. $15.00 www.polar.com DOI: https://doi.org/10.1145/3173574.3173719 appropriateness of the aforementioned wearables, we perform commonly rely on using optical Photoplethysmography (PPG) the following research activities: sensors to extract heart rate from peaks in the blood flow under the skin [68]; this happens predominantly on the wrist. De- 1. Comparing accuracy in physiological data measured by the pending on the placement of the optical sensor, there is a time different wearable technologies delay in the detected peak in blood flow caused by a heart beat 2. Examining how physiologically measured stress is affected called Pulse Transit Time (PTT). This can potentially lead to under stationary and physical activity errors in beat detectiion; still, various studies compared both 3. Investigating correlations between subjective measures and technologies for their ability to detect heart beats and heart physiological data beat intervals and found good correlation in the ECG gold 4. Introducing a Design Space for Measurement Tools, reflect- standard and PPG [42, 49, 60]. ing four dimensions which are important to consider for the choice of measurement technology in research contexts Instead of considering the beat-per-beat detection, various studies focused on comparing the reported heart rate; there is RELATED WORK evidence that PPG devices show a decline in accuracy com- Our research addresses the feasibility of consumer smart wear- pared to the gold standard with increased physical activity ables for research settings with a focus on stress assessment. and heart rate [35, 47, 68]. Further, Spierer et al. [65] found We therefore provide background information on physiologi- particular differences in heart rate agreeability depending on cal and subjective perceived measures to detect stress and how skin pigmentation between the two wrist-worn devices Mio wearables can provide sufficient data in this field. Alpha and Omron HR500U; while sensitive skin types (Type II on Fitzpatrick Scale [24]) showed similar low error rates, the Physiological Data and Stress error rates significantly increased for the Mio Alpha (for skin The human body is a complicated and continuously working type V). These findings highlight a manufacturer dependent system. We receive many physiological responses indicating variation in accuracy. stress; we breath faster, blood pressure and pulse rate increase, and we begin to sweat, to name only a few indicators. These Subjective Measures as Indicators for Stress reactions are attributable to the activation of the sympathetic Apart from the physiological indicators of stress based on nervous system which autonomously triggers a series of phys- the sympathetic nervous system’s reactions, there are self- iological changes [14, 64]. Those changes can be picked up rating measures to assess stress. While in the beginning of by sensors to make stress predictions. the research established around stress, subjective assessment The heart rate signal, as an indicator of physiological changes, methods alone were used [13], later questionnaires have been has been used as in various studies among different disciplines applied as ground truth measures to compare against other such as medicine [27], psychology [23, 58], and HCI [44] due measures, i.e. physiological sensors. Kramer [38] argues that to its sufficient reliability and data richness. Electrodermal physiological sensors captures changes that can be monitored activity (EDA), which is mostly referred to as the activation of within seconds whereas subjective rating of one’s stress level sweat glands and hence can be called Galvanic Skin Response only provide snapshots. On the contrary, subjective assessment (GSR) or skin conductivity [15], can be found in prior work as are tools easy to operate for participants and experimenters. an indicator for cognitive load [62], stress [30] and also as a One tool to assess affective states is the Self-Assessment "predictor of emotional responses to stressful life events" [46]. Manikin (SAM) [12]. This tool quickly and reliably collects As a third measure, we choose skin temperature due to its good the participants’ perception of their moods on three dimen- prediction ability indicating stress [36] through significant sions: arousal, valence and dominance. Its values have been changes in body temperature [71]. shown to match emotional and stress responses [25, 51] and the responses were found to be cross-cultural observable [45]. Assessing Physiological Data through Wearables Recently, fitness trackers and smart watches became in- COMPARING DIFFERENT SENSING TECHNOLOGIES creasingly popular and ubiquitous. While early devices fo- In this work, we aim to validate devices with 3 different heart cused merely