A Mobile Application for Data Acquisition from Wearable Devices in Affective Computing Experiments∗

BandReader – A Mobile Application for Data Acquisition from Wearable Devices in Affective Computing Experiments∗

Krzysztof Kutt AGH University of Science and Technology [email protected]

Grzegorz J. Nalepa AGH University of Science and Technology [email protected]

Barbara Gizycka˙ AGH University of Science and Technology [email protected]

Paweł Jemio{o AGH University of Science and Technology [email protected]

Marcin Adamczyk AGH University of Science and Technology

Abstract

As technology becomes more ubiquitous and pervasive, special attention should be given to human-computer interaction, especially to the aspect related to the emotional states of the user. However, this approach assumes very speciﬁc mode of data collection and storage. This data is used in the affective computing experiments for human emotion recognition. In the paper we describe a new software solution for mobile devices that allows for data acquisition from wristbands. The application reads physiological signals from wristbands and supports multiple recent devices. In our work we focus on the Heart Rate (HR) and Galvanic Skin Response (GSR) readings. The recorded data is conveniently stored in CSV ﬁles, ready for further interpretation. We provide the evaluation of our application with several experiments. The results indicate that the BandReader is a reliable software for data acquisition in affective computing scenarios.

I. IntroductionDRAFTeach user, mobile systems offer various methods of personalization and customization of eople nowadays are getting more and these devices. However, as they become more more accustomed to technology that is and more miniature, they can be ﬁt to pieces Ppervasive and ubiquitous. Besides using of hardware of a size of an accessory. Today’s computers at work and at home, many own at wearable technologies, such as wristbands, pro- least one mobile device, be it a smartphone or vide a number of advanced sensors to moni- a tablet. To address individual preferences of tor human activity and health condition. Fur- thermore, these devices could be used for de- ∗The paper is supported by the AGH University Grant.

1 BandReader livering data on human physiological states choice of words for slogans, titles, etc. How- for affective computing applications. Affec- ever, as technology is becoming more advanced tive Computing (AfC) [1] is a new computing and pervasive, a prescribed design begins to paradigm that has been gaining a lot of inter- lag behind the possibilities brought by the po- est recently. Affective computing systems can tential modern conveniences. An important use a range of signals for the classification of aspect of AfC systems is their ability to gather a person’s emotional state. Physiological signal data, process it and generate an answer dynam- monitoring is one of the commonly considered ically, and in real time – a mechanism widely cases. For many AfC systems it is important known as an affective loop. This basic concept to use non-intrusive and pervasive technolo- of AfC applications refers to the continuous gies. Therefore, wearable devices, including chain of interactions between the user and the wristbands, seem a promising solution. software. As a response to the user’s oper- Emotional aspect of human-computer inter- ations in the software, the system generates action is crucial for the process to have a nat- new content or modifies its characteristics, by ural and easy flow [2]. A number of studies means of colors, animations, or more subtle on usability1 has proven that careful system changes in the parameters. Then, the user can design, mindful of user’s emotional states and react to these new conditions, and so on. In responses has a great impact on the overall order to implement this in the application, how- user’s performance with the application. Our ever, one needs to adapt a specific approach tool can simplify the implementation of practi- and be aware of several issues that arise. cal AfC software on mobile devices, while also One difficulty in this context regards proper taking the affective aspect of interaction into data acquisition. Oftentimes, AfC systems use consideration. multiple sensors or complex platforms con- The rest of the paper is organized as fol- sisting of various devices. To ensure reliable lows: In Section II we stress the affective loop’s data interpretation, it is reasonable to con- role in HCI, In Section III we describe selected sider signals from more than one modality – hardware. Then in Section IV we specify our by which we mean both audiovisual data and application. In Section V we provide an experi- other user’s behavior metrics. Another prob- mental evaluation of our solution and situate lem is related to the temporal resolution of our approach in the context of other studies in data handling. Collecting affective information Section VI. Finally, in Section VII we summa- from the environment in asynchronous man- rize the paper and give directions for future ner and processing it offline is not sufficient work. for applying the affective loop [3] in the interaction. Synchronization and integration of II. Affective Loop in multimodal and multichannel data is therefore Human-Computer Interaction a grave issue. From the practical point of view, in order Recent methods in HCI emphasize the use of to create AfC systems that support wristbands information regarding human emotion on the as input devices, specific software solutions interface and interactionDRAFT level. The so-called need to be provided. In our work, we are in- emotive interfaces aim at such an interface de- terested in ubiquitous solutions that integrate sign that addresses the user’s affective state wearable sensors with mobile technologies. To directly, with use of various emotion inducing acquire and process data from wristbands, ded- techniques such as colors, photographs of hu- icated mobile applications have to be devel- man facial expressions of emotions, and careful oped. These applications need to address several of important requirements, such as oper- 1Usability refers to the characteristics of software that determine the degree of ease and efficiency with which ating in real time, or the support for different the application is used. hardware solutions. In this paper we present

2 BandReader such an application that we developed and ments are not supported by any satisfactory successfully evaluated in a number of AfC ex- software enabling to collect, store and mean- periments. BandReader is an important step in ingfully interpret data. satisfying these concerns, as it offers software Xiaomi Mi Band 2 is a simple band aimed at support for wearable devices that provide phys- fitness and sleep tracking. It is equipped with, iological data, which is necessary to develop among others, an optical HR sensor and a step the affective loop. counter. Xiaomi Mi Band 2 allows to check one’s HR at a certain point of time and shows III. Selected Wristbands for it on its display. In order to track it continu- ously, though, the device has to maintain the Affective Computing connection with the official smartphone soft- Analysis of current market state led us to selec- ware, Mi Fit. The full functionality is brought tion of bands that could be useful for presented by using the band with one or both of the devel- applications: Empatica E4, Microsoft Band 2, oper’s applications: Mi Band Notify & Fitness Xiaomi Mi Band 2, Apple Watch 3 and Scosche and Mi Band Tools. Rhythm+. All of the devices listed here pro- Apple Watch 3 is a brand new technology for vide at least a certain mode of HR monitoring, smartwatches. This wristband is essentially the which forms a basic condition from pervasive only smartwatch we included in our analysis. AfC development point of view. Considering The integrated components include accelerom- fitness tracking appliances developed by Fit- eter, barometric altimeter, and ambient light, bit, Garmin, Polar, Xiaomi or Apple, contin- gyro and HR sensors. With the preloaded Ap- uous HR measurement is provided in each ple watchOS operation system and Siri soft- case. Some companies, such as Scosche, de- ware, unlike Mi Band 2, Rhythm+ or E4, it is cided to focus just on rough cardiovascular meant to be a fully-developed smartwatch that activity monitoring. Few developers, namely supports SMS messaging, email, push mail and Empatica E4 and Microsoft Band 2, enhanced IM. their wristbands with more complex biomet- Scosche Rhythm+ is a HR sensor for exercise rics tracking. Both devices include Electroder- intensity measurement. The distinctive feature mal Activity (EDA) or GSR sensors, which aid for this device, as claimed by the developer, the user’s feedback with more comprehensive is its unmatching accuracy provided with the physiological body state overview. Wristbands application of Green/Yellow Optical Sensors are a solution that provides a compromise be- technology. Another notable feature is its dual- tween convenience and capability. This is not, mode processor, which allows simultaneous however, sufficient for achieving decent level data transmission between the Rhythm+, mul- of affect recognition and enhancing interaction. tiple ANT+ displays (such as sport equipment Easy and user-friendly body activity tracking at fitness centers) and the smarthphone app comes with numerous disadvantages regard- (paired via Bluetooth SMART) as well. The ing unusual use cases. Collected data is of- device, however is not equipped with any sort ten preprocessed before being presented to the of display or screen, so any real-time data mon- user. Therefore, the informationDRAFT that the user itoring has to be performed with external soft- acquires is already interpreted in a specific, ware. undocumented way. This is unfortunate for re- Empatica E4 is an advanced sensory wrist- searchers who would prefer access to raw data. band based on the technologies previously de- Information processing should be fully control- veloped in the Affective Computing division lable in order to apply best models for further of MIT Media Lab. The device is equipped interpretation. In the following paragraphs we with Blood Volume Pulse (BVP) and Electro- show that most of currently purchasable (from dermal Activity (EDA) sensors, as well as an everyday user perspective) biometrical instru- accelerometer and an infrared thermometer.

3 BandReader

To benefit from full capability of this device, perspective of the everyday user, Apple Watch the connection with user’s smartphone and of- provides exhaustive statistical information like ficial app via low energy Bluetooth Smart is amount of burned calories, sleep length or necessary. In order to access the detailed infor- training progress. On the other hand, the de- mation, assembled in the CSV file generated vice is supported only by iOS and thanks to by the app, the user needs to download and that its accessibility is restricted. install the Empatica Manager software. Rhythm+ comes with no official mobile, PC Microsoft Band 2 (MS Band 2), was designed or Web app, but there are more than one hun- as a wearable device for health, fitness, and dred unofficial apps available in Google Play basic everyday activity purposes. The sensors or App Store that support it and allow activity on board include barometer sensor, gyrometer, tracking. The device uses Bluetooth SMART GPS, accelerometer, capacitive sensor, optical and ANT+ protocols, so the user can connect HR monitor, UV sensor, ambient light sensor, it the application of choice, including other fit- and GSR sensor. ness watches, bike computers and fitness equip- The software provided by the developers to- ment. Producers of Scosche Rhythm+ provide gether with each commercially released wrist- developers with API support and Fitness Util- band is designed in a way to support the ev- ity app for iOS to test third-party apps. This eryday user. In consequence, each feature is software can be used for accessing raw data devised in a way that balances the efficiency, from the band and generating real-time charts. on the developer’s end, and user-friendliness, Despite having a dual-mode processor, this on the consumer’s part. While the specific band is not equipped with any internal stor- hardware itself is often theoretically capable age, nor does the developer allow for direct, of providing reliable data, its full capacity is raw data download. restricted by the software that processes and Data gathered in the Empatica E4 internal interprets the signals before presenting them memory can be easily imported through USB to the user. Based on the available software and stored in the developer’s cloud platform overview, we provide a critique of the present (E4 Connect) using a software tool, E4 Manager. alternatives and propose a set of features that In turn, the platform, alongside with mobile would comprise to the application suitable for app (E4 Realtime app), enables an overview of various developers needs. the data. The user is able to inspect realtime Xiaomi Mi Band 2 comes along with the Mi charts of gathered data, check status of battery Fit mobile app, which can be used both on An- and duration of current session. After upload- droid and iOS. It allows for band management ing data to the cloud platform, the processed and visualisation of historical data. Unfortu- data and charts can be downloaded anytime nately, there is no open API or SDK available from any device. Nonetheless, the quality of for developers. However, there are successful data provided and, hence, amount of informa- attempts to “hack” the transmission done via tion that can be derived from it, is erroneously Bluetooth and gather the data sent from the limited and flawed. band by accessing proper bytes of raw signal. MS Band 2 is supported by most mobile de- Many values can be accessedDRAFT this way, which vices. In order to use its full functionality, it has been demonstrated in previous work done is mandatory to pair it via Bluetooth with the by our team [4]. mobile Microsoft Band app. Similarly to Em- The first thing that makes Apple Watch 3 patica, all of the data collected by MS Band unique is that it comes with its own opearting 2 is stored in the cloud and can be accessed system called Watch OS. It can be easily ex- using the provided software. After connect- tended using external applications. Apple pro- ing to Microsoft Account, the user is able to vides developers with full support of API and see her activiy history, including burned calo- Software Development Kit (SDK). From the ries (based on amount of steps), track sleep

4 BandReader activity and record changes in weight. Using 7. The data should be available for other ap- Microsoft Health Web app, the user can access plications that run on the same device, a lot of statistical information, which is based i.e. there should be a service that feeds on the processed data. Raw data collected by clients (other applications) with the cur- MS Band 2 is practically unaccessible for the rent stream of data. user. 8. The application should output raw data gathered from wristband(s). IV. Design and Implementation of BandReader .csv .csv .csv .csv i. Requirements for a new application DataStreamer

Review of current software status and our pre- <> vious preliminary research in AfC ﬁeld [5, 6] WristBandService

<> led us to speciﬁcation of a set of use cases and <> requirements for the software to be a convenient asset for studies: WristBand WristBandFragment

1. The technology should introduce modular EmpaticaBand EmpaticaFragment MSBand MSBandFragment architecture to be easily expanded with

new devices support and new functionali- <> <> <> <>

ties. EmpaticaService MSBandService

2. The solution should give a possibility to <> <> EmpaticaAPI MSBandAPI save data from all available sensors for each band. It should take care of the fact that different devices have different sets of Figure 1: BandReader application architecture. sensors.

3. The applications should facilitate recording data from several different wristbands ii. Details of implementation simultaneously, as it is a common experi- Considering the features of described hard- mental scenario. ware solutions, we decided to focus the implementation of the prototype on MS Band 2 4. The software should allow synchroniza- and Empatica E4. tion with other data sources (e.g. proce- Therefore, to ensure the greatest openness, dure markers from PC). Incorporation of availability and the possibility of further inde- Lab Stream Layer system2 should be con- pendent development, the BandReader appli- sidered, as it is a mature solution for this cation was created for the Android Operating issue. System. The developed application was writ- 5. There should be aDRAFT possibility for arbitrary ten in Java language and consists of five main file naming and tagging to provide more modules (see Figure 1): control over data and to facilitate future • WristBand (31 LOC3) is an entity that processing. describes the type of the band and its 6. The data should be saved in convenient possible states (e.g. connecting, connected) file format (such as i.e. CSV). and list of sensors available.

2See: https://github.com/sccn/ labstreaminglayer. 3Lines of Code.

5 BandReader

• DataStreamer (156 LOC) is a module for LoggerFragment responsible for display- handling serialization of data streams to ing the logs collected from DataStreamer CSV files. It converts the data to specified and WristBandService. file format and physically writes the files to smartphone storage. • Utils (4 classes, 192 LOC in total) is a set of additional functionalities (not mentioned • WristBandService (287 LOC) is the main on Figure 1), responsible for, among oth- module of BandReader. It supports ev- ers, tags handling and saving and read- ery stage of communication with the band ing application settings from the phone’s with the use of band definition written memory. down as WristBand entity. Among the important methods of the service there are: – connect() connects with the wristband using the provided API or other available method; – startSubscription() switches the application to listening mode, i.e. it registers callback methods that are called when new data arrives from the band. Current sensor readings are now displayed in the application UI; – startRecording() turns on the recording mode, i.e. received values are not only displayed, but also saved to CSV files; – stream() is a generic method exe- cuted for each new sensor reading. Figure 2: A screencapture of BandReader application It checks the current mode and ex- during the recording session. ecutes the proper actions, i.e. dele- gates data saving to DataSteamer in The modular architecture of the application the recording mode. This method can allows it to be easily extended in the future. To be easily extended in the future to support a new band, one must define three handle more modes of operation, e.g. classes that inherit from the abovementioned synchronization through Lab Stream abstract classes: (a) WristBand, with the defi- Layer system or broadcasting of data nition of band’s states and sensors, (b) Wrist- to other Android applications. BandService, with the implementation of band It is important to note that WristBand- connection and (c) WristBandFragment, with the Service is implemented as real service, UI panel for the user. The range of extension i.e. it receives andDRAFT handles data even if of the application impacts the complexity of BandReader UI is closed. implementation. If it is a high-level change, e.g. adding synchronization through Lab Stream • WristBandFragment (455 LOC) is respon- Layer, it will be associated only with the exten- sible for UI panel for specific wirstband. sion of the appropriate method (e.g. stream()). With the use of methods provided by Low-level functionalities that depend on the WristbandService, it allows user to specific device implementation, may require connect with the band, display the current changes in individual wristband classes. De- status or start recording. There is also pending on the communication methods used

6 BandReader by the wirstbands’ developers, the addition of nates and then presented for users in aggre- a new device may involve a diversified number gated form as a chart of emotions for given of LOC that need to be written in order to be area. able to handle the instrument. In the case of Empatica E4, it is about 500 LOC, and MS Band Emotion as a context information 2 requires as many as 600 LOC. AWARE5 [11] is a framework for build- ing context-aware applications. Together with iii. Use Cases and Current Develop- many other possible sources, like GPS position ment or current battery usage, affective signals may be used as an input. Such a framework, com- BandReader is an important part of mobile sen- bined with the rule-based language designed sor platform we propose as a base for AfC use for inference on uncertain knowledge [8], cases [7, 8]. Various scenarios for BandReader gives a reliable base for development of more in such a setting are currently under develop- sophisticated AfC applications. ment in our team. V. Experimental Evaluation of Data collection in AfC experiments A subject takes part in experiment. She is exposed BandReader to some stimuli and at the same time her phys- The Bandreader application was developed in or- iological reaction is monitored by the Ban- der to support our practical research in the area dReader. Signals are saved and synchronised of affective computing. We used BandReader with stimuli-related markers from experimen- to record data collected by Empatica E4 and tal PC. Such data is then processed during post- MS Band2 wristbands in a research project re- experimental analysis in order to find valuable garding the development of affective design observations about emotional physiological re- patterns in computer games [6]. Thanks to the actions, e.g. in order to prepare models for portabilty of the developed solution, we are emotion prediction based on physiological sig- further extending it to provide the biofeedback nals. loop. Below we describe the most important results of the evaluation. Music recommendations system Ban- dReader gathers live HR and GSR streams while the user is listening to music. As music i. The Experimental Procedure tracks have assigned affective labels in valence The experimental procedure initially com- and arousal space [9], it is possible to train prised of two phases, both performed on a per- a neural network with the use of mobile sonal computer. In order to analyze the phys- 4 TensorFlow library . This model pairs current iological reactions and the stimuli occurences user state with tracks’ affective score and is correlation later, the synchronization between then used to propose music based on current the smartphone (with BandReader application mood. DRAFTrunning) and the computer was maintaned. During the whole study, the subject was wear- Geographical map of affect HR and GSR sig- ing either Empatica E4 or MS Band2 which nals from BandReader are transformed into was paired with BandReader application via Emotional Index [10], a mono-dimensional Bluetooth. In the first phase the subject is pre- measure that reflects more positive (> 0) or sented with affective pictures [12], one by one, more negative (< 0) emotions. This informa- for a fixed amount of time. After each pic- tion is stored along with current GPS coordi- ture, the task is to rate subjectively felt arousal

4See: https://www.tensorflow.org/. 5See http://www.awareframework.com/.

7 BandReader in response to the presented stimuli, on a 7- dReader was used in ”Multiband recording” level scale from 1 to 7. The actual experiment mode, which allows to gather data from two is preceded by a short training session with devices simultaneously. Both phases of pro- neutral images. The second and last phase cedure described above were performed, and involved playing a simple platform game, de- – what is more – an additional one has been signed speciﬁcally for research purposes [6]. prepared. After playing the game, a relaxing The wristband continued to collect the sub- picture was showed to a user, with a very un- ject’s physiological data throughout the game pleasant sound that followed. This step was and the BandReader recorded it for each partic- applied to acquire data on the strong physio- ipant individually. The whole procedure lasted logical response to a sudden affective stimuli about 22 minutes. of the subject. 18 persons participated in the study at that time.

Figure 3: Empatica E4 wristband (on the left), alongside the eHealth platform and the computer used in Figure 4: Bitalino platform, together with GSR and HR one of our studies. electrodes. ii. Summary of Conducted Experi- iii. Evaluation Results ments As a result of the experiments, several obser- So far we conducted two experiments when vations can be stated. First of all, BandReader the procedure involving use of BandReader proved to be a reliable solution that supported was applied. The first experiment took place a connection with Empatica E4 and MS Band 2 in late April 2017 (Eurokreator lab, Krakow, through provided API. It maintained the con- Poland), with 6 subjects in total. In this attempt nection for the duration of the whole experi- though, only the first phase was engaged. The ment, and generated proper data files (CSV). full procedure, consisting of both phases, was The main problems that occurred during the held in June 2017. ThisDRAFT time, the study group experiments were related to the fact that the de- involved 9 persons. vices require special configuration, including In the studies conducted afterwards, specif- repeated pairing attempts before the connec- ically in January 2018 and March 2018, we fo- tion takes place. However, these problems are cused on testing reliablity of data from Em- beyond the scope of BandReader, and are re- patica E4 and MS Band 2 which was recorded lated to the API implementation. with BandReader. We decided to augment our Currently BandReader effectively implements procedure with two new devices: BITalino and the core part of the requirements we have for- eHealth (see Fig. 3 and 4). This time, Ban- mulated (see Sect. IV). The application has

8 BandReader

modular architecture that can be easily ex- trend of mobile and ubiquitous computing. tended (Req. 1). It allows for saving the raw data (Req. 8) from all available sensors (Req. 2) to standard CSV files (Req. 6). These record- VII. Conclusions and Future ing may be done in parallel for many devices Work at once (Req. 3). The application also gives user a possibility to specify the file names by The objective of this paper was to present a new providing identifiers of subjects as well as a prototype software application Bandreader for possibility to add arbitrary tags to data (Req. affective computing experiments. In our work, 5). It can be then concluded that BandReader we are planning the development of ubiquitous is a suitable tool for AfC applications, and our computing systems. As such, we are focusing aim – which was to develop a tool for affec- on the use of mobile devices for data storage tive physiological data acquisition and storage and processing and wearable devices for prov- – seems to be achieved. ing practical measurmenets of biomedical signals. In the paper we present an overview of the devices currently available on the market, VI. Related Research which allowed us for selecting the ones most suitable for our work. We provided the de- To date, several other studies involving wear- scription of the design and implementation of able devices for affective computing applica- the application, as well as results of practical tions have been conducted. These regard either evaluation of its operation in our experiments. general overviews of sensors currently avail- As of future work, the application will be able on the market [13, 14, 15, 16, 17], compara- extended to support the third device: Scosche tive validation between wearables and medical- Rhythm+ wristband. In addition, implemen- quality platforms [18, 19, 20] or actual original tation of synchronization with the use of Lab experiments involving, for example, emotion Stream Layer is planned. This library provides detection or recognition [18, 21, 22, 23, 14, 24]. high accuracy methods for combining the data It is notable that some attempts of design- streams from various devices in local network. ing and developing custom measuring devices It will be primarily used to synchronize the worn on wrists are made [24, 25, 20]. data from variuos recording devices (smart- In this article’s context, namely the dedicated phones, PCs and laptops) in conducted experi- data recording software, it seems that not every ments (see also Req. 4). Finally, to support cur- research group considered this issue, since usu- rent (see Section iii) and future AfC use cases, ally information about this matter is lacking in broadcasting mechanism will be implemented the respective papers [17, 25, 19, 24, 26, 21, 15]. (see also Req. 7). As a result, BandReader will Often either, official producer’s applications be considered a middle layer that gathers data are used [13, 23], or custom-developed sen- from devices and provides the streams to other sor is followed by custom data collecting soft- applications, like music recommendations sys- ware [18, 20]. A notable exception is provided tem mentioned before. by [14], who used their custom program for real-time signal previewDRAFT from Empatica E4 Wristband, LG Watch Style and Zephyr Bio- References harness Module. In the light of our state-of-art analysis, we conclude that our application ad- [1] R. W. Picard, Affective Computing. MIT dresses an important niche and successfully Press, 1997. provides reliable tool for data recording and storage from existing off-the-shelf wearable de- [2] N. Thompson and T. McGill, “Affective vices. What is more, as our software is an An- human-computer interaction,” in Encyclo- droid application, it finely fits into the current pedia of Information Science and Technology,

9 BandReader

Third Edition. IGI Global, 2015, pp. 3712– [8] G. J. Nalepa, K. Kutt, and S. Bobek, “Mo- 3720. bile platform for affective context-aware systems,” Future Generation Computer [3] D. Bersak, G. McDarby, N. Augenblick, Systems, 2018, in press. [Online]. Avail- P. McDarby, D. McDonnell, B. McDon- able: https://doi.org/10.1016/j.future. ald, and R. Karkun, “Intelligent biofeed- 2018.02.033 back using an immersive competitive environment,” 2001, paper presented at the [9] J. Grekow, “Music emotion maps in Designing Ubiquitous Computing Games arousal-valence space,” in Computer Infor- Workshop at UbiComp 2001. mation Systems and Industrial Management - 15th IFIP TC8 International Conference, [4] M. Wosik, “Extension of aware plat- CISIM 2016, Vilnius, Lithuania, 2016, form providing support for sensory wrist- pp. 697–706. [Online]. Available: https: bands,” AGH University of Science and //doi.org/10.1007/978-3-319-45378-1_60 Technology, 2017, BSc Thesis. [10] G. Vecchiato, P. Cherubino, A. G. Maglione, M. T. H. Ezquierro, F. Mari- [5] G. J. Nalepa, J. K. Argasinski, K. Kutt, nozzi, F. Bini, A. Trettel, and F. Babiloni, P. Wegrzyn, S. Bobek, and M. Z. “How to measure cerebral correlates of Lepicki, “Affective computing experi- emotions in marketing relevant tasks,” ments in virtual reality with wearable Cognitive Computation, vol. 6, no. 4, pp. sensors. methodological considerations 856–871, 2014. [Online]. Available: http: and preliminary results,” in Proceedings //dx.doi.org/10.1007/s12559-014-9304-x of the Workshop on Affective Comput- ing and Context Awareness in Ambient [11] D. Ferreira, “Aware: A mobile context Intelligence (AfCAI 2016), ser. CEUR Work- instrumentation middleware to collabora- shop Proceedings, M. T. H. Ezquerro, tively understand human behavior,” 2013. G. J. Nalepa, and J. T. P. Mendez, ˙ Eds., vol. 1794, 2016. [Online]. Available: [12] A. Marchewka, Ł. Zurawski, K. Jednoróg, http://ceur-ws.org/Vol-1794/ and A. Grabowska, “The Nencki Affective Picture System (NAPS): Introduction to [6] G. J. Nalepa, B. Gizycka, K. Kutt, and J. K. a novel, standardized, wide-range, high- Argasinski, “Affective design patterns in quality, realistic picture database,” Behav- computer games. scrollrunner case study,” ior Research Methods, vol. 46, no. 2, pp. 596– in Communication Papers of the 2017 610, 2014. Federated Conference on Computer Science [13] F. El-Amrawy and M. I. Nounou, “Are cur- and Information Systems, FedCSIS 2017, rently available wearable devices for activ- 2017, pp. 345–352. [Online]. Available: ity tracking and heart rate monitoring ac- https://doi.org/10.15439/2017F192 curate, precise, and medically beneﬁcial?” Healthcare informatics research, vol. 21, no. 4, [7] K. Kutt, W. Binek, P. Misiak, G. J. pp. 315–320, 2015. Nalepa, and S.DRAFT Bobek, “Towards the development of sensor platform for pro- [14] S. Kye, J. Moon, J. Lee, I. Choi, D. Cheon, cessing physiological data from wearable and K. Lee, “Multimodal data collection sensors,” in Artiﬁcial Intelligence and Soft framework for mental stress monitoring,” Computing - 17th International Conference, in Proceedings of the 2017 ACM International ICAISC 2018, Zakopane, Poland, June Joint Conference on Pervasive and Ubiquitous 3-7, 2018, Proceedings, Part II, 2018, Computing and Proceedings of the 2017 ACM pp. 168–178. [Online]. Available: https: International Symposium on Wearable Com- //doi.org/10.1007/978-3-319-91262-2_16 puters. ACM, 2017, pp. 822–829.

10 BandReader

[15] H. Yates, B. Chamberlain, and W. H. Hsu, A. Chatzitoﬁs, D. Zarpalas, P. Daras, “A spatially explicit classiﬁcation model I. Chouvarda et al., “A computer-assisted for affective computing in built environ- system with kinect sensors and wristband ments,” in Affective Computing and In- heart rate monitors for group classes of telligent Interaction Workshops and Demos exercise-based rehabilitation,” in Precision (ACIIW), 2017 Seventh International Confer- Medicine Powered by pHealth and Connected ence on. IEEE, 2017, pp. 100–104. Health. Springer, 2018, pp. 237–241.

[16] S. Seneviratne, Y. Hu, T. Nguyen, G. Lan, [23] H. Koskimäki, H. Mönttinen, P. Siirtola, S. Khalifa, K. Thilakarathna, M. Hassan, H.-L. Huttunen, R. Halonen, and J. Rön- and A. Seneviratne, “A survey of wearable ing, “Early detection of migraine attacks devices and challenges,” IEEE Communica- based on wearable sensors: experiences tions Surveys & Tutorials, vol. 19, no. 4, pp. of data collection using empatica e4,” in 2573–2620, 2017. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous [17] S. Rank and C. Lu, “Physsigtk: Enabling Computing and Proceedings of the 2017 ACM engagement experiments with physiologi- International Symposium on Wearable Com- cal signals for game design,” in Affective puters. ACM, 2017, pp. 506–511. Computing and Intelligent Interaction (ACII), 2015 International Conference on. IEEE, [24] Y. Tajitsu, “Piezoelectret sensor made from 2015, pp. 968–969. an electro-spun ﬂuoropolymer and its use in a wristband for detecting heart-beat [18] M. Ragot, N. Martin, S. Em, N. Pallamin, signals,” IEEE Transactions on Dielectrics and J.-M. Diverrez, “Emotion recognition and Electrical Insulation, vol. 22, no. 3, pp. using physiological signals: Laboratory 1355–1359, 2015. vs. wearable sensors,” in International Con- ference on Applied Human Factors and Er- [25] Z. Zhang, “Photoplethysmography-based gonomics. Springer, 2017, pp. 15–22. heart rate monitoring in physical activities via joint sparse spectrum reconstruction,” [19] C. McCarthy, N. Pradhan, C. Redpath, IEEE transactions on biomedical engineering, and A. Adler, “Validation of the empatica vol. 62, no. 8, pp. 1902–1910, 2015. e4 wristband,” in Student Conference (ISC), 2016 IEEE EMBS International. IEEE, 2016, [26] Z. Zhang, Z. Pi, and B. Liu, “Troika: A pp. 1–4. general framework for heart rate monitoring using wrist-type photoplethysmo- [20] S. Preejith, A. Alex, J. Joseph, and graphic signals during intensive physical M. Sivaprakasam, “Design, development exercise,” IEEE Transactions on Biomedical and clinical validation of a wrist-based op- Engineering, vol. 62, no. 2, pp. 522–531, tical heart rate monitor,” in Medical Mea- 2015. surements and Applications (MeMeA), 2016 IEEE International Symposium on. IEEE, 2016, pp. 1–6. DRAFT [21] M. Gjoreski, M. Luštrek, M. Gams, and H. Gjoreski, “Monitoring stress with a wrist device using context,” Journal of biomedical informatics, vol. 73, pp. 159–170, 2017.

[22] A. Triantafyllidis, D. Filos, R. Buys, J. Claes, V. Cornelissen, E. Kouidi,