Management Faculty Publications Management 2014 Using Noninvasive Brain Measurement to Explore the Psychological Effects of Computer Malfunctions on Users during Human-Computer Interactions Leanne M. Hirshfield Syracuse University Philip Bobko Gettysburg College Alex Barelka Illinois State University See next page for additional authors Follow this and additional works at: https://cupola.gettysburg.edu/mgmtfac Part of the Applied Behavior Analysis Commons, Cognition and Perception Commons, Cognitive Psychology Commons, Computer Sciences Commons, and the Management Sciences and Quantitative Methods Commons Share feedback about the accessibility of this item. Hirshfield, Leanne M., et al. "Using Noninvasive Brain Measurement to Explore the Psychological Effects of Computer Malfunctions on Users during Human-Computer Interactions." Advances in Human-Computer Interaction (2014), Article ID 101038. This is the publisher's version of the work. This publication appears in Gettysburg College's institutional repository by permission of the copyright owner for personal use, not for redistribution. Cupola permanent link: https://cupola.gettysburg.edu/mgmtfac/19 This open access article is brought to you by The uC pola: Scholarship at Gettysburg College. It has been accepted for inclusion by an authorized administrator of The uC pola. For more information, please contact [email protected]. Using Noninvasive Brain Measurement to Explore the Psychological Effects of Computer Malfunctions on Users during Human-Computer Interactions Abstract In today’s technologically driven world, there is a need to better understand the ways that common computer malfunctions affect computer users. These malfunctions may have measurable influences on computer user’s cognitive, emotional, and behavioral responses. An experiment was conducted where participants conducted a series of web search tasks while wearing functional nearinfrared spectroscopy (fNIRS) and galvanic skin response sensors. Two computer malfunctions were introduced during the sessions which had the potential to influence correlates of user trust and suspicion. Surveys were given after each session to measure user’s perceived emotional state, cognitive load, and perceived trust. Results suggest that fNIRS can be used to measure the different cognitive and emotional responses associated with computer malfunctions. These cognitive and emotional changes were correlated with users’ self-report levels of suspicion and trust, and they in turn suggest future work that further explores the capability of fNIRS for the measurement of user experience during human-computer interactions. Keywords computer, technology, malfunctions, human response Disciplines Applied Behavior Analysis | Business | Cognition and Perception | Cognitive Psychology | Computer Sciences | Management Sciences and Quantitative Methods | Psychology Authors Leanne M. Hirshfield, Philip Bobko, Alex Barelka, Stuart H. Hirshfield, Mathew T. Farrington, Spencer Gulbronson, and Diane Paverman This article is available at The uC pola: Scholarship at Gettysburg College: https://cupola.gettysburg.edu/mgmtfac/19 Hindawi Publishing Corporation Advances in Human-Computer Interaction Volume 2014, Article ID 101038, 13 pages http://dx.doi.org/10.1155/2014/101038 Research Article Using Noninvasive Brain Measurement to Explore the Psychological Effects of Computer Malfunctions on Users during Human-Computer Interactions Leanne M. Hirshfield,1 Philip Bobko,2 Alex Barelka,3 Stuart H. Hirshfield,4 Mathew T. Farrington,4 Spencer Gulbronson,4 and Diane Paverman4 1 M.I.N.D. Lab, Department of Communications, SI Newhouse School, Syracuse University, Syracuse, NY 13244, USA 2 Departments of Management and Psychology, Gettysburg College, Gettysburg, PA 17325, USA 3 Department of Management and Quantitative Methods, Illinois State University, Normal, IL 61761, USA 4 Department of Computer Science, Hamilton College, Clinton, NY 13323, USA Correspondence should be addressed to Leanne M. Hirshfield; [email protected] Received 20 September 2013; Accepted 17 March 2014; Published 30 April 2014 Academic Editor: Francesca Morganti Copyright © 2014 Leanne M. Hirshfield et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. In today’s technologically driven world, there is a need to better understand the ways that common computer malfunctions affect computer users. These malfunctions may have measurable influences on computer user’s cognitive, emotional, and behavioral responses. An experiment was conducted where participants conducted a series of web search tasks while wearing functional near- infrared spectroscopy (fNIRS) and galvanic skin response sensors. Two computer malfunctions were introduced during the sessions which had the potential to influence correlates of user trust and suspicion. Surveys were given after each session to measure user’s perceived emotional state, cognitive load, and perceived trust. Results suggest that fNIRS can be used to measure the different cognitive and emotional responses associated with computer malfunctions. These cognitive and emotional changes were correlated with users’ self-report levels of suspicion and trust, and they in turn suggest future work that further explores the capability of fNIRS for the measurement of user experience during human-computer interactions. 1. Introduction human-computer interactions. For example, the NASA-TLX is one of the most commonly used surveys for assessing As the lines between humans and computers become blurred, workload [1]. As another example, when attempting to research is beginning to measure users’ experiences while measure self-report emotional states, users often complete interacting with a technology or information system. In par- surveys such as semantic differentials, the Self-Assessment ticular, dealing with computer malfunctions that involve slow Manikin, or the Positive and Negative Affect Schedule2 [ , internet connectivity, or those that involve the introduction 3].Thevastmajorityoftrustresearchtodatehasalso of malware onto one’s computer system, has unfortunately relied on surveys to assess people’s trust in others [4, 5]. become a somewhat regular part of users’ interactions with Although this method of measurement is commonplace and computers. Computer users’ cognitive load and emotional valuable for understanding and measuring changes in user state may change depending on the type and severity of states, it is limited by many of the well-known drawbacks of the malfunction. Additionally, one’s perceived trust in, or subjective, self-report measures. For example, subjects may suspicion of, the computer system may be correlated with have different frames of reference when completing surveys; these changes in cognitive load and emotional state. further, survey responses correlate only moderately with It is commonplace to use surveys to acquire users’ self- actual behavior and/or others’ perceptions of the subject’s reports of their cognitive load and psychological states during behavior [6]. Also, subjects’ use of rating scales is prone to 2 Advances in Human-Computer Interaction distortion due to social desirability [7], and surveys and self- reports are often administered after a task has been completed (postdictively). They are thus limited in their capacity to accurately collect valuable insight into the users’ changing experiences throughout a task. To compensate for the shortcomings of subjective, self- report techniques, in this study we use noninvasive brain measurement techniques to measure changes in user states objectively and in real time. Such measurement techniques have emerged in the literature with fMRI and the elec- troencephalograph (EEG) being used to measure workload and emotional states in human-computer interactions [8– 14]. Furthermore, some researchers have recently used fMRI Figure 1: A subject wearing a 52-channel fNIRS device. andassociatedbrainactivitytomeasureaspectsoftrustand distrust [15, 16]. Although fMRI provides valuable informa- tion about cognitive functioning in the brain, the device is area in the brain that activates when a person is experiencing quite constricting. It requires subjects to lie still in a large mental workload. However, these researchers look at specific magnet and is extremely expensive [17, 18]. Although fMRI areas in the brain to see which areas are activated while results suggest that we can measure trust objectively by subjects perform simple tasks [23–25]. We have used fNIRS assessing brain functioning, the tool cannot be used outside to measure spatial and verbal working memory load, as well the research lab, limiting its uses for monitoring trust in more as response inhibition load and visual search load [11, 12]. operational, real-world situations. Regarding emotional reactions, most researchers agree In order to enable the measurement of cognitive load, that emotions are affective states that exist in a short period emotion, and the correlated constructs of trust and suspicion of time and are related to a particular event [26, 27]. From a in real-world contexts, we employed a new, noninvasive psychological point of view, emotions are often mapped to brainsensingtechniquecalledfunctionalnearinfraredspec- points in a two-dimensional space of affective valence and troscopy (fNIRS) to make real-time, objective measurements arousal. Valence