Habituation Longitudinal Field Experiment

How People Habituate to Mobile Security Warnings in Daily Life: A Longitudinal Field Study Jeff Jenkins, Brock Kirwan, Daniel Bjornn, Bonnie Brinton Anderson, Anthony Vance Brigham Young University {jeffrey_jenkins, kirwan, dbjornn, bonnie_anderson, anthony.vance}@byu.edu Abstract Second, past studies did not examine how habituation influences actual warning adherence behavior Research in the fields of information security and in the field but instead used laboratory experiments human–computer interaction has shown that habit- that presented unrealistically high numbers of uation—decreased response to repeated stimula- warnings to participants in a short session. Because tion—is a serious threat to the effectiveness of se- users typically receive security warnings infre- curity warnings. Although habituation is a phe- quently, presenting an artificially high number of nomenon that develops over time, past studies have warnings in a short time is too far removed from only examined this problem cross-sectionally. Fur- real life to be ecologically valid [24]. Consequent- ther, past studies have not examined how habitua- ly, for these reasons, the full extent of the problem tion influences actual security warning behavior in of habituation is unknown. the field. For these reasons, the full extent of the Third, previous research [3; 4] proposed that re- problem is unknown. peatedly updating the appearance of a warning (i.e., We addressed these gaps by conducting a three- a polymorphic warning design) can be effective in week field experiment in which users were natural- reducing habituation. However, their findings were ly exposed to privacy permission warnings as they subject to the same limitations above. Therefore, it installed apps on their mobile devices. We found is not clear (1) whether polymorphic warnings are that (1) users’ warning adherence substantially de- effective over time or if users will quickly learn to creased over the three weeks, validating previous ignore them and (2) whether the polymorphic de- cross-sectional studies, (2) the general decline in sign can actually lead to better security warning warning adherence was partially offset by a recov- behavior. ery effect—a key characteristic of habituation— We address these gaps in this paper by presenting when permission warnings were not displayed be- the results of a longitudinal three-week field exper- tween days, and (3) for users who received poly- iment in which users were naturally exposed to morphic permission warnings—warnings that up- privacy permission warnings as they installed apps date their appearance with each repeated expo- on their mobile devices. Consistent with previous sure—adherence dropped at a substantially lower cross-sectional experimental results, users’ warning rate and remained high after three weeks compared adherence behavior substantially decreased over to users who received standard warnings. the three weeks. However, for users who received These findings provide the most complete view yet polymorphic permission warnings, adherence of the problem of habituation to security warnings dropped at a substantially lower rate and remained and demonstrate that polymorphic warnings can high after three weeks compared to users who re- substantially improve warning adherence behavior. ceived standard warnings. Together, these findings Keywords: habituation, security warning, longitu- provide the most complete view yet of the problem dinal field experiment, mobile devices. of habituation to security warnings and demonstrate that polymorphic warnings can substantially 1. Introduction improve warning adherence behavior. Research in the fields of information systems and 2. Literature Review and Theory human–computer interaction has shown that habituation—“decreased response to repeated stimula- Habituation has been identified as a key contributor tion” [26, p.419]—is a serious threat to the effec- to the failure of warnings [14; 19; 20]. Several re- tiveness of security warnings. However, past stud- searchers have inferred warning habituation in ies share three critical limitations. First, they only cross-sectional laboratory experiments [2-4; 8; 11; examined habituation cross-sectionally (see Table 12; 15; 19; 23; 25]. In addition, two studies sup- 1). This is a substantial limitation, because habitua- ported cross-sectional habituation in warning ad- tion is a phenomenon that develops over time [17]. herence behavior using Amazon Mechanical Turk Furthermore, a key characteristic of habituation is [6; 7]. While these studies provide important in- recovery—the increase of a response after a rest sights into the problem of habituation to security period in which the stimulus is absent [17]. With- warnings, they share a fundamental limitation: they out a longitudinal design, it is not possible to exam- only examine a single point in time. However, in ine whether recovery can sufficiently counteract the fields of neuroscience and neurobiology, it is the effect of habituation to warnings. well recognized that the effects of habituation change over time [17]. For this reason, cross- previously lost because of habituation” [28, p. 55]. sectional studies can only provide a partial view of Changing the appearance of a warning creates nov- the effects of habituation. For example, the two elty, and the warning will therefore be less similar most prevalent characteristics of habituation are (1) to existing mental models. As a result of this dis- response decrement—an attenuation of a response similarity, the response strength will recover [22]. after multiple exposures—and (2) response recov- DPT describes this as sensitization, an energizing ery—the increase of a response after a rest period process that strengthens attention [13]. Sensitiza- in which the stimulus is absent [17]. Without a lon- tion counterbalances or decreases habituation [17]. gitudinal design, it is not possible to observe how As a result of sensitization, users will pay closer (or whether) users recover from habituation to attention to warnings and reject risky permissions warnings between exposures. For this reason, it is more accurately. not clear from previous cross-sectional research H2: Users’ accuracy in rejecting risky permission whether response recovery can offset the negative warnings over time will decrease more slowly when impact of response decrements observed in previ- viewing polymorphic warnings as compared to ous habituation research. static warnings. Hypotheses 1 and 2 explore how users become less accurate in rejecting risky permissions over time 2.2 Response Recovery with repeated viewings and how polymorphic Although users will habituate to warnings, we pre- warnings can mitigate this effect. Hypotheses 3 and dict that they will partially recover from the habitu- 4 explore how users’ responses to warnings recover ation after a rest period. Decay theory [5] explains after the warning is withheld and how polymorphic that memory becomes weaker due to the passage of warnings enhance this recovery. time. When a warning is withheld for some time, the mental model of the warning weakens. There- 2.1 Response Decrement fore, when users see a warning in the future, it will We first hypothesize that users’ accuracy in reject- be less likely to match the mental model and will ing risky app permissions will decrease when view- appear novel, increasing sensitization and users’ ing multiple warnings across days. Dual-process attention to the warning [9]. This time between theory (DPT) [13] states that when users see a re- warnings should thus result in an increase in accu- peated stimulus, they compare it to a mental model racy in rejecting risky permissions. of that stimulus. If the two match, users evaluate H3: Time between warnings will improve users’ the actual stimulus less carefully and rely on the accuracy in rejecting risky permission warnings. mental model instead. This is referred to as a “response decrement,” and may result in paying less We predict that the amount of recovery after a time attention and responding less thoughtfully to the period will be greater for polymorphic warnings stimulus. In the context of mobile app permission than for static warnings. As previously discussed, warnings, users will unconsciously compare warn- the mental models of polymorphic warnings are ings to their mental model of warnings they have weaker and less stable than the models of static seen when previously downloading other apps. If warnings. Less stable mental models (i.e., mental users determine that a warning is similar to the models that have not received as much reinforce- mental model (even if, in fact, it lists different per- ment) fade more quickly than stable models [17]. missions), they will give it less attention. In future Thus, after users have not seen a warning for a time exposures, users will rely even more on the model period, they are more likely to perceive the poly- and respond even less thoughtfully. As a result, morphic warning as novel. As a result, the respons- users who view similar permission warnings over es of users who view polymorphic warnings will time will give less attention to them, and habitua- recover to a greater degree than the responses of tion will inhibit the ability to identify and reject users who view static warnings. risky permission warnings. H4: Users’ accuracy in rejecting risky permission H1: Multiple exposures to permission warnings warnings over time will increase more after a over time will decrease users’ accuracy

Load more