Retention and Transfer of Cognitive Bias Mitigation Interventions: A Systematic Literature Study J.E. (Hans) Korteling1, Jasmin Y.J. Gerritsma1, and Alexander Toet*1 1 TNO Human Factors, Soesterberg, Netherlands * Correspondence: Alexander Toet E: [email protected] Word count: 6428 1 2 Abstract 3 Cognitive biases can adversely affect human judgement and decision making and should therefore 4 preferably be mitigated, so that we can achieve our goals as effectively as possible. Hence, numerous bias 5 mitigation interventions have been developed and evaluated. However, to be effective in practical 6 situations beyond laboratory conditions, the bias mitigation effects of these interventions should be 7 retained over time and should transfer across contexts. This systematic review provides an overview of the 8 literature on retention and transfer of bias mitigation interventions. A systematic search yielded 41 studies 9 that were eligible for screening. At the end of the selection process, only seven studies remained that 10 adequately studied retention over a period of at least 14 days (six studies) or transfer to different tasks and 11 contexts (two studies). Retention of bias mitigation was found up to 12 months after the interventions. 12 Most of the relevant studies investigated the effects of bias mitigation training with specific serious games. 13 Only two studies (the second one being a replication of the first) found evidence for a transfer effect of bias 14 mitigation training across contexts. Our main conclusion is that there is currently insufficient evidence for 15 the retention and transfer of bias mitigation effects to consider the associated interventions as useful tools 16 that can help people to make better decisions in daily life. This is in line with recent theoretical insights 17 about the hard-wired neural and evolutionary origin of cognitive biases. 18 19 Keywords: cognitive biases, bias mitigation, retention, transfer of training, teaching, training interventions, 20 neural networks, brain, evolution, systematic literature study 21 22 23 Introduction 24 25 People constantly form judgments and make decisions, both consciously and unconsciously, without 26 certainty about their consequences. The decision to take another job, invest in company, or to start a 27 relationship, is generally made without knowing beforehand how internal and contextual success-factors 28 will develop, or what will happen when these decisions are really carried out. When making these kinds of 29 practical decisions our thinking is characterized by systematic distortions and aberrations, violating the 30 rules of logic and probability. These violations are manifested in all kinds of cognitive biases [1-4]. Cognitive 31 biases can be generally described as systematic and universally occurring tendencies, inclinations, or 32 dispositions that skew or distort decision making processes in ways that make their outcomes inaccurate, 33 suboptimal, or simply wrong [5, 6]. Biases occur in virtually the same way in many different decision 34 situations [7-9]. They distort our thinking in very specific ways, are largely implicit and unconscious, and 35 feel quite natural and self-evident [8, 10]. That is why they are often termed “intuitive” [11], “irrational” 36 [9], or a-rational [12, 13]. We tend to notice biased reasoning in others more than in ourselves and we 37 typically feel quite confident about our decisions and judgments, even when we are aware of our cognitive 38 biases and when there is hardly any evidence to support them [10, 14]. The robust, pervasive and 39 persistent cognitive bias phenomenon is extensively described and demonstrated in the psychological 40 literature [7-9, 15, 16]. Some well-known biases are: Belief bias (the tendency to base the power or 41 relevance of an idea on the credibility of the conclusion instead of on the argument: [17]), Confirmation 42 bias (the tendency to select, interpret, focus on and remember information in a way that confirms one’s 43 preconceptions, views, and expectations: [18]), Fundamental attribution error .(the tendency to 44 overestimate the influence of personality, while underestimating the importance of situational factors 45 when explaining events or behaviors of other people: [19, 20]), Hyperbolic discounting (the tendency to 46 prefer a smaller reward that arrives sooner over a larger reward that arrives later: [21]), Outcome bias (the 47 tendency to evaluate a decision based on its outcome rather than on what factors led to the decision: [22]), 48 Representativeness bias (the tendency to judge the likelihood of an entity by the extent to which it 49 ‘resembles the typical case’ instead of by its simple base rate: [23]), the Sunk-cost fallacy (the tendency to 50 consistently continue a chosen course or investment with negative outcomes rather than alter it: [24]) or 51 Social proof (the tendency to copy the actions and opinions of others [25]. 52 53 Biased thinking can result in outcomes that are quite acceptable in everyday situations, especially when the 54 time and processing cost of reasoning is taken into account [26, 27]. This is for example the case under time 55 pressure or when relevant or available information is too extensive or detailed, or when no optimal 56 solution is evident (e.g., “Bounded Rationality”; [27]). In these cases, we use pragmatic decision routines 57 (‘heuristics’), characterized by a high ratio of benefits to cost in terms of the quality of the outcomes 58 relative to invested time, effort, and resources [28]. However, biased or heuristic reasoning often leads to 59 outcomes that deviate from what may be considered optimal, advisable, or utile (in relation to our personal 60 objectives). These deviations are also not random, but very specific and systematic: in a wide range of 61 different conditions, people show the same, typical tendencies in the way they pick up and process 62 information to judge and decide. This applies to (almost) everybody, at all levels and in all parts of society, 63 not only in our daily life, but also in professional institutions like politics, government, business, and media. 64 This may have substantial practical consequences, for example in the context of corporate government or 65 policymaking when decision making is often very complex with far-reaching consequences [29-31]. For 66 example, in policymaking the outcomes of a plan can be ‘framed’ in terms of gains, which leads to a 67 preference of risk avoidance, whereas framing the outcomes in terms of losses can lead to risk seeking [6, 68 32, 33]. This means that people base their decisions on the way a problem is formulated rather than on its 69 mere content. Besides framing, the number of choice alternatives can influence decision making [29]. 70 Decision makers prefer the status quo when the number of alternatives is high (Status quo bias: the 71 tendency to prefer the current state of affairs; [34]). These are just a few practical examples of many 72 factors that have been shown to systematically affect the choices people make [7]. 73 74 Mitigating cognitive biases may lead to better decision making on all levels of society, which could 75 substantially promote long-term human wellbeing. Substantial research has already been conducted as to 76 whether and how cognitive biases can be mitigated. Merely teaching knowledge on the existence and 77 nature of cognitive biases has appeared insufficient to mitigate them [35]. Therefore, more elaborate 78 training methods and tools for debiasing have been developed, where people are intensively educated and 79 trained how to mitigate one or more cognitive biases [36]. One method is to ask people to consider why 80 their initial judgments could be wrong. This strategy is called “consider the opposite” and has been shown 81 to reduce various biases [37, 38]. Most studies investigate the bias mitigating effect just after finishing the 82 training while using the same type of tasks that were also used during the training [39-41]. However, to be 83 truly effective in real life, achieved effects of cognitive training should lead to enduring changes in the 84 decision maker’s behavior and should generalize toward other task domains or training contexts [42]. This 85 means that the retention of training should be high, and the bias mitigation effects should last for a longer 86 time than just a brief period immediately after the training. It cannot simply be assumed that an 87 intervention that is effective right after training, will still be effective at a later time [43]. In addition, to 88 have practical value, the bias mitigation effects should transfer to real-life decision situations beyond the 89 specific training environment or context. So, people should be able to apply what they have learnt in a bias 90 mitigation training to more than just one specific type of problem or situation. To date, there is no study 91 that systematically reviews and analyzes the retention and transfer effects of bias mitigation interventions. 92 Therefore, the present systematic literature study provides an overview of the available studies on 93 retention and transfer of bias mitigation interventions. 94 95 Methods 96 Protocol and systematic search Four databases were used: Scopus, Web of Science, PubMed and 97 Psychinfo. For the search, two elements were of interests: debiasing (title/abstract) and retention or 98 transfer (all fields). These elements were adapted to the respective databases. For example, the search 99 string used in Scopus was: (title-ABS-key (“bias mitigation” OR debiasing) AND ALL(retention OR transfer)). 100 The field specifications were made to search for papers that deal with retention and transfer, within the 101 more general topic of debiasing. 102 103 Inclusion and exclusion criteria Studies where included if they investigated the effectiveness of cognitive 104 bias mitigation interventions. Cognitive biases had to be investigated explicitly, while studies investigating 105 an improvement of performance in a broader sense were excluded [42, 44].