Bias Blind Spot: Structure, Measurement, and Consequences

Home , Bias blind spot, Outcome bias, Response bias

City Research Online

City, University of London Institutional Repository

Citation: Scopelliti, I., Morewedge, C. K., McCormick, E., Min, L., LeBrecht, S. and Kassam, K. (2015). Bias Blind Spot: Structure, Measurement, and Consequences. Management Science, 61(10), pp. 2468-2486. doi: 10.1287/mnsc.2014.2096

This is the published version of the paper.

This version of the publication may differ from the final published version.

Permanent repository link: https://openaccess.city.ac.uk/id/eprint/5901/

Link to published version: http://dx.doi.org/10.1287/mnsc.2014.2096

Copyright: City Research Online aims to make research outputs of City, University of London available to a wider audience. Copyright and Moral Rights remain with the author(s) and/or copyright holders. URLs from City Research Online may be freely distributed and linked to.

Reuse: Copies of full items can be used for personal research or study, educational, or not-for-profit purposes without prior permission or charge. Provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way.

City Research Online: http://openaccess.city.ac.uk/ [email protected] This article was downloaded by: [138.40.68.78] On: 21 October 2015, At: 07:31 Publisher: Institute for Operations Research and the Management Sciences (INFORMS) INFORMS is located in Maryland, USA

Management Science

Publication details, including instructions for authors and subscription information: http://pubsonline.informs.org Bias Blind Spot: Structure, Measurement, and Consequences

Irene Scopelliti, Carey K. Morewedge, Erin McCormick, H. Lauren Min, Sophie Lebrecht, Karim S. Kassam

To cite this article: Irene Scopelliti, Carey K. Morewedge, Erin McCormick, H. Lauren Min, Sophie Lebrecht, Karim S. Kassam (2015) Bias Blind Spot: Structure, Measurement, and Consequences. Management Science 61(10):2468-2486. http://dx.doi.org/10.1287/ mnsc.2014.2096

Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions

This article may be used only for the purposes of research, teaching, and/or private study. Commercial use or systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisher approval, unless otherwise noted. For more information, contact [email protected]. The Publisher does not warrant or guarantee the article’s accuracy, completeness, merchantability, fitness for a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, or inclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, or support of claims made of that product, publication, or service. Copyright © 2015, INFORMS

Please scroll down for article—it is on subsequent pages

INFORMS is the largest professional society in the world for professionals in the fields of operations research, management science, and analytics. For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org MANAGEMENT SCIENCE Vol. 61, No. 10, October 2015, pp. 2468–2486 ISSN 0025-1909 (print) ISSN 1526-5501 (online) http://dx.doi.org/10.1287/mnsc.2014.2096 © 2015 INFORMS

Bias Blind Spot: Structure, Measurement, and Consequences Irene Scopelliti Cass Business School, City University London, London EC1Y 8TZ, United Kingdom, [email protected] Carey K. Morewedge Questrom School of Business, Boston University, Boston, Massachusetts 02215, [email protected] Erin McCormick Dietrich School of Humanities and Social Sciences, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, [email protected] H. Lauren Min Leeds School of Business, University of Colorado, Boulder, Colorado 80309, [email protected] Sophie Lebrecht, Karim S. Kassam Dietrich School of Humanities and Social Sciences, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213 {[email protected], [email protected]}

eople exhibit a bias blind spot: they are less likely to detect bias in themselves than in others. We report Pthe development and validation of an instrument to measure individual differences in the propensity to exhibit the bias blind spot that is unidimensional, internally consistent, has high test-retest reliability, and is discriminated from measures of intelligence, decision-making ability, and personality traits related to self-esteem, self-enhancement, and self-presentation. The scale is predictive of the extent to which people judge their abilities to be better than average for easy tasks and worse than average for difﬁcult tasks, ignore the advice of others, and are responsive to an intervention designed to mitigate a different judgmental bias. These results suggest that the bias blind spot is a distinct metabias resulting from naïve realism rather than other forms of egocentric cognition, and has unique effects on judgment and behavior. Keywords: bias blind spot; judgment and decision making; metacognition; self-awareness; advice taking; debiasing History: Received December 5, 2013; accepted September 13, 2014, by Yuval Rottenstreich, judgment and decision making. Published online in Articles in Advance April 24, 2015.

Introduction We show that people differ in their propensity People exhibit numerous systematic biases in judg- to exhibit the bias blind spot, and develop a reli- ment (Tversky and Kahneman 1974, Kahneman et al. able and valid instrument to measure this individ- 1982, Nisbett and Ross 1980), many of which are due ual difference. Bias blind spot appears to be a unique construct independent of intelligence and personal- to unconscious processes (Morewedge and Kahneman ity traits related to self-esteem, self-enhancement, and 2010, Wilson and Brekke 1994). A lack of conscious self-presentation. Bias blind spot appears to be inde- access to judgment-forming processes means that pendent of general decision-making competence. In people are often unaware of their own biases (Nisbett other words, the belief that one is less biased than and Wilson 1977) even though they can readily spot one’s peers appears to reflect a biased perception of the same biases in the judgments of others. Conse- the self rather than a more general inferior decision- quently, most people tend to believe that, on average, making ability, or an accurate reflection of one’s supe- they are less biased in their judgment and behavior rior judgment and decision-making ability. We find Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. than are their peers. Most people recognize that other that the propensity to believe that one is less biased people are likely to be biased when judging an attrac- than one’s peers has detrimental consequences on tive person, for example, but think that their own judgments and behaviors that rely on self–other accu- judgment of an attractive person is unaffected by this racy comparisons, including decreasing receptivity to useful (i.e., debiasing) advice. type of halo effect. Because the majority of people cannot be less biased their peers, this phenomenon is Bias Blind Spot referred to as the bias blind spot (Pronin 2007; Pronin People exhibit a tendency to believe they are less et al. 2002, 2004). biased than their peers. College students believe

they are less biased than their classmates, airline Klayman et al. 1999; Stankov and Crawford 1996, passengers believe they are less biased than other pas- 1997; Stanovich and West 2000). Bruine de Bruin sengers, and Americans believe they are less biased et al.(2007) demonstrated that decision-making com- than their fellow citizens (Pronin et al. 2002). This competence, measured as composite performance on mul- mon asymmetry in the assessment of bias in the self tiple decision-making tasks, reflects the outcome of and in others has been observed across a variety of reliable individual differences and predicts real-world social and cognitive biases. This bias blind spot occurs decision-making ability. whether people explicitly rate the extent of their bias In addition to clarifying the robustness and unique- relative to their peers, or rate the absolute extent of ness of the bias blind spot, a major goal of the present their bias and separately rate the extent of bias exhib- research was to examine its relationship with gen- ited by their peers (Pronin et al. 2002, 2004; Pronin eral decision-making ability. If a single higher-order and Kugler 2007; Epley and Dunning 2000; West et al. construct determines the extent to which people rec- 2012, Van Boven et al. 2000; Wilson et al. 1996). ognize their own bias, three possible relationships The bias blind spot has been attributed to the inter- between the bias blind spot and general decision- play of two phenomena: the introspection illusion and making ability emerge. naïve realism (Pronin et al. 2004, Pronin and Kugler First, people who are lowest in decision-making 2007). The introspection illusion results from differ- ability may be least aware of their own bias, and ences in the availability and perceived diagnostic hence may be most likely to exhibit the bias blind value of introspection when assessing oneself and spot. Indeed, the least skilled are often least able to other people. When people evaluate the extent of assess their level of skill (Kruger and Dunning 1999, bias in their own behavior, they base their evalua- Dunning et al. 2003). If bias blind spot is simply one tions on introspection (e.g., “I find Rodger attractive, component of general decision-making ability, suscep- but I have no memory of that fact influencing how tibility to the bias blind spot should be negatively I judge his speaking ability”). Because introspection correlated with decision-making competence. is unlikely to reveal biased thought processes, people Second, people may accurately assess their degree conclude that their judgment and behavior are unbi- of bias relative to their peers. In other words, people who believe they are less biased than their peers ased. Naïve realism, the belief that one’s perception (those high in bias blind spot) may be correct. In this reflects the true state of the world, then generates a case, bias blind spot may not really reflect suscep- false sense that these charitable self-assessments are tibility to a blind spot at all, and there should be genuine rather than positively biased (Pronin et al. a positive correlation between bias blind spot and 2004; Ross and Ward 1995, 1996). decision-making competence. When people evaluate the extent of bias in others, Third, susceptibility to the bias blind spot may be their assessments rely on behavior rather than on pri- unrelated to general decision-making ability. People vate thoughts, because the private thoughts of others who have not been trained in decision making, and are not accessible. Consequently, the biased behav- are therefore unaware of common biases in reason- ior of others is not excused by an ostensibly unbi- ing and judgment, show considerable variation in ased thought process (e.g., “Joan might not realize their decision-making competence (Bruine de Bruin it, but she probably thinks Rodger speaks eloquently et al. 2007). This suggests that superior decision mak- because she is attracted to him”). Because people use ing is a function of calibrated intuitions rather than different types of evidence when assessing bias in the awareness of appropriate decision-making strategies. self and in other people, less bias is perceived in the Indeed, research on the limitations of introspection self than in others. The variety of social and cogni- shows that people have limited insight into the pro- tive biases for which this bias blind spot has been cesses by which their judgments and decisions are observed (Pronin et al. 2004, West et al. 2012) sug- made (Nisbett and Wilson 1977, Wilson and Dunn gests that susceptibility to the bias blind spot may be 2004). Furthermore, tests comparing bias blind spot a higher-order latent factor underlying the belief that for several specific biases and the commission of one is less likely to exhibit a variety of specific biases those specific biases have revealed little to no rela- Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. than are one’s peers. tion between recognition of bias and its commission (Stanovich and West 2008, West et al. 2012). Bias Blind Spot and Decision Making The propensity to exhibit decision-making biases Predictive Value of the Bias Blind Spot Scale appears to vary systematically (Levin et al. 2002, We suggest that the bias blind spot is weakly, if at Stanovich 1999, West and Stanovich 1997). Several all, related to general decision-making ability. More studies have shown that performance across decision- specifically, that the bias blind spot is a metabias— making tasks tends to have high internal consistency a bias in the recognition of other biases—that sys- (Blais et al. 2005; Bornstein and Zickafoose 1999; tematically biases judgment and behavior in unique Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2470 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

and predictable ways. Rather than indicate an overly report the development of our individual-difference positive view of the self, it reflects the tendency to measure to assess the extent to which a person consider different evidence when making self–other believes she is less biased than her peers, the evalu- accuracy comparisons. Thus, we predict that it should ation of its reliability, the verification of its factorial (a) serve as an index of bias for self–other com- structure, and the analysis of its discriminant valid- parisons that rely on similar cognitive mechanisms, ity in relationship with potentially related constructs whether those comparisons are favorable or unfavor- such as intelligence, cognitive reflection, decision- able to the self. As an index of self–other accuracy making ability, and personality traits related to self- comparisons, we suggest that bias blind spot should esteem, self-enhancement, and self-presentation. Our also influence judgments informed by these (social) last three studies report tests of our three judgmental comparisons such as (b) the weight placed on advice and behavioral predictions. from others and (c) receptivity to debiasing training. Next, we unpack each of these predictions in turn. A first prediction is that when evaluating their Study 1 own abilities, people exhibiting more bias blind spot In Study 1, we generated 27 scale items and then should exhibit more naïve realism in their initial submitted them to a purification process resulting in assessments of their self (e.g., “I’m great at using a 14-item bias blind spot scale with good reliabil- a computer mouse but terrible at juggling”) and ity and stability. We used an item-generation process less often correct those assessment for relevant infor- aimed at capturing a broad sense of the construct to mation about others’ abilities (e.g., “0 0 0 as are most develop the scale. In accordance with the underly- people”). Rather than simply view the self more pos- ing theory (Pronin 2008), we reasoned that the bias itively, they should be more likely to exhibit biases blind spot is a metabias that is exhibited across mul- reliant on similar cognitive mechanisms. For exam- tiple self-assessments. Therefore, we treated the bias ple, they should be more likely to exhibit a better- blind spot as a latent variable that causes an asymme- than-average effect when evaluating their ability to try in the assessment of bias in the self and in others perform easy tasks, and a worse-than-average effect with respect to several biases, both in the social and when evaluating their ability to perform difficult cognitive domains. Accordingly, we approached the tasks (Kruger 1999). conceptualization and the operationalization of sus- A second prediction is that commission of bias ceptibility to the bias blind spot as a reflective mea- blind spot should predict bias in judgments informed surement model (Bollen and Lennox 1991, Edwards by self–other accuracy comparisons. People who think and Bagozzi 2000). That is, a model in which the they are less vulnerable to bias than their peers, for direction of causality is from the construct to the example, may attend less to the opinions of others indicators, and in which changes in the underlying because they believe that others have a more biased construct are hypothesized to cause changes in the perception of the world, consistent with the operation indicators. According to the proposed conceptualiza- of naïve realism (Ross and Ward 1996, Pronin et al. tion, changes in individual susceptibility to the bias 2004, Liberman et al. 2012). This should lead them to blind spot (i.e., a latent variable), will cause changes place less weight on the advice of others when given in the extent to the bias blind spot is observed for the opportunity to incorporate that advice into their specific biases in judgment and behavior. own judgments. As another example, people who think they are Method less vulnerable to bias than their peers should believe that their judgment and decision making is in less Participants. Initial Sample0 A total of 172 Ama- need of correction (Wilson and Brekke 1994). Analo- zon Mechanical Turk (AMT) workers (86 women; gous to interventions aiming to curb addiction, where Mage = 3204 years, SD = 1005) accessed and completed awareness of the problem is a necessary first step a survey on the Internet and received $4 as com- in facilitating corrective action, awareness of bias pensation. In all studies we restricted participation

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. may be an important precursor to bias mitigation to residents of the United States. Participation in any (Kruger and Dunning 1999, Wilson and Brekke 1994). study reported here resulted in ineligibility for par- Consequently, greater commission of bias blind spot ticipation in subsequent studies (i.e., no person parti- should reduce the likelihood of beneﬁtting from bias- cipated in more than one study). Participants were reducing interventions. 79.7% White, 7% Black, 5.2% multiracial, 4.1% Asian, and 1.2% Native American; 2.9% did not indicate their Overview of the Studies ethnicity. AMT workers have been shown to exhibit We adopted a psychometric approach to the anal- susceptibility to biases in judgment and decision mak- ysis of the bias blind spot. Our ﬁrst two studies ing similar to traditional college samples with respect Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2471

to common tasks such as the Asian disease problem, Results the conjunction fallacy and outcome bias; to exhibit Purification0 We first reduced the number of items similar levels of risk aversion; and to exhibit similar to improve the psychometric properties of the instru- levels of cooperation in behavioral economics games ment. We removed items that assessed bias blind such as the prisoner’s dilemma (Berinsky et al. 2012, spot by describing different occurrences of the same Horton et al. 2011, Paolacci et al. 2010). underlying judgmental bias. We then computed the Follow-Up Sample0 A total of 83 participants in the correlations between each item and the rest of the initial sample completed a follow-up survey consist- scale, and removed items with item-to-total correla- ing of the items in the purified scale for an additional tions lower than 0.40. This purification process led to a final instrument containing 14 items (see Table1 for $4 in compensation (43 women; Mage = 3207 years, SD = 1000) yielding a 52% retention rate. Participants the full list of items). in this subsample were 75.9% White, 9.6% Black, Reliability0 The 14-item scale shows high reliabil- 7.2% Asian, 4.8% multiracial, and 1.2% Native Amer- ity ( = 0086), well above the acceptable threshold ican; 1.2% did not indicate their ethnicity. There were of 0.70 (Nunnally 1978). All items appeared to be no differences between the initial sample and the worth retention. No question eliminations would yield follow-up sample in terms of age (F = 0005, p > 0080), a higher value of the Cronbach’s alpha coefficient gender (2 = 1000, p > 0060), or ethnicity (2 = 2041, (-if-item-deletedi < ). Of the 91 pairwise correla- = p > 0070). tions between the items (M 0030), 85 were positive and significant, 3 were positive and marginally signifi- Materials. Item Generation0 We began developing cant, and 3 were positive but not significant (ps < 0042; scale items by identifying existing indicators of the see Table2). Each item correlated well with the scale, construct in the literature. We drew some questions as signaled by an average item-to-total correlation from Pronin et al.(2002) verbatim, modified questions equal to 0.52. All further analyses, in this and subse- from this source, and wrote new questions with the quent studies, use this 14-item scale. For all 14 biases, same structure. Each question was structured so that the majority of participants rated the average Amer- it described a bias and asked respondents to indi- ican as more susceptible to bias than themselves; all cate the incidence of the bias for two different tar- ts ≥ 6026, all ps < 00001 (for all means, see Table3). gets: the self and the average American (for examples, Exploratory Factor Analysis0 We submitted the see Table1). Each bias was described as a psycho- 14 bias blind spot items to a principal-components logical tendency or effect, avoiding (when possible) factor analysis (PCA) followed by a parallel analysis (Horn 1965). The parallel analysis suggests an the use of positively connoted words or negatively underlying single-factor structure, with only the first connoted words such as bias or error. After read- eigenvalue observed in the data being higher than ing a description of each bias, participants rated the a parallel average random eigenvalue based on the extent to which they exhibit that bias and the extent same sample size and number of variables. In the to which the average American exhibits that bias on single-factor model, all 14 items load onto a single 7-point scales with endpoints, not at all (1) and very factor accounting for 35% of the total variance, and much (7). each item has a high correlation with that factor (all Item Scoring0 Bias blind spot scores were calculated s > 0048; see Table3). by subtracting the perceived self-susceptibility to each Test-Retest Reliability0 An analysis of the second ad- bias from the perceived susceptibility of the average ministration of the scale indicated that the instrument American to that bias, for each bias, and then by aver- has consistently high reliability ( = 0088), and high aging those relative differences. For each participant, test-retest reliability (r4815 = 0080, p < 00001), signaling the size of the average difference between self and stability of the bias blind spot scale over time. average American bias ratings was used as a measure of the magnitude of her bias blind spot. Discussion Study 1 provided evidence for individual differences Procedure. Participants rated their vulnerability in susceptibility to the bias blind spot. High inter-

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. and the vulnerability of the average American to item correlations and factors loadings suggest that commit biases on 27 unique items. Item order was some people have a higher susceptibility to the bias random. After completing all 27 items, participants blind spot than do others across a variety of judg- reported their age, gender, and ethnicity. To assess mental biases. The results from this ﬁrst study sug- the test-retest reliability of the instrument, all partic- gest that our instrument reliably assessed individual ipants in the initial sample were invited by means differences in susceptibility to the bias blind spot. In of the AMT messaging system to complete the puri- addition, the instrument captures a single latent vari- ﬁed version of the scale eight days after the initial able as proposed by our construct operationalization. administration. Furthermore, the high test-retest reliability observed Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2472 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

Table 1 Scale Items and Corresponding Biases

Item Description Bias

1 Some people show a tendency to judge a harmful action as worse than an equally harmful inaction. For example, Action-inaction bias this tendency leads to thinking it is worse to falsely testify in court that someone is guilty, than not to testify that someone is innocent. 2 Psychologists have claimed that some people show a tendency to do or believe a thing only because many other Bandwagon effect people believe or do that thing, to feel safer or to avoid conflict. 3 Many psychological studies have shown that people react to counterevidence by actually strengthening their Confirmation bias beliefs. For example, when exposed to negative evidence about their favorite political candidate, people tend to implicitly counterargue against that evidence, therefore strengthening their favorable feelings toward the candidate. 4 Psychologists have claimed that some people show a “disconfirmation” tendency in the way they evaluate Disconfirmation bias research about potentially dangerous habits. That is, they are more critical and skeptical in evaluating evidence that an activity is dangerous when they engage in that activity than when they do not. 5 Psychologists have identified an effect called “diffusion of responsibility,” where people tend not to help in an Diffusion of responsibility emergency situation when other people are present. This happens because as the number of bystanders increases, a bystander who sees other people standing around is less likely to interpret the incident as a problem, and also is less likely to feel individually responsible for taking action. 6 Research has found that people will make irrational decisions to justify actions they have already taken. For Escalation of commitment example, when two people engage in a bidding war for an object, they can end up paying much more than the object is worth to justify the initial expenses associated with bidding. 7 Psychologists have claimed that some people show a tendency to make “overly dispositional inferences” in the Fundamental attribution error way they view victims of assault crimes. That is, they are overly inclined to view the victim’s plight as one he or she brought on by carelessness, foolishness, misbehavior, or naivetë. 8 Psychologists have claimed that some people show a “halo” effect in the way they form impressions of attractive Halo effect people. For instance, when it comes to assessing how nice, interesting, or able someone is, people tend to judge an attractive person more positively than he or she deserves. 9 Extensive psychological research has shown that people possess an unconscious, automatic tendency to be less Ingroup favoritism generous to people of a different race than to people of their race. This tendency has been shown to affect the behavior of everyone from doctors to taxi drivers. 10 Psychologists have identified a tendency called the “ostrich effect,” an aversion to learning about potential losses. Ostrich effect For example, people may try to avoid bad news by ignoring it. The name comes from the common (but false) legend that ostriches bury their heads in the sand to avoid danger. 11 Many psychological studies have found that people have the tendency to underestimate the impact or the strength Projection bias of another person’s feelings. For example, people who have not been victims of discrimination do not really understand a victim’s social suffering and the emotional effects of discrimination. 12 Psychologists have claimed that some people show a “self-interest” effect in the way they view political Self-interest bias candidates. That is, people’s assessments of qualifications, and their judgments about the extent to which particular candidates would pursue policies good for the American people as a whole, are influenced by their feelings about whether the candidates’ policies would serve their own particular interests. 13 Psychologists have claimed that some people show a “self-serving” tendency in the way they view their academic Self-serving bias or job performance. That is, they tend to take credit for success but deny responsibility for failure. They see their successes as the result of personal qualities, like drive or ability, but their failures as the result of external factors, like unreasonable work requirements or inadequate instructions. 14 Psychologists have argued that gender biases lead people to associate men with technology and women with Stereotyping housework.

indicates that the individual difference tapped by our assessing potentially related psychological constructs. scale is stable over time. Such comparisons determine whether individual differences in bias blind spot reflect individual differences in more basic or established personality traits. Study 2 Specifically, by using five different samples of re- In Study 2, we verified the factorial structure of the spondents, we examined the correlations between the

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. bias blind spot scale that emerged in Study 1 by sub- bias blind spot scale and measures of intelligence, mitting the 14 items to a conﬁrmatory factor analysis. inclination toward cognitive activities, and decision- Since the development of a valid and reliable measure- making ability (i.e., SAT scores, Need for cognition, ment scale introduces the possibility to clarify the rela- performance on the Cognitive Reﬂection Test, and tionship between susceptibility to the bias blind spot decision-making competence), measures of personal- and related constructs that compose its nomological ity traits related to self-esteem and self-enhancement network (Cronbach and Meehl 1955), we also tested (i.e., self-esteem, superiority, need for uniqueness, the discriminant validity of the bias blind spot scale narcissism, and over-claiming tendency), self-presen- in relation to a set of established scales and measures tation (i.e., self-monitoring, self-consciousness, and Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2473

Table 2 Correlations Between the 14 Selected Scale Items

Item 1 2 3 4 5 6 7 8 9 10 11 12 13 14

1 Action-inaction bias 2 Bandwagon effect 00287∗∗ 3 Conﬁrmation bias 00326∗∗ 00478∗∗ 4 Diffusion of responsibility 00440∗∗ 00347∗∗ 00331∗∗ 5 Disconﬁrmation bias 00189∗ 00236∗∗ 00347∗∗ 00240∗∗ 6 Escalation of commitment 00317∗∗ 00413∗∗ 00253∗∗ 00309∗∗ 00267∗∗ 7 Fundamental attribution error 00268∗∗ 00222∗∗ 00271∗∗ 00304∗∗ 00061 00216∗∗ 8 Halo effect 00356∗∗ 00469∗∗ 00328∗∗ 00273∗∗ 00250∗∗ 00379∗∗ 00331∗∗ 9 Ingroup favoritism 00217∗∗ 00321∗∗ 00371∗∗ 00329∗∗ 00245∗∗ 00286∗∗ 00352∗∗ 00398∗∗ 10 Ostrich effect 00256∗∗ 00368∗∗ 00313∗∗ 00359∗∗ 00390∗∗ 00329∗∗ 00134 00270∗∗ 00238∗∗ 11 Projection bias 00397∗∗ 00244∗∗ 00237∗∗ 00218∗∗ 00136 00082 00198∗∗ 00362∗∗ 00281∗∗ 00147 12 Self-interest bias 00336∗∗ 00272∗∗ 00358∗∗ 00323∗∗ 00296∗∗ 00281∗∗ 00249∗∗ 00197∗∗ 00302∗∗ 00261∗∗ 00228∗∗ 13 Self-serving bias 00383∗∗ 00403∗∗ 00399∗∗ 00418∗∗ 00357∗∗ 00295∗∗ 00250∗∗ 00447∗∗ 00300∗∗ 00412∗∗ 00172∗ 00334∗∗ 14 Stereotyping 00280∗∗ 00386∗∗ 00233∗∗ 00253∗∗ 00085 00305∗∗ 00340∗∗ 00426∗∗ 00313∗∗ 00150 00378∗∗ 00311∗∗ 00256∗∗

Note. Asterisks indicate signiﬁcant correlations: ∗∗p < 0001; ∗p < 0005; p < 0010.

social desirability), and more general personality traits Materials and Procedure. Sample 1. Participants (i.e., the Big Five Inventory (BFI)). (n = 260) first completed all 14 items of the bias blind spot scale in a random order. Participants then Method completed a series of personality scales measuring extant psychological constructs potentially related to Participants. Study 2 made use of five unique sam- the bias blind spot: the 10-item Rosenberg self-esteem ples that totaled to 661 respondents (344 women; scale (Rosenberg 1965), the need for uniqueness Mage = 3206 years, SD = 1102). For each sample, the scale (32 items; Snyder and Fromkin 1977), the self- content of the questionnaire varied as described next. consciousness scale (22 items; Feingstein et al. 1975), All participants were AMT workers residing in the the need for cognition scale (NFC; 18 items; Cacioppo United States who accessed and completed a survey et al. 1984), the self-monitoring scale (25 items; Snyder on the Internet in exchange for compensation vary- 1974), the superiority scale (10 items; Robbins and ing based on the length of the study (Sample 1: $5; Patton 1985), the NPI-16 short measure of narcissism Sample 2: $4; Sample 3: $6; Sample 4: $2; Sample 5: (16 items; Ames et al. 2006), and the social desir- $4). Participants were 77.0% White, 7.0% Black, 7.0% ability scale (33 items; Crowne and Marlowe 1960). Asian, 5.1% multiracial, 1.6% Native American, and Each scale was administered using its original answer 2.4% did not indicate their ethnicity. The five samples format and coding scheme. Both the order of the did not differ in terms of bias blind spot scores F < 1. scales and the order of the items were randomized,

Table 3 Bias Blind Spot by Scale Item, Factor Loadings from Exploratory Factor Analysis, and Completely Standardized Parameters from Conﬁrmatory Factor Analysis

Average American Self Factor Completely loading standardized Bias M SD M SD t(171) (EFA) parameter (CFA)

1 Action-inaction bias 5012 1033 3096 1069 9086∗∗∗ 00613 00660 2 Bandwagon effect 5073 1006 3060 1068 15007∗∗∗ 00675 00504 3 Conﬁrmation bias 5055 1027 3098 1060 11090∗∗∗ 00642 00552 4 Diffusion of responsibility 5051 1027 4011 1077 10005∗∗∗ 00613 00564 5 Disconﬁrmation bias 5031 1017 4048 1052 7031∗∗∗ 00482 00552 6 Escalation of commitment 5036 1013 3057 1067 14028∗∗∗ 00575 00583 Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. 7 Fundamental attribution error 4063 1024 3016 1054 12016∗∗∗ 00496 00476 8 Halo effect 5089 1002 4016 1069 12013∗∗∗ 00674 00511 9 Ingroup favoritism 5001 1029 3045 1069 11097∗∗∗ 00598 00599 10 Ostrich effect 5020 1015 4035 1074 6026∗∗∗ 00558 00520 11 Projection bias 5032 1033 4019 1065 7083∗∗∗ 00477 00476 12 Self-interest bias 5075 1024 4075 1062 8010∗∗∗ 00568 00355 13 Self-serving bias 5069 1008 4014 1059 13028∗∗∗ 00681 00677 14 Stereotyping 5046 1015 3062 1081 13032∗∗∗ 00579 00490

Notes. EFA, exploratory factor analysis; CFA, conﬁrmatory factor analysis. Asterisks indicate signiﬁcant results: ∗∗∗p < 00001. Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2474 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

with scale items nested within scale. Participants also Sample 4. Participants (n = 102) first completed all reported their scores on the math and verbal sections 14 items of the bias blind spot scale in a random of the SAT (if they had taken the SAT). To facilitate order. They then completed 11 Over-Claiming Ques- the accuracy of participants who might have taken tionnaire (OCQ) batteries. In these batteries, respon- the test several years before, participants reported dents indicate their familiarity with a list of items, approximate SAT scores on a 12-point scale increasing some of which exist (e.g., plate tectonics) and some in 50-point increments from 200 to 800. Last, partici- of which do not actually exist (e.g., plates of paral- pants reported their age, gender, ethnicity, and high- lax). These familiarity ratings serve as the basis for est level of education completed. a claiming response bias index (OCQ bias) measur- Sample 2. Participants (n = 101) first completed all ing self-enhancing tendencies: a statistical estimate of 14 items of the bias blind spot scale in a random order. how strong familiarity with an item has to be for a They then completed the original three-item Cogni- respondent to express familiarity with it (Macmillan tive Reflection Test (CRT; Frederick 2005) in open- and Creelman 1991). The 11 batteries covered topics ended format. Given the diffusion of the three original including rap artists, rock artists, country artists, hor- CRT on AMT and the ease of retrieving the solution to ror movies, comedies, dramas, foreign movies, soap those three questions, participants were also admin- operas, football, world leaders, and fashion design- istered nine additional and less common CRT ques- ers (Paulhus et al. 2003). Each battery included seven tions (Frederick, personal communication) resulting real items and three bogus items. Participants were in a total of 12 CRT items. Next, participants reported asked to rate their familiarity with each item on a 7- their scores on the math and verbal sections of the point scale with endpoints, not at all familiar (0) and SAT if they had taken the test (as did Sample 1). very familiar (6). Both the order of the batteries and Finally, participants reported their age, gender, eth- the order of the items were randomized, with items nicity, and highest level of education completed. nested within battery. Finally, participants reported Sample 3. Participants (n = 100) first completed their age, gender, ethnicity, and highest level of edu- all 14 items of the bias blind spot scale in a ran- cation completed. = dom order. Then they completed the Adult Decision- Sample 5. Participants (n 98) first completed all Making Competence inventory (A-DMC; Bruine de 14 items of the bias blind spot scale in a random order. Next, participants completed a short-form (44-item) Bruin et al. 2007), a comprehensive instrument assess- version of McCrae and Costa’s (1987) BFI (John and ing general judgment and decision-making ability. Srivastava 1999). Participants reported their scores on Participants completed the six behavioral decision- the math and verbal sections of the SAT if they had making batteries of measures composing the A-DMC: taken the test (as did Sample 1). Finally, participants resistance to framing (14 paired items), which measures reported their age, gender, ethnicity, and highest level whether value assessment is affected by irrelevant of education completed. variations in problem descriptions; recognizing social norms, which measures how well people assess peer Results social norms (16 items); under/overconfidence, which Confirmatory Factor Analysis0 To test the factorial measures how well participants recognize the extent structure that emerged in the exploratory factor anal- of their own knowledge (34 items); applying decision ysis in Study 1, we conducted a confirmatory fac- rules, which asks participants to indicate, for hypo- tor analysis on the data collected from Sample 1. We thetical individual consumers using different decision first evaluated the assumption of multivariate nor- rules, which products they would buy out of a choice mality of the data that is a necessary condition for the set (10 items); consistency in risk perception, which mea- use of maximum likelihood estimation. Toward this sures the ability to follow probability rules (16 paired end, we computed Mardia’s (1980) test of multivari- items); and resistance to sunk costs, which measures the ate skewness and kurtosis. The test was significant ability to ignore prior investments when making deci- (2 = 299051, p < 00001), signaling that the assumption sions (10 items). These tasks measure different aspects of multivariate normality of the data was violated. As

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. of the decision-making process and decision-making a consequence we opted for a robust maximum like- skills, such as the ability to assess value and the abil- lihood estimation. ity to integrate information. Both the order of the bat- The confirmatory factor analysis suggested that the teries and the order of the items were randomized, single-factor model emerged in the exploratory fac- with scale items nested within battery. Participants tor analysis from Study 1 fit the data in Study 2 reported their scores on the math and verbal sections well, with a goodness-of-fit index of 0.93, a Bentler’s of the SAT if they had taken the test (as did Sam- comparative fit index of 0.98, a non-normed fit index ple 1). Finally, participants reported their age, gender, of 0.97, and a root mean square error of approxima- ethnicity, and highest level of education completed. tion (RMSEA) of 0.05, suggesting a good fit between Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2475

Figure 1 Distribution of Bias Blind Spot Scores in Study 2

&REQUENCY

; ;n ; ; ; ; ; ; ; ; ; ; ; ; ; = ;n n ;n n ;n n ;n n ;n n ;n n ;n n ;n n ;n n ;n n ;n n ;n n ;n n "IAS"LIND3POT3CORE Notes. Scale ranges from 7 to −7. Scores greater than zero indicate bias blind spot. Median column is indicated by gray bar.

the model and the observed data. The factor-loading constructs: (i) measures of intelligence, inclination estimates are reported in Table3 and were all signifi- toward cognitive activities, and decision-making abil- cant; all ts > 5049, all ps < 00001. ity (i.e., SAT scores, NFC, CRT, A-DMC); (ii) personal- Descriptive Statistics0 To examine the distribution of ity traits related to self-esteem and self-enhancement susceptibility to the bias blind spot, we computed bias tendencies (i.e., self-esteem, superiority, need for blind spot scores on the five samples following the uniqueness, narcissism, over-claiming tendency); same procedure as in Study 1 ( = 0084). Across the (iii) personality traits related to self-presentation five samples, the distribution of the scores appeared (i.e., self-monitoring, self-consciousness, and social to be positively skewed with an observed range from desirability); and (iv) and general personality traits −1050 to 5.14 (see Figure1). On average, participants (i.e., BFI). exhibited a significant bias blind spot (M = 1048, SD = For each of the samples we computed bias blind 0095; significantly different from 0; t46605 = 40029, p < spot scores (Alphas ranging between 0.82 and 0.85). 00001). Furthermore, for each of the 14 biases that For each of the constructs that were measured composed the bias blind spot scale, participants rated using multi-item scales, we computed overall aver- the average American as more susceptible to that bias age scores after reverse scoring appropriate items, and than themselves; all Ms > 0090, all ts(659) > 16085, we examined the correlation of these scores with bias all ps < 00001. At the individual level, a large major- blind spot scores. Table4 presents these scales and ity (85.2%) of participants exhibited a significant bias measures, their Cronbach’s alpha coefficients, means, blind spot across the 14 items, with the averages of standard deviations, and their zero-order correlations their scores being marginally or significantly greater with the bias blind spot scale. than 0; all ts(13) > 1079, all ps < 0010. Only one par- Intelligence, Inclination Toward Cognitive Activi- ticipant (0.1%) had an average bias blind spot score ties, and Decision-Making Ability0 We first examined that was significantly lower than 0 (t4135 = −2046, whether bias blind spot is merely a manifestation of p = 0003). intelligence and cognitive ability in four ways. First, We then examined whether demographic variables we examined SAT scores, as both the verbal and math accounted for some differences in bias blind spot scores load highly on psychometric g or general intel- scores. Gender appeared unrelated to bias blind spot, ligence (Brodnick and Ree 1995, Frey and Detterman as the scores of male (M = 1045, SD = 0095) and 2004, Unsworth and Engle 2007). Of the 559 partic-

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. female participants (M = 1051, SD = 0095) were not ipants in Samples 1, 2, 3, and 5, 248 reported their significantly different; F < 1. Similarly, the correlation SAT test scores. The correlation between bias blind between age and bias blind spot scores was not sig- spot scores and the verbal SAT scores was not significant, r = −0004, p = 0030. Bias blind spot scores nificant (r42475 = 0010, p = 0013), whereas the corre- were significantly and negatively affected by level of lation between bias blind spot scores and the math education, following approximately a linear pattern SAT scores (r42475 = −0013, p = 0005), was significant, (linear contrast: F 411 6535 = 11042, p = 00001). albeit very small. Discriminant Validity0 We next examined whether Second, we examined the correlation between bias the bias blind spot scale is distinct from related blind spot and NFC, which refers to the extent to Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2476 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

Table 4 Scale Descriptive Statistics, Reliabilities, and Correlations with Bias Blind Spot (Study 2)

Construct Sample M SD N r

Intelligence 1–3, 5 SAT math 605004 111055 — 248 −0013∗ SAT verbal 640032 93022 — 248 0010 Need for cognition 1 63065 13021 0093 260 0016∗ Cognitive Reﬂection Test 2 12 items 3067 3000 0082 101 −0023∗ 3 original items 1030 1025 0079 101 −0019 9 new items 2038 2002 0071 101 −0022∗ Decision-making competence 3 Resistance to framing 1045 0054 0058 100 0014 Recognizing social norms 0051 0026 0089 100 0007 Under/overconﬁdence 0091 0007 0089 100 0006 Applying decision rules 0077 0016 0050 100 0002 Consistency in risk perception 0075 0015 0071 100 0009 Resistance to sunk costs 4040 0062 0040 100 0008 Big Five personality traits 5 Neuroticism 24030 5065 0087 98 0006 Extraversion 22084 7042 0088 98 0003 Openness 36062 6081 0085 98 0033∗∗ Agreeableness 31076 5030 0083 98 0010 Conscientiousness 33002 6042 0083 98 0007 Self-esteem 1 19096 6026 0092 260 0015∗ Superiority 1 35013 7039 0075 260 −0002 Need for uniqueness 1 101023 17034 0089 260 0016∗∗ Narcissism 1 20040 3065 0082 260 0004 Over-claiming 4 0050 0026 0095 102 −0025∗ Self-consciousness 1 Public 18062 4058 0082 260 −0005 Private 25045 4057 0073 260 0007 Social anxiety 16010 5001 0087 260 −0010 Self-monitoring 1 11018 4043 0074 260 0005 Social desirability 1 3030 6080 0097 260 −0005

p < 0010; ∗p < 0005; ∗∗p < 0001.

which people “engage in and enjoy effortful cogni- spot scores were significantly and negatively corre- tive activities” (Cacioppo and Petty 1982, p. 1). People lated with CRT scores (r4995 = −0022, p = 0002). This who are high in NFC expend more effort processing pattern of correlations was consistent irrespective of information, and perform better on arithmetic prob- which CRT items were considered: all 12 items: r = lems, anagrams, trivia tests, and college coursework −0023, p = 0002; the three “original” items: r = −0019, (Cacioppo et al. 1996). In line with the results of p = 0006; or the nine “new” items: r = −0022, p = 0003. West et al.(2012), we observed a small but signifi- Participants who believed themselves to be less vul- cant and positive correlation between bias blind spot nerable to bias than others were more likely to rely on scores and need for cognition (r42585 = 0016, p = 0001). their intuition and less likely to engage in cognitive This significant correlation suggests that people with reflection and deliberation (i.e., were less accurate). a high NFC tend to believe that they are less suscep- Finally, we computed scores for each A-DMC battery as recommended by Bruine de Bruin et al.1 tible to biases than their peers. (2007). Bias blind spot scores were uncorrelated with Third, we examined whether participants with performance on all of the A-DMC batteries; all rs < higher bias blind spot scores make more use of intu- 0014, all ps > 0018, average r = 0008. ition than of deliberate reasoning by examining the Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. Self-Esteem and Self-Enhancement0 We next examined correlation between bias blind spot scores and the the possibility that the bias blind spot is the manifes- CRT (Frederick 2005), a set of questions that each have tation of more basic perceptions of being superior to an incorrect intuitive response and a correct response others or being different from others (e.g., Taylor and that can be reached through deliberate reasoning. We Brown 1988, Snyder and Fromkin 1980) by examining computed a composite measure of performance on the CRT by assigning a score of one to each correct 1 The levels of reliability for each battery reported in Table4 answer, a score of zero to each incorrect answer, and are generally consistent with those reported by Bruine de Bruin summing all items (M = 3067, SD = 3000). Bias blind et al.(2007). Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2477

the correlations between bias blind spot and superior- sense of familiarity needs to be experienced in order ity, self-esteem, need for uniqueness, narcissism, and for a respondent to claim familiarity with an item. The over-claiming. If the domain of these constructs sig- correlation between bias blind spot scores and OCQ nificantly overlaps with that of the bias blind spot, we bias was negative and significant, r41015 = −0025, p = should observe high correlations between those traits 0001, suggesting that higher bias blind spot scores and bias blind spot scores. are associated with higher self-enhancing tendencies The superiority scale (Robbins and Patton 1985) as measured by over-claiming, but that the two con- measures whether people believe that they are supe- structs are sufficiently discriminated. rior to others. People who believe they are superior Self-Presentation0 We examined whether the bias to others may believe that they are less susceptible to blind spot is the manifestation of other personality cognitive and social biases than inferior others. How- traits related to self-presentation. Self-consciousness ever, perceived self-superiority did not correlate with is an acute sense of self-awareness, articulated in bias blind spot scores (r42585 = −0002, p = 0072). three dimensions: private self-awareness (i.e., a ten- We then examined the correlation between bias dency to introspect and examine one’s inner self and blind spot and self-esteem (Rosenberg 1965), since feelings), public self-consciousness (i.e., a tendency people holding high levels of self-esteem may be to think about how the self is viewed by others), more likely to see themselves as immune from bias and social anxiety (i.e., a sense of apprehensiveness and therefore show higher susceptibility to the bias over being evaluated by others in a social context; blind spot. We found evidence for a significant and Feingstein et al. 1975). We did not observe a signif- positive correlation between bias blind spot scores icant correlation between bias blind spot scores and and self-esteem (r42585 = 0015, p = 0002), suggesting private self-awareness (r42585=0007, p =0024), public that people who believe they are less vulnerable to self-consciousness (r42585=−0005, p =0046), or social bias than their peers tend to have higher levels of anxiety (r42585=−0010, p =0011). Self-monitoring mea- self-esteem. The small size of the correlation, how- sures the extent to which one consciously employs ever, suggests that the two constructs are theoretically impression management strategies in social interac- distinct. tions (Snyder 1974). We did not observe a significant We also examined the possibility that the belief that correlation between bias blind spot scores and self- = = one is less vulnerable to bias than one’s peers is a monitoring (r42585 0005, p 0040). Lastly, we sought manifestation of a need for uniqueness (Snyder and to test whether there was a negative correlation Fromkin 1977), the desire to be dissimilar from oth- between bias blind spot and the tendency to answer ers. The correlation between bias blind spot scores in a way that would be viewed favorably by others (Crowne and Marlowe 1960). The correlation was neg- and need for uniqueness was positive and significant ative but not significant (r42585=−0005, p =0041). (r42585 = 0016, p < 0001). This small significant correla- Big Five Personality Traits0 Finally, we examined tion suggests that people high in bias blind spot have the correlations between bias blind spot and the a higher desire to distinguish themselves from others, Big Five personality traits (McCrae and Costa 1987). but that the two constructs are theoretically distinct. Bias blind spot scores appeared to be uncorrelated Finally, we examined the relationship between the with agreeableness (r4965=0010, p =0032), extraver- bias blind spot and two different operationalizations sion (r4965=0003, p =0080), neuroticism (r4965=0006, of self-enhancement, a narcissism scale (Ames et al. p =0057), and conscientiousness (r4965=0007, p =0049), 2006), and over-claiming (Paulhus 1998). Narcissism, but moderately correlated with openness to experi- a personality trait that involves a pervasive pattern ence (r4965=0033, p <0001). Participants who reported of grandiosity, self-focus, and self-importance accom- significantly higher openness to experience had higher panied by preoccupation with success and demands bias blind spot scores, although the moderate size of for admiration (Morf and Rhodewalt 2001), has been the coefficient suggests that the two constructs are suf- used as a measure of trait self-enhancement (Paulhus ficiently discriminated. This result was not predicted. 1998). The narcissism scale did not correlate with Because open-minded thinking has been shown to cor- = = bias blind spot scores (r42585 0004, p 0049). Over- relate with superior cognitive ability in a number of Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. claiming, a measure that can discriminate accuracy domains (Stanovich and West 2007, West et al. 2008), from self-enhancement in response patterns, was it is possible that if people high in openness to expe- assessed by using the OCQ (Paulhus et al. 2003). rience have both superior cognitive ability and some We computed OCQ bias indexes, corresponding to awareness of that ability, they might accurately report the mean of the hit rate and the false-alarm rate for that they are less susceptible to bias than their peers. real and false OCQ items (Paulhus et al. 2003), for each battery of items, and averaged across batter- Discussion ies. Higher values of OCQ bias indicate lower self- Bias blind spot appears to be a latent unidimensional enhancing tendencies, as they indicate that a higher construct that induces people to see themselves as less Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2478 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

biased than other people across multiple judgmental If the bias blind spot is simply due to overly pos- biases (i.e., a meta bias). Bias blind spot scores did itive self-perceptions, then people high in bias blind not appear to be correlated with demographic vari- spot should be more susceptible to the better-than- ables such as gender and age, although bias blind average effect, but not to the worse-than-average spot decreased significantly as level of education effect. That is, they should rate themselves as bet- increased. The results of Study 2 support the dis- ter than average for both easy and difficult activi- criminant validity of the bias blind spot by showing ties. If susceptibility to the bias blind spot is due to that the construct is not a derivative of several poten- greater naïve realism and confidence in initial self- tially related constructs such as measures of intelli- assessments (e.g., “I’m great at using a computer gence and cognitive ability, decision-making ability, mouse but terrible at juggling”), coupled with a fail- self-esteem and self-enhancement, self-presentation, ure to correct those assessment to account for other’s and general personality traits. Many of the correla- abilities (e.g., “0 0 0 as are most people”), people high in tions between these traits and bias blind spot were susceptibility to the bias blind spot should be more not significant. Other correlations were in the low to susceptible to both the better-than-average effect and moderate range (0.17 to 0.33; Cohen 1988), suggesting the worse-than-average effect. that the bias blind spot scale is assessing a distinct construct. Most importantly, the perception that one is less Method vulnerable to bias than one’s peers truly appears to Pretest. In a pretest, 100 AMT workers (47 women; constitute a blind spot in self-awareness. The lack of Mage =2806 years, SD =1004) were presented with a correlation with general decision-making competence list of 34 activities in an online survey and com- suggests that susceptibility to bias blind spot (a) does pensated $1 for participating. Included in that list not reflect generally poorer decision-making ability, were the activities used by Kruger(1999, Study 2). and (b) does not reflect an accurate self-assessment Participants rated the difficulty of each activity on of decision-making ability. The negative correlation a 9-point scale (1=not difficult at all; 9=very diffi- observed between susceptibility to bias blind spot and cult). We selected the two activities rated least difficult CRT scores could be interpreted as supporting a posi- and the two activities rated most difficult: operating a tive relation to more general cognitive ability, but also computer mouse (M =1034, SD =0084), copying and lends support our suggestion that the bias blind spot difficulty pasting text (M =1043, SD =1013), programming is a distinct meta-bias reflected in the different con- difficulty a computer (M =7058, SD =2058), and juggling sideration of evidence when making self and other difficulty (M =7068, SD =2028). A one-way repeated mea- assessments. In other words, people who are more difficulty likely to rely on their intuition may more biased, but sures ANOVA accompanied by planned comparisons also be less likely to correct their initial introspec- revealed that the two difficult activities were perceived tive self-assessments by considering additional evi- as significantly more difficult than the two easy activ- dence, such as their behavior. A more direct test of ities (F 411995=11105052, p <00001), that the two easy this assumption was performed in Study 3. activities were perceived as equally easy, and that the two difficult activities were perceived as equally difficult (Fs <1). Study 3: Bias Blind Spot and Self-Assessment Participants. A new sample of 156 AMT workers In Study 3, we further elucidated the cognitive mech- (90 women; Mage = 3200 years, SD = 1200) received anisms underlying the bias blind spot by examin- $2.50 for participating in Study 3. Participants were ing the relationship between susceptibility to the bias 79.5% White, 8.3% Black, 4.5% Asian, 3.2% multira- blind spot and the asymmetric use of evidence in cial, and 0.6% Native American; 3.8% did not indicate another set of self–other comparisons: relative perfor- their ethnicity. mance assessments. When evaluating their own skills Materials and Procedure. Participants completed Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. and abilities relative to their peers, people tend to focus on their own capabilities and neglect the capa- the 14-item bias blind spot scale in a random order. bilities of others. Because they neglect the fact that Then they rated their comparative ability on the two easy activities are easy for most people and difficult easy (operating a computer mouse and copying and past- activities are difficult for most people, they rate them- ing text) and on the two difficult activities (program- selves as better than the average person when assess- ming a computer and juggling) on 100-point scales ing their ability at easy activities, but as worse than (0 = I’m at the very bottom; 100 = I’m at the very top). the average person when assessing their ability at dif- Activity rating order was random. Last, participants ficult activities (Kruger 1999). reported their age, gender, and ethnicity. Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2479

Results and Discussion t = −3051, p < 00001). These results suggest that people Bias blind spot scores were computed following the high in susceptibility to the bias blind spot are more procedure described in Study 1 ( = 0083). A mixed susceptible to both the better-than-average effect and model analysis was used to estimate the effect of sus- the worse-than-average effect. These results provide ceptibility to the bias blind spot, type of activity (easy further evidence that the bias blind spot is a form of versus difﬁcult), and their interaction on compara- egocentric cognition driven by an asymmetric consid- tive ability ratings. Since each participant assessed her eration of evidence when evaluating the self and oth- ability on both the easy and the difﬁcult activities, we ers, rather than by a general self-enhancement motive let the intercept vary randomly to take into account that induces a person to see herself as superior to the lack of independence between the observations. her peers. We estimated the following model:

CAij = 0 + 1 BBSi + 2 TAj + 3 BBSi × TAj + Ui + ij Study 4: Bias Blind Spot and where CA refers to comparative ability, index i refers Advice Taking In Study 4, we began to examine our last set of predic- to participants and index j refers to the type of tions, that bias blind spot affects judgments informed activity (easy versus difficult). The dependent variable was thus the average comparative ability rat- by self–other accuracy comparisons. Specifically, we ing expressed by each participant on easy or difficult examined the effect of the bias blind spot on the activities. The explanatory variables were each partic- weight given to advice from other people (Yaniv and ipant’s bias blind spot score (BBS 5, the type of activ- Kleinberger 2000). If bias blind spot does influence i bias in judgments reliant on self–other accuracy com- ity (TAj , dummy coded, with 0 = easy, 1 = difficult) and the interaction between these two variables, and parisons, people who are more susceptible to the bias blind spot should view their judgments to be Ui indicated each participant’s random effect. Both bias blind spot scores and comparative ability ratings more accurate than the judgments of other people, were standardized. Results are based on a total of because they believe their judgments are based on 624 observations, where each observation is a compar- more objective perceptions of the world (Pronin et al. ative ability rating on an activity provided by a par- 2004). Consequently, people high in bias blind spot ticipant. The results revealed a significant interaction should give less weight to advice received from oth- between susceptibility to the bias blind spot and type ers. They should be less likely to correct their initial of activity (easy vs. difficult) on comparative ability (self-informed) judgments when subsequently given assessments (b = 0027, SE = 0006, t = −4084, p < 00001). the opportunity to incorporate others’ judgments into Table5 reports the results of this estimation and the their own. comparison of this factorial model with an alternative Method model that includes only the main effects. The interaction between bias blind spot and the Participants. A total of 178 AMT workers (109 type of activity was further explored by conducting a women; Mage = 3308 years, SD = 1202) received $8 for simple slope analysis. Bias blind spot was associated completing Study 4. Participants were 75.3% White, with higher comparative ability ratings on easy activ- 6.7% Black, 6.7% multiracial, 6.2% Asian, and ities ( = 0013, t = 3019, p = 00002), and lower compar- 1.7% Native American; 3.4% did not indicate their ative ability ratings on difficult activities ( = −0014, ethnicity. Table 5 Regression Results for the Effects of Bias Blind Spot and Type of Activity (Easy vs. Difficult) on (Standardized) Comparative Ability Ratings (Study 3)

Main effect model Factorial model

Variable b a SE t p b a SE t p

Intercept 0072 0004 19067 <00001 0072 0004 19053 <00001 Type Activity b −1044 0006 −24021 <00001 −1044 0006 −25035 <00001 BBS c 0004 0003 1022 0022 0013 0004 3055 <00001 Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. Type Activity b × BBS c −0027 0006 −4084 <00001 −2 Log-likelihood 1,286.19 1,263.85 Degrees of freedom 6 7 Akaike information criterion 1,298.19 1,277.85 Bayesian information criterion 1,324.81 1,308.90 Likelihood ratio test 22.34 (1), p < 00001

Materials and Procedure. Participants completed WOA score value was more than ﬁve standard devia- the 14-item bias blind spot scale in a random order. tions above the mean WOA score was excluded from Next, they estimated the weight of 30 household all subsequent analyses. No other participants were objects (e.g., a trash can, a statue, a doll house), one at excluded. Results are based on a total of 5,049 obser- a time. For each object they were shown a color pic- vations, each observation being the weight of an ture and were provided with the object’s dimensions object estimated by a participant. (length, height, and width). After making each initial A mixed model analysis was used to estimate the weight estimate, participants were given advice in the effect of susceptibility to the bias blind spot on WOA. form of the guess of another “randomly selected par- Because each participant evaluated the weight of mul- ticipant” in the study (i.e., the average estimate of two tiple objects, we let the intercept vary randomly to pilot participants), and were then given the opportu- take into account the lack of independence between nity to estimate the weight of the object again. Thus, the observations. We estimated the following model, participants had the option to revise their original estimate based on the advice that they received from WOAij = 0 + b1BBSi + Ui + ij 1 the other participant. The order in which objects were judged was random. Finally, participants reported where index i referred to participants and index j their age, gender, and ethnicity. refers to objects. The dependent variable was thus the value for WOA for each participant and for each Results and Discussion object. The explanatory variable was each partici-

We computed bias blind spot scores following the pro- pant’s bias blind spot score (BBSi5, and Ui indicated cedure described in Study 1 ( = 0082). We computed each participant’s random effect. The results revealed the weight on advice (WOA) relative to each object, a signiﬁcant and negative effect of susceptibility to

by computing the difference between the original esti- the bias blind spot on WOA (b1 = −0003, SE = 0001, t = mate and the estimate made after receiving the advice −3023, p = 00001). On average, participants with bias of the other participant (as in Gino and Moore 2007, blind spot scores one standard deviation below the Gino 2008, Harvey and Fischer 1997, Yaniv 2004). mean placed six percent more weight on the advice WOA reflects how much a participant uses the advice of others than participants with bias blind spot scores she receives (Yaniv 2004) and is defined as one standard deviation above the mean (WOA = 00223 final estimate − initial estimate vs. WOA = 00164). WOA = 0 advice − initial estimate This analysis was complemented by the estimation WOA is equal to 0 when participants entirely dis- of a logit mixed model that examined the relationship count the advice. In such a case, final estimates between bias blind spot scores and the rate at which are equal to initial estimates, meaning that partici- participants completely ignored advice (WOA = 0). pants did not revise their estimates after receiving We estimated the following model: the advice. WOA is equal to 1 when participants’ NO_WOA = + b BBS + U + 1 final estimate is equal to the advice received. In ij 0 1 i i ij this case participants give maximum weight to the where index i referred to participants and index j advice received. Finally, WOA equals a value between refers to objects. The dependent variable was a 0 and 1, when participants weigh both their initial dummy variable that indicates the complete igno- estimate and the received advice positively. Follow- rance of advice (i.e., D = 1 when WOA = 0; D = 0 oth- ing the procedure used in prior research (Gino and erwise) for each participant and for each object. The Moore 2007, Gino 2008, Yaniv 2004), cases in which explanatory variable was each participant’s bias blind the advice equaled the initial estimate were excluded spot score (BBSi), and Ui indicated each participant’s from the analysis, since WOA in those cases equaled a random effect. Consistent with the previous model, number divided by 0, making it impossible to quan- the results revealed a significant and positive effect of tify how much a participant did or did not use the susceptibility to the bias blind spot on the likelihood advice. For cases in which the final estimate did not of completely ignoring advice (b = 00104, SE = 00044, fall between the initial estimate and the advice (e.g., 1

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. z = 2034, p = 0002). Susceptibility to the bias blind spot participants provide revised estimates that are fur- increased participants’ likelihood of conﬁrming their ther away from the advice than initial estimates), initial estimate and completely neglecting advice. and WOA was thus greater than 1, values above 1 In short, participants who exhibited high levels of were adjusted to 1.2 One participant whose aggregate bias blind spot were more likely to ignore the advice provided, whereas participants who exhibited low 2 We estimated the same model on WOA scores that were not subject to this recommended adjustment, and the effect of susceptibil- levels of bias blind spot weighed the advice more

ity to the bias blind spot is even stronger: b1 = −0005, SE = 0001, heavily. These results are consistent with the results t = 3065, p < 00001. of Liberman et al.(2012), who showed that naïve Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2481

realism, by making people believe that their own behaviors whereas underestimating the impact of sit- perceptions of the world are objective, leads them uational variables (Jones and Harris 1967). The article to underweight the input of their peers. Our results described the existence of the FAE, provided exam- extend those by Liberman et al.(2012) by showing ples of its occurrence, and explained how one might how individual differences in susceptibility to the bias correct the bias (see the section “debiasing training” blind spot significantly affect the extent to which peo- in the appendix). Participants assigned to the con- ple incorporate others’ advice in their judgments. Fur- trol condition read an article of equal length report- thermore, our results provide additional evidence in ing the results of a research study on trust (see the support of the role of naïve realism as the process section “control training” in the appendix). All par- underlying the bias blind spot (Pronin et al. 2004). ticipants then answered nine questions designed to test the occurrence of the FAE in judgments inspired Study 5: Bias Blind Spot and Bias by existing paradigms, in a random order (Jones and Harris 1967, Snyder and Frankel 1976, Scopelliti et al. Reduction 2015). Finally, participants reported their age, gender, In Study 5, we expanded our analysis of bias blind and ethnicity. spot to see whether people characterized by higher susceptibility to the bias blind spot are more resis- Results and Discussion tant to debiasing procedures. There is little evidence Bias blind spot scores were computed following the that people who are more aware of their own biases procedure described in Study 1 ( = 0080). Answers are better able to overcome them. West et al.(2012) to the nine FAE questions were highly correlated with examined the effect of the bias blind spot on the inci- one another ( = 0080). We summed the number of dence of six cognitive biases, and observed no evi- questions in which participants exhibited the FAE that dence that people characterized by lower bias blind served as our primary dependent variable (M = 4017, spot scores were less likely to commit bias. A rela- SD = 2073). The FAE scores were neither normally tionship between bias blind spot and decision-making distributed, nor could be normalized by log transfor- ability might emerge, however, after an intervention mation. Therefore we estimated a Poisson regression, aimed at reducing bias. appropriate to model count dependent variables that We predicted that the perception that one is less have only non-negative integer values. vulnerable to bias than others may constitute a bar- We predicted that the debiasing training would rier to the activation of corrective strategies aimed at reduce the incidence of bias more for participants avoiding bias (Wilson and Brekke 1994). This barrier who were less susceptible to the bias blind spot. This may prevent people who believe that others are more prediction was tested using the procedures outlined susceptible to bias from actively responding to train- by Aiken and West(1991) to decompose the pre- ing procedures designed to correct biased judgments. dicted interaction using regression analysis. First, bias In order to test this prediction, we had participants blind spot scores were mean-centered by subtract- read an article designed to increase awareness of the ing the mean bias blind spot score from all obser- occurrence of a specific bias, the fundamental attribu- vations. Second, we created the interaction term of tion error (FAE) that also suggested how to mitigate (dummy-coded) training factor (debiasing vs. control) that bias. We predicted that susceptibility to the bias by (mean-centered) bias blind spot score. Next, the blind spot would moderate the effect of this debiasing FAE scores were regressed on the training factor (debi- training on subsequent commission of the FAE. asing vs. control), bias blind spot scores, and the inter- Method action between these two variables. The analysis revealed a significant interaction Participants. A total of 297 AMT workers (170 between training and susceptibility to the bias blind women; Mage = 3107 years, SD = 1007) completed an spot (2415 = 4073, p = 0003). To shed light on the online survey and received $2 for their participa- nature of this interaction, we conducted a floodlight tion. Participants were 76.4% White, 7.4% multiracial, analysis (Spiller et al. 2013) estimating the marginal 6.4% Black, 5.7% Asian, and 1.0% Native American; Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. effect of training on commission of the FAE at all levels 3.0% did not indicate their ethnicity. of susceptibility to the bias blind spot. Table6 reports Procedure. Participants completed the 14-item bias the results of this estimation and the comparison of blind spot scale with items in random order. Next, this factorial model with an alternative model that participants were randomly assigned to one of two includes only the main effects. We used the Johnson- conditions: debiasing training or control. Participants Neyman technique to identify the range of bias blind assigned to the debiasing training condition read an spot scores for which the effect of the debiasing train- explanatory article about the FAE, a tendency to over- ing was significant. This analysis revealed that there estimate the impact of individual dispositions on was a significant positive effect of debiasing training Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2482 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

Table 6 Poisson Regression Results for the Effects of Bias Blind Spot and Type of Training on Fundamental Attribution Error Commission (Study 5)

Main effect model Factorial model

Variable b a SE Wald 2 p b a SE Wald 2 p

Intercept 1071 0006 819041 <00001 1081 0007 605062 <00001 Type of Training b −0036 0006 39027 <00001 −0057 0011 26020 <00001 BBS c −0008 0003 5087 0002 −0015 0005 10048 <00001 Type Training b × BBS c 0015 0007 4073 0003 −2 Log-likelihood 1,471.535 1,466.813 Degrees of freedom 4.000 3 Akaike information criterion 1,477.535 1,474.813 Bayesian information criterion 1,488.617 1,489.588 Likelihood ratio test 4.72 (1), p = 0003 aUnstandardized coefﬁcients. bDummy coded. cMean centered.

on reducing FAE commission for bias blind spot scores abilities of others when judging their own abilities, lower than 2.55 (original scale) (2415 = 3087, p = 0005), resulting in a greater propensity to believe that they but not for bias blind spot scores higher than 2.55. were better than average on easy tasks and worse than The effect of a training procedure designed to average on difficult tasks than their peers (Study 3). reduce commission of the FAE was stronger for par- Second, bias blind spot predicts bias in judgments ticipants with a lower susceptibility to the bias blind reliant on self–other accuracy comparisons. It pre- spot than for participants with a higher susceptibility dicted the extent to which people ignored the advice to the bias blind spot. The results suggest that high of others, as participants high in bias blind spot placed susceptibility to the bias blind spot may constitute a less weight on advice than did participants low in bias barrier to bias reduction. blind spot (Study 4). Susceptibility to the bias blind spot was also associated with reduced responsiveness to training designed to reduce the incidence of a dif- General Discussion ferent judgmental bias (Study 5). The foregoing studies provide an in-depth portrait of the bias blind spot. We found that the tendency for Consequences and Implications of the Bias people to be better able to identify bias in the judg- Blind Spot ments and behaviors of others than in their own judg- The identification of individual differences in suscepti- ment and behavior is captured by a single latent fac- bility to the bias blind spot has important implications tor, and that there is substantial individual variation in for the incidence of bias and the improvement of judg- susceptibility to this meta-bias. Using a psychometric ment and decision making. As we have demonstrated, approach, we developed and validated an instrument awareness of one’s vulnerability to bias is an impor- to measure individual differences in susceptibility to tant antecedent of openness to advice that in turn the bias blind spot. We found that the bias blind spot affects decision quality. Research maintains that inte- is discriminated from intelligence, cognitive ability, grating advice from external sources improves deci- cognitive reflection, and personality traits related to sion making (Larrick and Soll 2006, Johnson et al. 2001, self-esteem, self-enhancement, and self-presentation. Budescu and Rantilla 2000, Yaniv 2004; see Bonaccio Moreover, the bias blind spot does not appear to be a and Dalal 2006 for a review). People high in bias blind facet of general decision-making ability or an accurate spot, for example, may be particularly likely to ignore perception of the extent to which one is biased. Rather, the advice of peers or experts when engaging in finan- susceptibility to the bias blind spot appears to be a cial or medical decision making and require alterna- true blind spot in self-awareness, because of asymmet- tive forms of guidance to improve the quality of their

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. ric consideration of evidence when assessing the self decisions. and other people. Second, awareness of one’s susceptibility to bias Susceptibility to bias blind spot appears to predict blind spot appears to be an important indicator of at least two kinds of bias in social judgment. First, receptivity to efforts to improve one’s decision mak- bias blind spot predicts the extent to which people ing. Indeed, Wilson and Brekke(1994) suggest that a exhibit biases reliant on similar cognitive mechanisms, critical impediment to the correction of the contami- namely the consideration of different evidence in self– nating inﬂuence of biases on judgment is people’s lack other comparisons. Participants higher in susceptibil- of humility about their vulnerability to bias. When ity to bias blind spot were more likely to ignore the people are unaware of their bias, they are unlikely Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2483

to adopt corrective strategies to avoid the sources of exacerbate the perception of the gap between antago- bias that influence their judgment. Consequently, peo- nists in a controversy (i.e., false polarization, Robinson ple who are more susceptible to bias blind spot are et al. 1995, Pronin et al. 2002). People high in bias blind less prone to improve their decision making by engag- spot, being more likely to believe that their view is ing in bias reduction strategies, responding to training, correct, may be less likely to take the necessary steps and taking advice. Consequently they may need more to arrive at an agreement. More generally, susceptibil- persuasion or evidence that their decision making is ity to the bias blind spot is likely to predict problem- subject to error to acknowledge their potential for bias. atic interpersonal interactions when those interactions Our results corroborate this view by showing that require one to acknowledge the potential for bias or people who believe that they are relatively immune to error in one’s own thinking and reasoning. bias are less likely to enact corrective strategies, even when correcting strategies are explained and explic- Limitations and Directions for Future Research itly suggested. In the work place, the results suggest Of course, the present research is not without its own that employees high in bias blind spot may be less limitations. We relied upon samples drawn from AMT receptive to training designed to improve their deci- that are not representative samples. Fortunately, AMT sion making, and may need more or different kinds of samples have been shown to exhibit vulnerability to training. At home, the results suggest that people high biases in judgment and decision making similar to train bias blind spot may need more or different forms ditional college samples (Berinsky et al. 2012, Horton of guidance to improve their ability to make conse- et al. 2011, Paolacci et al. 2010). We thus expect that quential decisions affecting their personal life. It has the results should generalize to other samples. been proposed that decision making may be a teach- In the discriminant validity testing conducted in able skill (Baron and Brown 1991, Bruine de Bruin et al. Study 2, we collected measures to test multiple rela- 2007, Fischhoff 1982, Larrick 2004), and correlational tionships within the same sample (Sample 1). Most of evidence has suggested that people who have received the correlation coefficients computed on this sample formal training in decision making may obtain bet- were not significant, but we did observe three small ter life outcomes (Larrick et al. 1993). If so, then the significant correlations. Considering the fact that these bias blind spot represents an obstacle to improving significant correlations were obtained within a larger the quality of both work and life because it bolsters set of multiple comparisons, it is possible that their resistance to debiasing training aimed at improving significance level is inflated. We have refrained from decision-making ability. This influence is not irrevo- drawing strong inferences from those results. cable. We have found that propensity to exhibit bias Although our discriminant validity test shows how blind spot can be reduced by as much as 39% in a the bias blind spot is distinct from and independent related research program consisting of scenarios in of several other constructs, in Studies 3–5 we did not which participants could exhibit bias blind spot and pit our scale against other potential predictors of the were then provided with critical feedback and training judgments and behaviors investigated. Our choice not (Symborski et al. 2014). to include measures for which we would expect to Third, some have argued that the bias blind spot is find null results was driven by the tradeoff between a critical determinant of misunderstanding, mistrust, ruling out a potential alternative and using a length- and pessimism when attempting to reach agreements ier study likely to induce participant fatigue and at an in interpersonal, political and professional relation- additional cost. Future research may compare the pre- ships as it obstructs the resolution of conflicts (Pronin dictive power of the bias blind spot scale with that of et al. 2004), and have highlighted the importance of other traits and biases. mitigating bias blind spot to promote dialogue, diplo- macy, and peace processes (Pronin et al. 2004). Pronin Conclusion et al.(2006), for example, elucidated the consequences We find that bias blind spot is a latent factor in of perceiving others as more biased than oneself in the self-assessments of relative vulnerability to bias. This

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. context of terrorism. When terrorists were depicted as meta-bias affected the majority of participants in our biased and irrational rather than objective and ratio- samples, but exhibited considerable variance across nal, people indicated a greater preference for a mili- participants. We present a concise, reliable, and valid tary over a diplomatic resolution to a conflict, which is measure of individual differences in bias blind spot likely to result in a spiral of conflict escalation. Assess- that has the ability to predict related biases in self- ing individual differences in bias blind spot may help assessment, advice taking, and responsiveness to bias predict the likelihood of interpersonal conflict, misun- reduction training. Given the influence of bias blind derstanding, and the need for dispute resolution. For spot on consequential judgments and decisions, as example, high susceptibility to the bias blind spot may well as receptivity to training, this measure may prove Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2484 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

useful across a broad range of domains such as per- Baron J, Brown RV (1991) Teaching Decision Making to Adolescents sonnel assessment, information analysis, negotiation, (Erlbaum, Hillsdale, NJ). consumer decision making, and education. Berinsky AJ, Huber GA, Lenz GS (2012) Evaluating online labor markets for experimental research: Amazon.com’s mechanical Turk. Political Anal. 20:351–368. Acknowledgments Blais AR, Thompson MM, Baranski JV (2005) Individual differences The authors gratefully recognize the research support of a in decision processing and confidence judgments in compar- contract [FA8650-11-C-7175] from the Intelligence Advanced ative judgment tasks: The role of cognitive styles. Personality and Individual Differences Research Projects Activity (IARPA) via the Air Force 38:1701–1713. Bollen K, Lennox R (1991) Conventional wisdom on measurement: Research Laboratory (AFRL) to Carey K. Morewedge and A structural equation perspective. Psych. Bull. 110:305–314. Karim S. Kassam. The U.S. Government is authorized to Bonaccio S, Dalal RS (2006) Advice taking and decision-making: reproduce and distribute reprints for Governmental pur- An integrative literature review, and implications for the orga- poses notwithstanding any copyright annotation thereon. nizational sciences. Organ. Behav. Human Decision Processes 101: The views and conclusions contained herein are those of 127–151. the authors and should not be interpreted as necessarily Bornstein BH, Zickafoose DJ (1999) “I know I know it, I know I saw it”: The stability of the confidence–accuracy relationship representing the official policies or endorsements, either across domains. J. Experiment. Psych.: Appl. 5:76–88. expressed or implied, of IARPA, AFRL, or the U.S. Govern- Brodnick RJ, Ree MJ (1995) A structural model of academic perfor- ment. The authors thank Nino Miceli for advice on the data mance, socio-economic status, and Spearman’s g. Educational analysis. Psych. Measurement 55:594–605. Bruine de Bruin W, Parker AM, Fischhoff B (2007) Individual dif- Appendix. Fundamental Attribution Error ferences in Adult Decision-Making Competence. J. Personality Training Soc. Psych. 92:938–956. Budescu DV, Rantilla AK (2000) Confidence in aggregation of expert opinions. Acta Psychologica 104:371–398. Debiasing Training Cacioppo JT, Petty RE (1982) The need for cognition. J. Personality Please read the following: Soc. Psych. 42:116–131. A mistake in judgment that people often make is under- Cacioppo JT, Petty RE, Kao CE (1984) The efficient assessment of estimating how much a situation can determine someone’s need for cognition. J. Personality Assessment 48:306–307. behavior. This error occurs when we think another person’s Cacioppo JT, Petty RE, Feinstein JA, Jarvis WBG (1996) Disposi- action tell us something meaningful about their personal- tional differences in cognitive motivation: The life and times ity when instead anyone placed in their shoes would have of individuals varying in need for cognition. Psych. Bull. 119: 197–253. acted the same way. For example, one might see people Cohen J (1988) Statistical Power Analysis for the Behavioral Sciences, in anxiety-inducing situations (e.g., bungee jumping for the 2nd ed. (Lawrence Erlbaum Associates, Hillsdale, NJ). first time), and infer that they are anxious people in general, Crowne DP, Marlowe D (1960) A new scale of social desirabil- rather than infer that they are average people in a nerve- ity independent of psychopathology. J. Consulting Psych. 24: racking situation. Similarly, one may think that an author 349–354. paid to write in favor of a political leader has actually a Cronbach LJ, Meehl PE (1955) Construct validity in psychological tests. Psych. Bull. 52:281–302. favorable opinion of that political leader. Dunning D, Johnson K, Ehrlinger J, Kruger J (2003) Why people To combat this error, we have to acknowledge the impor- fail to recognize their own incompetence. Current Directions tance of situations in determining behavior. Psych. Sci. 12:83–87. Edwards JR, Bagozzi RP (2000) On the nature and direction of the Control Training relationship between constructs and measures. Psych. Methods Please read the following: 5:155–174. Our decisions to trust people with our money are based Epley N, Dunning D (2000) Feeling “holier than thou”: Are self- more on how they look then how they behave, according serving assessments produced by errors in self- or social prediction? J. Personality Soc. Psych. 79:861–875. to a new study. Researchers found people are more likely Feingstein A, Scheier M, Buss A (1975) The self-consciousness scale. to invest money in someone whose face is generally per- J. Consulting and Clinical Psych. 43:522–527. ceived as trustworthy, even when they are given negative Fischhoff B (1982) Debiasing. Kahneman D, Slovic P, Tversky A, information about this person’s reputation. eds. Judgment Under Uncertainty: Heuristics and Biases (Cam- The researchers found that 13 out of 15 participants bridge University Press, Cambridge), 422–444. invested more, on average, in the trustworthy identities. Frederick S (2005) Cognitive reflection and decision making. J. Econom. Perspect. 19:24–42. In a second experiment, the researchers gave the volun- Frey MC, Detterman DK (2004) Scholastic assessment or g? The teers information about whether the trustees had good or relationship between the scholastic assessment test and general

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. bad histories. Even with this inside information, the average cognitive ability. Psych. Sci. 15:373–378. amount invested in those who looked “trustworthy” was Gino F (2008) Do we listen to advice just because we paid for 6% higher. it? The impact of advice cost on its use. Organ. Behav. Human Decision Processes 107:234–245. Gino F, Moore DA (2007) Effects of task difﬁculty on use of advice. J. Behav. Decision Making 20:21–35. References Harvey N, Fischer I (1997) Taking advice: Accepting help, improv- Aiken LS, West SG (1991) Multiple Regression: Testing and Interpret- ing judgment and sharing responsibility. Organ. Behav. Human ing Interactions (Sage, Newbury Park). Decision Processes 70:117–133. Ames DR, Rose P, Anderson CP (2006) The NPI-16 as a short Horn JL (1965) A rationale and test for the number of factors in measure of narcissism. J. Res. Personality 40:440–450. factor analysis. Psychometrika 32:179–185. Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences Management Science 61(10), pp. 2468–2486, © 2015 INFORMS 2485

Horton JJ, Rand DG, Zeckhauser RJ (2011) The online laboratory: Paulhus DL, Harms PD, Bruce MN, Lysy DC (2003) The over- Conducting experiments in a real labor market. Experiment. claiming technique: Measuring self-enhancement independent Econom. 14:399–425. of ability. J. Personality Soc. Psych. 84:890–904. John OP, Srivastava S (1999) The Big Five trait taxonomy: History, Pronin E (2007) Perception and misperception of bias in human measurement and theoretical perspectives. Pervin LA, John judgment. Trends in Cognitive Sci. 11:37–43. OP, eds. Handbook of Personality: Theory and Research (Guilford, Pronin E (2008) How we see ourselves and how we see others. New York), 102–138. Science 320:1177–1180. Pronin E, Kugler MB (2007) Valuing thoughts, ignoring behavior: Johnson TR, Budescu DV, Wallsten TS (2001) Averaging probabil- The introspection illusion as a source of the bias blind spot. J. ity judgments: Monte Carlo analyses of asymptotic diagnostic Experiment. Soc. Psych. 43:565–578. value. J. Behavioral Decision Making 14:123–140. Pronin E, Lin DY, Ross L (2002) The bias blind spot: Perceptions Jones EE, Harris VA (1967) The attribution of attitudes. J. Experi- of bias in self versus others. Personality Soc. Psych. Bull. 28: ment. Soc. Psych. 3:1–24. 369–381. Kahneman D, Slovic P, Tversky A (1982) Judgment Under Uncer- Pronin E, Gilovich TD, Ross L (2004) Objectivity in the eye of the tainty: Heuristics and Biases (Cambridge University Press, beholder: Divergent perceptions of bias in self versus others. New York). Psych. Rev. 111:781–799. Klayman J, Soll JB, Gonzalez-Vallejo C, Barlas S (1999) Overconfi- Pronin E, Kennedy K, Butsch S (2006) Bombing versus negotiating: dence: It depends on how, what and whom you ask. Organ. How preferences for combating terrorism are affected by per- Behav. Human Decision Processes 79:216–247. ceived terrorist rationality. Basic Appl. Social Psych. 28:385–392. Kruger J (1999) Lake Wobegon be gone! The “below-average effect” Robbins SB, Patton MJ (1985) Self-psychology and career devel- and the egocentric nature of comparative ability judgments. J. opment: Construction of the superiority and goal instability Personality Soc. Psych. 77:221–232. scales. J. Counseling Psych. 32:221–231. Robinson RJ, Keltner D, Ward A, Ross L (1995) Actual versus Kruger J (2003) Return of the ego—Self-referent information as a assumed differences in construal: “Naive realism” in inter- filter for social prediction: Comment on Karniol (2003). Psych. group perception and conflict. J. Personality Soc. Psych. 68: Rev. 110:585–590. 404–417. Kruger J, Dunning D (1999) Unskilled and unaware of it: How Rosenberg M (1965) Society and the Adolescent Self-Image (Princeton difficulties in recognizing one’s own incompetence lead to University Press, Princeton, NJ). inflated self-assessments. J. Personality Soc. Psych. 77:1121–1134. Ross L, Ward A (1995) Psychological barriers to dispute resolution. Larrick RP (2004) Debiasing. Koehler DJ, Harvey N, eds. Blackwell Adv. Experiment. Soc. Psych. 27:255–304. Handbook of Judgment and Decision Making (Blackwell Publish- Ross L, Ward A (1996) Naiïve realism in everyday life: Implications ers, Oxford, UK), 316–337. for social conflict and misunderstanding. Reed ES, Turiel E, Larrick RP, Soll JB (2006) Intuitions about combining opinions: Brown T, eds. Values and Knowledge. The Jean Piaget Symposium Misappreciation of the averaging principle. Management Sci. Series (Lawrence Erlbaum Associates, Hillsdale, NJ), 103–135. 52:111–127. Scopelliti I, Morewedge CK, Min HL, McCormick E, Kassam KS Larrick RP, Nisbett RE, Morgan JN (1993) Who uses the cost- (2015) Neglect of external demands (NED) scale: Measuring the coherence, consequences, and correction of correspondent benefit rules of choice? Implications for the normative status of inference making. Working paper, City Univeristy London, microeconomic theory. Organ. Behav. Human Decision Processes London. 56:331–347. Symborski C, Barton M, Quinn M, Morewedge CK, Kassam K, Levin IP, Gaeth GJ, Schreiber J, Lauriola M (2002) A new look Korris J (2014) Missing: A serious game for the mitigation at framing effects: Distribution of effect sizes, individual dif- of cognitive bias. Interservice/Industry Training, Simulation Ed. ferences, and independence of types of effects. Organ. Behav. Conf., Orlando, FL, December. Human Decision Processes 88:411–429. Snyder M (1974) Self-monitoring of expressive behavior. J. Person- Liberman V, Minson JA, Bryan CJ, Ross L (2012) Naïve realism and ality Soc. Psych. 30:526–537. capturing the “wisdom of dyads.” J. Experiment. Soc. Psych. Snyder ML, Frankel A (1976) Observer bias: A stringent test 48:507–512. of behavior engulfing the field. J. Personality Soc. Psych. 34: Macmillan NA, Creelman CD (1991) Detection Theory: A User’s 857–864. Guide (Cambridge, New York). Snyder CR, Fromkin HL (1977) Abnormality as a positive char- Mardia KV (1980) 9 Tests of unvariate and multivariate normal- acteristic: Development and validation of a scale measuring ity. Rao CR, ed. Handbook of Statistics, Vol. 1 (Elsevier B.V., need for uniqueness. J. Abnormal Psych. 86:518–527. Snyder CR, Fromkin HL (1980) Uniqueness (Plenum, New York). Amsterdam), 279–320. Spiller SA, Fitzsimons GJ, Lynch JG Jr, McClelland GH (2013) McCrae RR, Costa PT (1987) Validation of the five-factor model Spotlights, floodlights, and the magic number zero: Simple of personality across instruments and observers. J. Personality effects tests in moderated regression. J. Marketing Res. 50: Soc. Psych. 52:81–90. 277–288. Morewedge CK, Kahneman D (2010) Associative processes in intu- Stankov L, Crawford JD (1996) Confidence judgments in studies itive judgment. Trends in Cognitive Sci. 14:435–440. of individual differences. Personality and Individual Differences Morf CC, Rhodewalt F (2001) Unraveling the paradoxes of nar- 21:971–986. cissism: A dynamic self-regulatory processing model. Psych. Stankov L, Crawford JD (1997) Self-confidence and performance Inquiry 12:177–196. on tests of cognitive abilities. Intelligence 25:93–109. Nisbett RE, Ross L (1980) Human Inference: Strategies and Shortcom- Stanovich KE (1999) Who Is Rational? Studies of Individual Differences in Reasoning (Psychology Press, Mahwah, NJ).

Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved. ings of Social Judgment (Prentice-Hall, Englewood Cliffs, NJ). Nisbett RE, Wilson TD (1977) Telling more than we can know: Stanovich KE, West RF (2000) Individual differences in reasoning: Verbal reports on mental processes. Psych. Rev. 84:231–259. Implications for the rationality debate? Behavioral Brain Sci. 23:645–665. Nunnally JC (1978) Psychometric Theory, 2nd ed. (McGraw-Hill, Stanovich KE, West RF (2007) Natural myside bias is independent New York). of cognitive ability. Thinking and Reasoning 13:225–247. Paolacci G, Chandler J, Ipeirotis PG (2010) Running experiments Stanovich KE, West RF (2008) On the relative independence of using Amazon mechanical Turk. Judgment and Decision Making thinking biases and cognitive ability. J. Personality Soc. Psych. 5:411–419. 94:672–695. Paulhus DL (1998) Interpersonal and intrapsychic adaptiveness of Taylor SE, Brown JD (1988) Illusion and well-being: A social trait self-enhancement: A mixed blessing? J. Personality Soc. psychological perspective on mental health. Psych. Bull. 103: Psych. 74:1197–1208. 193–210. Scopelliti et al.: Bias Blind Spot: Structure, Measurement, and Consequences 2486 Management Science 61(10), pp. 2468–2486, © 2015 INFORMS

Tversky A, Kahneman D (1974) Judgment under uncertainty: West RF, Meserve RJ, Stanovich KE (2012) Cognitive sophistication Heuristics and biases. Science 185:1124–1130. does not attenuate the bias blind spot. J. Personality Soc. Psych. Unsworth N, Engle RW (2007) The nature of individual differences 103:506–513. in working memory capacity: Active maintenance in primary Wilson TD, Brekke N (1994) Mental contamination and mental cor- memory and controlled search from secondary memory. Psych. rection: Unwanted influences on judgments and evaluations. Rev. 114:104–132. Psych. Bull. 116(1):117–142. Wilson TD, Dunn EW (2004) Self-knowledge: Its limits, value, and Van Boven L, Dunning D, Loewenstein G (2000) Egocentric empa- potential for improvement. Ann. Rev. Psych. 55:493–518. thy gaps between owners and buyers: Misperceptions of the Wilson TD, Houston CE, Etling KM, Brekke N (1996) A new look at endowment effect. J. Personality Soc. Psych. 79:66–76. anchoring effects: Basic anchoring and its antecedents. J. Exper- West RF, Stanovich KE (1997) The domain specificity and gener- iment. Psych.: General 125:387–402. ality of overconfidence: Individual differences in performance Yaniv I (2004) Receiving other people’s advice: Influence and ben- estimation bias. Psychonomic Bull. Rev. 4:387–392. efit. Organ. Behav. Human Decision Processes 93:1–13. West RF, Toplak ME, Stanovich KE (2008) Heuristics and biases as Yaniv I, Kleinberger E (2000) Advice taking in decision mak- measures of critical thinking: Associations with cognitive abil- ing: Egocentric discounting and reputation formation. Organ. ity and thinking dispositions. J. Educational Psych. 100: 930–941. Behav. Human Decision Processes 83:260–281.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. You are free to download this work and share with others for any purpose, even commercially if you distribute your contributions under the same license as the original, and you must attribute this work as “Management Science. Copyright 2015 INFORMS. http://dx.doi.org/10.1287/mnsc.2014.2096, used under a Creative Commons Attribution License: http://creativecommons.org/licenses/by-sa/4.0/.” Downloaded from informs.org by [138.40.68.78] on 21 October 2015, at 07:31 . For personal use only, all rights reserved.