Does Anything Predict Anchoring ?

Matthew Brian Welsh ([email protected]) Australian School of Petroleum, University of Adelaide North Terrace, Adelaide, SA 5005, Australia

Abstract perhaps being more inclined to duly consider the anchor and Anchoring – the tendency for recently seen numbers to affect thus be affected by it, or more agreeable people being more estimates – is a robust bias affecting expert and novice willing to go along with it as a suggested answer, but this judgements across many fields. An anchoring task, in which paper also specifically failed to replicate the finding relating people (N=301) estimated the number of circles in 10 openness to anchoring. stimulus figures after comparison to an anchoring value, was Given anchoring can include both a effect, conducted within a larger study including numerous altering the information called from , and an intelligence, personality, decision style and adjustment process in which iterative changes are made to measures. Individual anchoring susceptibility was calculated and compared to potential predictor variables. Two of eight an estimate (Kahneman, 2011), it also seems plausible that it broad ability measures (from Catell-Horn-Carroll intelligence would be reduced in people with better cognitive abilities – theory) correlated weakly but significantly with anchoring such as memory, numerical or reasoning skills. (Gq = 0.16, Gf = 0.12). No decision style or attention Links with intelligence, however, have been weak. measures correlated significantly with anchoring, nor did the Oechssler, Roider and Schmitz (2009), for example, found Big 5 personality traits, directly. Indirectly, however, as the no evidence of higher cognitive ability predicting lessened anchoring task continued and fatigue increased, people relied anchoring whereas Bergman et al (2010) found evidence for more on anchors and higher neuroticism may have increased this tendency. Overall, results suggest our ability to predict it reducing but not eliminating the effect. Welsh et al (2014) anchoring is poor and implications of this are discussed. found suggestive evidence for it reducing susceptibility to anchoring and somewhat stronger evidence for a rational Keywords: anchoring bias; intelligence; personality; decision decision style and cognitive reflection (Frederick, 2005) styles; attention. doing the same - but only as a result of learning across the Introduction duration of an extended task in both cases. Part of the problem here, though, as Welsh, Burns and Anchoring-and-adjustment (hereafter ‘anchoring’) describes Delfabbro (2013) point out, is that most tests of cognitive the tendency for people’s numerical estimates to be biased ability used in bias research are simple proxies for the far towards recently seen numbers – regardless of the relevance more complex hierarchical intelligence suggested by of those numbers to the task at hand (Tversky & Kahneman, modern Catell-Horn-Carroll theory (CHC; McGrew, 2009) 1974). This effect is robust, affecting judgements across a and selection of personality traits and decision style wide range of tasks and domains (see, e.g., Chapman & measures is often based on convenience and simplicity due Johnson, 1994; Furnham & Boo, 2011) and resisting many to the time and cost of conducting complete measures. efforts at (see, e.g., Mussweiler, Strack, & Given this, a gap exists for a more complete look at how Pfeiffer, 2000; Wilson, Houston, Etling, & Brekke, 1996). anchoring relates to the wide range of human abilities This effect has significant practical implications, affecting falling under the individual differences umbrella. This will experts judgements in: pricing and negotiations (Northcraft enable us to state with greater certainty whether individual & Neale, 1987); technical estimates under uncertainty traits can predict susceptibility to this bias and, thus, the (Welsh & Begg, 2016); and forecasting (Lawrence & extent to which less biased personnel can be selected or O'Connor, 1992). Given this, there is significant utility in identified for roles in which anchoring is likely to have predicting susceptibility to anchoring – to enable significant economic or human consequences. identification of people in need of extra support to avoid the A wide-reaching exploration of individual differences effect or selection of people more resistant to the bias. The also has the potential to shed light on the processes that give literature, however, shows inconsistent results, perhaps rise to the anchoring effect. For example, the CHC model of because the application of individual differences research to intelligence includes ten broad cognitive abilities that the study of anchoring has been somewhat piecemeal. correspond to different aspects of memory, types of Consider, for example, the Big 5 personality traits (Costa reasoning and cognitive speed. If anchoring is linked to & McCrae, 1992). McElroy and Down (2007), starting with these, this could indicate fruitful paths for further research the hypothesis that openness-to-experience includes a into the cognitive processes underlying the bias. greater willingness to take on outside influences, found evidence that people with higher openness were more Method susceptible to anchoring. Eroglu and Croxton (2010), however, linked anchoring with high conscientiousness and Participants agreeableness and with low extraversion. Again, this has some face validity, in terms of more conscientious people Participants were 301 university students, graduates and

3089 members of the general public recruited as part of a larger participants completed the tasks in the order shown in Table study; 172 identifying as female, 120 male and 9 non- 1 but circle locations were randomly generated in each new binary. Participants listed 34 countries-of-origin but the stimulus figure. majority (207) indicated English was their first language. Most were university educated, including 26 with post- Table 1. Order of stimuli and anchors (H-high or L-low) graduate degrees, 101 with bachelor degrees (including 38 1: 150H 2: 250L 3: 200H 4: 250H 5: 350L current post-grads) and 107 current university students. 6: 150L 7: 200L 8: 300L 9: 300H 10: 350H Ages ranged from 18 to 79 (M = 28.8, SD = 12.8).

Materials Anchoring scores were calculated from the difference between estimates on the high and low anchor pairs, Demographics adjusted for the actual number of circles in the stimulus Demographics were gathered via online survey prior to figure. That is: online and in-person individual differences and experimental measures. These included: age; gender; native Anch = (EstHighAnchor – EstLowAnchor) / True # of circles vs non-native English speaker; and educational attainment - recorded via options ranging from ‘Did not complete High Higher numbers thus reflect greater impact of the anchors. School’ to ‘Doctorate’. The average value across the 5 pairs was taken as a participant’s final anchoring susceptibility score. Anchoring The anchoring task was written as a graphical user Cognitive Abilities interface (GUI) in Matlab. The program showed participants Tasks reflecting 8 broad abilities from the Cattell-Horn- stimuli consisting of a number of small circles. Pre-task Carroll model of intelligence (CHC; McGrew, 2009) were instructions were as follows: included. Six of these abilities were represented by 3 tasks When the program begins, you will be shown a series of each, allowing a factor score to be extracted using PAF with patterns of circles drawn here. For each, your task is to direct oblimin rotation in SPSS (NB – in each case a single decide whether the number of circles is less or greater than factor was returned using eigenvalues>1). The remaining a number you will be shown and then to give your best two were represented by single tasks, as described below. estimate of the number of circles. There is no time limit but Gf. Fluid ability extracted (KMO = .666, Bartlett’s Test of the task is to estimate – not count – the circles. After a Sphericity χ2(3) = 218, p<.001) from 12-item Ravens APM participant ID has been entered, press START when you are (Arthur Jr & Day, 1994), CAB-I (Hakstian & Bennet, 1977) ready to see the first figure. and WJ-IV Number Series (this and all subsequent measures labelled WJ-IV are from the Woodcock-Johnson IV Tests of Figure 1. Anchoring GUI showing first stimulus figure. Cognitive Abilities, Schrank & Wendling, 2018). Gc. Crystallized ability extracted (KMO = .668, Bartlett’s Test of Sphericity χ2(3) = 243, p<.001) from the Mill-Hill Vocabulary Scale (Raven & Court, 1998), Spot-the-Word (Baddeley, Emslie, & Nimmo‐Smith, 1993) and WJ-IV Oral Vocabulary. Gsm. Short term memory extracted (KMO = .669, Bartlett’s Test of Sphericity χ2(3) = 173, p<.001) from WJ- IV Numbers Reversed, Memory Span Forward and Memory Span Backwards. The memory span tasks were written for this project in Matlab. Each displayed numbers of increasing length – from 1 to 10 – presented one digit at a time at 1 second intervals. After presentation, participants were asked to enter the digits in either the order presented or reversed order. Scores were the number of digits correctly recalled out of the total of 55 (10+9+8+… +1) from each task. Glr. Long term retrieval ability extracted (KMO = .612, On starting the task, participants were shown a stimulus Bartlett’s Test of Sphericity χ2(3) = 239, p<.001) from WJ- figure as shown in Figure 1 and asked to indicate whether IV Rapid Picture Naming, WJ-IV Retrieval Fluency and a the number of circles was greater or less than the anchoring Comprehension task. The comprehension tasks was written number shown before entering their best estimate. in Matlab for this project and required participants to read Participants completed 10 trials, consisting of 5 pairs of two ~500 word passages about historical events and answer figures with 150, 200, 250, 300 and 350 circles. One four multiple (4) choice questions after each. Scores were member of each pair was shown with a low anchor (50% of the number of questions correctly answered out of eight. the actual value) and the other a high anchor (150%). All Gq. Quantitative ability extracted (KMO = .642, Bartlett’s

3090 Test of Sphericity χ2(3) = 113, p<.001) from the 12-Item participant information, consent and demographic data Numerical Aptitude Test (Welsh et al., 2013), Berlin collection. Four-hundred and four participants complete this Numeracy Test (Cokely, Galesic, Schulz, Ghazal, & Garcia- initial survey. Retamero, 2012) and Subjective Numeracy Test (Fagerlin et Following this, participants invited to a second survey al., 2007). including: the personality and decision style questions; a Gt. Decision Speed extracted (KMO = .613, Bartlett’s subjective attention scale; the subjective numeracy scale; Test of Sphericity χ2(3) = 136, p<.001) from Inspection and the spot-the-word test. On average, participants spent 55 Time (Preiss & Burns, 2012), Simple Reaction Time and minutes on these tasks. Go-No Go Reaction Time tasks written for this project in On completion, participants were invited to a third survey, Matlab. The SRT asked participants to press a key as soon which included: the cognitive reflection test; the cognitive as a red ‘R’ appeared onscreen for 10 trials. The Go-No Go abilities measures not conducted in person (see below); and required responses only to the letter ‘E’, which occurred on a wide variety of bias tasks not discussed herein (e.g., 10 out of 100 trials (ten each of each letter from A to J, framing, , gamblers fallacy, sample size presented to participants in the same, randomized order. invariance, etc). Participants took around 2 hours to Gs. WJ-IV Letter-Pattern. complete these tasks. Gv. WJ-IV Visualization. Finally, participants were invited to a 2 hour, in-person All measures were scored such that higher numbers session, during which they completed the remaining indicated better performance. cognitive ability measures including: the WJ-IV tasks; the numerical abilities test; the comprehension task; the Personality Traits memory span tasks; and a series of computerised tasks The Big 5 personality traits were measured using the full yielding the inspection time, reaction time and attention version of the NEO-PI3 personality test (McCrae, Costa, & measures. The anchoring task was completed as part of the Martin, 2005). This yields 30 facets scales in addition to the computerised tests noted above. Additional bias tasks were five main traits. also included at the end of this session, examining tendencies such as hindsight bias and susceptibility to the Decision Styles availability effect. Overall, 301 of the 404 participants who Six decision style measures were included. These were: a signed up online completed the testing. cognitive reflection test (CRT) incorporating both Frederick’s (2005) and Thomson and Oppenheimer’s (2016) Results questions (NB - whether CRT is a decision style, a cogntiive measure or a combination of the two is still debated in the Anchoring Bias literature, see, e.g., Welsh et al., 2013); the Rationality and Intuition scales from the Decision Styles Scale (Hamilton, The mean anchoring score was 0.18 – indicating that Shih, & Mohammed, 2016); Need for Cognitive Closure estimates made after seeing the high anchor were 18% (of (Webster & Kruglanski, 1994); Actively Open-Minded the true value) higher than those made after seeing the low Thinking (Haran, Ritov, & Mellers, 2013); and the Brief anchor on equivalent stimuli. The overall effect of Maximization Scale (Nenkov, Morrin, Schwartz, Ward, & Anchoring bias was tested for using a single sample t-test, Hulland, 2008), which distinguishes between preferences comparing participants’ anchoring scores with the value of for satisficing versus optimization. zero expected if anchors had no effect on estimates. This confirms a significant impact of anchors on participants’ Attention Measures estimates, t(300) = 18.46, p<.001, CI95% [0.161, 0.200]. Six tasks were included in the in-person tasks measuring the following aspects of attention: focused attention; Demographics sustained attention; selective attention; alternating attention; Participant demographics were examined to test whether divided attention; and inhibition (defined as per Welsh, any of these mediated anchoring susceptibility. Age did not 2020a). These overlap the decision speed measures to some correlate significantly with anchoring, r(299) =.059, p = extent, being derived from a set of reaction time tasks, but .309. Likewise, an independent samples t-test indicated additionally incorporate errors of omission and commission native English speakers did not differ from non-native in and changes in RT within and across conditions and tasks. anchoring susceptibility, t(299) = .89, p = .369. Female participants had a slightly higher mean anchoring Procedure scores (M = .191) than males (0.165) or non-binary The anchoring task and other measures described above participants (.164) but a one-way ANOVA indicated these were part of a larger study on . Participants were differences were not significant F(2, 298) = 0.83, p = .437. recruited online and on campus but needed to be available to The effect of educational level was examined using a one- come to campus for testing sessions. The testing was way ANOVA after dividing the participants into 5 groups – conducted in four parts and participants were paid $100 on high school educated, post-secondary diplomas and completion. The first was an online survey that combined certificates, university students, university graduates and post-doctoral graduates. The mean anchoring scores for 3091 these groups ranges from 0.167 (diplomas, etc) to 0.187 Looking at Table 4, one sees that all correlations with (postgraduate degrees) but the test indicated no significant anchoring are less than ±0.1 and none are significant. By differences between the means, F (4,296) = .142, p = .966. contrast, there is evidence that the different decision styles measures are related, with several moderate relationships Cognitive Ability between them. Interestingly, in light of debate about what Pearson correlations calculated between anchoring scores the CRT measures, it shows the weakest relationships to the and the eight cognitive ability scores are shown in Table 2. other decision style measures. (Additional analyses showed it correlated significantly with all of the intelligence Table 2. Correlations between anchoring and cognitive measures - including r = 0.59 with Gf and r = 0.45 with Gq, abilities. p<.001 in both cases.) Gf Gc Gsm Glr Anch -.119* -0.098 -0.087 -0.001 Attention Measures Gq Gt Gs Gv Correlations coefficients were calculated for twenty-one Anch -.162** -0.070 -0.063 -0.064 measures extracted from the six attention tasks with *- sig at .05 (2-tailed). **- sig at 0.01 (2-tailed). N=297-301. anchoring, as shown in Table 5. Looking at Table 5, ones sees all correlations are less than Looking at Table 2, all eight cognitive abilities are ±0.1 and none are significant, showing no evidence of negatively correlated with anchoring scores – i.e., relationships between attention and anchoring susceptibility. suggesting higher cognitive abilities lessen anchoring susceptibility. Only the correlations with Gf and Gq, Changes in Anchoring however, are significant and these are very weak In light of previous work finding changes in anchoring relationships. (NB - a Bonferroni correction for family-wise across an experiment (Welsh et al., 2014), an examination alpha accounting for 8 comparison would result in only the of anchoring across the 10 trials was also conducted. For Anchoring-Gq correlation remaining significant, p=.04, this, an anchoring reliance score was calculated for each two-tailed, but given the overall pattern of results, known trial as the absolute difference between the anchor and the relationships between the cognitive measures and two estimate, adjusted for the magnitude of the estimated value significant results out of eight, such a correction seems (i.e., |A-E|/E), with lower scores thus indicating greater unnecessary). reliance. To see if people’s reliance on the anchor changed across Personality Traits the task, average anchor reliance scores were calculated for Pearson correlations calculated between anchoring scores each participant on the first 5 and last 5 trials on the overall and the Big 5 personality traits are shown in Table 3. task. These were compared using a Wilcoxon sign-rank test, revealing reliance on the anchor values increased as the Table 3. Correlations between anchoring and Big 5 task went on, Median1-5 = 0.662, Median6-10 = 0.445, z(300) personality traits (N=301). = -11.38, p <.001. N E O A C Changes in reliance were largely unrelated to the Anch -0.074 -0.060 -0.064 0.056 0.076 demographic, cognitive ability, decision style and attention measures. Of the personality traits, only Neuroticism Looking at Table 3, the personality traits show no clear correlated significantly with the change in reliance, r = - relationship to anchoring, with all correlations less than ±0.1 .123, p = .033 (two-tailed). That is, more neurotic people and none significant. Further examination using the 30 showed greater increases in reliance across the task. facets showed four weak, significant relationships with anchoring: Neuroticism-Depression (r = -.113, p=.049); Table 4. Correlations between anchoring and decision style Openness-Fantasy (r= -.139, p=.016); Openness-Ideas (r= - measures. .113, p=.05); and Conscientiousness-Deliberation (r= .155, 1 2 3 4 5 6 7 p=.007). Correcting the family-wise alpha values for 30 1.Anch -.09 .07 -.01 -.03 -.02 -.04 .103 .12 -.11 -.03 -.17 -.10 comparisons, however, left none of these as significant. 2.CRT 3.DSR .235 .039 -.24 .16 .03 .20 4.DSI .808 .049 <.001 .13 .31 .06 Decision Styles 5.NFCC .659 .596 .005 .021 .13 .33 Pearson correlations were calculated between anchoring 6.AOT .752 .004 .649 <.001 .020 .29 scores and the decision style measures. Given the 7.BMax .469 .084 .001 .287 <.001 <.001 relationships between the different decision style measures Note: top triangle shows correlation coefficients; bottom are less well-known than those between the cognitive and triangle shows p-values (2-tailed). Sig. results are bolded. personality traits, the full correlation matrix is shown in N=301 except for CRT where N=300. Table 4 - rather than just the relationships between the decision styles measures and anchoring. Table 5. Correlations between anchoring score and attention measures. 3092 Foc. Foc. Sus. Sus. Sus. Sel. Sel. variety of potential strategies to be used. This lack of RT Err. RT Err. Corr. dRT dErr constraints on how to solve the problem could enable a much greater diversity of outcomes from people with the r .091 .054 .030 .014 -.007 -.028 -.012 same level of ability as differences in performance could be Alt. Alt. Div. Div. Div. Div. Inh. caused by differences in the adopted strategy rather than dRT dErr RT Corr. RT Err. RT ability per se. Corr. Err. Corr. r .067 -.083 -.015 -.097 -.057 .036 -.026 Caveats and Future Research Inh. Inh. Inh. Sus. Inh Div. All A key result, however, needs to be discussed in greater Corr. RT Err. dRT dRT dRT Err. detail as it may impact the interpretation of the experiment. Err. This is the observation that reliance on anchoring values r -.035 .013 .035 -.059 -.063 -.054 -.022 increased during the task. The simplest explanation of this is Note: N=259-301. No correlations are significant at the .05 that participants were getting increasingly tired or unhappy, level. Foc. = focused, Sus. = sustained, Sel. = selective, Alt. making them more susceptible to bias as the trials continued = alternating, Div = divided, Inh. = inhibition. RT = reaction (see, e.g., Danziger, Levav, & Avnaim-Pesso, 2011; Englich time, Err. = number of errors, Corr. = number correct, dRT & Soder, 2009). = change in RT, dErr. = change in # errors. If this is correct, the fact that higher neuroticism may exacerbate this – correlating with increased reliance on the Discussion anchor as the task proceeded (r = -0.123) - is unsurprising as this trait measures emotional stability. The above results paint a clear, if bleak, picture: even It does, however, indicate a potential confound in this with a wide range of personal traits and a sizeable sample, experiment’s design – where the anchoring task – in our ability to predict susceptibility to anchoring is poor. addition to being included near the end of a two-hour None of the personality, decision style or attention measure session incorporating multiple tasks, required participants to considered herein showed any significant relationship to complete a series of trials and may, thus, have been doubly people’s susceptibility to anchoring. taxing. (NB – this may also hold true for the attention tasks, The only two traits that correlated significantly with which occurred around the same time and also required anchoring susceptibility in our experiment were two of the multiple trials.) eight intelligence measures – quantitative intelligence (Gq) The relationship seen here with anchoring reliance and at ~0.16 and fluid intelligence (Gf) at ~0.12. That these neuroticism indicates the potential for interaction effects would predict performance does seem reasonable – as they between the durations of the task and the experiment and reflect a person’s ability with numbers and new problems specific traits that could obscure relationships of greater respectively – and the anchoring task was a novel, interest. For example, the role of intelligence, openness or numerical estimation task. The relationships are weak, decision style traits could be larger in less fatigued however, offering little predictive power for those interested participants. (Of course, the relevance of this should be in personnel selection or identification and, given they viewed through the lens of personnel selection and the correlate with one another at 0.6 in our sample, there is no specific tasks being selected for – i.e., whether the role additional predictive power from considering both. requires estimates to be made continuously over an This is, at once, disappointing and not entirely surprising. extended period or more occasionally.) Anchoring has long stood somewhat apart from other biases This could be tested in future research, with the results – an observation recently confirmed in Ceschi et al’s (2019) herein shedding light on the most promising predictors of taxonomy wherein anchoring emerged as distinct from all anchoring to test in a shorter format – Gq and Gf. Another 16 other biases they examined. open question, given the discussion above, is whether This seems to reflect a fundamental difference in the anchoring susceptibility will remain stable across domains. underlying processes involved. While a case can be made that a number of other biases are related to memory Conclusions processes and thus to aspects of intelligence (e.g., hindsight bias, overconfidence, availability. See, e.g., Despite the use of best-practice measures for individual Welsh, 2018; Welsh, 2020b) or emerge from simple differences including the CHC model of intelligence, the logical/numerical problems, anchoring is more complex. Big 5 personality measures and a range of attention and The discussion on the root cause of anchoring has focused decision style measures, we did not find any traits that on whether it is a conscious adjustment process (see, e.g., predict anchoring strongly. In fact, only two measures – Tversky & Kahneman, 1974) or a priming/activation effect quantitative (Gq) and fluid (Gf) intelligence from the CHC linked to confirmatory search (Chapman & Johnson, 1999). model - showed significant relationships with anchoring and In either case, the underlying processing involves both of these were weak (~0.16 and ~0.12). consideration of values for their adequacy – which sounds These relationships may, however, have been deflated by like it should involve Gq and Gf (and memory in the form fatigue, given the length of the experiment and the of domain specific knowledge) but would allow a wide observation that participants were increasingly relying on 3093 the anchors as the task continued (and high neuroticism may Academic Achievement Criteria. Educational and have increased this somewhat). Future research should focus Psychological measurement, 37(2), 425-437. on these traits in smaller experiments where fatigue can be Hamilton, K., Shih, S.-I., & Mohammed, S. (2016). The eliminated as a covariate. development and validation of the rational and intuitive decision styles scale. Journal of Personality Assessment, References 98(5), 523-535. Arthur Jr, W., & Day, D. V. (1994). Development of a short Haran, U., Ritov, I., & Mellers, B. A. (2013). The role of form for the Raven Advanced Progressive Matrices Test. actively open-minded thinking in information Educational and Psychological measurement, 54(2), acquisition, accuracy, and calibration. 394-403. ScholarlyCommons, (5-2013). Baddeley, A., Emslie, H., & Nimmo‐Smith, I. (1993). The Kahneman, D. (2011). Thinking, Fast and Slow. New York, Spot‐the‐Word test: A robust estimate of verbal NY: Farrar, Straus, Giroux. intelligence based on lexical decision. British Journal of Lawrence, M., & O'Connor, M. (1992). Exploring Clinical , 32(1), 55-65. judgemental forecasting. International Journal of Bergman, O., Ellingsen, T., Johannesson, M., & Svensson, Forecasting, 8(1), 15-26. C. (2010). Anchoring and cognitive ability. Economics McCrae, R. R., Costa, J., Paul T, & Martin, T. A. (2005). Letters, 107(1), 66-68. The NEO–PI–3: A more readable revised NEO Ceschi, A., Costantini, A., Sartori, R., Weller, J., & Di personality inventory. Journal of Personality Fabio, A. (2019). Dimensions of decision-making: An Assessment, 84(3), 261-270. evidence-based classification of heuristics and biases. McElroy, T., & Dowd, K. (2007). Susceptibility to Personality and Individual Differences, 146, 188-200. anchoring effects: How openness-to-experience Chapman, G. B., & Johnson, E. J. (1994). The limits of influences responses to anchoring cues. Judgment and anchoring. Journal of Behavioral Decision Making, 7(4), Decision Making, 2(1), 48. 223-242. McGrew, K. S. (2009). CHC theory and the human Chapman, G. B., & Johnson, E. J. (1999). Anchoring, cognitive abilities project: Standing on the shoulders of activation, and the construction of values. the giants of psychometric intelligence research. Organizational Behavior and Human Decision Intelligence, 37(1), 1-10. Processes, 79(2), 115-153. doi:https://doi.org/10.1016/j.intell.2008.08.004 Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia- Mussweiler, T., Strack, F., & Pfeiffer, T. (2000). Retamero, R. (2012). Measuring risk literacy: The Berlin Overcoming the inevitable anchoring effect: considering numeracy test. Judgment and Decision Making, 7(1), 25. the opposite compensates for selective accessibility. Costa, P. T. J., & McCrae, R. R. (1992). NEO PI-R Personality and Bulletin, 26(9), Professional Manual. Odessa, FL: Psychological 1142-1150. Assessment Resources. Nenkov, G. Y., Morrin, M., Schwartz, B., Ward, A., & Danziger, S., Levav, J., & Avnaim-Pesso, L. (2011). Hulland, J. (2008). A short form of the Maximization Extraneous factors in judicial decisions. Proceedings of Scale: Factor structure, reliability and validity studies. the National Academy of Sciences, 108(17), 6889-6892. Judgment and Decision Making, 3(5), 371-388. Englich, B., & Soder, K. (2009). Moody experts---How Northcraft, G. B., & Neale, M. A. (1987). Experts, amateurs mood and expertise influence judgmental anchoring. and real estate: an anchoring-and-adjustment perspective Judgment and Decision Making, 4(1), 41. on property pricing decisions. Organizational Behavior Eroglu, C., & Croxton, K. L. (2010). Biases in judgmental and Human Decision Processes, 39, 84-97. adjustments of statistical forecasts: The role of Oechssler, J., Roider, A., & Schmitz, P. W. (2009). individual differences. International Journal of Cognitive abilities and behavioral biases. Journal of Forecasting, 26(1), 116-133. Economic Behavior & Organization, 72(1), 147-152. Fagerlin, A., Zikmund-Fisher, B. J., Ubel, P. A., Jankovic, Preiss, A. K., & Burns, N. R. (2012). Accurately measuring A., Derry, H. A., & Smith, D. M. (2007). Measuring inspection time with computers. International Journal of numeracy without a math test: Development of the Intelligence Science, 2(4). subjective numeracy scale. Medical Decision Making, doi:DOI:10.4236/ijis.2012.24013 27(5), 672-680. Raven, J. C., & Court, J. H. (1998). Raven's progressive Frederick, S. (2005). Cognitive reflection and decision matrices and vocabulary scales (Vol. 759): Oxford making. Journal of Economic Perspectives, 19(4), 25- pyschologists Press Oxford, England. 42. Schrank, F. A., & Wendling, B. J. (2018). Woodcock– Furnham, A., & Boo, H. C. (2011). A literature review of Johnson IV Tests of Cognitive Abilities Examiner's the anchoring effect. The Journal of Socio-Economics, Manual, Standard and Extended Batteries. Itasca, IL.: 40(1), 35-42. Riverside Publishing. Hakstian, A. R., & Bennet, R. W. (1977). Validity Studies Thomson, K. S., & Oppenheimer, D. M. (2016). Using the Comprehensive Ability Battery (CAB): 1. Investigating an alternate form of the cognitive

3094 reflection test. Judgment and Decision Making, 11(1), 99. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124- 1131. Webster, D. M., & Kruglanski, A. W. (1994). Individual differences in need for cognitive closure. Journal of Personality and Social Psychology, 67(6), 1049. Welsh, M. B. (2018). Bias in Science and Communication. Bristol, UK: IOP Publishing. Welsh, M. B. (2020a). May I Have Your Attention? Testing a Subjective Attention Scale. In S. Denison, M. Mack, Y. Xu, & B. C. Armstrong (Eds.), Proceedings of the 42nd Annual Conference of the Cognitive Science Society (pp. 2535-2541): Cognitive Science Society. Welsh, M. B. (2020b). Overconfident in Hindsight: Memory, Hindsight Bias and Overconfidence. In S. Denison, M. Mack, Y. Xu, & B. C. Armstrong (Eds.), Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (pp. 822-828): Cognitive Science Society. Welsh, M. B., & Begg, S. H. (2016). What have we learnt? Insights from a decade of bias research. APPEA Journal, 56, 435-450. Welsh, M. B., Burns, N., & Delfabbro, P. (2013). The Cognitive Reflection Test: how much more than numerical ability. Paper presented at the Proceedings of the 35th Annual Conference of the Cognitive Science Society. Welsh, M. B., Delfabbro, P. H., Burns, N. R., & Begg, S. H. (2014). Individual differences in anchoring: traits and experience. Learning and Individual Differences, 29, 131-140. doi:10.1016/j.lindif.2013.01.002 Wilson, T. D., Houston, C. E., Etling, K. M., & Brekke, N. (1996). A new look at anchoring effects: basic anchoring and its antecedents. Journal of Experimental Psychology: General, 125(4), 387-402.

3095