Confirmation Bias
Total Page:16
File Type:pdf, Size:1020Kb
A Thesis Submitted for the Degree of PhD at the University of Warwick Permanent WRAP URL: http://wrap.warwick.ac.uk/95233/ Copyright and reuse: This thesis is made available online and is protected by original copyright. Please scroll down to view the document itself. Please refer to the repository record for this item for information to help you to cite it. Our policy information is available from the repository home page. For more information, please contact the WRAP Team at: [email protected] warwick.ac.uk/lib-publications UNIVERSITY OF WARWICK The importance of making assumptions: why confirmation is not necessarily a bias by Jess Whittlestone A thesis submitted in partial fulfillment for the degree of Doctor of Philosophy in the Behavioural Science Group Warwick Business School July 2017 Contents List of Figures vi List of Tables vii Acknowledgements viii Declaration of Authorship x Abstract xii 1 Introduction 1 2 When is confirmation a bias? 7 2.1 Introduction . .7 2.2 Different types of confirmation bias: a review . .9 2.2.1 Bias in search . 12 2.2.1.1 Bias in hypothesis testing - Wason's original experiments 14 2.2.1.2 Subsequent research on hypothesis testing . 15 2.2.1.3 Selective exposure . 17 2.2.1.4 Bias in amount of time spent searching . 19 2.2.1.5 `Myside bias' in producing arguments . 20 2.2.1.6 Bias in search: discussion . 21 2.2.2 Bias in inference . 24 2.2.2.1 Biased interpretation of evidence . 26 2.2.2.2 Biased evaluation of evidence . 30 2.2.2.3 Bias in inference: discussion . 31 2.2.3 The relationship between bias in search and inference . 34 2.2.4 Biased beliefs: belief persistence and overconfidence . 35 2.2.4.1 Belief persistence . 36 2.2.4.2 Overconfidence . 39 2.2.4.3 Belief persistence and overconfidence: discussion . 41 2.2.4.4 Biased beliefs independent of search and inference? . 42 2.2.5 Summary: the evidence for confirmation bias . 44 2.3 The challenges to confirmation bias . 46 i CONTENTS ii 2.3.1 Strategies that look ‘confirmatory’ do not necessarily lead to a confirmation bias in all circumstances . 47 2.3.2 Strategies that look like they produce `bias' may in fact be accurate on average . 48 2.3.3 Problems with experimental design . 50 2.3.4 Lack of normative standards . 51 2.3.5 Bias in search and inference cannot be studied in isolation . 54 2.4 What remains of confirmation bias? . 56 2.5 Conclusion . 64 3 The mixed evidence for selective exposure 65 3.1 Introduction . 65 3.2 Background . 67 3.3 Past research on selective exposure . 69 3.3.1 A broad overview . 70 3.3.2 A narrower look at selective exposure - social and political attitudes 71 3.3.2.1 Strength of opinions and dissonance . 72 3.3.2.2 Goals and motivations . 74 3.3.2.3 Personality differences . 75 3.3.2.4 Other factors influencing information choice . 76 3.3.3 Naturalistic and field experiments . 78 3.3.4 Summary: the state of selective exposure research . 80 3.4 The present research . 82 3.4.1 Experimental paradigm . 82 3.4.2 Experiment 1: setting up a selective exposure paradigm . 83 3.4.2.1 Background . 83 3.4.2.2 Design and procedure . 84 3.4.2.3 Handling of moderates . 86 3.4.2.4 Results . 87 3.4.2.5 Analysis of moderates . 89 3.4.3 Experiments 2 and 3: how robust is the evidence for selective exposure? . 90 3.4.3.1 Background . 90 3.4.3.2 Design and procedure . 91 3.4.3.3 Results . 92 3.4.4 Experiment 4: a replication of Taber and Lodge, 2006 . 95 3.4.4.1 Background . 95 3.4.4.2 Design and procedure . 95 3.4.4.3 Results . 96 3.4.5 Experiments 5 and 6 . 97 3.4.6 Experiment 5: information source manipulation . 99 3.4.6.1 Design and procedure . 99 3.4.6.2 Results . 99 3.4.6.3 Participant's reported reasons for choices . 101 3.4.7 Experiment 6: information source manipulation 2 . 102 3.4.7.1 Design and procedure . 103 3.4.7.2 Results . 103 CONTENTS iii 3.5 Summary and discussion . 105 3.5.1 Summary . 105 3.5.2 Discussion: just another failed replication? . 106 3.6 General discussion . 110 3.6.1 Design issues in selective exposure research . 110 3.6.2 Different ways of measuring `selective exposure' . 111 3.6.3 Selective exposure and bias . 113 3.6.4 Selective exposure in the `real world' . 115 3.6.5 Final thoughts and implications . 116 4 Bias, rationality and improving human reasoning 118 4.1 Introduction . 118 4.2 What does it mean to be biased? . 120 4.2.1 `Bias' in different areas of research . 120 4.2.2 `Bias' in the confirmation bias literature . 124 4.3 Normative models . 127 4.3.1 The use of normative models in psychology . 127 4.3.2 Normative models in the study of confirmation bias . 129 4.4 What does it mean to be `rational'? . 132 4.4.1 The relationship between bias and rationality . 132 4.4.2 Different types of rationality . 134 4.4.2.1 Summary of different types of rationality . 135 4.4.2.2 Links and overlaps between different types of rationality 138 4.4.3 Disagreements about rationality: summary . 140 4.4.3.1 Substantive disagreement or terminological confusion? . 141 4.4.3.2 Disagreements about rationality in the confirmation bias literature . 144 4.4.4 Why does this matter? Rationality and improving human reasoning147 4.5 Summary . 150 4.5.1 What does it mean to be biased? . 150 4.5.2 Normative models in psychology . 151 4.5.3 Different types of rationality . 151 4.5.4 Bias, rationality, and improving reasoning . 153 4.6 Implications for confirmation bias . 153 5 Open-mindedness 157 5.1 Introduction . 157 5.2 What is open-mindedness? . 159 5.2.1 An intuitive picture of open-mindedness . 159 5.2.2 Open-mindedness in psychology . 161 5.2.2.1 Early accounts of open-mindedness . 161 5.2.2.2 Openness as a personality dimension in the five-factor model . 161 5.2.2.3 Open-mindedness as behaviour . 164 5.2.2.4 Open-mindedness in psychology: summary . 165 5.2.3 Open-mindedness in philosophy . 166 5.2.3.1 Open-mindedness as uncertainty . 166 CONTENTS iv 5.2.3.2 Open-mindedness as intellectual humility . 167 5.2.3.3 Open-mindedness as detachment or engagement . 169 5.2.3.4 Open-mindedness in philosophy: summary . 171 5.2.4 Summary: what is open-mindedness? . 172 5.3 Should we be more open-minded? . 173 5.3.1 The costs of open-mindedness . 174 5.3.2 Why open-mindedness is not a normative concept . 175 5.3.3 Why open-mindedness does not need to be a normative concept . 177 5.3.4 Open-mindedness as an explore-exploit tradeoff . 179 5.3.5 Open-mindedness and science . 180 5.3.6 Summary: would more open-mindedness be better? . 183 5.4 Implications . 184 5.4.1 Implications for the psychological study of open-mindedness . 185 5.4.2 Implications for promoting open-mindedness . 188 5.5 Conclusion . 190 6 Summary and discussion 192 6.1 Summary . 192 6.1.1 When is confirmation a bias? . 192 6.1.2 The mixed evidence for selective exposure . 193 6.1.3 Bias, rationality, and confirmation . 194 6.1.4 Open-mindedness . 195 6.1.5 Confirmation bias, open-mindedness, and Bayes' rule . 196 6.2 Discussion . 198 6.2.1 Why is the belief in confirmation bias so pervasive? . 198 6.2.1.1 A theoretical case for confirmation bias? . 199 6.2.1.2 Confirmatory reasoning as a (not necessarily irrational) tendency . 200 6.2.1.3 Confirmation bias as a convenient explanation . 201 6.2.1.4 Confusing confirmation bias for something else . 201 6.2.1.5 The role of emotions . 202 6.2.1.6 The role of trust in sources . 203 6.2.2 Confirmation bias and real-world problems . 205 6.2.2.1 Confirmation bias and politics . 208 6.2.2.2 Improving forecasting accuracy . 209 6.2.2.3 A new way of thinking about bias . 210 6.2.3 Implications for future research . 211 6.2.3.1 More attention and clarity around.