The Impact of Cognitive Biases on Information Searching and Decision Making
Total Page:16
File Type:pdf, Size:1020Kb
The impact of cognitive biases on information searching and decision making Annie Ying Shan LAU Thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy Centre for Health Informatics University of New South Wales December 2006 I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstracts International. I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation. Annie Ying Shan Lau November 2007 I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project's design and conception or in style, presentation and linguistic expression is acknowledged. Annie Ying Shan Lau December 2006 Dedicated to my parents Abstract This research is possibly the first study investigating the impact of cognitive biases on information searching and decision making. Set in the context of making health-related decisions, this research tests the hypotheses that (i) people experience cognitive biases during information searching; (ii) cognitive biases can be corrected during information searching; and (iii) correcting for biases during information searching improves decision making. Using a retrospective data analysis, a Bayesian model and a series of prospective empirical experiments, the cognitive biases investigated are anchoring effect, order effects, exposure effect and reinforcement effect. People may experience anchoring effect, exposure effect and order effects while searching for information. A person’s prior belief (anchoring effect) has a significant impact on decision outcome (P < 0.001). Documents accessed at different positions in a search journey (order effects) and documents processed for different lengths of time (exposure effect) have different degrees of influence on decision making (order: P = 0.026; exposure: P = 0.0081). To remedy the impact of cognitive biases, a series of interventions were designed and trialled to test for their ability to modify the impact of biases during search. A search engine interface was modified to allow for a document-level intervention, which attempts to debias order effects, exposure effect and reinforcement effect; a decision-focussed intervention for debiasing the anchoring effect; and an education-based intervention to inform users about the biases investigated in this research. Evaluation of these alterations to the search interface showed that some of the interventions can reduce or exacerbate cognitive biases during search. Order effects are no longer apparent amongst subjects using a “keep document tool” (i.e. order debiasing intervention) (P = 0.34); however, it is not associated with any significant improvement in decision accuracy (P = 0.23). Although the anchoring effect remains robust amongst subjects using a “for/against document tool” (i.e. anchor debiasing intervention) (P < 0.001), the intervention is marginally associated with a 10.3% increased proportion of subjects who answered incorrectly pre-search to answer correctly post-search (P = 0.10). Overall, this research concludes with evidence that using a debiasing intervention can alter search behaviour and influence the accuracy and confidence in decision making. Acknowledgement This thesis would not have been produced without the invaluable opportunity given by Professor Enrico Coiera and Professor Nigel Lovell. I am indebted to Enrico for guiding me into the world of research with great wisdom, support and inspiration. His continuous pursuit for innovation and excellence and his respect for every individual are qualities that inspire me deeply in my training as a researcher. His insight to see the core of a problem and his ingenious approaches to research questions are attributes that I believe every researcher should aspire to develop. Special thanks go to current and past members of the decision support team at the Centre for Health Informatics for their technical assistance and their intellectual stimulation in the research (in alphabetical order): Dr Farah Magrabi, Mr Ken Nguyen, Dr Victor Vickland and Mr Martin Walter. Also, special thanks go to Professor Johanna Westbrook for providing the data that enabled this research to commence and for her assistance in research matters over the years. In addition, special thanks go to Dr David Thomas and Dr Isle Blignault for their assistance in the study recruitment and the case scenario design, and the hundreds of participants who took part in the study. Thanks go to the following people who have given me assistance during my candidature, from statistical matters, study design, usability and pilot studies to thesis draft revisions (in alphabetical order): Associate Professor Deborah Black, Miss Michelle Brear, Dr Grace Chung, Miss Nerida Creswick, Dr Sally Galbraith, Mr Andrew Georgiou, Dr Bob Jansen, Dr Geoff McDonnell, Dr Conrad Newton, Dr Marilyn Rob, Ms Sam Sheridan, Dr Vitali Sintchenko and Ms Margaret Williamson. I also want to thank Dr Yusuf Pisan for encouraging and facilitating my pursuit of a postgraduate research degree, and to my colleagues at the Centre for Health Informatics for their friendship and encouragement throughout the years. Acknowledgement goes to the Australian Research Council for its financial support in this research. Finally, I want to thank my parents, my siblings and Trevor for being my pillar of strength, and for walking this entire journey with me with unconditional love, patience and support. Contents PART I: INTRODUCTION............................................................................................................... 1 1 BACKGROUND INFORMATION ................................................................................................. 2 1.1 PROBLEM STATEMENT ....................................................................................................... 2 1.2 RESEARCH GAP .................................................................................................................. 2 1.3 AIM.................................................................................................................................... 5 2 GUIDE TO THESIS .................................................................................................................... 6 PART II: WHAT DO WE UNDERSTAND ABOUT INFORMATION SEARCHING BEHAVIOUR?.................................................................................................................................... 7 3 EVIDENCE JOURNEY: DATA EXPLORATION.............................................................................. 8 3.1 INTRODUCTION .................................................................................................................. 8 3.2 DATA DESCRIPTION............................................................................................................ 9 3.3 DATA EXPLORATION ........................................................................................................ 13 3.4 DISCUSSION ..................................................................................................................... 18 3.5 CONCLUSION ................................................................................................................... 19 PART III: DO COGNITIVE BIASES INFLUENCE INFORMATION SEARCHING AND DECISION MAKING?.................................................................................................................... 20 4 COGNITIVE BIASES: A LITERATURE REVIEW .......................................................................... 21 4.1 INTRODUCTION ................................................................................................................ 21 4.2 COGNITIVE BIASES ........................................................................................................... 22 4.3 STUDIES OF COGNITIVE BIAS IN DECISION MAKING .......................................................... 25 4.4 HEURISTICS AND COGNITIVE BIAS IN THE CONTEXT OF INFORMATION SEARCHING .......... 29 4.5 CONCLUSION ................................................................................................................... 31 5 DO COGNITIVE BIASES OCCUR IN SEARCH BEHAVIOURS? A PRELIMINARY ANALYSIS ....... 32 5.1 METHOD .........................................................................................................................