Effect of a Metacognitive Intervention on Cognitive
Total Page:16
File Type:pdf, Size:1020Kb
EFFECT OF A METACOGNITIVE INTERVENTION ON COGNITIVE HEURISTIC USE DURING DIAGNOSTIC REASONING by Velma Lucille Payne Bachelor of Science in Computer Science, Oral Roberts University, 1984 Master of Science in Computer Information Systems, Robert Morris University, 1996 Master of Business Administration, Robert Morris University, 1997 Master of Science in Biomedical Informatics, University of Pittsburgh, 2008 Submitted to the Graduate Faculty of School of Medicine in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2011 UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE This dissertation proposal was presented by Velma Lucille Payne To the following committee members: Claudia Mello-Thoms, MSEE, PhD, Assistant Professor of Biomedical Informatics and Radiology, University of Pittsburgh Mark S. Roberts, MD, MPP, Professor of Medicine, Health Policy and Management, and Industrial Engineering, University of Pittsburgh Cleotilde Gonzalez, PhD, Associate Research Professor, Department of Social and Decision Sciences, Carnegie Mellon University Pat Croskerry, MD, PhD, Professor, Department of Emergency Medicine, Dalhousie University Dissertation Advisor: Rebecca S. Crowley, MD, MS, Associate Professor of Biomedical Informatics, Pathology and Intelligent Systems, University of Pittsburgh ii Copyright © by Velma Lucille Payne 2011 iii EFFECT OF A METACOGNITIVE INTERVENTION ON COGNITIVE HEURISTIC USE DURING DIAGNOSTIC REASONING Velma Lucille Payne, M.S., MBA, M.S. University of Pittsburgh, 2011 Abstract Medical judgment and decision-making frequently occur under conditions of uncertainty. In order to reduce the complexity of diagnosis, physicians often rely on cognitive heuristics. Use of heuristics during clinical reasoning can be effective; however when used inappropriately the result can be flawed reasoning, medical errors and patient harm. Many researchers have attempted to debias individuals from inappropriate heuristic use by designing interventions based on normative theories of decision-making. There have been few attempts to debias individuals using interventions based on descriptive decision-making theories. Objectives: (1) Assess use of Anchoring and Adjustment and Confirmation Bias during diagnostic reasoning; (2) Investigate the impact of heuristic use on diagnostic accuracy; (3) Determine the impact of a metacognitive intervention based on the Mental Model Theory designed to reduce biased judgment by inducing physicians to ‘think about how they think’; and (4) Test a novel technique using eye-tracking to determine heuristic use and diagnostic accuracy within mode of thinking as defined by the Dual Process Theory. Methods: Medical students and residents assessed clinical scenarios using a computer system, specified a diagnosis, and designated the data used to arrive at the diagnosis. During case analysis, subjects either verbalized their thoughts or wore eye-tracking equipment to capture eye iv movements and pupil size as they diagnosed cases. Diagnostic data specified by the subject was used to measure heuristic use and assess the impact of heuristic use on diagnostic accuracy. Eye- tracking data was used to determine the frequency of heuristic use (Confirmation Bias only) and mode of thinking. Statistic models were executed to determine the effect of the metacognitive intervention. Results: Use of cognitive heuristics during diagnostic reasoning was common for this subject population. Logistic regression showed case difficulty to be an important factor contributing to diagnostic error. The metacognitive intervention had no effect on heuristic use and diagnostic accuracy. Eye-tracking data reveal this subject population infrequently assess cases in the Intuitive mode of thinking; spend more time in the Analytical mode of thinking, and switches between the two modes frequently as they reason through a case to arrive at a diagnosis. v TABLE OF CONTENTS DEDICATION........................................................................................................................ XVII ACKNOWLEDGEMENTS .................................................................................................XVIII 1 INTRODUCTION ................................................................................................................ 1 1.1 BACKGROUND.................................................................................................. 1 1.2 PROBLEM DESCRIPTION .............................................................................. 2 1.3 SIGNIFICANCE OF THIS RESEARCH ......................................................... 2 1.4 GUIDE FOR THE READER ............................................................................. 4 2 MEDICAL ERRORS ........................................................................................................... 5 2.1 MEDICAL ERROR DEFINITIONS ................................................................. 5 2.2 MEDICAL ERROR RESEARCH STUDIES ................................................... 6 2.3 DIAGNOSTIC ERRORS.................................................................................... 8 3 COGNITIVE HEURISTICS AND BIASES .................................................................... 14 3.1 STUDY OF COGNITIVE HEURISTICS AND BIASES .............................. 14 3.2 APPLICATION OF HEURISTICS TO DIAGNOSTIC REASONING ...... 16 4 THEORIES OF DECISION MAKING............................................................................ 21 4.1 NORMATIVE AND DESCRIPTIVE DECISION THEORY....................... 21 4.2 NORMATIVE THEORIES OF DECISION MAKING ................................ 22 4.2.1 Expected Utility Theory ......................................................................................22 vi 4.2.1.1 Application of Expected Utility Theory to Diagnostic Reasoning.. 22 4.2.2 Bayes’ Theorem....................................................................................................23 4.2.2.1 Application of Bayes’ Theorem to Diagnostic Reasoning............... 24 4.3 DESCRIPTIVE THEORIES OF DECISION MAKING .............................. 25 4.3.1 Mental Model Theory..........................................................................................25 4.3.1.1 Principles of the Mental Model Theory of Deductive Reasoning... 25 4.3.1.2 Predictions of the Mental Model Theory.......................................... 28 4.3.1.3 Application of Mental Model Theory to Diagnostic Reasoning ..... 30 4.3.2 Dual Process Theory............................................................................................32 4.3.2.1 Intuitive Mode of Thinking................................................................ 34 4.3.2.2 Analytical Mode of Thinking............................................................. 35 4.3.2.3 Conversion of Analytical Thinking to Intuitive Thinking .............. 36 4.3.2.4 Application of Dual Process Theory to Diagnostic Reasoning ....... 36 4.3.2.5 Relationship of Heuristics to Mode of Thinking in DPT................. 39 5 PRIOR RESEARCH ON COGNITIVE HEURISTIC DEBIASING AND COGNITIVE FEEDBACK............................................................................................... 41 5.1 PRIOR RESEARCH ON COGNITIVE HEURISTIC DEBIASING........... 41 5.2 PRIOR WORK ON COGNTIVE FEEDBACK ............................................. 46 5.2.1 Impact of Different Types of Feedback .............................................................46 5.2.2 Temporal Aspects of Feedback...........................................................................47 5.2.3 Feedback and Diagnostic Errors ........................................................................48 6 RESEARCH STATEMENT.............................................................................................. 50 6.1 RESEARCH OBJECTIVES............................................................................. 50 vii 6.2 RESEARCH QUESTIONS............................................................................... 50 6.3 HEURISTIC AND BIAS UNDER STUDY ..................................................... 51 6.4 RESEARCH METHODS.................................................................................. 52 6.4.1 Subjects .................................................................................................................52 6.4.1.1 Type and Number of Subjects ........................................................... 52 6.4.1.2 Subject Approval, Recruitment and Consent .................................. 53 6.4.2 Instrument ............................................................................................................53 6.4.3 Expert Case Models .............................................................................................55 6.4.4 Study Design.........................................................................................................57 6.4.5 Case Review Process............................................................................................59 6.4.6 Determining Subject’s Mental Model ................................................................62 6.4.7 Intervention ..........................................................................................................63 6.4.8 Application of Mental Model Theory to Feedback...........................................64 6.4.9 Data Capture ........................................................................................................68 6.4.9.1 Case Analysis Computer-Based