Clinical Decision Making + Strategies for Cognitive Debiasing
Pat Croskerry MD, PhD, FRCP(Edin)
International Association of Endodontists Scottsdale, Arizona June 2019
Financial Disclosures or other Conflicts of Interest
None It is estimated that an American adult makes 35,000 decisions a day i.e. about 2200 each waking hour
Sollisch J: The cure for decision fatigue. Wall Street Journal, 2016 Decision making
‘The most important decision we need to make in Life is how we are going to make decisions’
Professor Gigerenzer Is there a problem with the way we think and make decisions? 3 domains of decision making
Patients Healthcare leadership Healthcare providers
Patients
Leading Medical Causes of Death in the US and their Preventability in 2000
Cause Total Preventability (%)
Heart disease 710,760 46 Malignant neoplasms 553,091 66
Cerebrovascular 167,661 43 Chronic respiratory 122,009 76
Accidents 97,900 44 Diabetes mellitus 69,301 33
Acute respiratory 65,313 23
Suicide 29,350 100 Chronic Liver disease 26,552 75
Hypertension/renal 12,228 68 Assault (homicide) 16,765 100
All other 391,904 14
Keeney (2008)
Healthcare leadership Campbell et al, 2017 Healthcare providers US deaths in 2013
• 611,105 Heart disease • 584,881 Cancer • 251,454 Medical error
Medical error is the 3rd leading cause of death
Estimated number of preventable hospital deaths due to diagnostic failure annually in the US
40,000 – 80,000
Leape, Berwick and Bates JAMA 2002 Diagnostic failure is the biggest problem in patient safety
Newman-Toker, 2017 Sources of Diagnostic Failure
The System 25%
The Individual 75%
Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what’s the goal? Acad Med. 2002 What is the individual’s problem?
Isn’t bright enough
Doesn’t know enough
Isn’t trying hard enough
Isn’t thinking rationally
Is not having a good day
Structure of knowledge in endodontics following Bloom’s taxonomy (Adapted from Mack et al, Asia Pacific J Ophthal, 2018)
Level Type of Description Examples Endodontics-specific knowledge example
A Factual Basic elements that must Technical Anatomy of dentition knowledge be known to solve vocabulary problems B Concept Interrelationships among Theory of Pathophysiology of pulp knowledge these elements that evolution disease enable them to function together C Procedural Methods of enquiry and Scientific method Operating skills knowledge criteria for using subject- specific skills
D Metacognitive Knowledge of cognition Awareness of Knowledge of the use knowledge and awareness and one’s own of heuristics and their knowledge of one’s own metacognitive associated biases cognition ability Diagnostic Failure
15% It varies by specialty
Dermatology Radiology (1-2%) Anatomic pathology
Internal medicine Family medicine (~15%+) Emergency medicine
Endodontics ? Legal outcome by critical incident
CMPA Data : 347 legal actions closed 2005 - 2009
240 Number of 200 patients
160
120
80
40
0 Perform Comm Diagnosis Admin Medication Conduct Legal outcome by critical incident
CMPA Data : 347 legal actions closed 2005 - 2009
240
200
160
120
80
40
0 Perform Comm Diagnosis Admin Medication Conduct Legal outcome by critical incident
CMPA Data : 347 legal actions closed 2005 - 2009
240
200
160
120
80
40
0 Perform Comm Diagnosis Admin Medication Conduct So how good is our decision making? Eight Quick Questions
Take a piece of paper and write down your answers to each of these 8 questions
You have about 10 seconds for each response On a standard Ottawa fire truck, there are 2 drivers up front, one at the rear and three additional fire-fighters. What is the total personnel required for 5 standard trucks?
A. 25 B. 35 C. 30 A. the ball follows a parabola travelling forward as it falls B. the ball drops straight downwards from the point of release C. The ball moves backwards and downwards A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.
How much does the ball cost?
A. 5¢ B. 15¢ C. 10¢
If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?
A. 5 minutes B. 100 minutes C. 50 minutes
In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half the lake?
A.28 days B. 24 days C. 47 days The graph shows changes in variable A (black line) and variable B (red line) from 2000 - 2009
How would you describe the relationship between A and B?
(a) Closely related (b) Independent of each other (c) Cannot say
Year A B
A: The ball continues in a straight line when the string breaks B: The ball follows a curved path when the string breaks C. The ball follows some other trajectory when the string breaks A 50 year old woman, with no symptoms, takes part in routine mammographic screening, and tests positive. She is very worried and wants to know what her chances are of having breast cancer. You know nothing else about her. The test has a specificity of 90%, and a false positive rate of 9%
A. 9 in 10 B. 1 in 100 C. 1 in 10
Answers
1. (C) 30 firemen 2. (A) The ball follows a parabola forwards 3. The ball costs 5 cents 4. (A) Five minutes 5. (C) 47 days 6. Cannot say if A is related to B. 7. (A). The ball continues in a straight line 8. (D) 1 in 10
Cognitive Reflective Test (questions 3, 4, 5)
• The test distinguishes intuitive from analytical processing • It tests your ability to resist first response that comes to mind • Of 3428 people tested only 17% got all 3 correct • 33% answered all three incorrectly
Frederick 2002 (MIT)
So how much can we trust our intuitions?
Intuitions in Healthcare
. Fast . Compelling . Frequent . Minimal cognitive effort required . Addictive . Mostly serve us well . Occasionally catastrophic (not rational)
How does rationality fail? 3 basic problems with decision making
• hasty judgments • distorted probability estimates • biased judgments Mindware Rationality Failure
Processing problems Content problems
Cognitive Mindware Mindware miserliness gaps contamination
Minimising cognitive effort Knowledge deficits Cognitive biases Accepting things at face value Impaired numeracy Affective biases Intemperate heuristic use Impaired scientific thinking Endorsement of implausible beliefs Superficiality Impaired probability thinking Cultural conditioning Insufficient breadth and depth Illogical reasoning Indoctrination Close-mindedness Lack of awareness of bias WYSIATI Incomplete critical thinking tools
(Hasty judgments) (Distorted probability estimates) (Biased judgments) ‘Everything from investment fiascos to medical error has been linked to ‘contaminated mindware’. Facing up to it is the first step towards better decisions’
Stanovich, Toplak and West, 2010 What does rationality look like?
Deliberate Objective reasoning style Knowledgeable Evidence based Minimal bias Careful, considered Thoughtful, Not impulsive Reflective Mindful
Mostly, it’s not what we don’t know, it’s how we think
We need to know more about how we think…
2005 2006
How then do we think, reason and make decisions? Clinical decision making
Rational Intuitive
Slow Fast Deliberate Autonomous Objective Context dependent Scientific Qualitative Few errors Error prone
The Capacity to Reflect
• Less impulsivity in decision making • Metacognition • De-coupling from the intuitive mode • Increased likelihood of bias detection • Increased likelihood of de-biased decisions • Hallmark of a critical thinker
Dual Process Theory 2011 Dual Process Decision Making Dual Process Decision Making
System 1: System 2: Automatic/streamlined Cautious/complex Healthcare decision making
Racing bike Mountain bike
Slow Fast Deliberate Autonomous Objective Context dependent Scientific Qualitative Few errors Error prone
Type 1 and Type 2 processes (dual process theory) Hope he’s not just trusting his intuition on this one
X4 C C 2 3 X3
C X2 X1 1
X4
Intuition RECOGNIZED
Pattern Pattern Executive Dysrationalia Patient Recognition T Calibration Solution Problem Processor override override
Repetition
NOT Analytical RECOGNIZED
“Getting” medicine is not easy
Life is about learning the basic patterns
COW “Getting” medicine is not easy
“Getting” medicine is not easy
“Getting” medicine is not easy
Intuition RECOGNIZED
Pattern Pattern Executive Dysrationalia Patient Recognition T Calibration Solution Problem Processor override override
Repetition
NOT Analytical RECOGNIZED System RECOGNIZED 1
Expertise
Proficiency Initial percept Pattern Competence Calibration Decision or Processor Calibra problem
Advanced Beginner
Novice
System NOT RECOGNIZED 2 Axial view of fMRI activation of the brain as a function of practice over 60 minutes Hill and Schneider, 2006 B C
Type Type RECOGNIZED 1 1 Process Process
Adaptive Critical thinking Expertise Metacognitive processes Mindware Rationality, biases Routine Routine Gap Lateral thinking Expertise Expertise Humanities
Proficiency Proficiency
Clinical Pattern problem Processor Competenc Competenc features e e
Advanced Advanced Beginner Beginner
Novice Novice
Type Type NOT 2 2 RECOGNIZED Process Processes
Unconsciously Consciously Consciously Unconsciously Adaptive incompetent incompetent competent competent expertise
40 Hours 10,000 Hours
Toggle Function
(Hypothesis Hopping)
8 Main Features of the Model
• We spend most of our time in System 1 • Most heuristics and biases are in System 1 • Most errors occur in System 1 • Repetitive operations of System 2 >>> 1 • System 2 override of System 1 • System 1 override of System 2 • Toggle function • Cognitive Miser function (being comfortably numb)
The Cognitive Miser Function Ann
. Jack is looking at Ann . Ann is looking at George . Jack is married . George is not married Jack George
Is a married person looking at an unmarried person? 1. Yes 2. No 3. Can’t say
Cognitive Miser Function
• 80% of people get this wrong – usual response is ‘cannot say’ • Correct answer is ‘yes’ • Easiest inference is made and people do not consider other possibilities • Successful solutions are not correlated with IQ
When does the Cognitive Miser kick in?
• All the time • Especially when the decision maker is fatigued • Especially when sleep deprived • Especially when feeling down/depressed • Especially when resources are limited • During overcrowding/limited resources Do we have control over which system we are in?
‘Cognitive thought is the tip of an enormous iceberg. It is the rule of thumb among cognitive scientists that unconscious thought is 95% of all thought – this 95% below the surface of conscious awareness shapes and structures all conscious thought’ Lakoff and Johnson, 1999 So, we have to learn how best to deal with System 1 Factors that influence decision making that formal decision makers do not acknowledge
• Cognitive and affective biases, COIs • Information gaps • Decision fatigue • Ambient conditions • Availability of resources • Well-being of decision maker • Gender • Personality
CBM Cognitive Bias Mitigation
Intuition RECOGNIZED
Pattern Pattern Executive Dysrationalia Patient Recognition T Calibration Solution Problem Processor override override
Repetition
NOT Analytical RECOGNIZED Executive override
• Thinking about how we think • Metacognition • Reflection • Mindfulness
• System 2 monitoring of System 1 • System 2 modulation of System 1 • Cognitive decoupling from System 1 • Cognitive debiasing
Four major issues
• Getting people to recognize there is a problem • Accepting that change must occur • Choosing an appropriate debiasing strategy • Teaching and sustaining cognitive debiasing strategies
Many people are simply unaware of the problem…
And some people will never change…
We need to maintain a feral vigilance to detect biases It ain’t easy…
• Even though bias detected • Very unlikely one strategy works for all • Need for multiple approaches • Very unlikely one shot will work • Need for multiple innoculations • Need for extra vigilance in critical conditions • Need for lifelong maintenance
Issues that impede cognitive de-biasing Variable Descriptor Clinical relevance Cognition has not been seen as the business of healthcare. Cognitive processes are not usually studied by clinicians except in disease states such as brain injury, dementia, autism and others. Lack of awareness Many clinicians are naïve about cognitive processes and unaware that cognitive and affective biases may significantly impair clinical judgment. Usually, not covered in medical undergraduate or postgraduate training. Invulnerability to Some clinicians may be aware of cognitive and affective biases but do not believe that they are cognitive and affective vulnerable to them (cognitive egocentrism, optimism bias, blind spot bias) or that they might affect error their own practice. Myside bias If clinicians (and researchers) believe cognitive and affective bias are unimportant in clinical reasoning, they will have a prevailing tendency to evaluate data, evidence, and hypotheses in a manner supportive of their opinions. Status quo bias Cognitive de-biasing usually requires some Type 2 processing and significant cognitive effort, but it is considerably easier to continue with the status quo, rather than make the effort to learn a new approach and change current practice. Overconfidence Clinicians’ overconfidence in their own judgments may be the most powerful factors that contribute to diagnostic failure. Hubris and lack of intellectual humility characterize the uncritical thinker.
Vivid-pallid dimension Discussions of cognitive processes per se are dry, abstract and uninteresting to the clinical mind. They typically lack the vividness and concrete nature of clinical disease presentations that are more appealing to clinicians. What strategies do we have for debiasing? General CBM Strategies
• Promote rationality: know what undermines it and what improves it • Provide specific training: in cognitive factors involved in decision making (Dual Process Theory) • Develop insight/awareness: Provide descriptions of known cognitive biases and clinical examples • Consider alternatives: Forced consideration of alternative possibilities. “What else could this be?” • Metacognition: Train for a reflective approach to problem solving • Decrease reliance on memory: Accuracy of judgments improved through cognitive aids including smartphone apps and clinical guidelines • Specific training: Train to overcome flaws in thinking (eg, in probability, distinguishing correlation from causation). • Simulation: Simulated clinical scenarios contrasting biased and debiased (correct) approaches. • Make task easier: Unpack more information about the specific problem to reduce task difficulty • Minimize time pressures: Adequate time for quality decision-making • Minimise decision fatigue: Throughput pressure, diurnal effects, sleep deprivation, dysphoria • Accountability: Establish clear accountability and follow-up for decisions made and lessons learned • Feedback: Provide rapid and reliable feedback so that errors can be immediately appreciated and corrected • Cognitive forcing strategies: Strategies to avoid predictable bias in particular clinical situations
A universal forcing strategy Always ask the question:
What else might this be? Hey… what’s a mountain goat doing way up here in a cloud bank? Additional CBM Strategies
Encourage metacognition, reflection, and mindfulness Slow down Think the opposite Maintain a healthy skepticism about intuitions Promote group decision making when appropriate Educate intuition Promote less hubris, less overconfidence Know when you are vulnerable High Risk Situations o o Cognitive overloading o Fatigue o Sleep deprivation/sleep debt o Negative mood o Alcohol/drug influence Decision fatigue Proportion of rulings in favor of the prisoners by ordinal position
Parole decisions 1112 judicial rulings 10 month period 8 Judges
Danziger et al., PNAS 2011 Clinical examples o o Breast cancer screening o Colorectal cancer screening o Flu vaccination o Inappropriate ABx prescribing o Opioid prescribing for back pain o Colonoscopy vigilance o Hand washing The failure to change? “You will make the same foolish mistakes you have made before, not only once but many, many times again” We have covered
The importance of decision making in patient safety The basics of dual process theory The essential differences between the two systems Where they are in the brain The importance of bias in our intuitive decision making When bias is more likely to occur How de-biasing occurs The over-arching importance of rationality
The Critical Thinking Program at Dalhousie Medical School http://medicine.dal.ca/departments/ core-units/DME/critical-thinking.html Critical Thinking Website
Educational resources One-pagers Downloadable books Other (lists of biases, graphics, self-evaluate CT)
Videos
On-Line resources CReMe CT website Skeptical medicine – John Byrne SIDM
Dalhousie Medicine
Teaching and Assessing Critical Thinking (TACT) TACT Teaching and Assessing Critical Thinking
. On-line course at Dalhousie University Medical School . Approx 20 hours . 6 Faculty . Variety of materials (videos, lectures, refs, discussion boards) . 20 hours CME Credit . Option to go on to more advanced TACT 2 . Check Dal FacDev website for more information