Critical Thinking for Helping Professionals Ii Iii

Critical Thinking for Helping Professionals ii iii

Critical Thinking for Helping Professionals

A Skills-Based Workbook Fourth edition

EILEEN GAMBRILL LEONARD GIBBS

1 iv

Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and certain other countries. Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America. © Eileen Gambrill and Leonard Gibbs 2017 First Edition published in 1996 Second Edition published in 1999 Third Edition published in 2009 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by license, or under terms agreed with the appropriate reproduction rights organization. Inquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above. You must not circulate this work in any other form and you must impose this same condition on any acquirer. Library of Congress Cataloging-in-Publication Data Names: Gambrill, Eileen D., 1934- author. | Gibbs, Leonard E., author. Title: Critical thinking for helping professionals : a skills-based workbook / Eileen Gambrill, Leonard Gibbs. Description: Fourth edition. | New York, NY : Oxford University Press, [2017] | Includes bibliographical references and index. | Description based on print version record and CIP data provided by publisher; resource not viewed. Identifiers: LCCN 2016052257 (print) | LCCN 2016038878 (ebook) | ISBN 9780190297312 (updf) | ISBN 9780190297305 (alk. paper) Subjects: LCSH: Critical thinking—Problems, exercises, etc. | Critical thinking—Study and teaching. | Professional employees—Decision making—Problems, exercises, etc. | Decision making—Study and teaching. | Human services personnel—Decision making—Problems, exercises, etc. Classification: LCC BF441 (print) | LCC BF441 .G36 2017 (ebook) | DDC 153.4/2—dc23 LC record available at https://lccn.loc.gov/2016052257 9 8 7 6 5 4 3 2 1 Printed by WebCom, Inc., Canada vâ

Contents

Prefaceâix Acknowledgmentsâxiii Note from Eileen Gambrillâ xv Detailed Table of Contentsâ xvii

PART 1 CRITICAL THINKING AS A GUIDE TO DECISION MAKING

Exercise 1 Making Decisions About Interventionâ 67 Exercise 2 Reviewing Your Beliefs About Knowledgeâ 73 Exercise 3 Controversy: Invaluable for Problem Solving and Learningâ 79 Exercise 4 Critical Thinking and Advocacyâ 89

PART 2 RECOGNIZING PROPAGANDA: THE IMPORTANCE OF QUESTIONING CLAIMS

Exercise 5 Critically Appraising Human Services Advertisementsâ 103 Exercise 6 Does Scaring Youth Help Them “Go Straight?”â 109 Exercise 7 Detecting Misleading Problem Framingâ 113 Exercise 8 Following the Moneyâ 117 Exercise 9 The Language of Propagandaâ 123

PART 3 INCREASING YOUR SKILL IN AVOIDING FALLACIES, BIASES, AND PITFALLS IN DECISION MAKING

Exercise 10 Using the Professional Thinking Formâ 137 Exercise 11 Reasoning-âin-âPractice Game A: Common Practice Fallacies and Biasesâ 149 Exercise 12 Reasoning-âin-âPractice Game B: Group and Interpersonal Dynamicsâ 167

v vi

Exercise 13 Reasoning-âin-âPractice Game C: More Biasesâ 179 Exercise 14 Preparing a Fallacy/âBias Festivalâ 191 Exercise 15 Fallacy Spotting in Professional Contextsâ 193 Exercise 16 Avoiding Groupthinkâ 197

PART 4 EVIDENCE-âINFORMED DECISION MAKING

Exercise 17 Applying the Steps in Evidence-âBased Practiceâ 205 Exercise 18 Working in Interdisciplinary Evidence-âInformed Teamsâ 219 Exercise 19 Preparing Critically Appraised Topicsâ 223 Exercise 20 Involving Clients as Informed Participantsâ 229 Exercise 21 Asking Hard Questions: Enhancing Assertive Skillsâ 237 Exercise 22 Evaluating Service Outcomesâ 241 Exercise 23 Reviewing Your Expertiseâ 249

PART 5 CRITICALLY APPRAISING RESEARCH

Exercises 24 Evaluating Effectiveness Studies: How Good Is the Evidence?â 269 Exercise 25 Critically Appraising Research Reviews and Practice Guidelinesâ 279 Exercise 26 Critically Appraising Self-âReport Measuresâ 287 Exercise 27 Estimating Risk and Making Predictionsâ 293 Exercise 28 Critically Appraising Diagnostic Testsâ 303 Exercise 29 Evaluating Research Regarding Causesâ 309

PART 6 REVIEWING DECISIONS

Exercise 30 Critically Appraising Argumentsâ 317 Exercise 31 Critical Thinking as a Guide to Making Ethical Decisionsâ 327 Exercise 32 Reviewing Intervention Plansâ 333

viâContents viiâ

PART 7 IMPROVING EDUCATIONAL AND PRACTICE ENVIRONMENTS

Exercise 33 Encouraging a Culture of Thoughtfulnessâ 341 Exercise 34 Evaluating the Teaching of Critical Thinking Skillsâ 347 Exercise 35 Forming a Journal Clubâ 351 Exercise 36 Encouraging Continued Self-âDevelopment Regarding the Process of Evidence-âInformed Practice and Policyâ 355 Exercise 37 Increasing Self-âAwareness of Personal Obstacles to Critical Thinkingâ 361

Glossaryâ371 Referencesâ377 Indexâ413

Contentsâvii viii ixâ

Preface

This workbook has a single purpose: those who do its exercises will reason more effectively about life-âaffecting practice and policy decisions. Critical thinking involves the critical appraisal of beliefs, arguments, claims, and actions to arrive at well-âreasoned judgments. Will sending a youthful offender to boot camp be more effective in decreasing future offenses than placing him on probation? Will a prescribed drug forestall the progression of confusion among Alzheimer’s patients in a nursing home? Will children with developmental disorders learn better if main- streamed into regular classrooms? Professionals make many such judgments and decisions daily. Deciding which actions will help clients is an inescapable part of being a professional. Thinking critically is important in all areas of the helping professions, including practice, research, policy, administration, and advocacy. The need for critical appraisal is highlighted by the increasing revelations of bogus claims in many sources, including the peer-âreviewed literature and related fraud and corruption (see Part 1). Critical thinking skills will help you spot policies and procedures that benefit agencies but not their clients, and those that maintain discriminatory patterns of service. Related skills, values, and attitudes, such as being open-âminded and flexible as well as self-âcritical, encourage recognition of cultural differences. This workbook is designed to learn by doing. Revisions in this fourth edition include greater attention to propaganda in the helping professions that may mislead both helpers and clients, and the greater accessibility of tools and material of value to help us avoid misleading claims that may harm clients if acted on. This workbook involves you in making decisions and allows for immediate feedback about decisions made. Think as much as you like, you cannot assess the effects of your thinking until you act and determine the outcome. For instance, did your thinking result in decisions that benefit clients? We have tried to create exercises that are enjoyable as well as instructive. Some involve cooperative learning in which you work with peers in learning adventures designed to hone your critical-âthinking skills. The exercises are designed to be useful in all helping professions curricula. Some have been pretested; others have

ix x

not. Each exercise includes the following sections: Purpose, Background, Instructions, and Follow-up Questions. The exercises illustrate the overlap between values, knowledge, and skills involved in research and practice. Research courses are typically taught separately from practice and policy courses, encouraging the false impression that research and practice are quite different enterprises. This arrangement hinders understanding of shared values, attitudes, content knowledge, and performance skills. For example, critical thinking is integral to all. Research and practice are complementary, not competing, areas. Failure to draw on practice and policy-related research is a concern in all helping professions. Related gaps were a key reason for the creation of the process and philosophy of evidence-based practice described in Parts 1 and 4. Too often, professionals do not take advantage of research related to decisions that have life-affecting consequences for clients. Because of this, clients may receive ineffective or harmful interventions. Part l, “Critical Thinking as a Guide to Decision Making,” defines critical thinking, discusses why it matters in the helping professions, and describes related values, attitudes, knowledge, and skills. This part contains four exercises. The first provides an opportunity to review criteria you use to make decisions. Exercise 2 offers an opportunity to assess your beliefs about knowledge (what it is and how to get it). Exercise 3 highlights the vital role of clashing views in problem solving, and Exercise 4 emphasizes the connection between critical thinking and advocating for clients. The five exercises in Part 2, “Recognizing Propaganda: The importance of questioning claims,” demonstrate the importance of skepticism. Human service advertisements, including the promotion of treatment programs, take advantage of propaganda methods such as vivid emotional appeals to convince us that a method works. Exercises 5 and 6 engage you in critically appraising human services advertisements and program promotion. Exercise 7 provides an opportunity to critically examine how problems are framed. Exercises 8 and 9 offer opportunities to “follow the money” (recognize the influence of profit making in the helping profession) and to increase your awareness of how language may lead you astray (e.g., weasel words). The seven exercises in Part 3, “Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making,” are designed to help you to identify and avoid common biases and fallacies in making life- affecting decisions. Vignettes are provided to illustrate situations that arise

x Preface xi

in everyday practice. Exercise 10 contains twenty-five vignettes that can be used to assess practice reasoning. The Reasoning-in-Practice Games (Exercises 11–13) involve working with other students to identify biases and fallacies. In the Fallacies Film Festival (Exercise 14), students work together to prepare a skit to demonstrate a fallacy. Exercise 15 provides an opportunity to spot fallacies in professional contexts (including your classroom). Exercise 16 describes group think ploys and provides an opportunity to learn how to spot and avoid them. Part 4, Evidence-Informed Decision Making, contains seven exercises designed to help you acquire knowledge and skills concerning the process of evidence-informed practice, including working in teams. Exercise 17, Applying the Steps in Evidence-Based Practice, guides you in this process. Exercise 18, Working in Interdisciplinary Evidence-Informed Teams, offers an opportunity to apply the steps in a team. Exercise 19, Preparing Critically Appraised Topics, guides you in preparing user- friendly summaries of research regarding important questions that arise in practice. Exercise 20 describes how you can involve clients as informed participants. Exercise 21 offers tips and practice opportunities for raising “hard questions” about claims that must be asked if our decisions are to be informed (about ignorance as well as knowledge). Exercise 22 engages you in reviewing gaps between an agency’s services and what research suggests is most effective, as well as in reviewing how you evaluate outcomes with your client. Exercise 23 guides you in reviewing your expertise. Part 5, “Critically Appraising Research,” contains six exercises. Exercise 24 provides guidelines for reviewing the quality of effectiveness studies. Exercise 25 guides you in reviewing the quality of reviews. Exercise 26, Critically Appraising Self-Report Measures, describes concerns regarding reliability and validity, and offers an opportunity to appraise a measure. Exercise 27 provides guidelines for estimating risk, making predictions, and accurately communicating risk to clients. Exercise 28 provides guidelines for reviewing diagnostic measures. Last, Exercise 29 suggests important concerns when critically appraising claims about causation. Part 6, “Reviewing Decisions,” contains three exercises that apply critical thinking skills to key components of the helping process. Exercise 30 provides guidelines for reviewing the quality of arguments. Exercise 31 provides an opportunity to think critically about practice and

Preface xi xii

policy-related ethical issues. Exercise 32 engages you in reviewing the quality of intervention. Part 7, “Improving Educational and Practice Environments,” includes five exercises. Exercise 33 provides a checklist for reviewing the extent to which an educational or work environment demonstrates a culture of thoughtfulness. Exercise 34 includes a rating form for evaluating the extent to which instructors encourage critical thinking in their classroom. Exercise 35 describes how to set up a journal club, and Exercises 36 and 37 offer guidelines for life-long learning. If working through the exercises contained in this workbook results in better services for clients, all our efforts—both yours and ours—will be worthwhile. We welcome your feedback about each exercise. In the spirit of critical thinking, we welcome negative as well as positive comments, especially those that offer concrete suggestions for improving exercises. We hope you enjoy and learn from participating in the exercises in this book. With adoption of this book, instructors have access to a website including the Instructor’s Manual. The manual contains descriptions of suggestions for using each exercise, scoring instructions as relevant, and possible answers to follow-up questions. Eileen Gambrill Leonard Gibbs

xii Preface xiiiâ

Acknowledgments

We owe a great deal to kindred spirits both past and present who cared enough and had the courage to raise questions about the quality of services offered to clients, and who have worked to create tools and processes to help practitioners and clients to make informed decisions—âinformed about related ignorance as well as knowledge. All value (or did value) critical appraisal of claims to protect clients from ineffective or harmful services. We thank Macmillan Publishers (for permission to use the Professional Thinking Form). Eileen Gambrill extends a special note of thanks to the Hutto-â Patterson Chair funders, to the University of California at Berkeley for Internet and library resources, and to Sharon Ikami for her patience, goodwill, and word processing skills. Leonard Gibbs acknowledges the influence of a great teacher, Professor Emeritus Michael Hakeem of the University of Wisconsin at Madison, and the encouragement and financial support of the University of Wisconsin at Eau Claire Foundation and the College of Professional Studies, whose support contributed to this work. Thanks to Dana Bliss, Senior Editor, Social Work, Oxford University Press, for his support and to Andrew Dominello, also of Oxford University Press.

xiii xiv xvâ

Note from Eileen Gambrill

My dear friend and co-âauthor, Emeritus Professor Leonard Gibbs, died June 13, 2008, following a valiant battle with metastatic prostrate cancer. He is deeply missed.

xv xvi xviiâ

Detailed Table of Contents

PART 1 CRITICAL THINKING AS A GUIDE TO DECISION MAKING

The introduction defines critical thinking, describes how it relates to scientific thinking and evidence-âinformed practice, and reviews related knowledge, skills, values, and attitudes. The purpose of both critical thinking and evidence-âinformed decision making is to make well-âreasoned decisions.â Exercise 1 Making Decisions About Interventionâ 67 Professionals differ in the criteria they use to select assessment, intervention, and evaluation methods. This exercise offers an opportunity to compare the criteria used to make decisions about intervention methods in different contexts.â Exercise 2 Reviewing Your Beliefs About Knowledgeâ 73 This exercise offers readers an opportunity to review their beliefs about knowledge (what it is and how to get it). Presented are common misconceptions and misunderstandings that may interfere with offering clients the benefits of available knowledge.â Exercise 3 Controversy: Invaluable for Problem Solving and Learningâ 79 Critical discussion of different views is vital to making evidence-âinformed decisions. This exercise provides an opportunity to address controversial issues, drawing on guidelines that contribute to a productive dialogue.â Exercise 4 Critical Thinking and Advocacyâ 89 Ethical obligations to clients require identifying, describing, exposing, and advocating to alter sources of avoidable misery for clients. Students work together in groups to identify a related goal and to design an advocacy plan. Additional activities are descried for further work in this area.â

xvii xviii

PART 2 RECOGNIZING PROPAGANDA: THE IMPORTANCE OF QUESTIONING CLAIMS

Exercise 5 Critically Appraising Human Services Advertisements 103 Professionals and laypeople alike hear many claims about how to help people. In this exercise, students watch a human service advertisement and complete a questionnaire. This exercise identifies hallmarks of such advertisements and raises questions about relying on them as a guide to making decisions. Exercise 6 Does Scaring Youth Help Them “Go Straight?” 109 This exercise assesses viewers’ skills in reasoning critically about a presentation that advocates a method for preventing criminal behavior among delinquents. It relies on the “Scared Straight” videotape. Exercise 7 Detecting Misleading Problem Framing 113 How problems are framed affects services clients receive. Are they framed as medical, psychological, and/or social problems? Misleading problem framing abounds— promoted by pharmaceutical companies, for example, describing anxiety as a medical problem. Ten activities are suggested to help students enhance their skills in detecting misleading framing Exercise 8 Following the Money 117 The helping professions and related industries are big businesses. Whenever money is concerned, conflicts of interest and related fraud and corruption may occur that drains money away from helping clients. Twelve activities are offered from which students and instructors can draw to increase their awareness of the play of special interests that may compromise quality of services offered to clients. Exercise 9 The Language of Propaganda 123 The purpose of this exercise is to increase students’ skill in recognizing language ploys, such as inappropriate vagueness and loaded questions that may get in the way of making sound decisions. Five activities are described, including a Bingo game designed to enhance skills in detecting misuse of language.

xviii Detailed Table of Contents xix

PART 3 INCREASING YOUR SKILL IN AVOIDING FALLACIES, BIASES, AND PITFALLS IN DECISION MAKING

Exercise 10 Using the Professional Thinking Form 137 This exercise includes twenty-five vignettes that may or may not contain a fallacy (error in reasoning). This short-answer exercise can be used to generate classroom discussion or as a measure. Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 149 Students work together in teams to read aloud or act out vignettes that may or may not contain a fallacy. Remedies for handling each fallacy are described. Game A concerns common informal fallacies in reasoning about practice decisions. Exercise 12 Reasoning-in- Practice Game B: Group and Interpersonal Dynamics 167 Vignettes in this game depict common sources of error that occur in case conferences, group meetings, and interdisciplinary teams. Exercise 13 Reasoning-in-Practice Game C: More Biases 179 Vignettes in this game illustrate common reasoning errors described in the literature on clinical reasoning, problem solving, decision making, and judgment. Exercise 14 Preparing a Fallacy/Bias Festival 191 In this exercise, participants work in groups to write a two- page paper that defines a chosen bias or fallacy, describes how to avoid it, and includes an original thirty- to sixty-second script for a vignette. Participants then act out their vignette while being videotaped. These vignettes are edited and then shown to others (e.g., an entire class), who try to identify the bias or fallacy. Exercise 15 Fallacy Spotting in Professional Contexts 193 This exercise provides practice in spotting fallacies in professional contexts. From the internet, journals, class lectures, books, students select a quote they believe demonstrates a fallacy. They record the full quote and note its source and name, define the fallacy, and explain why they think the reasoning is faulty.

Detailed Table of Contents xix xx

Exercise 16 Avoiding Groupthink 197 Many practice decisions take place in groups and team meetings. This exercise introduces participants to “groupthink” tactics that decrease the quality of decisions, identifies related indicators, and provides practice in identifying and avoiding groupthink ploys such as abusive ad hominem arguments.

PART 4 EVIDENCE-INFORMED DECISION MAKING

Exercise 17 Applying the Steps in Evidence-Based Practice 205 This exercise describes the process of EBP and offers an opportunity to practice implementing it. Exercise 18 Working in Interdisciplinary Evidence-Informed Teams 219 This exercise highlights the importance of interdisciplinary decision making and guides students in applying the process of EBP as a team. Exercise 19 Preparing Critically Appraised Topics 223 Components and purpose of CATs are described, and students are guided in preparing CATs. Exercise 20 Involving Clients as Informed Participants 229 Professional codes of ethics call for informed consent on the part of clients and for professionals to draw on practice and policy-related research. An Evidence-Informed Client Choice Form is included for involving clients as informed participants. (See also Exercise 27 regarding accurate communication of risks to clients.) Exercise 21 Asking Hard Questions: Enhancing Assertive Skills 237 Offering clients effective services requires asking questions regarding the evidentiary status of practices and policies such as: How good is the evidence? Suggestions for raising such questions are given in this exercise, as are practice opportunities.

xx Detailed Table of Contents xxi

Exercise 22 Evaluating Service Outcomes 241 Agency services differ in the extent to which they are likely to help clients attain hoped-for outcomes. In this exercise, students compare services in their agency with what research suggests is most likely to help clients attain hoped-for outcomes. Exercise 23 Reviewing Your Expertise 249 Components of expertise are described as well as challenges in developing expertise, including avoiding common errors in different problem-solving practice. In Exercise 23.1 students select one component of expertise they would like to enhance, design a plan, and try it out. Exercise 23.2 engages students in describing an error they tend to make as well as contributing factors and planning how to decrease it.

PART 5 CRITICALLY APPRAISING RESEARCH

Exercise 24 Evaluating Effectiveness Studies: How Good Is the Evidence? 269 This exercise provides an opportunity to evaluate an effectiveness study related to a practice or policy of interest, drawing on critical appraisal tools. It also provides an opportunity to learn about different ways to estimate effect size. Exercise 25 Critically Appraising Research Reviews and Practice Guidelines 279 Characteristics of rigorous systematic reviews and meta- analyses are described and contrasted with incomplete, unrigorous reviews. The importance of critical appraisal of practice guidelines is emphasized. Exercise 26 Critically Appraising Self-Report Measures 287 This exercise provides an opportunity to review concepts central to self-report measures, such as reliability and validity, and to apply them to measures. Exercise 27 Estimating Risk and Making Predictions 293 Helping clients involves estimating risk and making predictions about what people may do in the future. Students learn how to accurately represent risk by using

Detailed Table of Contents xxi xxii

frequencies instead of probabilities. The importance of providing information about absolute as well as relative risk is emphasized. Exercise 28 Critically Appraising Diagnostic Tests 303 Professionals make decisions about which assessment measures to use. In this exercise, readers review criteria that should be relied on when evaluating diagnostic tests. Exercise 29 Evaluating Research Regarding Causes 309 This exercise involves students in critically appraising research reports and claims regarding presumed causes of problems. Background material identifies related concepts, including necessary and sufficient causes, and describes different kinds of evidence used in support of claims. Students then apply this background information to related material in the professional literature and/or popular sources.

PART 6 REVIEWING DECISIONS

Exercise 30 Critically Appraising Arguments 317 Helping clients requires reviewing arguments for and against certain beliefs and actions. Accepting or rejecting these arguments can profoundly affect client welfare. This exercise describes key features of an argument (i.e., conclusion, premises, warrants) as well as the characteristics of sound arguments. Exercise 31 Critical Thinking as a Guide to Making Ethical Decisions 327 Some writers argue that the most important purpose of critical thinking is to help professionals arrive at ethical decisions. In this exercise, students consider practice situations from an ethical point of view using vignettes from the Reasoning-in- Practice games. Exercise 32 Reviewing Intervention Plans 333 Policies and plans may succeed or fail depending on how soundly they have been conceived. This exercise includes a form for rating the soundness of plans and provides an opportunity to apply it to a case example.

xxii Detailed Table of Contents xxiii

PART 7 IMPROVING EDUCATIONAL AND PRACTICE ENVIRONMENTS

Exercise 33 Encouraging a Culture of Thoughtfulness 341 In this exercise, students plan how to maintain critical thinking values and skills in educational and work settings guided by a list of possibilities from which they can choose. Exercise 34 Evaluating the Teaching of Critical Thinking Skills 347 Students (and/or instructors) can use the form included in this exercise to rate the extent to which an instructor encourages critical thinking (e.g., encourages questions, describes well- argued alternative views on controversial issues). Exercise 35 Forming a Journal Club 351 The purpose and facilitating characteristics of journal clubs are described as well as how to create a journal club. Exercise 36 Encouraging Continued Self-Development Regarding the Process of Evidence-Informed Practice and Policy 355 The importance of continued self-development of evidence- informed practice skills is discussed, examples of specific skills are given, and students are guided in increasing a skill. Exercise 37 Increasing Self-Awareness of Personal Obstacles to Critical Thinking 361 Students are encouraged to examine potential obstacles to critical thinking including the kind of excuses used for poor quality services and to work toward decreasing specific obstacles.

Glossary 371 References 377 Index 413

Detailed Table of Contents xxiii xxiv xv

Critical Thinking for Helping Professionals xxvi 1â

PART 1 Critical Thinking as a Guide to Decision Making

Reasoning, problem solving, and decision making are closely related, and the tasks they involve overlap. We make decisions to address concerns and problems. Professionals and clients make decisions about which problems and risks to focus on, how to frame them (e.g., which kind they are—âis anxiety a “mental illness?”), which information to collect, which interventions to consider, how to evaluate progress, and which criteria to use to evaluate the accuracy of related claims (see Box 1.1). Their views are shaped by societal values and related contingencies, for example, about requisites of a “just society” and which problems should be focused on. Decisions are made about what to do—ânothing, watchful waiting, active intervention. Consider the following: • An advertisement for a residential treatment center for children claims, “We’ve been serving residents for more than fifty years with success.” Would you refer a client to this center? What kind of evidence could you seek to evaluate this claim? • A social worker says: “This child is at risk of abuse. She should be taken into care immediately.” What questions would you ask? Why? • You read “Cognitive–âBehavior Therapy: Proven Effectiveness” (Leahy, 2011). Is it true? Effective for what? • Your physician recommends arthroscopic surgery for your degenera- tive knee. Should you take her advice?

1 2

Box 1.1 Questions Regarding Different Kinds of Claims

1. About “problems”

• What problems are selected for attention: How important is each? Who says so and on what basis? • Exactly how is it defined? What are specific examples? • What kind of problem is it claimed to be? What are underlying assumptions? • What controversies exist regarding this problem? • Is there a remedy? • Should action be taken? What should be done? • What evidence is there regarding the previous questions? Are claims true?

2. About assessment, diagnosis, risk, and prediction

• Is a measure reliable? Were the most important kinds of reliability checked? • Is a measure valid? Does it measure what it is designed to measure? What kinds of validity were investigated? • What is the false-positive rate? • What is the false-negative rate? • What is the absolute risk reduction (see Exercise 27)? • Are key-valued “end states” accurately predicted (rather than surrogates)? • What percentage of predictions are accurate? • How good is the evidence for all of the above? Are claims true?

3. About causes

• Is correlation confused with causation? • How strong are associations? • Could associations found be coincidental? • Could a third factor be responsible? • Are root causes distinguished from secondary causes? • Are boundaries or necessary conditions clearly described (circumstances where relationships do not hold) (Haynes, 1992)? • Are well-argued alternative views accurately presented? • Are the interventions based on presumed causes effective? • Are vague multifactorial claims made that do not permit critical tests? • How good is the evidence for all the entries in no. 3? Are claims true?

4. About effectiveness/prevention

• Are claims true? Were critical tests carried out? What were the results? • What is the number needed to treat (NNT)? • How rigorous were the tests? • Were outcomes of key value to clients focused on? • Are reviews of related research of high quality (e.g., rigorous, comprehensive in search, and transparent in description of methods and findings)? • Was the possibility of harmful effects investigated? What is the number needed to harm? • How long do effects persist? What was the duration of follow-up? 3

• You read on the website of the National Alliance on Mental Illness: “One in four adults—approximately 61.5 million Americans—experience mental illness in a given year.” Is this claim true? What information would you seek? • You read an article suggesting that collective bargaining fights gentri- fication. What questions would you raise? Why? Questionable criteria for evaluating claims are shown in Box 1.2. There are great stakes in how problems are framed, and people with vested interests devote time, money, and effort to influence framing (Loeske, 1999). Is it true that “the treatment of diabetes can be a useful metaphor for understanding the treatment of generalized anxiety disorder (GAD)” (Marker & Aylward, 2012, p. 33)? Is obesity a disease as now claimed? Does psychotropic medication do more harm than good (Gøtsche, 2015a, 2015b)? How a problem is framed (e.g., as an individual and/or social problem) influences the selection of intervention methods.

Box 1.2 Questionable Criteria for Evaluating Knowledge Claims

Criteria Example Authority (what the “If Freud said it, it must be true.” “experts” say) Popularity (argument “Many social workers use genograms. I’m going to use this too.” ad populum) Anecdotal experience “I’ve used facilitated communication successfully with five clients. This works!” Tradition “That’s the way we have always done it. We should continue to use these methods.” What’s new “It’s the latest thing. We should try it too.” Uncritical Accepting a claim based on vague, undocumented evidence documentation Case examples “I used narrative therapy with my client and she improved dramatically.” Testimonials “I believe it works because Mrs. Rivera said she tried it and it helped.” Characteristics of the “She presents a good argument, but look at the school she gradu- person ated from” (ad hominem). Manner of presentation “She gave a convincing talk. I’m going to use her methods.” Good intentions In response to a question about an agency’s effectiveness you say, “We really care about our clients.” Intuition “I just knew that support groups would be best.” Entertainment value “This is a fascinating account of depression. I think it is correct.” Emotional reactions “I trust my feelings when making decisions.”

Source: Gambrill, E. (2013a). Social work practice: A critical thinker’s guide (3rd Ed.). New York: Oxford University Press, p. 75.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 3 4

Ethical and Moral Issues

Decisions made involve moral and ethical issues in a number of ways. One pertains to which problems/behaviors are selected for attention and how they are defined—for example, as legal, ethical, medical, or moral (Conrad, 2007; Szasz, 1961, 2007). Views of problems have life-affecting consequences for clients. If we act on inaccurate accounts, we may focus on irrelevant factors, recommend ineffective or harmful intervention methods, or continue intervention too long or withdraw it too soon. History shows that good intentions do not protect us from harming clients (e.g., McCord, 2003; Rose, Bisson, & Wessley, 2004; Scull, 2005, 2015; Silverman, 1980). Examples of iatrogenic effects (helper-induced harm) include removing all teeth in women with depression (Scull, 2005). Gøtzsche (2015a) argues that prescribed psychotropic medication taken by people 65 and older kills more than 500,000 people per year and disables tens of thousands more. Medical errors in American hospitals are now the third leading cause of death in the Unites States (James, 2013). Medication errors are common (Aspden, Wolcott, Bootman, & Cronenwett, 2007). When ineffective methods fail, clients may feel more hopeless about achieving hoped-for outcomes. Szasz (1961, 2007) has long argued that ethical and moral issues are obscured by claiming that distress, such as anxiety, and (mis)behaviors, such as aggression, are medical (mental health) issues. Viewing overeating, gambling, and violence toward others as brain diseases removes responsibility from those involved. Szasz (1965) suggests that such beliefs “act as social tranquilizers that obscure the everyday fact that life for most people is a continuous struggle … for a ‘place in the sun,’ ‘peace of mind,’ or some other moral value” (p. 24). Attention to environmental circumstances, such as lack of employment paying a living wage, that create distress encourages empathic understanding of clients; “there, too, may go I.” It is in this sense that Gøtzsche (2008) considers humanistic thinking as two of the four components that form the basis of clinical decisions: ethical norms (e.g., to help and to avoid harm) and “understanding the client as a fellow human being” (p. 150).

Uncertainties, Ambiguities, and Competing Contingencies

Judgments and decisions are made in the face of uncertainty. Some can be removed; much cannot. Uncertainty may concern (1) the nature of

4 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 5

the problem, (2) the outcomes desired, (3) what is needed to attain them, (4) the likelihood of attaining outcomes, and (5) measures that best reflect the degree of success. Decisions are influenced by ignorance as well as knowledge. Ignorance may be personal (e.g., a physician may not be aware of the dangers of prescribing psychotropic medication to older people) or objective (e.g., no one knows the answer to many questions). Was important information missing? Was this a matter of “strategic ignorance”—deliberately created by someone or some organization (McGoey, 2012)? Decisions are characterized by ill-defined goals, ambiguity, missing data, and shifting and competing goals and values. They are influenced by agency policies and practices (Abramovitz & Zelnick, 2015). They often involve high stakes and multiple players, and are made under time pressures. Social control functions in child welfare, mental health systems, and the criminal justice system may compete with the goals of clients. These different functions highlight ethical, moral, and value issues and their potential clash. Problems that confront clients, such as lack of housing or healthcare, may be “wicked” problems with no clear formulation (Rittel & Webber, 1973). Rarely is all relevant information available, and it is a challenge to integrate different kinds of data. Even when empirical information is available about the probability that different remedies result in desired outcomes, this knowledge is usually in the form of general principles that do not allow specific predictions about individuals. The criteria on which decisions should be based are in dispute, and empirical data about the effectiveness of different options are often lacking. People have different beliefs about the kinds of evidence that should be used to make decisions and how much should be shared with clients. Judgments may require distinguishing between causes and secondary effects, problems and the results of attempted solutions, personal and environmental causes, and links between clinical assumptions and related research. A variety of biases and fallacies compromise problem solving. And, we are gullible, often accepting views uncritically.

Critical Thinking: Integral to Problem Solving and Ethical Behavior

Critical thinking is a unique kind of purposeful thinking in which we use standards such as clarity and fairness. It involves the careful examination and evaluation of beliefs and actions to arrive at well-reasoned decisions.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 5 6

As Paul and Elder (2014) suggest, “much of our thinking, left to itself, is biased, distorted, partial, uninformed, or downright prejudiced … . Critical thinking begins, then, when we start thinking about our thinking with a view to improving it” (p. 366). Critical thinkers attempt to “live rationally, fairmindedly, and self-reflectively” (p. 366). Related characteristics suggested by Paul (1993, p. 63) and Paul and Elder (2014) are as follows: • Clear versus unclear • Accurate versus inaccurate • Relevant versus irrelevant • Deep versus narrow • Consistent versus inconsistent • Logical versus illogical • Complete versus incomplete • Significant versus trivial • Adequate (for purpose) versus inadequate • Fair versus biased or one-sided Critical thinking involves clearly describing and critically evaluating claims and arguments, no matter how cherished, and considering alternative views when needed to arrive at decisions that do more good than harm. This means paying attention to reasoning (how we think), not just the product. It involves asking questions you, as well as other people, may prefer to ignore such as: Do our services do more good than harm? (see Box 1.1). It may require blowing the whistle on harmful practices and policies (e.g., Grant, 2012). It requires paying attention to context (to link personal troubles to public issues (Mills, 1959). This is why there is so often lots of talk about critical thinking, but little actual critical inquiry, and it is why caring about clients is so important; it provides a source of courage to ask questions that have life-affecting consequences. Our ethical obligations of helping clients and avoiding harming them also provide a vital source of courage. Critical thinking can help you to clarify and solve problems or to discover they are not solvable. What problems are clients trying to solve? How would they like their lives to be different? How can you discover client strengths and environmental resources? Philosopher Karl Popper (1994) views all of life as problem solving and notes that we often seek problems (e.g., how to traverse a river on a raft). The skills, values, and traits related to critical thinking can help you minimize mistakes, such as not recognizing a problem;

6 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 7

confusing the consequences of a problem for the problem; ignoring promising alternatives; delaying a decision, which results in harm; and not following up your client (Caruth & Handlogten, 2000). Critical thinking can help you avoid confirmation biases. Dewey (1933) views reflection as “active, persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends” (p. 118). This self-reflection is integral to critical thinking (e.g., Schön, 1983).

Critical Thinking as Radical: Raising Questions and Understanding Context

Critical thinking is one of the most radical activities in which anyone can engage. The term reflection is popular; but, as Steven Brookfield notes, “Reflection is not by definition critical” (1995, p. 8). Like any subject, critical thinking can be approached from a narrow view or a broad view. A narrow view focuses on reasoning and related biases and fallacies, such as post hoc ergo propter hoc—assuming that because you get better after taking a pill, the pill was responsible for the change, when you were just about to get over your cold in the natural course of events (Skrabanek & McCormick, 1998). Recognizing the fallacies and biases described in this book—and avoiding their influence—should result in sounder decisions. Critical thinking requires attention to context: political, social, and economic factors that affect both problems and decisions, including research drawn on (see later discussion of science in this chapter). Such factors influence which problems we focus on and how we do so. Neither clients nor professionals may be aware of the extent to which decisions are shaped by such influences. Paul (1993) uses the term sociocentric biases to refer to societal influences on our beliefs (see also Paul and Elder [2014]). Who knows what and when, and who is permitted to ask probing questions, and what happens when they do so are part of our history, as illustrated by the death of Socrates. You may be considered a troublemaker by asking questions that may reveal knowledge others prefer to hide. Who has the right to know what and when? Consider, also, the fate of William Tynedale, who was burned at the stake when finally caught because he translated the Bible into English. Only the priests were supposed to have access to “the word.” What is “the word” today? What words cannot be spoken? What words cannot be questioned? What problems are hidden? What problems are created, for example, by those with special interests

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 7 8

(e.g., the pharmaceutical industry)? These questions illustrate the role of political, social, and economic factors in shaping what is viewed as a problem and what kind; often, there is a social control interest and effect (e.g., Foucault, 1977; Illich, Zola, McNight, Caplan, & Shaiken, 1977; Szasz, 1987). Evans and Giroux (2015) argue that dissent is ever more oppressed in the United States facilitated by increasing surveillance. Some groups and individuals have the resources to hide knowledge and promote ignorance, such as the harmful effects of prescribed medication (see the later discussion of fraud and corruption in this chapter). Public relations firms and advertising agencies are key in this process.

The Technological Society in Which We Live

We live in a technological society. Advertising, therapy, classification systems, human relations, and management are techniques that involve a “set of steps to produce desired outcomes” (Stivers, 2001, p. 9). There is a press for ever-greater efficiency and standardization, as can be seen in the widespread use of psychiatric labels that obscure individual differences, and epidemic uses of prescribed medication to solve life’s challenges (one out of every four women now takes a psychotropic medication [Holland, 2015]). Professional, corporate, and governmental interests as well as diverse technologies are ever-more intertwined. Conrad (1979) views technology (e.g., prescription drugs) as one of three forms of medical social control. (The other two are collaboration between healthcare provider institutions and ideology conveyed by the use of language.) Ellul (1965) argues that propaganda, encouraging action with “as little thought as possible” (p. 180), is an integral part of such a society in which moral problems are translated into social problems, and in which we expect technology to solve our problems (Stivers, 2001). It helps us to “adjust” to the alienating effects of such a society. It both creates and fulfills needs. It may be intentional or not. It must affect all people but appear personal. Propaganda distributed via schools, television, newspapers, magazines, radio, the Internet, professional education, and peer-reviewed publications is designed to integrate us into our society. The main function of such integrative propaganda is to maintain the status quo—(adjust) us into our society as happy, unthinking consumers.

8 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 9

Propaganda is most vicious not when it angers but when it ingratiates itself through government programs that fit our desires or world views sic[ ], through research or religion that supplies pleasing answers, through news that captures our interest, through educational materials that promise utopia, and through pleasurable films, TV, sports, and art … . the chief problem of propaganda is its ability to be simultaneously subtle and seductive—and to grow in a political environment of neutralized speakers and disempowered communities. (Sproule, 1994, p. 327)

Propaganda prevents confusion and anxiety created by competing propagandas; it provides group belonging in a society in which stress is endemic because of the faster pace, overorganization, loss of community, and competition.

Follow the Money

The helping professions and related activities are huge businesses (e.g., the nursing home industry; hospitals and healthcare systems, including the insurance industry; the pharmaceutical and medical device industries; the substance abuse treatment industry; the residential youth program industry; and the nutritional supplement industries). Closely related industries include the public relations and advertising industry; the contract research industry, which conducts research and prepares articles; and the publishing industry. Medical writing firms prepare articles and “push” therapies produced by those who pay them (e.g., see Singer, 2009). Whenever large sums of money are involved, conflicts of interests that compromise pursuit of avowed aims, such as helping clients and avoiding harm, are inevitable, including those that result in crimes (Barak, 2015). Professional organizations such as the American Psychiatric Association, the American Psychological Association, and the National Association of Social Workers compete for turf and may have conflicts of interest that harm clients (Camilleri & Parke, 2010). Certain states/behaviors/ conditions are promoted as a problem (and others ignored), and certain causes and remedies are highlighted. Loeske (1999) uses the term social problems industry to refer to all related groups, including politicians, the government, and the media.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 9 10

There has been increasing medicalization of problems-in-living (Conrad, 2007; Szasz, 1987). Health and progress are two great cultural motifs in our current society. The more well people who can be convinced they are sick, the more products can be sold. (Mis)behaviors as well as many other problems, including depression and anxiety, are viewed as “health” problems addressed by medical solutions, such as prescribed medication, obscuring the role of environmental factors such as poverty in contributing to such concerns (Abrams, 2012; Brown & Harris, 1978; Calderón-Garcidueñas, Torres-Jardón, Kulesza, Park, & D’Anqilli, 2014; Cutrona, Wallace, & Wesner, 2006). Related costs were estimated to be $77 billion in 2005 in the United States (Conrad, Mackie, & Mehrotra, 2010). The term biomedical industrial complex refers to the reinforcing and interlocking connection between the pharmaceutical, biotechnological, and medical industries that—together with academic experts in the helping professions, governmental funding, and regulatory bureaucracies, such as the National Institute of Mental Health (NIMH) and the Food and Drug Administration (FDA), and professional and family lobbies—promote and support a biomedical model of psychosocial distress (moving the cause from the social to the clinical) and disability. (Gomory, Wong, Cohen, & Lacasse, 2011, p. 137)

As Moncrieff (2008a) suggests, the medicalization of distress (moving causes from the social to the clinical) serves the neoliberal agenda of focusing on individuals as the source of their own distress. This medicalization has received increasing critique (e.g., Conrad, 2007; Szasz, 2007), including international conferences on “Selling Sickness.” Marketing values and strategies, prevalent throughout time in selling nostrums, have entered increasingly into the realm of professional practice, education, and even peer-reviewed literature (Bauer, 2004b; Gambrill, 2012a). These values include the creation of needs, desires, and alleged risks, and the use of marketing strategies to sell products and services to satisfy these needs and avoid these risks. Public relations and advertising agencies receive billions of dollars from industries such as the pharmaceutical and higher education industries to promote certain phenomena as problems and as particular kinds of problems that can be solved by buying their product. The helping professions and related industries such as pharmaceutical companies play a key role in

10 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 11

shaping our beliefs about what is a problem, what kind of problem it is, and how it should be approached. Problems are commodified (e.g., into an industry) and consumerism rules the day; individuals are viewed as the source of their problems whereas environmental causes, such as low- paying jobs and harsh working conditions, are ignored. Governmental agencies play a key role in shaping policies and practices. Consider the war on drugs, now viewed by most as a failure and one that has (and does) discriminate against poor people of color, including thousands in jail on drug charges—many because they cannot afford bail. Because you have grown up at a particular time in a particular society and are surrounded by content promoting grand narratives such as health and adjustment, it will be a challenge for you even to think of questioning them.

The Promotion of Ignorance

Political, social, and economic influences include the strategic cultivation of ignorance. Paul and Elder (2014) use the term activated ignorance to refer to the use of false information thought to be true. Ignorance is a resource (McGoey, 2012). For the past several years there has been a tsunami of publications revealing the hiding of adverse side effects of prescribed medication, failure to publish all clinical trials, harmful promotion of off-label uses of prescribed medication, lying on the part of pharmaceutical companies, and related conflicts of interest between academic researchers and the pharmaceutical industry. Ghostwriting is common; doctors pose as authors of articles that are actually written by writers in a marketing firm (e.g., Lacasse & Leo, 2010). Pharmaceutical firms “engage in massive lobbying to extend patent protection, increase tax credits, reduce the standards in the drug approval process, and maintain secrecy over clinical trials data” (Gagnon, 2013, p. 573). This tsunami includes continuing revelations of the flawed nature of peer review (e.g., it is full of inflated and misleading claims, Ioannidis, 2005, 2016). Politicians, advertisers, those in public relations and, sorry to say, even many academics and professionals, thrive because of bogus claims (e.g., Gøtzsche, 2013, 2015a & b; Moncrieff, 2008b; Whitaker & Cosgrove, 2015). Those who promote bogus claims may themselves have been fooled by false reports in the media, misleading content on websites of professional organizations, educational programs, texts, and the peer- reviewed literature. A related naiveté contributes to the sincerity with

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 11 12

which related claims are promoted. This highlights the importance of thinking for yourself—of the “critical spirit” (Paul, 1993; Siegel, 1993) to understand the context of professional practice and education.

Marketing in the Guise of Scholarship

Much of the material in peer-reviewed sources has more of the quality of advertisements (e.g., inflated claims based on misleading appeals to statistical significance, hiding negative information) than scholarly discourse (Gambrill, 2012a). Marketing values often dominate scholarly values (see the later discussion of the social aspects of science in this chapter). Ioannidis (2005) kicked off this increasing scrutiny of research in his article “Why Most Published Findings Are False.” “There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims” (p. 696). Freedman, Cockburn, and Simcoe (2015) estimate that $28 billion are spent each year in the United States on preclinical research that is not reproducible. There are fake journals, fake reviews, (e.g., authors reviewing their own manuscripts), and fake findings (Callaway, 2015). Two thousand articles have been flagged as flawed on retractionwatch.com. Biases in design of research, data analysis, and reporting contribute to exaggerated claims of “what we know.” Revelations of the flawed nature of peer review continue to emerge. We see the following:

• Inflated claims, including in peer-reviewed publications, such as hiding or minimizing limitations of research, including lack of a match between questions addressed and methods used to explore them (e.g., Ioannidis, 2005, 2008; Rubin & Parrish, 2007) • Biased estimates of the prevalence of concerns; advocacy in place of critical appraisal (e.g., Best, 2004) • Incomplete unrigorous literature reviews (e.g., Littell, 2006, 2008) • Misleading problem framing, such as labeling distress caused by environmental inequities as psychiatric problems (e.g., Gambrill, 2014a; Moncrieff, 2008a) • Failures of replication (Baker, 2015) This is encouraged by predatory open-access journals that ignore peer-review standards to make money. In 2014, 400,000 papers were

12 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 13

published in such journals (Shen & Björk, 2015). Lapses of scientific integrity include • Data fabrication • Data falsification (“cooking” or altering data) • Plagiarism • Unethical treatment of animals or human subjects • Undisclosed conflicts of interest • Violation of privileged material • Selective reporting of findings • Failure to publish • Unwillingness to share data • Misleading statistical tests and procedures • Inaccurate authorship credit, as in ghostwriting • Redundant publication • Data dredging • Bogus citations Concerns about flaws in the peer-reviewed literature are not new. What is new are electronic means of deception (e.g., fake reviewers and fake journals), the increasing revelations of the prevalence of such flaws, and description of contributing circumstances, including fraud and corruption on the part of professional organizations such as the American Psychiatric Association; academic researchers with conflicts of interest (e.g., receiving money from pharmaceutical companies); and deceptions on the part of the pharmaceutical industry going back decades (e.g., Angell, 2005; 2009; Brody, 2007; Kassirer, 2005). Consider these quotes from current and former editors-in-chief of major medical journals: It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine. (Angell, 2009, p. 11) The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 13 14

trends of dubious importance, science has taken a turn towards darkness… . scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data… [A]‌cquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale … . Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication. National assessment procedures, such as the Research Excellence Framework, incentivize bad practices. (Horton, 2015, p. 1380) Rigging medical studies, misrepresenting research results published in even the most influential medical journals, and withholding the findings of whole studies that don’t come out in a sponsor’s favor have all become the accepted norm in commercially sponsored medical research. To keep the lid sealed on this corruption of medical science—and to ensure its translation into medical practice—there is a complex web of corporate influence that includes disempowered regulatory agencies, commercially sponsored medical education, brilliant advertising, expensive public relations campaigns, and manipulation of free media coverage. And last, but not least, are the financial ties between many of the most trusted medical experts and the medical industry. These relationships bear a remarkable resemblance to the conflicts of interest the Securities and Exchange Commission recently brought to a halt after learning that securities analysts were receiving bonuses for writing reports that drove up stock prices with the intent of bringing in more investment banking business. (Kassirer, 2005, p. 91)

Related concerns are reflected in the use of the question “How good are you in detecting bullshit?” on testingtreatments.org, which was created to help both clients and professionals appraise claims critically. Misinformation often remains uncorrected (Doshi, 2015; Lewandowsky, Ecker, Seifert, Schwarz, & Cook, 2012). Promising developments include AllTrials (www.alltrials.net), which is dedicated to registering and reporting all clinical trials; the

14 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 15

Restoring Invisible and Abandoned Trials (or RAIT) initiative; a new center, Meta-Research Innovation Center at Stanford (or METRICS), which was established to decrease the enormous waste in conducting research that cannot answer the questions addressed; and the Science Exchange Reproducibility Initiative. Valuable websites include Bad Science, Bad Science Watch, Center for Open Science, Berkeley Initiative for Transparency in the Social Sciences, Healthy Skepticism, Integrity in Science, Open Science Collaboration, Project for Scholarly Integrity, Center for Scientific Integrity, and Sense About Science.

Corruption: Institutional and Individual

Social, political, and economic pressures and incentives create conflicts of interests that may result in misdirection of resources and lack of transparency (Doshi, 2015; Fava, 2010; Urbina, 2009). Political corruption refers to “Manipulation of policies, institutions and rules of procedure in the allocation of resources and financing by political decision makers, who abuse their position to sustain their power, status and wealth” (transparency.org). Transparency International defines corruption as “misuse of entrusted power for private gain.” It may include bribery, extortion, and/or embezzlement. Corrupt practices include bribes, kickbacks, false invoicing, theft, and fraud (www. u4.no) (Baker, 2015). Revelations of incestuous relationships between academic researchers and the pharmaceutical industry continue to emerge (Gøtzsche, 2013, 2015a & b; Whitaker & Cosgrove, 2015). Institutional corruption refers to “widespread or systemic practice, usually legal that undermines an institution’s objectives or integrity” (Rodwin, 2013a, p. 544). “It can result from improper dependency (for money or for information), from financial incentives that are at odds with the needs of patients and public health, from market failure, or from marketing that has compromised medical practice” (Rodwin, 2013a, p. 544). There is a failure of organizations and/or professions to honor their professed obligations (e.g., to help clients) (see wiki. lessig.org). It is reflected in biased research encouraged by the quest for status and prestige. Consider the exposure of collusion between key staff members in the American Psychological Association, including the ethics director, during George W. Bush’s administration to allow torture of suspected terrorists (see Hoffman et al., 2015). Strategic

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 15 16

actions were taken by the American Psychological Association to neutralize dissent (see also Risen, 2015). Savedoff and Hussmann (2006) suggests that vulnerability to corruption is high in the health area because of uncertainties surrounding need for service, many actors, unequal information among various players, and the large amounts of public money allocated for healthcare. (For a conceptual model of health corruption see Vian [2008].) Failure of regulatory agencies such as the FDA to do their jobs contributes to corruption and fraud. The FDA is “financially dependent on industry user fees from reviewing applications to market new drugs” (Light, Lexchin, & Darrow, 2013). Whitaker and Cosgrove (2015) argue that related “problematic behaviors have become normalized within the institution and thus accepted” (p. 75). Examples include academic researchers taking money from pharmaceutical companies. “Harvard University researchers (Dr. Joseph Biederman, Thomas Spencer, and Timothy E. Wilens) who “discovered” bipolar disorder in children and promoted treatment with psychiatric drugs did not report a combined $4.2 million in income from drug companies to their university (Harris & Carey, 2008). Between 1994 and 2003, “the number of American children and adolescents treated for bipolar disorder increased 40-fold” (p. A1) and the sale of drugs used to treat it doubled (Carey, 2007). Treatment usually included prescribed medications such as Risperdal, an antipsychotic. This corruption of psychiatry has received a great deal of attention (e.g., Moncrieff, 2008a; Whitaker & Cosgrove, 2015). Gøtzsche (2013, 2015a) argues that psychiatric drugs are the most corrupted area.

How come we have allowed drug companies to lie so much, commit habitual crime and kill hundreds of thousands of patients, and yet we do nothing? Why don’t we put those responsible in jail? Why are many people still against allowing citizens to get access to all the raw data from all clinical trials … . (Gøtzsche, 2015, p. 3)

The American Psychiatric Association “receives millions of dollars from pharmaceutical companies for advertising and grants” (Cosgrove & Wheeler, 2013). Millions more are made from publication of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (American Psychiatric Association, 2013). The biological view promoted in this manual supports the financial interests of drug companies. Most members

16 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 17

of many Diagnostic and Statistical Manual of Mental Disorders task forces have financial ties to the pharmaceutical industry (Cosgrove & Krimsky, 2012). Pharmaceutical companies have been found to manipulate data in favor of drugs they produce (Brown, 2013), including hiding trials with negative or adverse effects (see the following section on fraud). Drug companies make use of opinion leaders to promote their drugs and woo physicians with gifts. Legislators are wooed by contributions from drug firms encouraged by the hundreds of pharmaceutical lobbyists in Washington. Kickbacks are common (Jain, Nundy, & Abbasi, 2014). Corruption and fraud also occur in nonprofits, which provide foster care for children as well as in-residential care for adolescents, including failure to select appropriate foster parents; and related corporate interests, including involvement of venture capital and equity firms (e.g., Roston & Singer- vine, 2015) Environmental options may be ignored because of their costs, such as moving families from poor to less poor neighborhoods.

Fraud

Fraud and corruption are interlinked but are not the same; one can occur without the other (e.g., Iyer & Samociuk, 2006). Fraud refers to intentional misrepresentation of yourself or the effect of certain actions for unlawful gain (e.g., see U4 Anti-Corruption Training Course). It involves deception and misrepresentation (e.g., Callaway, 2011). Examples include bogus invoicing (charging for services never provided) and transferring public money into private account, and selective trial reporting (e.g., Wolfe, 2015). It may occur on the part of individuals and/or organizations. For example, a Florida ophthalmologist was indicted on forty-six charges of medical fraud. He diagnosed serious eye diseases falsely and charged Medicare for treatments that were not needed (Dyer, 2015). In 2005, it was estimated that New York Medicaid fraud may have amounted to billions of dollars (Levy & Luo, 2005). A jury found that claims of Jonah, a counseling group in New Jersey offering “conversion therapy” that homosexuality was a curable disorder, amounted to consumer fraud (Eckholm, 2015). The pharmaceutical industry leads in fraud rulings in the United States (Corcoran, 2010). Johnson & Johnson paid more than $2.2 billion to resolve criminal and civil investigations for promoting the use of Risperdal, Invega, and Natrecor for unapproved use, and payment of kickbacks to

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 17 18

physicians and to a long-term care pharmacy provider (Department of Justice, Office of Public Affairs, 2013). Findings were based on the False Claims Act: “J & J’s promotion of Risperdal for unapproved uses threatened the most vulnerable populations of our society—children, the elderly and those with developmental disabilities” (p. 2). Quality- of-care concerns regarding the use of second-generation antipsychotic drugs revealed by an analysis of Medicaid claims in five states in 2014 found problems in 67% of the claims reviewed, including the drug was taken too long (34%), at the wrong dose (23%), too many drugs (39%), poor monitoring (53%), wrong treatment (41%), the patient was too young (17%), and side effects (7%). The Office of Research Integrity was established in 1989 in response to fabrication of data in the biomedical sciences. This office describes scientific fraud as follows:

• Fabrication: “inventing data, making up results and recording or reporting them in any way” • Falsification: manipulating research material, data, equipment, or processes and/or omitting data or results in such way that the actual results of the study do not adequately or accurately represent the research records (not mentioning the reality) • Plagiarism: appropriation of someone else’s ideas, processes, results, data, or words without noting their origin or without giving appropriate credit to the original author Fraud and corruption in the helping professions are so extensive that special organizations have been formed and newsletters written to help consumers evaluate claims (e.g., Health Letter published by Public Citizens Health Research Group; see also the Transparency International website at www.transparency.org). The sheer enormity and ethical lapses of corruption and fraud are hard to take in. Related activities reflect marketing, public relations, and advertising aims and strategies rather than scholarly aims of accurate description and critical appraisal (Gambrill, 2012a). Here are some examples: • Fudging the results of studies of selective serotonin reuptake inhibitors for depression and claiming they are more effective than older drugs. (See Whitaker and Cosgrove [2015] for a riveting description of how data were fudged.) • Publishing peer-reviewed articles using fake data (Callaway, 2015)

18 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 19

• Hiding the superiority of eight-week outcomes of placebo subjects compared with those who took Xanax for anxiety; the four-week outcome was the focus (Whitaker & Cosgrove, 2015). Nationwide campaigns were sponsored by the American Psychiatric Association and the National Mental Health Association to alert the public “about depression and how often it went undiagnosed” (p. 31). Key messages included the framing of depression as a medical illness and assurance of effective treatments. National Depression Screening days were launched. Indeed, I took the students in my class on social problems and psychopathology to a screening day held in Berkeley at the Student Union of the University of California a few years ago for us all to get screened. Does screening do more harm than good?

Pseudoscience

The term pseudoscience refers to material that makes sciencelike claims but provides no evidence for them. Pseudoscientists use the trappings of science without the substance (Bauer, 2004a). Pseudoscience can be found in all fields (e.g., Lilienfeld, Lynn, & Lohr, 2015; Moncrieff, 2008b; Thyer & Pignotti, 2015). Pseudoscience is characterized by a casual approach to evidence (weak evidence is accepted as readily as strong evidence). Hallmarks of pseudoscience include the following (Bunge, 1984; Gray, 1991): • Uses the trappings of science without the substance • Relies on anecdotal evidence • Is not self-correcting • Is not skeptical and exaggerates claims • Equates an open mind with an uncritical one • Ignores or explains away falsifying data • Uses vague language, psychobabble, and biobabble • Produces beliefs and faith but not knowledge • Is often not testable • Does not require repeatability • Is indifferent to facts and lacks connectivity to research • Often contradicts itself • Creates mystery where none exists by omitting information • Relies on the wisdom of the ancients; the older the idea, the better

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 19 20

• Appeals to false authority (or authority without evidence), emotion, sentiment, or distrust of established fact • Argues from alleged exceptions, errors, anomalies, and strange events

Indicators of pseudoscience include irrefutable hypotheses and a continuing reluctance to revise beliefs even when confronted with relevant criticism. It makes excessive (untested) claims of contributions to knowledge. Pseudoscience is a billion-dollar industry (see “How to Sell a Pseudoscience” by Pratkanis [1995]). Products include self-help books, “subliminal” tapes, and call-in advice from “authentic psychics” who have no evidence that they accomplish what they promise. The terms science and scientific are often used to increase the credibility of a claim although no evidence is provided to support it. Proselytizers of many sorts cast their advice as based on science. They use “trappings” of science to pull the wool over our eyes by suggesting critical tests of claims that do not exist. The misuse of appeals to science to sell products or to encourage certain beliefs is a form of propaganda. Classification of clients into psychiatric categories lends an aura of scientific credibility (Boyle, 2002; Kirk, Gomory, & Cohen, 2013). A critical attitude, which Karl Popper (1972, 1992) defines as a willingness and commitment to open up favored views to severe scrutiny, is basic to science, distinguishing it from pseudoscience. Popper uses the criterion of falsifiability to demark what is or could be scientific knowledge from what is not or could not be. For example, there is no way to refute the claim “there is a God,” but there is a way to refute the claim that “assertive community outreach services for the severely mentally ill reduce substance abuse.” We could, for example, randomly distribute clients to a group providing such services and compare those outcomes with those of clients receiving no services or other services. Bauer (2001) argues that demarcation is fuzzy as revealed by what scientists actually do—for example, fail to reject a favored theory in the face of negative results (e.g., perhaps a test was flawed) and the prevalence of pseudoscience within science (e.g., belief in N rays).

Quackery

Quackery refers to the promotion and marketing for a profit of untested, often worthless, and sometimes dangerous products and procedures

20 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 21

by either professionals or others (Jarvis, 1990; Young, 1992a). Quack reasoning reflects pseudoscience. A quack

• Promises quick, dramatic, miraculous cures • Describes problems and outcomes in vague terms • Uses anecdotes and testimonials to support claims • Does not incorporate new ideas or evidence; relies on dogma • Objects to testing claims • Forwards methods and theories that are not consistent with empirical data • Influence by a charismatic promoter • Claims effects cannot be tested by usually accepted methods of investigation such as clinical trials • Mixes bona fide and bogus evidence to support a favored conclusion (e.g., Jarvis, 1990; Porter, 2000) • Attacks those who raise questions about claims

Millions of dollars are spent by consumers on worthless products, such as magnetic devices to treat pain, with no evidence that they are effective (e.g., Winemiller, Billow, Laskowski, & Harmsen, 2003). Fads are often advanced on the basis of quackery (Jacobson, Foxx, & Mulick, 2005). Fraud takes advantage of pseudoscience and quackery. The prevalence of propaganda (e.g., censorship and inflated claims) in the professional literature and other sources, and related avoidable harms, highlights the importance of the “critical spirit”—critically appraising what you see, hear, and read, not only in the media, but also in professional venues including conferences, workshops, and professional publications. For every claim that has survived critical tests, there are thousands of bogus claims in advertisements, newscasts, films, TV, newspapers, the Internet, and professional sources. You must “think for yourself” so you are not misled by others who value status and money over helping clients, and/or are themselves misled by alleged experts (Rampton & Stauber, 2001).

Knowledge, Skills, and Values Related to Critical Thinking

Paul (1993) lists purpose first as one of the components of critical thinking (see Box 1.3; see also Paul and Elder [2014]). If our purpose is to help

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 21 22

Box 1.3 Characteristics of Critical Thinking

1. It is purposeful. 2. It is guided by intellectual standards: relevance, accuracy, precision, clarity, depth, and breadth. 3. It supports the intellectual traits of humility, integrity, perseverance, empathy, and self-discipline. 4. The thinker can identify the elements of thought in thinking about a problem. The critical thinker asks the following questions:

• What is the purpose of my thinking (goal/objective)? • On what questions (or problems) is it focused? • Within what point of view (perspective) am I thinking? • What concepts or ideas are central to my thinking? • What am I taking for granted? What assumptions am I making? • What information am I using and how am I interpreting it? • What are my conclusions? • If I accept the conclusions, what are the implications?

5. It is self-assessing and self-correcting using intellectual standards. 6. There is an integrity to the whole system. The thinker can critically examine her thought as a whole and consider its parts as well. The thinker is committed to being intellectu- ally humble, persevering, courageous, fair, and just. The critical thinker is aware of the ways in which thinking can become distorted, misleading, prejudiced, superficial, unfair, or otherwise defective. 7. It yields a well-reasoned answer. If we know how to check our thinking, do so, and practice it, then our thinking is likely to be productive. 8. It is responsive to the social and moral imperative to argue enthusiastically from opposing points of view and to seek and identify weakness and limitations in one’s own position. Critical thinkers are aware there are many points of view, each of which—when thought through—may yield some insight.

Source: Adapted with permission from Paul, R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (Rev. 3rd. Ed.). Santa Rosa, CA: Foundation for Critical Thinking, pp. 20–23. www.criticalthinking.org.

clients, we must think critically—make informed guesses about what may be a solution—then appraise our guesses critically using standards such as clarity, accuracy, relevance, and completeness. Critical thinking involves evaluating evidence, considering well-argued alternative views, and presenting opposing views accurately. Accuracy is valued over “winning” or social approval. Many kinds of evidence are drawn on to make decisions, including descriptive data, experience, research, personal opinion, tradition, and popularity. Which ones provide a sound guide? Is a problem framed in a misleading way? Are statistical analyses in research reports misleading? Related questions include the following:

22 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 23

• Is a claim accurate? What evidence is provided? Have critical tests of this claim been carried out? What were the results? Were studies relatively free of bias? How representative were the samples? Have the results been replicated? Claims may concern values, alleged facts, and/or interpretations. • Who said the claim was accurate? How reliable are these sources? Do they have vested interests in one point of view? • Are key concepts described clearly (e.g., risk)? • Are the facts presented correct? Have important facts been omitted? • Are there alternative well-argued views? Are these described accurately? • Are weak appeals used, for example, to emotion or special interests? Specialized knowledge may be required to think effectively in a domain, and Internet sources can help us to locate this. Questions that may arise are shown in Box 1.4. (See also the discussion of background and foreground knowledge in Part 4.)

Related Skills

Skills include recognizing assumptions and claims, and appraising their accuracy (see Box 1.5). Making accurate inferences about the causes of behavior often requires skill in gathering and synthesizing different kinds of information guided by well-argued theory, which requires paying attention to context. Valuable skills include identifying assumptions and their implications (consequences), suspending judgment in the absence of sufficient evidence to support a claim/decision, understanding the difference between reasoning and rationalizing, and stripping an argument of irrelevancies and phrasing it in terms of its essentials. Seeking counterevidence to preferred views and understanding the difference between the accuracy of a belief and the intensity with which it is held is vital.

Knowledge

Some define knowledge as information—for example, identifying sources of childcare in a community or effective interventions for enhancing positive parenting skills. Nickerson (1986) defines knowledge as information that decreases uncertainty about how to achieve a certain outcome (I would add, or reveals uncertainty.) He suggests that three

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 23 24

Box 1.4 Socratic Questions Related to Decision Making and Problem Solving

Questions of Clarification

• What do you mean by __ _ _ _ ? • What is your main point? • How does __ _ _ _ relate to __ _ _ _ ? • Could you put that another way? • Is your basic point __ _ _ _ ? • Let me see if I understand you. Do you mean __ _ _ _ or __ _ _ _ ? • Could you give me an example?

Questions About Assumptions

• What are you (am I) assuming? • What could we assume instead? • You seem to be assuming _____. Do I understand you correctly? • Is it always the case? Why do you think the assumptions holds here?

Questions About Reasons and Evidence

• What is an example? • Are these reasons adequate? • Why do you think this is true? • Do you have any evidence for this? • How does this apply to this case? • What would change your mind? • What other information do we need? • How could we find out whether it is true?

Questions About Viewpoints or Perspectives

• You seem to be approaching this from _____ perspective. Why have you chosen this view? • How could you answer objections to this? • What is an alternative view?

Questions About Implications and Consequences

• What are you implying by this? • When you say __ _ _ _ , are you implying __ _ _ _ ? • If this happened, what might occur as a result? Why? • What is an alternative?

Questions About the Question

• Do we all agree this is the key question? • Is this the same issue as __ _ _ _ ? • What does this question assume?

24 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 25

• Why is this question important? • How could someone settle this question? • Can we break this question down? • Is the question clear? Do we understand it? • Is this question easy or hard to answer? Why? • To answer this question, what questions do we have to answer first?

Source: Adapted with permission from Paul R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (Rev. 3rd Ed.). Santa Rosa, CA: Foundation for Critical Thinking,www.criticalthinking.org .

kinds of knowledge are important in critical thinking: critical thinking itself, domain-specific knowledge, and self-knowledge.

Domain-Specific Knowledge Domain-specific knowledge, including both content (knowing what) and procedural knowledge (knowing how to apply content knowledge), may be needed to make sound decisions. To think critically about a subject, you must know something about that subject. The “possession of relevant bodies of information and a sufficiently broad experience with related problems to permit the determination of which information is pertinent, which clinical findings are significant, and how these findings are to be integrated into appropriate hypotheses and conclusions” (Elstein et al., 1978, p. x) are foundation components related to competence in clinical problem solving. Knowledge of content influences the questions asked and the information professionals seek and share with clients (Keen, Klein, & Alexander, 2003). Background knowledge is required to evaluate claims. Consider the following example: • Depression always has a psychological cause. • Mr. Draper is depressed. • Therefore, the cause of Mr. Draper’s depression is psychological in origin. The conclusion is false because the first claim is false. Thus, critical thinking skills cannot replace content knowledge. Both background and foreground questions arise in working with clients. Background questions concern knowledge about a concern such as depression. Foreground questions concern information about an individual client (see the later discussion of evidence-based practice [EBP] in Part 4). The greater the knowledge that can decrease uncertainty about what decision is best,

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 25 26

Box 1.5 Examples of Critical Thinking Skills

• Raise and pursue important questions. • Gather relevant information and evaluate its accuracy. • Critically appraise claims. • Identify unstated assumptions. • Identify important similarities and differences. • Recognize contradictions and inconsistencies. • Avoid oversimplification. • Clarify issues, conclusions, or beliefs. • Critically evaluate arguments, interpretations, beliefs, and theories. • Clarify and analyze the meaning of words or phrases. • Use sound criteria for evaluation. • Clarify values and standards. • Detect bias. • Distinguish relevant from irrelevant questions, data, claims, or reasoning. • Transfer insights to new contexts; make interdisciplinary connections. • Arrive at well-reasoned conclusions. • Make well-reasoned inferences and predictions. • Compare and contrast ideals with actual practice; locate gaps. • Evaluate one’s own reasoning. • Analyze and evaluate arguments, interpretations, beliefs, practices, and policies, including their consequences. • Communicate effectively with others. • Make interdisciplinary connections. • Explore thoughts underlying feelings and feelings underlying thoughts.

Source: See for example Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In B. Baron & R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9–26). New York: W. H. Freeman; and Paul R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (Rev. 3rd Ed.). Santa Rosa, CA: Foundation for Critical Thinking. www.criticalthinking.org

the more important it is to be familiar with this. Thus, just as domain- specific knowledge is necessary but not sufficient for making informed decisions, critical thinking skills cannot replace knowledge of content. (For discussion of expertise, see Exercise 23.)

Self-Knowledge Critical thinking requires evaluating your thinking and learning styles. Nickerson (1986) views knowledge about oneself as one kind of knowledge central to critical thinking. The term meta-cognitive refers to being aware of and influencing your reasoning process by asking questions such as: How am I doing? What assumptions am I making? Is this claim true? What does “trauma” mean? How good is the evidence? Do I understand this

26 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 27

point? What mistakes may I be making? Am I discarding good ideas too soon? Such questions highlight the self-correcting, reflective role of critical thinking—criticism of conjectures about what may be true (Popper, 1972). Increasingly meta-cognitive levels of thought include the following:

• Tacit use: Thinking without thinking about it • Aware use: Thinking and being aware that you are thinking • Strategic use: Thinking that is organized using “conscious” strategies that enhance effectiveness • Reflective use: “[R]‌eflecting on our thinking before and after—or even in the middle of—the process, pondering how to proceed and how to improve” (Swartz & Perkins, 1990, p. 52) Self-knowledge includes familiarity with the strengths and limitations of thinking in general as well as knowledge of your personal strengths and limitations that influence how you approach learning, problem solving, and decision making. Resources include self-criticism, such as asking: What are my biases? Is there another way this problem could be structured? They include tools such as drawing a diagram of an argument. Three of the basic building blocks of reasoning suggested by Paul in Box 1.3—ideas and concepts drawn on, what is taken for granted, and point of view used—concern background knowledge, which influences how we approach problems. Without criticism (self- reflection and self-knowledge), unrecognized fallacies and biases, such as the illusion of validity, are more likely to interfere with problem solving. You are less likely to recognize your ignorance in areas that affect clients’ lives (see Part 4). Learning requires recognizing ignorance (Do I really understand a concept?). It requires asking questions about important information needs, and a willingness to acknowledge and learn from mistakes. Perkinson (1993) suggests that if we are not making mistakes, we are probably not learning. Self-knowledge includes accurate estimates of ignorance regarding a question (both personal and collective).

Related Values, Attitudes, and Dispositions

Valuable attitudes include recognizing the fallibility of our beliefs and the probability of bias in them, and valuing the discovery of ignorance as well as knowledge. Predispositions and attitudes include fair-mindedness (accurate understanding of other views) and open-mindedness (critically

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 27 28

appraising your views as well as those of others), a desire to be well informed, a tendency to think before acting, and curiosity (Baron, 2000; Ennis, 1987; Paul & Elder, 2014). These attitudes reflect underlying values about human rights, and the dignity and intrinsic worth of all human beings (Brookfield, 1987; Paul, 1993). Popper (1994) argues they are vital to an open society in which we are free to raise questions and are encouraged to do so (see Boxes 1.6 and 1.7). Critical thinkers question what others take for granted. They ask questions such as: What does it mean? How good is the evidence? They question values and positions that may be common in a society, group, or their own family. It takes courage to raise questions in settings in which there is “a party line.” And, you must pick your battles, focusing on assumptions that have life-affecting consequences for clients. Skill in raising questions in a diplomatic way are important (see Exercise 21). Even when posed with tact, questions may be viewed as threatening. Only by an open dialogue, in which there is critical appraisal of claims and

Box 1.6 Values and Attitudes Related to Critical Thinking

• Belief in and respect for human rights, and the dignity and intrinsic worth of all people. • Respect for the truth above self-interest. • Value learning and critical discussion. • Value open-mindedness, seriously consider other views, reason effectively from premises with which you disagree, and withhold judgment when the evidence and reasons are insufficient. • Value being well-informed. • Seek reasons for beliefs and claims. • Rely on sound evidence. • Consider the total situation (the context). • Remain relevant to the main point. • Seek alternatives. • Take a position (and change it) when the evidence and reasons are sufficient to do so. • Seek clarity. • Deal in an orderly manner with the part of a complex whole. • Be sensitive to the feelings, level of knowledge, and degree of sophistication of others. • Think independently. • Persevere in seeking clarity and evaluating arguments.

Source: Adapted with permission from Paul, R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (Rev. 3rd Ed.). Santa Rosa, CA: Foundation for Critical Thinking. www.criticalthinking.org. See also Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In B. Baron & R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9–26). New York: W. H. Freeman; and Popper, K. R. (1972 [1963]). Conjectures and refutations: The growth of scientific knowledge (4th Ed.). London: Routledge and Kegan Paul.

28 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 29

Box 1.7 Valuable Intellectual Traits

• Intellectual humility: Recognizing the limits of our own knowledge, including circumstances in which we are likely to deceive ourselves, and maintaining a sensitivity to bias, prejudice, and limitations of our viewpoint. Recognizing that we should never claim more knowledge/expertise than we have. This does not imply spinelessness or submissiveness. It implies a lack of intellectual pretentiousness, boastfulness, or conceit, combined within insight into the logical foundations (or lack of such foundations) of our beliefs. Questions here include: How much do I really understand/know about _____? Am I competent to help this client? • Intellectual courage: Facing and fairly addressing ideas, beliefs, or viewpoints toward which we have strong negative emotions and to which we have not given a serious hearing. This courage is connected with the recognition that ideas considered dangerous or absurd may be reasonable and that our conclusions and beliefs are sometimes false or misleading. To determine for ourselves what is accurate, we must not “accept” what we have “learned” passively and uncritically. Intellectual courage comes into play here, because inevitably we will come to see some truth in some ideas held strongly by others. We need courage to be true to our own thinking in such circumstances. The penalties for nonconformity can be severe. • Intellectual empathy: Being aware of the need to put ourselves in the place of others to understand them; accurately describing the viewpoints and reasoning of others, and reasoning from premises, assumptions, and ideas other than our own. It includes remembering occasions when we were wrong despite a conviction that we were right. • Intellectual integrity: Honoring the same rigorous standards of evidence to which we hold others, practicing what we advocate, and admitting discrepancies and inconsistencies in our own thoughts and actions • Intellectual perseverance: Pursuing accuracy despite difficulties, obstacles, and frustra- tions; relying on rational principles despite the irrational opposition of others; recognizing the need to struggle with confusion and unsettled questions to achieve deeper understanding or insight • Confidence in reason: Confidence that, in the long run, our higher interests and those of humankind at large will be best served by giving the freest play to reason by encouraging others to develop their rational faculties; faith that, with proper encouragement and education, people can learn to think for themselves, form rational views, draw reasonable conclusions, think coherently and logically, persuade each other by reason, and become reasonable persons, despite obstacles to doing so • Fair-mindedness: Treating all viewpoints alike, without reference to our own feelings or vested interests, or the feelings or vested interests of our friends, community, or nation. This implies adherence to intellectual standards without reference to our own advantage or the advantage of our group. • Intellectual autonomy: Being motivated to think for ourselves

Source: Adapted with permission Paul, R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (Rev. 3rd. Ed). Santa Rosa, CA: Foundation for Critical Thinking, pp. 467–472. www.criticalthinking.org.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 29 30

consideration of opposing views, may you determine that you are wrong and discover a better idea. Critical thinking involves taking responsibility for the claims you make and the arguments you offer. It requires flexibility and a readiness to recognize and welcome the discovery of mistakes in your own thinking. Critical thinking is independent thinking—thinking for yourself. Problem solving requires coming up with ideas (conjectures) as well as appraising them critically (reasoning). Creativity may be required to discover alternative views. Related thinking styles, attitudes, and strategies include • A readiness to explore and to change, including restructuring of understanding • Attention to problem finding as well as problem solving • Immersion in a task • Belief that knowing and understanding are products of one’s intellectual process • An emphasis on understanding • Valuing complexity, ambiguity, and uncertainty combined with an interest in finding order • Valuing feedback but not deferring to convention and social pressures • Recognizing multiple perspectives on a topic • Deferring closure in the early stages of a task (Kaufman & Sternberg, 2006).

Benefits of Critical Thinking

Critical thinking is integral to making decisions that do more good than harm. Any intervention, including use of an assessment instrument, may harm as well as help. For example, using invalid assessment measures may result in selection of services that do more harm than good. As a critical thinker, you are more likely to discover problem- related circumstances, select effective practices and policies, make accurate predictions, accurately access the likelihood of attaining hoped- for outcomes, and make timely changes in plans. Because you critically appraise claims, you are less likely to be bamboozled by deceptive research, misleading advertisements, pseudoscience, and quackery. You are more likely to detect flawed arguments and to recognize errors and mistakes as learning opportunities. You are more likely to avoid cognitive, motivational, and affective biases as well as fallacies that may

30 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 31

result in poor decisions. You are more likely to minimize avoidable ignorance.

Pay Attention to Context: Understand the Big Picture

Critical thinking encourages us to think contextually, to consider the big picture—to connect personal problems, such as depression, and public issues, such as lack of affordable housing and an inequitable tax structure (Case & Deaton, 2015; Marmot, 2015; Mills, 1959). Many problems are not solvable by professionals, such as the lack of well-paying unskilled jobs, poor-quality education, and the lack of healthcare for all residents. Only by attending to context may you identify policies and related legislation that affect clients’ well-being and that need changing. What is the history of a problem? What is the goal of a discussion? (See Exercise 30.) Propaganda hides context such as corporations and professional organizations that promote the “self” as the locus of causes of problems such as anxiety. Oversimplifications ignore context, as do many biases, including the fundamental attribution error, in which we focus on attributes of a person and ignore environmental circumstances. Problems-in-living such as lack of access to quality healthcare often reflect social and economic inequities encouraged by institutional corruption and strategic use of ignorance, as discussed earlier in the section “Follow the Money.”

Cultivate Curiosity

Reflective thinking encourages curiosity about the world; it discourages an “I know everything” attitude that encourages illusions of knowledge and discourages a lively interest in the world. If you think you know everything or accept uncritically what authorities say, you lose opportunities to discover new insights for yourself; you lose opportunities to understand clients because you think you already “know” them, as if we could ever “know” another person.

Have Empathy for Others and Ourselves

Understanding the context of behavior encourages empathic rather than judgmental reactions, even when confronted with challenging situations that “push your buttons.” Valuing truth means having a sincere interest in

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 31 32

understanding other points of view. We know we may be (and often are) wrong. Kuhn (1970) argues that we cannot talk fruitfully (learn from one another) if we have different frameworks. Karl Popper argues that what is important are theories and problems, not frameworks. He points out that we share many problems, regardless of our particular frameworks. Critical thinking involves being fair-minded—accurately describing opposing views and using the same rigorous standards to critique both preferred and disliked views. It discourages arrogance, the assumption that we know better than others or that our beliefs should not be subject to critical evaluation. As Popper emphasized, “In our infinite ignorance we are all equal” (Popper, 1992, p. 50). These attitudes reflect a belief in and respect for the intrinsic worth of all human beings, for valuing learning and truth without self-interest, and for respecting opinions that differ from one’s own (Nickerson, 1988–1989).

Acknowledge Mistakes and Errors

Errors and mistakes when making decisions in the helping profession are inevitable. Values and skills related to critical thinking encourages their recognition so we can learn from and minimize them.

Encourage Openness to Criticism and Disagreement

Solving problems requires critically appraising assumptions questioning beliefs, and welcoming criticism from others. This helps us to evaluate arguments and claims. Being familiar with common fallacies and biases encourages an openness to criticism, both from yourself and others. You will not take criticism personally; indeed, you will seek it, because this is how we discover errors and mistakes in our thinking.

Evaluate Claims and Arguments

Making decisions involves offering and evaluating arguments in favor of one course of action rather than another. In an argument, some statements (the premises) are claimed to support or provide evidence for another statement (the conclusion). We are often patsies for false claims because we do not think carefully about them. A key part of an argument is the claim, conclusion, or position put forward. A second part

32 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 33

comprises the reasons or premises offered to support the claim. Are they accurate? A third consists of the reasons given for assuming the premises are relevant to the conclusion. These are called warrants (Toulmin, 2003). Here’s an example of an argument not supported by its warrant:

• Premise: After extensive counseling, Mrs. Elman reported being sexually abused by her father as a child. • Conclusion: Her father abused Mrs. Elman sexually as a child. • Warrant: The (incorrect) assumption that all memories are accurate.

An argument is unsound if there is something wrong with its logical structure, it contains false premises, or it is irrelevant or circular (see Exercise 30).

Recognize Fallacies

Knowledge of fallacies and skill in spotting them helps you to avoid dubious claims and related, unsound arguments. Fallacies that evade the facts appear to address them, but do not. Examples include “begging the question” (assuming what must be argued) and circular reasoning (see Part 3). Vacuous guarantees may be offered, such as assuming that because a condition ought to be, it is the case. In the fallacy of sweeping generalization, a rule or assumption that is valid in general is applied to a specific example for which it is not true. Consider the assertion that parents abused as children abuse their own children. In fact, a large percentage do not. Other fallacies distort facts or positions, as in “strawperson arguments,” in which an opponent’s view is misrepresented, usually to make it easier to attack. Diversions such as raising trivial points, irrelevant objections, and emotional appeals may be used to direct attention away from the main point of an argument. Some fallacies work by creating confusion, such as feigned lack of understanding and excessive wordiness that obscures arguments. Some informal fallacies could also be classified as social psychological persuasion strategies. They work through our emotions (see Part 3).

Minimize Affective and Motivational Influences

Motivational biases include influence of monetary incentives on decisions including use of misleading performance metrics to maintain agency funding. Affective biases include mood changes, perhaps created by a

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 33 34

difficult interpersonal encounter (Slovic, 2010; Slovic, Finucane, Peters, & MacGregor, 2002). Negative feelings about a client may compromise decisions; they may encourage the fundamental attribution error (focusing on characteristics of the individual and overlooking environmental factors). Labels such as “personality disorder” may have emotional effects that get in the way of making sound decisions. Stress created by noisy offices and work overload may hamper the quality of reasoning. Because we like to please people we like, we may not question their use of unfounded authority to support questionable decisions (Cialdini, 2009). Affective influences based on liking include the “buddy–buddy syndrome” (not criticizing questionable comments by our friends) (Meehl, 1973). People often try to persuade others by offering reasons that appeal to accepted beliefs and values, for example, in progress, health, and happiness. Others may pressure us into maintaining a position by accusing us of being inconsistent, as if we could not (or should not) change our minds (Cialdini, 2009). A marketeer may appeal to scarcity (if we don’t act now, we will lose a valuable opportunity) or to fear (if you do not get screened for breast cancer you will die sooner), when this may not be so (e.g., Welch, Schwartz, Woloshin, 2011).

Minimize Cognitive Biases

Thinking critically can help you to avoid cognitive biases that get in the way of making sound decisions, such as confirmation biases (searching only for data that support a preferred view), assuming that causes are similar to their effects, underestimating the frequency of coincidences (chance occurrences), and premature closure (the tendency to decide prematurely on one option) (Ariely, 2010; Croskerry, 2003; Gambrill, 2012b). There are more than one hundred such biases (see Part 3). You will learn about many in later exercises. As Feynman (1974, p. 4) noted: “The first principle is that you must not fool yourself-and you are the easiest person to fool.” An interest in accurately understanding the views of others and examining the soundness of your own reasoning helps you to minimize biases.

Recognize and Avoid Influence of Propaganda

The purpose of propaganda is not to inform, but to encourage belief and action with little thought (Ellul, 1965; Pratkanis & Aronson, 2001). Much

34 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 35

is distributed by organized groups. Ignorance is actively promoted; it is used strategically (McGoey, 2012). Propagandists discourage questions that would reveal the evidentiary status of claims, such as: Has anyone been harmed by this method? This illustrates the difference between propaganda (encouraging beliefs and actions with the least thought possible) and critical thinking (arriving at well-reasoned beliefs and actions). In the former, strategies such as censoring (not mentioning) alternative well-argued views are used to keep the invisible, invisible. Critical thinking is an antidote to being propagandized by others (or by propagandizing yourself) in ways that decrease opportunities to help clients. Many scholars, although vastly disparate in many or even most of their views, emphasize empowerment through self-education (e.g., Freire, 1973; Popper, 1994; Skinner, 1953). A key part of this self-emancipation through knowledge is the critical appraisal of accepted views. For example, who benefits from emphasizing “self-esteem” as a cause of problems? Who loses? Don’t federal, state, and county governments save millions of dollars by attributing young people’s less-than-hoped-for academic and job performance to their low self-esteem rather than providing education, housing, employment, healthcare, and recreational opportunities that provide the experience on which self-esteem is grounded? Although appeals to self-esteem and willpower may sound informative and as though they give us control over our fate, do they provide guidelines for achieving hoped-for outcomes? (See the critique of self-esteem by Baumeister, Campbell, Krueger, and Vohs [2003].) Key propaganda methods include confusion, distortion, fabrication, and censorship. Examples include misrepresenting disliked positions, presenting opinion as fact, omitting critical information, appealing to emotion and use of slogans, and putdowns. Funding sources may be hidden (Hochman, Hochman, Bor, & McCormick, 2008). Advertisements describing alleged “therapeutic advances” often rely on deceptive methods, such as implied obviousness. Results of studies revealing negative or harmful effects of medication may be hidden. This was the reason for the creation of clinical trial registries in which proposed trials must be clearly described and registered (see AllTrials at www.alltrials.net). Tufte (2007) uses the term “corruption of evidence” to refer to such ploys. People who use propaganda methods may attempt to influence us by creation of fear, such as enticing social workers to buy malpractice insurance by alluding to lawsuits and use of vague innuendos. How many social workers get sued?

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 35 36

Recognize Pseudoscience, Fraud, Corruption, and Quackery

Critical thinking can help you to spot pseudoscience, fraud, corruption, and quackery and to avoid their influence (as discussed earlier). Seek out and scan related websites such as the Center for Evidence Based Psychiatry (http://cepuk.org ), the National Council Against Health Fraud (www.ncahf.org), and Transparency International (www.transparency. org). Seek out blogs related to your work that have a reputation for critical appraisal of claims. Examples include the Carlat Psychiatry Report (www.thecarlatreport.com), (www.Pharmedout.org [http://plos. org/mindthebrain]), and DC’s Improbable Science (www.dcscience.net).

Communicate Effectively

Language is important whether we speak, write, or use tools such as graphics (Tufte, 2007). Effective communication skills include listening carefully to what others say and providing accurate, clear (jargon-free) descriptions of theories and methods. Professionals have a responsibility to write and speak clearly. “Words are of course, the most powerful drug used by mankind” (Kipling, 1923; see also Orwell, 1958). If important terms are not clarified (e.g., “evidence- based practice”), confused discussions may result from the assumption of one word, one meaning. Vague terms that vary in their meaning include abuse, aggression, and addiction. On the other hand, as Popper (1994) suggests, we should never be more precise than we have to be to solve problems. Technical terms are often used carelessly, resulting in “biobabble,” or “psychobabble”—words that sound informative but are not. Such words are often used to give the illusion of scientific (critical) inquiry and profundity when, in reality, they reflect pseudoscience in the guise of science. To the gullible, obscurity heightens the appearance of profoundness. Indeed, Armstrong (1980) found that clear writing was viewed as less profound than obscure writing. Misuse of speculation is common—assuming that what is true can be discovered by merely thinking about it. Using a descriptive term as an explanatory one offers an illusion of understanding without providing any real understanding. A teacher may describe a student as “aggressive.” When asked to clarify what she means, she may say he hits other children. If then asked why she thinks he does this, she may say, “Because he is aggressive.” This is a pseudoexplanation; it goes

36 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 37

round in a circle. The degree to which a “culture of thoughtfulness” exists is reflected in the language used (see Exercise 9).

Identify Mistakes and Errors

Thinking critically about decisions requires attention to errors and mistakes that are inevitable in professional practice. What kinds occur? What are related causes? How can they be minimized? (See Exercise 23.)

Increase Self-Awareness, Including Accurate Estimates of Knowledge and Ignorance

Self-awareness and critical inquiry (reflexivity) go hand in hand. Self- awareness includes recognition of the ways in which you have been and are influenced by your environments, including the society in which you live (see earlier discussion of sociocentric biases) as well as awareness of personal biases (egocentric biases) (Paul, 1993). You may discover beliefs you have accepted that, on reflection, you find problematic. You are more likely to detect contradictions between what you do and what you say you value and to become aware of how your emotions influence your beliefs and actions (Slovic, 2010). Critical thinking requires self-criticism—making inferences explicit and examining them critically. What am I assuming? Can I make a well- reasoned argument for my position? Is there a more sound alternative view? Have I been fooled by misleading claims in advertisements and research reports? Critical thinking encourages recognition of important uncertainties and accurate estimation of ignorance (both personal and objective). Arranging ongoing monitoring of outcome is needed to minimize self-serving biases, such as assuming you have helped a client when you have not (see Exercise 22). Self-reflection includes recognition of self-handicapping strategies, such as not studying for a test so you have an excuse for failure. Complete Box 1.8 to review your approach to critical thinking.

Continue to Learn

A willingness to challenge accepted views and an eagerness to understand alternative views contribute to life-long learning spurred by curiosity and a recognition of our shared vast ignorance (Popper, 1994).

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 37 38

Box 1.8 Are You a Critical Thinker?

Characteristic Rating* SD D N A SA 1. I take responsibility for explaining the reasons for my views. 1 2 3 4 5 2. I evaluate claims of effectiveness critically. 1 2 3 4 5 3. I like to discuss controversial issues with people who disagree 1 2 3 4 5 with me. 4. I often discover that something I believe is incorrect. 1 2 3 4 5 5. I am grateful to people who point out flaws in my thinking. 1 2 3 4 5 6. It is important to examine the accuracy of claims that affect 1 2 3 4 5 clients’ lives. 7. I search for evidence against my assumptions. 1 2 3 4 5 8. It is embarrassing for me to admit that I was wrong or made a 1 2 3 4 5 mistake. 9. Changing one’s mind is a sign of weakness. 1 2 3 4 5 10. People do not respect me if they ask me to support claims I make. 1 2 3 4 5 11. Professionals should base their decisions on well-reasoned 1 2 3 4 5 arguments. 12. Learning from a discussion is more important to me than 1 2 3 4 5 winning. 13. I can spot questionable claims. 1 2 3 4 5 14. I often say, “I could be wrong.” 1 2 3 4 5 15. I take responsibility for evaluating the consequences of actions 1 2 3 4 5 I propose. 16. I seek data that support my point of view only. 1 2 3 4 5 17. I take responsibility for clarifying vague statements I make. 1 2 3 4 5 18. I change my mind when I have good reason to do so. 1 2 3 4 5

SD, strongly disagree; D, disagree; N, neutral; A, Agree; SA, strongly agree.

Source: Gambrill (2013a, p. 114). See, for example, Paul, R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (Rev. 2nd Ed.) Santa Rosa, CA: Foundation for Critical Thinking, pp. 367–368. http://www.criticalthinking.org .

Encourage Needed Advocacy

Many writers highlight the emancipating quality of critical thinking and argue that such reasoning is essential to a democracy (e.g., Baron, 2000; Freire, 1973; Paul, 1993). Brookfield (1987) emphasizes that “one fundamental purpose of encouraging adults to become critical thinkers

38 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 39

is to help them feel a sense of personal connection to wider happenings” (p. 53). Political, social and economic interests in the helping professions often warp priorities and hide avoidable miseries, including oppression and discrimination. A quest for profit at the expense of helping can be seen in residential treatment centers that harm rather than help, privatized jails in which prisoners are mistreated, and nursing homes in which residents are overmedicated. These harms require action to bring them to light and to encourage protest for change. It calls for advocacy. The stark realities that often confront professionals, including limited resources, may result in overlooking opportunities. Related gaps are a compelling reason to draw on critical thinking values and skills as well as the philosophy, process, and tools of evidence-informed practice and policy to help clients, including working with others to reveal and alter unjust conditions and minimize related, avoidable miseries. Thinking critically helps you to recognize obstacles to taking any responsibility for avoidable miseries, including poor-quality service. These may include pressures to conform and to obey authorities, and identifying with authorities, so becoming an authoritarian yourself. Critical thinking helps you avoid the extremes of helplessness, hopelessness, and grandiosity in your professional life. Work together with concerned others to minimize such harms, taking advantage of Internet resources to expose harmful practices, and mobilize other caring people to seek needed changes (see Exercise 4).

Solve More Problems

Thinking critically about problems and related factors should enable you to help clients solve more problems as well as identify those you cannot solve. You may have to work with others over years, even decades, to identify, describe, expose, and minimize avoidable harms such as discrimination and oppression. Many await your help. Many organizations share concerns about inequities, such as the Center for Investigative Journalism and Public Citizen.

Critical Thinking: Integral to Evidence-Informed Practice and Policy

A key choice is how to view EBP (Gambrill, 2006): 1. The process of EBP described in original sources 2. The EBPs approach

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 39 40

3. The propaganda approach Choices have implications, not only for clients, practitioners, and administrators, but also for researchers and educators. (Some people prefer the term evidence-informed practice [Chalmers, 2003]. I use both terms interchangeably in this book.) Misrepresentations of the process and philosophy of EBP emphasize the importance of reading original sources for yourself (e.g., Straus, Glasziou, Richardson, & Haynes, 2011). Criticism of ideas is vital, but should be based on accurate descriptions. Common misrepresentations of the process of EBP include the following: • Ignores client values and preferences • Ignores clients’ unique circumstances and characteristics • Ignores clinical expertise • Considers randomized controlled trials only • Saves money • Cannot be done • Uses a cookbook approach (Gibbs & Gambrill, 2002)

The Process of Evidence-Informed Practice

The process of evidence-based practice EBP, as described by its originators, is a way to handle the inevitable uncertainty in making decisions in an informed, ethical manner, attending to ignorance as well as knowledge. It is designed to decrease the gaps between research and practice to maximize opportunities to help clients attain outcomes they value and to avoid harm (Gray, 2001a, 2001b; Straus et al., 2011). It is hoped that professionals who consider research findings together with other vital information will provide more effective, ethical services than those who rely on criteria such as anecdotal experience or popularity. (For examples of improvement in clinical outcomes via EBP, see testingtreatments. org.) Critical thinking values, skills, and knowledge are integral to EBP (Gambrill, 2012b, 2013a; Jenicek & Hitchcock, 2005). Professionals often need information to make practice and policy decisions, for example, about which services are most likely to help a client attain hoped-for outcomes. Ethical obligations require practitioners to involve clients as informed participants concerning the costs and benefits of recommended services and alternatives. In their discussion of EBP, Guyatt and Rennie (2002) include obligations of professionals to advocate for changes in environmental conditions that contribute to problems.

40 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 41

EBP as described by its originators involves “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual [clients]” (Sackett, Richardson, Rosenberg, & Haynes, 1997, p. 2). It requires “the integration of the best research evidence with our clinical expertise and our [client’s] unique values and circumstances” (Straus et al., 2011) “Best evidence” refers to clinically relevant research. (There may be none, in which case well-argued theory must be drawn on as a guide.) Clinical expertise refers to use of practice skills, including effective decision making and relationship skills, and past experience to rapidly identify each clients unique circumstances and characteristics, including their preferences and expectations, and “their individual risks and benefits of potential interventions” (Straus et al., 2011, p. 1). It includes knowledge of relevant theory. Clinical expertise is drawn on to integrate information from varied sources (Haynes, Devereaux, & Guyatt, 2002) ,including information about resources (Health Sciences Library [hsl.mcmaster.libguides.com]). Without clinical expertise, practice risks becoming tyrannized by external evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual [client]. Without current best external evidence, practice risks becoming rapidly out of date, to the detriment of [clients]. (Sackett et al., 1997, p. 2)

Client values refer to “the unique preferences, concerns and expectations each [client] brings to a clinical encounter and which must be integrated into clinical decisions if they are to serve the [client]” (Straus et al., 2011, p. 1). Evidence-informed practice arose as an alternative to authority- based practice, in which decisions are based on criteria such as consensus, anecdotal experience, and tradition. EBP describes a philosophy as well as an evolving process designed to forward effective use of professional judgment in integrating information about each client’s unique characteristics, circumstances, preferences, and actions with external research findings. “It is a guide for thinking about how decisions should be made” (Haynes et al., 2002). Although the philosophical roots of EBP are old, its blooming as an evolving process attending to evidentiary, ethical, and applicable issues is fairly recent, facilitated by the Internet. The process of EBP requires drawing on research findings related to important questions and sharing what is found (including nothing) within a supportive relationship with clients. It involves a search not

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 41 42

only for knowledge, but also for ignorance. Such a search is required to involve clients as informed participants (e.g., to identity uncertainties related to decisions). When little or no research is available regarding a concern, well-argued theory is drawn on, which should be informed by empirical research—for example, about behavior and/or physiology. Client values and expectations are vital to consider (see description of Step 4 in the process of evidence-based practice). The process of EBP highlights the uncertainties involved in making decisions and offers tools to handle them constructively and ethically—for example, by locating and appraising critically research related to decisions, taking advantage of technologies such as systematic reviews (see Exercise 25). Uncertainties include the relevance of research to individual clients, client characteristics and circumstances that may influence outcome, and resources available. Steps in EBP include the following: Step 1: Convert information needs related to decisions into well- structured questions. Step 2: Track down the best evidence with which to answer these questions. Step 3: Appraise that evidence critically for its validity (closeness to the truth), impact (size of the effect), and applicability (usefulness in our clinical practice). Step 4: Integrate the critical appraisal with our clinical expertise and with our [clients’] unique characteristics, including their values and circumstances (e.g., Is a client similar to those studied? Is there access to services needed?). Step 5: Evaluate our effectiveness and efficiency in executing Steps 1 to 4 and seek ways to improve them for next time (p. 4)” (Straus et al., 2011, p. 3).

Reasons for the Creation of the Evidence-Based Practice Process

A key reason for the creation of the process of EBP was the discovery of gaps showing that professionals were not acting systematically or promptly on research findings. There was increased recognition of harming in the name of helping. There was a failure to start services that work and to stop services that did not work or harmed clients (Gray, 2001a, 2001b). There was increasing attention to variations in treatment for similar problems across regions (Wennberg, 2002). There still are wide variations in practices, including rates of cesarean births and hysterectomies, for

42 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 43

example (e.g., Brownlee et al., 2011; McPhearson, Gon, & Scott, 2013; Wennberg & Thomson, 2011). Children are prescribed medication for (mis)behavior at far higher rates in the United States compared with France (Cohen, 2012). Children in foster care in the United States are prescribed higher rates of psychotropic medication compared with other children (US Government Accountability Office, 2011, 2014). The Wennberg International Collaborative (www.wennberg.collaborative) tracks variations in medical practices. We should have similar sites in other professions. Economic concerns (e.g., money spent on ineffective and/or harmful services) were another concern in creating the process of EBP. Inventions in technology were key to the origins of EBP, such as the Web revolution that allows quick access to databases and preparation of meta-analyses and systematic reviews (research syntheses) that, if done well, make it easier to discover the evidentiary status of interventions and claims about causes. The Cochrane and Campbell Databases provide systematic reviews regarding thousands of questions; examples include antidepressants for treatment of depression in people with cancer, social skills programs for schizophrenia, and exercise programs for people with dementia. Increased recognition of the flawed nature of traditional means of knowledge dissemination such as texts, editorials, and peer review was another factor. Revelations of the flawed nature of the peer-reviewed literature continue, as discussed earlier.

The Evidence-Based Practices (EBPs) Approach

The most popular view is the EBPs (evidence-based practices)approach in which some authority recommends or mandates use of certain practice guidelines and/or manuals. Many people confuse the process of EBP as described in original sources with the EBPs approach. (If someone uses the term evidence-based practice or policy, find out how she is using this term.) Websites such as the National Institute for Health and Care Excellence (NICE), the California Evidence Based Clearinghouse for Child Welfare (www.cebc4cw.org), and the Social Care Institute for Excellence (SCIE) (www.scie.org.uk) include lists of what are described as “evidence-based practices.” Are they sound? Usually, making decisions about individual clients is much more complex. There are many other considerations, such as the need to consider the unique circumstances and characteristics of each client, as illustrated by critiques of practice guidelines and manualized treatments (e.g., Norcross, Beutler, & Levant,

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 43 44

2006). “The leading figures in EBM [evidence-based medicine] … emphasized that clinicians had to use their scientific training and their judgment to interpret [guidelines] and individualize care accordingly” (Gray, 2001b, p. 26). Promotion of EBPs is often done in an uncritical manner; that is, related research may be deeply flawed and not warrant promotion. And, research in psychotherapy, suggests that, “no one treatment is clearly more effective than another” (Wampold & Imel, 2015a, p. 272)—common factors such as the alliance, warmth, and empathy have a greater impact on outcome than specific interventions.

The Propaganda Approach

Many descriptions of EBP in the literature could be termed “business as usual,” inflating claims of effectiveness and hiding harming in the name of helping. The old is relabeled as the new (as an “evidence-based” practice or policy); the term is used without the substance (e.g., uncritical reviews are labeled as evidence based [Littell, 2008]). Given the clash with authority-based practice, it is not surprising that the original vision of the process of EBP, which highlights ignorance and uncertainty, is so often misrepresented (Gambrill, 2010; Gibbs & Gambrill, 2002). Misrepresenting new ideas saves time in understanding and describing them accurately, and allows marketing in the guise of scholarship to continue.

Helpful Distinctions

The following distinctions are valuable in helping clients and avoiding harming them.

Reasoning/Truth

Reasoning does not necessarily yield the truth. “People who are considered by many of their peers to be reasonable people often do take, and are able to defend quite convincingly, diametrically opposing positions on controversial matters” (Nickerson, 1986, p. 12). Also, the accuracy of a conclusion does not necessarily indicate that the reasoning used to reach it was sound. Lack of evidence for a claim does not mean that it is incorrect. Similarly, surviving critical tests does not

44 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 45

mean a claim is true. Further tests may show it is false. Thus, it is best to avoid words and phrases such as proven and well-established, which convey an unwarranted certainty.

Justifying/Falsifying

Many people focus on gathering support for (justifying) claims, theories, and arguments. This is the source of confirmation biases described in later exercises. Let’s say you see 3000 swans and they are all white. Does this mean that all swans are white? Can you generalize from the particular (seeing 3000 swans, all of which are white) to the general (“All swans are white.”)? Karl Popper argues that we cannot discover what is true by induction (generalizing from the particular to the general) because we may later discover exceptions. In fact, black swans are found in some parts of the world. Rationalizing involves a selective search for evidence in support of a belief or action. This selective search may occur without awareness or deliberately. When we rationalize, we focus on building a case rather than weighing evidence for and against an argument. Popper (1972, 1994) argues that falsification (attempts to falsify, to discover the errors in our beliefs by critical testing of claims—by criticism) is the only sound way to develop knowledge. By critically testing our guesses, we discover errors and, if we are lucky, learn from them to make more informed guesses in the future, which, in turn, should be criticized. Reasoning involves reviewing both the evidence against and in favor of a position. Miller (2005) argues:

[M]‌uch of our thinking consists of exploratory problem solving that is not constrained by rules, and is not argumentative. Faced with a problem, we generate guesses … hoping that one may offer some kind of solution. Then we use reasoning or calculation, again more or less blindly, to find out where our guesses break down. (pp. 58–59)

Good Intentions and Good Outcomes

Good intentions do not ensure good results. Many publications document the harmful effects of efforts intended to help clients (e.g., Gøtzsche,

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 45 46

2013, 2015a & b; Jacobson et al., 2005; Scull, 2015; Welch et al., 2011). Approximately 10,000 babies were blinded as a result of giving oxygen at birth, resulting in retrolental fibroplasia (Silverman, 1980). No one cared enough to critically test whether this treatment did more harm than good. Medical errors are the third leading cause of death in the United States (James, 2013). Gøtzsche (2015a) argues that psychotropic mediation kills more than 500,000 people 65 years and older each year, with millions more disabled. Moncrieff and Cohen (2006) argue that medication prescribed to alter abnormal brain states assumed to be related to “mental illness” may create such states. Intensive social casework offered to a sample of frail elderly individuals in the Cleveland area increased mortality (Blenkner, Bloom, & Nielsen, 1971).

Widely Accepted/True

What is widely accepted may not be true. Consider the following exchange:

• Ms. Simmons (psychiatrist): I’ve referred this client to the adolescent stress service because this agency is widely used. • Ms. Harris (supervisor): How effective is this agency in helping adolescents like your client? • Ms. Simmons: They receive more referrals than any other agency for these kinds of problems. We’re lucky if they accept my client.

Many people believe in the influence of astrological signs (their causal role is widely accepted). However, to date, risky predictions based on related beliefs have not survived critical tests. Can you think of other beliefs that are widely accepted but not true?

A Feeling That Something Is True Versus Whether It Is True

People often use their “feeling” that something is true as a criterion to accept or reject possible causes. A “feeling” that something is true may not (and often does not) correspond with what is true. Not making this distinction helps to account for the widespread belief in many questionable causes. Basing actions and beliefs on feelings discourages an examination of their soundness and, in professional contexts, this may result in decisions that do not benefit clients.

46 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 47

Knowledge and Ignorance: Not Mirror Images

Personal knowledge refers to what you as an individual believe you “know.” Objective knowledge refers to assumptions that have survived critical tests or evaluation. It is public; it can be criticized by others. Personal and objective ignorance may overlap to different degrees. Knowledge of your own ignorance is a vital kind of personal knowledge that may be compromised by self-censorship (e.g., avoiding contradictory views). We typically overestimate what “we know”; that is, our self-assessments of our “knowledge” and skills are usually inflated (Dunning, Heath, & Suls, 2004). We tend to be overconfident of our views (Kahneman, 2011). Agnatology refers to the study of socially constructed ignorance. Ignorance may be avoidable or unavoidable. It may matter or not matter. Proctor and Schliebinger (2008) argue that the study of ignorance is just as important as the study of knowledge. Ignorance is often created deliberately; it serves a strategic purpose (McGoey, 2012). (See earlier discussion of corruption and fraud.) Roberts and Armitage (2008) use the term ignorance economy to refer to such activity and its consequences. There are many things people do not want you to know, such as the results of negative trials in drug studies and hidden changes in endpoints—the moving goal post (e.g., Gøtzsche, 2013, 2015a; Whitaker & Cosgrove, 2015). Collective avoiding (denial) of information is common (Norgaard, 2006, 2011). Some topics may be taboo. Gaudet (2013) argues that researchers value, produce actively, and thereby mobilize ignorance. Creation of doubt about the harms of smoking tobacco was a key strategy of the part of the tobacco companies (Oreskes & Conway, 2010).

Truth and Credibility

Karl Popper (1994) defines truthful statements as those that correspond to the facts. Credible statements are those that are possible to believe. Phillips (1992) points out that just about anything may be credible; this does not mean it is true. Simply because it is possible to believe something does not mean it is true. History often shows that what once seemed credible was false (e.g., the belief that tuberculosis was inherited) and what once seemed incredible was true (e.g., people could fly in airplanes). Accounts are often accepted when they “make sense,” although there is no evidence they are accurate. Only by critically appraising beliefs can we evaluate their soundness. Although

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 47 48

scientists seek accurate answers to questions/problems (statements that correspond to the facts), this does not mean that there is certain knowledge. Rather, certain beliefs (theories) have (so far) survived critical tests or have not yet been exposed to them. An error consists in “our regarding as true a theory that is not true” (Popper, 1992, p. 4). We can avoid error or detect it by doing “everything possible to discover and eliminate falsehoods” (p. 4).

Knowing and the Illusion of Knowing (Overconfidence)

There is a difference between accurately understanding content and the illusion of knowing—“a belief that comprehension has been attained when in fact, comprehension has failed” (Zechmeister & Johnson, 1992, p. 151). The illusion of understanding and validity gets in the way of taking remedial steps because you think “you know” when you do not (e.g., Kahneman, 2011). There is a failure of comprehension without the realization that this has occurred. The illusion of knowing is encouraged by mindless reading—for example, not reading material carefully and failing to monitor comprehension by asking questions such as: Do I understand this? What is this person claiming? What are his reasons? There is a failure to detect contradictions and unsupported claims (see discussion of uncritical documentation in Exercise 11). Redundant information may be collected, creating a false sense of accuracy, such as giving a client different standardized tests of depression (all self- report data) (Hall, Ariss, & Todorov, 2007). The illusion of knowing may be encouraged by a feeling of familiarity concerning a claim and by thinking in terms of absolutes (e.g., proven, well-established) rather than conditionally (e.g., “This may be … ”; “This could be … .”) (Zechmeister & Johnson, 1992). Claims may appeal to generally accepted but incorrect beliefs about causes of a problem such as depression (see Exercises 7 and 29).

What to Think and How to Think

Critics of the educational system argue that students are often told what to think and do not learn how to think. Thinking deeply about any subject requires criticism of what is considered. This is quite different from memorizing a list of alleged facts. Exploring the accuracy of claims requires thinking critically about them.

48 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 49

Intuitive and Analytic Thinking

Intuition (our “gut” reaction) involves a quick judgment. It may refer to looking back in time (interpreting experience) or forward in time (predictions). Intuitive judgments are often based on heuristics (simple rules-of-thumb). “Imitate the successful” is one heuristic suggested by Gigerenzer (2008): “Look for the most successful person and imitate his or her behavior” (p. 24). We make what Gigerenzer calls, a “fast and frugal decision.” It is rapid (fast) and relies on key environmental cues (it is frugal). We ignore irrelevant data; we do not engage in calculations such as balancing pros and cons. Gigerenzer (2008) suggests that we select a heuristic based on reinforcement learning. As he notes, logic may not be of help in many situations, and that it is correspondence with a certain environment that matters (p. 25). Such judgments are often superior to calculating pros and cons, but not always. The view that intuition involves responsiveness to information that, although not represented consciously, yields productive insights, is compatible with the research regarding expertise (Klein, 1998). No longer remembering where we learned something encourages attributing solutions to “intuition.” When an expert is asked what made her think a particular method would be effective, her answer may be, “My intuition.” When asked to elaborate, she may offer sound reasons reflecting related, extensive experience providing corrective feedback. That is, her “hunch” is an informed one. Dual process models encourage us to use our analytic skills to make best use of intuition (Croskerry, Petrie, Reilly, & Tait, 2014; Kahneman, 2011); we use controlled thinking as a check on automatic thinking. When “our gut reaction” is based on correct cues, it serves us well. When it is not (when, in Hogarth’s (2001) term, it is not an “informed intuition” based on multiple opportunities for corrective feedback), it is best to use a more analytic approach to making decisions. Intuition is not a sound guide for making decisions when misleading cues are focused on. Research comparing clinical judgments with those based on empirical relationships between variables and an outcome, such as future child abuse, shows the superior accuracy of the latter (Vaithianathan, Maloney, Putnam-Hornstein, & Jiang, 2013). Intuition cannot show which method is most effective in helping clients; a different kind of evidence is required for this, one that provides comparisons controlling for biases. Attributing judgments to “intuition” decreases opportunities to teach others. One has “it” but doesn’t know

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 49 50

how or why “it” works. If you ask your supervisor, “How did you know to do that at that time?” and she replies, “My intuition,” this will not help you to learn what to do.

Bias/Point of View (Guesses About What May be True or False)

People with a point of view describe their sources, state their views clearly, and avoid propaganda tactics (MacLean, 1981). They encourage rather than discourage critical appraisal. Bias refers to an emotional leaning to one side. Biased people may or may not be aware of their biases. Propagandists encourage biased views, leanings to one side. They exaggerate positive aspects of their views and hide or minimize negative ones, and exaggerate negative aspects of disliked views and hide or minimize positive ones (Rank, 1984a, 1984b). They play on our emotions. They may appeal to our fears to sell products and gain uncritical acceptance of a position. They present only one side of an argument, and hide and attack the motives of critics rather than respond to critical questions. For example, it may be assumed that anyone who raises questions about the effectiveness of services for battered women must be trying to undermine efforts to help women.

Thinking, Reasoning, and Persuasion

Both reasoning and persuasion strategies, such as appeal to scarcity (e.g., a claim that an offer is only available for one day), are used to encourage people to act or think in a certain way. Miller (2005) notes that critical thinking texts usually encourage use of sound reasoning skills to persuade others of the “truth” of a claim. He argues that such a position encourages dogmatism—that “a concern with persuasion impedes pursuit of truth” (p. 66) (see the discussion of different goals of arguments in Exercise 30).

Consistency, Corroboration, and Proof

People often use agreement among different sources of data, to justify their beliefs. For example, they may say that Mrs. X is depressed currently because she has a prior history of depression. However, saying that A (a history of “depression”) is consistent with B (alleged current “depression”) is to say only that it is possible to believe B given A. Two

50 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 51

or more assertions may be consistent with each other but yield little or no insight into the accuracy of a claim. Proof implies certainty about a claim, as in the statement, “The effectiveness of case management services to the frail elderly has been proved in this study.” Because future tests may show a claim to be incorrect, even one that has been tested critically, no assertion can ever be “proved” (Popper, 1972). Yet, this term is used often, even in the peer-reviewed literature. If nothing can ever be proved, we can at least construct theories that are falsifiable; specific hypotheses can be tested critically. The “Great Randi” has offered $1 million to anyone who can demonstrate parapsychology effects (such as psychic predictions) via a controlled test. So far, no one has won the prize.

Beliefs, Opinions, Preferences, and Facts

Beliefs and opinions are assumptions about what is true or false. They may be testable (e.g., support groups help the bereaved) or untestable (God exists). They may be held as convictions (be unquestioned) or as guesses, which we test critically. Popper (1972) suggests that facts refer to well-tested data evaluated intersubjectively. The data can be contrasted with “factoids”—claims with no related evidence, but which are believed because they are repeated often. Results of a study may be referred to in many different sources until they achieve the status of a law without any additional data being gathered. Richard Gelles (1982) calls this the “Woozle Effect” (p. 13). In science, it is assumed that the accuracy of an assertion is related to the uniqueness and accuracy of related critical appraisals. Facts can be checked (e.g., shown they are not true); beliefs may not be testable. Preferences reflect values. It does not make sense to consider preferences as true or false, because people differ in their preferences, as in the statement, “I prefer insight-oriented treatment.” This is quite different than the assertion: “Play therapy can help children overcome anxiety.” With regard to the latter statement, evidence can be gathered to determine whether it is accurate. Other examples of preferences and beliefs follow. The first is a preference; the last two are beliefs.

• I like to collect payment for each session at the end of the session. • Insight therapy is more effective than cognitive–behavioral treatment of depression.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 51 52

• My pet Rottweiler helps people with their problems (quote from a psychologist on a morning talk show, April 6, 1988). We can ask people what their preferences are, and some ways of exploring this are more accurate than others.

Science and Scientific Criteria

The corruption of science by special interests highlights the importance of understanding what science is and what it is not. The study of the social dimensions of scientific knowledge includes “the effects of scientific research on human life and social relations, the effects of social relations and values on scientific research, and the social aspects of inquiry itself” (Zalta, 2002, p. 1; see also Latour, 1987). With an understanding of science, you are less likely to be a patsy for bogus claims in the scientific literature. It will help you to avoid scientism—adherence to the methods of science when they are not appropriate (e.g., Phillips, 1987, p. 206). If you do not understand science, its social as well as knowledge functions and its history, you are likely to make the following errors: 1. Assume science can discover final answers and so make and act on inflated claims of knowledge that may harm clients. 2. Assume there is no way to discover what may be true and what may be false because scientists make errors and have biases. 3. Assume prematurely that those who question popular views—for example, about mental illness, prescribed medication, or screening— are crackpots (e.g., Boyle, 2002; Gøtzsche, 2012; 2015a; Welch et al., 2011). 4. Throw out the baby (science) with the bathwater (pseudoscience and scientism). A search for the truth, wherever this may lead, is hampered by religious, financial, political, and social influences. Pressure to publish to gain and maintain status and income encourages publication of exaggerated claims as discussed earlier (Ioannidis, 2012). The publication and dissemination of flawed research and related fraud and corruption has reached alarming levels. As “big science” has become more common (research institutes jockeying for limited research funds, and collaboration between industry and universities increasing), secrecy

52 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 53

and resistance to competing new ideas becomes more likely. Political correctness (censorship of certain topics and the castigation of those who raise questions) stifles inquiry. The ever-increasing corporate nature of universities may result in squelching researchers who pursue questions viewed as damaging the claimed “brand” of a university. Consider reactions from university administrators against Paul Frijters (Mujcic & Frijters, 2013) who reported that local bus company drivers discriminate against people of color (Robertson, 2015). The bus company complained to the university and the university tried to fire him. The university spokesman said the study reflected negatively on their brand as a community-friendly institution. Frijters sued, spending $50,000 of his own money and won. Langmuir (1989) used the term pathological science to refer to actions counter to open inquiry (activities that have little to do with science as open investigation). This includes pseudoscience and scientism (see earlier discussion).

What Is Science?

The essence of science is creative, bold guessing and rigorous testing in a way that offers accurate information about whether a guess (conjecture or theory) is accurate (Asimov, 1989; Feynman, 1969; Sagan, 1990, 1997). Science is a way of thinking about and investigating the accuracy of assumptions about the world. It is a way of “learning how not to fool ourselves” (Feynman, 1974, p. 4). As Feynman (1974) emphasizes, “in science there is a kind of utter honesty… . for example if you are doing an experiment, you should report everything that you think might make it invalid” (p. 3). Popper (1972) suggests it is a process for solving problems in which we learn from our mistakes. My whole view of scientific method may be summed up by saying that it consists of these four steps:

1. We select some problem—perhaps by stumbling over it. 2. We try to solve it by proposing a theory as a tentative solution. 3. Through the critical discussion of our theories our knowledge grows by the elimination of some of our errors, and in this way we learn to understand our problems, and our theories, and the need for new solutions. 4. The critical discussion of even our best theories always reveals new problems.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 53 54

Or to put these four steps into four words: problems— theories—criticisms— new problems. Of these four all-important categories the one which is most characteristic of science is that of error-elimination through criticism. For what we vaguely call the objectivity of science and the rationality of science, are merely aspects of the critical discussion of scientific theories. (Popper, 1994, pp. 158–159)

Both critical thinking and scientific reasoning provide a way of thinking about and testing assumptions of value to those in the helping professions. Both rely on standards that encourage us to challenge assumptions, attend to context, consider opposing points of view, be clear, and check for errors. Both encourage doubt of experts. Science rejects a reliance on authority—for example, pronouncements by officials or professors—as a route to knowledge. Authority and science are clashing views of how knowledge can be gained. The history of science and medicine shows that new ideas and the results of critically testing them often frees us from false beliefs and results in discoveries. Consider these examples:

• The cause of ulcers was found to be Helicobacter pylori, not stress or spicy foods (Marshall & Warren, 1984; Van der Weyden, Armstrong, & Gregory, 2005). • It was proclaimed that we are born with all the brain cells we will ever have. Research shows that we produce hundreds of new brain cells each day. Discovering what is true and what is false often requires ingenious experiments and the invention of new technologies such as the microscope and the long-range telescope. Consider the experiment conducted by a twelve-year-old to test the effectiveness of “therapeutic touch” (Rosa, Rosa, Sarner, & Barrett, 1998). All methods are vulnerable to error, including qualitative methods needed to explore many kinds of questions. Nonexperimental approaches include natural observation (the study of animal behavior in real-life settings), and correlational methods that use statistical analysis to investigate the degree to which events are associated. These methods are of value in suggesting promising experiments as well as when events of interest cannot be altered experimentally, or if doing so would destroy what is under investigation.

54 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 55

The view of science presented here, critical rationalism, is one in which the theory-laden nature of observation is assumed (i.e., our assumptions influence what we observe) and criticism of our assumptions is viewed as vital (Phillips, 1992; Popper, 1972). “There is no pure, disinterested, theory-free observation” (Popper, 1994, p. 8). “What we call scientific objectivity is nothing else than the fact that no scientific theory is accepted as dogma, and that all theories are tentative and are open all the time to severe criticism” (Popper, 1994, p. 160). Objectivity implies that the results of science are independent of any one scientist so that different people exploring the same problem will reach the same conclusions. Concepts are assumed to have meaning and value even though they are unobservable. By testing our guesses, we eliminate false theories and may learn a bit more about our problems; corrective feedback from the physical world allows us to test our guesses about what is true or false. It is assumed that nothing is ever “proved” (Miller, 1994; Popper, 1972). Science is conservative in insisting that a new theory account for previous findings. It is revolutionary in calling for the overthrow of previous theories shown to be false, but this does not mean the new theory has been “established” as true. Although the purpose of science is to seek true answers to problems (statements that correspond to facts), this does not mean we can have certain knowledge. Rather, we may say that certain beliefs (theories) have (so far) survived critical tests or have not yet been exposed to them. And, some theories have been found to be false. This humble approach is not reflected in the exaggerated claims of knowledge in the peer-reviewed literature. The interplay between theories (conjectures) and their testing (refutations) is central to science. Two different proposed theories for an event cannot both be true.

It most important to see that a critical discussion always deals with more than one theory at a time. For in trying to assess the merits or demerits even of one theory, it always must try to judge whether the theory in question is an advance: whether it explains things which we have been unable to explain so far—that is to say, with the help of older theories. (Popper, 1994, p. 160)

Some claims are testable but untested. If tested, they may be found to be true, false, or uncertain (Bunge, 1984). Consider the question: How many

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 55 56

teeth are in a horse’s mouth? You could speculate about this or you could open a horse’s mouth, look inside, and count the teeth. If an agency for the homeless claims it succeeds in finding homes for applicants within 10 days, you could accept this claim at face value or systematically gather data to determine whether this claim is true. Scientists are often wrong and find out they are wrong by testing their predictions. Popper argues, “The growth of knowledge, and especially of scientific knowledge, consists of learning from our mistakes” (1994, p. 93). The scientific tradition is “a tradition of criticism” (Popper, 1994, p. 42). Popper considers the critical method to be one of the great Greek inventions. “I hold that orthodoxy is the death of knowledge, since the growth of knowledge depends entirely on the existence of disagreement” (Popper, 1994, p. 34). For example, an assumption that verbal instructions can help people to decrease their smoking could be tested by randomly assigning smokers to an experimental group (receiving such instructions) and a control group (not receiving instructions), and observing their behavior to see what happens. There is a comparison. Let’s say you think you will learn some specific skills in a class you are taking. You could assess your skills before and after the class and determine whether your skills have increased. Testing your belief offers more information than simply thinking about it. What if you find your skills have increased? Does this show the class was responsible for your new skills? It does not. There was no comparison (e.g., with students who did not take the class). There are other possible causes (rival hypotheses). For example, maybe you learned these skills in some other context. Popper maintains that attempts to falsify, to discover the errors in our beliefs by means of critical discussion and testing, is the only sound way to develop knowledge (Popper, 1992, 1994). (For critiques of Popper’s views, see, for example, Schilpp [1974].) Some theories are not testable (falsifiable). There is no way to test them to find out whether they are correct. As Popper points out, irrefutability is not a virtue of a theory, but a vice. Theories can be tested only if specific predictions are made about what can happen and also about what cannot happen. Confirmations of a theory can readily be found if one looks for them. Although we can justify the selection of a theory by its having survived more risky tests concerning a wider variety of hypotheses, compared with other theories that have not been tested or that have not been falsified, we can never accurately claim that this theory is “the truth.” Further tests may show otherwise.

56 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 57

Some Tests Are More Rigorous Than Others

Some tests are more rigorous than others and so offer more information about what may be true or false. Compared with anecdotal reports, experimental tests are more severe tests of claims. Unlike anecdotal reports, they are carefully designed to rule out alternative hypotheses, such as the effects of maturation, history, or testing (Campbell & Stanley, 1963) and so provide more opportunities to discover that a theory is not correct. Every research method is limited in the kinds of questions it can address successfully. The question will suggest the research method required to explore it. Thus, if our purpose is to communicate the emotional complexity of a certain kind of experience (e.g., the death of an infant), then qualitative methods are needed (e.g., detailed case examples, thematic analyses of journal entries, open-ended interviews at different times).

A Search for Patterns and Regularities

It is assumed the universe has some degree of order and consistency. This does not mean unexplained phenomena or chance variations do not occur or are not considered. For example, chance variations contribute to evolutionary changes. Uncertainty is assumed. Because a future test may show an assumption to be incorrect, even one that is strongly corroborated (has survived many critical tests), no assertion can ever be “proved.” This does not mean all beliefs are equally sound; some have survived more rigorous tests than others (Asimov, 1989). In the physical sciences, there is a consensus about many of the phenomena that need to be explained, and some degree of consensus about explanations, as Bauer (2001) notes. This consensus does not mean a theory is true; it may be overthrown by one that accounts for more events and makes more accurate predictions. There are scores of different theories in the social sciences. They cannot all be correct, yet in the social sciences and helping professions, theories are often claimed to be true with excessive confidence.

Parsimony

An explanation is parsimonious if all or most of its components are necessary to explain most of its related phenomena. Unnecessarily complex explanations may get in the way of detecting relationships

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 57 58

between behaviors and related events. Consider the following two accounts: 1. Mrs. Lancer punishes her child because of her own unresolved superego issues related to early childhood trauma. This creates a negative disposition to dislike her oldest child. 2. Mrs. Lancer hits her child because this temporarily removes the child’s annoying behaviors (the child stops yelling) and because she does not have positive parenting skills (e.g., she does not know how to identify and reinforce desired behaviors). The second account suggests specific behaviors that could be altered. It is not clear that concepts such as “unresolved superego issues” and “negative disposition” yield specific guidelines for altering complaints.

A Skeptical Attitude

Scientists are skeptics. They question what others view as fact or “common sense.” They questions claims (e.g., Caroll, 2003). They do not have sacred cows. Science … is a way of thinking … . [It] invites us to let the facts in, even when they don’t conform to our preconceptions. It counsels us to consider hypotheses in our heads and see which ones best match the facts. It urges on us a fine balance between no-holds-bared openness to new ideas, however heretical, and the most rigorous skeptical scrutiny of everything—new ideas and established wisdom (Sagan, 1990b, p. 265).

Scientists and skeptics seek criticism of their views and change their beliefs when they have good reason to do so. Skeptics are more interested in arriving at accurate answers than in not ruffling the feathers of supervisors or administrators. They value critical discussion because it can reveal flaws in their own thinking. Karl Popper considers criticism the mark of rationality. Scientists question what others view as facts or “common sense.” They ask: How good is the evidence? This is why a scientific attitude is dangerous to those in power. Skepticism does not imply cynicism (being negative about everything). Scientists change their beliefs if additional evidence demands it. If they do not, they appeal to science as a religion—as a matter of authority and faith—rather than a way to critically test theories.

58 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 59

Other Characteristics

Science deals with specific problems that can be solved (that can be answered with the available methods of empirical inquiry)—for example, is the use of medication to decrease depression in elderly people more (or less) effective than cognitive–behavioral methods? Examples of unsolvable questions are: Is there a God? Do we have a soul? Saying that science deals with problems that can be solved does not mean other kinds of questions are unimportant or that a problem will remain unsolvable. New methods may be developed that yield answers to questions previously unapproachable in a systematic way. Science is collective. Scientists communicate with one another; they criticize each other’s ideas and data, and the results of one study inform the efforts of other scientists. The social uses of science may result in inflated claims and other misleading material.

Science and Normal Science

New ideas and related empirical evidence often show that currently accepted theories are not correct. However, as Kuhn (1970) argued, old paradigms may continue to be accepted uncritically until sufficient contradictions (anomalies) force recognition of the new theory. Spirited disagreements about evolution continue (see publications of the National Science Education Center [www.ncse.com]). The history of science shows that new ideas are often censored, and those proposing them may have difficulty getting a hearing in scientific journals and the media. Bell and Linn (2002) note that textbooks often omit controversy and personality, giving “an incorrect illusion of a logical progression of uncomplex discovery when indeed the history is quite different: serendipitous, personality-filled, conjectural, and controversial” (p. 324) (see also Latour [1987]). Prestigious journals typically rejected the work of scientists who made major discoveries and overturned prevailing beliefs (Barber, 1961; Companario & Acedo, 2013). Entrenched views may result in an inability even to conceive of radical new discoveries such as the existence of germs (Semmelweis, 1983). Kuhn argued that most investigators work within accepted (and often wrong) accepted paradigms. They do “normal science.” [T]‌he “normal” scientist, as Kuhn describes him, is a person one ought to be sorry for … . The “normal” scientist, in my

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 59 60

view, has been taught badly. I believe, and so do many others, that all teaching on the University level (and if possible below) should be training and encouragement in critical thinking. The “normal” scientist, as described by Kuhn, has been badly taught. He has been taught in a dogmatic spirit: he is a victim of indoctrination. He has learned a technique which can be applied without asking for the reason why … . As a consequence, he has become what may be called an applied scientist, in contradistinction to what I should call a pure scientist. He is, as Kuhn puts it, content to solve “puzzles.” (quoted in Notturno, 2000, p. 237; Popper, 1970)

Commenting on Kuhn’s notion of “normal science”—its concrete institutional embodiment—Popper (1970) wrote: “Normal” science, in Kuhn’s sense, exists. It is the activity of the non-revolutionary, or more precisely, the not-too-critical professional: of the science student who accepts the ruling dogma of the day; who does not wish to challenge it; and who accepts a new revolutionary theory only if almost everybody else is ready to accept it—if it becomes fashionable by a kind of bandwagon effect. To resist a new fashion needs perhaps as much courage as was needed to bring it about. (p. 52)

Consider promotion of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013) despite reliability and validity problems of this classification system (e.g., Gambrill, 2014a; Kirk et al., 2013). Consider also promotion of the use of psychotropic medicine by psychiatrists and other helping professionals, ignoring harms including life-long dependence and death. Gøtzsche (2015a & b) argues that Zoloft alone has resulted in the death of 200,000 people. Great clashes have, do, and will occur in science.

Misunderstandings and Misrepresentations of Science

Surveys show many people do not understand the basic characteristics of science (National Science Foundation, 2006). Misunderstandings and misrepresentations of science are so common that Dennis C. Phillips, a philosopher of science, titled one of his books The Social Scientist’s Bestiary: A Guide to Fabled Threats to and Defenses of Naturalistic Social Science

60 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 61

(2005). Even some academics confuse logical positivism and science as we know it today. Logical positivism emphasizes direct observation by the senses. It is assumed that observation can be theory free. It is justification focused, assuming greater verification yields closer approximations to the truth. This approach to knowledge was discarded decades ago because of the induction problem, the theory-laden nature of observation, and the utility of unobservable constructs (Miller, 1994; Shadish, 1995a & b). Misunderstandings about science may result in ignoring this problem- solving method and the knowledge it has generated. Misconceptions include the following: • There is an absence of controversy. • Theories are quickly abandoned if anomalies are found. • Intuitive thinking has no role. • There is no censorship and blocking of innovative ideas. • It is assumed that science knows, or will soon know, all the answers. • Objectivity is assumed. • Chance occurrences are not considered. • The accumulation of facts is the primary goal. • Linear thinking is required. • Passion and caring have no role. • There is one kind of scientific method. • Unobservable events are not considered. Misrepresentations of science are encouraged by those who view science as a religion—as offering certain truths. Science is often misrepresented as a collection of facts or as referring only to controlled experimental studies. People often confuse values external to science (e.g., what should be) with values internal to science (e.g., critical testing) (Phillips, 1987). Many people confuse science with pseudoscience and scientism (see Glossary). Some people protest that science is misused. Saying that a method is bad because it has been or may be misused is not a cogent argument; anything can be misused. Some people believe critical reflection is incompatible with passionate caring. Reading the writings of any number of scientists, including Loren Eiseley, Carl Sagan, Karl Popper, and Albert Einstein, should put this false belief to rest quickly. Consider a quote from Karl Popper (1994): I assert that the scientific way of life involves a burning interest in objective scientific theories—in the theories

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 61 62

in themselves, and in the problem of their truth, or their nearness to truth. And this interest is a critical interest, an argumentative interest. (p. 56) Far from reinforcing myths about reality, as some claim, science is likely to question them. All sorts of questions that people may not want raised may be raised such as: Does this residential center really help residents? Would another method be more effective? Is osteoporosis a disease? Should I get tested for cancer? (Gøtzsche, 2012; Welch, 2004). How accurate is this diagnosis? Many scientific discoveries, such as Charles Darwin’s theory of evolution, clashed with (and still does) some religious views of the world. Objections to teaching evolutionary theory remain common (see reports published by the National Center for Science Education, www.ncse.com). Only after 350 years did the Catholic Church agree that Galileo was correct in stating the Earth revolves around the sun. The “messiness” of inquiry (false starts and disappointing turns) is obscured by the brief format of journals. Dispute and controversy is the norm rather than the exception in science (e.g., Hellman, 1998).

Antiscience

Antiscience refers to rejection of scientific methods as a valid way to test claims. For example, some people argue there is no such thing as “privileged knowledge,” that some knowledge is more sound than others. Anti-intellectualism is common in America (Hofstadter, 1963), as is antiscience in some academic settings (Gross & Levitt, 1994; Patai & Koertge, 2003) and in popular culture (e.g., Burnham, 1987). Some people confuse science, scienticism, and pseudoscience, resulting in an antiscience stance (see Glossary).

Relativism

Relativists argue all methods are equally valid in testing claims (e.g., anecdotal reports and experimental studies). Postmodernism is a current form of relativism. It is assumed that knowledge and morality are inherently bounded by or rooted in culture (Gellner, 1992, p. 68). “Knowledge or morality outside of culture is, it claims, a chimera … . meanings are incommensurate, meanings are culturally constructed, and so all cultures are equal” (p. 73), so one cannot critique a culture if one is not a member of it. Gellner (1992)

62 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 63

argues that in the void created, some voices predominate, throwing us back on authority. If there is no means by which to tell what is accurate and what is not, if all methods are equally effective, the vacuum is filled by an “elite” who are powerful enough to say what is and what is not (Gellner, 1992). He argues that the sole focus on cognitive meaning in postmodernism ignores political and economic influences and “denies or obscures tremendous differences in cognition and technical power” (pp. 71–72). Gellner (1992) emphasizes that there are real constraints in society that are obscured within this recent form of relativism (postmodernism) and suggests that such cognitive nihilism constitutes a “travesty of the real role of serious knowledge in our lives” (p. 95). He argues that this view undervalues coercive and economic constraints in society and overvalues conceptual ones. “If we live in a world of meanings, and meanings exhaust the world, where is there any room for coercion through the whip, gun, or hunger?” (p. 63). Gellner (1992) suggests that postmodernism is an affectation: “Those who propound it or defend it against its critics, continue, whenever facing any serious issue in which their real interests are engaged, to act on the non-relativistic assumption that one particular vision is cognitively much more effective than others” (p. 70). Typically, such views are not related to real-life problems, such as building safe airplanes, and to a candid appraisal of the results of different ways of solving a problem. That is, they are not problem focused, allowing a critical appraisal of competing views. Gambrill and Gibbs (2002) found that social workers wanted their physicians to rely on the results of controlled experimental studies and demonstrated track record of success based on data collected systematically and regularly when making decisions about a serious medical problem of their own, but reported they relied on criteria such as intuition, testimonials, and experience with a few cases when making decisions about their clients. Some have mistaken Popper to be a postmodernist because of his insistence that nothing can ever be proved. Unlike postmodernists, he views science as the “unending quest” for truth through the process of eliminating what is found to be false.

Summary

Critical thinking and its reflection in the philosophy and evolving process of evidence-informed practice will help you and your clients

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 63 64

to make informed decisions—informed about uncertainties related to decisions as well as related questionable inferences. It will help you to honor ethical obligations to draw on practice and policy-related research, and to involve clients as informed participants. Professionals as well as clients are often bamboozled by bogus claims that have life-affecting consequences, including those in the peer-reviewed literature. It is not in the interests of many groups to reveal the lack of evidence for claims made and policies recommended, but it is in the interest of your clients. Problems may be created or may remain unsolved because we rely on questionable criteria to evaluate claims such as tradition, popularity, or authority. Critical appraisal skills and the “critical spirit” contribute to finding out who has fooled you. As you become immersed in the everyday world of practice, it is easy to forget about the economic, political, and social conditions that create social problems such as poverty and lack of affordable housing, which in turn may increase individual distress. Labeling such distress as a psychiatric disorder further obscures these influences. The biomedicalization of problems-in-living such as anxiety and depression distracts attention away from corporate interests in profit at our expense (e.g., Angell, 2009). Personal barriers include lack of education in critical thinking, including attention to common fallacies and cognitive bias, misunderstandings of science and how we learn, a reverence for “experts,” and overconfidence. Critical thinking, the process of evidence-based practice, and scientific reasoning all regard criticism (self-correction) as essential to forward understanding; all encourage you to challenge assumptions, consider well-argued opposing views, and check your reasoning for errors. All are anti-authoritarian. Differences, disagreements, and errors are viewed as opportunities to learn, to correct mistaken beliefs. All are designed to make the invisible visible, including uncertainties related to decisions. Because you are more likely to question assumptions, avoid influence by weak appeals, use language effectively, and minimize biases and fallacies, you are more likely to understand problems and appraise accurately the value of intervention and evaluation options. You are less likely to harm clients. You are more likely to spot bogus research reports and avoid the influence of propaganda pitches, pseudoscience, and quackery. Many costs of not thinking critically about beliefs and actions are hidden, such as failure to detect harm in the name of helping. By not raising critical questions, you are less likely to discover negative

64 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 65

consequences of widely accepted but inaccurate beliefs. You are less likely to speak up about and advocate for change in harmful practices and policies that contribute to the oppression of women, poor people, and people of color. Conflicts of interest that result in placing profit over helping may continue. Curiosity is likely to languish if vague, oversimplified accounts regarding client distress are accepted that obscure the complexity of lives, giving an illusion of understanding and posing an obstacle to helping clients. Janko (1996) suggests that exaggerated claims of knowledge result in indifference. Costs include “ruffling feathers,” foregoing the comfortable feeling of “certainty,” acknowledging ignorance (both avoidable and not), and spending the time and effort needed to understand alternative views. Critical thinkers often encounter an environment in which careful appraisal of assumptions that affect clients’ lives is viewed as a threat. Critical thinking and its reflection in evidence-informed practice and policy will not necessarily increase your popularity among “true believers” (those who accept claims based on faith and authority) or among marketeers (those interested primarily in selling a product). This is why caring about clients—as well as courage, integrity, and perseverance—is vital.

Gambrill & Gibbs Critical Thinking as a Guide to Decision Making 65 66 67â

EXERCISE 1â MAKING DECISIONS ABOUT INTERVENTION

Purpose

To provide an opportunity for you to review the criteria you use to make decisions.

Background

People in the helping professions often become so involved in the process of helping that they forget to step back and examine the basis for their decisions. This exercise provides an opportunity for you to examine the criteria you use to make decisions.

Instructions

1. Please answer the questions on the form that follows. 2. Review your answers using the guidelines provided. To get the most out of the exercise, complete the questionnaire before you read the discussion questions, and complete Situation 1 before you turn to the next page.

67 68 69

Practice Exercise 1 Making Decisions About Intervention

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ Date__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

SITUATION 1 Think back to a client (individual, family, group, agency, or community) with whom you have worked. Place a check mark next to each criterion you used to make your practice decision. If you have not yet worked with a client, think of the criteria on which you would probably rely.

CRITERIA

_____ 1. Your intuition (gut feeling) about what will be effective _____ 2. What you have heard from other professionals in informal exchanges __ _ _ _ 3. Your experience with a few cases _____ 4. Your demonstrated track record of success based on data you have gathered systematically and regularly __ _ _ _ 5. What fits your personal style __ _ _ _ 6. What was usually offered at your agency _____ 7. Self-reports of other clients about what was helpful _____ 8. Results of controlled experimental studies (data that show a method is helpful)* __ _ _ _ 9. What you are most familiar with _____ 10. What you know by critically reading professional literature

*Controlled experimental studies involve the random assignment of people to a group receiving a treatment method and one not receiving the treatment.

Gambrill & Gibbs Exercise 1 Making Decisions About Intervention 69 70

SITUATION 2 Imagine you have a potentially serious medical problem and you seek help from a physician to examine treatment options. Place a check mark next to each criterion you would like your physician to rely on when he or she makes recommendations about your treatment.

CRITERIA _____ 1. The physician’s intuition (gut feeling) that a method will work _____ 2. What he or she has heard from other physicians in informal exchanges __ _ _ _ 3. The physician’s experience with a few cases _____ 4. The physician’s demonstrated track record of success based on data he or she has gathered systematically and regularly __ _ _ _ 5. What fits his or her personal style __ _ _ _ 6. What is usually offered at the clinic __ _ _ _ 7. Self-reports of patients about what was helpful _____ 8. Results of controlled experimental studies (data that show that a method is helpful) __ _ _ _ 9. What the physician is most familiar with _____ 10. What the physician has learned by critically reading professional literature

SITUATION 3 Think back to a client (individual, family, group, agency, or community) with whom you have worked. Place a check mark next to each criterion you would like to use ideally to make practice decisions. If you have not yet worked with a client, think of the criteria on which you would ideally like to rely on.

CRITERIA _____ 1. Your intuition (gut feeling) about what will be effective _____ 2. What you have heard from other professionals in informal exchanges __ _ _ _ 3. Your experience with a few cases _____ 4. Your demonstrated track record of success based on data you have gathered systematically and regularly __ _ _ _ 5. What fits your personal style __ _ _ _ 6. What was usually offered at your agency _____ 7. Self-reports of other clients about what was helpful _____ 8. Results of controlled experimental studies (data that show a method is helpful) __ _ _ _ 9. What you are most familiar with _____ 10. What you know by critically reading professional literature.

70 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 71

SCORES Your instructor will provide scoring instructions. Situation 1 (Your Actual Criteria): Situation 2 (Physician’s Criteria): Situation 3 (Your Ideal Criteria):

DISCUSSION If you scored five to ten points, you are basing your decisions on criteria likely to result in a well- reasoned judgment (results from controlled experimental studies, systematically collected data, and critical reading). If you scored below two in any of the situations, you are willing to base decisions on criteria that may result in selecting ineffective or harmful methods. When making decisions, professionals often use different criteria in different situations. For instance, they may think more carefully in situations in which the potential consequences of their choices matter more to them personally (e.g., a health matter). Research on critical thinking shows that lack of generalization is a key problem; that is, people may use critical thinking skills in some situations but not in others.

FOLLOW-UP QUESTIONS Do your choices differ in these situations? If so, how? Why do you think they differ? If you scored below two on Situation 1 and two points or more on Situation 2, you may not believe that what is good for the goose is good for the gander. Your approach may be “science for you and art for them.” If you below 2 less in Situations 2 and 3, you may be prone to disregard sound evidence generally.

When is intuition (your “gut reaction”) a sound guide to making decisions about what practices or policies to recommend? When is it not? (See Part 3.)

Gambrill & Gibbs Exercise 1 Making Decisions About Intervention 71 72 73â

EXERCISE 2â REVIEWING YOUR BELIEFS ABOUT KNOWLEDGE

Purpose

To provide an opportunity to review your beliefs about knowledge and ignorance.

Background

The decisions professionals make reflect their underlying beliefs about knowledge and ignorance. Ignorance can be defined as a lack of knowledge that may be useful in making decisions and solving problems. Avoidable ignorance refers to ignorance that we can avoid, in the present and in the future. The term agnotology refers to the making and unmaking of ignorance. Proctor and Schiebinger (2008) argued that attention to ignorance is as important as attention to “what we know.” Much ignorance is unavoidable, reflecting the inevitable uncertainty involved in making decisions. Much is avoidable and may result in harm (e.g., Gotzsche, 2013, 2015a; Prasad & Ioannidis, 2014). Ignorance can be a benefit, especially if recognized. For example, it is a spur to solving problems and can prevent rushing to judgment in making life-âaffecting decisions. Kerwin and Witte (1983) identified six domains of ignorance. The first domain, known unknowns, refers to all the things we know we do not know; the second, unknown knowns, refers to all the things we do not know that we do not know. The third, errors are all the things you think you know but do not. The fourth domain, unknown knowns, refers to all the things we do not know we know. Unknown knowns require known knowns (assertions about the world shown to be accurate). The fifth,taboos refer to “socially enforced irrelevance.” This includes “what people must not know or ever inquire about” (Kerwin & Witte, 1983, p. 8)—âthings we are not supposed to know, but might be helpful to know. The sixth, denials refer to things too painful to know. Ignoring known knowns, such as sources of bias in research studies (e. g., Jadad & Enkin, 2007), is a major source of

73 74

avoidable ignorance. Inflated self-assessments make recognition of ignorance a challenge (Davis et al., 2006; Dunning, Health, & Suls, 2004).

Instructions

1. Circle the response that most accurately reflects your view in Practice Exercise 2 (A = Agree, D = Disagree, N = No opinion). Write a brief explanation below each statement to explain why you circled the response you did. 2. Compare your replies with those provided by your instructor.

74 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 75

Practice Exercise 2 Reviewing Your Beliefs About Knowledge

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _____ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ Instructor’s Name __ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

A = Agree D = Disagree N = No opinion

1. Because our beliefs influence what we see, we cannot gather A D N accurate knowledge about our world.

2. There are things we just can’t know. A D N

3. Researchers rarely hide limitations of their investigation. A D N

Note: Items 3 through 8 are based on W. Gray (1991).

Gambrill & Gibbs Exercise 2 Reviewing Your Beliefs About Knowledge 75 76

4. It’s good not to be too skeptical, because anything is possible. A D N

5. Ignorance is the mirror image of knowledge. A D N

6. Inflated claims of knowledge are rare in published literature. A D N

7. Everything is relative. All ways of “knowing” are equally true. A D N

8. Unlike knowledge, ignorance cannot be strategically used. A D N

9. Most research findings reported in the peer-reviewed literature A D N have been

76 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 77

10. Some things cannot be demonstrated scientifically. A D N

11. Recognizing ignorance can be a benefit. A D N

12. Trying to measure client outcome dehumanizes clients, A D N reducing them to the status of a laboratory rat.

13. Personal experience provides a sound guide about what is true A D N and what is not.

14. Our ignorance is vast compared with our knowledge. A D N

Gambrill & Gibbs Exercise 2 Reviewing Your Beliefs About Knowledge 77 78

15. Scientific reasoning and data are of little value in planning social A D N policy and social action.

16. Science is a way of thinking developed by white, male, Western A D N Europeans. It does not apply to other people and cultures.

SCORE_____ Your instructor will provide instructions.

Follow-up Question 1. Imagine a practitioner who agrees with your instructor’s suggested answers and reasons, and another who does not. Which one would be most likely to help clients and least likely to harm them? Why?

78 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 79â

EXERCISE 3âCONTROVERSY

INVALUABLE FOR PROBLEM SOLVING AND LEARNING

Purpose

To demonstrate the importance of identifying and discussing controversial issues related to life-âaffecting decisions, and to enhance argument analysis skills.

Background

Grappling with differences in beliefs and ideas is necessary for learning—â for correcting our background knowledge and discovering our ignorance. Popper (1994) suggests critical discussion is the only alternative to violence. The term debate often triggers negative reactions. This view is encouraged by “bad” behaviors such as personal attacks and distorting disliked views in presentations in which participants ignore, insult, harass, and interrupt others, and focus on winning rather than learning. Yet, even in the absence of “bad behaviors,” we may react negatively to even mild disagreements. This is why the goal of a discussion is so important to keep in mind. If the goal is to help clients, be on the lookout for ideas that may be more sound than those you hold, and therefore of more potential benefit to clients (see also Exercise 31). Discussing controversial issues encourages an appreciation of cultural diversity (Steiner, Bruzuzy, Gerdes, & Hurdle, 2003) and contributes to more informed and active citizens (Barton & McCully, 2007; Reitano, Kivunja, & Porter, 2008). Only if we discuss our differences can we learn from them. Limitations of views are more likely to be spotted and valuable options identified if multiple, well-âargued perspectives are considered. Only by questioning claims can we reveal avoidable miseries as a result of corrupt practices and/âor premature closure on problem framing. We need a community of inquiry in which clashing views are welcomed, including cultural differences, to

79 80

minimize biases shaped by our culture (sociocentric biases) and those unique to ourselves (egocentric biases)—one in which learning is the goal (Paul, 1993). Ignorance regarding the social aspects of science, such as the quest for prestige, may contribute to an unwarranted gullibility (dependence on “experts”). Many commonly accepted claims are wrong or controversial. Here are some examples of controversial issues (see also examples on www.procon.org): • Should prescription drugs be advertised directly to consumers? • Should all people older than 65 be screened for dementia? • Does the alliance in psychotherapy have a “marginal scientific status” (Baker, McFall, & Shoham, 2008)? (See, for example, Wampold and Imel [2015a].) • Are (mis)behaviors mental illnesses reflecting brain disorders treatable with specific drugs? • Is Welfare-to- Work effective? • Should private companies run prisons? • Have we become an oligarchy (e.g., see Lewis, 2015)? Controversial issues are often downplayed or hidden in the helping professions, including professional education programs, because of the pursuit of political, social, and economic interests. Examples include use of psychotropic medication, disease mongering in which well people are made to feel sick, and offering psychological help instead of correcting injustices that create distress. Controversy may have a “chilling effect,” encouraging self-censorship (e.g., Kempner, 2008). Public relations and advertising agencies promote products and views via “spin” (see Part 2). In many schools of social work, students are indoctrinated into the use of a medical classification system, theDiagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013), or DSM, regarding “deviant” behavior that ignores individual differences and downplays environmental factors that create distress. They become enablers of medicalization; they misrepresent the sources of distress (Gambrill, 2014a). We each have a different history with critical discussion. Some people learn to distinguish between evidence and beliefs from an early age, perhaps by listening to conversations between their parents. Others equate belief and evidence.

80 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 81

Rather than seeing their theories as belief states subject to disconfirmation and representing theory and evidence as distinct entities to be reconciled with one another, they may merge the two into a single representation of “the way things are” with little apparent awareness of the sources of their belief. (Kuhn, 1999, p. 21)

If critical dialogue is new to you, you may take it personally when someone disagrees with you. This is why the goal of the discussion is vital to keep in mind. If client welfare is involved, we are obligated to explore the cogency of different views.

Guidelines for Discussion Here are some valuable guidelines: • Listen carefully, with an open mind, to the contributions of others. Focus on accurately understanding their position and related reasons. • Ask for clarification when you do not understand a point (see Boxes 1.1 and 1.4). • Explain your reasons if you challenge ideas. Describe the relevance of issues you raise. • Critique ideas or positions, not people. Avoid negative reactions such as ridicule. • If others challenge your ideas, be willing to change your mind if they demonstrate flaws in your reasoning, such as inaccurate premises. Do not respond to criticism as personal attacks; focus on the goal of helping clients. • Seek input from all participants. • Reinforce others for raising good points. (Adapted from For Your Consideration #21: Teaching Controversial Issues [2004].) (See also descriptions of structured controversy.) Communication blockers include blaming, being distracted (e.g., body language suggesting inattention), avoidable vagueness, dismissing others’ points, interrupting others, rolling the eyes, smirking, lecturing, distorting what others say, and offering premature advice (Marchel, 2007). Communication enhancers include providing a clear description of arguments, giving your full attention, recognizing other points of view (e.g., “I can see how someone could believe this”), and remaining silent until

Gambrill & Gibbs Exercise 3 Controversy: Invaluable for Problem Solving and Learning 81 82

others are finished speaking. (For further discussion of arguments see Exercise 30.) Principles Damer (2005) suggests for effective, rational discussions, include the following:

Fallibility: A willingness to admit you could be wrong Truth-seeking : A commitment to search for the truth or best-argued position—to examine alternative positions and to welcome objections to your view Burden of proof: This rests on the person who presents it. Charity: Arguments are presented in their strongest version. Clarity: Positions, defenses, and challenges are clearly described. Relevance: Only reasons or questions related to the merit of a position are offered. Acceptability: The premises or reasons on which you rely meet standard criteria of acceptability. Sufficient grounds: Those who present an argument for or challenge a position should attempt to provide reasons sufficient in number, kind, and weight to support the conclusion. Rebuttal: The person who presents an argument for or challenges a position should attempt to provide effective responses to all serious challenges or rebuttals. Resolution: An issue should be considered resolved if the proponent of a position presents an argument that uses relevant and acceptable premises sufficient in number, kind, and weight to support premises and the conclusion, and provides an effective rebuttal to all serious challenges. Suspension of judgment: If no position can be successfully defended, or if two or more positions can be defended with equal strength, suspend judgment or, if practical considerations require a decision, proceed based on preferences. Reconsideration: Parties are obligated to reconsider the issue if flaws are found in an argument (see also Exercise 30).

Discussion Questions How are controversial issues handled in your classes and/or in your field placement? Do you feel free to note that a claim/assumption is

82 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 83

controversial? Are controversial issues acknowledged and discussed openly? If not, what topics are not raised? Are there “taboo” topics in your agency education program that affect the services clients receive? If yes, please describe one and discuss related reasons. What is a “safe” environment for discussion? Would everyone agree? Does this allow questions concerning the evidentiary status of services offered to clients? Consult resources on understanding differences and prejudices, such as the Southern Poverty Law Center (www.splcenter.org), the Anti-defamation League (www.adl.org), Understanding Prejudice (www.understanding- prejudice.org), the Society for the Teaching of Psychology (www.teach- psych.org), and www.procon.org.

Activity 1

1. Watch “The 52nd Maudsley Debate: More harm than good?” (www. youtube.com) of May 13, 2015. Review the list of myths asserted regarding the concept of “mental illness” (e.g., the myth of the chemical imbalance) on the website of the Council for Evidence-Based Psychiatry (www.cepuk.org). Compare content on this website with content on the website of the National Alliance for Mental Illness (www.nami.org). 2. Count off one to four until all students are in small groups. Each group will select a position on the issue, conduct related inquiry and discussion, and prepare a brief statement arguing its view (consult Exercise 30 as needed). List key points together with citations of key references. What is the theory behind your position (e.g., a biological view of (mis)behavior)? Focus on key concerns and tie each point clearly to the group’s position. Include a well-argued rebuttal statement arguing the reverse position. Do you think Gøtzsche (2013) is excessive using the words “organized crime” in his book Deadly Medicines and Organised Crime: How Big Pharma Has Corrupted Health Care. Be prepared to give reasons for your answer. Read the introduction to his book Deadly Psychiatry and Organized Denial (2015a) (also available on the Web). 3. Conduct a discussion. Each group has five minutes to present its view and five minutes to present a well-argued rebuttal. 4. Hold a general discussion in which different positions are discussed. What are ambiguities, uncertainties, and complexities concerning this

Gambrill & Gibbs Exercise 3 Controversy: Invaluable for Problem Solving and Learning 83 84

issue? Is there a consensus? If not, what are the specific differences? Was there an “open and civil exchange of views” (www.spj.org)? Did anyone change their mind? Were any important perspectives omitted? If so, which ones? Did cultural, race, class, gender, sexual orientation, and/or ability issues arise, such as stereotyping? If so, describe exactly how. How were these issues handled? 5. Identify flaws, fallacies, and biases reflected in the discussion, such as overclaiming (exaggerating evidentiary status), vague assertions, and begging the question. Other concerns include use of strawman arguments, cherry-picking, social loafing, and groupthink (see Exercises 11–13). How were they handled? Were they even noticed? Review pitfalls described in Carl Sagan’s Baloney Detection Kit. Were any of these on display? 6. Prepare a practice/policy brief clearly noting and arguing for a position (see www.jhsph.edu and/or www.pep-net.org ).

Activity 2 Complete Practice Exercise 3.

Activity 3 Read “What Do We Know About Psychotherapy?—And What Is There Left to Debate?” by Wampold and Imel (2015b). Together with other students, select a related issue you would like to discuss. Why is therapy so popular? Is it because thoughts and feelings are promoted as the cause of alienation, loneliness, and anxiety in our technological society, as Stivers (2001) argues?

Activity 4 Read and discuss “Lucrative Pseudoscience at the International Positive Psychology Association Meeting” by Coyne (2015a). Use the format of Activity 1 to consider this issue. See also Unintended consequences of universal mindfulness training for schoolchildren? By James Coyne (2016).

Activity 5 Additional examples of possible topics include the following: • Pros and cons of the “globalization of mental health” (e.g., Summerfield, 2012)

84 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 85

• Pros and cons of the privatization of prisons (e.g., Dolinar, 2015; Ludwig, 2015; Rabuy, 2014) • Pros and cons of using the concept of structural violence to understand inequities in life opportunities (e.g., Pilisuk & Rountree, 2015)

Gambrill & Gibbs Exercise 3 Controversy: Invaluable for Problem Solving and Learning 85 86 87

Practice EXERCISE 3 Addressing a Controversial Issue

Names of group members __ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Date __ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _____ _ _ _ ______ _ _ _ _ _ Instructor__ _ _ _ _ _ _ __ _ _ _ _ _ ______ _ _ _ _ _ _

Controversial issue (describe): __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Your position: __ _ Pro __ _ Con (check one)

Describe your reasons here under separate points, including related theory.

1. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _

2. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

3. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 3 Controversy: Invaluable for Problem Solving and Learning 87 88

4. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

5. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ __

Cite best source used: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _

Provide your rebuttal: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Conduct a discussion following guidelines in Activity 1.

Follow-up Questions What were the reactions to the different kinds of statements made during your discussion, including questions (e.g., about claims), gentle disagreements (e.g., “Could it be … ?”; “I think there is another point of view”), more assertive disagreements (e.g., “I think this view is incorrect”). Are there more effective ways to disagree that encourage listening and learning?

88 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 89â

EXERCISE 4â CRITICAL THINKING AND ADVOCACY

Purpose

To provide practice in using critical thinking to identify and minimize injustice and inequality.

Background

Helping professionals, including physicians, often encounter problems caused or made worse by discrimination based on race, sexual orientation, religion, age, ability, and/âor ethnicity and related inequities—âfor example, in housing, environmental pollution, healthcare, employment opportunities, access to transportation, and educational opportunities (e.g., Case & Deaton, 2015). Consider, for example, “Disproportionate Risk of Driving While Black” (LaFraniere & Lehren, 2015), “The Color of Debt” (Kiel, 2015), and “Children Caught in a Racist System” (2015). Obligations in professional codes of ethics require us to identify, describe, expose, and advocate for minimizing avoidable miseries (Gambrill, 2013b). Guyatt and Rennie (2002), in their discussion of the philosophy of EBP, include obligations of practitioners to advocate for changes in environmental conditions that contribute to health problems. Corporate and state crimes, and failure to address them contribute to avoidable miseries (e.g., Barak, 2015; Chambliss, Michalowski, & Kramer, 2010). Marmot (2015) argues that social injustice kills 200,000 individuals in the United Kingdom every year. Conflicts of interests are often involved. Strategic ignorance is used to hide inequities and global violence and their effects (Pilisuk & Rountree, 2015). Regulatory agencies often fail to do their job (e.g., Zetterquist & Mulinari, 2013). These events highlight the value of thinking critically about inequities and what can be done about them. Critical thinking is contextual thinking; it encourages you to use your “sociological imagination” (Mills, 1959) to understand the big picture (context). Web resources related to advocacy include Center for Media & Democracy (www.prwatch.org), Truthout (www.truthout.org),

89 90

Center for Public Integrity (www.publicintegrity.org), PolitiFact (www. politifact.com), and Center for Race Equity and Education (www.gse. upenn.edu/equity).

Instructions

Together with two or three of your classmates, select some avoidable misery in your local area. Examples include illegal evictions (Barker, 2015), extortion of poor people by private probation companies (Dewan, 2015a, 2015c), lack of access by the poor to effective legal aid, harmful conditions in for-profit nursing homes (e.g., Abramo & Lehman, 2015), and abuse and neglect in for-profit foster care (e.g., Roston & Singer-Vine, 2015). 1. Describe the problem clearly, including related causes. Document the exact nature and extent of this problem. Will you use pictures? If so, how will you get them? Who are the stakeholders? Will you talk to those involved? If so, who and how will you introduce yourself? How will you locate and appraise related material critically, including newspaper and agency reports, and videos? Are there any? Is the problem hidden (see Fawcett & Rabinowitz, 2007). 2. Describe the consequences of the problem for those affected. How will you combine qualitative and quantitative data (e.g., Mertens & Hesse-Biber, 2013; Saini & Shlonsky, 2015). Describe current contingencies at multiple levels (e.g., client, neighborhood, agency, lobby groups) that influence related practices and policies. (Consult guidelines for preparing a policy brief as needed.) 3. Describe ploys used to deny and/or maintain the problem (e.g., claiming lack of resources, and/or claiming there is no problem). See, for example, Living in Dignity in the 21st Century (Council on Europe, 2013). 4. Design a plan for exposing this avoidable misery, including use of the Internet to increase exposure and garner support. Who will you involve and how? Consider, for example, the argument that women of color bear the costs of mass incarceration (Dusenbery, 2015). 5. Carry out your plan over the course of a class and describe implementation problems and outcomes. 6. Make a class presentation regarding your advocacy efforts. How successful were you? If not at all, please suggest reasons. Related questions for class discussion include the following: Is social justice a socially

90 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 91

constructed concept? Is outrage a reasonable response to many avoidable miseries?

Further Activities

1. Minimizing avoidable suffering may require you to be a whistleblower (Thomas, 2015). Would you blow the whistle on a practice/policy in your agency or professional education program that you believe harms clients? Are you familiar with your school’s or agency’s whistle- blowing policies? What is the fate of whistleblowers? Consult related literature and discuss as a class (see www.exposefacts.org). 2. Should we focus on increasing happiness or decreasing avoidable miseries (e.g., see Lilienfeld & Arkowitz, 2011)? Discuss as a class. 3. Discuss the role strategic ignorance plays in creating harm and maintaining discrimination and oppression (e.g., Lanphear, 2011). 4. Institutional abuse refers to policies and practices on the part of public and private organizations that harm clients, such as poor-quality services. Describe and document an example (e.g., Mysyuk, Westendorp, Biggs, & Lindenberg, 2015). Discuss as a class. 5. Read “Environmental Racism Persists, and the EPA Is One Reason Why” by Lombardi, Buford, and Greene (2015), and discuss as a class. See also relevant material in Routledge International Handbook of the Crimes of the Powerful (Barak, 2015). 6. Discuss the relationship between discrimination and social immobil- ity. (see Council on Europe, 2013.) Watch the TED talk by Richard Wilkinson: “How Economic Inequality Harms Society” and discuss advocacy implications. 7. Review The School-to-Prison Pipeline (Mallett, 2015). Describe advocacy opportunities to discourage this. 8. Select a nonprofit group that claims to advocate for clients and describe key outcomes attained (e.g., Harris, 2009; Wegemer, 2015). 9. Describe lapses on the part of regulatory agencies such as the FDA, for example, concerning transparency of decisions and related consequence (e.g., Doshi, 2015; Zetterqvist & Mulinari, 2013). 10. Describe examples of needed advocacy for Native Americans (e.g., Royden, 2015). 11. Advocacy includes protecting vulnerable populations from harm created by fraud and corruption (e.g., Dewan, 2015a, 2015b). Select

Gambrill & Gibbs Exercise 4 Critical Thinking and Advocacy 91 92

an example and discuss advocacy opportunities, such as using skits to educate elders about scammers (Lelyveld, 2015; see also Olson, 2015). 12. Discuss implications of the article “Human Rights in the New Global Strategy,” by Sanghera et al. (2015).

92 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 93

Practice Exercise 4

Names of group members: ____ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Date: __ _ _ _ _ _ _ _ _ _ Course: __ _ _ _ _ _ _ _ _____ _ Instructor: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Avoidable misery (what and who): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Clear examples: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Involved stakeholders: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 4 Critical Thinking and Advocacy 93 94

Ploys used to hide/mystify misery: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Advocacy plan (describe actions, parties, and contexts): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Outcome: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Implementation problems: __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Follow-up Question What did you learn from this exercise?

94 Critical Thinking as a Guide to Decision Making Gambrill & Gibbs 95â

PART 2 Recognizing Propaganda

Propaganda “seeks to induce action, adherence and participation—âwith as little thought as possible” (Ellul, 1965, p. 180; see also Carl Sagan’s Baloney Detection Kit [2011]). Propaganda hides influences on our decisions and information of value in making decisions; it makes selective use of evidence. It hinders our autonomy to make our own decisions based on accurate information, including related uncertainties (see discussion of strategic use of ignorance in Part 1). Propaganda occurs in all venues, including advertisements (e.g., Loke, Koh, & Ward, 2002; Zetterqvist & Mulinari, 2013) and the professional literature, as discussed in Part 1 (inflated claims of effectiveness, distortions of disliked positions, misleading graphics and statistics to bolster claims, and hiding adverse effects of intervention). Major methods include distortion, confusion, censorship, and fabrication. Here are some examples of propaganda in professional sources: • A brochure designed to encourage nonsymptomatic women to be tested for breast cancer by having a mammogram does not describe possible harms, such as the high rate of false positives and resultant unnecessary biopsies. • Vague phrases in professional sources such as “It has long been known,” “It is generally believed”. • Lilly, the pharmaceutical company, hid the fact that taking Zyprexa increased the risk of diabetes. • Osteoporosis is described as a disease when it is a risk factor. • Antidepressants are marketed as effective and harm free.

95 96

Much propaganda is “a set of methods employed by an organized group that wants to bring about the active or passive participation in its actions of a mass of individuals, psychologically unified through psychological manipulations and incorporated in an organization” (Ellul, 1965, p. 61). This can be contrasted to critical appraisal of beliefs and actions. Some medical educators have been so concerned about the influence of pitches by pharmaceutical companies on medical students that courses are included designed to help students avoid these influences (Wilkes & Hoffman, 2001; Wofford & Ohl, 2005). Propaganda in the helping professions includes deep propaganda that obscures political, economic, and social factors that influence problems selected for attention and how they are framed—for example, focusing on individuals as the source of their own distress requiring the help of experts while ignoring environmental factors such as lack of jobs that provide a living wage, related affordable healthcare, high-quality education, and affordable housing. There are scores of propaganda ploys (see Box 2.1). Many involve common fallacies and biases (see Part 3). Many work via language (see Exercise 9). Some create doubt to protect special interests (e.g., harms of smoking tobacco, climate change). A number involve social psychological strategies such as promoting false claims of scarcity to encourage use of a product (e.g., “Act now or …”) (Cialdini, 2009). Wff’N Proof (Allen, 1970) divides propaganda techniques into six categories: (1) techniques of self-deception such as wishful thinking, rationalization, and prejudice; (2) techniques of language such as vagueness and use of emotional terms; (3) techniques of irrelevance such as appearance, degrees, and titles; (4) techniques of exploitation such as appeal to pity, flattery, ridicule, and bandwagon; (5) techniques of form such as post hoc (see Exercise 11), selected instances, and hasty generalizations; and (6) techniques of maneuver such as diversion, ad hominem, strawman, and question begging. Propagandists take advantage of all these ploys (e.g., http://energyskeptic.com). Misleading claims are a key form of propaganda—inflated claims about what works, what causes certain behaviors, what can keep us healthy, what a risk is, and how we can identify risks. Ignorance is promoted actively by hiding knowledge that is available, such as the harmful effects of psychotropic medication (Gøtzsche, 2013, 2015a & b) and the harms of smoking (Oreskes & Conway, 2010), for example. We are often patsies for others’ bogus claims because of our own inactions,

96 Recognizing Propaganda Gambrill & Gibbs 97

Box 2.1 Examples of Propaganda Ploys*

1. Ad hominem: inappropriately focusing on the person (attack/praise) rather than the argument 2. Appeal to anecdotal evidence: relying on one or more examples that do not represent the related population 3. Appeal to ignorance: assuming a claim is accurate because it has not or cannot be shown to be false 4. Appeal to unfounded authority: degrees, titles, jargon, popularity, tradition, use of celebrities, manner, numbers 5. Bandwagon: “Everybody is doing it”; attempting to convince us we should “jump on the band wagon” and follow the crowd 6. Begging the question: using a conclusion also as a premise; assuming a controversial point contested by others (Walton, 1991) 7. Causal fallacies such as post hoc ergo prop: assuming that correlation reflects causation. 8. Card stacking: selecting content to give the best (or worst) possible case for an idea, program, person, or product—suppressed evidence (also called cherry-picking) 9. Contrast effect: introducing an extreme view to encourage acceptance of a more moderate one or to establish a moderate stance and shift gradually to more extreme one 10. Creation of doubt: disseminating false or negative information to undermine a belief 11. Dictat: presenting an idea/cause as the only viable one 12. Disinformation: providing false or distorted information to encourage an action or a belief; creating false accounts or records, altering or removing existing ones to create support for or opposition to an idea or cause 13. Diversion/distraction: for example repeating inconsequential statements that ignore a problem 14. Divide and conquer: (e.g., Webster, 1992). 15. Either-or: presenting only two alternatives when there are others 16. Evading the issue: refusing to address the issue 17. Fallacy of composition: inferring that x is true of the whole based on the fact that it is true of some parts of the whole 18. Fallacy of division: inferring that something is true of one or more parts because it is true of the whole 19. False analogy (comparison) or metaphor: A is like B, B has property P, therefore A has property P (www.fallacyfiles.org) 20. Foot in the door: encouraging compliance with a large request by first asking for a small one (For other social psychological strategies see Part 3.) 21. Framing: misleading description of an issue 22. Glittering generalizations: associating something with a word to make us accept and approve the product, for example, without examining the evidence 23. Hasty generalization: arguing from a small number; making a conclusion based on a small sample—perhaps of one, as in a case example or testimonial 24. Irrelevant emotional appeals: appealing, for example, to fear, euphoria (perhaps created by spectacle), pity, nationalism, flattery, and/or prejudice; demonizing the enemy 25. Language ploys: relying on equivocation, vagueness (weasel words), leading questions, emphasis, euphemisms, repetition, hyperbole, innuendo, slogans, labels, biobabble, or psychobabble; using clichés to stifle dissent or validate faulty logic (see Exercise 9)

Gambrill & Gibbs Recognizing Propaganda 97 98

26. Milieu control: using peer or social pressure to create adherence to an idea or cause 27. Misuse of statistics: using statistics in a misleading manner (e.g., Best, 2004) 28. Name calling: giving an idea a bad label to make us reject and condemn the idea without examining the evidence 29. Oversimplification: oversimplifying complex topics; presenting information out of context 30. Plain folks: attempting to convince an audience an idea is good because it is “of the people,” the “plain folks” 31. Quote out of context: using selective and misleading quotations 32. Red herring: introducing irrelevant content as a distraction 33. Scapegoating: blaming a person or a group for a problem to distract attention from those responsible for it and/or to distract the audience from the problem itself and the need to fix it 34. Stereotyping: inciting prejudice by reducing a group to a set of undesirable traits 35. Strawman: misrepresenting or distorting a view and then attacking it 36. Testimonial: having a respected or hated person say that a given idea or program or product or person is good or bad 37. Third party: using a supposedly impartial person or group, such as a journalist or an expert, or a group represented falsely as a grassroots organization, to support an idea or cause or to recommend a product 38. Transfer (association): carrying the authority, sanction, and prestige of something respected and revered (such as virtue words) over to something to make the latter acceptable; or carrying authority, sanction, and disapproval over to something to cause us to reject and disapprove it

*Items 5, 8, 22, 28, 30, 36, and 38 are from the Institute of Propaganda Analysis (1938). Fallacies and propaganda methods overlap, see also Gambrill, 2012a, www.fallacyfiles.org, and Wikipedia.

our failure to ask questions, to be skeptical. Propagandists appeal to our values and emotions. (See discussion in Part 1.) An engaging and polished presentation by a charismatic speaker may lure us into believing that someone is deeply learned in a subject when he or she is not, as illustrated by Naftulin, Ware, and Donnelly (1973) more than a quarter of a century ago. Their study showed that even experienced educators “can be seduced into feeling satisfied that they had learned despite irrelevant, conflicting, and meaningless content conveyed by the lecturer” (p. 630). The authors concluded that “student satisfaction with learning may represent little more than the illusion of having learned” (p. 630). Many professional conferences present ideal conditions for the Dr. Fox Effect: The audience is exposed to a speech only once, the audience expects to be entertained, and the audience will not be evaluated on mastery of content in the speech. Student evaluations of their teachers may be based more on their style or charisma than on

98 Recognizing Propaganda Gambrill & Gibbs 99

how accurately they present course content (see, for example, Ambady & Rosenthal, 1993; Williams & Ceci, 1997). Avoiding propaganda ploys in our technological society are analogous to the challenge of Odysseus, a character in Greek mythology who had to guide his ship past the treacherous sirens’ song. He was forewarned the sirens’ song was so seductive that anyone who heard it would be lured to a reef, where the ship would wreck and all would drown. Odysseus put wax in his crew’s ears so they couldn’t hear the sirens’ song, but he had them chain him to the mast so that he would hear it but not take over the helm and steer the ship toward the sirens and the reef. Clients and practitioners must steer a course toward critical appraisal of claims while avoiding the siren calls of propaganda pitches that promote misleading views and harmful methods. To avoid being taken in, watch for the following:

1. What claim is promoted? Who or what organization is promoting it? Has this claim been critically tested? If so, what were the results? Is all related evidence presented? Is it accurate? What is missing? Is some relevant content hidden, such as clinical trials of a drug showing harm? Has there been cherry-picking (reporting only trials with positive results)? 2. Are emotional appeals used, such as a strikingly attractive person, background music to set a mood, or an appeal to fear? What slogans and images are used? 3. Beware of the style of presentation, including a presenter’s apparent sincerity, which suggests a belief a claim is true; a polished presentation, which lends credibility; a presenter’s attempts to appear similar to the audience; and the use of anecdotes and humor that entertain but do not inform. 4. Beware of the effect of the presenter’s status, degrees and titles (e.g., professor, MD, PhD, RN), affiliations with organizations familiar to the audience, and a favorable introduction by someone familiar. 5. Watch out for influence via information that is accurate but is misleading or irrelevant.

Advertising and Public Relations Agencies

Unlike material that is clearly an advertisement, the public relations industry often works in the background, obscuring their influence.

Gambrill & Gibbs Recognizing Propaganda 99 100

Indeed, such invisibility was key in Edward Bernays’ (1928) view of propaganda—to use scientific manipulation to mold public opinion, drawing on psychological and other research. Public relations firms are paid to reframe issues, create doubt, and use symbols and images to persuade us. Related tactics include staged publicity events, lobbying, robocalls, spin (e.g., selective presentation of facts, misleading language) (e.g., Rampton & Stauber, 2001; wikipedia.org). Cohen & Wolfe, a public relations agency hired by the pharmaceutical company GlaxoKlineSmith created the term “social anxiety disorder” and laid the groundwork over a two-year period for the introduction of Paxil (Moynihan & Cassels, 2005). Public relations firms help industries to create astroturfs (front groups). An astroturf presents itself as a grassroots organization, but in reality is created and “funded by corporations, industry trade associations, political interests, or public relations firms” (wikepedia). They may be phony or actually consist of real patient organizations “willing to serve as paid shills.” Brody (2007) highlights their use in fighting against transparency of drug trials conducted by pharmaceutical companies.

Advertisements Disguised as Professional Literature

Many publications in professional sources are of high quality; authors are honest brokers of knowledge and ignorance. They describe accurately what they hoped to find and what they did find, as well as conceptual and methodological limitations. But, many do not share these characteristics, as discussed in Part 1. We may find bogus citations (citations that do not provide support for claims) (Greenberg, 2009), weasel words, and phrases such as “It is well known that …,” “It is widely accepted that …” (when there is controversy). It was, in part, because of inaccurate statements in texts, editorials, and professional articles that the process of evidence-based practice was developed (Gray, 2001a). Peer-reviewed articles and reports that purport to “tell the truth” may, in reality, function as advertisements for the profession, the university, the agency, or an individual publishing the report. As a result of reading an article, you may think you are more informed, but you may have acquired or strengthened beliefs that are untrue.

100 Recognizing Propaganda Gambrill & Gibbs 101

Similar purposes and strategies are evident in both propagandistic professional literature and human services advertisements (Gambrill, 2012a). In both, the trappings of science are used to create credibility. In both, it is hoped that we will suspend critical appraisal and trust “the experts.” Propaganda in the media and in the professional literature interacts with groupthink in organizations, as well as with self- propaganda, such as denial, wishful thinking, and confirmation biases (searching only for material that supports our views), to compromise decisions. Examples of useful websites include www.healthnewsreview. org, www.procon.org, www.Pharmedout.org, and www.politifact.com

About the Exercises

The exercises in Part 2 are designed to give you practice in recognizing and minimizing the influence of propaganda ploys that lead you and your clients astray. Exercise 5 illustrates characteristics of human service advertisements. You will watch a presentation and evaluate what you have seen. In Exercise 6, you view and appraise content about the Juvenile Awareness Program at Rahway prison in New Jersey. Exercise 7 gives you practice in spotting misleading problem framing; Exercise 8 gives you an opportunity to “follow the money” to increase your awareness of how the pursuit of profit may harm clients. Exercise 9 highlights the language of propaganda. Please follow your instructor’s suggestions for completing the exercises. Some instructors may want you to read this section only after you have reacted to videotaped material.

Gambrill & Gibbs Recognizing Propaganda 101 102 103â

EXERCISE 5â CRITICALLY APPRAISING HUMAN SERVICES ADVERTISEMENTS

Purpose

To increase your skill in detecting misleading appeals in advertisements.

Background

Most people are somewhat skeptical about advertisements that appear on the Internet, in newspapers, and on television. Advertisements use emotional appeals, images, and claims to encourage you to use certain products: Buy this product and a lush growth of hair will sprout thickly like a rug on your head. If you’re older than 60, take these pills, and you’ll leap around like a kid again. Advertisements are prepared by public relations and advertising firms, as well as by organizations and individuals offering a service or product (e.g., alcohol abuse treatment centers) and are distributed through brochures and the Internet (e.g., videotapes, films, CDs) to encourage professionals and/âor potential clients to use a service. Direct-âto-âconsumer advertising of pharmaceuticals was ruled legal in 1997 in the United States. Content analysis of television direct-â to-âconsumer advertising shows that these provide little information of an educational nature and oversell the benefits of drugs in ways that conflict with the promotion of health (Frosch, Krueger, Hornik, Cronbolm, & Barg (2007). Many create needless worry (Hadler, 2008). Advertising works, which is why billions of dollars are spent on advertisements. It is one thing for people to spend a few dollars on a product they may not need or will not deliver what is promised, but is quite another for professionals to make decisions based on propagandistic appeals, for example, to emotion. Features of advertising identified by Rank (1984a) include the following: • Attention getting: using visual images (lighting, sound), and words and images with strong emotional associations. • Confidence building: establishing trust by stating you should believe the expert because he or she is sincere and has good intentions

103 104

• Desire stimulating: describing the pleasure to be gained, the pain to be avoided, the problem to be solved • Urgency stressing: encouraging buyers to act now • Response seeking: trying to learn whether the advertisement worked (p. 547). Profit is a key motive. Although a concern for profit is not incompatible with truthful accounts, advertising in general avoids describing negative data and arguments pro and con unless obligated to do so. Most advertisements are deceptive (e.g., they exaggerate benefits and use propaganda ploys such as appeal to authority and fear). Hospitals, drug companies, and a variety of other entities have been sued for false advertising. For example, Florida’s Memorial Hospital paid 178 million dollars for false advertising of its weight loss program as a ‘Center for Excellence’ (Caramenico, 2012). Many publications in the peer-reviewed literature have the characteristics of advertisements in both the tactics they use (e.g., inflated claims) and the goals pursued (e.g., consumption of a product such as a therapy (Gambrill, 2012a). Terms such as well-established and empirically validated convey a certainty that cannot be had. Human service advertisements tend to have the following features: 1. They rely on common fallacies such as testimonials (statements by those who claim to have been helped by the method) and case examples (descriptions of individual cases that supposedly represent the client population that has benefited from the treatment). (See Exercises 11–13.) 2. They involve persons of status who may believe in a program and argue the method works, but do not describe critical tests of claims. 3. There is a well-rehearsed, smooth presentation, relying on style, not evidence, to support claims. 4. Visual and auditory images are used to lull the audience into not asking questions about whether the method works. 5. The alleged positive effects of a product are highlighted. (In direct-to- consumer ads for drugs, if hoped-for outcomes are described, potential harms must also be described.) This is usually done in a rushed manner (if video) or appears in small print, if print source. Vague claims are made, such as “x works.” Typically, no related evidence that the service is effective in achieving the outcomes promised (e.g., an experimental study, a reference to studies evaluating the service) is described. Advertisements tend to oversimplify complex issues. Other

104 Recognizing Propaganda Gambrill & Gibbs 105

ploys identified by Schrank (1974) include use of weasel words such as helps control, virtually, and fights; incomplete claims (“gives you more”— more what?); claims that are true for all related products (“our product uses x,” but so do all the others); the “so-what claim” (e.g., “Geritol has twice the amount of x”—Is that good?); vague claims (luscious lips); bogus scientific claims (“33% more nutrition”), and appeals to you as special (“you pride yourself as x”). www.Pharmedout.org provides educational materials to help the unwary enhance their skills in avoiding the influence of pharmaceutical propaganda. This website was funded by money from the successful lawsuit against Pfizer for fraudulent sales and marketing of Neurontin—pushing off-label uses when insiders knew the drug was ineffective for several of these uses. The jury found that Pfizer violated the federal Racketeer Influence and Corrupt Organizations Act (Van Voris & Lawrence, 2010). In a related report it was claimed Pfizer engaged in “outright deception of the biomedical community, and suppression of scientific truth”—stalling or stopping the publication of negative study results, manipulating both trial designs and data to make the drug look more effective than it was, and using questionable tactics to enhance the drug’s image and increase its sales. These practices were “highly unethical, harmful to science, wasteful of public resources, and potentially dangerous to the public’s health” (Neurontin Marketing Sales Practices and Products Liability Litigation, MDL 1629, U.S. District Court District of Massachusetts [Boston]). Practitioners, hospitals, and organizations advertise their programs. Most have websites containing promotional content. Professional journals such as the American Psychologist often include full-page advertisements for certain medications (e.g., Adderall). Promotional television programs advertise weight loss, study skills, smoking cessation, and other types of programs. Professional conferences often include presentations that meet the criteria for an advertisement: A charismatic, well-known person describes an intervention and presents it in an engaging, entertaining way; makes claims of effectiveness; and does not describe related research. How accurate are the claims?

Activity 1

Read or watch a presentation selected by your instructor. Your instructor may use promotional material from Rogers Memorial Hospital,

Gambrill & Gibbs Exercise 5 Critically Appraising Human Services Advertisements 105 106

Oconomowoc, Wisconsin, or direct you to another source. Answer the questions in Practice Exercise 5. Discuss ploys used as a class.

Activity 2

Bring four ads related to the helping professions to class. The instructor will distribute “Bingo” cards (Brown undated) and describe instructions for playing “Drug Bingo” available at www.Pharmedout.org.

Activity 3

Watch the presentation “Overview of Pharmaceutical Industry Marketing Practice,” www.cme.ucsf.edu/cme by Lisa Bero. This is an excellent overview. The entire course is free, paid for by funds from the successful lawsuit against Pfizer for false marketing of Neurontin resulting in harmful off-label uses. Discuss as a class.

Activity 4

Review the website www.concerta.net. Are any dubious methods used? Or review the website of www.abilify.com. Is any thing missing? Discuss as a class.

106 Recognizing Propaganda Gambrill & Gibbs 107

Practice Exercise 5 Human Services Advertisement Spotting Form

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Please answer the following questions by circling your responses. The presentation …

1. Argues that some assessment or intervention method works Yes No

2. Makes vague claims, such as “Clinical studies show Yes No that _____,” “x gives you more,” “helps control,” “is better than”; claims uniqueness but offers no evidence

3. Uses testimonials as evidence (statements by those who Yes No claim to have been helped by a program)

4. Appeals to emotions such as sympathy or fear via images, Yes No music, or strikingly attractive or unattractive people and/or locations

5. Presents case examples as evidence (e.g., a professional Yes No describes an intervention used and how a client responded)

6. Minimizes or does not mention the possibility of harmful Yes No (iatrogenic) effects

7. Uses a speaker whose presentation is polished and attractive Yes No

Gambrill & Gibbs Exercise 5 Critically Appraising Human Services Advertisements 107 108

8. Uses a well-known or high-status person (e.g., physician), Yes No implying the claim is true because the person says it is

9. Is repeated often and in many venues Yes No

10. Uses misleading graphs/tables Yes No

11. Uses slogans Yes No

12. Urges you to “get on the bandwagon” Yes No

13. Presented evidence for and against the program Yes No

14. Encouraged you to think carefully about the effectiveness Yes No of the method

Score: Your instructor will provide scoring instructions. Score _____

108 Recognizing Propaganda Gambrill & Gibbs 109â

EXERCISE 6â DOES SCARING YOUTH HELP THEM “GO STRAIGHT?”

Purpose

To be learned as you do the exercise.

Background

The Juvenile Awareness Program at Rahway prison in New Jersey has served as a model for many similar programs. The program is run by lif- ers, who are inmates serving a life sentence. The program is intended to prevent delinquency.

Instructions

1. View and take notes on the videotape shown by your instructor. 2. Afterward, read the situation in Practice Exercise 6 and record you answers to the three questions regarding the video you watched.

109 110 111

Practice Exercise 6 “Scared Straight”

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ __ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _

SITUATION Assume you have taken a job as a probation–parole officer working with juvenile clients adjudi- cated by a local juvenile court. Your supervisor has asked you to view some material and indicate whether juveniles served by your agency should participate in a program such as the one in “Scared Straight.”

1. What is the main claim made in the videotape you watched? __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _

2. What kind(s) of evidence are provided? Please identify as many kinds as you see. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _

3. Would you, based purely on what you have seen in this videotape, recommend that your agency try such YES NO a program with its clients? (circle one)

Please explain the reasons for your answer below. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

SCORE_____. Your instructor will provide scoring instructions.

Gambrill & Gibbs Exercise 6 Does Scaring Youth Help Them “Go Straight?” 111 112

Follow-Up Questions for Discussion 1. What is the main form of evidence provided in this videotape? 2. Are other kinds of evidence available about this program? Please describe, including sources, and make a final recommendation to your supervisor. 3. Do you think this exercise is a valid test of critical thinking (see Gibbs et al., 1995)? (check one) __ _ Yes __ _ No Please explain your answer.

112 Recognizing Propaganda Gambrill & Gibbs 113â

EXERCISE 7â DETECTING MISLEADING PROBLEM FRAMING

Purpose

To increase awareness of misleading problem framing.

Background

There are great stakes in how problems are framed, and people with vested interests devote considerable time, money, and effort to influence what others believe. Sociologists highlight the social construction of problems—âwhy some are selected for attention and others are ignored—âand how social, political, and economic factors shape how they are framed. (See the discussion of the social problems industry in Part 1.) Problem selection and framing is influenced by professionals’ interest in maintaining and gaining power, status, and economic resources as well as differences of opinion about what makes one explanation better than another. For example, the more problems are medicalized, the more pills can be sold. Payer (1992) coined the term disease mongering to describe the confluence of interests among some doctors, drug companies, patient advocacy groups, and the media in exaggerating the severity of illness and the ability of drugs to “cure” them. The medicalization of problems includes various forms of disease mongering such as transforming common problems-âin-âliving into medical illnesses, viewing mild concerns as serious, exaggerating prevalence, use of words such as insidious, and claiming undertreatment and underdiagnosis. The coffers of helping professionals grow rich by the medicalization of problems. Anxiety and depression are viewed as “mental disorders,” focusing on individuals as the source of their own distress ignoring environmental contributors such as impoverished neighborhoods and the focus in our society on consumerism and making money. Some argue that the prime function of mental health professionals is to encourage values compatible with a neoliberal capitalistic culture (e.g., Moncrieff, 2008a).

113 114

Detecting propagandistic problem framing is a challenge because such framing is ever present in multiple sources. Consider the endless promotion of “wellness” and health risks in our society. Although a well- read person in an area may recognize censorship of well-argued competing perspectives, those who are not well informed are unlikely to do so. Given that problem framing guides selection of assessment/diagnostic measures and interventions, learning how to avoid being bamboozled, and so bamboozling clients, regarding problems and their framing is vital. Reporting guidelines designed to enhance the quality of research reports such as CONSORT focus on methodological considerations; they do not address concerning problem framing such as the medicalization of problems-in-living (Gambrill, 2015). Concerns about the intrusion of marketing values into health has become so extensive that a backlash has occurred. For example, the first international conference on disease mongering was held in 2006 in Australia. The following exercise will give you some practice in detecting misleading problem framing.

114 Recognizing Propaganda Gambrill & Gibbs 115

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Activity 1 Watch “Century of the Self” by Adam Smith (2002) and be ready to discuss as a class. What did you learn from watching this? What questions do you have?

Activity 2 Your instructor will select a randomized controlled trial report (RCT) regarding anxiety. Read the introduction in this report and underline any instance of misleading discourse. Return this copy with the underlining to your instructor. Review the introduction again using a filter provided by your instructor. Compare results of 2 and 4 following your instructor’s guidelines and discuss implications, including ethical concerns.

Activity 3 Select a problem of interest to you and a couple of other students. Describe current framing as well as any unacknowledged controversies.

Activity 4 Describe a practice or policy in your agency that compromises quality of services for clients. Examples include lack of careful evaluation of outcome, discouraging those who raise questions, and paperwork that does not contribute to helping clients. What reasons do administrators give for continuation of this policy? Are they sound? Describe any propaganda methods used.

Activity 5 Can you identify an example of institutional corruption encouraged by misleading problem framing in your local community (see Part 1)? If yes, please be ready to discuss this in class.

Activity 6 Addiction is often viewed as a brain disease. Read about the rat park experiment (Alexander, Beyerstein, Hadaway, & Coambs, 1981) and be ready to discuss the implications of this experiment regarding the claim that addiction is a brain disease. You could also listen to the audiotape of Bruce Alexander’s lecture upon acceptance of the 2007 Sterling Prize in Support of Controversy and read “The Myth of Drug Induced Addiction” (Alexander & Wong, 2010, www.brucealexander.com).

Activity 7 Read material on National Anxiety and Depression Awareness Week (www.freedomfromfear.org) and on the website of the Council For Evidence-Based Psychiatry (www.cepuk.org). Be ready to discuss differences in framing and implications. Related questions include the following: Why do

Gambrill & Gibbs Exercise 7 Detecting Misleading Problem Framing 115 116

you think there is an epidemic of “mental illnesses” in the developed world? Is there a relationship between use of psychiatric medications and violent behavior (see Stolzer, 2013)?

Activity 8 Review Selling Sickness by Moynihan and Cassels (2005) and be ready to discuss methods used by Cohn & Wolfe, a public relations agency hired by GlaxoSmithKline to lay the groundwork for the introduction of Paxil. Were any public relations methods used such as front groups, lobbying, “spin” (e.g., managing language), TV ads, mass mailings, and celebrity endorsement?

Activity 9 Assume you are a parent of a six-year-old child and are told by his teacher that his difficult behavior in class requires that be placed on a prescribed medication (Concerta). What would you do and why? Log on to the Lakota People’s Law Project (http://lakotapeopleslawproject.turm) and review content on the use of psychotropic medicine with foster children. Be prepared to discuss what professionals can do to discourage inappropriate use of psychotropic medication (see, for example, www.Pharmedout.org and www.CriticalThinkrx.org). (See also Cohen, Lacasse, Duan, & Sengelmann, 2013.)

Activity 10 Read Layng (2009) and be prepared to discuss as a class.

116 Recognizing Propaganda Gambrill & Gibbs 117â

EXERCISE 8â FOLLOWING THE MONEY

Purpose

To increase your awareness of the influence on the helping professions of special interests and related harms.

Background

As discussed in Part 1, the helping professions ability and related industries are big businesses. For example, sales of abilify in 2014 were 9.2 billion (ACS, Chemical Neuroscience, 2015). Many people are naive about the profit-âmaking side of the helping professions that may compromise quality of services. Consider, for example, the unnecessary force feeding of elderly patients in for-âprofit nursing homes to charge higher rates and save staff time (Warraich, 2015). This exercise provides an opportunity for you to increase your awareness of the role and consequences of profit and status seeking in the helping professions.

Activity 1

Select an example of interest to you and a couple of other students, follow the money, and prepare a follow-âthe-âmoney report to share with your class (see Practice Exercise 8).

Activity 2

How much funding does the Depression and Bipolar Alliance receive from pharmaceutical companies? Write the amount here: _â_â_â_â_â_â_â_â_â_â_â_â_. â Discuss possible consequences as a class.

117 118

Activity 3

Read the buzzfeed news investigation describing deaths, sex abuse, and blunders in screening and overseeing of foster parents (Roston & Singer- vine, 2015). How many children were involved? Who knew about the harms but said nothing? What happened to those in charge? Were they ever prosecuted? Describe what can be done to minimize related harms in the future.

Activity 4

Read Part 1 of an investigative series by J. Harlick (2015): “The Eating Disorders ‘Residential Treatment Industrial Complex.’ ” How effective are such programs? Cite best sources.

Activity 5

Read investigative reports in The Chicago Tribune regarding state run treatment centers (e.g., Eldeib & Jackson, 2014; Jackson & Eldeib, 2014; Jackson & Marx, 2014; Jackson, Marx, & Eldeib, 2014a, 2014b). What exactly is meant by “harsh treatment?” How many residents were in these facilities in the latest year of operation? Who knew about harm but said nothing? Suggest remedies for decreasing such harms. (See also Illinois Corruption Watch, Dec. 2014, www.illinoispolicy.org).

Activity 6

Read “Prisoners Pay Millions to Call Loved Ones Every Year: Now This Company Wants Even More” (Walsh, 2013). Watch the YouTube video “Prison Bankers Cash in on Captive Customers” (Rabuy, 2014). Is this a fair arrangement? Describe any recent changes in related practices.

Activity 7

Read “As IRS Takes Aim at False Social Welfare Organizations, Will Some in Congress Take Aim at the IRS?” (Rosenberg, 2013). Are there

118 Recognizing Propaganda Gambrill & Gibbs 119

fake social welfare organizations? If yes, identify one and indicate how much money this organization made last year; cite your sources. Are there fake charities? If yes, please name one. How much money did it make in the last year of operation? Who knew the charity was bogus but did nothing?

Activity 8

Watch “The Business of Recovery” (Finberg, 2016) and discuss as a class. How effective are related recovery programs? Cite best source. Read The Sober Truth: Debunking the Bad Science Beyond Twelve-Step Programs and the Rehab Industry by Dodes and Dodes (2014).

Activity 9

Are the health services offered in a jail or prison near you of high quality? Describe your sources of information. If services are of poor quality, what could be done to increase quality?

Activity 10

Discuss activities and related consequences of the International Franchise Association to maintain low wages of homecare attendants (see Strauss, 2015, p. 17). How many homecare attendants are there in the United States? What is the average hourly wage of in-home attendants? Is this a living wage? Describe how home care attendants’ low wages may affect quality of health services

Activity 11

Organizations such as the European Medicines Agency and the Pharmaceutical Research and Manufacturers of America work to maintain secrecy of trials. The AllTrials campaign presses for all trials to be registered (see www.alltrials.net). Do you think trial data should be shared with all parties, including clients?

Gambrill & Gibbs Exercise 8 Following the Money 119 120

Activity 12

Read “A ‘Three-Quarter’ Home Landlord in Brooklyn Faces Illegal Eviction Charges” (Barker, 2015) and discuss related issues. Does inaction by regulatory agencies contribute to avoidable harms? Please give an example.

120 Recognizing Propaganda Gambrill & Gibbs 121

Practice Exercise 8 Follow the Money Report

Your Name__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ Date__ _ _ _ _ _ _ _ _ _ _ _ _

Course__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name__ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _

1. Identify and describe an organization/agency of interest. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

2. Describe the major goal of the agency/organization (attach relevant material, for example, from its website). __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

3. What is the agency’s yearly funding? __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

4. Describe funding sources. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

5. How many clients were served last year? __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 8 Following the Money 121 122

6. Describe how outcomes are measured. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

7. Is the money being well spent? ___ yes___ no___ Please describe the reasons for your answer. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

122 Recognizing Propaganda Gambrill & Gibbs 123â

EXERCISE 9â THE LANGUAGE OF PROPAGANDA

Purpose

To increase your skill in detecting the role of language in propaganda.

Background

Language exerts a powerful influence on what we do, feel, and believe, and propagandists take full advantage of this (e.g., slogans, jingles, “fighting words,” weasel words), as can be seen in marketing-âbased medicine, nursing, social work, psychology, and psychiatry (in contrast to evidence-â informed). Doublespeak refers to “language that deliberately disguises, distorts, or reverses the meaning of worlds” (Wikipedia). The term spin refers to inaccurate views of products/âevents. Language ploys are shown in Box 9.1. Weasel words give an illusion that something informative is being said, as in: “It has long been known,” (meaning “I didn’t look up the reference),” “In my experience” (meaning, once), “It is generally believed that,” and “A couple of others think so too” (Gregory, 2008). In equivocation, words have multiple meanings and it is unclear which meaning is intended. Many fallacies described in later exercises involve language (labels, emphasis, euphemisms, repetition). Avoidable confusion is fos- tered by unclear writing, including use of “plastic words” such as development and strategy, which sound scientific but blur meaning (Poerksen, 1995). “The great enemy of clear language is insincerity. When there is a gap between one’s real and one’s declared aims, one turns as it were instinctively to long words and exhausted idioms, like a cuttlefish spurt- ing out ink” (Orwell, 1958, p. 142). Orwell (1958) introduced the term “newspeak” in his novel Nineteen Eight-âFour—âa tool to limit freedom of thought that threatens the powers that be. Coombs and Nimmo (1993) describe palaver as a kind of discourse in which truth and falsity are irrelevant. This is similar to Frankfurt’s (1986) concept of “bullshit” in which the purveyor does not know and does not care about the evidentiary status of what he or she is talking about. Truth

123 124

Box 9.1 Language Ploys

Eeuphemisms Use of an inaccurate word to refer to something offensive, such as referring to killing big game as “conservation” Reification: Acting as if an abstract term really exists Influence via emotional words Use of pseudotechnical jargon (biobabble, psychobabble) Misuse of speculation: assuming that what is can be discovered by merely thinking about it Confusing verbal and factual prepositions, trying to draw a conclusion about what is from a study of how a word is used Use of statements in which all is implied, but only some are true Being vague (using weasel words, ambiguity). Equivocation: use of words that have multiple meanings Asking leading questions (the question implies the answer) Insisting on a specific definition that oversimplifies the situation Using loaded questions Repeating phrases and terms Using slogans Using labels Using metaphors and similes Shifting meaning

is irrelevant. Frankfurt (1986, 2005) thus distinguishes between the liar— someone who knowingly deceives others –and the “bullshitter,” who simply does not care. Here again, we see the vital role of purpose in thinking and dialogue: What is the purpose of the speaker or writer (Walton, 1995, 2008)? There is an epidemic of management speak (Ali, Sheringham, & Sheringham, 2008). Stivers (2001) emphasizes the “magical” use of language in viewing a change in management as evidence of attainment of hoped-for outcomes. Vague terms are often used to conceal ignorance and/or hide plans (e.g., see rationalwiki.org, Lewis, 2013; Poole, 2013), “50 Office Speak Phrases You Love to Hate” (news.bbc.co.uk). Items in the latter include “going forward” and “deliver.” Here at the University of California at Berkeley, administrators now talk of “deliverables.” Some of my least favorite terms are “join up,” “incentivize,” and “leverage.”

Activity 1

Select content from a website, agency, or university report or published article in which you and a couple other students are interested. Underline

124 Recognizing Propaganda Gambrill & Gibbs 125

and identify any instances of the propagandistic use of language. Discuss as a class.

Activity 2

Form groups of four students. Ask each group to prepare a skit relevant to the helping professions illustrating the misuse of language. Ask other class members to “catch” ploys used. Compare notes after each skit.

Activity 3

Review sources describing “management speak” (e.g., Lewis, 2013; Poole, 2013). Examples include “ballpark figure,” “deliverables,” “taking it to the next level.” Download Buzzword Bingo Cards 1 and 2 (www.bollock- sphere.co.uk). You could play this game by yourself, with a few class mates or your instructor could arrange for you to play as a class. Track how often “management speak” is used in your classes or agency. Does this kind of language help clients?

Activity 4

Review Mysterious Phrases Explained (www.gcfl.net). Keep track—for a week—of how often you see these terms in your readings.

Activity 5

Review the examples in Rubin and Parrish (2007) and discuss possible effects as a class.

Gambrill & Gibbs Exercise 9 The Language of Propaganda 125 126 127â

PART 3 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making

Let’s say you attend a conference to learn about a new method for helping clients, and the presenter says you should adopt the method because it is new. Would that be sufficient grounds to use the method? What if she described a few clients who had been helped by the method? Would you use the method? Or let’s say that staff members who manage a refuge for battered women tested residents’ self-âesteem before and after residents participated in a support group and found that the women scored higher after taking part in the support group. Can we assume the support group caused an increase in residents’ self-âesteem? Last, let’s say the leader of an interdisciplinary team encourages the group to arrive at a unanimous decision about whether a child requires special education services. Can we assume that because no one raised objections that important evidence and relevant arguments have been heard? In the first situation, the presenter encourages acceptance of a method because it is new (appeal to newness). In the second example, acceptance is encouraged by describing a few selected instances (reliance on case examples). In the third, staff members assume that because improvement followed intervention, the intervention caused improvement (the post hoc-âergo-âproc fallacy). In the final example, group members may not share dissenting opinions because they fear upsetting group cohesion (groupthink). You will learn

127 128

about such fallacies as well as cognitive and affective biases that can result in avoidable errors as you engage in the exercises in Part 3. Literature in four major areas can help us to identify fallacies and biases that influence decisions: (1) philosophy (especially concerning critical thinking and informal logic); (2) psychology, including relevant social–psychological studies as well as research on judgment, problem solving, and decision making; (3) sociology (especially the study of political, social, and economic influences on what problems are selected for attention and how problems are defined); and (4) studies of clinical reasoning, decision making, and judgment in the helping professions, including corrupting influences such as conflicts of interest. The exercises in Part 3 draw on this literature.

Fallacies

Context is vital in reviewing related dialogue; for example, is critical appraisal of a claim the key interest? Many fallacies have been recognized for thousands of years. For example, ad hominem refers to attacking a person rather than critically examining their argument (e.g., Damer, 2005; Engel, 1994; Walton, 1995). (See Wikipedia’s list of fallacies.) In Walton’s pragmatic view of fallacy, “a fallacy is not just a weak or questionable argument, but an argument used to block or interfere with the legitimate goal of the type of dialogue the arguer is supposed to be engaging in” (Walton, 1997, p. 250). Fallacies of ambiguity refer to a confusing use of language (e.g., vagueness). In fallacies of presumption, unfounded assumptions are inserted into an argument; facts may be evaded or distorted. Fallacies that overlook the facts include hasty generalization and sweeping generalizations. Testimonials and case examples may be used to make generalizations, for example, as described in Exercise 11. Other fallacies include evading the facts and begging the question. Some fallacies distort facts, as in false analogy and false cause. Fallacies of relevance include appeal to emotion and ad hominem arguments. Damer (1994) divides fallacies into (1) those that violate the relevance criterion, such as appeal to questionable authority, force, or pity; (2) those that violate the acceptability criteria, such as linguistic confusion, question begging, and assumptions such as the is–ought fallacy; (3) fallacies that violate the sufficient grounds criteria, such as fake precision and irreproducible data; (4) causal fallacies, such as oversimplification and

128 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 129

confusion of cause and effect; and (5) fallacies that violate the rebuttal criteria, such as denying or ignoring counterevidence, ad hominem, and strawperson arguments.

Cognitive Biases

We are also prone to making a variety of cognitive and affective biases. There are more than one hundred cognitive biases that may result in avoidable errors (e.g., Croskerry, 2003) (see Box 3.1). Biases and fallacies can intrude at any point in the judgment process. The acquisition of information may be biased; how we direct our attention influences what we see (and what we miss). For example, we are subject to confirmation biases in which we seek support for preferred assumptions. Second, how we process information may be biased (we may not consider vital cues). Third, bias may be introduced by how we respond (an agency may require use of a misleading recording form) (Hogarth, 1980). “Finally, the outcomes of our judgments can create bias in both: (1) interpretation of their significance (for example, is the outcome attributable to one’s actions or simply a chance fluctuation?); and (2) learning relationships for predictive validity” (p. 158). There are individual differences in susceptibility to errors and biases. Tetlock (2003) found that respondents who valued closure and parsimony highly were more prone to biases that were rooted in excessive faith in the predictive and explanatory power of their preconceptions—biases such as overconfidence, cognitive conservatism, certainty of hindsight and selective standards of evidence and proof … more “open-minded,” lower-need- for-closure respondents … wound up being too imaginative and assigning too much subjective probability to too many scenarios … . (p. 234)

Propaganda ploys make use of fallacies and biases (See Box 3.1). (see also Wikipedia for lists of fallacies and cognitive biases as well as www. fallacyfiles.org and Thouless [1974]). Cognitive, motivational, and affective biases as well as fallacies such as equating correlation and causation, may result in avoidable errors. We are most likely to miss fallacies and biases in situations in which we are biased for (or against) a certain point of view. You can learn how

Gambrill & Gibbs Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making 129 130

BOX 3.1 EXAMPLES OF FALLACIES AND BIASES

1. Ad hominem, in which the person is focused on rather than the argument. 2. Anchoring and insufficient adjustment 3. Appeal to unfounded authority (Walton, 1997), uncritical documentation (relying on citation alone) 4. Affective bias arguing from emotion, appeal to pity/anger; using emotional language 5. Arguing from ignorance—assuming that an absence of evidence for an assumption indicates that it is not true 6. Assuming hard-headed; therefore, hard-hearted 7. Availability heuristic—overestimating the likelihood of events with greater “availability” (e.g., in memory) 8. Bandwagon 9. Base rate neglect, ignoring prior probability; ignoring prevalence rate 10. Begging the question (Walton, 1991) 11. Case example 12. Commission bias—he tendency toward action rather than inaction 13. Confirmation bias—searching only for confirming evidence; focusing on successes only—not objective 14. Confusing cause and effect (e.g., Does depression cause drinking or does drinking cause depression?) 15. Confusing correlation and causation 16. Contrast effect—the enhancement or reduction of a stimuli when compared with a recently observed, contrasting object 17. Diagnostic momentum—once a diagnosis or label is attached to clients, it tends to “stick” and other possibilities are ignored (see also anchoring) 18. Diversion, red herring 19. Dunning-Kruger effect—the tendency for unskilled individuals to overestimate their own ability 20. Ecological fallacy—assuming that something true for a group is true of an individual 21. Either–or, only two alternatives, false dilemma 22. False consensus effect—overestimating the degree to which others agree with you 23. Fallacy of accident—applying a general rule to a person or situation to which it does not apply 24. Fallacy of composition—assuming what is true of the parts is true of the whole 25. Fallacy of labeling 26. Framing effects—drawing different conclusions depending on how the same information is presented 27. Fundamental attribution error 28. Gambler’s fallacy 29. Groupthink 30. Hasty generalization, biased sample, sweeping generalization 31. Hawthorne effect—we behave differently (make ourselves look better than we really are) when we know we are being observed 32. Hindsight bias (“I knew it would be so”); hindsight does not equal foresight

130 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 131

33. Illusion of control—the tendency to overestimate degree of influence over external events 34. Illusion of validity—belief that further information creates additional relevant data for predictions even when it does not 35. Insensitivity to sample size—the tendency to underexpect variation in small samples 36. Illusory correlation—inaccurately perceiving a relationship between two unrelated events 37. Is–ought fallacy—assuming that because something is the case, it should be the case 38. Jargon 39. Law of small numbers 40. Leading, loaded, biased questions 41. Manner, style, charisma, stage presence. 42. Naturalism bias—a preference for natural over artificial products even when the two are identical 43. Mere exposure effect—expressing undue liking for things because of familiarity 44. New, newness, tradition 45. Omission bias—judging harmful actions as worse than equally harmful omissions (inactions) 46. Outcome bias—judging a decision by its eventual outcome instead of based on the quality of the decision at the time it was made 47. Oversimplification 48. Overconfidence—excessive confidence in one’s own views 49. Overlooking regression effects, regression to the mean, regression fallacy 50. Popularity, peer pressure, bandwagon, appeal to numbers, because everybody … 51. Post hoc ergo propter hoc, after this, therefore because of this 52. Premature closure—accepting prematurely a case formulation or diagnosis 53. Representativeness—making decisions based on similarity (e.g., believing causes are similar to their effects) 54. Selection bias, biased selection of clients 55. Semmelweis reflex—tendency to reject new evidence that contradicts a paradigm 56. Slippery slope—assuming (mistakenly) that if one event occurs, others will follow, when this is not necessarily true 57. Social desirability bias—overreporting socially desirable characteristics or behaviors in oneself and underreporting socially undesirable characteristics or behaviors 58. Status quo bias—the tendency to like things to stay the same (related to loss aversion, endowment effect, and system justification) 59. Stereotyping 60. Strawperson argument 61. Sunk costs—the more we invest in a particular diagnosis (or outcome), the less likely we may be to consider alternatives 62. Tautology, word defines itself 63. Testimonial 64. Two questions, double-barreled question, ambiguous 65. Vagueness, unclear term, undefined term, vague outcome criterion 66. Wishful thinking

See also Box 2.1.

Gambrill & Gibbs Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making 131 132

to avoid common fallacies and biases by becoming familiar with them and developing strategies to avoid them, such as active, open-minded thinking (e.g., questioning initial assumptions; see Part 1). Other debiasing strategies include ongoing training, arranging feedback regarding decisions, decreasing time pressures, altering the task environment (e.g., using a checklist [Gwande, 2009]), decreasing excessive workloads, and using decision aids such as apps and decision algorithms (e.g., Croskerry, Singhal, & Mamede, 2013; Informed Medical Decisions Foundation [www.informedmedicaldecision.org]).

Warm-up Example

A state human service agency licenses foster homes and places children in them. One worker makes this comment about a coworker: Ms. Beyer forms impressions of potential foster homes very early. Once she forms an impression, she never budges from it. She bases her initial impression on her own housekeeping standards (whether the potential foster home smells and looks clean). She seems to ignore the parent’s ability to care for the kids, criminal records, references from others in the community, how the foster parent’s own children have adjusted, and so on.

What’s wrong here? Initial impressions “anchor” all that goes after. No matter what new evidence emerges, the initial impression prevails. This kind of faulty reasoning is called anchoring and insufficient adjustment. It encourages premature closure that may prevent the discovery of helpful data and alternative perspectives (Chapman & Elstein, 2000; Kassirer & Kopelman, 1991).

About the Exercises

The exercises in Part 3 describe fallacies, biases, and pitfalls as well as suggestions for avoiding them. By illustrating each one and encouraging active participation, we hope you will enhance your skills in spotting and avoiding fallacies and biases in your work with clients. Exercise 10, Using the Professional Thinking Form, is the only exercise in Part 3 that

132 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 133

does not require group participation. You can use this form to evaluate what you have learned in Part 3 by completing it both before and after Exercises 11 to 13. Games A, B, and C concern, respectively, common practice fallacies, faulty reasoning related to group and interpersonal dynamics, and cognitive and affective biases in practice. The fallacies and biases in Game A are grouped together because they are possibly the most universal and deceptive. Many involve partiality in use of evidence (e.g., case example, testimonial, focusing only on successes). Those in Game B describe fallacies often occur in team meetings and committees. Additional cognitive and affective biases are illustrated in Game C, drawing on research on judgment and decision making. Many others could be added, such as the ecological fallacy (assuming what is true for a group is true for an individual) and biases encouraged by emotional reasoning (e.g., creating anger or empathy; see Part 2). Sources of bias on clinical decisions include gender, ethnic, racial, sexual orientation, and social class biases (e.g., Garb, 2005). The situations in the games are taken from real life. The purpose of these games is to become familiar with common fallacies and biases that compromise sound decision making and acquire skills in avoiding them Merely knowing about fallacies and biases may not help you to avoid them. Practice in spotting and countering fallacies is vital, as is the use of debiasing strategies. The Reasoning-in-Practice Games will start you along this path. Completing Exercises 11 to 13 paves the way for a Fallacies Festival (Exercise 14) in which you team up with two other students and act out an original, thirty- to sixty-second script illustrating one fallacy. Vignettes can be videotaped and posted on the class website. The vignettes entertain best if actors ham it up, wear outlandish costumes, add props, and humorously overstate practice situations. Fallacy Spotting in Professional Contexts (Exercise 15) asks you to select an example of faulty reasoning, note its source, and explain why it is a fallacy. Exercise 16 describes indicators of groupthink and offers practice opportunities in detecting and avoiding them. Although we encourage you to have fun with the exercises, we also ask you to remember that the kinds of decisions involved are serious business, such as deciding whether sexually abused siblings should be placed for adoption in the same home or in homes distant from each other, or whether a speech therapist working with a child with cerebral palsy who cannot speak should use a particular augmentative procedure (computer, signing, picture pointing) to help the child.

Gambrill & Gibbs Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making 133 134

General Instructions for Games A, B, and C (Exercises 11–13)

Please read these general instructions before completing Exercises 11 to 13. In the three Reasoning-in-Practice Games, two or more teams compete. Working in teams allows learning from each other. The goal of each team is to identify the fallacies or biases in the vignettes. Either a narrator in each group reads a vignette aloud or participants act it out. Games last about sixty minutes. If time is limited, you can set a predetermined time limit to end the game or resume the game later.

1. Study the “Definitions” section for the game you want to play. Imagine how the fallacy or bias and its countermeasures might apply to your clients and to your work with fellow professionals. The games work best with four to six participants in a group. We recommend that as many persons as possible get a chance to read aloud and act out parts in starred (*) vignettes. The vignettes can be made into individual cards by copying the workbook pages onto cardstock and then cutting them apart. 2. Select a moderator from the class to serve as referee, timekeeper, and answer reader. (Your instructor may elect to be moderator.) Before the game, the moderator makes sure all groups agree on some small reward (actual or symbolic) to be given to the most successful group. Possible incentives include help with a task. For example, the high scorers could give the low scorers five minutes of help reviewing fallacy definitions.The moderator needs (1) a watch or timer that counts seconds, (2) access to the game’s answer key in the Instructor’s Manual, and (3) a pencil and paper to record and periodically announce group points as the game progresses. The moderator also reminds participants to shield their answers so others cannot see them. If the class contains eighteen students, the moderator can divide the class into thirds, starting at any point, by counting off “one, two, three.” When all have counted off, different groups go to different parts of the room, far enough away so that within-group discussions are not overheard by members of other groups. If the class contains more students, the moderator can divide the class into groups (about four to six in a group) so that Group A can compete against Group B, Group C against Group D, and so on. If the noise gets too distracting, groups can conduct their games in other classrooms (if available) or in the hallway.

134 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 135

3. Each group picks a leader. Participants should sit in a circle facing each other, far enough away from other groups so as not to be heard. 4. When participants are ready, they take turns either reading or acting out the vignettes. Starred (*) items are acted out; unstarred items are read. Ham it up if you like, but stick to the text. 5. Participants then have two minutes to write down their chosen fallacy from the list of fallacies in a game. Each participant should place his or her game card facedown so others cannot see it. Participants do not discuss the item’s content at this time. 6. When all group members have made their decisions, they share their choice with others in their group. 7. After the two minutes are up, each leader tells the moderator whether their group is unanimous or has a disagreement. The moderator then consults Box 3.2 to determine which group gets what points. The moderator gives points for unanimity only if the group’s choice agrees with the answer key in the Instructors’ Manual. 8. If both teams have some disagreement, each group talks privately to arrive at a choice. Each group’s leader should try to ensure all members of his or her group get a chance to express an opinion. After a maximum of three minutes of discussion, the leader takes a vote, notes the majority choice, and places the card facedown on the table, where it remains until the leader of the group signals that his or her group has also made a choice. The leaders then show the moderator their choice. 9. If the leaders mark the correct choice, all groups receive five points (see Box 3.2).

BOX 3.2 AWARDING POINTS WHEN THERE ARE TWO GROUPS

Without discussion Either Group A or Group Neither group has agreement on the among group members, B agrees on the correct correct fallacy. when all in each group fallacy but one group Both groups get up to two min- show each other their does not. The group with utes more to discuss which fallacy selection and all agree agreement on the correct to pick. Groups with the correct on the correct fallacy fallacy gets five points. answer get five points. number in both Group A and Group B. Each group gets five points.

Gambrill & Gibbs Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making 135 136

10. This process continues until all the vignettes are finished, until the class runs out of time, or until one group gets a certain number of points. The instructor may also decide that whoever has the most points at some predetermined time limit is the winner. 11. At the end of each game, all groups may be rewarded for participating, but the winning group should get the greater reward. These procedures and rules are only suggested. If your group can agree on changes that make the game more fun, go for it! Please send an e- mail to the first author describing changes that can improve the game. (If you play a game by yourself, you could work through each vignette and keep a score of your “hits” and your “misses.”) You could also practice responding to each item and compare your responses with suggestions provided by your instructor.

136 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 137â

EXERCISE 10â USING THE PROFESSIONAL THINKING FORM

Purpose

To test your skill in identifying common practice fallacies and to help you to identify fallacies in reasoning about practice.

Background

The Professional Thinking Form is designed to evaluate your skill in spotting fallacies that cloud thinking in the helping professions. Each vignette describes an example of thinking in practice. Some involve a fallacy; others do not. Vignettes include examples of decisions related to individuals, families, groups, and communities in various areas including health, mental health, child welfare, chemical dependency, and research.

Instructions

Each situation describes something that you may encounter in practice. 1. Consider each situation from the standpoint of critical, analytic scientific thinking. 2. In the space provided, write brief responses, as follows: a. If an item is objectionable from a critical standpoint, then write a statement that describes what is wrong with it. Items may or may not contain an error in thinking. b. If you cannot make up your mind on one, then mark it with a question mark (?), but leave none blank. c. If you are satisfied with the item as it stands, mark it “OK.” Please write your main point(s) as concisely as possible. The form takes about thirty minutes to complete.

137 138 139

Practice Exercise 10 The Professional Thinking Form*

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

SITUATIONS FROM PRACTICE 1. “Did you attend the workshop on strategic family therapy? Marian Steinberg is an excellent speaker, and her presentation was so convincing! She treated everyone in the audience like colleagues. She got the whole audience involved in a family sculpture, and she is such a warm person. I must use her methods with my clients.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

2. “Have you heard of thrombolytics [clot-dissolving medications] being given immediately after a cerebrovascular accident [stroke]? It’s a new treatment that seems to minimize the amount of damage done by the stroke, if the medication is given soon enough. The treatment has just been tried, with promising results. You ought to try it with your patients.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

* Revised by Leonard Gibbs and Joan Stehle-Werner (School of Nursing, University of Wisconsin-Eau Clare) (1996) and adapted from Gibbs (1991).

Gambrill & Gibbs Exercise 10 Using the Professional Thinking Form 139 140

3. “I know that open adoptions, in which birth parents and adoptive parents know each other’s identity and can communicate with each other, works well. I read an article in a recent professional journal that says it works.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

4. “Dr. Hajdasz, a surgeon at Luther Hospital, concerned about a recent case of MRSA [methicillin- resistant Staphylococcus aureus], has made several MRSA-positive cultures from hospital objects. He told members of our Infection Control Committee about his findings, but they discounted his views, in part because they dislike him.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

5. “I note that the authors never define the wordcodependency in their article on codependency among people who abuse alcohol. I need clarification of this term before I can understand what is being discussed.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

140 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 141

6. “I know Ms. Sanchez just completed a two-year study with random assignment of forty subjects to experimental and control groups with a six-month follow-up, to study the effects of treatment for chemical dependency here at Hepworth Treatment Center, but my experience indicates otherwise. My experience here as a counselor shows me that Ms. Sanchez’s results are wrong.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

7. Workers from the Bayberry County Guidance Clinic were overheard at lunch as saying, “You mean you don’t use provocative therapy? I thought everyone used it by now. Provocative therapy is widely used at this facility. Most of the staff is trained in its use. We have all used it here. You should too.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

8. “Dr. Noland has degrees from Harvard and Stanford. He has held the prestigious Helms Chair of Human Service Studies for ten years. He has been director of psychiatry departments in two universities and has served as a consultant to the National Institute of Mental Health. His stature supports the truth of his ideas in his book on psychotherapy.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 10 Using the Professional Thinking Form 141 142

9. “I think that we need to exercise caution when we make judgments that our efforts are helping clients. Other possible reasons may account for change. Perhaps people just mature. They may get help from some other source. Maybe they get better simply because they expect to get better.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

10. At a professional conference, a colleague leans over to you and whispers in your ear, “I don’t understand how anyone could accept an opinion from Ms. Washington. Just look at her. Her hair is unkempt. How can we accept an idea from someone who looks like that?”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

11. A director of a research consulting firm was overheard saying, “We conduct studies for agencies to determine how effective their programs are. We never agree to do an evaluation unless we are sure we can produce positive results.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

142 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 143

12. Here is a statement made by an agency supervisor to a colleague: “Michelle is one of the most difficult staff members to deal with. I asked her to decide between supporting either nutritional or healthcare programs to meet the needs of the elderly here in Dane County. She responded that she needed some time to get evidence to study the matter. She said that there may be other alternatives for our resources. As I see it, there are only two ways to go on this issue.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

13. At a professional conference, Dr. McDonald asked a family who had participated in Strategic Family Therapy to tell the audience how the method worked for them. The husband said to the audience, “Frankly, I didn’t think we had a prayer of saving our marriage. Dr. McDonald requested, my wife and I to bring in our thirteen-year-old, David, and our eleven-year-old, Emily, with us to counseling. We do exercises as a family. Last time we went on a treasure hunt with me as a leader. His exercises teach us about our family system. The methods have really helped us, and I highly recommend them to you.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

14. Shortly after the city planners announced their intent to build a vocational training facility, they were deluged with phone calls and letters from angry citizens protesting the plan. Planners were surprised that the whole community opposed the plan so strongly.

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 10 Using the Professional Thinking Form 143 144

15. “Most likely this client is depressed.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

16. Joe Armejo is a typical war veteran, like most of the clients we see at the Veterans Administration. At seventeen, he entered the marines, went through his basic training, and then “all hell broke loose,” as he tells it. Joe served in Iraq sixteen months, often in combat, with a small unit. Among those in his unit, he lost two close buddies. After being discharged, Joe drifted from job to job, seemed unable to form a lasting relationship, and descended into an alcohol addiction. Joe occasionally encountered counselors, but never opened up to them—not until he joined an Iraq War veterans’ group. After six months of weekly visits, Joe began to turn his life around. He got and held a job, and he has been dating the same woman for a while now. His dramatic change is typical of men who join such groups.

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

17. An interviewer asks the following question: “Will you be able to drive yourself to the hospital weekly and eat without dentures until January 1st?”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

144 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 145

18. An interviewer asks a female victim of domestic abuse the following question: “You don’t want to stay in a home with a violent wife-beater, do you?”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

19. “Electroconvulsive (shock) therapy is the most effective treatment for psychotic depression.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

20. “One way of describing ‘progress’ in clients seeking independence from their families is to assess their gradual increase in independence from their families.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

21. “The effectiveness of our program in family therapy is well documented. Before families enter our program, we have them fill out a Family Adjustment Rating Scale, which has a Cronbach’s alpha reliability of .98 and is validly associated with indices of sexual adjustment and marital communication. After intervention, family members fill out the Scale

Gambrill & Gibbs Exercise 10 Using the Professional Thinking Form 145 146

again. Statistically significant improvement in these scores after family therapy proves that our program is effective.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

22. A psychologist remarks to a client, “It is very difficult to work with people who have adolescent adjustment reactions. Adolescents have not had sufficient experience to reality test. This is why those who work with adolescents use existential and reality-oriented approaches.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

23. Don Jaszewski, a teacher at Parkview Elementary School, administered the Rosenberg Self-Concept Scale to all one hundred students in the school’s fifth grade. For the ten students who scored lowest, Don designed a special program to raise their self-esteem. All ten participated in a weekly rap session, read materials designed to foster self-acceptance and self-assurance, and saw Don individually at frequent intervals during the academic year. When Don again administered the Scale at the end of the program, he found a statistically significant improvement from their preintervention scores. In fact, seven of the ten students in his program scored almost average this time. Because of this evidence, Don urged the school administration to offer his program in the future.

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

146 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 147

24. Mr. Rasmussen, director of the Regional Alcoholic Rehabilitation Clinic, is proud of his facility’s success rate. The clinic draws clients who are usually leading citizens in the area and whose insurance companies are willing to pay premium prices for such treatment. He points out that 75% of those who complete this treatment, according to a valid and reliable survey done by an unbiased consulting firm, abstain completely from alcohol during the six months following intervention. In contrast, the same firm reports that alcoholics who completed intervention at a local halfway house for unemployed men have a 30% absti- nence rate for the six months after their treatment. Mr. Rasmussen says, “The difference between 75% and 30% cannot be ignored. It is obvious that our clinic’s multidisciplinary team and intensive case-by-case treatment are producing better results than those at the halfway house.”

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

25. With help from a researcher, the Cree County Social Service Department developed a screening test for families to identify potential child abusers. Experience with this test in the Cree County School District showed that, among confirmed abusers who took the test, the result was positive (indicating abuse) for 95% of couples who abused their child within the previous year (sensitivity). Also, among nonabusers the test results were negative (indicating no abuse) for 95% (specificity). Cree County records show that abuse occurs in three of one hundred families (prevalence rate of 3%) in the Cree County School District. County Social Service Department workers note that the Donohue family tested positive (indicating abuse). They conclude that the Donohue family has a 95% chance that they will abuse their child. Do you agree with this estimate? If not, what is the probability that the Donohue family will abuse their child?

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

SCORE Your instructor will provide scoring instructions. ___________________________________________

Gambrill & Gibbs Exercise 10 Using the Professional Thinking Form 147 148

FOLLOW-UP QUESTION

Do any of the Professional Thinking Form’s situations reflect real situations particularly well? Which one(s)?

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

148 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 149

EXERCISE 11 REASONING-IN-PRACTICE GAME A: COMMON PRACTICE FALLACIES AND BIASES

Purpose

To learn how to spot and avoid fallacies common across the helping professions.

Background

The fallacies and biases in this game stalk unwary practitioners in all helping professions. Watch for them creeping into thinking during interdisciplinary case conferences when participants assume a client’s improvement following intervention was caused by the intervention (post hoc ergo prop), that what may be true of one person or many is true for all (case example), or that unclear descriptions of hoped-for client outcomes offer sufficient evidence to judge client improvement (vagueness). The terms trick and stratagem refer to fallacies that are used deliberately as persuasion strategies (see Part 2). They may also occur because of sloppy thinking or lack of critical thinking skills and result in poor decisions. Thus, fallacies may be intentional or unintentional. Walton’s (1995) pragmatic view of fallacy highlights their ethically questionable role in blocking critical appraisal. When trying to arrive at accurate answers, such blocking is never appropriate. This pragmatic view of fallacy emphasizes the importance of context. Is critical appraisal of a claim the key interest? Thus, “a fallacy is defined as an argument that not only does not contribute to the goal of a dialogue but actually blocks or impedes the realization of that purpose” (Walton, 1995, p. 255). So we should always ask: What is the context of the dialogue?

149 150

Instructions

1. Read the description of each fallacy or bias. 2. See Instructions for Games A, B, and C in Part 3. Act out starred (*) vignettes and read others aloud.

Definitions, Examples, and Countermeasures 1. Relying on Case Examples This fallacy refers to drawing conclusions about many people based on only one or a few unrepresentative individuals. A generalization is made about the effectiveness of a method, or about what is typically true of clients based on one or just a few people. This is a hasty generalization and reflects the law of small numbers—the belief that because a person has intimate knowledge of one or a few cases, he or she knows what is generally true about clients. This fallacy is also referred to as the fallacy of experience (Skrabanek & Mc Cormick, 1998, pp. 56–58). Experience with a few cases may be highly misleading (see discussion of the law of small numbers in Exercise 13). Case examples often por- tray individuals so vividly that their emotional appeal distracts from seeking evidence about what helps clients or is generally true of clients. We can easily become immersed in the details of a case, forgetting that it is just one instance. Brief case examples encourage oversimplification of what may be complex problems. They encourage confirmation biases in which we seek examples that support our favored assumption and overlook contradictory evidence. If we search long enough for it, we can find a case that supports almost any conclusion. Anecdotal evidence can be important (e.g., observational evidence) when an intervention has a very large impact, as in the discovery of the cause of ulcers. (For a related discussion see Black [1994].) Case examples can be used to raise important questions about assessment and to demonstrate and teach practice skills. A videotape of an interview with an adolescent mother may demonstrate important practice competencies such as high-quality empathic skills. An instructor may model a family therapy technique. Such use of case material is a valuable part of professional education. The problem arises when we generalize to all clients from case examples. (See also the discussion of the post hoc fallacy discussed later in this exercise.)

150 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 151

Example: The Greenfield Nursing Home has been known to provide very poor care. This shows that nursing home care for the elderly is not a good option. Countermeasures: To make accurate generalizations about a population, collect a representative sample from this population. For example, to judge whether change is related to a particular intervention, search for a systematic review of well-designed experimental studies. You may find a high-quality systematic review in the Cochrane or Campbell Libraries.

2. Relying on Testimonials Claims that a method is effective may be based on one’s own experience. Testimonials are often given in professional conferences, or on film or videotape. Clients may report how much participating in a particular intervention benefited them. To qualify as a testimonial, a person must (1) assert that a given method was helpful, (2) offer his or her own experience as evidence that the method works, and (3) describe the experience, not to demonstrate how the method is applied, but to argue that the method is effective. Testimonials do not provide evidence that an intervention is effective. Although people who give testimonials may be sincere, their sincerity does not ensure accuracy. Promoters often choose people to give testimonials because the vividness of their presentation encourages gullibility. Those who give a testimonial may feel pressure to please the person who requested the testimonial; they may not be aware of the kind of data needed to determine whether change did occur and was related to the intervention (e.g., an experimental study).

Example:

After taking so many other medicines without being helped, you can imagine how happy and surprised I felt when I discovered that Natex was doing me a lot of good. Natex seemed to go right to the root of my trouble, helped my appetite and put an end to the indigestion, gas and shortness of breath. (Local Lady took Natex year ago—had good health ever since, 1935, May 27, p. 7).

This woman’s testimonial appeared on the same page of a newspaper as her obituary!

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 151 152

Countermeasures: Conduct a controlled study to evaluate the effects of the intervention or consult literature that describes such studies. Both case examples and testimonials involve partiality in the use of evidence—looking at just part of the picture. They rely on selected instances, which often give a biased view.

3. Vagueness Descriptions of client concerns and related causes, hoped-for outcomes, and progress measures may be vague. Problem-related behaviors, thoughts, or feelings may not be clearly described. Examples of vague terms include aggression, antisocial behavior, poor parenting skills, and poor communication. The Barnum Effect, in which we assume ambiguous descriptions apply to us and indicate the accuracy of advice (e.g., from astrologers), takes advantage of vague words and phrases. Common terms referring to vague accounts include bafflegab, psychobabble, biobabble, bureaucratese, and gobbledygook (Kahane & Cavender, 1998, p. 135). Vague descriptions of hoped-for outcomes and progress indicators make it impossible to clearly determine whether progress has been made. Vague terms foster fuzzy thinking and obscure the results of efforts to help clients. Examples of vague terms that describe outcomes include improved, better, coming along nicely, somewhat better, functioning at a higher level, and substan- tially improved. If the client “improved” without our defining how, how would we know whether this was the case? Examples of clear outcomes include initiating three conversations a day (a conversation is defined as more than a greeting and at least one minute long), a client with a weight problem losing ten pounds within a given six-week interval, or a client with hypertension maintaining a blood pressure of 140/80 mmHg or less at all six monthly meetings at the clinic.

Example: “Our community prevention programs have been effective. After six weeks of meetings, residents seemed to feel more in charge of their health.” Countermeasures: Clearly describe presenting concerns, related hoped-for outcomes, and progress measures. Descriptions of outcomes should be so clearly stated that all involved parties can agree readily about when they have been attained. The descriptions should answer the questions: Who? What? Where? When? How often?

152 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 153

4. Assuming Hard-headed, Therefore Hard-hearted This fallacy refers to the mistaken belief that one cannot be a warm, empathic caring person and an analytic, scientific, rational thinker. As Meehl (1973) argued, it is precisely because clinicians do care (are soft- hearted) that they should rely on the best evidence available (be hard- headed) when making decisions. Assuming that one has to be either caring or rational misses the point; a person can be both. Paul Meehl (1973) documented in 1954 that, despite the fact that statistical prediction (statistical tables based on experience with many clients) consistently out predicted judgments made by senior clinicians, helpers still relied on their gut-level feelings when making important predictions. Meehl (1973) speculated that clinicians often ignore better statistical evidence because they believe that they would be less feeling and caring about clients if they based their judgments on statistical evidence. Scores of studies support Meehl’s conclusions about the superiority of statistical prediction over gut-level (intuitive) feelings in making predictions (e.g., Grove & Meehl, 1996; see also Johnson, Clancy, & Bastion, 2015; Vaithianathan, Maloney, Putnam-Hornstein, & Jiang, 2013).

Example: Supervisor to student: I think the use of this new actuarial prediction system will interfere with caring about our clients and seeing the client as a whole person. We should not use it. Countermeasures: Be hard-headed (analytic and data informed) because you are soft-hearted (care about what helps people).

5. Confirmation Biases This fallacy refers to the tendency to look only for data that supports initial beliefs and to ignore disconfirming evidence (Nickerson, 1998). For example, we may attend only to events consistent with a preferred practice theory. This may occur with or without our awareness. We cherry- pick (Tufte, 2007). An administrator may infer a method is effective by focusing only on successes—only on those instances when improvement followed the use of a method. Failures, instances of spontaneous recovery, and persons not treated who got worse are ignored. When we examine associations to infer cause, we often rely on evidence that confirms our hypothesis—that is, those who received an intervention and improved (Cell A in Box 11.1) and ignore counterevidence (Cell B in Box 11.1). We

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 153 154

BOX 11.1 EXAMINING THE ASSOCIATION BETWEEN INTERVENTION AND OUTCOME

Client Outcome Improved Not Improved Yes Cell A successes, Cell B failures, Proportion successful Client n = 75 n = 35 A = × 100 Participated AB+ in Intervention No Cell C sponta- Cell D untreated, Proportion in neous recovery, unimproved; spontaneous recovery n = 75 n = 60 C = × 100 CD+ Source: Adapted from Gibbs, L. E. (1991). Scientific reasoning for social workers (p. 70). New York: Macmillan.

may be so committed to a view that we ignore counterarguments and evidence against it. This kind of biased (one-sided) thinking may result in decisions that harm clients.

Example: “I sought information related to my belief that the client was depressed and found many instances of depressed feelings and related indicators.” For other examples of confirmation biases, see professional advertisements, presentations at conferences by those seeking to sell a method of intervention (particularly if they want you to pay for related training), and articles that present only one point of view about an issue. Countermeasures: Question initial assumptions. Search for data that do not support your preferred view. Keep in mind that your initial assumption may be wrong. All four cells of the table in Box 11.1 must be examined to get an accurate picture of whether an intervention may be effective. In addition to considering successes, look for failures, persons not treated who got better, and those not treated who got worse. Don’t trust your memory. Keep a systematic record of successes, failures, those not treated and improved, and those not treated and not improved. The latter two groups might be estimated by reading literature about what generally happens to untreated persons. Look fearlessly at all the evidence, not just data that support a hypothesis (i.e., cases in which the treatment worked). How else can an accurate judgment be made? Be skeptical of anyone who presents just one side of anything. The world’s not that simple. Seek and

154 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 155

present well-argued alternative views and data in your own work. The more you are committed to a particular view, the more vigorously you should seek counterevidence.

6. Relying on Newness or Tradition This fallacy occurs if (1) an assertion is made about how to help clients or what is true of clients; (2) the assertion is said to be true because it has been held to be true or practiced for a long time (tradition),or because the idea or practice has just been developed (newness); and (3) no studies or data are given to support the claim. The practice of bleeding (applying leeches, cutting into a vein with a scalpel) as a treatment for infection was practiced for hundreds of years, despite the fact that there was no evidence that it worked (see Box 11.2). Conversely, the mere fact that an intervention has just been developed does not mean that it is effective. All interventions were new at some time, including ones that promised cures but were later found to be ineffective or harmful. Many popular treatments, such as the use of “magnetic” devices to cure ailments, are popular even though there is no evidence they are effective (Pittler, Brown, & Edwards, 2007; Winemiller, Billow, Laskowski, & Harmsen, 2003).

Example of appeal to tradition: A nursing home social workers says, “We have always classified our residents according to their level of nursing care on the four floors of Rest Haven. No matter what reasons you might give for changing this practice, I doubt that the administration would change a practice that has been in place for many years.” Example of appeal to newness: “This method of family therapy is described in a new book by Dr. Gerbels. It’s the latest method. We should use it here.” Countermeasures: Point out that being new or old does not make an idea or practice valid. Ask to see evidence to judge the effects of methods.

7. Appeal to Unfounded Authority (ad Verecundium) Here, there is an attempt to persuade someone to accept a claim by focusing, for example, on the “status” of an individual as an alleged expert, perhaps to block efforts to critically appraise the claim. Authority may refer to cognitive authority “which is always subject to critical questioning,” or “institutional or administrative authority, which often tends to be

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 155 156

BOX 11.2 DEATH OF GENERAL GEORGE WASHINGTON

The death of this illustrious man, by an abrupt and violent distemper, will long occupy the attention of his fellow citizens. No public event could have occurred, adapted so strongly to awaken the sensitivity and excite the reflections of Americans. No apology will therefore be needful for relating the circumstances of this great event. The particulars of his disease and death being stated by the physicians who attended him, their narrative deserves to be considered as authentic. The following account was drawn up by doctors Craik and Dick: Some time in the night of Friday, the 13th of December, having been exposed to a rain on the preceding day, General Washington was attacked with an inflammatory affection of the upper part of the wind pipe, called, in technical language, Cynanche Trachealis. The disease commenced with a violent ague, accompanied with some pain in the upper and fore part of the throat, a sense of stricture in the same part, a cough, and a difficult, rather than a painful, deglutition, which were soon succeeded by fever and a quick and laborious respiration. The necessity of blood-letting suggesting itself to the General, he procured a bleeder in the neigh- bourhood, who took from his arm, in the night, twelve or fourteen ounces of blood. He could not be prevailed on by the family, to send for the attending physician till the following morning, who arrived at Mount Vernon at about eleven o’clock on Saturday. Discovering the case to be highly alarming, and foreseeing the fatal tendency of the disease, two consulting physicians were immediately sent for, who arrived, one at half after three, and the other at four o’clock in the afternoon: in the mean time were employed two copious bleedings, a blister was applied to the part affected, two moderate does of calomel were given, and an injection was administered, which operated on the lower intestines, but all without any perceptible advantage, the respiration becoming still more difficult and painful. On the arrival of the first of the consulting physicians, it was agreed, as there were yet no signs of accumulation in the bronchial vessels of the lungs, to try the effect of another bleeding, when about thirty-two ounces of blood were drawn, without the least apparent alleviation of the disease. Vapors of vinegar and water were frequently inhaled, ten grains of calomel were given, succeeded by repeated doses of emetic tartar, amounting in all to five or six grains, with no other effect than a copious discharge from the bowels. The power of life seemed now manifestly yielding to the force of the disorder; blisters were applied to the extremities, together with a cataplasm of bran and vinegar to the throat. Speaking, which had been painful from the beginning, now became almost impracticable: respiration grew more and more contracted and imperfect, till half after eleven on Saturday night, when, retaining the full possession of his intellects, he expired without a struggle!

Source: Death of General George Washington. (1799). e Monthly Magazine and American Review, 1(6), 475–477.

more coercive and absolutistic in nature” (Walton, 1997, p. 250). Illicit shifts in dialogue may occur in which there is an “unlicensed shift from one type of ‘authority’ to another” (p. 251). The error or deception is to take an expert opinion that should be treated as presumptive, subjective, and open to critical questioning and invest it with “infallibility and finality,” even perhaps invoking “external force” to give it

156 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 157

“sanction.” The error is an unlicensed shift from one type of “authority” to another, portraying an argument as something it is not. (Walton, 1997, p. 251)

We are often reluctant to question the conclusions of a person with high status or who is viewed as an “expert” (see Walton, 1997). There are many forms of this fallacy, including appeal to tradition and expert opinion, as in “Experts agree that cognitive–behavioral methods are best.” Such appeals are often accompanied by a convincing manner. An author or presenter may appeal to his or her experience with no description of what this entails. Other sources of authority include legal, religious, and administrative (Walton, 1997).

Accepting uncritical documentation is an example of appeal to questionable authority. It refers to the mistaken belief that if an idea has been described in a book, journal, article, or newspaper, or if a reference is given following a claim, the claim is true. To be classified as uncritical documentation, literature must be cited, but no detail provided (e.g., research method used, reliability and validity of measures used, sample size), as in “This test is reliable and valid” (Trickster, 2008). Indeed, in an analysis of references used in one article, Greenberg (2009) found that most offered no support for related claims. Even the most preposterous ideas have advocates. For example, see the National Inquirer to find that Elvis still lives and that a woman revived her gerbil after it had been frozen stiff in her freezer for six months. And, keep in mind that many research findings reported in peer-reviewed journals are false (Ioannidis, 2005, 2012). Example: A master of ceremonies introduces a speaker to a professional audience: “Dr. MacMillan is one of the most renowned experts on therapeutic touch in the world. He has published three books on therapeutic touch and he now holds a prestigious William B. Day Lectureship at the University of Pennsylvania. His reputation supports what he’ll tell us about the effectiveness of his approach.” Countermeasures: Ask to see the authority’s evidence and evaluate that. How good is the evidence? Here again we see the vital role of questions. To discover whether a cited reference provides evidence for a claim; you may have to read that reference yourself. Questions suggested by

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 157 158

Walton (1997) include: Is x an expert in related fields of concern? Is x personally reliable as a source?

8. Oversimplifications Here we assume there is just one cause when there may be many. We overlook important information. This could involve the omission of outcomes vital to clients, such as quality of life and mortality; simplistic views of causes (e.g., “It’s in the brain,” “It’s in the genes”); or selection of intervention methods that do not address causes of concerns (e.g., use of manualized intervention that ignores important, unique client characteristics). They are encouraged by biases that contribute to a partiality in use of evidence likely to result in errors. Examples including the following • Assuming different entities as more similar than they are • Treating dynamic phenomena as static • Incorrectly assuming that a general principle accounts for all phenomena when it does not • Treating multidimensional phenomena as unidimensional • Treating continuous variables as discrete • Treating interconnected concepts as separable • Treating the whole as merely the sum of its parts (e.g., Feltovich, Spiro, & Coulson, 1993; Woods & Cook, 1999, p. 152) Oversimplifications are common in case reports by students with whom I have worked. Problems are often complex with many related personal and environmental circumstances (e.g., Jensen & Fraser, 2016). Oversimplifications obscure complexities vital to understand to help clients and avoid harm. (For a related discussion, see Haynes [1992].) Oversimplifications that result in poor decisions may arise at many points in decision making, including problem framing, selecting interventions, and evaluating progress. Labeling a behavior and believing that you then understand what it is and what causes it is a common fallacy (equating labeling with explaining). The fallacy of labeling is an oversimplification. Treating multidimensional phenomena as unidimensional, and viewing changing events as static are examples of oversimplifications. “Overinterpretation” may occur, in which we consider data suggestive of an alternative that does not support a preferred view as consistent with this preferred view.

158 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 159

Example: “It is clear that social anxiety is a mental disorder. It is a brain disease. We should place the client on Paxil.” It is not clear that social anxiety is a mental disorder. Indeed, this view was promoted by a public relations agency hired by GlaxoSmithKline, a pharmaceutical company that markets Paxil (Moynihan & Cassels, 2005). (See, for example, the study of fear over the centuries [Naphy & Roberts, 1997].) Countermeasures: Ask questions regarding other potentially important factors. For example, if a client is anxious in social situations, find out whether he has requisite social skills and whether he uses them appropriately. Become historically informed (e.g., Scull, 2005, 2015). Critically appraise claims common in a profession (e.g., Gambrill, 2012a & b; Moncrieff, 2008b; Szasz, 2007).

9. Confusing Correlation with Causation: Assuming Associations Reflect a Causal Relationship Tindale (2007) identifies three kinds of problematic causal reasoning: (1) assuming a causal relation based on a correlation or mere temporal order (post hoc reasoning), (2) confusing causal elements involved (misidentified causes), and (3) predicting a negative causal outcome for a proposal or action, perhaps on the basis of an expected causal chain (slippery slope reasoning) (pp. 173–174). It may be assumed that statistical association reflects causal relationships. Just because two events are associated does not mean one causes the other. A third variable may cause both. Pellagra, a disease characterized by sores, vomiting, diarrhea, and lethargy, was thought to be related to poor sanitation. It is caused by an inadequate diet.

Example: “We studied the correlation between a number of risk factors and depression and found that having parents who are depressed is a risk factor. Depression in parents causes depression in their children.” Countermeasures: Keep in mind that correlations—for example, as found in descriptive studies exploring relationships among variables—cannot be assumed to reflect causal relationships (see also the discussion of oversimplification in this exercise). Here, again, questions provide a pathway for avoiding errors, such as: Does X always occur together with Y? Does X (the presumed cause) occur before Y (the presumed effect)? Does the presumed effect occur without the presumed cause?

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 159 160

10. Post Hoc Ergo Propter Hoc (After This Therefore Because of This) This fallacy refers to the mistaken belief that if event A precedes event B in time, then A caused B. It occurs because of a confounding of correlation with causation (see also item 9). As Medawar (1967) notes, “If a person (a) feels poorly or is sick, (b) receives treatment to make him better, and (c) gets better, then no power of reasoning known to medical science can convince him that it may not have been the treatment that restored his health” (pp. 14–15). If A causes B, it is true that A must precede B, but there may be other events preceding B that could be the cause. A preceding B is a necessary but not a sufficient (always enough) condition to infer cause. This fallacy occurs in practice when (1) a problem exists, (2) the practitioner takes action to remove the complaint (event A), and (3) the complaint disappears (event B). The practitioner then assumes that her action caused the complaint to disappear. In fact, some other event may have caused the change, such as spontaneous remission.

Example: “Mr. James started coming to our support group for the recently bereaved. A few meetings later he seemed much less depressed. That support group must work.” Countermeasures: Think of other possible causes for improvement, or deterioration. For example, you may think your client acquired a new social skill as a result of your program, but your client may have learned it from interactions with friends or family. You may believe cognitive–behavioral therapy helped a depressed client, but the client may have improved because she saw a psychiatrist who prescribed an antidepressant. A break in hot weather, rather than your community crisis team’s efforts to head off violence, may have been responsible for a decrease in street violence. There are cyclical problems that get worse, improve, and get worse again. A large percentage of medical problems clear up by themselves (Skrabanek & Mc Cormick, 1998). A well-designed study can help rule out these and other explanations of client change.

11. Nonfallacy Items: Items That Do Not Contain Fallacies In these items, a fallacy is named and avoided (e.g., “You are attacking me personally, not examining my argument; that’s an ad hominem appeal”), or the helper applies sound reasoning and evidence (e.g., cites and critiques a study, uses a valid outcome measure to judge client change). Use Box 11.3 to review the names of the fallacies.

160 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 161

BOX 11.3 FALLACIES IN GAME A

1. Case examples 2. Testimonials 3. Vagueness (vague descriptions of problems, outcomes, and/or progress measures) 4. Assuming soft-hearted, therefore soft-headed 5. Confirmation biases 6. Reliance on newness or tradition 7. Appeals to unfounded authority, including uncritical documentation 8. Oversimplification 9. Confusing correlation with causation 10. Post hoc ergo propter hoc (after this) 11. Nonfallacy item

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 161 162 163

Practice Exercise 11 Vignettes for Game A: Common Practice Fallacies and Biases

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

REMINDERS Act out the starred items (3, 9, 13). Take turns reading the other numbered items out loud. Remember that some items do not contain fallacies. In these items, a fallacy is named and avoided (e.g., “You are attacking me personally, not examining my argument; that’s an ad hominem appeal”), or the helper applies sound reasoning and evidence (e.g., cites and critiques a study, applies a valid outcome measure to judge change). Use Box 11.3 to review the names of the fallacies.

1. Client speaking to potential clients: I participated in six weekly encounter-group meetings conducted by my nurse, and the group helped. My scores on the Living With Cancer Inventory have increased. I recommend that you attend the group too. 2. One counselor speaking to another: I think that Tom’s chemical dependency problem and codependency have definitely worsened in the past six months. *3. Two administrators speaking to each other: First administrator: In what proportion of hard-to-place adoption cases did the child remain in the placement home at least two years? Second administrator: We have had fifty successful placements in the past two years. First administrator: How many did we try to place? I’m trying to get some idea of our success rate. Second administrator: We don’t have information about that. 4. Politician critical of welfare benefits and welfare fraud among recipients of Aid for Families with Dependent Children: One “welfare queen” illustrates the extent of the problem. She used twelve fictitious names, forged several birth certificates, claimed fifty nonexistent children as dependents, received Aid for Families with Dependent Children for ten years, and defrauded the state of Michigan out of $40,000. She drove an expensive car, took vacations in Mexico, and lived in an expensive house. 5. Psychologist: Our agency offers communication enrichment workshops for couples having some rough spots in their relationships. Four to five couples participated as a group in ten weekly two-hour sessions. Each participant completed the Inventory of Family Feelings during the first and last meetings. These scores show marked improvement. Our workshops enhance positive feelings. 6. A supervisor arguing against critical thinking: There are two kinds of helpers: those who have people skills and who can interact warmly with clients, and those who lack this natural gift but try to make up for it by consulting studies, measures, surveys, and other such trash.

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 163 164

7. Author in a professional journal: This literature review summarizes six articles. Our library assistants were instructed to find articles that support the effectiveness of family-based treatment. All six articles support the effectiveness of family-based treatment for adolescent runaways and related problems. 8. Psychiatrist: My client, Mr. Harrison, had a Beck Depression Inventory (BDI) score that placed him in the severe range when I saw him at intake. I worked with him using cognitive–behavioral methods for six weeks. In the seventh week, his score was in the normal range. My methods worked with Mr. Harrison. His BDI scores were lower after treatment. *9. An intern speaking to another intern: First intern: Mrs. A was very anxious in our first interview. She was so nervous that I ended the interview early and gave her a prescription for Paxil. Second intern: I think you did the right thing because social anxiety is a brain disorder. 10. Situation: A county board meeting Jenny: My staff and I have conducted a survey of Hmong families here in Davis County to determine their service needs. We obtained a list of families from county census records and records kept by the Hmong Mutual Assistance Organization (HMAO). Fifty-seven Hmong families live in the county—a total of 253 persons. With the help of an HMAO interpreter, we asked two head persons from each family about their needs. You have the interview guide before you that we used in the survey. In that interview, we asked them to rank their needs from most important to least important. As a result, their most pressing need is … . Board member (speaking softly to his neighbor): Jenny has done her homework, but I don’t agree with her assessment of the situation. Remember Dr. Morrison, who spoke for an hour to us about the needs of Hmong communities? I place much more confidence in his conclusions. Dr. Morrison is more widely known on this topic. 11. Two nurses discussing the effectiveness of therapeutic touch in decreasing pain. First nurse: I looked up research regarding therapeutic touch and found some well-designed experimental studies that do not support the effectiveness of this method in reducing pain. Second nurse: Thanks for taking the time to take a look at the evidentiary status of this method that we have been using. Let’s see if we can locate methods to reduce pain that have been critically tested and have been found to reduce pain. 12. Senior practitioner speaking to a student: If you try to measure your client’s progress, you will destroy your rapport with the client. Clients know when they are being treated like a guinea pig and resent it. You will be better off if you rely on your intuition and attend to how you react toward your client. As I see it, you’re either an intuitive type or an automaton. *13. Dean, School of Arts and Sciences speaking to Chair, Department of Social work: Dean: How did the social work majors who graduated last june fare in the job market? Department Chair: We’ve been pretty successful. Thirty are employed in social work, and one is in graduate school. 14. Speech therapist speaking to a teacher: Have you heard about facilitated communication? It has just been developed as a way to communicate with autistic children. A facilitator can help the child type messages out on a computer keyboard that communicates the child’s thoughts. These thoughts would remain locked inside the child without this new technology and its skillful use.

164 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 165

15. An advertisement, including pictures of Bill in The American Journal of Psychiatry: Name: Bill. Occupation: Unemployed administrative assistant. Age: 48. Height: 5 ft 10 in. Weight: 170 lb. History: Patient complains of fatigue, inability to concentrate, and feelings of worthlessness since staff cuts at the corporation where he worked for twenty-one years resulted in the loss of his job. He has failed to begin a company- sponsored program and to look for a new job. Initial treatment: After two months of antidepressant treatment, patient complained of sexual dysfunction (erectile failure and decreased libido), which had not been a problem before antidepressant treatment… . Recommendation: Discontinue current antidepressant and switch to a new-generation, nonserotonergic antidepressant. Start Wellbutrin to relieve depression and minimize risk of sexual dysfunction. Outcome After Four Weeks of Therapy with Wellbutrin: Patient reports feeling more energetic. Sexual performance is normal. He has enrolled in job retraining program …. Wellbutrin (BUPROPION HCL) relieves depression with few life-style disruptions (Wellbutrin, 1992). 16. An administrator in a group home for developmentally disabled adults: According to a study I read about functional communication training, this treatment reduced severe aggressive and self-injurious behaviors in self-injuring adults. Let’s try this method with Mark and Olie. 17. Director of a refuge home for battered women: The women who attend our program for physically and emotionally abused women report on their levels of self-esteem. Generally, their self-esteem improves. 18. One psychologist to another: I read a study that reported a positive correlation between parenting styles in early childhood and later antisocial behavior. This shows that parenting style is a cause of later delinquency. 19. Child-welfare worker to students in class: Open adoption is one of the newest advances in adoptions. In open adoption, the biological parents are allowed to stay in touch with the adoptive parents, and in many cases the biological parents contribute to rearing the child. Your agency should try this increasingly popular option. 20. One counselor to another: “Clearly young adults use heroin because they have nothing else to do.”

Gambrill & Gibbs Exercise 11 Reasoning-in-Practice Game A: Common Practice Fallacies and Biases 165 166 167

EXERCISE 12 REASONING-IN-PRACTICE GAME B: GROUP AND INTERPERSONAL DYNAMICS

Purpose

To learn how to identify and avoid fallacies and biases that occur often in group settings, such as case conferences, staff meetings, interdisciplinary teams, and conferences.

Background

Professionals participate in a wide variety of groups, including multidisciplinary teams, case conferences, task groups, seminars, and workshops, where decisions are made that affect the lives of clients. Some groups include both professionals and laypersons such as self-help and support groups (e.g., renal dialysis support groups). Group work is a common part of practice, including community advocacy groups and group cognitive– behavioral therapy. Community action groups include neighborhood block organizations, conflict resolution, and other grassroots groups. Advantages of groups include multiple points of view and approaches to problems, and a variety of skills and knowledge among members. Without sound leadership—and knowledge and skills regarding group process,—common biases, and fallacies, unwise decisions may be made. The fallacies described in this exercise can occur without awareness and stall or sidetrack effective group decision making.

Instructions

1. Before playing Game B, review the instructions in Part 3. 2. Read the descriptions of each fallacy given in Exercise 12, including the definition, example, and suggested countermeasures. 3. Read each vignette aloud when playing the game. This will make the situations more real. Starred items require volunteers to take turns acting out the example while others follow along in the script or watch the actors.

167 168

Definitions, Examples and Countermeasures 1. Ad Hominem (at the Person) This refers to attacking (or praising) the person, or feeling attacked (or praised) as a person, rather than examining the substance of an argument—arguing ad hominem rather than arguing ad rem (at the argument). The ad hominem fallacy may arise when someone lacks supporting evidence but nonetheless wants his or her point of view to prevail. It is a variety of the genetic fallacy (devaluing an argument because of its source (see www.fallacyfiles or skepdic.com). Instead of addressing the substance of your argument, he or she may seek to discredit you by calling you a name or by attacking your character or motives. Or, that person may try to “seduce” you by offering irrelevant praise of you and/or some characteristic you have.

Example: Team meeting in hospital Social worker to psychiatrist: “Could you clarify how you are using the term bipolar personality disorder?” Psychiatrist: I always wondered what they teach you at Berkeley, and now I can see it’s not much. Countermeasures: Address the issue. Argue ad rem. Examine the argument and evidence related to claims.

2. Begging the Question Here, we assume the truth or falsity of what is at issue; we engage in circular reasoning (Walton, 1991). “A statement that is questionable as a conclusion is equally questionable as a premise” (Engel, 1994, p. 53). Different words are often used, making these seemingly obvious ploys difficult to spot. Opinions may be presented as facts. Emotional terms may be used. This is a remarkably common and effective ploy—one that often goes undetected, especially when pronounced with an air of confidence.

Example: Manualized treatments are best because they provide detailed instructions which improve effectiveness. (Notice that the reason given restates the conclusion, but in different words.) Countermeasure: First, be on the lookout for such assertions. Second, ask the proclaimer to give her argument for her conclusion. Here again, raising questions such as “How good is the evidence?” is key to avoiding such “sleight of hand” (Browne & Keeley, 2006, p. 96).

168 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 169

3. Diversion (Red Herring) Here, there is an attempt to sidetrack people from an argument. Red herring originally referred to a fugitive’s use of a dead fish scent to throw tracking dogs off a trail. Unethical adversaries may create a diversion because they know their argument is too weak to stand up to careful scrutiny; they sidetrack the group’s attention to a different topic (they drag a red herring across the trail of the discussion). Angering your opponent creates a diversion (Walton, 1992b). More commonly, the diversion just happens as attention wanders, gets piqued by a new interest, or is sidetracked by humor.

Example: Discussion during a case conference Paul: Edna, my eighty-seven-year-old client, lives alone. She has looked frail lately and I’m worried that she is not eating a balanced diet. Her health seems generally good—no major weaknesses or inju- ries, just dietary problems. What do you think of her as a candidate for the Meals-on-Wheels Program? Craig: I saw a Meals-on-Wheels meal recently. The fish looked pulpy. John: Speaking of fish, did you know that the Walleyed Pike were bit- ing last Sunday on Halfmoon Lake? Countermeasures: Gently bring the discussion back to the point at issue (e.g., “We were talking about … .”).

4. Stereotyping “A stereotype is an oversimplified generalization about a class of individuals, one based on a presumption that every member of the class has some set of properties that is (probably erroneously) identified with the class” (Moore & Parker, 1986, p. 160). Stereotypes can bias judgment. Racism, sexism, “classism,” and ageism are based on stereotypes that can lead to inappropriately negative or positive attitudes and behaviors toward individuals.

Example: Income maintenance worker: Mrs. Owens is a typical low-income client. She lacks the coping skills she needs to be an effective parent. Countermeasures: Judge individuals and their ideas based on a careful assessment of their behavior and contextual thinking, not from some preconceived notion about what to expect from them because of their membership in some group or class of individuals

Gambrill & Gibbs Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics 169 170

5. Manner or Style This fallacy refers to believing an argument because of the apparent sincerity, speaking voice, attractiveness, stage presence, likeability, or other stylistic traits of an argument’s presenter. The reverse of this argument—not believing an argument because you find the speaker’s style or appearance offensive or distracting—can also be a problem. This fallacy captures many gullible victims in our media-saturated age of the Internet, television, podcasts, cell phones, and film. Williams and Ceci (1997) found that simply using a more enthusiastic tone of voice increased student ratings of effectiveness (see also Ambady and Rosenthal [1993].) Advertisements are carefully designed to influence via color, images, and language (see Exercise 5). Slick propaganda is often used in place of clear data about outcomes (e.g., How many clients benefit in what ways? Do any clients get worse?). Beware of advertisements for residential facilities, as well as slick descriptions and portrayals of intervention methods that focus on how pleasant and clean the facility’s grounds are or how enthusiastically attractive clients may advocate for the program.

Example: First student: Take Ames’ class. You’ll love it. She has a great sense of humor. She rivals some standup comics on TV. You’ll learn a lot. Countermeasures: Base your judgments and decisions on the evidence presented, not on the speaker’s style or lack of it. The idea’s utility and soundness are what matter.

6. Groupthink Here, “concurrence-seeking [seeking agreement] becomes so domi- nant in a cohesive group that it tends to override realistic appraisal of alternative courses an action” (Janis, 1971, p. 43). Group members of interdisciplinary teams and staff meetings may avoid sharing useful opinions or data with the group because they fear they might be “put down,” hurt the feelings of others, or cause disunity. Indicators of groupthink include stereotyping or characterizing the leaders of opposing groups as evil or incompetent, pressuring group members to stay in line and fostering an (incorrect) belief that group members are unanimous in their opinion—a false consensus effect (Janis, 1982). Such behaviors may interfere with sound decision making by obscuring negative consequences of a preferred view and discouraging

170 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 171

discussion of alternative views. Unless a culture of inquiry is encouraged, groups may stifle dissenting opinions. For further discussion concerning groupthink see Rose (2011).

Example: A student is in a seminar on psychology given by a well-known expert in his area. The instructor makes a claim the student knows is wrong, but does not bring it up because she is afraid she will be criticized. Countermeasures: Strategies Janis (1982) suggested to counter groupthink include assigning the role of critical evaluator to one or more of the group’s members and, for important decisions, setting up independent committees to gather evidence and deliberate independently of the other groups, with each committee led by a different person (pp. 262–265). You can decrease vulnerability to groupthink by considering arguments, both pro and con, regarding issues; being aware of indicators of groupthink; and keeping in mind consequences for clients of groupthink, such as making decisions that harm rather than help them (Nemeth & Goncalo, 2005).

7. Bandwagon (Popularity) With this fallacy, “there is an attempt to persuade us that a claim is true or an action is right because it is popular—because many, most, or all people believe it or do it” (Freeman, 1993, p. 56; see also Walton, 1999). Examples include the belief that if many people have a particular belief or use a particular method, then the conclusion must be true or the method must be effective. The bandwagon appeal (appeal to consensus) implies that, by the sheer number of people, the point in question cannot be wrong.

Example: Two social workers speaking during lunch of an alcohol and other drug abuse (AODA) treatment facility First social worker: A lot of the AODA treatment facilities in our area are adopting the matching hypothesis. More facilities try to systematically match clients with treatment. Second social worker: I think we should, too. Countermeasures: Critically evaluate popular notions. Examine the evidence before you join the herd. For example, is there a systematic review related to the question?.

Gambrill & Gibbs Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics 171 172

8. Either–Or (False Dilemma) This fallacy refers to stating or implying that there are only two alternatives when there may be more than two. Either–or reasoning prematurely limits options for problem solving. Other options may be available.

Example: “The way I see it, you’re either for us, or against us. Which is it?” Countermeasures: Identity alternative views of what might be done. Ask each group member to write down independently a list of possible courses of action. Assure group members that whatever they write will be discussed seriously (see previous discussion of groupthink in this exercise).

9. Strawperson Argument This fallacy refers to misrepresenting a person’s argument and then attacking the misrepresentation. This is often used as a diversion to block critical appraisal.

Example: Here is an example from the first author’s experience at a faculty meeting. Professor A: We think we should offer two courses on diversity to our students. Professor Strawman: How can we possibly pay for five to ten new courses? Countermeasures: Restate your position as accurately as you can.

10. Slippery Slope (Domino Effect) Fallacy With this fallacy there is an objection to an argument on the grounds that once a step is taken, other events will occur inevitably (Walton, 1992a). This is a common ploy designed to discourage acceptance of a disliked position. The fallacy often lies in the assumption that the events alluded inevitably follow from the initial action (when they may not).

Example: “If we adopt socialized medicine in this country, all other areas will become socialized, including even where we live. I don’t want to live in a country like that.” Countermeasures: Point out that the additional alleged events do not necessarily follow from the initial action.

172 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 173

11. Nonfallacy Items: Items That Do Not Include a Fallacy Be ready to encounter a few examples of sound reasoning. Use the list of fallacies in Box 12.1 as a reminder when playing Game B.

BOX 12.1 FALLACIES IN GAME B

1. Ad hominem (at the person) 2. Begging the question 3. Diversion (red herring) 4. Stereotyping 5. Manner or style 6. Groupthink 7. Bandwagon (popularity) 8. Either–or (false dilemma) 9. Strawperson argument 10. Slippery slope 11. Nonfallacy item

Gambrill & Gibbs Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics 173 174 175

Practice Exercise 12 Vignettes for Game B: Group and Interpersonal Dynamics

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

REMINDER The vignettes are more vivid if each item is read aloud. The starred items may be more effective and fun if class members act out the parts. Refer to Box 12.1 for a summary of fallacies.

*1. Situation: A multidisciplinary team (special education teacher, school psychologist, speech therapist, social worker, school nurse, and child’s parent) meet to decide whether Jason, age four, should be admitted to an Early Childhood–Exceptional Education Needs (EC- EEN) program. Special education teacher: I know Jason’s score on the Battelle Developmental Inventory was above the cutoff score for admission to your program, but I think his behavior, as I saw during his visit to my classroom, qualifies him for admission to the EC-EEN program. He ran around the room almost all the time, was not task focused, and did not follow instructions. School psychologist: Maybe you’re right. Why didn’t you say something about this during the team meeting? Special education teacher: Nobody, including the parents, seemed to think Jason’s behavior was a problem except me. School psychologist: It’s too bad that you didn’t feel comfortable enough to bring this up. You were the team member who had the best chance to observe him. *2. Situation: Monthly meeting of agency administrators First administrator: Your idea to give more money to work with the elderly is a good one, but in the long run it is not a good idea because we would have to give more money to services for all other groups. Second administrator: Gee, I didn’t think of that. 3. Situation: Workshop on alcohol abuse Presenter: Alcohol abuse is a disorder. Participant: What is a disorder? Presenter: It is a lack of order. *4. Situation: Case conference at a mental health clinic Sandra: We may be overusing the category of borderline personality disorder when assessing our clients. Diana: I don’t think so. This diagnosis is included in the DSM, so it must be valid category. Sandra: But I have read critiques of this classification system and there are real problems with reliability and validity. For example, continuous variables such as social anxiety are

Gambrill & Gibbs Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics 175 176

transformed into dichotomous ones (“social anxiety disorder” or not), many terms are vague (such as often), and complaints such as “insomnia” included as a sign of depression could have many different causes. Alex (whispering in Roger’s ear): There she goes again. Always trying to be a know-it-all. *5. Situation: Discussion of whether to release a client from an inpatient psychiatric facility Clinical psychologist: I don’t know if Mr. Myers should be released so early. I am concerned that, now that his depression is lifting, he may still be suicidal— Social worker (interrupting): I noted that Mr. Myers cracked a joke in group this morning. Nurse: Yes, I recall that joke. It was something about how the president’s great expectations had unraveled into great expectorations. *6. Situation: Juvenile court worker talking to her supervisor Juvenile court worker: I just read a study that suggests that early intervention may reduce the number of kids needing institutional placement. The study did not involve random assignment, but maybe we could conduct a trial here. We could offer more intensive services to some clients and standard services to others, then compare the outcome. Supervisor: Thanks for sharing this. Let’s do a more systematic search for related research after we form a clear question. For example, what age children are we most interested in? And what are characteristics of these children—for example, are they from poor families? 7. Situation: Case conference First social worker: Have we considered use of behavioral methods to help Rafa manage his behavior at school? Second social worker: Behavioral methods manipulate people against their will. I do not think we should use such methods. 8. Situation: Case conference at a protective service agency Chairperson: The Armejo family presents us with a dilemma. Should we conduct an investigation for potential child abuse or not? Polly: As I understand the situation, we are in a gray area. A friend of one of their neighbors said another neighbor reported he heard children screaming and worried the children might be being abused. The family has undergone hard times lately. The father, a custo- dian at a local Air Force base, has been laid off from work. We have a report from a fellow worker that the Armejos are having marital difficulties. Jennifer: I am uncomfortable with initiating an investigation for child abuse on the basis of such shaky evidence. I think we should do nothing at this time. What do you think? We must file a formal complaint (initiate a full investigation) or leave the family alone; which is it? 9. Situation: Two psychiatric nurses discussing a patient First nurse: His behavior on the ward is erratic and unpredictable. He warrants a diagnosis of bipolar. Second nurse: What makes you think so? First nurse: Because of the behavior on the unit I have described. 10. Situation: All staff members in the Methodist Hospital Social Service Department are female. Members of the department will interview three job candidates, one of whom is male. One staff member to another (as they walk down the hill): Just between you and me, I think male social workers are out of their element in hospital social work. They lack the empathy

176 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 177

and patience required to do this job well. I am not optimistic about our male candidate’s ability to do the job. *11. Situation: Discussion among alcohol and other drug abuse counselors Richard: One study I read suggested that the best hope for improving services for alcohol- dependent persons is to classify alcoholics into types and to match each type with its most effective intervention. It seems there are interactions between intervention and type for mean level of sobriety, but no differences for mean success across intervention. What do you think? Onesmo: The idea that alcoholics are all unique (each one is different) seems wrong to me. If they were all unique, how would they all experience the same physiological symptoms of withdrawal after they have built up a tolerance for alcohol? 12. Comment in an interdisciplinary case conference: I notice the diagnosis of attention deficit hyperactivity disorder more and more frequently in records from children referred to us. Perhaps we should classify our children in this category more often. 13. Situation: An interdisciplinary case conference in a nursing home Psychologist intern: I don’t think you should use those feeding and exercise procedures for Mrs. Shore. They don’t work. Because she has Parkinson’s, she often spills her food. I also don’t think you should walk her up and down the hall for exercise. I have read reports that argue against everything you’re doing. Nurse: I’m not sure you’re in the best position to say. You haven’t completed your degree yet. *14. Situation: Two nurses are attending a professional conference. Their hospital has sent them to the conference for continuing education. There are about one hundred people attending the two-day conference, for which all paid a $100 fee: First nurse (whispering in friend’s ear): I wonder if this imaging method affects the longevity of cancer patients and what kind of evidence these presenters might give us. Second nurse: Why don’t we ask the presenter? First nurse: That’s a good idea. How does this sound: Could you tell us if any controlled trials have been conducted testing the effectiveness of imaging in decreasing mortality of cancer patients and, if so, could you describe them? *15. Situation: Two physicians attending a conference on validation therapy as a method for helping the confused elderly First physician: I wonder if validation therapy helps elderly people to become more oriented to time, place, and person? Second physician: You’ll enjoy this presentation by Diggelman this afternoon. He presents reality therapy so well that the time flies. He is sincere; he gets the audience involved. He walks into the audience and jokes with us during the breaks. His enthusiasm is exciting. Anyone so sincere and enthusiastic must be giving us accurate information. *16. Situation: Confrontation between supervisor and worker Supervisor (to worker): You’re late for work. Worker: So, you’re telling me that Bill saw me come in late. I don’t think it’s ethical to have one worker report on another. 17. Psychiatrist says to himself at a team meeting: Here comes Ms. Carey again. She’s usually not very well prepared but she talks with such confidence. *18. Situation: Judge consulting with a social worker Judge Calhoun: The Chicago police have referred a family to social services. The police found the parents and their two children living in their car without food, adequate

Gambrill & Gibbs Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics 177 178

clothing—and it’s November! What should we do? Put the children in foster care or leave the family members alone to fend for themselves? Social worker: I think that, in such a situation, I would place the children in foster care. *19. Situation: Case conference at a juvenile court probation agency Ron: This boy has committed a very dangerous act. He constructed an explosive device and set it off in the field next to town. There wasn’t anyone, other than the stone deaf, who didn’t hear the boom! Jonathan: Yes, that’s true, but he has no prior delinquent act on his record. Ron: We either have to place him in juvenile detention to protect society or let him off. Which is it? *20. Situation: Child Protective Service case conference Mike (a police officer): I know we are not supposed to interview a child more than once, but I think we could find out more if we did. Antonio: There are sound reasons not do to this. In 1993, Ceci and Bruck reviewed research about suggestibility in young children. It seems that small children, especially if inter- viewed repeatedly, may construct an untrue story. In one study 38% of the children who went to the doctor for a routine examination in which no pelvic examination was done reported their genitals were touched. In successive interviews with the same children, the children gave progressively more elaborate descriptions of acts the doctor did not perform. I’m worried that the same thing might have occurred here. Is there any clue in the progression of her ideas, from interview to interview, that Janie might have picked up unintentional cues to shape her story? Mike: Your saying that I would intentionally mislead a child into giving false testimony is ridiculous. I would never help a child to lie. 21. Faculty member speaking in a medical school to faculty: Problem-based learning is used more frequently in medical schools to teach clinical reasoning skills. We should use this with our students.

Follow-up Question

Do any of this game’s vignettes reflect real situations particularly well? Which one(s)?

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

178 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 179â

EXERCISE 13â REASONING-âIN-âPRACTICE GAME C: MORE BIASES

Purpose

To learn to identify and avoid common cognitive and affective biases that influence practice beliefs and actions.

Background

Research related to judgment and decision making highlights affective and cognitive biases and errors that may lead us astray, as well as the vital role of corrective feedback (Gambrill, 2012b; Jenicek & Hitchcock, 2005; Koehler & Harvey, 2005). Examples include the status quo bias (preferring current conditions), premature closure, and sunk costs (the greater our investment in a view/âaction, the less likely we may be to consider alternatives). Gender, race, and personal attractiveness may influence decisions (Garb, 2005). Fast and frugal heuristics (making decisions based on cues that first come to mind) are a sound guide when such cues are accurate (Gigerenzer, 2008). Simplifying strategies such as the satisfying heuristic (search through alternatives and select the first one that exceeds your aspiration level) (Gigerenzer, 2008, p. 24) often result in rapid adaptive choices. Although such strategies may be a sound guide when based on specific and recurrent characteristics of the environment (cues have ecological rationality), when misleading cues are relied on, they can result in incorrect judgments and poor decisions. Nisbett and Ross (1980) emphasized two heuristics (simplifying strategies) that encourage bias: (1) availability (e.g., vividness and preferred theory) and (2) representativeness (e.g., depending on resemblance, such as similarity of causes to events). Biases related to representativeness include stereotyping, ignoring sample size, and overlooking a regression effects. People tend to believe that causes are similar to their effects. Analytic thinking provides a check on the accuracy of intuitive thinking (Croskerry, 2005, 2009; Kahneman, 2011) (see also related discussion in Part 1). The vignettes in Game C illustrate misleading cognitive and affective biases (see also discussion of confirmation biases and oversimplifications in Exercise 11). Many

179 180

others could be added, such as “naturalism bias” (“a preference for natural over artificial products even when the two are indistinguishable” [Chapman, 2005, p. 590]).

Instructions

1. Review the instructions in Part 3 before playing this game. 2. Read the description of each bias. 3. Read each vignette aloud when playing the game. Act out starred items.

Definitions, Examples, and Countermeasures 1. Hindsight Bias This fallacy refers to the tendency to think you could have predicted an event “before the fact,” when indeed you could not have done so (often because you did not have the information at the time in the past that you now have). We tend to remember successful predictions and to forget or ignore unsuccessful ones (Hoffrage & Pohl, 2003). There is a false sense of predictive accuracy even among experts (Tetlock, 2003). Those who fall prey to hindsight bias often say, “I told you so!” or “Wasn’t I right?” But, they rarely say, “I told you this would be true, but I was wrong.” Hindsight bias may result in unfairly blaming yourself or others for not predicting a tragic outcome (murder, suicide, return to drug abuse). You review the person’s history, searching for something you “should have noticed,” and then hold yourself (or someone else) responsible for not taking timely action while ignoring cases in which the same events occurred unaccompanied by the tragic outcome. This fallacy wins lawsuits for attorneys.

Example: First supervisor: That story about the client who shot his wife, his children, and then himself was a tragic one. Second supervisor: Yes, I understand he attempted suicide once before. Wouldn’t you think his counselor would have noted this and taken corrective action? Countermeasures: Keep records of your predictions as you make them, not after the fact. When looking back, people tend to overestimate

180 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 181

the accuracy of their predictions. Learn how to assess risk (see Exercise 27).

2. Fundamental Attribution Error This fallacy refers to the tendency to attribute behavior to enduring quali- ties (personality traits) considered typical of an individual and to overlook environmental influences (Kahneman, Slovic, & Tversky, 1982). This results in focusing on client characteristics and overlooking environmental factors related to hoped-for outcomes. For example, we may overlook police brutality in gaining confessions. Asymmetries in attribution to person or environment or between actors and observers may create a self- serving pattern (attributing personal lapses to environmental variables and lapses of others to their personality characteristics). Croskerry (2003) suggests that helpers are especially prone to this error with people labeled with psychiatric problems (e.g., medical causes are overlooked). For a description of the complexities of findings in this area, see Malle (2006).

Example: A family therapist says, “I know the couple has faced severe financial hardships because of the husband’s being laid off, the flood destroying much of their furniture and household goods, and the wife’s illness and surgery, but I still think their personality clash explains their problems. He is aggressive and she has a passive personality.” Countermeasures: Always ask: Are there influential environmental circumstances? There is a long line of research showing that the environments in which we live can create stress and contribute to anxiety and depression (e.g., Abrams, 2012; Brown & Harris, 1978; Cutrona et al., 2006). Contextual views emphasize the role of environmental influences (Gambrill, 2013a, 2013b; Staats, 2012).

3. Framing Effects Posing a decision in a certain way influences decisions. For example, framing a decision in a way that emphasizes potential benefits increases the likelihood that the decision maker will say yes. On the other hand, we are more likely to say no when a decision is posed in a way that emphasizes possible adverse consequences (see discussion of framing effects in Paling [2006]). Framing effects are more powerful when life-affecting decisions are being made, such as whether to undergo a complex surgical procedure.

Gambrill & Gibbs Exercise 13 Reasoning-in-Practice Game C: More Biases 181 182

Example: Counselor: Perhaps I can help you with your decision. We know that two-thirds of those who get treatment at Anderson Hospital for the Alcohol Dependent remain alcohol free for two years. We also know that one-third of those treated at Luther Hospital’s Alcohol Dependency Unit return to drinking within two years. Client: I think I’ll choose Anderson because, from what you said, my chances seem better there. Countermeasures: Describe negative as well as positive consequences for all alternatives.

4. Overconfidence This refers to inflated (inaccurate) belief in the accuracy of your judgments. We often have inaccurate beliefs about the accuracy of our predictions. Because of self-inflated assessments of our skills and knowledge (Dunning et al., 2004), we may fail to question them and, as a result, offer clients ineffective or harmful services. David Burns (2008) collected data concerning degree of agreement between clients and professionals regarding the helpfulness of each therapy session for hundreds of exchanges. He reported a correlation of zero. Overconfidence is encouraged by confirmation biases in which we focus on data that support a preferred view and ignore counterevidence (see a discussion of such biases in Exercise 11). Overconfidence is encouraged by the illusion of control—a tendency to believe we can influence outcomes when we cannot.

5. Overlooking Regression Effect This refers to the tendency for people with very high or very low scores on a measure or variable to have scores closer to the center or mean of the distribution when measured a second time. Let’s say an individual scores very low or high on some assessment measure or test and is given a program designed to improve performance. If the client’s posttest score is different, the regression fallacy lies in assuming that the intervention accounts for the change. Extreme pretest results tend to contain large errors that are corrected at posttest. Consider an average student who took a test and got one of the lowest scores in the class. In subsequent testing, the student will probably do better (regress toward the mean or average). Why? Perhaps during the pretest the student was ill or distracted, failed

182 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 183

to understand instructions, or didn’t see the items on the back of the last page, The test may have included questions about content in the one area he or she did not study. The same principle holds for very high scores on a pretest that may have been due to unusually effective guessing or chance study of just the right topics for the test. Regression can account for the apparent effectiveness or ineffectiveness of programs designed to help those who pretest unusually low or high in some characteristic.

Example: A school social worker says, “We pretested all the fifth graders at Lowell Middle School on social skills, then involved the 10% who scored lowest in a five-week Working Together Program. At posttest, the fifth graders scored much higher on the same measure. This program seems to work. Countermeasures: Be wary of studies that single out extreme groups for observation.

6. The Law of Small Numbers This refers to the belief that because a person has intimate knowledge of one or a few cases, he or she knows what is generally true about clients. It involves an insensitivity to sample size, placing greater confidence in conclusions based on a small sample than on a much larger one (see also discussion of case examples and testimonials in Exercise 11). The misleading law of small numbers is the reverse of the law of large numbers, which states that, as samples include successively greater proportions of a population, the characteristics of the sample more accurately represent those of the population (unless the variance is very low). In other words, many observations usually provide a basis for more accurate generalizations.

Example: A childcare worker says, “Thanks for summarizing the study of 421 children that reported significantly lower intelligence among children whose mothers drank three drinks per day, but I doubt those findings. My sister regularly drinks more than three drinks per day and her children are fine. Countermeasures: Give greater weight to conclusions based on randomly drawn, representative samples; give less weight to experience with one or a few clients.

Gambrill & Gibbs Exercise 13 Reasoning-in-Practice Game C: More Biases 183 184

7. Ignoring Prevalence Rate This refers to the mistaken belief that the same assessment or screening tool will identify individuals just as well in a low-prevalence group (in which few people have a problem) as it will in a high-prevalence group (in which many people have the problem).

Example: A mental health worker says, “Among those hospitalized for a serious mental illness [high prevalence group] who took the Suicide Prediction Instrument (SPI), 10% of those who scored in the high-risk category committed suicide within two years of their release from the hospital. If we administer the SPI to all our outpatient clients [low prevalence] at our clinic, we can be sure that if a client scores as high risk on the SPI, then that client has a 10% chance of committing suicide in the next two years. Countermeasures: In the low base-rate situation, there will be many more false positives (persons judged to have the problem who do not) than in the high base-rate situation. Seek information about base rate regarding topics of discussion. What is regarded as “abnormal” behavior may indeed be normative, as reflected in base-rate data.

8. Omission Bias The refers to the tendency to favor potentially harmful omissions over less harmful acts.

Example: “I don’t think I will have my child vaccinated because I think this has harmful effects.” Countermeasures: Consider information regarding the outcome of all decisions.

9. Gambler’s Fallacy This refers to the mistaken belief that, in a series of independent events, in which a run of the same event occurs, the next event is almost certain to break the run because that event is “due.” For example, if you toss a coin fairly and four heads appear, you tend to believe that the next coin tossed should be a tail because the tail is “about due” to even things out.

Example: “My husband and I have just had our eighth child. Another girl, and I am really disappointed. I suppose I should thank God she

184 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 185

was healthy, but this one was supposed to have been a boy. Even the doctor told me that the law of averages were [sic] in our favor 100 to 1 (“Dear Abby,” June 28, 1974; cited in Dawes, 1988, p. 275). Countermeasures: Remember that for truly independent events—tosses of a fair coin, birth of boy or girl in a given hospital—what happened previously cannot affect the next in the series. The doctor’s advice was in error, because on the eighth trial, the chance was essentially 0.5, as it was for the other births. “Like coins, sperm have no memories, especially not for past conceptions of which they know nothing” (Dawes, 1988, p. 291). No matter how many times you enter the lot- tery, your chances of winning the next time you play are the same no matter how many times you have played in the past.

10. Availability This refers to influence by the accessibility of data. For example, we may judge the likelihood of an event by how easy it is to recall it (Nisbett & Ross, 1980). We tend to make judgments based on the accessibility of concepts/memories—how easy it is to think, see, or hear them. People judge events to be more likely if they are vivid, recent, or familiar. Reliance on availability is a sound guide if cues are accurate (Gigerenzer, 2008). However, if they are not, not questioning available theories and vivid data may result in avoidable errors. Biases related to availability include the fundamental attribution error, emotional influences, recency effects, and confirmation biases. Anchoring and insufficient adjustment refer to the tendency to base estimates of the likelihood of events on an initial piece of information and then not adjust the estimate in the face of new information. An example of this fallacy is a physical therapist who says, “I always base decisions about a client’s chances for rehabilitation on my first few moments with the patient” (see the discussion of use of both intuition and analytic thinking in Part 1).

Example: “I think she has Asperger’s syndrome. I just read a book about this disorder.” Countermeasures: Use strategies that encourage alternative hypotheses, such as asking: Could I be wrong? Select a hypothesis “at the other end of the pole,” or one that directly counters your initial estimate or belief. If needed, consult surveys that describe the relative frequencies of events (Arkes, 1981).

Gambrill & Gibbs Exercise 13 Reasoning-in-Practice Game C: More Biases 185 186

11. Affective Biases This term refers to the influence of emotions on decisions. Affective biases such as a negative reaction to a client and cognitive biases are intertwined. Our moods influence our decisions, which in turn are influenced by our work environments. Poor emotion management skills contribute to poor decisions.

Example: One counselor to another: “I did not get much sleep last night, it was really difficult to pay attention to my last client.” Countermeasure: Here, too, metacognitive skills that encourage us to question initial views are vital.

12. Nonfallacy Items Nonfallacy items do not contain fallacies. These items illustrate use of sound premises to reach a conclusion. Nonfallacy items may show someone pointing out or avoiding a fallacy. Refer to the list of fallacies in Box 13.1 as needed when playing Game C.

BOX 13.1 FALLACIES IN GAME C

1. Hindsight bias 2. Fundamental attribution error 3. Framing effects 4. Overconfidence 5. Overlooking regression effects 6. Law of small numbers 7. Base rate neglect, ignoring prevalence rate 8. Omission bias 9. Gambler’s fallacy 10. Availability (misleading)/anchoring and insufficient adjustment 11. Affective biases 12. Nonfallacy item

186 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 187

Practice Exercise 13 Vignettes for Game C: Cognitive Biases

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

REMINDER The starred items work best if the narrator reads the background and several actors act out the parts. Acting out the situation vividly portrays the content of each vignette. We hope that this active participation helps you to retain and transfer knowledge and skills to practice context. Consult the general instructions for playing the Reasoning-in-Practice Games (Part 3) as well as list of fallacies for Game C (Box 13.1) as needed.

*1. Situation: A new supervisor has just been hired as an early childhood/special education director. The school administration is concerned that too many children who don’t need special education are admitted to the school’s special education program; then, in the spring when the program fills, too few children are admitted to the program who really need it. New supervisor: I think that we need to administer standardized tests to see which children should be admitted to the new program. First special education teacher: We haven’t used standardized tests before, and we have done a good job of identifying those needing the program. Remember the Williams’ boy? We admitted him and he clearly needs our services. Second special education teacher: Yes! And there’s the Gordan’s girl. She clearly needed speech therapy. *2. Situation: School officials have requested a study to evaluate their district’s preschool enrichment program. The childcare worker responsible for the study is reporting. Childcare worker: We administered the Bailey’s Developmental Inventory to all four-year- old children in the Washington County School District. Those who scored in the lowest 5% were enrolled in the district’s Preschool Enrichment Program. The children in the Enrichment Program scored 25% higher one year later, just before entering kindergarten. School official: The enrichment program really helps preschool kids approach the average level for children starting kindergarten. *3. Situation: Orthopedic surgeon speaking to his patient Doctor: If you have arthroscopic surgery on your knee, you have a good chance for full use of your knee. Patient: How good a chance? Doctor: In about 75% of such cases, the operation is a complete success. Patient: And what about with cortisone treatment? Doctor: About a quarter of those who get cortisone do not improve to full use of the knee. Patient: Let’s do the knee surgery.

Gambrill & Gibbs Exercise 13 Reasoning-in-Practice Game C: More Biases 187 188

*4. Situation: Two psychologists discussing the grade-school performance of children from a local low-income housing area Maria: Remember that envelope full of paint chips I sent to the county health department? I got the chips off the window sills and floors of tenement housing on Bridge Street. The county health nurse called today to tell me that the paint chips are toxic—full of lead! The nurse said anyone breathing dust from the paint or ingesting food contaminated with lead, or infants and toddlers eating the chips as they crawl around the floor could suffer long-term cognitive deficits and other health problems. Joe: I was a little worried about that as a factor in school performance. Still, I think the major determinant of performance is cultural. The Bridge Street people just don’t value education. They are simply not motivated enough to do anything about education in their area. 5. Situation: Two psychologists at lunch First psychologist: Now that I have been practicing for two years I can tell just how much my client likes me and feels my sessions helped. Second psychologist: Me, too, but I do wonder sometimes about why so many of my clients drop out early. 6. Nurse administrator: I looked for the best evidence I could find regarding the value of decision aids for people facing health treatment and screening decisions. I found a systematic review in the Cochrane database conducted by O’Connor and her colleagues in 2009. In the absence of counterevidence, which I looked for, I support the use of decision aids for clients. *7. Situation: Two alcohol and drug abuse counselors are talking in their office during lunch Maureen: Who would have thought that Rodrigues would be first among the eight in the recovery group to start using drugs again? Penny: Oh, it didn’t surprise me. There was something about him that tipped me off. I still can’t put my finger on it, but I would have guessed it. 8. Client: I’d much rather have a slim— 10%—chance to overcome the problem than face a likely 90% failure. 9. School social worker: Your study of fifty high school boys that found no relationship between level of knowledge learned in a sex education program and more permissive attitudes toward sex does not impress me. I know a student at King High School who took the same kind of program who swore his permissiveness began because of it. He just found out he has AIDS and he has transmitted it to at least one female student. 10. Social-work supervisor: We arranged for one hundred social workers employed by Megalopolis County to take the State Social Work Competency Examination. The top ten social workers were awarded an engraved gold plaque with their name on it. During the year immediately after the examination, we arranged a series of in-service training programs for all one hundred participants. We then administered the same examination to all test-takers a year later. Much to our surprise, the top ten on the prior test averaged 12% worse on their second test. These top ten social workers must have relaxed during the training and not paid much attention. *11. Situation: Two social workers talking during a break First social worker: I think arranging for parents to monitor outcome at home has harmful effects. Second social worker: I agree. This is not necessary I just ask the client about outcomes.

188 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 189

12. Caseworker planning to visit an Aid-for-Dependent-Children case in a dangerous area of the city: Three employees from our office have gotten through to their cases with backup support in the past with only minor confrontations. I’m sure the next one will have trouble. 13. Situation: A researcher describing a risk assessment instrument to an audience of protective- service workers Researcher: My child abuse prediction instrument accurately identified 90% of protective- service clients who reabused their child within a year. Protective-service worker: Wow! If we could administer your test to all families in the community, we could identify 90% there, too. 14. Surgeon: I evaluated a seventy-eight-year-old man for lethargy, stomach pain, and sleep disturbance after he retired and his wife died. I conducted tests to investigate physiological causes, including lung cancer, thyroid disease, and an infection of the stomach and intestines. I’m sure that I didn’t overlook anything. 15. Situation: Two social workers talking about a client at lunch First social worker: I took a continuing education course on trauma last week. This client is clearly traumatized and we should seek out more information related to support this in her history. *16. Situation: Two counselors talking during lunch First counselor: The argument with my supervisor this morning really upset me. I’m still angry and don’t look forward to seeing clients this afternoon. 17. Hospital physician: I try to get a good look at a patient’s chart before seeing the patient. Usually, all I need to know about whether the patient should be discharged to a community program, a nursing home, or some other program, is in the chart. Then, I look for these indicators when I see the patient. *18. Situation: Two psychologists discussing how to help poor readers in an elementary school First child psychologist: I have some information that might help your poor reader and his parents. In 1986, Miller, Robson, and Bushell studied thirty-three failing readers and their parents. The children were ages eight to eleven and had reading delays of at least eighteen months. The parents read with their kids over six weeks for an average of 7.6 hours per family. Reading accuracy and comprehension scores for the paired reading-program kids were compared with those of kids who did not participate in the program. Results favored kids in the program. You might try paired reading. Second child psychologist: About a year ago, one of our psychologists tried paired reading. The reading developed into a battleground. The kid bugged his parents constantly while they tried to read with him. I don’t think I’ll try paired reading. 19. One probation officer to another: My most recent three sex offenders have been apprehended for a new offense within two months of when their cases were assigned to me. This next one is bound to be a success. *20. Situation: Two occupational therapists talking during lunch First occupational therapist: I think it’s important to kept track of harms to clients in our work. I keep track of each time a client seems worse off with a treatment. I have found few instances of harming my clients. Second occupational therapist: That’s a good idea. I’m going to keep track of times the methods I use harm clients. *21. Situation: A psychologist describing a new instrument to predict outcome for parolees (In the United States, parole is a conditional release from prison; probation is a suspended

Gambrill & Gibbs Exercise 13 Reasoning-in-Practice Game C: More Biases 189 190

prison sentence to be served in the community provided that the probationer follows certain rules.) Psychologist: Our study found that 95% of offenders who scored in the high-risk group and were released from our maximum-security prison went on to commit a new offense within a year. Community probation officer: I would like to give your parole prediction measure to my clients so I can identify high-risk clients, too. I’ll be able to tell the judge in my presentence report which offenders should be handled more conservatively.

Follow-up

Select one of the fallacies discussed in this game and give an example of when this applies to your work. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

190 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 191â

EXERCISE 14â PREPARING A FALLACY/âBIAS FESTIVAL

Purpose

To become familiar with a fallacy or bias that you and a partner have chosen to demonstrate before the class in a brief vignette, and to learn more about other fallacies and biases by watching others demonstrate theirs.

Background

The credit for devising an exercise in which professionals purposefully mess up for instructional purposes may go to clinical scholars at the University of North Carolina (Michael, Boyce, & Wilcox, 1984, p. xi). Apparently, a clinical scholars’ skit in “Clinical Flaw Catching” left such an impression on Max Michael and his colleagues that they wrote the delightful book, Biomedical Bestiary (Michael et al., 1984), complete with humorous illustrations of thirteen fallacies from the medical literature. In this exercise, student presentations illustrate each fallacy, much as the cartoons in Biomedical Bestiary do.

Instructions

1. Sign up with a partner to depict one of the practice fallacies or bias in a skit. Consult references in this book as well as other sources as needed, including Internet sources (e.g., www.rationalwiki.org, www. fallacyfiles.com, www.skepdic.com, Carl Sagan’s Baloney Detection Kit, Guide to Logical Fallacies [Downes, 1995–â2001], and Wikipedia). 2. Define the fallacy or bias drawing on relevant sources and describe how you could avoid it. You may use conceptual definitions, examples, or measures. Attach a brief reference list using American Psychological Association style as well as a script for actors to follow, including

191 192

BOX 14.1 SAMPLE VIGNETTE SCRIPT

Situation: Four patients sit bedraggled with spots painted on their faces [Hold up a sign that reads “9: 00 A.M.”] Doctor: Today we are trying an experimental drug for people such as yourselves, who have blotchy skin disease. This should take care of your disease in a matter of seconds. [Pours water into four glasses containing dry ice, i.e., solid carbon dioxide. Everybody appears to take a drink. (Don’t drink, it will burn the mouth.)] [Hold up a sign that reads “9: 01 A.M.”] Doctor [looking at first patient]: Wow! Your skin really cleared up. How do you feel? First patient: I feel great! Doctor: This stuff really does work. At last, a new miracle drug! First patient [looking at the other three patients]: But what about these other three uncured, sorry-looking specimens? [The other three hang their heads.] Doctor: That’s OK. It doesn’t matter. We did have one success! It works. What a breakthrough! I must tell my colleagues to use it.

Source: Werner, M., & Lehman T. (1995). “Focusing on successes only.” University of Wisconsin at Eau Claire.

descriptions of props (see the sample script included in Box 14.1). Your vignette should last, at most, about a minute. Vignettes seem to work best if they are brief (about thirty seconds), are a bit overdone, make use of props, and demonstrate just one fallacy clearly. 3. Demonstrate your chosen bias or fallacy to the class with your partner or with help from other students whom you direct. (They’ll vol- unteer because they’ll probably need help with their vignettes.) Post your example on YouTube so other students can see and comment on it. Your demonstration should include a short introductory statement describing who is involved, where it takes place, and what is going on so your audience gets the gist of what they will see. Your vignette can be highly realistic or be overacted and humorous, with overdressing, engaging props, or eccentric mannerisms.

Follow-up Question

What did you learn from this exercise?

192 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 193â

EXERCISE 15â FALLACY SPOTTING IN PROFESSIONAL CONTEXTS

Purpose

To hone your skills in spotting fallacies and biases in professional contexts.

Background

This is one of our students’ favorite exercises. Students select some quote relevant to their profession and critique it. You can select quotes from one of your professors or critique a statement in this book. Although we have tried to avoid fallacies and biases, we are sure we are guilty of using some and would be grateful if you would inform us about them so we can correct them.

Instructions

1. Review the fallacies and biases described in the Reasoning-âin-âPractice Games and in the Professional Thinking Form’s scoring key. 2. Identify an example of professional content that you think illustrates one fallacy. 3. Note the complete source in Practice Exercise 15.1 using the American Psychological Association reference style. 4. Provide a verbatim quote that states a claim (include page numbers as relevant). You could duplicate relevant portions of an article/âchapter and attach a copy highlighting the quote. To be fair, do not take a sentence out of its context in a way that alters its meaning. 5. Identify (name) the fallacy or bias involved and explain why you think it represents a fallacy in the critique section of the worksheet.

193 194 195

Practice Exercise 15.1 Fallacy and Bias Spotting in Professional Contexts

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ __ _ _ _

Source*

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Claim (Give verbatim quote or attach a copy showing the content in question.)

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _

Critique (Identify the main fallacy or bias, describe why you think it applies to the quoted material, and describe possible consequences. Have there been any critical tests of the claim? If so, what was found? Consult relevant sources as needed.)

Main fallacy/bias _________________________________________________________________________

How it applies to quote __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _

*If a newspaper article, give date, title of article, author, and page numbers. If journal article, give title, author, date, volume number, and page numbers. If book, give full title, author, date, and publisher. Use American Psychological Association style. If in a conversation, describe context and position of person. If Internet, give website address and date accessed.

Gambrill & Gibbs Exercise 15 Fallacy Spotting in Professional Contexts 195 196

Follow-up Question What did you learn from this exercise?

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

196 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 197â

EXERCISE 16â AVOIDING GROUPTHINK

Purpose

To learn about and practice avoiding strategies used in team meetings and case conferences that decrease the likelihood of making well-âinformed decisions.

Background

Team meetings and case conferences are everyday occurrences in professional practice. As Meehl (1973) suggests in his classic chapter “Why I Do Not Attend Case Conferences,” discussions do not always forward careful appraisal of alternatives. One tendency he notes is the “buddy–âbuddy” syndrome in which we are reluctant to raise questions about other people’s comments because of the false belief that this requires harsh or dis- courteous methods. Groupthink, the tendency to prematurely choose one alternative and to “cool out” dissention, has resulted in grievous consequences, as described by Janis (1982) and others (Tuchman, 1984). Conditions that encourage groupthink include high cohesiveness, insula- tion of the group, lack of procedures to critically appraise judgments and decisions, an authoritarian leader, and high stress with little hope of discovering and forwarding a choice that differs from the one preferred by the leader of the group. These conditions encourage seeking agreement among group members. Indicators of groupthink include the following: • An illusion of invulnerability that results in overoptimistic and excessive risk taking • Belief in the group’s inherent morality • Pressure applied to any group member who disagrees with the majority view • Collective efforts to rationalize or discount warnings • A shared illusion of unanimity • Self-âappointed “mind guards” who protect the group from information that might challenge the group’s complacency

197 198

• Self-censorship of deviation from what seems to be the group’s consensus • Stereotypical views of adversaries as too evil to make negoti- ating worthwhile or too stupid or weak to pose a serious treat (Janis, 1982) Results of groupthink include poor decisions as a result of lack of consideration of well-argued alternatives, vague or incomplete description of objectives, overlooking risks of preferred choices, confirmation biases (seeking only data that confirm preferred views), and failure to critically appraise choices and alternatives (Janis & Mann, 1977; Myers, 2002). Methods Janis (1982) suggests for avoiding groupthink include the following: • The leader should assign the role of critical evaluation to each member. Every group member should be encouraged to air objections and doubts, and to look for new sources of information. • The leader should not state his or her own judgments or preferences at the outset. • Several independent policy planning groups should be established, each with a different leader. • The group should divide into subgroups and meet separately and then come together later to work out differences. • Members should discuss deliberations of the group with qualified outsiders. • Qualified outsiders should be invited to attend group deliberations. • One member of the group should be assigned the role of devil’s advocate. (Assigning just one devil’s advocate in a group may not be effective because of the strong tendencies of groups to persuade a lone dissenter. See, for example, the classic study by Asch [1956]). • After the group has reached an agreement, another meeting should be held during which every member is encouraged to express any doubts and to rethink the issue.

198 Increasing Your Skill in Avoiding Fallacies, Biases, and Pitfalls in Decision Making Gambrill & Gibbs 199

Instructions

Activity 1 Keep track of groupthink indicators in conferences, team meetings, and/ or class for one week using the form in this exercise. What was the most common indicator? Who used groupthink ploys most often? (Divide duration of meeting into number for each indicator to obtain rate.) (See Exercise 1C.)

Activity 2 Together with a few other students, practice countering groupthink ploys in a role-played team conference using the fishbowl technique in which class members observe a role play. Observers should keep track of groupthink indicators using the form in this exercise. Keep track of ploys used, whether effective responses followed, and with what consequences.

Activity 3 Select a method designed to decrease groupthink (see “Background”), encourage other group members to use it, and discuss what happened.

Gambrill & Gibbs Exercise 16 Avoiding Groupthink 199 200 201

Practice Exercise 16 Nature and Frequency of Groupthink Indicators

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ __ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

INSTRUCTIONS Keep track of indicators of groupthink for one week. Be sure to note overall time observed:

Situation Source Statement Kind of Ploy Consequences

Key: Situation: T (team meeting), CC (case conference), C (class), O (other ______ ). Source: L (leader), M (member), V (visitor), O (other __ _ _ _ _ _ _ _ ) Kind of Ploy: buddy–buddy, for example. See background information in Exercises 11 through 13. Consequence: For example, detracted from making a sound decision

Gambrill & Gibbs Exercise 16 Avoiding Groupthink 201 202 203â

PART 4 Evidence-âInformed Decision Making

The process and philosophy of evidence-âbased practice EBP was introduced in the health area and has spread to other professions (see Part 1). The exercises in Part 4 provide guidance in carrying out the steps in the process, including posing well-âstructured questions that guide an efficient, effective search for related research. Exercise 17 offers an opportunity to carry out the process. Exercise 18 involves applying the process in interdisciplinary team meetings. Exercise 19 provides instructions for preparing critically appraised topics (CATs) and guides you in preparing a CAT for your supervisor. Exercise 20 reviews informed consent obligations and describes related options. Example 21 is designed to give you practice in raising hard questions that may be necessary to help clients and avoid harming them—âthe need for assertive skills. Exercise 22 involves reviewing the evidentiary status of an agency’s service as well as how you evaluate outcome. Last, Exercise 23 engages you in reviewing your expertise. We hope these exercises help you and your clients to make informed decisions.

203 204 205â

EXERCISE 17â APPLYING THE STEPS IN EVIDENCE-âBASED PRACTICE

Purpose

To describe the steps involved in the process of evidence-âinformed practice and to offer practice in implementing these steps.

Background

Ethical obligations require practitioners to draw on practice and policy-â related research findings and to involve clients as informed participants concerning the costs and benefits of recommended services and alternatives. EBP provides a process and related tools designed to fulfill these obligations (Straus et al., 2011). Part 1 offers an overview of this process. Here, the steps involved are described in detail, illustrating the close connection of values, skills, and knowledge related to critical thinking (see Box 1.1 in Part 1). They are designed to help professionals to make “conscientious, explicit and judicious use of current best evidence in making decisions” (Sackett et al., 1997, p. 2; Straus et al., 2011). This means that you must carefully consider the extent to which research, including clinical guidelines, applies to particular clients and their life circumstances. Key questions that guide a search for information reflect the close relationship between the question and information relevant to it:

1. What type of question is being asked?; 2. What sort of information would provide evidence [reduce uncertainty] to answer this type of question?; 3. What type of study [or studies] would provide such information?; 4. What types of information resources would give us access to (the results of) such studies?; and 5. How do we get the best out of the resources, to answer each type of question? (adapted slightly from Snowball [1999])

What questions are most important to raise? How many can be answered by current research?

205 206

Types of Questions That May Occur in Your Work with Clients Both generic and specific questions may arise when trying to understand a client’s concerns and related circumstances, and resources. Questions may concern background knowledge, such as: What percentage of five- year-old boys wet the bed? Generic clinical questions physicians asked include: What is the cause of physical finding X? What test is indicated in situation X? What is the cause of symptom X? (Ely et al., 2000). Foreground questions concern a particular client, such as: What is the risk that Mrs. Stower will fall again in the next few days? Questions differ in terms of whether research is available, its quality, and whether it applies to a client (Patrick & Li, 2012). A systematic review reported that about half of questions during care are not pursued (Del Fiol, Workman, & Gorman, 2014). No one knows the percentages in other professions. Individual differences in both life circumstances and personal characteristics may limit the extent to which available research decreases uncertainties (see Exercise 23). Description/assessment/ diagnostic questions include background questions such as: In adolescents with anger management problems, which social skills do they often lack? In frail elderly adults, what are common needs? In students with behavior management problems in the classroom, is classroom observation more accurate than teacher self- report? Both theory and data may be pursued. Questions may concern base rate (estimate of the frequency of a concern in a given population based on a sample of individuals from that population or what has been found regarding similar clients). An example is: What is the base rate of teenage pregnancy in this city? Both qualitative research and survey research contribute to pursuit of answers to many questions, including descriptive questions, such as: What negative and positive events are of most concern to residents of city jails? Other questions here include: What assessment methods are most useful in diagnosing autism? What theory of behavior will be most useful in understanding (framing) this concern? Are multiple problems interrelated? Examples of foreground questions include: Is there a reliable, valid measure of depression or substance abuse, or parenting skills that will be valuable with my client? What is the best instrument to screen for depression among the elderly at Syveresn Lutheran Home? Risk/prognosis/ prediction questions concern the likelihood that a person will engage in a behavior or experience a certain event in a given

206 Evidence-Informed Decision Making Gambrill & Gibbs 207

period. Examples are: What is the likelihood that a sex offender like Joe will commit a new offense within the first two years of his parole? What is the risk Mrs. Jones will continue to physically abuse her children? What is the risk Mrs. Sample will make another suicide attempt in the next six months?” Effectiveness questions include: What feeding methods will work best for infants born with a cleft lip/palate? What method, if any, will most effectively forestall the progression of Alzheimer’s disease among nursing home residents such as e those here at Lakeside? Prevention questions concern the most effective way to prevent the initial occurrence of a problem or undesirable event—for example: What is the most effective way to prevent SIDS sudden infant death syndrome? What is the most effective way to teach kindergarteners and first graders not to wander off with someone not authorized to take the child from school? Other kinds of questions include those regarding harm, cost–benefit of different practices and policies, quality of life, and self-development. Linguistic analysis of questions related to clinical uncertainties by professionals and clients includes comparison questions as well as how, what, when, and why questions (Lloyd, Cella, Tanenblatt, & Coden, 2009).

Steps in Applying Evidence-Based Practice Gibbs (2003) suggests becoming motivated to offer clients evidence- informed services as a first step. The history of the helping professions provides many examples of iatrogenic (harmful) effects produced uncaringly as well as inadvertently (e.g., Jacobson et al., 2005; Scull, 2005, 2015). Examples include the blinding of thousands of premature babies as a result of the use of excessive oxygen levels at birth, resulting in retrolental fibroplasias (Silverman, 1980); increased mortality of frail, elderly persons receiving intensive case management (Blenkner, Bloom, & Nielsen, 1971); and deaths resulting from prescribed psychotropic medication for people older than 65 (Gøtzsche, 2015a). Neither good intentions nor “good words” protect us from harming clients. Beware of the hard-headed- therefore- hard- hearted fallacy—the fallacy that you cannot be an empathic, caring professional as well as a critical thinker; caring about clients requires us to be critical thinkers.

Gambrill & Gibbs Exercise 17 Applying the Steps in Evidence-Based Practice 207 208

Step 1 Convert the need for information into a well-structured question of practical importance regarding a client or policy (see previous description of kinds of questions).

a. In the relevant spaces in Practice Exercise 17, Briefly describe your client and an important decision you must make in the relevant spaces on Practice Exercise 17. Your client may be one of the following: An individual: Include client name (use a pseudonym to protect confidentiality) and relevant demographic information, such as occupation, work history, brief social history, cultural considerations, when and why help was sought, efforts to alleviate concerns, how the client and significant others (e.g., family members) view concerns, how you view them, client strengths, and environmental resources, including sources of social support. A group: Describe specific goals of the group (desired outcomes), number in group, member ages, gender, and occupations/social roles. An organization: Include purpose, structure, policies of interest, culture and climate, resources, and goals. A community: Include geographic area, demographics (race, economic profile, ethnicity, and age distribution), businesses, recreational and educational opportunities, political climate, medical facilities, hoped-for outcomes, and neighborhood characteristics.

b. Write a well-structured question related to your information needs in Practice Exercise 17 and note the question type. Well-structured foreground questions state the client type and concern (e.g., depressed elderly), identify an intervention (which may be an assessment method), and describe some alternative course of action (such as watchful waiting) and a hoped-for outcome (e.g., decrease depression). This is called a PICO question (patient, intervention, comparison, and outcome). Gibbs (2003) refers to these as COPES questions: they are client oriented, of practical importance, and can guide a search especially when accompanied by relevant methodological filters, such as the term systematic review. Here is an example: In women who are depressed, is behavioral activation compared to cognitive therapy effective in decreasing depression.

208 Evidence-Informed Decision Making Gambrill & Gibbs 209

Obstacles to posing questions include lack of important background (about the problem) and/or foreground (about the client) information. Other obstacles include uncertainty about the scope of the question and unrecognized related questions, difficulty posing questions to fit a three- or four-part question (client, intervention, comparison, and outcome), posing too many questions at once, trying to answer the question while posing it, and lack of training and practice. Your questions may not be clear, encouraging failure to obtain valuable resources. Vague questions lead to vague answers. A variety of cognitive and affective biases, including the fundamental attribution error (FAE; focusing on personality characteristics of an individual and overlooking environmental factors), may encourage misleading questions. Frameworks that focus on individual characteristics and causes of distress, as in psychiatric diagnoses, encourage the FAE. Staff who pose questions may create discomfort perhaps because they are doing something unfamiliar or because other staff members view them as disloyal to the agency or profession. Supervisors may not have related experience and wonder why it is of value.

c. Write down your best answer to your question and describe the sources you used before searching for external research.

Step 2 Track down the best research related to your question. a. Write key terms in your question under the headings in Practice Exercise 17 (e.g., client, intervention, alternative, outcomes, and quality filters). Consider this question: In depressed adolescents, is self-help compared with no intervention effective in decreasing depression? In the first column client type, you would write “depressed adolescents”; in the second, “self-help”; in the third, “no intervention”; in the fourth, “decrease depression”; and in the fifth, a quality filter such as “trial” or “systematic review” (see Box 17.1 for a list of quality filters.) Combine search terms using Boolean indicators—AND (to connect terms), OR (to connect synonyms), and NOT (to exclude terms)—as needed to link key terms. Write down your best answer in Exercise 17 before searching.

Gambrill & Gibbs Exercise 17 Applying the Steps in Evidence-Based Practice 209 210

Box 17.1 Quality Filters for Locating Research Findings

Type of Practice Question Useful Terms to Find Best Evidence Assessment (assessment or diag- Interrater reliability, interrater agreement, assess- nosis or client evaluation and ment, diagnosis, kappa, sensitivity, specificity, descriptors to the right positive predictive value, negative predictive value, likelihood ratio, pretest odds Description (survey or needs Random select, stratified random, representative sam- assessment or client satisfaction ple, pretested, response rate and descriptors to the right) Effectiveness (and descriptors to Random, control group, statistical significance, exper- the right) imental group, randomized controlled trial, RCT, experimental design Prevention (and prevent and Random, control group, statistical significance, exper- descriptors to the right) imental group, randomized controlled trial, RCT, experimental design Risk/prognosis (risk, prognosis, or Validation sample, gold standard, positive predictive prediction and descriptors to the value, negative predictive value, predictive validity, right) risk reduction, estimating risk, risk estimation, prediction study Synthesis of studies(and descrip- Meta-analysis, systematic review, synthesis tors to the right)

b. Select a search resource. Resources include summaries and guidelines, preappraised research, nonappraised research, and federated sources that search across all levels (Guyatt et al., 2015) (see Box 17.2). Search first for a systematic review (see Cochrane and Campbell Libraries). In an example of preappraised research, Guyatt et al. (2015) started with 45,000 articles screened via McMaster PLUS; 2600 articles were left after clinical relevance filters were applied—“a 99.9% noise reduction” (p. 64). Improvement in resources has made searching easier. Guyatt et al. (2015) suggest that “Googling” (www.google.com) can be valuable in obtaining general background information (e.g., Wikipedia), especially about new topics. They also note its utility in helping to refine search terms by quickly locating any relevant citations. They suggest that Google Scholar is better for answering foreground questions (www.scholar.google.com) (Guyatt et al., 2015, p. 76). TRIP offers options for advanced searches, PICO searches, and rapid review. PubMed includes a Clinical Inquiries section for asking clinical questions. Using the search terms postpartum depression and systematic review I found a Cochrane review: “Psychosocial and Psychological Interventions for Preventing Postpartum Depression,” by

210 Evidence-Informed Decision Making Gambrill & Gibbs 211

Box 17.2 Useful Resources for Practitioners

Preappraised Research

• Cochrane Database of Systematic Reviews (www.cochrane.org) • Campbell Database of Systematic Reviews (www.campbellcollaboration.org) • Center for Reviews and Dissemination (www.york.ac.ukcrd) • ACP Journal Club (www.acpjc.org) • SCIE (www.scie.org.uk) • NICE (www.nice.org.uk)

Summaries and Guidelines

• Clinical evidence (www.clinicalevidence.bmj.com) • Best practices (www.us.bestpractice.bmj.com) • US National Guidelines Clearinghouse (www.guideline.gov) • UpToDate (www.update.com)

Nonpreappraised Research

• Cumulative Index to Nursing and Allied Health Literature (CINAHL) • ERIC (documents regarding issues in education; accessible for free at https://eric.ed.gov) • PsychInfo (psychological literature; www.apa.org/psychinfo) • Medline/PubMed US National Library of Medicine (www.nlm.nih.gov) • Medscape (medical references; www.medscape.com) • Netting the Evidence (www.nettingtheevidence.org.uk) • Database of Uncertainties about the Effectiveness of Treatments (DUETS) (www.library. nhs.uk/duets) • Agency for Health Care Research and Quality (www.ahcrq.gov) • Research into practice (www.ahrq.gov) • Clinical queries (www.ncbi.nlm.nih.gov) • Essential Evidence Plus (www.essentialevidenceplus.com) • PubMed (Medline; www.nlm.nsh.gov)

Federal Sources*

• TRIP (www.tripdatabase.com) • McMaster PLUS (www.plus.mcmaster.ca/access)

*Retrieves material from multiple sources and organizes results by kind of resource (Guyatt et al., 2015, pp. 51–52)

Dennis and Dowswell (2012). This review indicated that psychological and psychosocial intervention decreased the number of women developing postpartum depression. Results of systematic reviews are often inconclusive, perhaps because of limited relevant research related to a

Gambrill & Gibbs Exercise 17 Applying the Steps in Evidence-Based Practice 211 212

question, as in “empty reviews” in which no studies are eligible for inclusion ([email protected]). (See also Ioannidis, 2016.) Also, keep in mind that “absence of evidence is not evidence of absence” (Aldersen, 2004, p. 476). It may be impossible to tell whether any important differences exist among interventions, and, statistical differences may not be clinically important. Medline/PubMed allows users to link to full text articles. MeSH (Medical Subject Headings), provided by the US National Library of Medicine, is a thesaurus that permits searches at different levels of specificity; you can use a mobile app to gain access to this site. The Center for Reviews and Dissemination at the University of York is maintained by the NHS National Health Services. The Agency for Health Care Research and Quality (www.ahrq.gov) includes clinical research summaries, such as “Off-Label Use of Atypical Antipsychotics: An Update.” A note is included on some reviews that the review may be out of date, alerting the reader to possible new, relevant research. Sources differ in terms of the rigor with which research is appraised and in access to material located. See also www.evidence.nhs.uk.

Box 17.3 Search History Log

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ___ _ _ _ _ _ _ ______ _ _ _ Date __ _ _ _ ___ _ _ _ ____ _ _ _ _ _ _ _

Well-structured question __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _

Search Database Searched Search Terms Number Comments Number of Hits

212 Evidence-Informed Decision Making Gambrill & Gibbs 213

Describe what you learned from your search __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

c. Carry out your search and keep a search history log, noting sources searched, terms used, and number of hits on the Search History Log in Box 17.3. Obstacles include searching too broadly or too narrowly, lack of familiarity with important databases, and lack of ready access to high-speed computers with relevant databases. If your initial search yields no hits, use less restrictive search terms. If it yields too many irrelevant items, make your search terms more precise.

Step 3 Critically appraise what you find “for its validity (closeness to the truth), impact (size of the effect), and applicability (usefulness in clinical practice)” (Straus et al., 2011, p. 4). This applies to systematic reviews (pre- appraisal resources) as well as other material. Skill and knowledge in critically appraising claims regarding practice and policy guidelines and accurately evaluating their applicability is essential. Research studies may be rigorous but not applicable to your client. Many special interests as well as individual biases may compromise the accuracy of claims, including those in peer-reviewed publications, as discussed in Part 1; many incentives compete with “telling the truth.” On a scale of one to five, what is the likelihood that the research method used in a study can answer the question posed: 0, none; 1, slight (10%); 2, fair (30%); 3, moderate (50%); 4, good (70%); and 5, very good (90%)? Are findings relevant to your client? Criteria for appraising different kinds of research reports are available on the Internet (e.g., www.testingtreatments.org) and are discussed in Part 5. Consult these sources as needed. Discovery of conflicting guidelines is not unusual (Oxman, Glasziou, & Williams, 2008). Which guideline is based on the most rigorous, complete review of related research? You may discover that no informative research is available. This is an important

Gambrill & Gibbs Exercise 17 Applying the Steps in Evidence-Based Practice 213 214

finding to share with clients. Does a systematic review or meta-analyses show that an intervention is ineffective or harmful? You could use the following scale:

No. of Points –3 –2 –1 0 +1 +2 +3 points Strong Moderate Slight No Slight Moderate Strong harmful harmful harmful effect positive positive positive effect effect effects effect effect effect

Step 4 This step is the most complex; it involves using your clinical expertise to integrate diverse sources and types of information, including available research knowledge; your client’s unique life circumstances and characteristics, including their values and expectations; organizational capacity; local community-based knowledge such as available resources; and, together with your client, making a decision about what to do. Relevant questions here are: Do research findings apply to my client? Are they important? How definitive are findings? (See the discussion in Part 5 of questions to raise about all research.) Will benefits of a method outweigh risks and costs? What are client preferences? How can I determine my client values and preferences? (see Exercise 20). Do I have the resources needed to offer the most promising intervention? If not, are there any alternatives and what are the policy implications? Clinical expertise includes quality of relationship skills (see Exercise 23). The better they are, the more likely you and your client will forge a helping alliance in which views of concerns and possible solutions are shared (Frank, 1961; Wampold & Imel, 2015a). Clinical expertise includes problem-related knowledge and assessment skills including high quality relationship skills such as warmth and empathy, and intervention and evaluation skills. The more complete and accurate your assessment knowledge, including your bias and fallacy detection skills—for example, in avoiding premature closure and “pathologizing” clients—the more likely you will accurately estimate the likelihood that hoped-for outcomes can be attained and provide effective services, given a supportive agency culture (e.g., time).

214 Evidence-Informed Decision Making Gambrill & Gibbs 215

Step 5 Evaluate the outcome of this process and seek ways to improve it in the future. Were outcomes pursued clear and relevant to clients and significant others (such as family members)? How did you assess progress? Did you compare baseline data with data collected during intervention? What did you find? A variety of single case designs may be feasible to use (e.g., Barlow, Nock, & Hersen, 2008; Gambrill, 2013a). Timely feedback concerning outcome contributes to positive outcome; for example, lack of progress can be quickly detected and corrective steps taken. Collecting client feedback after each session using brief forms regarding how helpful a session was and the quality of the alliance contributes to positive outcome (Miller, Duncan, Brown, Sorrell, & Chalk, 2006; Owen & Imel, 2010) (see www.scottmiller.com for the Session Rating Scale and the Outcome Rating Scale).

Next Steps Help others learn how to pose clear questions regarding information needs, and seek and evaluate related research. Exchange summaries with each other (see Exercise 35). Keep up-to-date with research regarding how to encourage implementation of effective practices and policies. How many questions do you pursue that arise with your clients? Are you asking the most important questions and seeking answers to them (see Exercise 36)?

Gambrill & Gibbs Exercise 17 Applying the Steps in Evidence-Based Practice 215 216 217

Practice Exercise 17 Posing Questions and Seeking Answers

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _____ _ _

Course __ _ _ _ _ _ _ ______ _ _ _ _ _ _____ _ _ Agency __ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Brief description of client __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _

Presenting concern(s) __ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Important decision I must make __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _____ _ _ _

Well-structured question __ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 17 Applying the Steps in Evidence-Based Practice 217 218

Question type □ Effectiveness □ Risk/Prognosis □ Description/Assessment □ Prevention □ Cost-benefit

Circle key words in your question, including quality filters that will help you to focus your search (e.g., Client type, intervention, alternative, hoped-for outcome, quality filter).

My best answer before searching for external evidence __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _____ _ _ _ _ __ _ _

Source(s) used (e.g., supervisor, intuition) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _

Select the most useful database(s) or website address(es) (see Box 17.2) and record your search history in Box 17.3. My answer based on a review of related research (Describe databases and descriptor terms used, hits, and the quality of evidence found. Attach a copy of your best source. Briefly summarize what you learned regarding your question and attach your search log.)

Will the results of your search improve the quality of services offered to your clients? __ _ _ _ Yes __ _ _ _ _ No

Please describe the reasons for your answer __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _

218 Evidence-Informed Decision Making Gambrill & Gibbs 219â

EXERCISE 18â WORKING IN INTERDISCIPLINARY EVIDENCE-âINFORMED TEAMS

Purpose

To give you experience working in an interdisciplinary team using the steps of evidence-âbased practice.

Background

Helping clients attain outcomes they value may require the coordinated efforts of different kinds of professionals. Designing and maintaining effective multidisciplinary teams has received increasing attention (e.g., Edmondson, Bohmer, & Pisano, 2001; Salas, Tannenbaum, Cohen, & Latham, 2013). Kim, Barnato, Augus, Fleisher, and Kahn (2010) reported a decreased mortality using multidisciplinary care teams in intensive care units. Interprofessional communication is viewed by Elwyn and Edwards (2009) as a determinant of an informed client choice environment (e.g., tracking decisions). Rudeness has been found to have negative effects on team performance regarding diagnostic and procedural tasks (Riskin et al., 2015).

Suggestions for Enhancing Team Effectiveness

• Sense of mission: Focus on helping clients and avoiding harm by making informed decisions. • Shared problem-âsolving process: The team shares a process that guides problem solving—âone in which a search for evidence pertinent to decisions and controversy are viewed as vital for discovering possible solutions. Team members should know (1) how basic technologies and procedures work, (2) how to carry out team tasks, (3) be familiar with team members’ particular skills and knowledge, and (4) understand how the EBP team process can be used to seek solutions together (Kozlowski & Ilgen, 2006, pp. 81–â83). Effective communication skills are vital (e.g., questioning claims), and discouraging dysfunctional behaviors such as talking too much (e.g., Gambrill, 2013a).

219 220

• Team environment: The team needs a supportive organizational environment that provides the time and material support needed to identify important questions and to seek and evaluate related research. A team may arrive at conclusions at odds with organizational policy and process, and there should be an arrangement for later discussions of possible needed changes. • Team learning: Members should be experienced in EBP skills. • Leadership: Team leadership should be based on which member wants to take on a problem, assuming all members have equal skill in applying the EBP process. • Necessary support and equipment: The required support and equipment should be available, such as access to up-to-date speedy computers.

Instructions

Please complete Practice Exercise 18. Part of this exercise was given as a final examination counting toward a grade in a course designed by Len Gibbs to help students think critically and to work as a team in applying EBP skills. Students, who had practiced the EBP process as a team before, were given thirty minutes to work as a group to answer the question. If you work as a team in a computer laboratory, try to complete the exercise in thirty minutes.

220 Evidence-Informed Decision Making Gambrill & Gibbs 221

Practice Exercise 18 Working in Evidence-Based Teams

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ Date __ ______ _ _ _ _

Course __ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ Names of group members __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

This exercise assumes knowledge regarding the process of evidence–based practice (EBP), including how to pose and seek answers to questions. Use this process to make your recommendations, working as a team. Try to complete the exercise in thirty minutes.

Topic: Preventing Alcohol Misuse in Young People Assume one of you has taught for several years and you now are the principal of a middle school and high school that includes grades 7 through 12. Alcohol misuse among young people has resulted in several tragic situations, including fatal car accidents. You wonder what primary prevention program (preventing the initial occurrence of a problem) would most effectively prevent alcohol misuse among young people. You have been given a mandate by the school board that you must try something. What approach would you try?

1. Describe your PICO question here (include all three or four elements).

2. Enter your search plan, below including relevant methodological search filters.

Client type Intervention Alternative Outcome Quality filter

Gambrill & Gibbs Exercise 18 Working in Interdisciplinary Evidence-Informed Teams 221 222

3. Record in Box 18.1 your search histories or history for your group, including databases searched, terms used, and numbers of hits to locate your best document.

4. How sound is your best source relative to criteria on the appropriate quality of evidence form? Summarize your assessment of the evidentiary quality here in a brief paragraph.

5. What intervention does this source support? Can you determine number needed to treat (NNT)? (See Box 18.1.)

Box 18.1 Search History Log

Search Database Searched Search Terms Number Comments Number of Hits

Follow-up Question Describe what you learned from your search.

222 Evidence-Informed Decision Making Gambrill & Gibbs 223â

EXERCISE 19â PREPARING CRITICALLY APPRAISED TOPICS

Purpose

To acquaint you with elements in a critically appraised topic (CAT) and to prepare a CAT for your supervisor.

Background

CATs are short (one-â to two-âpage) summaries of the available evidence related to a specific clinical or policy question. A CAT summarizes a process that begins with a question, proceeds to a well-âbuilt one, describes the search strategy used to locate current best evidence, critically appraises what is found, and makes a recommendation based on what is found (the clinical bottom line). The cost-âeffectiveness of different programs should be considered as well as evidentiary concerns (Guyatt et al., 2015; Straus et al., 2011). CATs may be prepared for journal club presentations (see Exercise 35). First, review the process of EBP in Exercise 17. Consult related sources on the Internet to learn more about how to construct CATs and how to locate ones prepared by others. The Centre for Evidence-âBased Medicine (http:ktclearinghouse.ca/âceb) provides sample scenarios, searches, completed worksheets, and CATS for evidence-âbased general practice. The Evidence-âBased Medicine toolbox includes a CAT maker, critical appraisal worksheets, likelihood ratio, number needed to treat (NNT), tables, stats calculator and an odds ratio-âto-ânumber to treat converter. The CAT maker (1) prompts a clinical question, search strategy, and key information about what you find; (2) provides online critical appraisal guides for evaluating the validity and usefulness of studies found; (3) automates the calculation of clinically useful measures (and their 95% confidence intervals); (4) helps you formulate the clinical “bottom line” and creates one-âpage summaries (CATs); (5) helps you remember when to update each CAT; and (6) helps you teach others how to carry out the steps in the process of evidence based practice.

223 224

Instructions

1. Complete Practice Exercise 19.1. Use visual aids in your presentation, which should be about six minutes. Read background information in Exercise 17 as needed. 2. Complete Practice Exercise 19.2. Agency staff donate their time to students as field instructors. Students can reciprocate by helping staff members acquire valuable information. a: Give Practice Exercise 19.2 to your field instructor; bring it to class when completed. b: What kind of question did your supervisor pose? ______________ c: Prepare a CAT (critically appraised topic) regarding this question and send it via e-mail to your instructor and all other class members. Include cost–benefit information if possible, noting both short- and long-term costs and benefits. d: Present your CAT in class. e: Integrate class feedback regarding your CAT, including further search and appraisal as needed. Send the revised CAT via e-mail to your instructor and class members, and give a copy to your supervisor. f: Seek your supervisor’s feedback regarding the usefulness of your CAT and describe it here.____________________________________ ___________________________________ ___________________________________ ___________________________________

224 Evidence-Informed Decision Making Gambrill & Gibbs 225

Practice Exercise 19.1 Preparing and Presenting a Critically Appraised Topic

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ____ _

Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ Course __ _ _ _ _ _ _ _ _ _ _ _

1. State who you are, who generated your question, where that person works, and why their question is important.

2. Describe your well-structured question here and on a PowerPoint slide. Include the client type, intervention, alternate action, hoped-for outcome, and quality filter.

3. Describe your search history, including databases searched, search terms used, and number of documents retrieved for each search string.

Gambrill & Gibbs Exercise 19 Preparing Critically Appraised Topics 225 226

4. Present your best source.

5. Based on critical appraisal of this source, how would you answer your question? The evidence may not be sufficiently clear to make a recommendation. There may be contradictory results. If so, how will you weigh their relative impact?

226 Evidence-Informed Decision Making Gambrill & Gibbs 227

Practice Exercise 19.2 Preparing a Critically Appraised Topic for Your Supervisor

To: Field Instructor __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

From: Student __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Re: Request for Practice or Policy Question

Field instruction and internships are a key part of the education of professionals. We hope you will help us integrate such instruction more closely in our courses by suggesting a practice or policy question related to work with clients by your students to pursue and provide feedback to you. The attached form asks you to pose a question about a method or procedure being used in your agency or being considered for use. Any question regarding the effectiveness of a method you use or plan to use that has life-affecting consequences for clients is appropriate. A question may concern whether a pregnancy prevention program would be effective in reducing the frequency of pregnancy among girls in a local high school, or the effect of daily calls to frail elderly persons in the community on the frequency of calls to the agency. Please complete the attached form and return it to the student you supervise so he or she can bring the completed form to class.

Practice or Policy Question Please return to the student you supervise.

Name of agency __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Your name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Your email __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ ____ _ _ _ _ _ _ _ _ _ _ _ _

Address of Agency __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Agency phone number __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _

Type of clients served by your agency __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _

What important question concerns you about your agency and its effectiveness? You may wonder which of two approaches to treating residents who have Alzheimer’s disease results in a longer period of self-sufficiency for residents; you may wonder if preschool children who are exposed to sex education films falsely report sexual abuse more frequently than children not exposed to such material.

Gambrill & Gibbs Exercise 19 Preparing Critically Appraised Topics 227 228

Please describe your question here as clearly as possible. If you can, define key words in your question.

228 Evidence-Informed Decision Making Gambrill & Gibbs 229â

EXERCISE 20â INVOLVING CLIENTS AS INFORMED PARTICIPANTS

Purpose

To illustrate how clients can be involved as informed participants.

Background

Professional codes of ethics require informed consent regarding the risks and benefits of recommended methods and alternatives. However, most clients are not involved as informed participants (e.g., Brownlee et al., 2011b). Informed consent regarding use of assessment frameworks and measures is just as important, or even more important, than informed consent regarding selection of intervention and evaluation methods. Indeed, framing of problems and selection of assessment and diagnostic measures are interventions and drive selection of intervention methods. Using shared decision making (SDM), both clients and professionals participate in the process (Edwards, Elwyn, & Thompson, 2016). Shared decision making and being informed are patient priorities (Schattner, Bronstein, & Jellin, 2006) and increasing attention is being devoted to involving clients as informed participants, especially in the health area, including considering their wishes for degree of participation (e.g., Brownlee, Hurley, & Moulton, 2011a). Professions such as social work, psychiatry, and psychology are not as advanced in this area, which is an ethical concern. We can draw on advances in the area of health to inform other areas such as social work. Steps taken to help clients to make informed decisions include decision aids that may be computer based. (See, for example, the Informed Medical Decision Foundation [www.informedmedicaldecision.org]; the Ottawa Decision Support Framework (ODSF) [https://âdecisionaid.ohri.ca]; the Patient Oriented Outcomes Research Institute (PCORI) [www.pcori. org]; Coulter, 2002; Stacey et al., 2014). Organizational culture and policy, including data systems and interprofessional communication, influence decision making, including the degree to which it is informed (Edwards,

229 230

Elwyn, & Thompson, 2016). The Interprofessional Shared Decision Making Model is designed to attend to the multiple factors that affect shared decision making, including broad policies and social contexts (https:// decisionaid.ohri.ca). CollaboRATE is a brief patient self-report measure of shared decision making (Barr et al., 2014). Decision coaches are used in healthcare to guide clients (see Patient Decision Aids [http://decisionaid. ohri.ca]). The Cochrane Collaboration has a consumer website that offers a variety of ways to involve clients (consumers.cochrane.org) (see also www. ithinkwell.org). Independent consumer watchdog organizations such as Healthwatch Oxfordshire provide information to clients about services (www.healthwatchoxfordshire.co.uk) (see also the Picker Institute of Europe [www.pickereurope.org]). Informing clients about number needed to treat (see and accurately communicating risk to clients involves clients as informed participants [see Exercise 27]). Shared decision making requires presenting information in an accurate, clear manner. It involves introducing choice, describing options, and helping clients explore their preferences and make decisions. There is a search for “what matters most” to clients (Elwyn et al., 2012). Essential elements suggested by Makoul and Clayman (2006) include problem framing, considering options, discussing pros and cons, exploring client preferences/values, discussing client strengths, checking understanding, making decisions, and arranging follow-up. The International Patient Decision Aid User Checklist (IPDAS)includes sixty-four items organized into three categories: (1) information and value clarification, (2) development process, and (3) effectiveness (e.g., Elwyn & O’Connor, 2009). Recommendations regarding information and value clarification include describing the problem (including the natural course without intervention) and describing intervention options and their likelihood (including the option of doing nothing and their risks), benefits, and harms. Additional items for tests include describing what the test is designed to measure; chances of true-positive, true- negative, false-positive, and false-negative test results; possible next steps based on test results; and chances the problem can be found with or without screening. Recommendations for presenting probabilities of outcomes in an unbiased and understandable way include the following:

230 Evidence-Informed Decision Making Gambrill & Gibbs 231

• Use event rates that describe the population and time period. • Compare outcome probabilities using the same denominator, time period, and scale. • Describe uncertainty around probabilities. • Use multiple methods to view probabilities (words, numbers, diagrams). • Allow clients to view probabilities based on their own situation (e.g., age). • Place probabilities in context of other events (see Paling, 2006). • Use both positive and negative frames (show both survival and death rates)

Methods for clarifying and exploring clients’ values and preferences include describing interventions and outcomes to help clients imagine what it may be like to experience related effects, and asking clients to consider which features matter most. Question lists may be valuable. The second area pertains to how the decision tool was developed— for example, is related research up-to-date?. Are conflicts of interests disclosed? Is plain language used? Are instructions clear? The third area (effectiveness) concerns whether the aid ensured informed value- based decisions (e.g., helping clients to understand options and their features, clarifying what options matter most to clients, and becoming involved in preferred ways).

Instructions

Complete Practice Exercise 20. Select a client with whom you are working or have referred or will refer to another source, or contact someone who works directly with clients or refers them to other sources. Describe the key outcome being pursued as well as the assessment and intervention methods used. Describe the most relevant research regarding how to attain this outcome. Give complete reference and complete Part A of the Evidence-Informed Client Choice Form in Box 20.1. Gather information needed to complete Parts B and C of Box 20.1 and respond to questions noted.

Gambrill & Gibbs Exercise 20 Involving Clients as Informed Participants 231 232

Box 20.1 Evidence-Informed Client Choice Form*

Agency __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _

Client __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Hoped-for outcome(s) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _____ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _

Referral agency (as relevant) and/or department or program within your agency ________ Staff member who will offer (or is providing) services __ _ _ _ _ _ _ _ _ _ _

A. Related External Research _____ 1. This program has been critically tested and found to help people like me to attain hoped-for outcomes. _____ 2. This program has been critically tested and found not to be effective in attaining hoped-for outcomes. __ _ _ _ 3. This program has never been tested with regard to hoped-for outcomes. _____ 4. Other programs have been critically tested and found to help people like me attain hoped-for outcomes. 5. This program has been critically tested and been found to have harmful effects (e.g., decrease the likelihood of attaining hoped-for outcomes or make me worse).

B. Agency’s Background Regarding Use of This Method _____ 1. The agency with which I am involved or to which I have been referred has a track record of success in using this program with people like me.

C. Staff Person’s Track Record in Use of This Method _____ 1. The staff member who will work (or is working) with me has a track record of success in using this method with people like me.

*See Entwistle, Sheldon, Sowden, and Watt (1998).

Note: This form is to be completed by the professional, who then gives it to the client. One form is prepared for each outcome pursued (e.g., increasing positive parenting skills).

232 Evidence-Informed Decision Making Gambrill & Gibbs 233

Practice Exercise 20 Does Your Agency Involve Clients as Informed Participants?

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ ___ _ Course__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Agency __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Describe client (basic demographic information)

Describe assessment measures, methods/frameworks used

Describe key outcome pursued

Describe intervention selected

Give complete citation for best source of related research about how to attain the outcome

Gambrill & Gibbs Exercise 20 Involving Clients as Informed Participants 233 234

1. Based on the above, complete Part A of Box 20.1. 2. Gather the information needed to complete Parts B and C of Box 20.1. This may require reviewing an agency’s website, visiting a referral agency, and/or reviewing agency reports. Questions include the following: a. How do staff assess progress with their clients?______________________________ b. What criteria do they use? __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ c. Are progress measures used valid (see Exercise 22)? __ _ _ _____ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _

3. Describe gaps between the assessment and intervention method(s) used and what research suggests is most accurate and effective.

Discussion Questions/Further Activities

• Review the model consent form for psychiatric drug treatment (Cohen & Jacobs, 2000). Discuss the pros and cons of using this form as a class. Are parents who are urged to place their children on medication for (mis)behaviors fully informed about benefits and harms? Read “The Ethics of Informed Parental Consent to the Psychiatric Drugging of Children” by Breeding and Baughman (2001) and discuss as a class. • Discuss the pros and cons of referring to clients/patients as “consumers” (e.g., Goldstein & Bowers, 2015). • Discuss the ethical implications of gaps between services offered and what is most likely to help clients attain hoped-for outcomes. • Should clients receive a copy of a completed Evidence-Informed Client Choice Form for each major service recommended? ___ Yes ___ No. Discuss reasons for your answers. • Do clients have a responsibility to be informed about life affecting decisions (e.g., see Deber et al., 2007)? • Review the measure of shared decision making described in Elwyn et al. (2013). What do you think of this measure? • Check the following methods your agency uses to involve clients as informed participants:

___ Clients have access to relevant databases on computers in agency waiting rooms. ___ Leaflets are available in the agency waiting room describing services offered, their evidentiary status, and outcomes attained. If yes, are the materials up-to-date, accurate, and user friendly? ___ Clients have access to decision aids on the agency website.

234 Evidence-Informed Decision Making Gambrill & Gibbs 235

___ Clients are involved in describing questions important to them and valued characteristics of staff. If yes, please describe how as well as results. ___ Decision coaches are available as needed. ___ The agency website clearly describes outcomes addressed, and assessment, intervention, and evaluation methods used, including their evidentiary status.

• Discuss changes that could (and should) be made to increase shared decision making. For example, you could involve clients and staff in creating a user-friendly education site (e.g., Clayman, Boberg, & Makoul, 2008). • Design a client-oriented website for helping clients make informed decisions relevant to your agency (e.g., selection of substance abuse programs, diagnosis of dementia).

Gambrill & Gibbs Exercise 20 Involving Clients as Informed Participants 235 236 237â

EXERCISE 21â ASKING HARD QUESTIONS: ENHANCING ASSERTIVE SKILLS

Purpose

To give you practice in asking questions such as: Do our services really help clients? Asking such questions is vital to making sound decisions and involving clients as informed participants. Many people fear nothing more than to take a position which stands out sharply and clearly from the prevailing opinion. The tendency of most is to adopt a view that is so ambiguous that it will include everything and so popular that it will include everybody. —âRev. Martin Luther King, Jr.

Background

Offering clients effective services and honoring ethical obligations requires asking questions others may prefer to avoid, such as: Does our service help clients? How do we know whether it does more good than harm? How good is the evidence? Are there unwarranted variations (Wennberg & Thomson, 2011)? What are key uncertainties here? What does “antisocial” mean? Common views of asking questions include the following: • Don’t ask. You may offend people. • Don’t ask. It may slow down the group. • Don’t ask. You may be the only one who doesn’t know. • Don’t ask. If it’s important, someone else will ask it. • Don’t ask. Others may not want to deal with it now (Matthies, 1996). Let’s say that someone claims that multisystemic therapy for social, emotional and behavioral problems of youths is more effective than other forms of interventions. You might ask: I wonder if you have seen the 2005 systematic review by Littell, Popa, and Forsythe in which it is argued that this intervention is no more effective than other interventions? People may find such questions threatening (Baer, 2003). Indeed, you may be

237 238

threatening the financial survival of an agency that offers ineffective or harmful services. You will often have to be persistent—that is, raise a question again, perhaps in a different way (see Gambrill, 2013a). You may have to enhance your skills in responding to neutralizing efforts, including fallacies used to avoid answering questions, such as ad hominem replies (see Exercise 12). Questions differ in their “threat” level. You could avoid terms such as evidence or research, which may “turn off” others by asking: I wonder if this service is effective for all our clients? Are there uncertainties we have not considered? Does this service work better for some clients than for others? If you work in a community of inquiry, such questions are welcomed. If you work in an environment in which informed decision making is not valued and actively encouraged, such questions may be ignored and/or punished.

Instructions

Please complete Practice Exercise 21.

238 Evidence-Informed Decision Making Gambrill & Gibbs 239

Practice Exercise 21 Asking Hard Questions

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ ______ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

1. Review the questions Richard Paul suggests for thinking critically about decisions and judgments in Box 1.4 as well as the questions in Box 1.1, which are related to different kinds of claims. 2. Describe a question you would like to practice raising.

3. Describe how you would feel and respond if someone asked you that question.

I would feel __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

I would respond (what would I say) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

4. Is there a more diplomatic way to raise this question? Please suggest one example here.

Gambrill & Gibbs Exercise 21 Asking Hard Questions: Enhancing Assertive Skills 239 240

5. Describe obstacles to raising this question.

6. Describe feasible remedies to obstacles you suggest, including assertive replies (see Chapter 16 in Gambrill [2013a]).

7. Practice asking your question over the next week. Keep track of the following in a diary: situation, question, what happened, and results. Give some examples below.

8. Practice asking questions about the evidentiary status of agency practices and policies in a small group of other students. What questions work best (result in clear answers with the least negative reactions)? Which questions do not work well? (Give exact wording.)

Questions that work well. (Give exact wording.) ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _

Questions that do not work well. (Give exact wording.) ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ __ _

240 Evidence-Informed Decision Making Gambrill & Gibbs 241â

EXERCISE 22â EVALUATING SERVICE OUTCOMES

Purpose

To provide an opportunity to review the evidentiary status of an agency’s services (at least in one area) and compare it with what research suggests is most likely to result in hoped-âfor outcomes (including services purchased from other agencies), and to review how you evaluate outcomes with your clients.

Background

There may be gaps between services an agency offers and what should be offered to maximize the likelihood of helping clients. Are there unwarranted variations in your agency? Variations in services used to achieve the same outcome raise questions such as: Are they all of equal effectiveness? Are some more effective than others? Are any harmful? Services are often purchased from other agencies. How effective are they? (See literature and websites on evidence-âbased purchasing, for example, such as the Centre for Evidence-âBased Purchasing [http://ânhscep.useconnect. co.uk].) Misleading ways to evaluate service include focusing on vague outcomes, relying on testimonials, selecting measures that are easy to use but are not valid, using surrogate outcomes (measuring plaque in arteries rather than mortality), and neglecting to follow up (Gambrill, 2013a). Subjective (client reports) as well as objective (e.g., observation of parent–âchild interaction) data should be gathered and compared; thus both qualitative and quantitative data are important in reviewing progress. Self-âmonitoring—âfor example, via mobile phone apps—âprovides user-â friendly options (e.g., Kauer et al., 2012). Administrative data gathered by an agency may not provide an accurate account of outcomes attained. For example, process data are often provided on agency websites (number of clients seen) which is not necessarily correlated with outcome. Staff members may not be required to gather ongoing data regarding each client’s progress in achieving hoped-âfor outcomes, foregoing opportunities

241 242

to make timely changes in plans. They may not be familiar with data analyses that can be of value in graphing data from each client so progress can be reviewed in a timely manner (e.g., Bulté & Onghena, 2013; Zaslofsky & Volpe, 2010).

Instruction

Please complete Practice Exercises 22.1 and 22.2.

242 Evidence-Informed Decision Making Gambrill & Gibbs 243

Practice Exercise 22.1 How Do You Evaluate Outcomes?

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ Date __ _ _ ______ _ _ _ _ _

Course__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Agency __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _

1. Are staff required to monitor progress in achieving hoped-for outcomes on an ongoing basis and to share information with clients? ___ Yes ___ No 2. Describe an outcome you recently pursued.

3. Describe how you evaluated success in achieving this outcome. Give specific examples and discuss the validity of outcome measures.

4. Describe the outcome. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 22 Evaluating Service Outcomes 243 244

5. Collecting data from each client after each interview regarding how helpful it was to the client, and assessing the quality of common factors contribute to positive outcomes (Owen & Imel, 2010). Are staff in your agency required to gather such data? ___ Yes ___ No If yes, please describe exact nature of data collected.

If no, do you think this should be required? __ Yes ___ No

Would you be willing to do this? ___ Yes ___ No

244 Evidence-Informed Decision Making Gambrill & Gibbs 245

Practice Exercise 22.2 Reviewing the Evidentiary Status of Agency Services

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ Date __ ______ _ _ _ _ _ _ _

Course__ _ _ _ _ _ _ _ _ ______ _ _ _ _ Instructor’s Name__ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _

Agency__ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1. Describe the most frequently pursued hoped-for outcome in this agency.

2. Ethical obligations of professionals require appraisal of the evidentiary status of services offered, including those purchased from other agencies. Draw and complete pie charts that depict current and optimal distribution for interventions offered to attain the most frequently pursued hoped-for outcome using the following categories based on Gray (2001b):

a. Services critically tested and found to be effective; they do more good than harm b. Services critically tested and found to be ineffective c. Services of unknown effect d. Services critically tested and found to be harmful; they do more harm than good e. Services are of unknown effect (they have not been tested) but are in a well-designed research study

3. Are there unwarranted variations in services offered in your agency? ___ Yes ___ No If yes, please give an example.

Gambrill & Gibbs Exercise 22 Evaluating Service Outcomes 245 246

4. For any services you describe as falling under a, describe the intervention and give the complete related citation for the highest quality study or review.

Intervention __ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Citation __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

5. If you checked b or d, describe the interventions and give complete related citations. Interventions __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Citations __ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Should these services be terminated (e.g., see Prasad & Ioannidis, 2014)? __ _ Yes __ _ No 6. If you checked e, describe the intervention and provide information regarding this in- progress study (e.g., site of study, author, design)

246 Evidence-Informed Decision Making Gambrill & Gibbs 247

Intervention __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Study details __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _

7. Describe gaps found between the evidentiary status of current and ideal service distribution.

8. Discuss the ethical implications of any gaps found.

9. Describe reasons for gaps found.

10. Describe how gaps could be decreased (e.g., involving staff and clients in advocating for more effective services). Consult implementation literature as needed.

Gambrill & Gibbs Exercise 22 Evaluating Service Outcomes 247 248 249â

EXERCISE 23â REVIEWING YOUR EXPERTISE

Purpose

To identify components of expertise and their relationship to problem solving, and to offer an opportunity for you to think about your expertise.

Background

There is a rich body of literature that describes the differences between experts and novices, and how expertise can be developed (e.g., Gambrill, 2012a & b; Schraagen, Militello, Ormerod, & Lipshitz, 2008). Experts, compared with novices, possess domain-âspecific knowledge and can identify more rapidly information needed to solve a problem. Effective communication and relationship skills are a vital component of clinical expertise, including factors that contribute to positive outcomes, such as empathy, warmth, and creating a positive alliance; as well as interpersonal skills that contribute to a critical appraisal of claims, such as asking related questions (e.g., Is there evidence that x is effective?), resist- ing interruptions, requesting changes in dysfunctional behaviors such as monopolizing conversations, and providing positive feedback (e.g., see Chapter 16 in Gambrill, 2013a). Experts pay more attention to problem definition and they structure problems at a deeper (more abstract) level compared with novices, who tend to accept problems as given. For example, helpers skilled in functional analysis look beyond the topog- raphy (form) of behavior to examine its functions (Gambrill, 2013b; Staats, 2012). Experts do not necessarily perform better than novices in unstructured problem areas such as psychology and psychiatry (Tracey, Wampold, Goodyear, & Lichtenberg, 2014). Skill in solving unstructured problems seems to require a great deal of experience with the domain in which corrective feedback is gained (Hogarth, 2001; Kahneman, 2011). This kind of experience permits the building of an extensive “library” of distinguishable situations (pattern recognition). Expertise entails the use

249 250

of meta-cognitive skills (thinking about your thinking), which decreases premature closure on misleading views. Values, skills, and knowledge related to critical thinking, such as keeping your purpose clearly in view (to help clients) and questioning assumptions, contribute to expertise in decision making. Challenges in developing expertise include the dynamic nature of situations (they change), the unpredictability of human behavior, lack of opportunities for corrective feedback, and infrequent occurrence of a task (Shanteau, 1992, p. 282). Decisions may be improved by the use of simple, empirically based rules, as in the use of a fast-and- frugal decision tree to make predictions about heart disease patients (Gigerenzer, 2002a; 2014) (see, also, the discussion of decision aids for clients in Exercise 20). Barriers to problem solving include limited knowledge, information processing barriers, the task environment, motivational and emotional blocks, perceptual and intellectual blocks, and expressive blocks (see Exercise 37). Cultural blocks may also be present, such as a distain for intellectual rigor.). Biases and fallacies discussed in other exercises may intrude. Some barriers to problem solving are self-inflicted, such as overconfidence, which may result in avoidable errors (see Part 3). Some are created by dysfunctional management practices, including failure to arrange corrective feedback and lack of practice opportunities (Tracey et al., 2014). Avoidable errors may result in overlooking relevant environmental circumstances and client assets, focusing on irrelevant outcomes, and/ or selecting ineffective or harmful services. Mistakes include failure to recognize a problem, the collection of misleading information, incorrect problem framing, premature closure, and harmful delay in making decisions (Caruth & Handlogten, 2000). Some errors are unavoidable because of the lack of information. Many are avoidable, such as failure to acquire readily available information that would contribute to helping a client. Common errors in different problem-solving phrases can be seen in Box 23.1. Errors may be knowledge based (ignorance of important information), rule based (failure to implement a rule appropriately), skill based (mistakes made in carrying out an important activity), and/ or argument based (faulty arguments). Many are system based (related to dysfunctional agency organizations) (Jenicek, 2011). You may confuse

250 Evidence-Informed Decision Making Gambrill & Gibbs 251

Box 23.1 Problem-Solving Phases and Examples of Common Errors

Step Common Errors 1. Clarify the problem. • Jump to conclusions (overlook alternative views) • Seek to justify views rather than critically evaluate them • Ignore environmental causes • Gather irrelevant data • Ignore problem-related theory and research • Overestimate personal problem-related knowledge • Rely on invalid data (e.g., small biased samples) • Disregard conflicting evidence • Stereotyping 2. Search for solutions. • Overlook options • Look only for data that confirm assumptions • Overlook constraints • Overlook resources • Fail to revise views based on new information • Items under Step 1 3. Decide on a plan. • Overlook options • Overlook constraints • Fail to fully inform clients about options and their potential costs and benefits 4. Implement plans. • The “dilution” effect (i.e., offering an ineffective version of plans) • Do not arrange for timely, corrective feedback • Use vague outcome measures 5. Evaluate results. • Use invalid measures • Fail to plan for generalization and maintenance • Do not gather both subjective and objective measures • Post hoc fallacy (assume that because there is a change, services were responsible) • Overlook harmful effects 6. Try again? • Give up too soon

labeling with understanding. Partiality in the use of evidence contributes to errors as well (e.g., jumping to conclusions).

Instructions

Complete Practice Exercise 23.1 and discuss related questions following your instructor’s guidance.

Gambrill & Gibbs Exercise 23 Reviewing Your Expertise 251 252 253

Practice Exercise 23.1 Reviewing Your Expertise

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ Date __ _ _ _ _ _ ______ _ _ _ _

Course__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ Instructor’s Name__ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _

Expertise Checklist

No. Competencies B LL G VG 1. Help clients to clearly describe hoped for outcomes and related 1 2 3 4 circumstances drawing on well argued theory and related research. 2. Accurately identify information needs and pose clear related 1 2 3 4 questions. 3. Locate and critically appraise research related to important 1 2 3 4 decisions in a timely manner. 4. Use valid assessment tools. 1 2 3 4 5. Accurately interpret information gathered. 1 2 3 4 6. Recognize important patterns, drawing on well-argued theory 1 2 3 4 and data. 7. Recognize important uncertainties. 1 2 3 4 8. Integrate different kinds of information effectively. 1 2 3 4 9. Respond effectively to cultural differences. 1 2 3 4 10. Avoid common biases and fallacies; use effective debiasing 1 2 3 4 strategies. 11. Recognize when individual differences call for deviation from 1 2 3 4 “treatment manuals.” 12. Offer high levels of common factors as tracked by ongoing 1 2 3 4 feedback.* 13. Suggest interventions compatible with assessment information 1 2 3 4 and client preferences. 14. Make well-reasoned tradeoffs among different kinds of evidence 1 2 3 4 (e.g., client preferences and research). 15. Arrange ongoing monitoring of outcomes.† 1 2 3 4 16. Arrange valuable learning opportunities. 1 2 3 4 17. Troubleshoot; overcome obstacles. 1 2 3 4

Gambrill & Gibbs Exercise 23 Reviewing Your Expertise 253 254

No. Competencies B LL G VG 18. Accurately estimate your innumeracy (e.g., see 1 2 3 4 www.innumeracy.com). 19. Recall valuable knowledge. 1 2 3 4 20. Identify important gaps in knowledge and skill (ignorance). 1 2 3 4 21. Revise views and strategies when needed. 1 2 3 4 22. Recognize the need for and acquire consultation. 1 2 3 4 23. Detect and minimize errors. 1 2 3 4 24. Involve clients as informed participants. 1 2 3 4 25. Avoid burnout. 1 2 3 4 26. Acquire needed resources. 1 2 3 4 27. Help clients attain hoped-for outcomes. 1 2 3 4 28. Arrange effective “handovers.” 1 2 3 4

B, beginner; LL, a lot to learn; G, good; VG, very good.

*Common factors include warmth, empathy, and quality of alliance (e.g., Wampold & Imel, 2015).

†A user-friendly feedback measure can be found at www.scottmiller.edu.

Source: Adapted from Gambrill (2013b).

Questions/Activities 1. We are often unaware that we are unskilled or lack important knowledge (e.g., Dunning et al., 2007). How could you explore the accuracy of your ratings? Draw on related research. 2. Select one of the items from the checklist in Box 23.1 that you would like to improve and suggest a plan for enhancing your expertise in this area as well as a way to monitor your progress.

254 Evidence-Informed Decision Making Gambrill & Gibbs 255

Practice Exercise 23.2 Errors

Your Name __ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ Date __ _ _ _ _ ______ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Describe an error you tend to make, drawing on Box 23.1.

Describe contributing causes, including agency characteristics.

Describe a plan for decreasing this error.

Does your agency track errors? ___ Yes __ No If yes, please describe.

Gambrill & Gibbs Exercise 23 Reviewing Your Expertise 255 256

If not, describe a method to do so, drawing on related literature.

Describe a mathematical mistake that influences practice/policy decisions (e.g., Cox, 1999; Paulos, 1988).

256 Evidence-Informed Decision Making Gambrill & Gibbs 257â

PART 5 Critically Appraising Research

Knowledge and skill in critically appraising research will help you to discover the evidentiary status of relevant studies and involve clients as informed participants. As emphasized on the website www.testingtreatments.org, we need fair tests of intervention because sometimes we get better anyway, new interventions may not be better, harm may result from acting on untested theories or flawed evidence, more treatment is not always better, earlier diagnosis is not always better, and sometimes we don’t know which intervention is best. Many sources of bias can intrude, intentionally or not, that compromise confidence in the results claimed (see Box 5.1). Inflated claims and misleading problem framing both in the media and in peer-â reviewed literature are common, as described in Part 1. Too often, the limitations of studies are not mentioned, are glossed over, or are minimized (Ioannidis, 2016). Poor reporting of a study does not necessarily mean it was poorly constructed; it may just be poorly reported (Soares et al., 2004). www. testingtreatments.org contains a pdf of the entire text of Testing Treatments (Evans, Thornton, Chalmers, & Glasziou, 2011) and a plain-âlanguage glossary of research terms. testingtreatments.org is a valuable website for both clients and professionals. In a chapter in this book entitled “Research: Good, Bad and Unnecessary,” the authors list the following key points: • Unnecessary research is a waste of time, effort, money, and other resources; it is also unethical and potentially harmful to patients.

257 258

Box 5.1 Examples of Potential Bias in Randomized Controlled Trials

DURING PLANNING Choice of question bias Regulator bias (e.g., internal review board requirements) Selection bias

DURING CONDUCT Ascertainment bias (not blinded)\ Population choice bias (may be overly narrow) Intervention choice bias Comparison (or control) group choice bias Outcome choice bias (relevant and/or just easy to measure)

DURING REPORTING Dropouts not reported Protocol violations not reported Selective reporting of results Data dredging bias

DURING DISSEMINATION Publication bias Language bias Time-lag bias

DURING UPTAKE Careless reader bias (does not read key sections of a report) Rivalry bias (does not like author, so ignores article) Personal habit bias (over- or underrates study because disagrees with personal beliefs) Clinical practice bias (disregards because disagrees with clinical experience) Prominent author bias (overrates value of studies by well-known authors) Printed word bias (overrates just because it is printed) Flashy title bias Geographic bias (judgment based on location) Favored design bias (dislikes design) Small-trial bias (underestimates value of small trial) Vested interest bias (e.g., uptake will decrease my profits) Belligerence bias (underrates value for the sake of being difficult) Empiricism bias (underrates because it challenges readers’ clinical experience) Institution bias (we don’t do things that way)

Source: Based on Jadad, R., & Enkin, M. R. (2007) Randomized controlled trials: Questions, answers and musings (2nd ed.). Malden, MA: Blackwell.

258 Critically Appraising Research Gambrill & Gibbs 259

• New research should only proceed if an up-to-date review of earlier research shows that it is necessary, and after it has been registered. • Evidence from new research should be used to update the previous review of all the relevant evidence. • Much research is of poor quality and done for questionable reasons. • There are perverse influences on the research agenda, from both industry and academia. • Questions that matter to patients are often not addressed. (p. 129) Different questions require different kinds of research. Surveys and qualitative research are used for descriptive studies. Experimental (e.g., randomized controlled trials [RCTs]) and observational studies are used to explore the effects of intervention. RCTs are also used to explore etiology and risk factors (if ethical). Alternatives to RCTs include cohort studies (outcomes are compared in matched groups with and without exposure to the risk factors [prospective study]) and case control studies (subjects with and without the outcome of interest are compared for previous exposure or risk factor [retrospective study]). The Critical Appraisal Skills Program (CASP) offers checklists for appraising the quality of different kinds of research, including qualitative research (see also www.testingtreatments. org). Checklists and flowcharts for reporting different kinds of research can be found at www.equator-network.org:

Synthesis of qualitative research (ENTREQ) Observational studies (MOOSE) Statistical reporting (SAMPL) Evaluations with nonrandomized design (TREND) Randomized trials (CONSORT) Observational studies (STROBE) Case reports (CARE) Systematic reviews (PRISMA) Qualitative research (SRQR) Diagnostic/prognostic studies (STARD) Quality improvement studies (SQUIRE) Economic evaluations (CHEERS) Animal preclinical studies (ARRIVE) Study protocols (SPIRIT)

Certain questions are important to raise across all research because of the potential for flaws that may result in misleading conclusions. These

Gambrill & Gibbs Critically Appraising Research 259 260

include concerns about the size and source of samples used, whether there is a comparison and/or control group, the accuracy and validity of measures used, and the appropriateness of data analysis. A review of these characteristics will shed light on both the internal and external validity of a study. Internal validity refers to whether a study provides valid (accurate) results. External validity refers to the extent to which results can be generalized to other populations/situations. Rival hypotheses must be considered. “If threats to valid causal inference cannot be ruled out in the design, they should at least be measured and their importance estimated” (Farrington, 2003, pp. 51–52) (see Box 5.2). Language in research reports may be misleading. An example Greenhalgh (2010) offers includes authors saying “We measured how often GPs ask patients whether they smoke” rather than “We looked in patients’ medical records and counted how many had their smoking status recorded” (p. 50). Here, there is the incorrect assumption that records are completely accurate, as Greenhalgh notes. Or, the authors may say, “We measured how doctors treat lower back pain.” They should have said, “We measured what doctors say they do when faced with a patient with lower back pain.” There is a mistaken assumption that “what doctors say they do reflects what they actually do.”

Box 5.2 Possible Confounding Causes (Rival Explanations) for Change

1. History: Events that occur between the first and second measurement, in addition to the experimental variables, may account for changes (e.g., clients may get help elsewhere). 2. Maturation: Simply growing older or living longer may be responsible, especially when long periods of time are involved 3. Instrumentation: The way something is measured changes (e.g., observers may change how they record) 4. Testing effects: Assessments that may result in change 5. Mortality: Different loss of people from different groups 6. Regression: Extreme scores tend to return to the mean 7. Self-selection bias: Clients are often self-selected rather than randomly selected (They may differ in critical ways from the population they are assumed to represent and differ from clients in a comparison group.) 8. Helper selection bias: When certain kinds of clients are selected to receive certain methods 9. Interaction effects: Only certain clients may benefit from certain services; others may even be harmed

Source: Based on Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-experimental Designs for Research. Chicago, IL: Rand McNally.

260 Critically Appraising Research Gambrill & Gibbs 261

Is the Research Question Clear?

Do the authors clearly describe their research question? Examples of clear research questions are: What factors contribute to the reabuse of children returned to their biological parents? Do substance abuse programs to which adolescents are referred help them to decrease alcohol consumption compared with no intervention? Unclear questions do not allow for clear tests at the point of data analysis, set in advance.

What Kind of Question Is It?

Does the article address the effectiveness of a practice method? Is it an assessment question? Does it describe a new risk assessment measure for depression in the elderly? What kind of question does it concern?

What Theories and Related Assumptions Are Involved?

Popper (1994) views theories as guesses as to what is true or false. Researchers always have assumptions and use concepts that may or may not be explicitly described. Systematic reviews are based on available research which is conducted within a certain viewpoint (e.g., that anxiety is a “brain disease,” that obesity is “a disease”). Are related assumptions true? What is the evidence? How well tested is the theory? Has it led to successful interventions?

Is It Relevant to My Clients? Is It Important?

Does the information apply to my clients? If you knew the answer, could you and your clients make more informed decisions? Does it concern outcomes of interest to your clients? Is the setting similar to your practice setting? Are the clients similar?

Does the Research Method Match the Question?

Can the research method used address the question (see Box 5.1)? Whether qualitative or quantitative research is best depends on the

Gambrill & Gibbs Critically Appraising Research 261 262

question. Oxman and Guyatt (1993) suggest using a scale ranging from 1 point (not at all) to 6 points (ideal) in relation to the potential that a research method can critically test a question.

Is There a Comparison Group?

Critically testing certain kinds of questions requires a comparison. Ransom distribution of subjects to two or more different conditions is a hallmark of randomized controlled trials (RCTs) (e.g., an alternate intervention and/or control). If all we have is a pretest and a posttest describing how anxious people are before and after some intervention, there is no comparison with a group receiving no service or a different one. Thus, there could be a variety of other reasons for changes seen (Howick et al., 2013) (see Box 5.1). Are comparison groups bona fide (e.g., intended to be therapeutic) (see Wampold & Imel, 2015a)?

Is the Study Design Rigorous?

The general research method may be appropriate but be carried out in an unrigorous manner that allows the play of many biases. Farrington (2003) suggests five methodological criteria: (1) internal validity, demonstrating that the intervention caused an effect on the outcome: (2) descriptive validity, information about key features of research; (3) statistical conclusion validity (see Exercise 24); (4) construct validity (see Exercise 26); and (5) external validity.

What Is the Sample Size and Source?

Most research involves a sample that is assumed to be characteristic of the population from which it is drawn. Selection biases are one kind of bias related to how subjects are selected. Does the sample on which a study is based offer a sound opportunity to answer questions raised? Can we accurately generalize from a sample to the population from which it is drawn? Questions here include the following: • Is the sample selection process clearly described? • How was the sample selected?

262 Critically Appraising Research Gambrill & Gibbs 263

• From what population was it selected? • Is it representative of the population? • Were subjects lost at follow-up? If so, how many? The answers to these questions provide clues about biases that may limit the value of a study to answer questions. Small samples drawn by convenience, rather than by random selection, may not provide information that reflects characteristics of the population of interest. Often, researchers do not clearly describe the source of their sample. A number of “filtering” decisions may be made to obtain a final sample. CONSORT reporting guidelines for RCTs include a flowchart for describing samples used. We can see how many people were excluded at different points and for what reasons. Sample size and the testing of hypotheses are closely related. That is, some studies do not find effects—not because there are no effects to be found, but because the sample size does not have the power to test whether there is an association. As Farrington (2003) notes, “a statistically significant result could indicate a large effect in a small sample or a small effect in a large sample” (p. 52). A clear description of the source of samples used is important in qualitative as well as quantitative research.

Are Measures Used Reliable and Valid?

The validity of measures is a key concern in all research. Reliability places an upward boundary on validity. That is, a measure cannot be valid if it is not reliable (cannot be assessed consistently). And a measure may be reliable but invalid, perhaps because of shared biases among raters and/or because the assumptions on which they are based are faulty.

Did Authors Report Attrition (Dropout Rates)?

Subjects who drop out during the study should be reported and is reflected in an intention-to-treat analysis. This involves

analysis of experimental data in which participants remain in the group to which they were originally assigned,

Gambrill & Gibbs Critically Appraising Research 263 264

regardless of whether they received that treatment” (Littell, Corcoran, & Pillai, 2008, p. 175)

Was There Any Follow-Up? If So, How Long?

An intervention may be effective in the short term but not in the long term. How long were subjects followed up. do not change followed up to monitored?

Are Procedures Clearly Described ?

Are the interventions used clearly described? If not, it will not be possible to replicate them.

Are the Data Analyses Sound?

Statistics are widely used to explore relationships among variables; we ask what is the probability of finding an association by chance in samples of different sizes. The statistical approach dominates much of research, as illustrated in the use of RCTs and hierarchies of evidence. Inferences are made about population parameters based on samples; we estimate these parameters. Is the population parameter normally distributed, as assumed? You should have rudimentary knowledge of statistics, including ways in which statistics are misused. Misleading statistical analyses may result in bogus claims. Examples of ways to cheat described by Greenhalgh (2010) include: “Ignore all ‘dropouts’ and nonresponders,” If your data analysis “does not give the result you wanted, run the figures through a selection of other tests” (p. 62). Authors of an article using many variables with a large sample may claim that five significant differences were found. How many correlations were run? A certain percentage would be significant by chance. Other problems include failing to adjust for baseline differences, ignoring withdrawals and nonresponders, and ignoring outliers. Penston (2010) argues that selective reporting and data manipulation have become “standard practice” (p. 19). Examples include describing results in terms

264 Critically Appraising Research Gambrill & Gibbs 265

of relative risk rather than absolute risk (see Exercise 27) and conducting ever-larger RCTs in pursuit of significance. He argues that “the size of the study is inversely proportionate to … knowledge of the subject matter, the size of treatment effect, the value of the results to individual patients, and the overall importance of the study” (Penston, 2010, p. 23) and that the small effects produced by large RCTs “are seldom important” and are “a poor guide of cause and effect relationships” (p. 25). He notes that there is a “leap from statistical significance to causation” (p. 31). A variety of biases compromise results. (See Part 3.) Data describing individual variations are lost in changing a continuous variable into a binary one (either one is or is not an alcoholic). Consider drinking. One could have no drinks, one drink, or many drinks per day. Journal Club Presentation Resource describes specific kinds of tests, kinds of biases, odds ratios, confidence intervals, number needed to treat, and survival analyses (http://libraryguides.neomed.edu). See also “Top 10 Ways to Save Science From Its Statistical Self” (Siegfried, 2015). The term statistical significance refers to whether a test falls at or below a certain probability (e.g., .01 or .05). We may assume there is a relationship when there is not (type I error) or assume there is no relationship when there is (type II error). Different statistical tests make different assumptions about variables in relation to their underlying distribution (e.g., are variables continuous or discontinuous?). Statistical testing is based on the frequentist theory (e.g., Fisher’s significance testing and, Neyman-Pearson hypothesis) (Penston, 2010). Critiques of this approach have appeared for decades, yet most are ignored (e.g., Oakes, 1986). Criticisms include (1) uncertainties about the underlying distribution of variables, (2) failure to satisfy assumptions of tests used, (3) unfounded claims of objectivity, (4) problems with statistical theory and causation, and (5) lack of application to individuals (“probabilities based on relative frequency don’t apply to individuals” [Penston, 2010, p. 119]) (see also Colquhoun, 2014; Ziliak & McCloskey, 2013). Reliance on statistical appraisals encourage dichotomous thinking. Penston (2010) argues that reliance on confidence intervals has problems (p. 137). Yet, significance testing is widely used in research and is often not questioned. Complex statistical methods will not correct major flaws in the design or conduct of a study. Researchers as well as practitioners make mistakes in how they word findings. Rather than stating “no statistically significant difference was found,” they may say there was “no difference/change” (Weisburd, Lum, & Yang, 2003).

Gambrill & Gibbs Critically Appraising Research 265 266

Are Claims Accurate?

Problems in any of the characteristics discussed, including samples, measures used, and data analysis, may not allow clear conclusions. Possible confounding causes may be overlooked (see Box 5.2). Do claims match the kind of design used? For example, authors may claim “Our results show that X was effective” based on a pretest and a posttest. Such a design cannot tell us whether the intervention was responsible for the results because there is no comparison group. Experimenter biases influence results in a number of ways. Experimenter effects are not necessarily intentional. If the experimenters know the group to which a subject is assigned, they may change their behavior—for example, subtly lead the person in a certain direction. This is why it is vital in RCTs for raters of outcome to be “blind” (unaware of the group to which a person is assigned). Biases in the conduct, reporting, and use of research findings are common. Some argue that placebo effects account for as much or more than the effects of intervention (e.g., Antonuccio, Burns, & Danton, 2002; Wampold, Minami, Tierney, Baskin, & Bhati, 2005). Research suggests that selective serotonin reuptake inhibitors prescribed to decrease depression do not help most depressed people more than placebos (Kirsch et al., 2008; Turner, Matthews, Linardatos, Tell, & Rosenthal, 2008).

Are Findings Clinically Important?

Will research findings be of value in helping clients? Is the intervention effective in real-world circumstances? Weisburd, Lum, and Yang (2003) argue that even a modest reduction in future delinquency is important. What may be true of a group may not be true of an individual; thus, aggregate studies must be interpreted with caution in relation to generalizing to an individual. Otherwise you may make the ecological fallacy—assume that what is true of a group is true of an individual.

Who Sponsored the Study? Did Authors Report Conflicts of Interest?

Sponsorship of a study may suggest possible biases. Studies conducted by or on behalf of pharmaceutical companies report more positive findings

266 Critically Appraising Research Gambrill & Gibbs 267

compared with those that are conducted independently (e.g., Lexchin, Bero, Djulbegovic, & Clark, 2003; see also Goldacre, 2011). Conflicts of interest may bias conclusions, including the development of practice guidelines and diagnostic classification systems such as the DSM (see Part 1). For example, most members of some DSM-5 task forces had financial ties to drug companies (Cosgrove & Krimsky, 2012). Did a drug company fund the study? Sponsorship of research or a continuing education program by a company with vested interest in a product, such as a pharmaceutical company, encourages presentation of biased material (e.g., Brody, 2007). In Part 5, Exercise 24 provides guidelines for evaluating effectiveness studies focusing on RCTs. Exercise 25 describes criteria for critically appraising research reviews and includes a related exercise. Guidelines for critically appraising self-report measures are offered in Exercise 26. Exercise 27 suggests guidelines for estimating risk and making predictions. Exercises 28 and 29 provide an opportunity to review a diagnostic test and a classification system. Exercise 30 suggests important points to check when critically appraising research regarding causes. Take advantage of checklists and flowcharts for reporting and critiquing different kinds of research.

Gambrill & Gibbs Critically Appraising Research 267 268 269

EXERCISE 24 EVALUATING EFFECTIVENESS STUDIES: HOW GOOD IS THE EVIDENCE?

Purpose

To identify the hallmarks of well-designed intervention studies and to accurately appraise practice and policy-related research.

Background

This exercise is designed to help you to enhance your knowledge and skills in evaluating the quality of effectiveness studies, focusing on randomized controlled trials (RCTs). There are many other kinds of studies investigating effectiveness, including observational studies and single case designs. Here is an example of a hierarchy regarding quality of evidence: 1. Evidence from a high-quality systematic review (see Exercise 25) 2. Evidence from at least one well-designed RCT 3. Evidence from well-designed controlled trials without randomization 4. Evidence from well-designed cohort or case-controlled analytic studies, preferably from more than one center or research group 5. Evidence from multiple time series with or without the intervention; dramatic results in uncontrolled experiments (e.g., the results of the introduction of penicillin in the 1940s) 6. Opinions of respected authorities based on clinical experience, descriptive studies, and case reports, or reports of expert committees (Berg, 2000, p. 25) Suggested hierarchies of evidence may be misleading in ignoring limitations of research, including RCTs. Characteristics of fair tests of intervention noted on www.testingtreatments.org include the following: • They take into account the benefits of optimism and wishful thinking (placebo effects) • They compare like with like • They allocate participants randomly to different interventions

269 270

• They follow up with everyone who took part • They use a fair measure of outcome • They consider the play of chance (e.g., adequate sample sizes) • They consider results together with those of all other relevant studies Some interventions have such dramatic effects that bias is unlikely in assuming the intervention was responsible, such as insulin for diabetes (Glasziou, Chalmers, Rawlins, & McCulloch, 2007). Data regarding changes in quality of life for clients/patients require both qualitative (subjective) and quantitative (objective) information. As discussed in Part 1, we must be skeptical of all claims, including those in peer-reviewed literature (e.g., McGauran et al., 2010). Key features of RCTs include random distribution of clients to different groups (e.g., another intervention and/ or control group) and a comparison of outcomes across groups. Unlike pre/post studies, which are subject to many rival hypotheses regarding outcomes (see Box 5.2), there is a comparison. What does a study tell you (if anything) about the effectiveness of a method compared with another possibility, such as watchful waiting? How does effectiveness (results in real-world conditions) compare with efficacy (results under ideal conditions)? Many kinds of biases may intrude in RCTs (see Box 5.1). Biases include publication bias, delayed publication, duplicate publication, ease- of-access bias, noncitation bias, language bias, and outcome reporting bias (selective reporting). Lack of a comparison group allows the play of alternative explanations (see Box 5.2). Questions for appraising an RCT suggested by CASP include the following: • Did the trial address a clearly focused issue? • Are people in the study similar to my clients? • Was assignment of clients to intervention randomized using an appropriate method? • Was everyone included in the study blind to intervention? • Were groups similar at the start of the trial? • Aside from the experimental intervention, were groups treated equally? • Were all clients who entered the trial accounted for properly at its conclusion? • Are results statistically as well as clinically significant? • How precise was the estimate of the treatment? • Can results be applied in your context? Or to the local population?

270 Critically Appraising Research Gambrill & Gibbs 271

• Were all clinically important outcomes considered? • Are the benefits worth the harms and costs? Wampold and Imel (2015a) argue that many RCTs concerning the effectiveness of psychotherapy are not useful because they do not include a bona fide comparison group—one intended to be therapeutic. Misleading practices in the conduct of RCTs funded by pharmaceutical companies include failure to report dropout rates resulting from adverse side effects, using very high doses of comparison drugs, multiple publication of the same study, hiding negative trials, short time periods of use, lack of follow-up, and exclusion of people with co-occurring problems. (See, for example, the discussion by Whitaker and Cosgrove [2015] of the marketing of selective serotonin reuptake inhibitors.)

Instructions

Appraise a study selected by your instructor. At the top of 24.1 note (1) the type of client who participated (e.g., dyslexic children, older persons with Parkinson’s disease), (2) the intervention method(s) used, (3) important outcome measures, and (4) the reference for the study in American Psychological Association format. Complete items 1 through 17 in Box 24.1. Place a check in the relevant section if the study meets the criterion.

1. The author describes who is included by stating the subjects’ average age, proportion of males and females, and characteristics (e.g., those labeled schizophrenic). 2. The authors clearly describe the intervention or they refer you to a book, videotape, or article that describes the intervention. 3. The authors state where the intervention occurred. 4. The authors describe the when of intervention (how long subjects participated in days, weeks, or months, or how many sessions were attended). 5. The authors either discuss a specific theory that describes why they used the intervention methods selected or they cite relevant literature. 6. Random selection means subjects are taken from some pool of subjects for inclusion in a study by using a table of random numbers or other random procedures.

Gambrill & Gibbs Exercise 24 Evaluating Effectiveness Studies: How Good Is the Evidence? 271 272

Box 24.1 Quality of Study Rating Form

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ Date __ _ _ _ _ _ _ _ _ _ ______ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name ______ _ _ _ _ _ _ _ _ _ _ _____ _ _ _ _ _

Client type(s) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Intervention method(s) ______ _ _ _ _ _ ___ _ _ _

Outcome measure to compute effect size (ES) 1 __ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Outcome measure to compute ES2 __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Outcome measure to compute ES3 __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _

Source (in American Psychological Association format) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _

Clear Definition of Intervention 1. Who 2. What 3. Where 4. When 5. Why 6. Subjects 7. Subjects 8. randomly randomly Subjects assigned assigned blind to to different group groups assignment 9. 10. There 11. 12. 13. Valid outcome measures are used Analysis is a control Number Reliability shows (untreated) of subjects of out- equal group in small- come groups est group measures before exceeds is greater interven- 20 than .70 tion or rater agreement is more than 70% 14. Raters 15. 16. 17. of out- Outcome Follow-up Statistical come was mea- was more signifi- were blind sured after than 75% cance was to group interven- explored assign- tion was ment completed

Adapted from Gibbs (2003). See also Gibbs (1989, 1991). 273

7. The authors state subjects were randomly assigned to groups or refer to the assignment of subjects to intervention or control groups on the basis of a table of random numbers or other accepted randomization procedure. Randomization implies each subject has an equal chance of being assigned to either an intervention group or a control group. If authors “place” subjects by assigning every other one to a different group or allow subjects to choose their group, the study participants are not assigned randomly. 8. Subjects were blind to group assignment. 9. Analysis shows subjects were similar on key variables before intervention. 10. Members of the untreated control group receive no intervention. Subjects in the control group might receive intervention at a later date. If a comparison group consists of another kind of psychothera- peutic intervention, is the latter a bona fide one (one intended to be therapeutic)? Does it contain common factors that contribute to positive outcomes such as allegiance effects, empathy, and warmth, in addition to another kind of intervention intended to be therapeutic (Wampold & Imel, 2015a)? 11. The number of subjects in the smallest intervention group should be determined by a power analysis. 12. Reliability of measures used is clearly described (see Exercise 26). The authors describe the kind(s) of reliability explored, include the most important kinds, give numerical values, and describe sample sizes and sources, and the reliability coefficient is 0.70 or greater. 13. Validity of outcome measures used is clearly described. (For a discussion of different kinds of validity, see Exercise 26.) Kind(s) of validity explored are noted, and related figures, samples sizes, and sources given. The most important kind of validity is explored. Reliance on surrogate indicators that do not reflect important outcomes (such as mortality) is a deceptive practice. For example, plaque in arteries may be used as a misleading surrogate for mortality. 14. Raters of outcome were blind to group assessment. 15. Outcome measures were obtained both before and after intervention. 16. The authors include an “intention-to- treat” analysis. The proportion of subjects successfully followed up refers to the number contacted to measure outcome compared with the number who began the study. (If there is more than one follow-up period, use the longest one.) Then, for each group, divide the number of subjects followed up by

Gambrill & Gibbs Exercise 24 Evaluating Effectiveness Studies: How Good Is the Evidence? 273 274

the number that began in each group and multiply each quotient by 100. For example, if twenty participants entered an intervention group, but only fifteen were followed up in that group, the result would be (15 ÷ 20) × 100 = 75%. Compute the proportion followed up for all groups. If the smallest of these percentages equals or exceeds 75%, the study meets the criterion. 17. The authors make appropriate use of statistical tests (e.g., analysis of variance, chi square, t-test) and gives p values, confidence intervals, and effect sizes. Statistical testing is controversial, and misunderstandings are common (see the introduction to Part 5). Effect size is a measure of the strength and direction of a relationship between variables. Effect sizes may refer to mean differences between groups (Cohen’s d), correlation/regression (e.g., Pearson’s r), and contingency tables (odds ratios or relative risk association between two variables). An effect size approaching zero suggests there is no difference in effectiveness of the compared groups. A negative effect size (ES) 1 suggests a harmful (iatrogenic) effect. As a rough rule, a small effect size is .2; a medium one, about .5; and a large one, .8 or greater (Cohen, 1977, p. 24). Effect size can be computed as follows:

X − Xc (Mean of intervetnion –r Mean of control or alternate intervention group) ES1 = t = . Sc Standard deviation of control or alternative intervention

This formula is for computing ES1 when outcome means of both groups are given. (See Effect Size Calculator www.campbellcollaboration.org.) To compute an effect size from information in an article, select two means to compare (e.g., mean of an intervention group and control group). Subtract the mean of the second group from the mean of the first group and divide this value by the standard deviation of the second group. If the number is positive, the first group may have the greater effect (this assumes that a positive outcome on a measure implies larger numbers on the measure). The larger a positive ES1, the stronger the effect of intervention may be.

274 Critically Appraising Research Gambrill & Gibbs 275

18. We can also compute ES2 for proportions or percentages:  Number improved in intervention  ESP2: −=P ×100 tc Total number in iintervention group

 Number improved in alternative intervention or control −   ×1000.  Totaal number in alternative intervention or control 

This effect size measures the difference between the percent of subjects improved in one group compared with the percent improved in another intervention (or control) group. If 30% improve in one intervention group and 20% improve in the other, then ES2 is 10% (i.e., 30% – 20% = 10%). Many studies fail to include sufficient information to compute ES2. A third effect size (ES3) is computed as follows: Number needed to treat =÷100 ES2 (Furuka & Leucht, 2011). Odds ratios can also be calculated to determine the association between two binary factors. We calculate the odds of the chance that something will occur compared with the chance it will not. If the odds ratio is greater than one, an association is assumed (note that odds are not risks) (e.g., Littell, Corcoran, & Pillai, 2008).

Discussion Questions

1. Discuss the difference between specific medical interventions (such as use of antibiotics for infections) and psychotherapy (including the role of common factors such as empathy, warmth, and allegiance effects). Wampold and Imel (2015a) argue that such common factors account for more variance in outcome of psychotherapy than specific interventions. 2. Discuss the role of placebos in medical and psychotherapy research, drawing on relevant resources (e.g., Kaptchuk & Miller, 2015; Wampold et al., 2005).

Gambrill & Gibbs Exercise 24 Evaluating Effectiveness Studies: How Good Is the Evidence? 275 276

3. A confidence interval “offers a range of values within which it is probable that the true value of a parameter (e.g., a mean or relative risk) lies” (Guyatt et al., 2015). Discuss the difference between effect size and confidence intervals. 4. Describe misuses of effect size drawing on related sources (e.g., Penston, 2010).

276 Critically Appraising Research Gambrill & Gibbs 277

Practice Exercise 24 Evaluating Effectiveness Studies: How Good Is the Evidence?

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ Course __ _ _ _ ______ _ _ _ _ _ _ _ _ _ _

1. Read the research report selected by your instructor. Give a complete reference (in American Psychological Association style) here.

2. What desired outcome is of interest in this study?

3. How is the problem framed (e.g., biological, social, psychological)?

Does the empirical literature support such a framing? ___ Yes ___ No If yes, cite best source here.

Are well-argued alternatives described? __ _ Yes __ _ No If yes, what is an alternative view?

Gambrill & Gibbs Exercise 24 Evaluating Effectiveness Studies: How Good Is the Evidence? 277 278

4. Appraise this study using information in Exercise 24. You could also consult the CONSORT reporting flowchart and checklist. (Ask your instructor for clarification as needed.)

5. Explore effect size (ES).

a. What is ES1?__ _ _ _ _ _ _ _ _ _ b. What is ES2? __ _ _ _ _ _ _ _ _ _ _ _ _ c. What is number needed to treat? __ _ _ _ _ _ _ _ _ _ _ _

6. Describe any biases that compromise conclusions, such as comparison of an intervention with an alternative that is not bona fide (e.g., see Box 5.1).

7. Were results overstated? __ _ Yes __ _ No Please describe the reasons for your answers.

8. Would you recommend this intervention? __ _ Yes __ _ No

278 Critically Appraising Research Gambrill & Gibbs 279

EXERCISE 25 CRITICALLY APPRAISING RESEARCH REVIEWS AND PRACTICE GUIDELINES

Purpose

To accurately evaluate the quality of research reviews and practice/policy guidelines, and to make informed decisions.

Background

Research reviews differ in their rigor, complexity, and the clarity with which procedures used are described (Ioannides, 2016; Littell, Corcoran, & Pillai, 2008) (see also Rapid Evidence-Assessments from HLWIKI Canada [www.hlwiki.slais.ubc.ca]). Different kinds of reviews include the following (Pettigrew & Roberts, 2006):

Type of Review Description Systematic Exhaustive search for research including the gray literature (e.g., unpublished studies) related to a clear question using transparent, rigorous criteria for reviewing quality of research found (see Cochrane and Campbell Libraries); systematic reviews may or may not include a meta-analysis (a statistical analysis of effect sizes) Narrative Does not use systematic methods; may or may not be (traditional) rigorous Expert Reviews written by acknowledged experts or group of experts State-of-the-art Designed to bring readers up-to-date on recent review research on a topic Conceptual Reviews of theories related to a problem/question Scoping Review of type of studies available carried out in advance of full review Realist Studies synthesized to produce general theories Critical Critically examines methods and results of primary studies related to theory without using formal appraisals of systematic review (pp. 39–41)

279 280

Concerns about incomplete, unrigorous reviews resulted in the creation of the Cochrane and Campbell Collaborations, which prepare, dissem- inate, and maintain reviews regarding specific questions, such as: Are Scared Straight programs for preventing delinquency effective (Petrosino, Turpin-Petrosino, Hollis-Peel, & Lavenberg, 2013)? A key purpose of a systematic review is to discover the evidentiary status of an intervention, such as multisystemic family therapy (Littell, Popa, & Forsythe, 2005). Results may indicate benefit, harm, or, most often, a lack of clarity concerning outcome. Rigorous systematic reviews are designed to reveal flaws in research studies such as selection bias, misleading data analyses, and publication bias (among others; see Box 5.1). (For reporting guidelines see PRISMA.) Criteria for evaluating credibility of systematic reviews suggested by Murad et al. (2014) include the following: • Did the review address a clear question? • Did the authors look for the right type of papers? • Was the search for relevant studies exhaustive (including the gray literature)? • Were research studies critically evaluated? • If study results were combined, was it appropriate to do so? • What were the overall results of the review? • How serious is the risk of bias in studies reviewed? • Are the results consistent across studies? • How precise are the results? • Do the results directly apply to my clients? • Is there concern about reporting bias? • Were all important outcomes considered? • Are the benefits worth the harms and costs? • What is the clinical bottom line? Conflicts of interest may compromise quality of systematic reviews as well as practice guidelines (e.g., Forsyth, Odierna, Krauth, & Bero, 2014). Disclosure of industry ties in editorial comments, letters regarding policy, and related systematic reviews, is often incomplete (Forsyth et al., 2014). As Straus, Richardson, Glasziou, and Haynes (2005) caution: misleading systematic reviews may result in faulty decisions. Systematic reviews of intervention effectiveness pay little or no attention to how problems are framed in research reviewed, including Cochrane and Campbell reviews (Gambrill, 2014b). Both qualitative as well as quantitative data may be important in answering a question. Sources for reviewing qualitative

280 Critically Appraising Research Gambrill & Gibbs 281

research include Greenhalgh (2010). A checklist for reviewing survey data is also included in Greenhalgh (2010).

What About Practice/Policy Guidelines? Thousands of practice guidelines have been created (e.g., Institute of Medicine, 2011). Can we trust them? Does a guideline include an economic analysis (Ramsey, 2002)? Lenzer (2013) argues that we cannot automatically accept claims because of ongoing “conflicts of interest among authors and sponsors of clinical guidelines” (p. 1). Guidelines sponsored by pharmaceutical companies tend to be biased (e.g., Angell, 2009). Are they relevant to your client (see related material in Norcross, Beutler, and Levant [2006])? Just as research reviews differ in their rigor and inclusiveness, practice and policy guidelines differ in their evidentiary status (see Greenhalgh, 2010).

Instructions

Please complete Practice Exercises 25.1 and 25.2. The example in Practice Exercise 25.1 concerns an effectiveness question. Your instructor may also give you practice in critically appraising a review regarding a diagnostic test, assessment measure, quality of life measure, or risk factor, for example. (See Exercise 25.2.)

Gambrill & Gibbs Exercise 25 Critically Appraising Research Reviews and Practice Guidelines 281 282 283

Practice Exercise 25.1 Critically Appraising Research Reviews: How Good Is the Evidence?

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ ______ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ Instructor’s Name __ _ _ _ _ ____ _ _ _ _ _ _ _ _ _ _ _ _ _ _

1. Your instructor will select a review of an effectiveness question for you to evaluate. Access it on the Internet or obtain a copy from your instructor. Write a full reference here:

2. How is the problem framed? Are related controversies accurately described? Does the framing match the rationale for interventions used?

3. Download CASP’s Critical Appraisal Checklist for Systematic Reviews (www.casp-uk.net ) and complete it related to the review article selected (see also Miser, 2000). You can also review the checklist and flowchart in PRISMA reporting guidelines www.prisma-( state - ment.org) available on Equator (www.equator-network.org). See also “How to Read a Systematic Review and Meta-analysis and Apply the Results to Patient Care” by Murad et al. (2014); Guyatt et al. (2015), and GRADE guidelines.

4. If the review concerns an effectiveness or prevention question, prepare a forest plot of effect sizes of all trials (Littell et al., 2008). Your instructor will give you examples of such plots and discuss their value. Review material discussing confidence intervals and effect sizes.

Gambrill & Gibbs Exercise 25 Critically Appraising Research Reviews and Practice Guidelines 283 284

5. Prepare a brief overall critique of this review.

6. What is the clinical or policy “bottom line?”

Further Activities You could select a systematic review that addresses a question calling for qualitative data.

284 Critically Appraising Research Gambrill & Gibbs 285

Practice Exercise 25.2 Critically Appraising a Practice Guideline

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ ______ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ ______ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _

1. Select a practice guideline that pertains to your work with clients. Note this here as well as its source.

2. Download and apply the Agree II guidelines (www.agreetrust.org). See also Guyatt et al. (2015).

3. Read “Why We Can’t Trust Clinical Guidelines” by Jeanne Lenzer (2013). Describe any instances of the following concerning the guidelines you reviewed: the manufacturing of consensus, exaggerated claims, conflict of interests.

4. Would you recommend using this guideline? ___ Yes ___ No

Gambrill & Gibbs Exercise 25 Critically Appraising Research Reviews and Practice Guidelines 285 286 287â

EXERCISE 26â CRITICALLY APPRAISING SELF-âREPORT MEASURES

Purpose

To enhance skills in critically appraising self-âreport measures.

Background

Hundreds of self-âreport measures are described in the professional literature. Are they valid? Do they measure what they claim to measure? Assessment provides a foundation for intervention. A key part of assessment is clearly describing client concerns and related client characteristics and circumstances. Examples of vague descriptions include “antisocial behavior” and “poor parenting skills.” Invalid self-âreport measures may give an incorrect view of a client’s concerns, repertoires, and life circumstances. You may be misled by initial impressions and fail to change your views in light of new information. (See discussion of anchoring and insufficient adjustment in Part 3.) Misleading data can result in selection of ineffective or harmful interventions. Biases that interfere with accurately describing concerns and related factors are more likely to remain unrecognized when descriptions are vague. You may be misled by the vividness of behavior, such as extreme temper tantrums, and may overlook less vivid alternative positive behaviors that can be increased to function as competing alternative repertoires.

Some Useful Concepts A measure is reliable when different observers arrive at very similar ratings using that measure; it is valid when it measures what it is designed to measure. Assuming standardized measures are valid is mistake.

Reliability refers to the consistency of results provided by the same person at different times (time-âbased reliability), by two different raters of the same events (interrater reliability), or by parallel forms or split-â halfs of a measure (item-âbound reliability). The first kind is known as

287 288

test–retest reliability, or stability. Reliability places an upward boundary on validity. Unreliable measures cannot be valid. For example, if responses on a questionnaire vary from time to time in the absence of real change, you cannot use it to predict what a person will do in the future. Reliability can be assessed in a number of ways, all of which yield some measure of consistency. With test–retest reliability, the scores of the same individuals at different times are correlated with each other. We might administer the Beck Depression Inventory to several persons whom we think might be “depressed,” then administer it again with the same instructions a few days or weeks later to determine whether the scores are similar over time. Correlations may range from +1 to –1. The size of the correlation coefficient indicates the degree of association. A zero correlation indicates a complete absence of consistency. A correlation of +1 indicates a perfect positive correlation. The stability (reliability of a measure at different times in the absence of related events that may influence scores) of some measures is high. That is, you can ask a client to complete a questionnaire this week and five weeks from now, and obtain similar results (in the absence of real change). Other measures have low stability. Coefficients of reliability are usually sufficient if they are .70 or better. However, the higher the better. Homogeneity is a measure of internal consistency. It assesses the degree to which all the items on a test measure the same characteristics. The homogeneity of a test (as measured, for example, by “coefficient alpha”) is important if all the items on it are assumed to measure the same characteristics. If a scale is multidimensional (e.g., many dimensions are assumed to be involved in a construct, such as “loneliness” or “social support”), then correlation among all items would not be expected. We could calculate the internal consistency by computing the correlations of each item with the total score of a measure and averaging these correlations. We could compute a measure’s split-half reliability by dividing the items randomly into two groups of ten items each, administering both halves to a group of subjects, then seeing if the halves correlate well with each other. Validity concerns the question: Does the measure reflect the characteristics it is supposed to measure? For example, does a client’s behavior in a role play correspond to what the client does in similar real-life situations? Direct measures are typically more valid than indirect measures. For instance, observing teacher–student interaction probably

288 Critically Appraising Research Gambrill & Gibbs 289

offers more accurate data than asking a student to complete a questionnaire assumed to offer information about the classroom. There are many kinds of validity. Predictive validity refers to the extent to which a measure accurately predicts behavior at a later time. For example, how accurately does a measure of suicidal potential predict suicide attempts? Can you accurately predict what a person will do in the future from his or her score on the measure? (For a valuable discussion of challenges in predicting future behavior and the importance of considering base-rate data, see Faust [2007].) Concurrent validity refers to the extent to which a measure correlates with a valid measure gathered at the same time. For example, do responses on a questionnaire concerning social behavior correspond to behavior in real-life settings? Criterion validity is used to refer to predictive and concurrent validity. Content validity refers to the degree to which a measure adequately samples the domain being assessed. For example, does an inventory used to assess parenting skills include an adequate sample of such skills? Face validity refers to the extent to which items included on a measure make sense “on the face of it.” Given the intent of the instrument, would you expect the included items to be there? For example, drinking behavior has face validity as an outcome measure for decreasing alcohol use. Construct validity refers to the degree to which a measure successfully measures a theoretical construct—the degree to which results correspond to assumptions about the measure. For example, a finding that depressed people report more negative thoughts compared with nondepressed people adds an increment of construct validity to a measure designed to tap such thoughts. Different methods of assessing a construct (e.g., direct observation, self-report) should yield similar results. Do scores on a measure correlate in predicted ways with other measures? They should have positive correlations with other measures of the same construct, and negative correlations with measures that tap opposite constructs.

Instructions

Your instructor will select an assessment measure for you to review or will ask you to select one used in your agency. Please use this to complete Practice Exercise 26.

Gambrill & Gibbs Exercise 26 Critically Appraising Self-Report Measures 289 290 291

Practice Exercise 26 Critically Appraising Self-Report Measures

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

1. Indicate the measure to be reviewed.

2. Describe the purpose of this measure.

3. Describe the reliability of this measure. What kind of reliability was evaluated? What were the results? Give facts and figures—for example, size of correlations and samples used, including source and size. Was the kind of reliability reported the most important?

4. Describe the kind of validity evaluated. What were the results? Give facts and figures—for example, the size of correlations found, and sample sizes and sources. Was this the most important kind of validity to report?

Gambrill & Gibbs Exercise 26 Critically Appraising Self-Report Measures 291 292

5. Are claims made regarding the reliability and validity of this self-report measure accurate based on your review? __ _ Yes __ _ No Please discuss. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

6. Describe ethical problems in using self-report measures of unknown or low reliability and validity.

292 Critically Appraising Research Gambrill & Gibbs 293â

EXERCISE 27â ESTIMATING RISK AND MAKING PREDICTIONS

Purpose

To introduce you to concepts basic to risk assessment and prediction, to increase your skills in spotting misleading information (e.g., relative risk), and to provide easy ways to estimate risk, such as use of natural frequencies.

Background

Assessing risk and communicating it accurately to clients is an important skill. Mental health staff members assess risk of suicide. Risk of future child abuse may be estimated based on the correlation between certain characteristics and the outcome, as in predictive risk modeling (Vaithianathan et al., 2013). Typically, we neither calculate risk accurately nor communicate it clearly to clients (Gigerenzer, 2014a & b; Paling, 2006; Paulos, 1988; Woloshin, Schwartz, & Welch, 2008). This has resulted in overdiagnosis and overtreatment. Vague words such as probably, likely, or high risk have very different meanings to different people. We tend to assess risks based largely on our emotions rather than on facts, and we overreact to descriptions of relative risks that provide misleading views of risk and effectiveness of a treatment (Slovic, 2010). Inaccurate information in brochures and ads is the norm rather than the exception (Meyer, Steckelberg, & Mühlhauser, 2007; Gigerenzer, 2014a). This example from Gigerenzer (2002a), concerning the reporting of HIV test results, shows just how inaccurate counselors may be in describing risk:

293 294

Session 1: The Counselor Was a Female Social Worker

Sensitivity?

• False negatives really never occur. Although, if I think about the literature, there were reports of such cases. • I don’t know exactly how many. • It happened only once or twice.

False positives?

• No, because the test is repeated; it is absolutely certain. • If there are antibodies, the test identifies them unambiguously and with absolute certainty. • No, it is absolutely impossible that there are false positives; because it is repeated, the test is absolutely certain.

Prevalence?

• I can’t tell you this exactly. • Between about 1 in 500 and 1 in 1000.

Positive predictive value?

• As I have now told you repeatedly, the test is absolutely certain.

The counselor was aware that HIV tests can lead to a few false negatives but incorrectly informed Ebert that there are no false positives. Ebert asked for clarification twice, in order to make sure that he correctly understood that a false positive is impossible. The counselor asserted that a positive test result means, with absolute certainty, that the client has the virus; this conclusion follows logically from her (incorrect) assertion that false positives cannot occur. (pp. 129–130)

Sensitivity refers to the proportion a test or measure indicates as having a problem among those known to have a problem. Specificity refers to the proportion the test or measure indicates as not having the problem among those known not to have the problem. Base rate (prevalence) refers to the proportion of people showing some characteristics. The predictive value of a positive test is greater when the base rate is relatively high. Ignoring base- rate data is a key cause of overestimating the predictive accuracy of test results. The predictive accuracy of a test depends on the initial risk of a

294 Critically Appraising Research Gambrill & Gibbs 295

characteristic in the person receiving the test. Test accuracy varies, depending on whether a test is used as a screening device, in which there are large numbers of people who do not have some condition of interest, or whether it is used for clients with known signs or symptoms. In the latter case, the true-positive and true-negative rates are much greater than in the broad screening situation, and so there will be fewer false positives and false negatives. Overlooking this difference results in overestimations of test accuracy in screening situations, resulting in a high percentage of false positives. We may assume there is no risk when there is a risk (the zero-risk illusion) or incorrectly assume that a risk can be calculated (the “turkey illusion”) (Gigerenzer, 2014b). We may fail to use a satisfying heuristic (simple solution); “ignoring information can lead to better, faster, and safer decisions” (Gigerenzer, 2014b, p. 31). (See discussion of intuitive and analytic thinking in Part 1.) As Gigerenzer (2014a) notes, “certainty is an illusion” (p. 31); “defensive decision making” in which there is a “fear of blame, criticism and litigation” (p. 65) encourages faulty estimates and bogus claims. Gigerenzer also notes that most physicians select treatments that differ from those they recommend to their patients (see also Garcia-Retemero & Galesic, 2012; Gambrill & Gibbs, 2002). Technologies promoted may create incorrect estimates of accuracy.

The Importance of Providing Absolute Risk Key concepts in understanding risk are illustrated in Box 27.1.

Absolute risk reduction: The absolute risk reduction is the proportion of patients who die without treatment (placebo) minus those who die with treatment. For example, a drug reduces the number of people who die from 41 to 32 in 1000. That is, the absolute risk reduction is 9 in 1000, which is 0.9%. Relative risk reduction: The relative risk reduction is the absolute risk reduction divided by the proportion of patients who die without treatment. For example, for the present data, the relative risk reduction is 9 divided by 41, which is 22%. Thus, the drug reduces the risk of dying by 22% Number needed to treat: The number of people who must participate in the treatment to save one life is the number needed to treat (NNT). This number can be easily derived from the absolute risk reduction. [See Box 27.1.] The number of people who needed to be treated to save one life is 111,

Gambrill & Gibbs Exercise 27 Estimating Risk and Making Predictions 295 296

Box 27.1 The 2 × 2 Table

Outcome Yes No Exposed a b Not exposed c d

aa/( + b) Relative risk (RR) = cc/ ()+ d cc/( +−da)/()ab+ Relative risk reduction (RRR) = cc/ ()+ d c a Absolute risk reduction (ARR)= − cd + ab + 1 Number needed to treat (NNT) = ARR ab/ ad Odds ratio (OR) == cd/ cb

Source: Adapted from Guyatt, G., Rennie, D., Meade, M. O., & Cook, D. J. (2008). Users’ guides to the medical literature: A manual for evidence-based clinical practice. (2nd ed.), p. 88. Chicago: American Medical Association.

because 9 in 1000 deaths (which is about 1 in 111) are prevented by the drug. (Gigerenzer, 2002a, p. 35)

Notice that relative risk reduction seems much more important than absolute risk reduction. Because of this, the former is misleading. For over a decade, experts in risk communication have been pointing out that statements of relative risks totally fail to provide “information” to patients because they have no context to know that, say a “50% increased risk” is measured in relation to. In view of this universal condemnation of the practice, it is shameful when health care agencies, pharmaceutical companies and the media persist in making public pronouncements about risks or benefits solely in this manner. It is well known that if patients only hear data expressed as relative risks, they take away deceptively exaggerated impressions of the differences. (Paling, 2006, p. 14) As Gigerenzer (2002a) notes, relative risk reduction suggests “higher benefits than really exist” (p. 35). Number needed to treat provides further information when making decisions (see Box 27.1). Consider the example “that of 111 people who swallow the tablets for 5 years, 1

296 Critically Appraising Research Gambrill & Gibbs 297

had a benefit, whereas the other 110 did not” (p. 36). Presenting risk reduction in relation to a common number (1 out of 1000) contributes to understanding. Paling (2006) urges professionals (and researchers) to provide absolute risk and to use easy-to-understand visual aids such as those he illustrates in his book. Say the absolute risk of developing a disease is 4 in 100 in nonsmokers. Say the relative risk of the disease is increased by 50% in smokers. The 50% relates to the “4”—so the absolute increase in the risk is 50% of 4, which is 2. So, the absolute risk of developing this disease in smokers is 6 in 100. (Paling, 2006, p. 16) Say that the records show that for a defined population of people, about 2 out of 100 are at risk of having a heart attack over the next year. Then imagine that a new study comes out reporting that if such patients take an aspirin daily, their risks of a heart attack will be lowered. Instead of 2 out of 100 suffering a heart attack, only 1 person out of 100 would be expected to do so. (Paling, 2006, p. 15)

Be on your guard for those who present only relative risk reduction. Inform clients about absolute risk (Welch, Schwartz, & Woloshin, 2011). Read Risk Savvy: How to Make Good Decisions by Gigerenzer (2014b).

Using Natural Frequencies to Understand and Communicate Risk It is much easier to calculate risk using natural frequencies than it is to use probabilities. Consider an example from Gigerenzer (2014a) regarding Down syndrome. Sensitivity of first trimester screening is 90%; the false-positive rate is about 5%. So what does a positive test mean for a woman younger than 40 years old? • “About 1% of babies have Down Syndrome. • If a baby has Down Syndrome, there is a 90% chance that the test will be positive. • If the baby is unaffected, there is still a 5% chance that the test will be positive.” (p. 171)

Gambrill & Gibbs Exercise 27 Estimating Risk and Making Predictions 297 298

So let’s say a test is given and the result is positive.

Natural Frequencies 1,000 pregnant women

10 999 Down Syndrome no Down Syndrome

9 1 50 940 positive negative positive negative

Out of every 1000 women age forty, we expect fifty-nine babies (nine plus fifty) to receive a positive test result, and only nine of these women have a baby with Down syndrome. The other fifty women are falsely alarmed. Only one of every six or seven women with a positive result actually have a baby with Down syndrome (p. 122). Most obstetricians got this wrong.

Instructions

Complete Practice Exercises 27.1 and 27.2. Practice Exercise 27.1 offers practice in using natural frequencies. Practice Exercise 27.2 offers an opportunity to critically appraise a prediction/risk instrument; this exercise can be completed in small groups.

298 Critically Appraising Research Gambrill & Gibbs 299

Practice Exercise 27.1 Translating Probabilities into Frequencies

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ Course __ _ _ ______ _ _ _ _ _ _ _ _ _ _

Instructions Please read the following examples and calculate risk using frequencies.

Situation 1 Sally, a medical social worker, is employed in a hospital. Her client, Mrs. Sabins, age 45, said her doctor recommends she get a mammogram to screen for breast cancer. She is asymptomatic. She asked about possible risks, but said that the doctor brushed aside her questions. She would like to know more about the accuracy of this test and asks for your help. Let’s say: “The following information is available about asymptomatic women aged 40 to 50 in such a region who participate in mammography screening”: The probability that one of these women has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer? (Gigerenzer, 2002a, p. 41)

Your answer? ___________________________ Translate probabilities into frequencies and illustrate them in a diagram.

Situation 2 Another patient approaches Sally (from Situation 1) regarding how to interpret risk—in this case, a symptom-free 50-year-old man. His

Gambrill & Gibbs Exercise 27 Estimating Risk and Making Predictions 299 300

physician recommended that he get a hemoccult test to detect occult blood in the stool. This test is used in routine screening for early detection of colon cancer. He wants more information about the accuracy of the test. Imagine that, based on information from screening symptom- free people older than 50 years of age, we have the following data: The probability that one of these people has colorectal cancer is 0.3 percent [base rate]. If a person has colorectal cancer, the probability is 50 percent that he will have a positive hemoccult test. If a person does not have colorectal cancer, the probability is 3 percent that he will still have a positive hemoccult test. Imagine a person (over age 50, no symptoms) who has a positive hemoccult test in your screening. What is the probability that this person actually has colorectal cancer? (Gigerenzer, 2002a, pp. 104–105)

Your answer? ____________________________________________________ Translate probabilities into frequencies and illustrate them in a diagram.

Situation 3

About 0.01 percent of men with no known risk behavior are infected with HIV (base rate). If such a man has the virus, there is a 99.9 percent chance that the test result will be positive (sensitivity). If a man is not infected, there is a 99.99 percent chance that the test will be negative (specificity). (Gigerenzer, 2002a, p. 124)

What is the chance that a man who tests positive actually has the virus? Your answer _________________________________________________ Translate probabilities into frequencies and illustrate them in a diagram.

300 Critically Appraising Research Gambrill & Gibbs 301

Practice Exercise 27.2 Critically Appraising a Risk Measure

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ___ _ _ _ _ _ ______ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Name of instrument __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _

Purpose __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _

Cite best related article. ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Complete the information in Box 27.2. Overall critique __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Clinical bottom line __ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 27 Estimating Risk and Making Predictions 301 302

Box 27.2 Evaluating a Risk Measure

Criterion Explanation Insert Value Reported in Source or Zero If Not Reported Sensitivity a/(a + c) Specificity d/(b + d) Positive predictive a/(a + b) value Negative predictive d/(d + c) value Prevalence rate (a + c)/(a + b + c + d) Blinded prediction Were those who judged the gold standard Yes __ _ No __ _ blind to the prediction scale’s score? Follow-up Were clients followed up long enough to Yes __ _ No __ _ evaluate predictive accuracy? Follow-up rate Greater than 80% Percent followed up (0–100%): _ _ _ % Reliability checked Were ratings of risk level checked by inde- Yes __ _ No __ _ by independent pendent raters and compared? raters Reliability Ideally, greater than .70 Reliability coefficient coefficient __ _ Representativeness Were subjects in the study sufficiently Yes __ _ No __ _ similar to your clients so that results apply to them? Validation study Was the measure tested in a setting other Yes __ _ No __ _ than the one in which it was developed and found to have predictive value? Benefit to client Are the benefits of using the measure Yes __ _ No __ _ and significant worth the harms and costs? others

Note: See the contingency table in Box 22.1.

*Use one form per source

302 Critically Appraising Research Gambrill & Gibbs 303â

EXERCISE 28â CRITICALLY APPRAISING DIAGNOSTIC TESTS

Purpose

To enhance your skill in critically appraising diagnostic measures, highlight the harms of using tests that do not measure what they claim to measure, and to distinguish between assessment and diagnosis.

Background

The term diagnosis refers to accurately identifying the cause(s) of symptoms. The term was borrowed from medicine, in which a physician makes a diagnosis and then recommends a treatment based on this diagnosis. Giving a diagnosis involves classification (Belmonte-âSerrano, 2015). Information may be gathered from a variety of sources (e.g., observation, self-âreports) to make a diagnosis. Depending on the diagnosis, an estimate of risk may be given, such as the risk of developing prostate cancer. Evidence-âinformed diagnosis can be defined as the use of certain symptoms and signs (feeling hot and having a fever as determined by a ther- mometer) to identify an underlying condition (a bacterial infection). Such a diagnosis is evidence informed in two senses: (1) signs and symptoms indicate the underlying condition accurately and (2) accurate identification of these conditions points to an effective remedy (e.g., antibiotics). In medicine, we often have both signs (an X-âray) and symptoms (complaints of a cough). Is this true of concerns such as depression and anxiety? The answer from psychiatry is yes. It is assumed that symptoms (for example, self-âreported anxiety in social situations) reflect an underlying disorder (or dysfunction). There are no independent agreed-âon objective indicators of alleged underlying pathology; we have symptoms without signs. From an evidence-âinformed point of view, diagnosis and assessment are not either/âor endeavors. The Mental Health Parity Act requires all health insurers to provide equivalent benefits for “mental disorders” described in the DSM as they do for physical illnesses. Providing a diagnosis for a client from the DSM-â5 is a requirement for third-âparty payment. Is this system reliable and valid (e.g., Kirk et al., 2013)? Labels are classifications

303 304

and thus can suffer from all the ways in which classifications are wrong, such as over- or underinclusion. Providing a diagnosis does not mean you cannot or should not carry out a contextual assessment describing personal and environmental factors related to client concerns (Gambrill, 2014a; Layng, 2009). Assessment encourages the description of processes rather than the study of conditions. This kind of assessment often reveals that environmental factors contribute to client concerns, such as an abusive partner, lack of daycare, and/or lack of employment opportunities. Diagnostic tests may misinform rather than inform. Errors in diagnosis are the leading causes of harm in medicine (Balogh, Miller, & Ball, 2015). They account for about 30% of malpractice claims in medicine (Nendaz & Perrier, 2012). We have no related information in most other professions. If a test used for symptomatic people is used to screen nonsymptomatic individuals (those without known risk factors), there will be a high rate of false positives (see Exercise 27). Excessive confidence may be placed in a diagnostic test resulting in harming rather than helping people. Consider the reflex dilation test. In Britain, Hobbs and Wynne (1986), two pediatricians suggested that a simple medical test could be used to demonstrate that buggery or other forms of anal penetration had occurred. Here is their description: Reflex dilation, well described in forensic texts … usually occurs within about 30 seconds of separating the buttocks. Recent controversy has helped our understanding of what is now seen as an important sign of traumatic penetration of the anus as occurs in abuse, but also following medical and surgical manipulation…. The diameter of the symmetrical relaxation of the anal sphincter is variable and should be estimated. This is a dramatic sign which once seen is easily recognized…. The sign is not always easily reproducible on second and third examinations and there appear to be factors, at present, which may modify the eliciting of this physical sign. The sign in most cases gradually disappears when abuse stops. (Hanks, Hobbs, & Wynne, 1988, p. 153)

News of this test spread quickly, and because of this test many children were removed from their homes on the grounds they were being sexually

304 Critically Appraising Research Gambrill & Gibbs 305

abused. Questions related to interpreting diagnostic test results reported in research studies suggested by Guyatt et al. (2008) include the following:

• Did participants include a representative sample of those presenting with a diagnostic dilemma? • Was the test compared with an appropriate, independent reference standard? • Were those interpreting the test and reference standard blind to the other results? • Did all clients receive the same reference standard regardless of the test results? • What likelihood ratios were associated with the range of possible test results? • Will the reproducibility of the test results and their interpretation be satisfactory in my clinical setting? • Are the study results applicable to my clients? • Will clients be better off as a result of the test? (p. 421)

Clients may be misclassified as a result of using a faulty test. Confirmation biases contribute to misleading use of tests. Nendaz and Perrier (2012) estimate that cognitive issues, either alone or in association with system failures, are involved in about 75% of diagnostic errors. Flaws occurred during data collection, data integration, and data verification.

Instructions

Please complete Practice Exercise 28.

Gambrill & Gibbs Exercise 28 Critically Appraising Diagnostic Tests 305 306 307

Practice Exercise 28 Evaluating Diagnostic Tests

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ Date __ _ _ _ _ _ _ _ _ ______ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _____ _ Instructor’s Name __ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _

1. Identify a diagnostic test to be reviewed and give the most up-to-date relevant citation.

2. Describe the purpose of this test. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _

3. Is this test widely used? __ _ Yes __ _ No

4. Review material on reliability and validity in Exercise 26 as well as concepts such as false- positive and false-negative rates described in Box 28.1. You could also consult STARD reporting guidelines regarding diagnostic measures and related questions noted concerning a diagnostic test study. (See also CASP [Critical Appraisal Skills Program]).

Describe the reliability of the test. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Describe the validity of this test. __ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

5. Would you use this test? __ _ Yes __ _ No Please explain your answer.

Gambrill & Gibbs Exercise 28 Critically Appraising Diagnostic Tests 307 308

6. Is the accuracy of this test described correctly on the Internet and other sites and forms (e.g., client brochures)?

7. Screening tests for nonsymptomatic individuals are often used. To get some expertise in critically appraising such tests, read “Measuring the Performance of Screening Tests” by Peter Sedgwick (2014).

Box 28.1 Definitions and Calculations for a Perfect (“Gold Standard”) Diagnostic Test

Test Disorder Present Disorder Absent Total Test Positive A B A + B Test Negative C D C + D Total A + C B + D N = (A + B + C + D)

Definitions Sensitivity: A/(A + C) Specificity: D/(D + B) False-negative rate: C/(C + A) False-positive rate: B/(B + D) Positive predictive value: A (A + B) Negative predictive value: D/(C + D) Pretest disease probability: (A + C)/(A + B + C + D) Posttest disease probability, positive result: A/(A + C) Posttest disease probability, negative result: C/(C + D)

Test Disorder Present Disorder Absent Total Test Positive 100 0 100 Test Negative 0 100 100 Total 100 100 200

Calculations Sensitivity: 100 (100 + 0) = 100% Specificity: 100 (100 + 0) = 100% Positive predictive value: 100% Posttest disease probability negative test: 0%

Source: Elmore, J. G., & Boyko, E. (2000). Assessing accuracy of diagnostic and screening tests. In J. P. Geyman, R. A. Deyo, & S. D. Ramsey (Eds.), Evidence-based clinical practice: Concepts and approaches (p. 85) edited by. Boston, MA: Butterworth Heinemann. Reprinted with permission. 309â

EXERCISE 29â EVALUATING RESEARCH REGARDING CAUSES

Purpose

To highlight questions to raise regarding research about causes.

Background

Both professionals and researchers make decisions about problem framing, as discussed in Part 1. For example, is depression the result of differences in the brain? Are they created by different life experiences such as the loss of significant others or environmental stress, including noise pollution? Is it “in the genes?” Is it caused by negative thoughts? Does psychotropic medication cure or cause abnormal brain states (Gøtzsche, 2015a; Moncrieff & Cohen, 2006)? Kinds of causes include the following:

• Sufficient: Y occurs whenever X occurs; therefore, X is sufficient to cause Y. X must precede Y if X is a cause of Y. • Insufficient: That cause that, by itself, is insufficient to produce the effect, but can function as a causal variable in combination with other variables • Necessary: Y never occurs without X. • Necessary and sufficient: Y occurs whenever X occurs, and Y never occurs without X. • First: That cause upon which all others depend—âthe earliest event in a causal chain • Principal: That cause upon which the effect primarily depends • Immediate: That cause that produces the effect without any intervening events • Mediating: A cause that produces its effect only through another cause (Byerly, 1973; Haynes, 1992, p. 26) Causes may be proximal or distal (Krieger, 2008). Causal factors differ in how long it takes for a cause to affect behavior (latency) and the time required to stabilize an effect (equilibrium). There may or may not be critical periods. Clues to causality include temporal order, contiguity

309 310

in time and space, covariation, and availability of alternative possibilities (Einhorn & Hogarth, 1986). Causal effects may depend on critical periods such as developmental stage. Kuhn (1992) divided the kind of evidence used to support theories about alleged causes of a problem into three kinds. One is genuine evidence. Criteria here are (1) it is distinguishable from the description of the causal inference itself and (2) it bears on its correctness. Kinds of covariation evidence include correspondence (evidence that does no more than note a co-occurrence of antecedent and outcome), covariation (there is a comparison or quantification), and correlated change (does Y change after X?). Kinds of indirect evidence include analogy, assumption, discounting (elimination of alternatives), and partial discounting. A second category includes pseudoevidence. Kuhn (1992) describes this as taking the form of a scenario or general script, depicting how the phenomena might occur, usually expressed in general terms. In contrast to genuine evidence, pseudoevidence cannot be clearly distinguished from description of the causal sequence itself. Generalized scripts or scripts as unfalsifiable illustrations may be offered. Evidence is equated with an example. Counterexamples are dismissed as exceptions. A request for evidence may be followed by a restating or elaboration of the original theory; there is no distinction between theory and evidence. The third category is no evidence (either genuine or pseudo). No evidence is offered in relation to the theory proposed. Included here are implications that evidence is unnecessary or irrelevant, assertions are not connected to a causal theory, or the phenomena is cited itself as evidence regarding its cause. Questions suggested by Greenhalgh (2010) regarding quality of research regarding causes include the following: • Is there evidence from true experiments in humans? • Is the association strong? • Is the association consistent from study to study? • Is the temporal relationship appropriate (i.e., did the assumed cause precede the assumed effect)? • Is there a dose–response gradient (i.e., does more of the assumed effect follow more of the assumed cause)? • Does the association make epidemiological sense? • Does the association make biological sense?

310 Critically Appraising Research Gambrill & Gibbs 311

• Is the association specific? • Is the association analogous to a previously found association? (p. 68) Common errors in identifying causes include confusing correlation and causation (cause and effect), the cluster illusion (assuming that random events are a meaningful pattern), measurement error, oversimplification (ignoring related factors), and the regression fallacy (ignoring natural fluctuations in variables). (See the discussion of post hocergo propter hoc in Part 3. See also mistakes in thinking about causation at www. ma.utexas.edu.) Summary statistics are used in RCTs; statistical evidence does not permit interpreting causation in a deterministic manner. Averages are given, such as, “On average, people who take the medication have a decrease in blood pressure.” This does not mean that you will experience such a decrease. Other errors include ignoring a common cause and the fallacy of the single cause (assuming there is a single cause when there are many). The garbage bag approach may be used—assuming a cause is “biopsychosocial” with little or no unpacking (Tesh, 1988). It may be assumed that B and A are both causes when B is an effect of A. It may be assumed that A causes B when B causes A or both A and B have a common cause. Or, A and B may have no relation. We may believe A is a cause of B because it feels good to believe this (e.g., in God). Disadvantages of incomplete or misleading causal models include inaccurate assessment, diagnosis, and predictions, and subsequent selection of ineffective or harmful interventions (Haynes, 1992). This brief overview should alert you to the challenges in identifying causes, especially via studies that explore correlations among variables.

Instructions

1. Read the article assigned by your instructor. 2. Complete Practice Exercise 29.

Gambrill & Gibbs Exercise 29 Evaluating Research Regarding Causes 311 312 313

Practice Exercise 29 Evaluating Research Regarding Causes

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _

Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ Course __ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _

1. Describe problem of concern.

2. Provide a complete reference for the article. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

3. Describe the major claims regarding causality. You may use a brief direct quote (include page numbers in your reference).

4. Describe the research method used (e.g., correlational design) as well as any problems regarding claims about causality in this study, drawing on prior discussion and sources described. For example, has correlation been confused with regression? Has the correlation coefficient been calculated and interpreted correctly? Is correlation assumed to reflect causality? Are other possible variables that may contribute to an effect ignored?

Gambrill & Gibbs Exercise 29 Evaluating Research Regarding Causes 313 314 315â

PART 6 Reviewing Decisions

This section includes exercises designed to help you to review decisions you make. Exercise 31 is designed to enhance your skill in critically examining arguments related to claims that affect client’s lives. Exercise 32 provides an opportunity to consider ethical issues that arise in everyday practice based on the vignettes in Exercises 11 through 13. Deciding what is most ethical often requires considering the implications of different options. Exercises 33 and 34 offer an opportunity to review the quality of assessment and intervention. Exercise 35 offers an opportunity to consider errors you may make so you are more likely to spot and avoid them on future occasions.

315 316 317â

EXERCISE 30â CRITICALLY APPRAISING ARGUMENTS

Purpose

To highlight the role of argument in decision making and problem solving, to enhance argument analysis and presentation skills, to increase understanding of what arguments can and cannot achieve, and to encourage a focus on learning not winning.

Background

Here, an argument is viewed as a group of statements, one or more of which (the premises) are offered in support of another (the conclusion). Skill in offering and analyzing arguments helps you avoid the influence of propaganda ploys, biases, and fallacies (see Parts 2 and 3). Differences of opinion about what to do to help clients provide valuable learning opportunities. Disagreements may concern meanings, values, facts, and/â or theories. Arguments consist of parts; they can be taken apart as well as put together. They may be strong (convincing) or weak (unconvincing), simple or complex. Assertions may involve statements of fact (“a belief for which there is enough evidence to justify a high degree of confidence” [Nickerson, 1986, p. 36]), assumptions, or hypotheses. The term assumption refers to “an assertion that we either believe to be true in spite of being unable to produce compelling evidence of its truth, or are willing to accept as true for purposes of debate or discussion” (Nickerson, 1986, p. 37). A hypothesis is an assertion that we do not know to be true but that we think is testable. Assumptions, hypotheses, or statements of fact may be used as premises in an argument or they may serve as conclusions. Research reports are a form of argument regarding a claim (conclusion). A key part of an argument is the claim, conclusion, or position that is put forward. Let’s say someone claims that addiction is a brain disease. Is this true? Does Bruce Alexander’s rat park experiment shed light on this (Alexander, Beyerstein, Hadoway, & Coambs, 1981)? Excessive wordiness may make claims and a conclusion difficult to identify. A second critical

317 318

feature of an argument consists of the reasons, or premises, offered to support a claim. “The credibility of a conclusion can be no greater than the least credible of the premises from which it is drawn, so a conclusion cannot be considered a statement of fact unless all of the premises are statements of fact” (Nickerson, 1986, p. 37). Premises can be divided into two parts: grounds and warrants. The grounds (data or evidence) must be relevant to the claim as well as sufficient to support the claim— and here is where “warrants” come in. The term warrant is used to refer to the reasons given for assuming premises are relevant to a conclusion (Toulmin, 2003). Consider the following: • Johnny has difficulty paying attention, is falling behind in his school work, and has tantrums. • Johnny has attention deficit hyperactivity disorder (ADHD).

Warrant: Related behaviors are evidence of a “neurodevelopmental disorder.” • He should take a prescribed medication (Concerta).

Warrant: Biological problems call for biological remedies. • Johnny’s symptoms decrease showing that ADHD is a neurodevelopmental disorder.

Warrant: The effects of the drug show that ADHD is a brain disorder. What do you think? Many propaganda strategies give an illusion of argument, as in begging the question (Walton, 1991). Editorials may make a claim but provide no argument, give no reasons for the position taken. As Weston (1992) notes, it is not a mistake to have strong views; the mistake is to have nothing else. “By ‘fallacious’ is meant not only an incorrect argument but also a type of argument that is inherently deceptive in that it tends to fool people into accepting a bad argument as a good one” (Walton, 1997, p. 28). Deceptive strategies are used to block critical appraisal. An argument may be unsound because (1) there is something wrong with its logical structure, (2) it contains false premises, and/ or (3) it is irrelevant or circular. The latter two kinds are informal fallacies; they have a correct logical form but are still incorrect. So informal fallacies are related to the content of arguments rather than

318 Reviewing Decisions Gambrill & Gibbs 319

to their form. Let’s say Mrs. Charles, a staff member at a skilled nursing facility, notes that Mrs. Slater (a resident) seems incoherent. Another staff member asks why this is occurring. Mrs. Charles tells the other staff member that Mrs. Slater has dementia and that is why she is incoherent. What other possibilities should be considered? (For practice in identifying rival hypotheses, see Huck and Sandler [1979].) In deductive arguments, if the reasoning is logically valid, the conclusion necessarily follows, although it may not be true if one or more of the premises are false. Deductive arguments can produce false conclusions when one of the premises is false or when one of the rules of deductive inference is violated, as in the logical fallacy of affirming the consequent. Consider this example from Wikipedia:

If I have the flu, I have a sore throat. I have a sore throat. Therefore, I have the flu.

Here there is a confusion of necessary and sufficient criteria. Having a sore throat may be related to having the flu, but it is not sufficient to make a diagnosis (see discussion of fallacy in Part 3).

Seldom are the major premises as well as the conclusions stated clearly in deductive arguments. Typically, one or more premise is missing. Questions in evaluating an argument include: Is it complete? Is its meaning clear? Is it valid (Does the conclusion follow from the premises)? Do I believe the premises? (Nickerson, 1986, p. 88).

An argument may be worthy of consideration even though it has some defects. The following steps are helpful in analyzing incomplete logical arguments:

• “Identify the conclusion and premises. • List all the other explicit assertions that make up the argument as given. • Add any unstated assertions necessary to make the argument complete. (Put them in parentheses to distinguish them from assertions that are explicit in the argument.) • Order the premises (or supporting assertions) and conclusion (or key assertion) to show the structure of the argument” (Nickerson, 1986, p. 87).

Gambrill & Gibbs Exercise 30 Critically Appraising Arguments 319 320

Visual depictions of the relationship among premises, warrants, and conclusions as in argument mapping can be helpful. Goals of argument visu- alization include automatic description of human reasoning from text (e.g., www.arg-tech.org) and descriptions of corroborating evidence in courts of law (Walton, 2016). General rules for composing arguments suggested by Weston (1992) include the following:

1. “Distinguish between premises and conclusion 2. Present ideas in a natural order 3. Start from accurate premises 4. Use clear language 5. Avoid fallacies including loaded language (see Exercises 11 through 13) 6. Use consistent terms 7. Stick to one meaning for each term (p. v)”.

Arguments involve different goals. Examples include inquiry (considering the accuracy of a hypothesis), information exchange, and deliberation (deciding on the best course of action) (Walton, 2013). Other goals of dialogue include persuasion, discovery, negotiation, and eristic (personal conflict). Miller (2005) notes that most books on critical thinking emphasize use of arguments to persuade others and argues that “persuasion reeks of authority, or the attitude of a person who wants to teach rather than to learn” (p. 62). He suggests that the goal of persuasion encourages dogmatism and teaching rather than learning, and that pursuit of discovery (adding or extending knowledge) is also wrong, as the aim of argument as is the aim of justification (to justify or prove propositions). Miller (2005) argues that question begging is involved in assuming that if the premises are accurate, the conclusion follows, because premises are never certain. (See the discussion of a justification approach to knowledge in Part 1.) Damer (2005) offers the following criteria for evaluating arguments:

1. Relevance: The premises should be relevant to the conclusion. A premise is relevant if it makes a difference to the truth or falsity of a conclusion. The relevance of a premise may also be affected by its relation to other premises. Additional premises may be needed to show this relation.

320 Reviewing Decisions Gambrill & Gibbs 321

2. Acceptability: A premise is acceptable if it is adequately defended or at least capable of being adequately defended on request or with further inquiry, or is a conclusion of another good argument. A premise is unacceptable if it reflects a claim that contradicts a reliable source, or other premises in the same argument; is not adequately defended; is self-contradictory, linguistically confusing, or otherwise unintelligible; is no different from (or is as questionable as) the conclusion that it is supposed to support; and is based on a (usually) unstated but questionable assumption or unacceptable premise. 3. Sufficient grounds: The premises must provide sufficient grounds for the truth of its conclusion. The premises may be relevant and acceptable, but not sufficient in number, kind, and weight. A premise may be based on a small or unrepresentative sample, or on a faulty causal analysis, or crucial evidence may be missing. 4. Rebuttal: A good argument should provide an effective rebuttal to the strongest arguments against the conclusion and the strongest arguments in support of alternative positions. Rebuttal is often neglected. (See other sources for further detail such as Walton [1995, 2008].) Arguments are more likely to be productive and enjoyable if participants avoid “conversational terrorist tactics” such as negative ad hominems (“This is over your head”), nit-picking (focusing on minor points), bom- basting (e.g., “How dare you question such an obvious point?”), and vague statements such as, “Studies have shown that … ” (Vandruff, Dean, & Marshall, 1995). Foul ways to win an argument suggested by Paul and Elder (2004) include • Accuse your opponent of doing what he accuses you of or worse • Call for perfection (the impossible) • Use double standards (e.g., for evidence) • Evade questions • Flatter your audience • Ignore the evidence • Ignore the main point • Make an opponent look ridiculous • Raise only objections • Shift the ground • Introduce distracting jokes • Focus on a vivid case example • Shift the burden of proof

Gambrill & Gibbs Exercise 30 Critically Appraising Arguments 321 322

• Use double-talk • Use red herrings • Use bogus statistics • Claim the point is “old hat” • Use faint praise

Instructions

1. Review the guidelines for evaluating arguments. 2. Complete Practice Exercise 30. 3. Exchange your analysis with another student for review.

322 Reviewing Decisions Gambrill & Gibbs 323

Practice Exercise 30 Argument Analysis Form

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _

Select a practice or policy claim and related argument. Attach a copy so others can review it. We recommend that you use a short one of just a few sentences. Longer statements quickly become complex. 1. What is the claim (conclusion)?

Premise 1:

Warrant(s):

Premise 2:

Warrant(s):

Gambrill & Gibbs Exercise 30 Critically Appraising Arguments 323 324

Premise 3:

Warrant(s):

2. Draw an argument map depicting this argument (consult Web-based material as needed).

3. Be prepared to describe and examine each premise and warrant using the following criteria and write your answers here, including your reasons for them. Is it relevant to the conclusion? Explain how.

Is it acceptable? (Would a reasonable person accept it?)

Does it provide sufficient grounds? If so, explain how.

324 Reviewing Decisions Gambrill & Gibbs 325

4. Describe the strongest counterargument as well as your rebuttal.

5. Identify biases and/or fallacies (see Part 3). Draw on sources such as Carl Sagan’s (2011) Baloney Detection Kit, Stephen Downes’ “Guide to Logical Fallacies” (Stephen. downes@ualberta,ca) 1995-2001, “Twenty-Five Ways to Suppress the Truth: The Rules of Disinformation” by H. M. Sweeney (2000 www.whale.to), Fallacy Files(2016) by Caroll (www.fallacyfiles.org) ed, as well as description of biases and fallacies in Exercises 11 through 13 and propaganda methods described in Part 2. Bias/Fallacy (name):__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ How it appears:__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Bias/Fallacy (name):__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ How it appears:__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

6. What was the most difficult part of completing your argument analysis?

7. Would you like to have the premises in your arguments regarding life-affecting decisions critiqued on a routine basis? __ _ Yes __ _ No If yes, set up an argument clinic among classmates and submit a current example for review.

Gambrill & Gibbs Exercise 30 Critically Appraising Arguments 325 326 327â

EXERCISE 31â CRITICAL THINKING AS A GUIDE TO MAKING ETHICAL DECISIONS

Purpose

To illustrate the value of critical thinking as a guide to making ethical decisions in professional contexts.

Background

Baron (1985) suggests that the very purpose of critical thinking is to arrive at moral or ethical decisions. Human rights perspectives emphasize par- ticipatory decision making, nondiscrimination, and accountability (e.g., Sanghera et al., 2015). Professional codes of ethics describe ethical obligations of professionals. Such obligations highlight the call for transparency and accountability (reasoned decisions). Unfortunately, honoring these obligations is more the exception, as described in Exercise 20. Ethical dilemmas (e.g., situations in which there are competing interests) require careful consideration from multiple points of view to be resolved in the best way (e.g., Pope & Vasquez, 2010; 2011).

Instructions

Review the Checklist of Ethical Concerns in Box 31.1 as well as “Steps in Ethical Decision Making” by Pope and Vasquez (2010). Select vignettes in Exercises 11 through 13 to review. In Practice Exercise 31, note the game and vignette number and the ethical issue you think arises in that vignette. For each ethical issue selected, describe how it pertains to the vignette.

327 328

Box 31.1 Checklist of Ethical Concerns

A. Keeping confidentiality

____ 1. Limits on confidentiality are described. ____ 2. Confidentiality is maintained unless there are concerns about harm to others.

B. Selecting objectives

____ 3. Objectives result in real-life gains for clients. ____ 4. Objectives are related to key concerns of clients.

C. Selecting practices and policies

____ 5. Assessment methods provide accurate, relevant information. ____ 6. Assessment, intervention, and evaluation methods are acceptable to clients and significant others. ____ 7. Intervention methods are those most likely to help clients attain outcomes they value. ____ 8. Practice methods are culturally informed as needed.

D. Involving clients as informed participants

_____ 9. The accuracy of assessment methods is clearly described to clients. _____10. Risks and benefits of recommended services are clearly described. _____11. Risks and benefits of alternative options are described. _____12. Clear descriptions of the cost, time, and effort involved in suggested methods are given in language intelligible to clients. _____13. Your competence to offer needed services is accurately described to clients. _____14. Appropriate arrangements are made to involve others in decisions when clients cannot give informed consent.

E. Being competent

_____15. Valid assessment methods are used with a high level of fidelity. _____16. Promising intervention methods are used with a high level of fidelity. _____17. Effective communication and supportive skills are offered.

F. Being accountable and transparent

______18. Arrangements are made for ongoing feedback about progress using valid progress indicators. Data concerning progress are reviewed with clients at each session. ______19. Unethical practices are identified and brought to other’s attention (including fraud and corruption). __ _ _ _ _ 20. Conflict of interests are avoided. ______21. Opportunities for advocacy for better service are identified and pursued.

328 Reviewing Decisions Gambrill & Gibbs 329

G. Encouraging a culture of thoughtfulness

______22. Errors and mistakes are identified, and plans made and implemented to decrease them. ______23. Positive feedback is provided to colleagues for raising questions, and critically evaluating claims and arguments. ______24. Coordinated efforts are made to improve agency procedures and policies.

H. Attending to human rights

______25. Basic human rights are attended to, including the right to work, healthcare, food, and shelter.*

*See Council on Europe. (2013). Living in dignity in the 21st century: Poverty and inequality in societies of human rights: The paradox of democracies: Methodological guide.

Gambrill & Gibbs Exercise 31 Critical Thinking as a Guide to Making Ethical Decisions 329 330 331

Practice Exercise 31 Vignettes Reviewed for Ethical Concerns

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ `_ _ _ _ _ _ _ ______ __

Reasoning-in- Practice Vignettes Game Number Ethical Issue

Gambrill & Gibbs Exercise 31 Critical Thinking as a Guide to Making Ethical Decisions 331 332

Follow-up Questions 1. Identify any vignette or ethical issue that you think is particularly important or that spurs a question you would like to discuss.

2. Why do you think ethical issues are often overlooked or ignored in everyday practice?

3. Download the “Code of Ethics of the Society of Professional Journalists” (2014). Should some items here be added to your professional code of ethics?

4. It has been argued that extreme poverty is a violation of human rights and that this causes other violations of human rights, such as child labor and human trafficking. Discuss with peers and be ready to discuss in class (e.g., see the Council on Europe’s [2013] Living in Dignity in the 21st Century).

332 Reviewing Decisions Gambrill & Gibbs 333â

EXERCISE 32â REVIEWING INTERVENTION PLANS

Purpose

To enhance critical appraisal of intervention plans.

Background

Professionals make decisions about what intervention methods may result in hoped-âfor outcomes. These decisions may or may not be informed by an accurate assessment in which problem-ârelated individual and environmental circumstances are identified (Gambrill, 2013a). In the common elements approach, intervention components most correlated with outcome are focused on. (Chorpita, Becker, & Daleiden, 2007). The checklist included in this exercise suggests points to check when deciding on plans. For example, Are negative effects likely? Are cultural differences considered? Are plans acceptable to clients and significant others? Does related research suggest that the plans selected will be effective?

Instructions

1. Choose a client with whom you are working, or your instructor may provide a case example. Complete the Checklist for Reviewing Intervention Plans in Practice Exercise 32 and respond to questions that follow.

333 334 335

Practice Exercise 32 Checklist for Reviewing Intervention Plans

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ ______ _ _ _ _ _ _ _

No. Item N L S I 1. Assessment data support the plan’s selection. 1 2 3 4 2. The plan addresses the most important problem-related 1 2 3 4 circumstances. 3. The plan offers the greatest likelihood of success as shown by 1 2 3 4 critical tests. 4. Empirically based principles suggest the plan will be effective 1 2 3 4 with this client. 5. The plan is feasible. 1 2 3 4 6. The plan and rationales for it are acceptable to participants. 1 2 3 4 7. The plan, including intermediate steps, is clearly described. 1 2 3 4 8. The least intrusive methods are used. 1 2 3 4 9. The plan builds on available client skills. 1 2 3 4 10. Significant others (those who interact with clients, such as fam- 1 2 3 4 ily members) are involved as appropriate. 11. The plan selected is the most efficient in cost, time, and effort. 1 2 3 4 12. Positive side effects are likely. 1 2 3 4 13. Negative side effects are unlikely. 1 2 3 4 14. Cues and reinforcers for desired behaviors are arranged. 1 2 3 4 15. Cues and reinforcers for undesired behaviors are removed. 1 2 3 4 16. Chosen settings maximize the likelihood of success. 1 2 3 4 17. Cultural differences are considered as necessary. 1 2 3 4 18. Multiple services are well integrated. 1 2 3 4 19. Participants are given a clear, written description of the plan. 1 2 3 4 20. Arrangements are made for generalization and maintenance of 1 2 3 4 valued outcomes. 21. The plan meets legal and ethical requirements. 1 2 3 4 22. The probability that the plan will be successful in achieving 1 2 3 4 desired outcomes is high (p > .80). 23. You are competent to carry out this intervention. 1 2 3 4

I.=, ideal; S=satisfactory; L, a little satisfactory; N, not at all satisfactory.

Gambrill & Gibbs Exercise 32 Reviewing Intervention Plans 335 336

Cite and describe the best research that suggests that your plan is most likely (compared with others) to result in hoped-for outcomes.

Follow-up Questions 1. Is there any way you could increase the likelihood of success given available resources? If no, is this because ______ I don’t know how to offer a plan more likely to succeed __ _ _ ______ _ I know how to offer more effective services but don’t have the time __ _ _ ______ I don’t have the resources needed to offer a more effective plan. (Please describe what you need.) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ The client does not like the plan most likely to be successful and is not willing to participate. __ _ _ _ _ Other (Please describes) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

2. Are there items you think should be added to the checklist? ___ Yes ____ No If yes, please describe them and explain why you selected them.

3. What items do you think are especially important from the client’s point of view?

336 Reviewing Decisions Gambrill & Gibbs 337â

PART 7 Improving Educational and Practice Environments

The exercises included in Part 7 are designed to help you to apply critical thinking in your work and educational environments. Exercise 33 contains a checklist for reviewing the extent to which a setting reflects a culture of thoughtfulness. Exercise 34 offers a measure for evaluating the teaching of critical thinking. Exercise 35 describes how you can set up a journal club and Exercise 36 offers guidelines for encouraging continued self-âdevelopment regarding the process of evidence-âinformed practice. Exercise 37 offers an opportunity to increase self-âawareness of personal obstacles to critical thinking. Formidable obstacles lie ahead for those who resolve to critically appraise judgments and decisions. Our students, who confront these obstacles for the first time in their work and professional practice, often report a mixture of amazement, discomfort, aloneness, and feeling out of step. The examples that follow may help you to prepare for reactions to raising questions. A master’s degree student in one of my classes at the University of California at Berkeley had her fieldwork placement in a hospital. During a team meeting, a psychiatrist used a vague diagnostic category. The student asked, “Could you please clarify how you are using this term?” He replied, “I always wondered what they taught you at Berkeley, and now I know that it is not much.”

337 338

Students in my research classes at Berkeley were asked to seek an answerable question regarding agency services from their fieldwork supervisor and to offer to search for related research regarding effectiveness. One student who worked at an agency that offered play therapy to all clients for all problems said to the student, seemingly quite annoyed, “I really am not interested in what the research says. I do play therapy because I enjoy it.” Polly Doud, who graduated from the University of Wisconsin-Eau Claire, described events during a hospital case conference involving social workers, nurses, and a physician. She identified the problem as “appeal to authority.” The nurses and social workers had carefully examined the evidence regarding a patient’s care and had arrived at a consensus. The doctor entered the room and, after a superficial examination of the patient’s situation, decided on a course of action. Polly said, “If the nurses and social workers, myself included, had spoken up about the things that we had brought up before he walked in the room, I think things would have been different.” Polly was concerned because accepting the doctor’s conclusion, without counterargument, may have jeopardized patient care. Sandra Willoughby, another University of Wisconsin-Eau Claire student described events during an in-service training for professionals conducted by a woman advocating “alternative therapies,” including “feeling/touch” and “art therapy” as treatments for women in a refuge house for battered women. Sandra entered the conference room “planning to question her methods.” The presenter never referred to data regarding effectiveness, nor to studies evaluating it; she advocated for her methods based on “her personal experience with suffering and long depression, having lived through pain so that she can identify with clients and, therefore, help them.” Sandra felt uncomfortable asking for evidence about the method’s effectiveness. In her words, “We had all gone around and introduced ourselves before the speaker began talking, and they were all therapists and professionals in the field, and I introduced myself as a ‘student,’ so I also felt, “Who am I to say anything?’ ” Sandra also felt uncomfortable asking about effectiveness because “I’m looking around the room at the other professionals and I’m noticing a lot of ‘nodding’ and nonverbals that say, ‘That’s great.’ ” Sandra also “sensed from her [the presenter] a lot of vulnerability, and she even almost teared up a couple of times.” When the presentation was

338 Improving Educational and Practice Environments Gambrill & Gibbs 339

over, Sandra’s colleagues did not ask a single question about effectiveness, but asked only “supportive” questions like, “How do we refer clients to you?” Sandra said,

How can I ask the questions that I want to ask, but in a safe way? Feeling very uncomfortable, I did end up asking her. She talked [in response to Sandra’s question about effectiveness] a lot about spiritual emergence as a phenomenon that people go through and how she helps them through this …. She kept using “spiritual emergence” over and over without defining it…. She just described why she does it [the treatment] as far as energy fields in the body.

Sandra concluded from this experience that asking whether a method works and how this is known “is not commonplace.” We think Sandra’s experience may be typical across the helping professions. She was one of the first students who attended a professional conference, often attended by hundreds, who asked “Is your method effective? How do you know?” Here is the lesson from all this: Expect to be out of step. Expect to feel uncomfortable as a critical thinker and “question raiser.” But, take heart in knowing that raising “hard” questions regarding the evidentiary status of practices and policies is integral to helping clients and to avoiding harming them or offering ineffective services. Raising such questions is vital to the process and philosophy of evidence-based practice (EBP).

Gambrill & Gibbs Improving Educational and Practice Environments 339 340 341â

EXERCISE 33â ENCOURAGING A CULTURE OF THOUGHTFULNESS

Purpose

To identify changes you and your colleagues can pursue to enhance a culture of thoughtfulness in which critical appraisal of claims and related decisions is the norm and in which all parties, including clients, are involved as informed participants.

Background

The environments in which we work differ in the extent to which they encourage critical appraisal of claims, decisions, and suggested changes. They differ in the degree to which critical dialogue about issues that affect clients’ lives is sought and supported, and evidence-âinformed services are implemented. The more thoughtful an agency’s culture, the more likely decisions are to be informed about related uncertainties. Capacity for organizational change was one of seven domains included in checklists regarding determinants of practice (Flottorp et al., 2013). Other factors included incentives and resources; social, political, and legal factors; client factors; professional interactions; individual helper factors; and guideline factors (see also Gagnon et al., 2014; Shea, Jacobs, Esserman, Bruce, & Weiner, 2014). The environment is one of three factors that influence the quality of shared decision making. These include “social norms, organizational routines, and institutional standards (e.g., cultural values, governmental policies, professional organizations’ rules and institutional structures, such as the IP-âSDM MODEL [Légaré, Stacey, and the IP Team, 2010]) (www.ohri.ca/âdecisionaid). Of the seven sources that influence clinical practice suggested by Vincent and Taylor-âAdams (2001), five concerned agency characteristics: (1) agency context (funding sources, legal and administrative regulations, economic and regulatory context), (2) organizational policies and management practices (e.g., financial resources and constraints, organizational structure, policy standards

341 342

and goals, safety culture, and priorities), (3) work environment (staffing levels and skill mix; workload and shift patterns; design, availability, and maintenance of equipment, administrative, and management support), (4) team characteristics (verbal and written communication, supervision and help seeking, and team structure), and (5) task requirements (task design and clarity of structure, and availability and use of protocols). (The list of statements on the scale in this exercise have not been subjected to any item analysis, nor have reliability or validity checks been done, so we know little of the instrument’s measurement properties.)

Instructions

1. Complete Practice Exercise 33 and compute your total score. 2. Complete the follow-up activities.

342 Improving Educational and Practice Environments Gambrill & Gibbs 343

Practice Exercise 33 Culture of Thoughtfulness Scale

Your Name__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Setting (e.g., agency) __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Please circle the numbers to the right that best describe your views.

No. Characteristics of Your Work Environment SD D A SA 1. Critical appraisal of claims is welcomed; it is safe to disagree. 1 2 3 4 2. Evidence against, as well as in support of, favored views is actively 1 2 3 4 sought. 3. Knowledge gaps are rarely sought. 1 2 3 4 4. Criteria used to select practices and policies are clearly described. 1 2 3 4 5. Fear of retribution for disagreeing with “higher ups” is common. 1 2 3 4 6. Clients are involved as informed participants (clearly appraised of the 1 2 3 4 risks and benefits of recommended services as well as alternatives). 7. Testimonials and case examples are often used to promote practices. 1 2 3 4 8. Controversies and disagreements are welcomed and are viewed as 1 2 3 4 learning opportunities. 9. Staff prepare and share relevant critically appraised topics. 1 2 3 4 10. There are a number of taboo topics. 1 2 3 4 11. The agency’s website clearly and accurately describes the evidentiary 1 2 3 4 status of interventions used and outcomes achieved. 12. Interventions used have been critically tested and found to do more 1 2 3 4 good than harm. 13. Staff have ready access to up-to-date, relevant databases. 1 2 3 4 14. Client progress is evaluated based on clear, relevant outcomes and is 1 2 3 4 regularly shared with clients. 15. There is no time to be thoughtful about decisions because of excessive 1 2 3 4 workloads. 16. There is ready access to supervisory review. 1 2 3 4

Gambrill & Gibbs Exercise 33 Encouraging a Culture of Thoughtfulness 343 344

No. Characteristics of Your Work Environment SD D A SA 17. Rigid communication channels block effective problem solving. 1 2 3 4 18. Staff receive timely corrective feedback regarding key decisions. 1 2 3 4 19. Learning opportunities (e.g., coaching) are available. 1 2 3 4 20. Desired behaviors are reinforced; there is a positive culture. 1 2 3 4 21. It is safe to reveal ignorance, to say “I don’t know.” 1 2 3 4 22. Staff members communicate well. 1 2 3 4 23. Process measures are used to assess the effectiveness of services (e.g., 1 2 3 4 number of sessions attended). 24. Staff members are encouraged to consider the evidentiary status of 1 2 3 4 practices and policies. 25. Uncertainties in making decisions are recognized. 1 2 3 4 26. Participants accept the burden of proof principle, our obligation to pro- 1 2 3 4 vide reasons for our views. 27. Administrators encourage open channels of communication. 1 2 3 4 28. Agency case records clearly describe the presenting concerns of clients, 1 2 3 4 outcomes sought, and results. 29. Alternative views on controversial issues are sought and considered. 1 2 3 4 30. Reliance on questionable criteria such as popularity and tradition is 1 2 3 4 common. 31. The workforce is diverse. 1 2 3 4 32. Disagreements focus on important points and are made without sar- 1 2 3 4 casm, put-downs, or signs of contempt (e.g., rolling the eyes). 33. Needed resources are usually available. 1 2 3 4 34. Staff members are encouraged to blow the whistle on ineffective and 1 2 3 4 harmful practices, and a clear policy guides them. 35. A system is in place to identify errors and use them as learning oppor- 1 2 3 4 tunities to plan how to decrease them. 36. Staff members work well together in teams. 1 2 3 4 37. Some staff members have conflicts of interest that may compete with 1 2 3 4 offering quality services. 38. Most services used are of unknown effectiveness. 1 2 3 4 39. Staff members have effective conflict management skills. 1 2 3 4 40. Information is readily shared. 1 2 3 4

SD, strongly disagree; D, disagree; A, agree; SA, strongly agree.

Scoring A: Add weights for items 1, 2, 4, 6, 8, 9, 11, 12, 13, 14, 16, 18, 19, 20, 21, 22, 24 through 29, 31, 32, 33, 34, 35, 36, 39, 40. 453

A. Reverse weights for the following items and add them: 3, 5, 7, 10, 15, 17, 23, 30, 37, 38. Subtotal B __ _ . Subtotal A __ _ − Subtotal B __ _ = Total _ _ _

1. If you agreed with item 12 please identify the intervention used and cite your best source. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

2. Which two items reflect your workplace’s greatest strengths? a.__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ b.__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

3. Which two items reflect your workplace’s greatest room for improvement? a.__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ b.__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

4. Review items on the Alberta Context Tool (Estabrooks, Squires, Cummings, Birdsell, & Norton 2009) and the Organizational Social Context measure (Glisson, Green, and Williams [2012]). Which items apply to your work setting?

5. Does your agency culture encourage burnout? Review factors associated with burnout using Internet sources.

Follow-up Activity

Select one characteristic of a culture of thoughtfulness you would like to increase. Describe a plan for increasing this, implement the plan, and describe the results.

Gambrill & Gibbs Exercise 33 Encouraging a Culture of Thoughtfulness 345 346 347â

EXERCISE 34â EVALUATING THE TEACHING OF CRITICAL THINKING SKILLS

Purpose

To assess the extent to which an instructor models critical-âthinking skills.

Background

Classrooms vary in the extent to which critical-âthinking values, knowledge, and skills are emphasized. The Teaching Evaluation Form in this exercise describes related characteristics. (Thanks to the late Professor Emeritus Michael Hakeem of the University of Wisconsin-âMadison for his contributions to this list.) The list of items has not been subjected to any item analysis, nor have reliability or validity checks been done, so we know little of the instrument’s measurement properties. For example, a question to be pursued is: Do students who rate their instructors high on teaching critical thinking acquire and use more related values, knowledge, and skills compared with students who rate their instructors low?

Instructions

1. Indicate your degree of agreement with each item in Practice Exercise 34 by circling the answer that most accurately describes your view Do not put your name on the form. 2. Determine your score using the instructions given and include this at the end of the form.

347 348 349

Practice Exercise 34 Evaluating the Teaching of Critical Thinking

Your Name __ _ _ _ _ _ _ _ _ ______ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _

Course__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Instructor’s Name__ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Please circle the numbers in the columns that best describe your views.

No. Instructor’s Teaching Style SD D A SA 1. Presents arguments for, as well as against, different positions on 1 2 3 4 controversial issues. 2. Describes controversies concerning topics discussed. 1 2 3 4 3. Encourages students to critically appraise claims. 1 2 3 4 4. Thanks students who bring in research studies that argue against 1 2 3 4 her or his views. 5. Emphasizes that pursuit of accuracy is more important than “win- 1 2 3 4 ning” an argument. 6. Describes the evidentiary status of claims.* 1 2 3 4 7. Teaches students how to find and critically appraise evidence for 1 2 3 4 themselves about topics discussed. 8. Encourages students to base conclusions on sound documenta- 1 2 3 4 tion, such as high-quality research studies. 9. Gives assignments that emphasize what to think rather than how. 1 2 3 4 10. Clearly describes major concepts introduced in class. 1 2 3 4 11. Accurately presents disliked perspectives 1 2 3 4 12. Encourages students to identify assumptions related to their views. 1 2 3 4 13. Helps students generalize important principles to other situations. 1 2 3 4 14. Gives examinations and assignments that require application of 1 2 3 4 course content. 15. Describes how conclusions were reached. 1 2 3 4 16. Encourages students to seek research that contradicts their pre- 1 2 3 4 ferred views.

Gambrill & Gibbs Exercise 34 Evaluating the Teaching of Critical Thinking Skills 349 350

No. Instructor’s Teaching Style SD D A SA 17. Encourages students to think for themselves. 1 2 3 4 18. Makes fun of those who disagree with his or her position. 1 2 3 4 19. Presents conclusions tentatively, noting they may be found to be 1 2 3 4 false or a better theory may be found to account for them. 20. Identifies assumptions related to conclusions. 1 2 3 4 21. Teaches students that all ways of knowing are equally valid. 1 2 3 4 22. Often says, “I could be wrong.” 1 2 3 4 23. Sells a particular point of view. 1 2 3 4 24. Highlights the importance of recognizing ignorance. 1 2 3 4 25. Encourages creative guessing when problem-solving. 1 2 3 4

SD, strongly disagree; D, disagree; A, agree; SA, strongly agree.

*This refers to whether a claim has been critically tested, and with what rigor and outcome.

Scoring: A. Add weights for items 1, 2, 3, 4, 5, 6, 7, 8, 10 through 17, 19, 20, 22, 24, 25. Subtotal A:__ _ _ _ __ B. Reverse weights for the following items and add: 9, 18, 21, 23. Subtotal B:__ _ _ _ _ _ _ _ _ _ _ _ _ A − B = Total Score:_ __ 1. Which item(s) seem most important to encourage critical thinking?

350 Improving Educational and Practice Environments Gambrill & Gibbs 351â

EXERCISE 35â FORMING A JOURNAL CLUB

Purpose

To describe how to set up a journal club to encourage continued learning and to work with others to locate research vital to decisions that affect clients’ lives.

Background

The purpose of a journal club may be (1) to acquire the best evidence to inform decisions about a client (need driven), (2) to learn about new research related to your work (evidence driven), or (3) to acquire evidence-â informed practice skills (skill driven) (Straus et al., 2011, p. 245). Activities include the following (based on Straus et al., 2011): 1. Identify learning needs. For example, identify a practice or policy question in which there is uncertainty about what to do. Pose a well-â structured question. 2. Share best related available literature located between meetings. The American College of Physicians Journal Club of McMaster University uses the McMaster Online Rating of Evidence system to locate well-â conducted research (http://âhiru.mcmaster.ca/âMORE). Distribute pho- tocopies of abstracts, original articles, or abstracts of Cochrane or Campbell reviews. Decide which item(s) everyone will read before the next session. 3. Critically appraise research found using appropriate criteria during the next session and apply findings to the decision that must be made (see www.testingtreatments.org; CASP (Critical Appraisal Skills Program); Greenhalgh, 2010; as well as exercises in Part 5 of this volume, Journal Club Presentation Resources, NorthEast Ohio Medical Resources, http://âlibraryguides.neomed.edu and Index of Cochrane Journal Club articles www.cochranejournalclub.com). Suggestions for setting up a journal club include the following:

351 352

1. Identify other interested parties. 2. Agree on the goals of the club—for example, to enhance EBP skills. 3. Describe group learning methods that contribute to success, including norms for creating a productive task environment. 4. Arrange tools needed “to learn, practice, and teach in evidence-based ways, including quick access to evidence resources” (Straus et al., 2005, p. 229). 5. Share examples of critically appraisal topics (see Exercise19). 6. Enhance skills in facilitating group discussions and teaching the process of EBP. Recommendations for making your presentation: a. Briefly describe the client and clinical question . b. Provide a brief overview describing what you found, how you found it, and why it is relevant. You may want to prepare a handout or PowerPoint presentation. c. Describe the quality of the report found, drawing on relevant quality indicators. Include description of biases (see Box 5.1). Were any conflicts of interest involved, such as funding sources? If so, were they acknowledged? Are controversial issues noted (e.g., about problem framing)? d. Describe how what you found will affect your work with the client (e.g., Schwartz, Dowell, Aperi, & Kalet, 2007), including potential risks, benefits, and costs. Can you calculate the number needed to treat, if relevant (Sawhney, 2006)?

Instructions

1. Complete Practice Exercise 35.

352 Improving Educational and Practice Environments Gambrill & Gibbs 353

Practice Exercise 35 Forming a Journal Club

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

INSTRUCTIONS First, review the instructions for setting up a journal club. Your instructor may model a journal club session “in action” using the fish bowl technique in which you watch a session. Select three other classmates or three other staff members employed by your agency and set up a journal club, drawing on the background information in this exercise.

1. Indicate the location of the journal club.

2. List participants’ names.

3. Describe the goal of the journal club. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

4. Describe the learning techniques that will be used.

a. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

b. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

c. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

d. __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gambrill & Gibbs Exercise 35 Forming a Journal Club 353 354

5. Describe tools needed and indicate whether you have access to them.

a. _______________________________________________________ Yes ____ No

b. _______________________________________________________ Yes ____ No

c. _______________________________________________________ Yes ____ No

6. List the responsibilities of each participant.

Name __ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Name ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Name __ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

7. Describe progress in achieving your goal. Were you successful? __ _ Yes __ _ No If yes, please describe how.

If no, please describe obstacles.

8. Attach related documentation, including your CAT, and a copy of the best research report you found.

354 Improving Educational and Practice Environments Gambrill & Gibbs 355

EXERCISE 36 ENCOURAGING CONTINUED SELF-DEVELOPMENT REGARDING THE PROCESS OF EVIDENCE-INFORMED PRACTICE AND POLICY

Purpose

To encourage continued learning throughout your career.

Background

One advantage of being a professional is continuing to learn throughout your career. Questions pertinent to continued learning in integrating practice and research are shown in Box 36.1.

Instructions

1. Complete Practice Exercise 36.

355 356

Box 36.1 Self-evaluation Questions Regarding the Process of Evidence- Informed Practice and Policy

A. Posing Well-Structured Questions

1. Am I improving in recognizing information needs and asking related questions? 2. Am I asking well-structured questions? 3. Can I get “unstuck” when asking questions? 4. Am I saving my questions for later answering? 5. Is my rate of posing well-structured questions increasing?

B. Finding the Best Research Related to Decisions

1. Am I searching at all? 2. Do I know the best sources of current research for making my decisions? 3. Do I have easy access to the best research for questions that arise? 4. Am I finding useful research from a widening array of sources? 5. Am I becoming more efficient in searching?

C. Critically Appraising Research for Its Validity and Usefulness

1. Am I critically appraising external research at all? 2. Are critical appraisal guidelines becoming easier for me to apply? 3. Am I becoming more accurate and efficient in applying measures such as number needed to treat(NNT)? 4. Am I creating any CATS critical appraised topics and sharing them with others?

D. Using Clinical Expertise to Integrate Information and Make a Decision

1. Am I integrating my critical appraisals in my work with clients at all? Could I do better? 2. Am I becoming more efficient in clearly and accurately sharing vital information with clients? 3. Am I involving clients as informed participants in decision making more often, including clearly describing benefits and costs of both recommended and alternative options ? 4. Am I getting better at recognizing and avoiding fallacies and biases and the influence of special interests? 5. Can I resolve disagreements drawing on effective argument and relationship skills? 6. Am I getting better in recognizing and avoiding errors in integrating diverse kinds of information? 7. Am I obtaining feedback after each meeting from each client regarding their perception of my empathy and the helpfulness of sessions (see Miller et al., 2006; Wampold & Imel, 2015a)? 8. Are my empathy and helpfulness ratings from clients improving? 9. Are my critical dialogue skills in team meetings improving?

356 Improving Educational and Practice Environments Gambrill & Gibbs 357

E. Evaluating Outcomes and Helping Others

1. Am I helping more clients attain hoped-for outcomes as shown by subjective and objective measures of progress? 2. Am I using fewer unjustifiable excuses for lack of success (see Exercise 37)? 3. Is it getting easier to say I was wrong? 4. Am I helping others to pose well-structured questions regarding their information needs? 5. Am I raising more questions regarding services clients receive and receiving more informative responses? 6. Am I teaching and modeling searching skills? 7. Am I teaching and modeling critical appraisal skills? 8. Am I teaching and modeling the integration of external research with my clinical expertise and my clients’ preferences? 9. Am I helping others to enhance their skills in offering empathic and disarming responses?

Source: Some parts are adapted from Straus, S. E., Glasziou, P., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (2011). Evidence-based medicine: How to practice and teach EBM (pp. 206–209). New York: Churchill Livingstone.

Gambrill & Gibbs Exercise 36 Encouraging Continued Self-Development 357 358 359

Practice Exercise 36 Encouraging Continued Self-development Regarding the Process of Evidence-Informed Practice

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

1. Select a self-development goal from Box 36.1 and describe it here.

2. Describe your baseline. (How often you now engage in this step.)

3. Describe a plan for achieving your goal here (e.g., Watson & Tharp, 2014).

4. Describe how you will evaluate your success.

Gambrill & Gibbs Exercise 36 Encouraging Continued Self-Development 359 360

5. Carry out your plan and describe exactly what you did.

6. Describe your results. Were they what you hoped for? If yes, why do you think you were successful? If no, describe why you think you were unsuccessful. What obstacles got in your way?

7. Critique your plan based on self-management literature (e.g., Watson & Tharp, 2014).

8. What did you learn from this process?

360 Improving Educational and Practice Environments Gambrill & Gibbs 361

EXERCISE 37 INCREASING SELF-AWARENESS OF PERSONAL OBSTACLES TO CRITICAL THINKING

Purpose

To examine personal obstacles that interfere with sound decision making and continued learning, and to take steps to overcome them.

Background

Some obstacles to problem solving are personal, such as arrogance, which interferes with recognition of ignorance (see Box 37.1). Others are environmental, such as a supervisor who discourages questions about the effectiveness of agency services; or a blame culture, in which administrators single out and castigate an individual, ignoring system-related factors such as impossible goals and onerous recording requirements (see Exercise 33). There may be limited knowledge about a concern. Personal obstacles include misleading views of knowledge and how it can be gained (see Part 1) and lack of skill in raising questions in diplomatic ways (see Exercise 21). You may be informed about a problem but deny its existence, perhaps encouraged by “socially organized denial” (Norgaard, 2011; Zerubavel, 2006). Other obstacles include unrealistic expectations (that you can help everyone when this is not possible) and a lack of self-management skills such as poor time management (Watson & Tharp, 2014). Some are motivational, such as not caring about clients. Unjustifiable excuses may be offered for lack of success (see Practice Exercise 37.2). Remedies include increasing your skills in recognizing and avoiding fallacies and biases, arranging more effective supports for sound reasoning, increasing your willingness to recognize ignorance (both personal and objective) in areas that affect clients’ lives, embracing errors as learning opportunities (Bosk, 1979), and increasing your courage, both to speak and to listen.

361 362

Box 37.1 Personal Barriers to Critical Thinking/Problem Solving

1. Motivational Blocks

___ Value winning over discovering approximations to the truth ___ Vested interest in an outcome __ _ Cynicism ___ Unrealistic expectations ___ Lack of curiosity; lack of zeal __ _ Arrogance __ _ Lack of courage

2. Emotional Blocks

__ _ Fatigue __ _ Anger ___ Anxiety (e.g., regarding failure, social disapproval) __ _ Low tolerance for ambiguity/uncertainty ___ Inability to “incubate,” impatient

3. Perceptual Blocks

___ Define problem too narrowly (e.g., overlooking environmental causes) __ _ Overlook alternative views __ _ Stereotype ___ See what you expect to see

4. Intellectual Blocks

___ Judge rather than generate ideas as a first step ___ Rely on questionable criteria to evaluate claims; fail to critically evaluate beliefs/ claims ___ Use faulty learning and problem-solving strategies (e.g., do not seek corrective feedback; fail to retrieve information) ___ Fail to get accurate information concerning decisions ___ Use a limited variety of problem-solving languages (e.g., words, illustrations, models) __ _ Rely on misleading cues

362 Improving Educational and Practice Environments Gambrill & Gibbs 363

5. Cultural Blocks

___ Value “John Wayne thinking” (strong pro/con positions with little reflection) ___ Fear that the competition of ideas would harm the social bonding functions of false beliefs ___ Status differences that block open discussion __ _ Hold implicit biases

6. Expressive Blocks

___ Inadequate skill in writing and speaking clearly ___ Anxiety in social situations __ _ Poor conflict management skills

7. Excuses Used

See Practice Exercise 37.2.

Source: Adapted from Adams, J. L. (1986). Conceptual blockbusting: A guide to better ideas (3rd ed.). Reading, MA: Addison- Wesley. See also Gambrill (2013a, p. 179).

Instructions

1. Review the list of barriers described in Box 37.1 and check those that apply to you. 2. Complete Practice Exercise 37.1. 3. Complete Practice Exercise 37.2.

Gambrill & Gibbs Exercise 37 Increasing Self-Awareness of Personal Obstacles to Critical Thinking 363 364 365

Practice Exercise 37.1 Increasing Self-awareness of Personal Obstacles to Critical Thinking

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

1. Describe a personal obstacle you would like to work on (see Box 37.1).

2. What kind of obstacle is this?

3. How does it affect your work with clients?

4. Describe a plan for decreasing this obstacle drawing on related literature (e.g., Watson & Tharp, 2014).

Gambrill & Gibbs Exercise 37 Increasing Self-Awareness of Personal Obstacles to Critical Thinking 365 366

Cite the best source found.

5. Describe how you will evaluate outcome.

6. Did you carry out your plan? ___ Yes ___ No If not, describe what you did.

7. Describe your results.

8. Do you think your personal learning style influences your success? _ _ Yes __ No

366 Improving Educational and Practice Environments Gambrill & Gibbs 367

9. Review Pashler, McDaniel, Rohrer, and Bjork (2012). What are the implications of their work?

Gambrill & Gibbs Exercise 37 Increasing Self-Awareness of Personal Obstacles to Critical Thinking 367 368 369

Practice Exercise 37.2 Excuses Used For Poor-Quality Service: Justifiable Or Not?

Your Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ Date __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Course __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ______ _ _ _ _ _ Instructor’s Name __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Consider excuses you have heard others use as well as excuses you have used for offering poor- quality services. Which ones do you think are justified? Here are some examples (e.g., McDowell, 2000; Pope & Vasquez, 2011):

1. My supervisor (administrator) told me to do it. 2. Other people do it. 3. That’s the way it’s been done in the past. 4. I didn’t have time; I was busy. 5. We care about our clients. 6. This is the standard practice. 7. I was under a lot of stress. 8. My client was difficult. 9. I did not know about the ethical guidelines. 10. Doing something is better than doing nothing. 11. No one will find out. 12. My consultant said it is OK. 13. I didn’t mean it. 14. No one complained about it. 15. I didn’t have the needed resources. 16. Everything is relative. 17. If it sounds good, it is good. 18. If most people believe it, it’s true. 19. Other schools do it. 20. We can’t measure outcomes. 21. My professional organization says it is ok. 22. No law was broken.

Gambrill & Gibbs Exercise 37 Increasing Self-Awareness of Personal Obstacles to Critical Thinking 369 370

1. Circle the numbers above referring to excuses you think are justified.

2. Select one that you think is unjustified and describe a related real-life situation. Item number:

Describe your reasons for your selection and discuss with other students.

3. Describe an excuse you used that you think is justified below including your reasons.

370 Improving Educational and Practice Environments Gambrill & Gibbs 371â

Glossary

GLOSSARY OF CONCEPTS RELEVANT TO REVIEWING TESTS

Absolute risk Difference in risk between the control group and the treated group. (See Practice Exercise 27.) Absolute risk reduction The absolute arithmetic difference in rates of bad outcomes between experimental and control participants in a trial, calculated as the experimental event rate (EER) and the control event rate (CER), and accompanied by a 95% CI (Bandolier Glossary, accessed 10/â20/â07). Critical discussion “Essentially a comparison of the merits and demerits of two or more theories … The merits discussed are, mainly the explanatory power of the theories … the way in which they are able to solve our problems of explaining things, the way in which the theories cohere with certain other highly valued theories, their power to shed new light on old problems and to suggest new problems. The chief demerit is inconsistency, including inconsistency with the results of experiments that a competing theory can explain” (Popper, 1994, pp. 160–â161). Cynicism A negative view of the world and what can be learned about it. Eclecticism The view that people should adopt whatever theories or methodologies are useful in inquiry, no matter their source, and without undue worry about their consistency Empiricism “The position that all knowledge (usually, but not always, excluding that which is logico-â mathematical) is in some way ‘based upon’ experience. Adherents of empiricism differ markedly over what the ‘based upon’ amounts to—â‘starts from’ and ‘warranted in terms of’ are, roughly, at the two ends of the spectrum

371 372

of opinion” (Phillips, 1987, p. 203). Uncritical empiricism takes for granted that our knowledge is justified by empirical facts (Notturno, 2000, p. xxi). False negative rate Percentage of persons incorrectly identified as not having a characteristic. False positive rate Percentage of individuals inaccurately identified as having a characteristic. Hermeneutics “The discipline of interpretation of textual or literary material, or of meaningful human action” (Phillips, 1987, p. 203). Knowledge Problematic and tentative guesses about what may be true (Popper, 1992, 1994). Likelihood ratio Measure of a test result’s ability to modify pretest probabilities. Likelihood ratios indicate how many times more likely a test result is in a client with a disorder compared with a person free of the disorder. A likelihood ratio of 1 indicates that a test is totally uninformative. “A likelihood ratio of greater than 1 indicates that the test is associated with the presence of the disease whereas a likelihood ratio less than 1 indicates that the test result is associated with the absence of disease. The further likelihood ratios are from 1 the stronger the evidence for the presence or absence of disease. Likelihood ratios above 10 and below 0.1 are considered to provide strong evidence to rule in or rule out diagnosis respectively in most circumstances” (Deeks & Altman, 2004, p. 168). Likelihood ratio of a positive test result (LR +) The ratio of the true positive rate to the false positive rate: sensitivity/(1−specificity). Likelihood of a negative test result (LR –) The ratio of the false negative to the true negative rate: (1−sensitivity)/specificity (adapted from Pewsner, et al., 2004). Logical positivism The main tenet is the verifiability principle of meaning: “Something is meaningful only if it is verifiable empirically (i.e., directly, or indi- rectly, via sense experience) or if it is a truth of logic or mathematics” (Phillips, 1987, p. 204). The reality of purely theoretical entities is denied.

372 Glossary 373

Nonjustificationist epistemology The view that knowledge is not certain. It is assumed that although some knowledge claims may be warranted, there is no warrant so firm that it is not open to question (see Karl Popper’s writings). Negative predictive value (NPV) The proportion of individuals with negative test results who do not have the target condition. This equals 1 minus the posttest probability, given a negative test result. Number Needed to treat (NNT) The number of clients who need to be treated to achieve one additional favorable outcome, calculated as 1/ARR and accompanied by 95% CI (confidence interval). Paradigm A theoretical framework that influences “the problems that are regarded as crucial, the ways these problems are conceptualized, the appropriate methods of inquiry, the relevant standards of judgment, etc.” (Phillips, 1987, p. 205). Phenomenology “The study, in depth, of how things appear in human experience” (Phillips, 1987, p. 205). Positive predictive value (PPV) The proportion of individuals with positive test results who have the target condition. This equals the posttest probability, given a positive test result. Post positivism The approach to science that replaced logical positivism decades ago (see for example Phillips, 1987, 1992). Post-test odds The odds that a patient has the disorder after being tested (pretest odds × LR [likelihood ratio]). Posttest probability The probability that an individual with a specific test result has the target condition (posttest odds/[1 + posttest odds]). Pretest odds The odds that an individual has the disorder before the test is carried out (pretest probability/[1−pretest probability]). Pretest probability (prevalence) The probability that an individual has the disorder before the test is carried out. Pseudoscience Material that makes science like claims but provides no evidence for these claims. Predictive accuracy The probability of a condition given a positive test result.

Glossary 373 374

Prevalence rate (base rate, prior probability) The frequency of a problem among a group of people. The best estimate of the probability of a problem before carrying out a test. Quackery Relative risk Commercialization of unproven, often worthless and sometimes dangerous products and procedures either by professionals or others (Jarvis, 1990; Young, 1992). The ratio of risk in the treated group (EER) to risk in the control group (CER). RR = ERR/ CER Relative risk reduction (RRR) The relative risk reduction is the difference between the EER and CER (EER−CER) divided by the CER, and usually expressed as a percentage. Relative risk reduction can lead to overestimation of treatment effect. (Bandolier Glossary, accessed 10/20/07.) Relativism The belief that a proposition can be true for individuals in one framework of belief but false for individuals in a different framework. Relativists “insist that judgments of truth are always relative to a particular framework or point of view” (Phillips, 1987, p. 206). Retrospective accuracy The probability of a positive test given that a person has a condition. Science A process designed to develop knowledge through critical discussion and testing of theories. Scientific objectivity This “consists solely in the critical approach” (Popper, 1994, p. 93). It is based on mutual rational criticism in which high standards of clarity and rational criticism are valued (Popper, 1994; p. 70). (See also Critical discussion, mentioned earlier.) Scientism This term is used “to indicate slavish adherence to the methods of science even in a context where they are inappropriate” and “to indicate a false or mistaken claim to be scientific” (Phillips, 1987, p. 206). Sensitivity Among those known to have a problem, the proportion whom a test or measure said had the problem. Skepticism The belief that all claims should be carefully examined for invalid arguments and errors of fact.

374 Glossary 375

Specificity Among those known not to have a problem, the proportion whom the test or measure has said did not have the problem. Theory Myths, expectations, guesses, conjectures about what may be true. A theory always remains hypothetical or conjectural. “It always remains guesswork. And there is no theory that is not beset with problems” (Popper, 1994, p. 157). Theory-ladenness (of perception) “The thesis that the process of perception is theory-laden, in that the observer’s background knowledge (including theories, factual information, hypotheses, and so forth) acts as a ‘lens’ helping to ‘shape’ the nature of what is observed” (Phillips, 1987, p. 206). True negative rate Percentage of individuals accurately identified as not having a characteristic. True positive rate Percentage of individuals accurately identified as having a characteristic. Truth “An assertion is true if it corresponds to, or agrees with, the facts” (Popper, 1994, p. 174). People can never be sure that their guesses are true. “Though we can never justify the claim to have reached truth, we can often give some very good reasons, or justification, why one theory should be judged as nearer to it than another” (Popper, 1994, p. 161).

Glossary 375 376 377â

References

ABRAMO, A., & LEHMAN, J. (2015). How N.Y.’s biggest for profit nursing home group flourishes despite record of patient harm. www.propublica.org (accessed November 20, 2015). ABRAMOVITZ, M., & ZELNICK, J. (2015). Privatization in the human services: Implications for direct practice. Clinical Social Work Journal, 43, 283–â293. ABRAMS, L. (2012). Study: People living in poverty are twice as likely to be depressed. The Atlantic. October 31. www.theatlantic.com (accessed 11/â2/â2011). ACS CHEMICAL NEUROSCIENCE (2015). 2014 Global Prescription Medication Statistics: Strong growth and CND well represented. Editorial. ACS. Chemical Neuroscience, 6, 5905–â5906. ALDERSON, P. (2004). Absence of evidence is not evidence of absence: We need to report uncertain results and do it clearly. BMJ, 328, 476–â477. ALEXANDER, B. K., & WONG, L. S. (2010). The myth of drug-âinduced addiction. www.brucealexander.com (accessed 11/â14/â16). ALEXANDER, B. K., BEYERSTEIN, B. L., HADAWAY, P. F., & COAMBS, R. B. (1981). Effect of early and later colony housing on oral ingestion of morphine in rats. Pharmacology, Biochemistry & Behavior, 15, 571–â576. ALI, A., SHERINGHAM, J., & SHERINGHAM, K. (2008). Managing the curve downwards: A cross-âsectional study of the use of management speak in public health. Public Health, 122, 1443–â1446. ALLEN, L. E. (1970). WFF’N Proof: a game of modern logic. WFF’N Proof Learning Games. AMBADY, N., & ROSENTHAL, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64, 431–â441. AMERICAN PSYCHIATRIC ASSOCIATION. (2013). Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association. ANGELL, M. (2005). The truth about the drug companies: How they deceive us and what to do about it. New York: Random House. ANGELL, M. (2009). Drug companies & doctors: A story of corruption. New York Review of Books, January 15. www.nybooks.com (accessed 7/â10/â2010).

377 378

ANTONUCCIO, D. O., BURNS, D. D., & DANTON, W. G. (2002). Antidepressants: A triumph of marketing over science. Prevention & Treatment, 5, 1–17. ARIELY, D. (2010). Predictably irrational: The hidden forces that shape our decisions (rev. ed.). New York: Harper. ARKES, H. R. (1981). Impediments to accurate clinical judgment and possible way to minimize their impact. Journal of Consulting and Clinical Psychology, 49, 323–330. ARMSTRONG, J. C. (1980). Unintelligible management research and academic prestige. Interfaces, 10, 80–86. ASCH, S. E. (1956). Studies of independent and conformity: Minority of one against a unanimous majority. Psychological Monographs, 70, 91–70. ASIMOV, I. (1989). The relativity of wrong. Skeptical Inquirer, 14, 35–44. ASPDEN, P., WOLCOTT, J. A., BOOTMAN, J. L., & CRONENWETT, L. R. (eds.). (2007). Preventing medication errors. Washington, DC: National Academic Press. BAER, D. M. (2003). Program evaluation: Arduous, impossible, or political? In H. E. Briggs & T. L. Rzepnicki (eds.), Using evidence in social work practice: Behavioral perspectives (pp. 310– 322). Chicago: Lyceum. BAKER, M. (2015). First results from psychology’s largest reproducibility test. Nature. April 30. BAKER, T. B., MCFALL, R. M., & SHOHAM, V. (2008). Current status and future prospects of clinical psychology: Toward a scientifically principled approach to mental and behavioral health care. Psychol Sci Public Interest, 9, 67–103. BALOGH, E. P., MILLER, B. T., & BALL, J. R. (eds.). (2015). Improving Diagnosis in Health Care. Washington, DC: National Academies Press. BARAK, G. (ed.). (2015). The Routledge international handbook of the crimes of the powerful. New York: Taylor & Francis. BARBER, B. (1961). Resistance by scientists to scientific discovery. Science, 134, 596–602. BARKER, K. (2015). A “three-quarter” home landlord in Brooklyn faces illegal eviction charges. New York Times, August 11, p. A18. BARLOW, D., NOCK, M. K., & HERSEN, M. (2008). Single case experimental designs: strategies for studying behavior change. Boston: Allyn & Bacon. BARNES, D., & WOOD, P. (2008–2010). Business Bollocks. www.bol- locksphere.co.uk. BARON, J. (1985). Rationality and intelligence. New York: Cambridge University Press.

378 References 379

BARON, J. (2000). Thinking and deciding (3rd ed.). Cambridge, UK: Cambridge University Press. BARON, R. F. (2005). So right it’s wrong: Groupthink and the ubiquitous nature of polarized group decision making. In M. T. Zanna (ed.), Advances in experimental social psychology, vol. 37 (pp. 219–253). San Diego: Elsevier. BARR, P. J., THOMPSON, R., WALSH, T., GRANDE, S., OZANNE, E., & ELWYN, G. (2014). The psychometric properties of CollaboRATE: A fast and frugal patient-reported measure of the shared decision-making process. J Med Internet Res, 16, e2. BARTON, K., & MCCULLY, A. (2007). Teaching controversial issues … where controversial issues really matter. Teaching History, 127, 13–19. BAUER, H. H. (2001). Fatal attractions: The troubles with science. New York: Paraview Press. BAUER, H. (2004a). Science in the 21st century: Knowledge monopo- lies and research cartels. Journal of Scientific Exploration, 18, 643–660. BAUER, H. H. (2004b). Science or pseudoscience: Magnetic healing, psychic phenomena, and the other heterodoxies. Champaign, IL: University of Illinois Press. BAUMEISTER, R. F., CAMPBELL, J. D., KRUEGER, J. L., & VOHS, K. D. (2003). Does high self-esteem cause better performance, interpersonal success, happiness, or healthier life-styles? Psychological Science in the Public Interest, 4, 1–44. BELL, T., & LINN, M. C. (2002). Beliefs about science: How does science instruction contribute? In B. K. Hoffer & P. R. Intrich (eds.), Personal epistemology: The psychology of beliefs about knowledge and learning (pp. 321–346). Mahwah, NJ: Erlbaum. BELMONTE-SERRANO, M. A. (2015). The myth of the distinction between classification and diagnostic criteria.Reumatol. Clin, 11, 188–189. [letter to the editor.] BERG. A. O. (2000). Dimensions of evidence. In J. P. Geyman, R. A. Deyo, & S. D. Ramsey (eds.), Evidence-based clinical practice: Concepts and approaches (pp. 21–28). Boston, MA: Butterworths-Heineman. BERNAYS, E. (1928). Propaganda. New York: Routledge. BEST, J. (2004). More dammed lies and statistics: How numbers confuse public issues. Berkeley: University of California Press. BLACK, N. (1994). Experimental and observational methods of evaluation. BMJ, 309, 540. BLENKNER, M., BLOOM, M., & NIELSEN, M. (1971). A research and demonstration project of protective services. Social Casework, 52 (8), 483–499.

References 379 380

BOSK, C. L. (1979). Forgive and remember: Managing medical failure. Chicago: University of Chicago Press. BOYLE, M. (2002). Schizophrenia: A scientific delusion? (2nd ed.). London: Routledge. BREEDING, J., & BAUGHMAN, F. (2001). The ethics of informed parental consent to psychiatric drugging of children. Ethical Human Sciences and Services, 3, 175–188. BRODY, H. (2007). Hooked: Ethics, the medical profession, and the pharmaceutical industry. New York: Rowman & Littlefield. BROOKFIELD, S. D. (1987). Developing critical thinkers. San Francisco, CA: Jossey-Bass. BROOKFIELD, S. (1995). Becoming a critically reflective teacher. San Francisco: Jossey-Bass. BROWN, A. (2013). Understanding pharmaceutical research manipulation in the context of accounting manipulation. J Law & Med Ethics, 41, 611–619. BROWN, S. R. (n.d.). Bingo cards. BROWN, G. W., & HARRIS, T. (1978). Social origins of depression: A study of psychiatric disorders in women. New York: Free Press. BROWNE, M. N., & KEELEY, S. M. (2006). Asking the right questions: A guide to critical thinking (5th ed.). Upper Saddle River, NJ: Pearson. BROWNLEE, S., HURLEY, V., & MOULTON, B. (2011a). Patient decision aids and shared decision making. New York: New American Foundation. BROWNLEE, S., WENNBERG, J. E., BARRY, M. J., FISHER, E. S., GOODMAN, D. C., & BYNUM, J. P. W. (2011b). Improving patient decision-making in health care: A 2011 Dartmouth Atlas Report Highlighting Minnesota. Hanover New Hampshire: The Dartmouth Institute for Health Policy & Clinical Practice. BULTÉ, I., & ONGHENA, P. (2013). The single-case data analysis package: Analysing single-case experiments with R software. Journal of Modern Applied Statistical Methods, 12, 450–478. BUNGE, M. (1984). What is pseudoscience? The Skeptical Inquirer, 9(1), 36–47. BURNHAM, J. C. (1987). How superstition won and science lost: Popularizing science and health in the United States. New Brunswick, NJ: Rutgers University Press. BURNS, D. (2008). Workshop. Concord California. BYERLY, H. C. (1973). A primer of logic. New York: Harper & Row. CALDERÓN-GARCIDUEÑAS, L., TORRES-JARDÓN, R., KULESZA, R. J., PARK, S.- B., & D’ANGIULLI, A. (2014). Air pollution and detrimental effects on children’s brain: The need for a multidisciplinary approach to the issue complexity and challenges.

380 References 381

Frontiers in Human Neuroscience, article 613. www.frontiersin. org (accessed 8/3/2015). CALDWELL, P. H. Y., NANKIVELL, G., & SURESHKUMAR, P. (2013). Simple behavioural interventions for nocturnal enuresis in children. Cochrane Database of Systematic Reviews. CALLAWAY, E. (2011). Report finds massive fraud at Dutch universities. Nature, 479 Nov.1(7371). CALLAWAY, E. (2015). Faked peer reviews prompt 64 retractions. Nature August. CAMILLERI, M., & PARKE, D. W. (2010). Perspective: Conflict of interest and professional organizations: Considerations and recommendations. Acad Med, 85, 85–91. CAMPBELL, D. T., & STANLEY, J. C. (1963). Experimental and quasi- experimental design for research. Chicago, IL: Rand McNally. CARAMENICO, A. (2012). Hospital pays $178M for false advertising, inexperienced doc. FierceHealthcare, February 5. www.fiercehealthcare.com (accessed 10/9/2014). CAREY, B. (2007). Bipolar illness soars as a diagnosis for the young. New York Times, September 4. CARROLL, R. T. (2016). Fallacy files.www.fallacyfiles.org . CAROLL, R. T. (2003). The skeptic’s dictionary: A collection of strange beliefs, amusing deceptions and dangerous delusions. Hoboken, NJ: Wiley. CARR, E. G., LADD, M.V., & SCHULTE, C. F. (2008). Validation of the contextual assessment inventory for problem behavior. Journal of Positive Behavior Interventions, 10, 91–104. CARUTH, D. L., & HANDLOGTEN, G. D. (2000). Mistakes to avoid in decision making. Innovative Leader, 9(7), 86. www.innova- tiveleader.com. CAASE, A., & DEATON, A. (2015). Rising morbidity and mortality in midlife among white non-Hispanic Americans in the 21st century. PNAS, 112, 15078–15083. CHALMERS, I. (2003). Trying to do more good than harm in policy and practice: The role of rigorous, transparent, up-to-date evaluations Annals of the American Academy of Political and Social Science, 589, 22–40. CHAMBLISS, W., MICHALOWSKI R., & KRAMER, R. (eds.). (2010). State crime in the global age. Portland, OR: Willan. CHAPMAN, G. B. (2005). The psychology of medical decision making. In D. J. Koehler & N. Harvey (eds.), The Blackwqell handbook of judgement and decsision making (pp. 585–603). Malden, MA: Blackwell Pub. CHAPMAN, G. B., & ELSTEIN, A. S. (2000). Cognitive processes and biases in medical decision making. In G. B. Chapman & S. A.

References 381 382

Sonnenberg (eds.), Decision making in health care: Theory, psychology, and applications (pp. 183–210). New York: Cambridge University Press. CHILDREN CAUGHT IN A RACIST SYSTEM. (2015). New York Times, August 18, p. A16. [Editorial]. CHORPITA, B. F., BECKER, K. D., & DALEIDEN, E. L. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 647–652. CIALDINI, R. B. (2009). Influence: Science and practice (4th ed.). Boston, MA: Allyn & Bacon. CLAYMAN, M. L., BOBERG, E. W., & MAKOUL, G. (2008). The use of patient and provider perspectives to develop a patient-oriented website for women diagnosed with breast cancer. Patient Educ Couns, 72, 429–435. CODE OF ETHICS (2014). Society of Professional Journalists. COHEN, D. (2012). ADHD in France and America. www.madinamer- ica.com (accessed June 14, 2015). COHEN, D., & JACOBS, D. (2000). A model consent form for psychiatric drug treatment. Journal of Humanistic Psychology, 40, 59–64. COHEN, D., LACASSE, J. R., DUAN, R., & Y SENGELMANN, I. (2013). CriticalThinkRX may reduce psychiatric prescribing to foster youth: Results from an intervention trial. Research on Social Work Practice, 23, 284–293. COLQUHOUN, D. (2014). An investigation of the false discovery rate and the misinterpretation of p-values. Royal Society of Open Sci, 1, 14016. COMPANARIO, J. M., & ACEDO, E. (2013). Rejecting highly cited papers: The views of scientists who encounter resistance to their discoveries from other scientists. Journal of the American Society for Information Science & Technology, 58, 734–743. CONRAD, P. (1979). Types of medial social control. Sociology of Health & Illness, 1, 1–11. CONRAD, P. (2007). The medicalization of society: On the transfor- mation of human conditions into treatable disorders. Baltimore, MD: Johns Hopkins University Press. CONRAD, P., MACKIE, T., & MEHROTRA, A. (2010). Estimating the costs of medicalization. Social Science & Medicine, 70, 1943–1947. COOMBS, J. E., & NIMMO, D. (1993). The new propaganda: The dictatorship of Palaver in contemporary politics. New York: Longman.

382 References 383

CORCORAN, P. (2010). Drug makers take over from defense contrac- tors, dominate fraud payout list. News & Notes, Pharmaceutic Industry and Drugs, October 27. www.fairwarning.org (accessed 2/23/2012). COSGROVE, L., BURSZTAJN, H. J., KRIMSKY, S., ANAYA, M., & WALKER, J. (2009). Conflicts of interest and disclosure in the American Psychiatric Association’s clinical practice guidelines. Psychotherapy and Psychosomatics, 78, 228–232. COSGROVE, L., & KRIMSKY, S. (2012). A comparison of DSM- IV and DSM-5 panel members’ financial associations with industry: A pernicious problem persists. PLoS Medicine, 9, e1001190. COSGROVE, L., & WHEELER, E. E. (2013). Drug firms, the codifica- tion of diagnostic categories, and bias in clinical guidelines. Journal of Law, Medicine & Ethics, 411, 644–653. COSTIN, B., & TONGE, S. (2015, Jan.16). The residential. COUNCIL ON EUROPE. (2013). Living in Dignity in the 21st Century: Poverty and Inequality in Societies and Human Rights: The paradox of Democracies: Methodological Guide. Strausberg: Council on Europe. COULTER, A. (2002). The autonomous patient: Ending paternalism in medical care. London: Nuffield Trust. COYNE, J. (2014). Neurobalm: The pseudo-neuroscience of couples therapy. PLoS Blogs. http://blogs.plos.org/mindthebrain (accessed 3/10/2015). COYNE, J. (2015b). BMC medicine gets caught up in Triple P Parenting promoters’ war on critics and null findings. http://blogs.plos. org/mindthebrain (accessed 4/10/ 2- 16). COYNE, J. (2015b). Busting foes of post-publication peer review of a psychotherapy study. PLoS Blogs. http://blogs.plos.org/ mindthebrain. COYNE, J. (2015a). Lucrative pseudoscience at the International Positive Psychology Association meeting. http://blogs.plos.org/ mindthebrain/2015/ 07/ 15 (accessed 4/5/ 2016). COYNE, J. (2015c). Sordid tale of a study of cognitive behavioral therapy for schizophrenia gone bad. http://blogs.plos.org/mindthebrain (accessed 7/1/2016) COYNE, J. (2016). Unintended consequences of universal mindfulness training for schoolchildren? PLoS Biology, Nov.16. COX, P. (1999). Glossary of mathematical mistakes. www.psn.net (accessed 10/7/2001). CROSKERRY, P. (2003). The importance of cognitive errors in diagnosis and strategies to minimize them. Academic Medicine, 78, 775–780.

References 383 384

CROSKERRY, P. (2005). Diagnostic failure: A cognitive and affective approach. In K. Henriksen, J. B. Battles, E. S. Marks, et al. (eds.), Advances in patient safety: From research to implementation (vol. 2). Rockville, MD: Agency for Healthcare Research and Quality. CROSKERRY, P. (2009). Context is everything or how could I have been that stupid? Healthcare Quarterly, 12, e171–e177. CROSKERRY, P., PETRIE, D. A., REILLY, J. B., & TAIT, G. (2014). Deciding about fast and slow decisions. Academic Medicine, 89, 197–200. CROSKERRY, P., SINGHAL, G., & MAMEDE, S. (2013). Cognitive debiasing 1: Origins of bias and theory of debiasing. BMJ Qual Saf, 22, ii58–ii64. CUTRONA, C. E., WALLACE, G., & WESNER, K. A. (2006). Neighborhood characteristics and depression: An examination of stress processes. Curr Dir Psychol Sci, 15, 188–192. DAMER, T. E. (2005). Attacking faulty reasoning: A practical guide to fallacy free argument (3rd ed.). Belmont, CA: Wadsworth. DAVIS, D. A., MAZMANIAN, P. E., FORDIS, M., VAN HARRISON, R., THORPE, K. E., & PERRIER, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. Journal of the American Medical Association, 296, 1094–1102. DAWES, R. M. (1988). Rational choice in an uncertain world. San Diego, CA: Harcourt Brace Jovanovich. DEATH OF GENERAL GEORGE WASHINGTON. (1799). The Monthly Magazine and American Review, 1 (6), 475–477. DEBER, R. B., KRAETSCHMER, N., UROWITZ, S., & SHARPE, N. (2007). Do people want to be autonomous patients? Preferred roles in treatment decision-making in several patient populations. Health Expectations, 10, 248–258. DEEKS, J. J., & ALTMAN, D. J. (2004). Diagnostic Tests For Likelihood Ratios. British Medical Journal, 329, 168–169. DEL FIOL, G., WORKMAN, T. E., & GORMAN, P. N. (2014). Clinical questions raised by clinicians at the point of care: A systematic review. Journal of the American Medical Association Internal Medicine, 174, 710–718. DENNIS, C.- L., & DOWSWELL, T. (2012). Psychosocial and psychological interventions for preventing postpartum depression. Cochrane Database of Systematic Reviews. DEPARTMENT OF JUSTICE, OFFICE OF PUBLIC AFFAIRS. (2013). Johnson & Johnson to pay more than $2.2 billion to resolve criminal and civil investigations. Justice News, November 4. DEWAN, S. (2015a). Company extorts poor on probation, lawsuit says. New York Times, October 2, p. A14.

384 References 385

DEWAN, S. (2015b). Fighting court by court to end judicial policies that fall heavily on poor. New York Times National, October 25, pp. 16, 22. DEWAN, S. (2015c). On probation, lives can run far off track. New York Times, August 3, pp. A1, A12. DEWEY, J. (1933). How we think. Boston: D. C. Health. DODES, L., & DODES, Z. (2014). The sober truth: Debunking the bad science beyond 12-step programs and the rehab industry. Boston, MA: Beacon Press. DOLINAR, B. (2015). Prison services company Securus makes record profits in the face of FCC regulation. truth-out.org (accessed 7/ 1/2016). DOSHI, P. (2015). No correction, no retraction, no apology, no comment: Paroxetine trial reanalysis raises questions about institutional responsibility. BMJ, 351, h4629. DOWNES, S. (1995–2001). Guide to logical fallacies. Stephen. [email protected]. DROZDEK, B., WILSON, J. P., & TURKOVIC, S. (2012). Assessment of PTSD in non-Western cultures: The need for new contextual and complex perspectives. In J. G. Beck & D. M. Sloan (eds.), The Oxford handbook of traumatic stress disorders (pp. 302–314). New York: Oxford University Press. DUCKWORTH, K. (2013). Numbers of Americans affected by mental illness. www.nami.org (accessed 11/9/2014). DUNNING, D., HEATH, C., & SULS, J. M. (2004). Flawed self- assessment: Implications for health, education, and the work place. Psychological Science in the Public Interest, 5, 69–106. DUSENBERY, M. (2015). New report shows how women of color bear the costs of mass incarceration. Truthout, September 19. www. truth-out.org (accessed October 20, 2015). DUTTON, D. B. (1988). Worse than the disease: Pitfalls of medical progress. New York: Cambridge University Press. DYER, O. (2015). Top Medicare billing doctor is arrested for fraud. BMJ, 350, h2091. ECKHOLM, E. (2015). Gay conversion therapy group to close. New Jersey: New York Times, Dec. 18, A16. EDMONDSON, A. C., BOHMER, R. M. J., & PISANO, G. R. (2001). Speeding up team learning. Harvard Business Review. EDWARDS, A., ELWYN, G., & THOMPSON, (eds.). (2016). Shared decision making in health care: Achieving evidence-based patient choice (3rd ed.). New York: Oxford University Press. EINHORN, H. J., & HOGARTH, R. M. (1986). Judging probable cause. Psychological Bulletin, 99 (1), 3–19.

References 385 386

ELDEIB, D., & JACKSON, D. (2014). Sexually traumatized youths report new abuse at Indian Oaks. Chicago Tribune, December 5. ELLUL, J. (1965). Propaganda: The formation of men’s attitudes. New York: Vintage. ELMORE, J. G., & BOYKO, E. J. (2000). Assessing accuracy of diagnostic and screening tests. In J. P. Geyman, R. A. Deyo, & S. D. Ramsey (eds.), Evidence-based clinical practice: Concepts and approaches (pp. 83–93). Boston, MA: Butterworth Heinemann. ELSTEIN, A. S., SCHULMAN, L. W., SPRAFKA, S. A., ET AL. (1978). Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press. ELWYN, G., BARR, P. J., GRANDE, S. W., THOMPSON, R., WALSH, T., & OZANNE, E. M. (2013). Developing CollaboRATE: A fast and frugal patient-reported measure of shared decision making in clinical encounters. Patient Educ Couns, 93 (1), 102–107. ELWYN, G., & EDWARDS, A. (2009). “It’s what we do around here”: Shared decision-making as a future organizational competence. In A. Edwards & G. Elwyn (eds.), Shared decision making in health care: Achieving evidence-based patient choice (2nd ed., pp. 401–406). New York: Oxford University Press. ELWYN, G., FROSCH, D., THOMSON, R., ET AL. (2012). Shared decision making: A model for clinical practice. J Gen Intern Med, 27 (10), 1361–1367. ELWYN, G., & O’CONNOR, A. (2009). The International Patient Decision Aids Standards (IPDAS) collaboration: The checklist, the instrument, and next steps. In A. Edwards & G. Elwyn (eds.), Share decision-making in health care: Achieving evidence- based patient choice (2nd ed., pp. 237–242). New York: Oxford University Press. ELY, J. W., OSHEROFF, J. A., GORMAN, P. N., ET AL. (2000). A taxonomy of generic clinical questions: Classification study. BMJ, 321, 429–432. ENGEL, S. M. (1994). With good reason: An introduction to informal fallacies (5th ed.). New York: St. Martin’s Press. ENNIS, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In B. Baron & R. J. Sternberg (eds.), Teaching thinking skills: Theory and practice (pp. 9–26). New York: W. H. Freeman. ENTWISTLE, V. A., SHELDON, T. A., SOWDEN, A. J., & WATT, I. A. (1998). Evidence-informed patient choice. International Journal of Technology Assessment in Health Care, 14, 212–215. ESTABROOKS, C. A., SQUIRES, J. E, CUMMINGS, G. G., BIRDSELL, J. M., & NORTON, P. G. (2009). Development and assessment of the Alberta Context Tool. BMC Health Service Research, 15 (9), 234.

386 References 387

EVANS, B., & GIROUX, H. A. (2015). Disposable futures: The seduction of violence in the age of spectacle. SanFrancisco: City Lights Books. EVANS, I., THORNTON, H., CHALMERS, I., & GLASZIOU. (2011). Testing treatments: Better research for better healthcare. London: Pinter @ Martin. FARRINGTON, D. P. (2003). Methodological quality standards for evaluation research. Annals of the American Academy of Political and Social Science, 587, 49–68. FAUST, D. (2007). Decisions research can increase the accuracy of clinical judgment and thereby improve patient care. In S. O. Lilienfeld & W. T. O’Donohue (eds.), The great ideas of clinical science: 17 Principles that every mental health professional should understand (pp. 49–76). New York: Routledge. FAVA, G. A. (2010). Unmasking special interest groups: The key to addressing conflicts of interest in medicine. Psychother Psychosom, 79, 203–207. FAWCETT, S., & RABINOWITZ, P. (2007) Section 5. Addressing social determinants of health and development. Chapter 17: Analyzing community problems and solutions. Community Tool Box (ctb.ku.edu). FELTOVICH, P. J., SPIRO, R. J., & COULSON, R. L. (1993). Learning, teaching, and testing for complex conceptual understanding. In N. Frederickson, R. Mislevy, & I. Bejar (eds.), Test theory for a new generation of tests (pp. 181–218). Hillsdale, NJ: Lawrence Erlbaum. FEYNMAN, R. (1974). Cargo cult science. Caltech communication address. Also in Surely you’re joking Mr. Feyman, 1–4. FEYNMAN,R. (1969). What is science? Physics Teacher, 7, 313–320. FINEBERG, S. (2016). The business of recovery. A movie. FLOTTORP, S. A., OXMAN, A. D., KRAUSE, J., ET AL. (2013). Checklist For Identifying Determinants Of Practice: A Systematic Review And Synthesis Of Frameworks And Taxonomies Of Factors That Prevent Or Enable Improvement In Healthcare Professional Practice. Implementation Science, 8, 35. FOR YOUR CONSIDERATION #21: TEACHING CONTROVERSIAL ISSUES. (2004). Center for Teaching and Learning, University of North Carolina. http://ctl.edu/fyc21. FORSYTH, S. R., ODIERNA, D. H., KRAUTH, D., & BERO, L. A. (2014). Conflicts of interest and critiques of the use of systematic reviews in policymaking: An analysis of opinion articles. Systematic Reviews, 3, 122. doi:1186/2046- 4053- 3- 122,

References 387 388

FOSTER CHILDREN: HHS guidance could help states improve oversight of psychotropic prescriptions. (2011). Washington, DC: US Government Accounting Office. FOUCAULT, M. (1977). Discipline and punish: The birth of the prison. New York: Pantheon. FRANK, J. D. (1961). Persuasion and healing: A comparative study of psychotherapy. Baltimore, MD: Johns Hopkins University Press. FRANKFURT, H. G. (1986). On bullshit. Raritan, 6, 81–100. FRANKFURT, H. G. (2005). On bullshit. Princeton, NJ: Princeton University Press. FREEDMAN, L. P., COCKBURN, I. M., & SIMCOE, T. S. (2015). The economics of reproducibility in preclinical research. PLoS Biology, 13 (6), e1002165. FREEMAN, B. (1993). Thinking logically (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall. FREIRE, P. (1973). Education for critical conscienciousness. New York: Seabury Press. FROSCH, D. L., KRUEGER, P. M., HORNIK, R. C., CRONBOLM, P. F., & BARG, F. K. (2007). Creating demand for prescription drugs: A content analysis of television direct-to-consumer advertising. Annals of Family Medicine, 5, 6–13. FURUKA, T. A., & LEUCHT, S. (2011). How to obtain NNT from Cohen’s d: Comparison of two methods. PLoS One, 6 (4). GAGNON, M. A. (2013). Corruption of pharmaceutical markets: Addressing the misalignment of financial incentives and public health. Journal of Law, Medicine & Ethics, 41, 571–580. GAGNON, M,- P., ATTIEH, R., GHANDOUR, E. K., ET AL. (2014). A systematic review of instruments to assess organizational readiness for knowledge translation in health care. PLoS One, 9 (12), e114338. GAMBRILL, E. (2006). Evidence based practice: Choices ahead. Research on Social Work Practice, 16, 338–357. GAMBRILL, E. (2010). Evidence-based practice and the ethics of dis- cretion. Journal of Social Work, 11, 26–48. GAMBRILL, E. (2012a). Propaganda in the helping professions. New York: Oxford University Press. GAMBRILL, E. (2012b). Critical thinking in clinical practice: Improving the quality of judgments and decisions about clients (3rd ed.). New York: Wiley. GAMBRILL, E. (2013a). Social work practice: A critical thinker’s guide (3rd ed.). New York: Oxford University Press. GAMBRILL, E. (2013b). Birds of a feather: Applied behavior analysis and quality of life. Research on Social Work Practice, 23, 121–140.

388 References 389

GAMBRILL, E. (2014a). The Diagnostic and Statistical Manual of Mental Disorders as a major form of dehumanization in the modern world. Research on Social Work Practice, 24, 13–36. GAMBRILL, E. (2014b). Social work education and avoidable ignorance. Journal of Social Work Education, 50, 391–413. GAMBRILL, E. (2015). Avoidable ignorance and the role of the Campbell and Cochrane Collaborations. Research on Social Work Practice, 25, 147–163. GAMBRILL, E., & GIBBS, L. (2002). Making practice decisions: Is what’s good for the goose good for the gander? Ethical Human Sciences & Services, 4 (1), 31–46. GAMBRILL, E., & REIMAN, A. (2011). A propaganda index for reviewing problem framing in articles and manuscripts: An exploratory study. PLoS One, 6, e19516. GARB, H. N. (2005). Clinical judgment and decision-making. Annual Review of Clinical Psychology, 1, 67–89. GARCIA-RETAMERO, R., & GALESIC, M. (2012). The risks we dread: A social circle account. PLoS One, 7 (4), e32837. GAUDET, J. (2013). It takes two to tango: Knowledge mobilization and ignorance mobilization in science research and innovation. Prometheus: Critical Studies in Innovation, 31, 169–187. GELLES, R. J. (1982). Applying research on family violence to clinical practice. Journal of Marriage and the Family, 44, 9–20. GELLNER, E. (1992). Postmodernism, reason, and religion. New York: Routledge. GEYMAN, J. P., DEYO, R. A., & RAMSEY, S. D. (2000). Evidence- based clinical practice: Concepts and approaches. Boston, MA: Butterworth-Heineman. GIBBS, L. (1989). Quality of study rating form: An instrument for synthesizing evaluation studies. Journal of Social Work Education, 25, 55–67. GIBBS, L. E. (1991). Scientific reasoning for social workers. New York: Macmillan (Allyn & Bacon). GIBBS, L. (2003). Evidence-based practice for the helping professions. Pacific Grove, CA: Thomson Learning, Brooks/Cole. GIBBS, L., & GAMBRILL, E. (2002). Evidence–based practice: Counterarguments to objections. Research on Social Work Practice, 12, 452–476. GIBBS, L., GAMBRILL, E., BLAKEMORE, J., ET AL. (1995). A measure of critical thinking for practice. Research on Social Work Practice, 5, 193–204. GIBBS, L., & STEHLE-WERNER, J. (1996). University of Wisconsin, School of Social Work, Eau-Claire.

References 389 390

GIGERENZER, G. (2002a). Calculated risks: How to know when numbers deceive you. New York: Simon & Schuster. GIGERENZER, G. (2002b). Reckoning with risk: Learning to live with uncertainty. New York: Penguin. GIGERENZER, G. (2008). Why heuristics work. Perspectives on Psychological Science, 3 (1), 20–29. GIGERENZER, G. (2014a). Breast cancer screening pamphlets mislead women. BMJ, 348, g2636. GIGERENZER, G. (2014b). Risk savvy: How to make good decisions. New York: Viking. GLASZIOU, P., CHALMERS, I., RAWLINS, M., & MCCULLOCH, P. (2007). When are randomized trials necessary: Picking sig- nal from noise. BMJ, 334, 349–351. GLASZIOU, P., DEL MAR, C., & SALISBURY, J. (2003). Evidence-based medicine workbook. London: BMJ Books. GLISSON, C., GREEN, P., & WILLIAMS, N. J. (2012). Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse & Neglect, 36, 621–632. GOLDACRE, B. (2011). Battling bad science. TED talk. GOLDSTEIN, M. M., & BOWERS, D. G. (2015). The patient as consumer: Empowerment or commodification? Journal of Law, Medicine & Ethics, 43, 162–165. GOMORY, T., WONG, S. E., COHEN, D., & LACASSE, J. R. (2011). Clinical social work and the biomedical industrial complex. Journal of Sociology & Social Welfare, 38, 135–165. GØTZSCHE, P. (2008). Rational diagnosis and treatment: Evidence- based clinical decision making (4th ed.). Hobokan, N.J.: Wiley. GØTZSCHE, P. (2012). Mammography screening: Truth, lies and controversy. London: Radcliffe. GØTZSCHE, P. (2013). Deadly medicines and organised crime: How Big Pharma has corrupted health care. London: Radcliffe. GØTZSCHE, P. (2015a). Deadly psychiatry and organized denial. Copenhagen: People’s Press. GØTZSCHE, P. C. (2015b). Psychiatry gone astray. http://davidhealy. org (accessed June 13, 2015). GRANT, B. (2012). FDA staffers sue agency: Current and former employees of the US Food and Drug Administration have filed suit, alleging that their employer spied on them for whistleblow- ing. The Nutshell, February 6. http://the-scientist.com (accessed 8/24/2014). GRAY, W. D. (1991). Thinking critically about New Age ideas. Belmont, CA: Wadsworth.

390 References 391

GRAY, J. A. M. (2001a). Evidence-based health care: How to make health policy and management decisions (2nd ed.). New York: Churchill-Livingstone. GRAY, J. A. M. (2001b). Evidence-based medicine for professionals. In A. Edwards & G. Elwyn (eds.), Evidence-based patient choice: Inevitable or impossible? (pp. 19–33). New York: Oxford University Press. GREENBERG, S. A. (2009). How citation distortions create unfounded authority: Analysis of a citation network. BMJ, 339, b2680. GREENHALGH, T. (2010). How to read a paper: The basics of evidence- based medicine (4th ed.). Hoboken, NJ: Wiley-Blackwell. GREGORY, R. (2008). Guide to translating scientific papers into plain English. www.science20.com. Accessed 11/10/17. GROSS, P. R., & LEVITT, N. (1994). Higher superstition: The academic left and its quarrels with science. Baltimore, MD: Johns Hopkins University Press. GROVE, W. M., & MEEHL, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechani- cal, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy & Law, 2, 293–323. GUYATT, G., RENNIE, D., MEADE, M. O., & COOK, D. J. (2008). Users’s guides to the medical literature: A manual for evidence- based clinical practice (2nd ed.). JAMA. New York: McGraw Hill. GUYATT, G., & RENNIE, D. (Eds.) (2002). Users’ guides to the medical literature (first ed). The JAMA Network. GUYATT, G., RENNIE, D., MEADE, M. O., & COOK, D. J. (Eds.) (2015). Users’ guides to the medical literature: Essentials of evidence-based clinical practice (3rd ed.). The JAMA Network. New York: McGraw Hill. GWANDE, A. (2009). The checklist manifesto: How to get things right. New York: Metropolitan Books. HADLER, N. M. (2008). Worried sick: A prescription for health in an overtreated America. Chapel Hill, NC: University of North Carolina Press. HALL, C. C., ARISS, L., & TODOROV, A. (2007). The illusion of knowledge: When more information reduces accuracy and increases confidence. Organizational Behavior and Human Decision Processes, 103, 272–290. HANKS, H., HOBBS, C., & WYNNE, J. (1988). Early signs and recognition of sexual abuse in the pre-school child. In K. Browne, C. Davies, & P. Stratton (eds.), Early prediction and prevention of child abuse. Chichester, UK: Wiley. HARLICK, J. (2015). The eating disorders “Residential Treatment Industrial Complex”: Harm or help? Part 1 of an investigative

References 391 392

series. A Disordered World. http://adisorderedworld.com/ 2015/ 106. Accessed 10/14/2016 HARRIS, G. (2009). Drug makers are advocacy group’s biggest donors. The New York Times, Health sec Oct 22,A23. HARRIS, G., & CAREY, B. (2008). Researchers fail to reveal full drug pay. New York Times, June 8, A1. HAYNES, S. N. (1992). Models of causality in psychopathology: Toward dynamic, synthetic and nonlinear models of behavior disorders. New York: Macmillan. HAYNES, R. B., DEVEREAUX, P. J., & GUYATT, G. H. (2002). Clinical expertise in the era of evidence-based medicine and patient choice. ACP Journal Club, 136, A11, pp. 1–7. [Editorial]. RESOURCES FOR EVIDENCE-BASED PRACTICE: THE 6S PYRAMID. MCMASTER UNIVERSITY HEALTH SCIENCES. HTTP://HSL. MCMASTER.LIBGUIDES.COM (accessed 11/25/16). HELLMAN, H. (1998). Great Feuds in science: Ten of the livliest dis- putes ever. New York: John Wiley. HOBBS, C. J., & WYNNE, J. M. (1986). Buggery in childhood: A common syndrome of child abuse. Lancet, 4, 792–796. HOCHMAN, M., HOCHMAN, S., BOR, D., & MCCORMICK, D. (2008). News media coverage of medication research: Reporting pharmaceutical company funding and use of generic medication names. Journal of the American Medical Association, 300 (13), 1544–1550. HOFFMAN, D. H., CARTER, D. J., LOPEZ, C. R. V., ET AL. (2015). Report to the Special Committee of the Board of Directors of the American Psychological Association: Independent review relating to APA Ethics Guidelines National security interroga- tions and torture. July 2: Sidley Austin LLP. HOFFRAGE, U., & POHL, R. F. (2003). Research on hindsight bias: A rich past, a productive present, and a challenging future. Memory, 11, 329–345. HOFSTADTER, R. (1963). Anti-intellectualism in American life. New York: Vintage. HOGARTH, R. (1980). Judgement and Choice: The psychology of Decision. New York: John Wiley & Sons. HOGARTH, R. M. (2001). Educating intuition. Chicago: University of Chicago Press. HOLLAND, J. (2015). Medicating women’s feelings. New York Times, February 28. HORTON, R. (2015). Offline: What is medicine’s 5 sigma?Lancet , 385, 1380. HOWICK, J., FRIEDEMANN, C., TSAKOK, M., ET AL. (2013). Are treatments more effective than placebos? A systematic review and meta-analysis. PLoS One, 8, e62599.

392 References 393

HUCK, S. W., & SANDLER, H. M. (1979). Rival hypotheses: Alternative interpretations of data based conclusions. New York: Harper & Row. ILLICH, I., ZOLA, I. K., MCNIGHT, J., CAPLAN, J. A., & SHAIKEN, H. (1977). Disabling professions. Salem, NH: Marion Boyers. IMPROVING PATIENT DECISION-MAKING IN HEALTH CARE: A 2011 DARTMOUTH ATLAS REPORT HIGHLIGHTING MINNESOTA. LEBANON, NEW HAMPSHIRE: DARTMOUTH ATLAS PROJECT, THE DARTMOUTH INSTITUTE FOR HEALTH POLICY & CLINICAL PRACTICE. INSEL, T. (2013). Director’s blog: Transforming diagnostics. www. nimh-nih.gov (accessed November 10, 2015). INSTITUTE OF PROPAGANDA ANALYSIS (1937). New York: New York. IOANNIDES, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. IOANNIDES, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology, 19, 640–648. IOANNIDIS, J. P. A. (2012). Why science is not necessarily self- correcting. Perspectives on Psychological Science, 7, 645–654. IOANNIDIS, J. P. A. (2014). How to make more published research true. PLoS Medicine, 11, e1001747. IOANNIDIS, J. P. A. (2016). The mass production of redundant, misleading and conflicted systematic reviews and meta-analyses. Millbank Quarterly, Sept 13, online. IOANNIDIS, J. P. A., GREENLAND, S., HLATKY, M. A., ET AL. (2014). Increasing value and reducing waste in research design, conduct, and analysis. Lancet, 383, 166–175. INSTITUTE OF MEDICINE (IOM). (2011). Clinical Practice Guidelines we can Trust. Washington, DC: The National Academies Press. IYER, N., & SAMOCIUK, M. (2006). Fraud and corruption: Prevention and detection. Burlington, VT: Gower. JACKSON, D., & ELDEIB, D. (2014). Youths drawn into prostitu- tion while living at residential facilities. Chicago Tribune, December 7. JACKSON, D., & MARX, G. (2014). At Lawrence Hall, vulnerable kids terrorized, learn life of crime. Chicago Tribune, December 4. JACKSON, D., MARX, G., & ELDEIB, D. (2014a). DCFS inspects trou- bled youth living centers in response to Tribune series. Chicago Tribune, December 9. JACKSON, D., MARX, G., & ELDEIB, D. (2014b). DCFS stops placements, steps up monitoring at living center for youths. Chicago Tribune, December 15. JACOBSON, J. W., FOXX, R. M., & MULICK, J. A. (eds.). (2005). Controversial therapies for developmental disabilities: Fad,

References 393 394

fashion, and science in professional practice. Mahwah, NJ: Erlbaum. JADAD, A. R., & ENKIN, M. W. (2007). Randomized controlled trials: Questions, answers and musings (2nd ed.). Malden, MA: Blackwell Publishing. JAIN, A., NUNDY, S., & ABBASI, K. (2014). Corruption: Medicine’s dirty open secret. BMJ 348, g4184. JAMES, J. (2013). A new, evidence-based estimate of patient harms associated with hospital care. Journal of Patient Safety, 9, 122–128. JANIS, I. L. (1971). Groupthink. Psychology Today, 5, 43–46, 74– 76. JANIS, I. L. (1982). Groupthink: Psychological studies of policy decisions and fiascoes (2nd ed.). Boston, MA: Houghton Mifflin. JANIS, I. L., & MANN, L. (1977). Decision making: A psychological analysis of conflict, choice and commitment. New York: Free Press. JANKO, R. (1996). Literature, criticism and authority: The experience of antiquity. Inaugural Lecture, October, University College, London. JARVIS, W. T. (1990). Dubious dentistry. Loma Linda, CA: Loma Linda University School of Medicine, Department of Public Health and Preventive Medicine. JENICEK, M. (2011). Medical error and harm: Understanding prevention and control. New York: CRC Press. JENICEK, M., & HITCHCOCK, D. L. (2005). Evidence based practice: Logic and critical thinking in medicine. Chicago: American Medical Association Press. JENSEN, J. M., & FRASER, M. W. (eds.). (2016). Social policy for children and families: A risk and resilience perspective (3rd ed.). Thousand Oaks, CA: Sage. JOHNSON, H. M. (2006). Alas for Tiny Tim, he became a Christmas cliché. New York Times, Op. Ed., 12, 25. JOHNSON, W., CLANCY, T., & BASTIAN, P. (2015). Child abuse/ neglect risk assessment under field practice conditions: Tests of external and temporal validity and comparison with heart disease prediction. Children and Youth Services Review, 56, 76–85. KAHANE, H., & CAVENDER, N. (1998). Logic and contemporary rhetoric: The use of reason in everyday life (8th ed.). Belmont, CA: Wadsworth. KAHNEMAN, D. (2011). Thinking fast and slow. New York: Farrar, Straus & Giroux. KAHNEMAN, D., SLOVIC, P., & TVERSKY, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.

394 References 395

KALINOWSKI, P. (2010). Understanding confidence intervals (CIs) and effect size estimation. Observer, 23 (4). www.psychologi- calscience.org (accessed December 15, 2015). KAPTCHUK, T. J., & MILLER, F. G. (2015). Placebo effects in medicine. New England Journal of Medicine, 373, 1. KASSIRER, J. P. (2005). On the take: How medicine’s complicity with big business can endanger your health. New York: Oxford University Press. KASSIRER, J. P., & KOPELMAN, R. I. (1991). Learning clinical reasoning. Baltimore, MD: Williams & Wilkins. KAUER, S. D., REID, S. C., HEW, A., ET AL. (2012). Self-monitoring using mobile phones in the early stages of adolescent depression: Randomized controlled trial. J Med Internet Res, 14 (3), e67. KAUFMAN, J. C., & STERNBERG, R. J. (eds.). (2006). The Cambridge handbook of creativity. New York: Cambridge University Press. KEEN, A. J. A., KLEIN, S., & ALEXANDER, D. A. (2003). Assessing the communication skills of doctors in training: Reliability and sources of error. Advances in Health Sciences Education, 8, 5–16. KEMPNER, J. (2008). The chilling effect: How do researchers react to controversy? PLoS Medicine, 6 (1), e1000014. KERWIN, A., & WITTE, M. (1983). Map of ignorance (Q-Cubed Programs): What is ignorance? http://www.ignorance.medicine. arizona.edu/ignorance.html (accessed October 6, 2011). KIEL, P. (2015). The color of debt: How collection suits squeeze black neighborhoods. www.propublica.org (accessed October 14, 2015). KIM, M. M., BARNATO, A. E., ANGUS, D. C., FLEISHER, L. A., & KAHN, J. M. (2010). The effect of multidisciplinary care teams on intensive care unit mortality. Archives of Internal Medicine, 170, 369–376. KIPLING, R. (1923). Speech in A Book of Words. February 14. www. kiplingsociety.co.uk KIRK, S., GOMORY, T., & COHEN, D. (2013). Mad science: Psychiatric coercion, diagnosis, and drugs. New Brunswick, NJ: Transaction. KLEIN, G. (1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press. KLEIN, G. (2008). Naturalistic decision making, human factors. Journal of the Human Factors and Ergonomics Society, 50, 456–460. KOEHLER, D. J., & HARVEY, N. (eds.). (2005). The Blackwell handbook of judgment and decision making. Malden, MA: Blackwell Publishing. KOZLOWSKI, S. W. J., & ILGEN, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 17, 77–124.

References 395 396

KRIEGER, N. (2008). Proximal, distal, and the politics of causation: What’s level got to do with it? American Journal of Public Health, 98, 221–230. KUHN, D. (1992). Thinking as argument. Harvard Educational Review, 62, 155–178. KUHN, D. (1993). Connecting scientific and informal reasoning. Merrill-Palmer Quarterly, 39, 74–103. KUHN, D. (1999). A developmental model of critical thinking. Educational Researcher, 28, 16–25/ 46. KUHN, T. S. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. LACASSE, J. R., & LEO, J. (2010). Ghostwriting at elite academic medical centers in the United States. PLoS Medicine, 7(2), 1–4. LAFRANIERE, S., & LEHREN, A. W. (2015). The disproportionate risk of driving while black. New York Times National, October 25, pp. 1, 18. LANGMUIR, I. (1989). Pathological science. Physics Today, 42, 36–48. LANPHEAR, B. (2011). Crime of the century: The failure to prevent the lead poisoning pandemic. The Nora and Ted Sterling Prize in Support of Controversy. Simon Fraser University. www.sfu. ca/sterlingprize/rec (accessed 4/20/2013). LASKA, K. M., & WAMPOLD, B. E. (2014). Ten things to remember about common factor theory: Author response to commentary. Psychotherapy, 51, 519–524. LATOUR, B. (1987). Science in action: How to follow scientists and engi- neers through society. Cambridge, MA: Harvard University Press. LAYNG, J. (2009). In search for an effective clinical behavior analysis: The nonlinear thinking of Israel Goldiamond. Behavior Analysis, 32, 163–184. LEAHY, R. L. (2011). Cognitive–behavior therapy: Proven effectiveness. November 23. www.psychologytoday.com/blog (accessed 6/25/2013). LÉGARÉ, F., STACEY, D., & THE IP TEAM. (2010). IP-SDM Model. www.ohrs.ca/decisionaid (accessed 1/4/2012). LELYVELD, N. (2015). City beat: Senior actors tell scams. Los Angeles Times, April 8. www.latimes.com (accessed 8/4/2016). LENZER, J. (2013). Why we can’t trust clinical guidelines. BMJ, 346, f3830. LEVY, C. J., & LUO, M. (2005). New York Medic4555 vaid fraud may reach into billions. The New York Times, July 18. LEWANDOWSKY, S., ECKER, U. K. H., SEIFERT, C. M., SCHWARZ, N., & COOK, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13, 106–131.

396 References 397

LEWANDOWSKY, S., LITTLE, D. R., & KALISH, M. (2007). Knowledge and expertise. In F. T. Durso, R. Nickerson, S. Dumais, S. Lewandowsky, & T. Perfect (eds.), Handbook of applied cognition (2nd ed.). Chichester: Wiley. LEWIS, B. (2013). Management speak translation guide. Stokely Consulting. Collected and originally published in Keep the Joint Running. www.stokeley.com LEWIS, L. (2015). Oxfam: Wealth of one percent will exceed that of the rest of humanity in 2016. www.dailykos.com (accessed 2/ 12/16). LEXCHIN, J., BERO, L. A., DJULBEGOVIC, B., & CLARK, O. (2003). Pharmaceutical industry sponsorship and research outcome and quality: A systematic review. BMJ, 326, 1167–1170. LIGHT, D. W., LEXCHIN, J., & DARROW, J. J. (2013). Institutional corruption of pharmaceuticals and the myth of safe and effective drugs. Journal of Law, Medicine and Ethics, 14, 590–610. LILIENFELD, S. O., & ARKOWITZ, H. (2011). Can positive thinking be negative? Scientific American, April 14. www.scientificameri- can.com (accessed 11, 2015). LILIENFELD, S. O., LYNN, S. J., & LOHR, J. M. (2015). Science and pseudoscience in clinical psychology (2nd ed.). New York: Guilford. LITTELL, J. H. (2006). The case for multisystemic therapy: Evidence or orthodoxy? Children and Youth Services Review, 28, 458–472. LITTELL, J. H. (2008). Evidence-based or biased? The quality of published reviews of evidence-based practices. Children and Youth Services Review, 30, 1299–1317. LITTELL, J. H., CORCORAN, J., & PILLAI, V. (2008). Systematic reviews and meta-analysis. New York: Oxford University Press. LITTELL, J. H., POPA, A. M., & FORSYTHE, B. (2005). Multisystemic therapy for social, emotional, and behavioral problems in youth aged 10–17. Cochrane Database of Systematic Reviews, Issue 3 (3). LLOYD, K., CELLA, M., TANENBLATT, M., & CODEN, A. (2009). Analysis of clinical uncertainties by health professionals and patients: an example from mental health. BMC Medical Informatics and Decision Making, 9, 34. doi:10.1186/ 1472-6947-9-34 LOESKE, D. R. (1999). Thinking about social problems: An introduction to constructionist perspectives. New York: Aldine de Gruyter. LOKE, T. W., KOH, F. C., & WARD, J. E. (2002). Pharmaceutical advertisement claims in Australian medical publications: Is evidence accessible, compelling and communicated comprehensively? Medical Journal of Australia, 177, 291–293.

References 397 398

LOMBARDI, K., BUFORD, T., & GREENE, R. (2015). Environmental racism persists, and the EPA is one reason why. The Center for Public Integrity. www.publicintegrity.org (accessed 9/12/2016). LUDWIG, M. (2015). Private prison firms buy access to public officials at lavish industry conferences. Truthout, August 20. www. truthout.org. MACDONALD, I. (2013). Hospital medical errors now the third leading cause of death in the U.S. Fierce Healthcare, September 20. www.fiercehealthcare.com (accessed October 2, 2015). MACDONALD, I. (2015). Communication lapses between clinicians lead to poor care coordination, readmissions. FierceHealthcare (http://www.fiercehealthcare.com) (accessed 8/22/2016). MACLEAN, E. (1981). Between the lines: How to detect bias and propaganda in the news and everyday life. Montreal: Black Rose Books. MAKOUL, G., & CLAYMAN, M. L. (2006). An integrative model of shared decision making in medical encounters. Patient Educ Couns, 60, 301–12. MALLE, B. F. (2006). The actor–observer asymmetry in attribution: A (surprising) meta-analysis. Psychological Bulletin, 132 (6), 895–919. MALLETT, C. A. (2015). The school-to- prison pipeline. New York: Springer. MARCHEL, C. A. (2007). Learning to talk/talking to learn: Teaching critical dialogue. Teaching Educational Psychology, 2, 1–15. MARKER, C. D., & AYLWARD, A. G. (2012). Generalized anxiety disorder. Cambridge, MA: Hogrefe. MARLETT, C. A. (2015). The school-to- prison pipeline: A comprehensive assessment. New York: Springer. MARMOT, M. (2015). The health gap: The challenge of an unequal world. New York: Bloombury. MARSHALL, B. J., & WARREN, J. R. (1984). Unidentified curved bacilli in the stomach of patents with gastritis and peptic ulcer- ation. Lancet, 1 (8390), 1311–1315. MATTHIES, D. (1996). Precision questioning. Stanford University Center for Teaching and Learning. Stanford University: Mindworks. MCATEE, M., CARR, E. G., SCHULTE, C., & DUNLAP, G. (2004). A contextual assessment inventory for problem behavior. Journal of Positive Behavior Interventions, 6, 148–165. MC CORD, J. (2003). Cures that harm: Unanticipated outcomes of crime prevention programs. Annals of the American Academy of Political and Social Science, 587, 16–30. MC DOWELL, B. (2000). Ethics and excuses: The crisis in professional responsibility. Westport, CT: Praeger.

398 References 399

MCGAURAN, N., WIESELER, B., KREIS, J., SCHÜLER, Y-. B., KÖLSCH, H., & KAISER, T. (2010). Reporting bias in medical research: A narrative review. Trials, 11, 37, 1–15. MCGOEY, L. (2012). The logic of strategic ignorance. British Journal of Sociology, 63, 533–576. MCMASTER ONLINE RATING AND EVIDENCE (MORE). Canada. McMaster University. MCPHEARSON, K., GON, G., & SCOTT, M. (2013). International variations in a selected number of surgical procedures. Directorate for Employment, Labor and Social Affairs Health Committee. OECD Health working paper no. 61. Paris, France. MEDAWAR, P. B. (1967). The art of the soluble. London: Methuen. MEEHL, P. E. (1973). Why I do not attend case conferences. In P. E. Meehl (ed.), Psychodiagnosis: Selected papers (pp. 225–323). Minneapolis, MN: University of Minnesota Press. MERTENS, D. M., & HESSE-BIBER, S. (eds.). (2013). Mixed methods and credibility of evidence in evaluation: New directions in evaluation. Hobokan, NJ: Wiley. MEYER, G., STECKELBERG, A., & MÜHLHAUSER, I. (2007). Analysis of consumer information brochures on osteoporosis prevention and treatment. Ger Med Sci. http://www.egms.de (accessed 3/4/ 2009). MICHAEL, M., BOYCE, W. T., & WILCOX, A. J. (1984). Biomedical bestiary: An epidemiologic guide to flaws and fallacies in the medical literature. Boston, MA: Little, Brown. MILLER, D. (1994). Critical rationalism: A restatement and defense. Chicago, IL: Open Court. MILLER, D. (2005). Do we reason when we think we reason, or do we think? Learning for Democracy, 1, 57–71. MILLER, S. D., DUNCAN, B. L., BROWN, J., SORRELL, R., & CHALK, M. B. (2006). Using formal client feedback to improve retention and outcome: Making ongoing, real-time assessment feasible. Journal of Brief Therapy, 5, 5–22. MILLS, C. W. (1959). The sociological imagination. New York: Grove Press. MISER, W. F. (2000). Applying a meta-analysis to daily clinical practice. In J. P. Geyman, R. A. Deyo, & S. D. Ramsey (eds.), Evidence- based clinical practice: Concepts and approaches (pp. 57–64). Boston, MA: Butterworth & Heinemann. MONCRIEFF, J. (2008a). Neoliberalism and biopsychiatry: A marriage of convenience. In C. I. Cohen & S. Timimi (eds.), Liberatory psychiatry: Philosophy, politics and mental health (pp. 235–256). New York: Cambridge University Press. MONCRIEFF, J. (2008b). The myth of the chemical cure: A critique of psychiatry drug treatment. New York: Palgrave Macmillan.

References 399 400

MONCRIEFF, J., & COHN, D. (2006). Do antidepressants cure or create abnormal brain states? PLoS Medicine, 3(7), e240. MONTE, C. F. (1975). Psychology’s scientific endeavor. New York: Praeger. MOORE, B. N., & PARKER, R. (1986). Critical thinking: Evaluating claims and arguments in everyday life. Palo Alto, CA: Mayfield. MOYNIHAN, R., & CASSELS, A. (2005). Selling sickness: How the world’s biggest pharmaceutical companies are turning us into patients. New York: Nation. MUJCIC, R., & FRIJTERS. (2013). Still not allowed on the bus: It matters if you’re black or white. Institute for the Study of Labor discussion paper no. 7300. Bonn: IZA. MURAD, M. H., MONTORI, V. M., IOANNIDIS, J. P., ET AL. (2014). How to read a systematic review and meta-analysis and apply the results to patient care. Journal of the American Medical Association, 312, 171–179. MYERS, D. G. (2002). Social psychology. New York: McGraw-Hill. MYSYUK, Y., WESTENDORP, J., BIGGS, S., & LINDENBERG, J. (2015). Listening to the voices of abused older people: Should we classify system abuse? BMJ, 350, 1–3. NAFTULIN, D. H., WARE, J. E., & DONNELLY, F. A. (1973). The Doctor Fox lecture: A paradigm of educational seduction. Journal of Medical Education, 48, 630–635. NAPHY, W. G., & ROBERTS, P. (eds.). (1997). Fear in early modern society. Manchester, UK: Manchester University Press. NATIONAL ALLIANCE FOR MENTAL ILLNESS. MENTAL ILL- NESS FACTS AND NUMBERS. WWW.NAMI.ORG (accessed October 2015). NATIONAL SCIENCE FOUNDATION. (2006). Surveys of public understanding of science and technology: 1979–2006. www.nsf. govhttp://www.ropercenter.uconn.edu (accessed August 22, 2011). NEMETH, C. J., & GONCALO, J. A. (2005). Inference and persuasion in small groups. In T. C. Brock & M. C. Green (eds.), Persuasion: Psychological insights and perspectives (2nd ed., pp. 171–194). Thousand Oaks, CA: Sage. NENDAZ, M., & PERRIER, A. (2012). Diagnostic errors and flaws in clinical reasoning: Mechanisms and prevention in practice. Swiss Med Weekly, 142, w13706. NICKERSON, R. S. (1986). Reflections on reasoning. Hillsdale, NJ: Erlbaum. NICKERSON, R.S. (1988–1989). On improving thinking through instruction. In E. Z. Rothkopf (ed.), Review of research in education (pp. 3–57). Washington, DC: American Educational Research Association.

400 References 401

NICKERSON, R. S. (1998). Confirmation bias: A ubiquitous phenomena in many guises. Review of General Psychology, 2, 175–220. NISBETT, R., & ROSS, S, L. (1980). Human inference: Strategies and shortcomings in social judgment. Englewood Cliffs, NJ: Prentice-Hall. NORCROSS, J. C., BEUTLER, L. E., & LEVANT, R. F. (eds.). (2006). Evidence-based practices in mental health: Debate and dialogue on the fundamental questions. Washington, DC: American Psychological Association. NORGAARD, K. (2006). “People want to protect themselves a little bit”: Emotions, denial, and social movement nonparticipation. Sociological Inquiry, 76, 372–396. NORGAARD, K. M. (2011). Living in denial: Climate change, emotions and everyday life. Cambridge Massachusetts Institute of Technology. NOTTURNO, M. A. (2000). Science and the open society: The future of Karl Popper’s philosophy. New York: Central European University Press. NUIJTEN, M. B., HARTGERINK, C. H. J., VAN ASSEN, M. A. L. M., EPSKAMP, S., & WICHERTS, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985–2013). Behav Res. OAKES, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. Chichester, UK: Wiley. OLSON, E. (2015). A hidden scourge: Preying on the elderly. New York Times, pp. B1, 5. OPENCULTURE. A GUIDE TO LOGICAL FALLACIES EXPLAINED IN 2-MINUTE VIDEOS. WWW.OPENCULTURE.COM (accessed 11/20/ 16) ORESKES, N., & CONWAY, E. M. (2010). Merchants of doubt: How a handful of scientists obscured the truth on issues from tobacco smoke to global warming. New York: Bloomsbury Press. ORWELL, G. (1958 [1946]). Politics and the English language. In S. Orwell & I. Angus (eds.), The collected essays, journalism and letters of George Orwell: Vol. 4. In front of your nose, 1945–1950 (pp. 127–140). London: Secker & Warburg. OWEN, J., & IMEL, Z. E. (2010). Utilizing rating scales in psychotherapy practice: Theory and application. In L. Baer & M. A. Blais (eds.), Handbook of clinical rating scales and assessment in psychiatry and mental health (pp. 257–270). New York: Humana Press. OXMAN, A. D., GLASZIOU, P., & WILLIAMS, J. W., JR. (2008). What should clinicians do when faced with conflicting recommendations? BMJ, 337, a2530. OXMAN, A. D., & GUYATT, G. H. (1988). Guidelines for reading literature reviews. Canadian Medical Association Journal, 133, 697–703.

References 401 402

OXMAN, A.D., & GUYATT, G. H. (1993). The science of reviewing research. In K. S. Warren & F. Mostellar (eds.), Doing more good than harm: The evaluation of health care interventions (pp. 125– 133). New York: New York Academy of Sciences. PALING, J. (2006). Helping patients understand risks: 7 Simple strategies for successful communication. Gainesville, FL: Risk Communication Institute. PASHLER, H., MCDANIEL, M., ROHRER, D., & BJORK, R. (2012). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9, 105–119. PATAI, D., & KOERTGE, N. (2003). Professing feminism: Education and indoctrination in women’s studies. Lanham, MD: Lexington Books. PATRICK, J., & LI, M. (2012). An ontology for clinical questions about the contents of patient notes. Journal of Biomedical Informatics, 45, 292–306. PAUL, R. (1993). Critical thinking: What every person needs to survive in a rapidly changing world (rev. 3rd ed.). Santa Rosa, CA: Foundation for Critical Thinking. PAUL, R. W., & ELDER, L. (2004). Critical thinking: Tools for taking charge of your professional life. Upper Saddle River, NJ: Pearson Education. PAULOS, J. A. (1988). Innumeracy: Mathematical illiteracy and its consequences. New York: Hill & Wang. PAYER, L. (1992). Disease mongers: How doctors, drug companies, and insurers are making you feel sick. New York: Wiley. PENSTON, J. (2010). stats.con: How we’ve been fooled by statistics- based research in medicine. London: The London Press. PERKINS, D. (1992). Smart schools: From training memories to educating minds. New York: Free Press. PERKINSON, H. J. (1993). Teachers without goals, students without purposes. New York: McGraw-Hill. PETTIGREW, M., & ROBERTS, H. (2006). Systematic reviews in the social sciences: A practical guide. Maldon, MA: Blackwell. PHILLIPS, D. C. (1987). Philosophy, science, and social inquiry: Contemporary methodological controversies in social science and related applied fields of research. New York: Pergamon Press. PEWSNER, D., PATTAGLIA, M., MINDER, C., MARKS, A., BUCHER, M. C., & EGGER, M. (2004). Ruling a diagnosis in or out with “SpPIn” and “SnNOut”: a note of caution. BMJ, 329, 209–313. PHILLIPS, D. C. (1992). A social scientists bestiary: A guide to fabled, threats to, and defenses of, naturalistic social science. New York: Pergamon Press.

402 References 403

PILISUK, M., & ROUNTREE, J. A. (2015). The hidden structure of violence: Who benefits from global violence and war? New York: Monthly Review Press. PITTLER, M. H., BROWN, E. M., & EDWARDS, E. (2007). Static mag- nets for reducing pain: Systematic review and meta-analysis of randomized trials. Canadian Medical Association Journal, 177(7), 736–742. POERKSEN, U. (1995). Plastic words: The tyranny of modular language. J. Mason & D. Cayley (trans.). University Park, PA: Pennsylvania State University Press. POHL, R. F. (ed.). (2008). Cognitive illusions: A handbook on fallacies and biases in thinking, judgment and memory. New York: Psychology Press. POOLE, S. (2013). 10 of the worst examples of management-speak. The Guardian, April 25. www.theguardian.com. (accessed July 27, 2015). POPE, K. S., & VASQUEZ, M. J. T. (2010). Steps in ethical decision making. http://kspope.com. (Accessed 6/19/2016). POPE, K. S., & VASQUEZ, M. J. T. (2011). Ethics in psychotherapy and counseling: A practical guide (4th ed.). Hobokan, NJ: Wiley. POPPER, K. R. (1970). Normal science and its dangers. In I. Lakatos & A. Musgrave (eds.), Criticism and the growth of knowledge (pp. 52–53). New York: Cambridge University Press. POPPER, K. R. (1972 [1963]). Conjectures and refutations: The growth of scientific knowledge (4th ed.). London: Routledge and Kegan Paul. POPPER, K. R. (1992). In search of a better world: Lectures and essays from thirty years. New York: Routledge. POPPER, K. R. (1994). The myth of the framework: In defense of science and rationality. M. A. Notturno (ed.). New York: Routledge. PORTER, R. (2000). Quacks: Fakers & charlatans in English medicine. Charleston, SC: Tempus. PRASAD, V. K., & CIFU, A. S. (2015). Ending medical reversal: Improving outcomes, saving lives. Baltimore, MD: John Hopkins University Press. PRASAD, V., & IOANNIDIS, J. P. A. (2014). Evidence-based de- implementation for contradicted, unproven, and aspiring healthcare practices. Implementation Science, 9, 1. PRATKANIS, A. R. (1995). How to sell a pseudoscience. Skeptical Inquirer, 19, 19–25. PRATKANIS, A. R., & ARONSON, E. (2001). Age of propaganda: The everyday use and abuse of persuasion (rev. ed.). New York: Owl Books.

References 403 404

PROCTOR, R. N., & SCHIEBINGER, L. (eds.). (2008). Agnatology: The making and unmaking of ignorance. Palo Alto, CA: Stanford University Press. RABUY, B. (2014). Prison bankers cash in on captive customers. www. prisonpolicy.org (accessed 2/20/15). RAMPTON, S., & STAUBER, J. (2001). Trust us. We’re experts: How industry manipulates science and gambles with your future. New York: Penguin. RAMSEY, S. D. (2002). Incorporating economic analysis into clinical practice guidelines: A guide for hopeful users. Evidence-Based Medicine, 7, 164–166. RANK, H. (1984a). The peptalk: How to analyze political language. Park Forest, IL: Counter-Propaganda Press. RANK, H. (1984b). The pitch. Park Forest, IL: Counter-Propaganda Press. RATIONAL WIKI. (2015). Management speak. http://rationalwiki.org/ wiki/Management_speak. (Accessed 10/29/2016). REITANO, P., KIVUNJA, C., & PORTER, K. (2008). Teaching controversial issues in schools to prepare children for a sustainable global village. www.procon.org (accessed October 10, 2015). RISEN, J. (2015). Outside psychologists shielded U.S. torture program, report finds.The New York Times, July 10, A1. RISKIN, A., EREZ, A., FOULK, T. V., ET AL. (2015). They impact of rudeness on medical team performance: A randomized trial. Pediatrics, 136, 487–495. RITTEL, H. W. J., & WEBBER, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169. ROBERTS, J., & ARMITAGE, J. (2008). The ignorance economy. Prometheus, 26, 335–354. ROBERTSON, J. (2015). University suppressed study into racism on buses and “victimized” its co-author. The Guardian, February 26. www.theguardian.com/australia- news (accessed April 6, 2015). RODWIN, M. A. (2013a). Five un-easy pieces of pharmaceutical policy reform. Journal of Law, Medicine & Ethics, 41, 581–589. RODWIN, M. A. (2013b). Introduction: Institutional corruption and the pharmaceutical policy. Journal of Law, Medicine & Ethics, 41, 544–552. ROSA, L, ROSA, E., SARNER, L., & BARRETT, S. (1998). A close look at therapeutic touch. Journal of the American Medical Association, 279, 1005–1010. ROSE, J. D. (2011). Diverse perspectives on the groupthink theory: A literary review. Emerging Leadership Journeys, 4, 37–57. ROSE, S., BISSON, J., & WESSELY, S. (2004). Psychological debriefing for preventing post traumatic stress disorder (PTSD). Cochrane Database of Systematic Reviews, Issue 3. senberg, L. (2013). As

404 References 405

IRS takes aim at false social welfare organizations, will some in Congress take aim at the IRS? November 27. sunlightfoundation.com (accessed 6/5/2015). ROSENBERG, L. (2013). As IRS takes aim at fake social welfare organizations, will some in congress take aim at the IRS? http:// sunlightfoundation.com Nov. 27. (accessed 11/2014). ROSTON, A., & SINGER-VINE, J. (2015). Fostering profits: Abuse and neglect at America’s biggest for-profit foster care company. A buzzfeed news investigation. http://buzzfeed.com/aramros- ton/fostering-profits (accessed June 13, 2015). ROYDEN, D. (2015). The other 1%: Healing the wounds of Native American tragedies on Turtle Island. www.occupy.com (accessed November 30, 2015). RUBIN, A., & PARRISH, D. (2007). Problematic phrases in the conclusions of published outcome studies: Implications for evidence-based practice. Research on Social Work Practice, 17, 334–347. RUGGIERO, V. R. (1996). A guide to sociological thinking. Thousand Oaks, CA: Sage. SACKETT, D. L., RICHARDSON, W. S., ROSENBERG, W., & HAYNES, R. B. (1997). Evidence-based medicine: How to practice and teach EBM. New York: Churchill Livingstone. SAGAN, C. (1990). Why we need to understand science. Skeptical Inquirer, 14, 263–269. SAGAN, C. (1997). The demon-haunted world: Science as a candle in the dark. New York: Ballantine Books. SAGAN, C. BALONEY DETECTION KIT. BASED ON SAGAN (1997). http://users.tpg.com.au (accessed 5/22/2012). SAINI, M., & SHLONSKY, A. (2015). Systematic synthesis of qualitative research. New York: Oxford University Press. SALAS, E., TANNENBAUM, S., COHEN, D., & LATHAM, G. (2013). Developing and enhancing teamwork in organizations: Evidence- based best practices and guidelines.Hobokan, N.J.: Wiley. SANGHERA, J., GENTILE, L., GUERRAS-DELGADO, O’HANLON, L., BARRAQUES, A., HINTON, R. L., & STAHLHOFER, M. (2015). Human rights in the new global strategy. BMJ, 351(Suppl.), 42–45. SAVEDOFF, W. D., & HUSSMANN, K. (2006). Why are health systems prone to corruption? Global Comparison Report 2006. Transparency International (pp. 4–13). Ann Arbor, MI: Pluto Press. SAWHNEY, R. (2006). How to prepare an outstanding journal club presentation. Hermatologist. www.hematology.org. accessed 7/ 28/2008

References 405 406

SCHATTNER, A., BRONSTEIN, A., & JELLIN, N. (2006). Information and shared decision-making are top patients’ priorities. BMC Health Services Research, 6, 21. doi:10.1186/1472-6963-6-21 SCHILPP, P. A. (ed.). (1974). The philosophy of Karl Popper, vol. 2. LaSalle, IL: Open Court. SCHÖN, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books. SCHRAAGEN, J. M., MILITELLO, L. G., ORMEROD, T., & LIPSHITZ, R. (2008). Naturalistic decision making and macrocognition. Burlington, VT: Ashgate. SCHRANK, J. (1974). The language of advertising claims. Media and Methods, 10, 44–51. Home.olemiss.edu (accessed May 1, 2009). SCHWARTZ, I. M. (1989). (In)justice for juveniles: Rethinking the best interests of the child. Lexington, MA: Heath. SCHWARTZ, M. D., DOWELL, D., APERI, J., & KALET, A. (2007). Improving journal club presentations, or, I can present that paper in under 10 minutes. ACP J Club, 147, A8–A9. SCULL, A. (2005). Madhouse: A tragic tale of megalomania and modern medicine. New Haven, CT: Yale University Press. SCULL, A. (2015). Madness in civilization: A cultural history of insanity, from the Bible to Freud, from the madhouse to modern medicine. Princeton, NJ: Princeton University Press. SEDGWICK, P. (2014). Measuring the performance of screening tests. BMJ, 348, g4438. SEECH, Z. (1993). Open minds and everyday reasoning. Belmont, CA: Wadsworth. SEMMELWEISS, I. (1983). The etiology, concept and prophylaxis of childbed fever. K. C. Carter (trans.).Madison,Wis: University of Wisconsin Press. SHADISH, W. R. (1995a). Philosophy of science and the quantitative– qualitative debates: Thirteen common errors. Evaluation and Program Planning, 18, 63–75. SHADISH, W. R. (1995b). The quantitative–qualitative debates: “DeKuhnifying” the conceptual context. Evaluation and Program Planning, 18, 47–49. SHANTEAU, J. (1992). How much information does an expert use? Is it relevant? Acta Psychologica, 81, 75–86. SHEA, C. M., JACOBS, S. R., ESSERMAN, D. A., BRUCE, K., & WEINER, B. J. (2014). Organizational readiness for implementing change: A psychometric assessment of a new measure. Implement Sci, 9 (7). SHEN, C., & BJÖRK, B.- C. (2015). “Predatory” open access: A longitu- dinal study of article volumes and market characteristics. BMC Medicine, 13, 230.

406 References 407

SIEGEL, H. (1993). Not by skill alone: The centality of character to critical thinking. Informed Logic, XV.3, 163–177. SIEGFRIED, T. (2015). Top ten ways to save science from its statistical self. Science News, July 10, accessed 9/16/2015. SILVERMAN, W. A. (1980). Retrolental fibroplasia: A modern parable. New York: Grune & Stratton. SINGER, N. (2009). Medical papers by ghostwriters pushed therapy. The New York Times, August 5. SKINNER, B. F. (1953). Science and human behavior. New York: Knopf. SKRABANEK, P., & MC CORMICK, J. (1998). Follies and fallacies in medicine (3rd ed.). Chippenham, UK: Tarragon Press. SLOVIC, P. (ed.) (2010). The feeling of risk: New perspectives on risk perception. Washington, DC: Earthscan. SLOVIC, P., FINUCANE, M., PETERS, E., & MACGREGOR, D. G. (2002). The affect heuristic. In T. Gilovich & D. Griffin (eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 397–420). New York: Cambridge University Press. SMITH, A.(2002). Century of the self. BBC documentary. https://freed- ocumentaries.org video. SMITH, C. (2014). The 13 worst office jargon phrases staff love to hate. www.theguardian.com. Accessed 11/20/16. SNOWBALL, R. (1999). Finding the evidence: An information skills application. In M. Dawes, P. T. Davies, A. M. Gray, J. Mant, K. Seers, & R. Snowball (eds.), Evidence-based practice: A primer for health professionals (pp. 15–40). New York: Churchill Livingston. SOARES, H. P., DANIELS, S., KUMAR, A., ET AL. (2004). Bad reporting does not mean bad methods for randomized trials: Observational study of randomized controlled trials performed by the Radiation Therapy Oncology Group. BMJ, 328, 22–24. SOUMERAI, S. B., STARR, D., SUMIT, R., & MAJUMDAR, S. R. (2015). How do you know which health care effectiveness research you can trust? A guide to study design for the perplexed. Preventing Chronic Disease, 12, 1–42 (on line). SPROULE, J. M. (1994). Channels of propaganda. ERIC Clearinghouse. on Reading English and CommunicationBloomington In. STAATS, A. W. (2012). The marvelous learning animal: What makes human nature unique. Amhurst, NY: Prometheus. STACEY, D., LÉGARÉ, F., COL, N. F., BENNETT, C. L., BARRY, M. J., EDEN, K. B., … WU, J. H. C. (2014). Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews Issue, 1. STEINER, S., & BRZUZY, S., GERDES, K., & HURDLE, D. (2007). Using structured controversy to teach diversity content and

References 407 408

cultural competence. Journal of Teaching and Social Work, 23, 55–71. STIVERS, R. (2001). Technology as magic: The triumph of the irrational. New York: Continuum Publishing. STOLZER, J. M. (2013). The systemic correlation between psychiatric medications and unprovoked mass murder in America. New Male Studies: An International Journal, 2, 9–23. STRAUS, S. E., GLASZIOU, P., RICHARDSON, W. S., & HAYNES, R. B. (2011). Evidence-based medicine: How to practice and teach it (4th ed.). New York: Churchill Livingstone. STRAUS, S. E., RICHARDSON, W. S., GLASZIOU, P., & HAYNES, R. B. (2005). Evidence-based medicine: How to practice and teach EBM (3rd ed.). New York: Churchill Livingstone. STRAUSS, M. (2015). Treating old people like hamburgers. In These Times, August, p. 17. SUMMERFIELD, D. (2012). Afterword: Against “global mental health.” Transcultural Psychiatry, 49, 1–12. SWARTZ, R. J., & PERKINS, D. N. (1990). Teaching thinking: Issues and approaches. Pacific Grove, CA: Critical Thinking Press & Software. SWEENEY, H. M.(2000) Twenty-five ways to suppress the truth: The rules of disinformation. www.whale.to (accessed 10/15/2016). SZASZ, T. S. (1961). The myth of mental illness: Foundations of a theory of personal conduct. New York: Harper & Row. SZASZ, T. S. (1965). The myth of mental illness. In O. Milton (ed.), Disorders: Perspectives and trends (pp. 15–25). New York: Lippincott. SZASZ, T. S. (1987). Insanity: The idea and its consequences. New York: Wiley. SZASZ, T. S. (2007). The medicalization of everyday life. New York: Syracuse University Press. TANNE, J. H. (2010). AstraZeneca pays $520m fine for off label marketing. BMJ, 340, c2380. TESH, S. N. (1988). Hidden arguments of political ideology and disease prevention policy. New Brunswick, NJ: Rutgers University Press. TETLOCK, P. E. (2003). Correspondence and coherence: Indicators of good judgment in world politics. In D. Hardman & L. Macchi (eds.), Thinking: Psychological perspectives on reasoning, judgment and decision making (pp. 233–250). New York: Wiley. THE DARTMOUTH INSTITUTE. (2011). Improving patient decision- making in health care: A 2011 Dartmouth Atlas Report Highlighting Minnesota. Lebanon New Hampshire. 52ND MAUDSLEY DEBATE: MORE HARM THAN GOOD? (2015). www.youtube.com (accessed 3/4/2016).

408 References 409

THE SOCIAL DIMENSIONS OF SCIENTIFIC KNOWLEDGE. (2002). Stanford Encyclopedia of Philosophy. (accessed December 3, 2015). THOMAS, J. R. (2015). Parts I–III. So you want to be a whistleblower? A lawyer explains the process. www.retractionwatch.com. Accessed 1/2/2016 THOULESS, R. H. (1974). Straight and crooked thinking: Thirty-eight dishonest tricks of debate. London: Pan. THYER, B. A., & PIGNOTTI, M. G. (2015). Science and pseudoscience in social work practice. New York: Springer. TINDALE, C. W. (2007). Fallacies and argument appraisal. New York: Cambridge University Press. TORJESEN, I. (2015). Study links SSRIs to violent crime in young adults. BMJ, 351, h4945. TOULMIN, S. E. (2003 [1958]). The uses of argument. Cambridge: Cambridge University Press. TRACEY, T. J. G., WAMPOLD, B. E., GOODYEAR, R. K., & LICHTENBERG, J. W. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69, 218–229. TUCHMAN, B. W. (1984). The march of folly: From Troy to Vietnam. New York: Ballantine. TUFTE, E. (2007). Beautiful evidence. Cheshire, CT: Graphics Press. TURNER, E. H., MATTHEWS, A. M., LINARDATOS, E., TELL, R. A., & ROSENTHAL, R. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine, 358 (3), 252–260. URBINA, I. (2009). Despite red flags about judges, a kickback scheme flourished. New York Times, March 28, pp. A1, A13. US GOVERNMENT ACCOUNTABILITY OFFICE. (2014). Foster children: Additional federal guidance could help states better plan for oversight of psychotropic medications administered by managed-care organizations. GAO-14-362. Washington, DC: US Government Accountability Office. US GOVERNMENT ACCOUNTABILITY OFFICE. (2015). Report to congressional requesters: Antipsychotic drug use: HHS has ini- tiatives to reduce use among older adults in nursing homes, but should expand efforts to other settings. GAO-15- 211. Washington, DC: US Government Accountability Office. VAITHIANATHAN, R., MALONEY, T., PUTNAM-HORNSTEIN, E., & JIANG, N. (2013). Children in the public benefit system at risk of maltreatment: Identification via predictive modeling. American Journal of Preventive Medicine, 45, 354–359. VAN DER WEYDEN, M. B., ARMSTRONG, R. M., & GREGORY, A. T. (2005). The 2005 Nobel Prize in physiology or medicine. Med Journal Aust, 183, 612–614.

References 409 410

VANDRUFF, DEAN & MARSHALL (1995). Conversational terriorism. www.vandruff.com (accessed 11/15/2016). VAN VORIS, R., & LAWRENCE, J. (2010). Pfizer told to pay $142.1 million for Neurontin marketing fraud. www.bloomberg.com (accessed July 26, 2015). VIAN, T. (2008). Review of corruption in the health sector: Theory, methods and interventions. Heath Policy and Planning, 23, 83–94. VINCENT, C., & TAYLOR-ADAMS, S. (2001). The investigation and analysis of clinical incidents. In C. Vincent (Ed.), Clinical risk management: Enhancing patient safety (2nd ed., pp. 439–460). London, UK: BMJ. WABER, R. L., SHIV, B., CARMON, E., & ARIELY, D. (2008). Commercial features of placebo and therapeutic efficacy. Journal of the American Medical Association, 299 (9), 1016–1017. WALSH, B. (2013). Prisoners pay mullions to call loved ones every year. Now this company wants even more. www.huffington- post.com (accessed 6/10/2016). WALTON, D. N. (1991). Begging the question: Circular reasoning as a factor of argumentation. New York: Greenwood Press. WALTON, D. (1992a). Slippery slope arguments. New York: Oxford University Press. WALTON, D. (1992b). The place of emotion in argument. University Park, PA: Pennsylvania State University Press. WALTON, D. (1995). A pragmatic theory of fallacy. Tuscaloosa, AL: University of Alabama Press. WALTON, D. (1997). Appeal to expert opinion: Arguments from authority. University Park, PA: Pennsylvania State University Press. WALTON, D. (1999). Appeal to popular opinion. University Park, PA: Pennsylvania State University Press. WALTON, D. (2008). Informal logic: A pragmatic approach (2nd ed.). New York: Cambridge University Press. WALTON, D. (2013). Methods of argumentation. New York: Cambridge University Press. WALTON, D. (2016). Argument Evaluation and Evidence. Series 23, Law, Governance and technology. New York: Springer. WAMPOLD, B. E., & IMEL, Z. E. (2015a). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York: Routledge. WAMPOLD, B. E., & IMEL, Z. E. (2015b). What do we know about psychotherapy?—and What is there left to debate? The Society for the Advancement of Psychotherapy. http://societyforpsycho- therapy.org (accessed July 20, 2015).

410 References 411

WAMPOLD, B. E., MINAMI, T., TIERNEY, S. C., BASKIN, T. W., & BHATI, K. S. (2005). The placebo is powerful: Estimating placebo effects in medicine and psychotherapy from randomized clinical trials. Journal of Clinical Psychology, 61, 835–854. WARRAICH, H. J. (2015). Stop force-feeding the elderly. New York Times, Op. Ed., November 24, p. 25. WATSON, D. L., & THARP, R. G. (2014). Self-directed behavior: Self- modification for personal adjustment (10th ed.). Belmont, CA: Thompson/Wadsworth. WEBSTER, Y. O. (1992). The racialization of America. New York: St. Martin’s Press. WEGEMER, C. (2015). Nonprofit foundation rakes in cystic fibro- sis drug revenues. Truthout. October 7. www.truth-out.org (accessed 2/4/16). WEISBURD, D., LUM., C. M., & YANG, S.- M. (2003). When can we conclude that treatments or programs “Don’t Work?” Annals of the American Academy of Political and Social Science, 587, 31–48. WELCH, H. G. (2004). Should I be tested for cancer? Maybe not and here’s why. Berkeley, CA: University of California Press. WELCH, H. G., SCHWARTZ, L. M., & WOLOSHIN, S. (2011). Over- diagnosed: Making people sick in the pursuit of health. Boston, MA: Beacon Press. WELLBUTRIN. (1992). WELLBUTRIN (bupropion HCL), the non serotonergic alternative to Prozac (fluoxetine HCL) for many paden. American Journal of Psychiatry, 149 (5), A33–A31. WENNBERG, J. E. (2002). Unwarranted variations in healthcare delivery: Implications for academic medical centres. BMJ, 325, 961–964. WENNBERG, J. E., & THOMSON, P. Y. (2011). Time to tackle unwarranted variations in practice. BMJ, 342, d1513. WERNER, M. & LEHMAN, T. (1995). Focusing on successes only. University of Wisconsin, Eau Claire, Wis. WESTON, A. (1992). A rulebook for arguments (2nd ed.). Indianapolis, IN: Hackett. WHITAKER, R., & COSGROVE, L. (2015). Psychiatry under the influence: Institutional corruption, social injury, and prescriptions for reform. New York: Palgrave Macmillan. WILKENSON, R. (2011). How economic inequality harms society. TED Talk. https://www.ted.com (accessed October XX, 2015). WILKES, M. S., & HOFFMAN, J. R. (2001). An innovative aproach to educating medical students about pharmaceutical promotion. Academic Medicine, 76, 1271–1277.

References 411 412

WILLIAMS, W. M., & CECI, S. J. (1997). How’m I doing? Problems with student ratings of instructors and courses. Change: The Magazine of Higher Learning, 29, 12–23. WINEMILLER, M. H., BILLOW, R. G., LASKOWKI, E. R., & HARMSEN, W. S. (2003). Effect of magnetic vs sham-magnetic insoles on plantar heel pain: A randomized controlled trial. Journal of the American Medical Association, 290, 1474–1478. WOFFORD, J. L., & OHL, C. A. (2005). Teaching appropriate interactions with pharmaceutical company representatives: The impact of an innovative workshop on students attitudes. BMC Medical Education, 5, 5. WOLFE, S. M. (2015). Selective trial reporting: betraying trial participants, harming patients. BMJ, 350, h2753. WOLOSHIN, S., SCHWARTZ, L. M., & WELCH, H. G. (2008). Know your chances: Understanding Health Statistics. Berkeley, CA: University of California Press. WOODS, D. D., & COOK, R. I. (1999). Perspectives on human error: Hindsight biases and local rationality. In F. T. Durso, R. S. Nickerson, R. W. Schzaneveldt, S. T, Dumais, D. S. Lindsay, & M. T. Chi (eds.), Handbook of applied cognition (pp. 141–171). New York: Wiley. YOUNG, J. H. (1992a). American health quackery. Princeton, NJ: Princeton University Press. YOUNG, J. H. (1992b). The medical messiahs: A social history of health quackery in twentieth century America. Princeton, NJ: Princeton University Press. ZAMIR, E. (2014). The bias of the question posed: A diagnostic “invisible gorilla.” Diagnosis, 1, 245–248. ZASLOFSKY, A. F., & VOLPE, R. J. (2010). Graphing single-case data in Microsoft Excel 2007. School Psychology Forum, 4 (2). ZECHMEISTER, E. G., & JOHNSON, J. E. (1992). Critical thinking: A functional approach. Pacific Grove, CA: Brooks/Cole. ZERUBAVEL, E. (2006). The elephant in the room: Silence and denial in everyday life. New York: Oxford University Press. ZETTERQVIST, A. V., & MULINARI, S. (2013). Misleading advertising for antidepressants in Sweden: A failure of pharmaceutical industry self-regulation. PLoS One, 8 (5), e62609. ZILIAK, S. T., & MCCLOSKEY, D. N. (2013). The cult of statistical significance: How the standard error costs us jobs, justice and lives. Ann Arbor, MI: University of Michigan Press.

412 References 413â

Index

Absolute risk, 295, 371 Automatic thinking, 49 Absolute risk reduction, 295, 297, 371 Availability biases, 179, 185 Acceptable premises, 321 Avoidable ignorance, 73 Accessibility of data, 185 Activated ignorance, 11 Background questions, 206 Ad hominem, 168 Bafflegab, 142 Administrative data, 241 Bandwagon fallacy, 171 Ad verrecundium, 155–â58 Barnum Effect, 152 Advertising, 100–â101, 103–â8 Base rate, 294. See also Prevalence rate Advocacy, 38–â39, 89–â94 Begging the question, 168 Affective biases, 33–â34, 179–â90 Beliefs, 51 Affirming the consequent, 318–â19 about evidence, 5 Ageism, 169 about knowledge, 73–â78 Agency services, reviewing, 245–â47 about problems, 11 Agnatology, 47, 73 changing, 58 Ambiguity, 5 critical appraisal of, 96 Analytic thinking, 49–â50 overconfidence in, 182 Anchoring, 185 testing, 45 Anecdotal evidence, 150 theories as, 81 Anecdotal reports, 57 “Best evidence,” 41 Antiscience, 62 Bias(es), 50 Anxiety, 113 affective, 33–â34, 179–â90 Appeals to specialness, 105 avoiding, 127–â32 Applicability of resources, 213 cognitive, 34, 129, 132, 179–â90 Arguments, 317 common across helping professions, critical appraisal of, 317–â25 149–â61, 163–â65 evaluating, 32–â34 context-ârelated, 31 manner or style of presenter, 170 examples of, 130–â31 Assertive skills, 237–â40 in group settings, 167–â73, 175–â78 Assessment questions, 206 motivational, 33–â34 Assumptions, 23, 24, 261, 317 preparing a bias festival, 191–â92 of causal relationships, 159, 160 in professional contexts, spotting, 193, 195–â96 of hard-âheartedness, 153 in propaganda ploys, 96–â98 Astroturfs, 100 in randomized controlled trials, 258 Attitudes, 27–â30, 58 recognizing and avoiding influence of, 7 Attractiveness, personal, 179 in research, 12 Attrition, 263–â64 self-âserving, 37 Authority, 3, 155–â58 See also Fallacies; specific biases

413 414

Biobabble, 36, 142 Comparison groups, 262 Biomedical industrial complex, 10 Competing contingencies, 4–5 Bogus claims, 11, 96, 98, 105 Conclusions, 317–18 Boolian indicators, 209 Concurrent validity, 289 “Buddy–buddy syndrome,” 34, 197 Confirmation biases, 7, 150, 153–55, 182, 185 “Bullshit,” 123, 124 Conflicts of interests, 9, 13, 15–17, 89, 266–67 Bureaucratese, 142 Confusion, 12, 35, 95 Conjectures, 55 Campbell database, 211 Consistency, 50–51 Caring about clients, 6, 65 Construct validity, 262, 289 Case conferences, 197–99, 201 Consumer fraud, 18 Case examples, 150 Content validity, 289 Causal reasoning, 159 Context, 6–8, 11–12, 31 Causation, correlation and, 159, 160 Contingencies, competing, 4–5 Causes, research regarding, 309–11, 313 Contingency table, 154, 297 Censorship, 35, 95 Control, illusion of, 182 Circular reasoning, 168 Controlled thinking, 49 Claims Controversy, 79–85, 87–88 accuracy of, 266 COPES questions, 208–9 in arguments, 317–18 Corrective feedback, 249, 250 bogus, 11, 96, 98, 105 Correlation, causation and, 159, 160 critical appraisal of, 99–100 Corroboration, 50–51, 57 evaluating, 23, 32–34 Corruption, 15–19 false research findings, 12 recognizing, 36 in human services advertisements, 104–5 in scientific/medical research, 13–14 misleading, 96 in scientific research, 52–53 questionable criteria for evaluating, 3 Courage, intellectual, 29 questions about different kinds of, 2 Credibility, truth and, 47–48 testing, 55–56 Crimes, 89 Classifications, 303–5 Criterion validity, 289 “Classism,” 169 Critical discussion, 373 Clients Critically appraised topics (CAT), 223–28 feedback from, 215 Critical rationalism, 55 as informed participants, 229–35 Critical thinking, 5 values of, 41 benefits of, 30–31 Clinical expertise, 41, 214 characteristics of, 6, 22–23, 38 Cochrane database, 211 context in, 6–8, 31 Cognitive biases, 34, 129, 132, 179–90 costs of avoiding, 64–65 Commodification of problems, 11 in educational system, 48 Communication for evidence-informed practice and policy, 39–44 with controversial issues, 79–82 knowledge, skills, and values related to, 21–39 effective, 36–37 personal obstacles to, 363–65, 367–69 in science, 59 for problem solving and ethical behavior, 5–7 Community of inquiry, 79–80 purpose in, 21, 22

414 Index 415

questions arising in, 7–8, 23, 24 Disease mongering, 113, 114 as radical activity, 7–8 Distortion, 35, 95 teaching of, 351–52 Diversion, 169 valuable intellectual traits for, 29 Documentation, 157 Criticism, 27, 54, 64 Dogmatism, 50 of assumptions, 55 Domain-specific knowledge, 25–26 changing beliefs based on, 58 Domino effect, 172 in developing knowledge, 45 Doublespeak, 12 openness to, 32 Dr. Fox Effect, 99 self-, 37 Dual process models, 49 Culture of thoughtfulness, 341–45 Curiosity, 31 EBPs approach, 43–44 Cynicism, 371 Eclecticism, 373 Ecological fallacy, 266 Data analysis, 264–65 Effectiveness questions, 207 Debate, 79 Effectiveness studies, 269–78 Decision making, 1–3 Effect size, 274–75 about interventions, 67–71 Efficiency, 8 acknowledging mistakes and errors in, 32 Egocentric biases, 37 ambiguity in, 5 Either–or fallacy, 172 avoiding fallacies, biases, and pitfalls Emotional influences, 185 in, 127–32 Emotions, appeal to, 98–99 awareness of personal obstacles in, Empathy, 31–32 363–65, 367– 69 Empiricism, 373–74 critical appraisal of arguments in, 317–25 Environments of education/practice, 337– 39 ethical concerns, 4, 327–29, 331–32 culture of thoughtfulness, 341–45 evaluating claims and arguments, 32–34 excuses for poor-quality services, 369–70 ignorance in, 5 journal club, 353–54 moral issues in, 4 self-awareness of personal obstacles to shared, 229–31, 341– 42 critical thinking, 361–63, 365– 67 Socratic questions related to, 24–25 self-development in EBP process, uncertainties in, 4–5 355–57, 359–60 See also Critical thinking teaching of critical thinking skills, Deductive inference, 318–19 347, 349–50 Delinquency prevention program, 109, 111–12 Errors, 48, 56, 73, 250–52, 255–56 Denials, 73–74 acknowledging, 32 Depression, 113 diagnostic, 304 Description questions, 206 iatrogenic effects, 4 Descriptive validity, 262 identifying, 37 Diagnosis, 303 medical, 4, 46 Diagnostic questions, 206 medication, 4 Diagnostic tests, 303–5, 307– 8 from not understanding science, 52 Disagreements, 32, 79, 317 and problem-solving phases, 251 Discrimination, 89 See also Bias(es); Fallacies

Index 415 416

Ethical behavior, critical thinking integral to, 5–7 Excuses, 369–70 Ethical issues, 4, 6, 229, 327–29, 331–32 Experience, fallacy of, 150 Evaluation Experimental tests, 57 of agency services, 245–47 Expertise, 249–56 of arguments, 320–21, 323–25 Explanations, rival, 260 of claims and arguments, 23, 32–34 External validity, 262 of diagnostic tests, 303–5, 307–9 of EBP service outcomes, 241–47 Fabrication, 18, 35, 95 of effectiveness studies, 269–78 Face validity, 289 of evidence, 22, 23, 269–78 Facts, 51 of expertise, 249–56 Fallacies of intervention plans, 333, 335–36 in arguments, 318–19 of knowledge claims, 3 avoiding, 7, 127–32, 137, 139–48 of practice/policy guidelines, 279–81, 285 common across helping professions, questionable criteria for, 3 149–61, 163– 65 of research regarding causes, 309–11, 313 examples of, 130–31 of research reviews, 279–81, 283–84 in group settings, 167–73, 175–78 of self-report measures, 287–89, 291– 92 involving language, 12 of teaching of critical thinking skills, preparing a fallacy festival, 191–92 347, 349–50 in professional contexts, spotting, Evidence, 310 193, 195–96 absence of, 212 Professional Thinking Form, 137, 139–48 anecdotal, 150 in propaganda ploys, 96–98 evaluating, 22, 23, 269–78 recognizing, 7, 33 Evidence-based (evidence-informed) practice See also Bias(es); specific fallacies (EBP), 39–40, 203 Fallacy of experience, 150 applying steps in, 205–15 Fallacy spotting, 193–96 approach of, 43–44 False advertising, 104 asking hard questions in, 237–40 False dilemma, 172 continued self-development in, False negative rate, 374 355–57, 359–60 False positive, 294 critical thinking for, 39–44 False positive rate, 304, 372 evaluating service outcomes in, 241–47 Falsifiability, 20, 51, 56 involving clients as informed Falsification, 18, 45 participants, 229–35 Fast and frugal heuristics, 179, 250 posing questions and seeking answers Financial issues in, 217–18 in helping professions, 9–11 preparing critically appraised topics, 223–28 influence of special interests, 117–22 process of, 40–42, 205–15, 355–57, 359– 60 quest for profit at expense of helping, 39 purchasing, 241 spending on quackery, 21 reasons for creating, 42–43 Follow-up rate, 302 reviewing your expertise in, 249–56 Foreground questions, 206 working in interdisciplinary evidence- Framing effects, 181–82 informed teams, 219–22 Framing problems, 1, 3, 113–16

416 Index 417

Fraud, 13, 17–19, 36, 52–53 avoidable, 73 Frequencies, 296–300 in decision making, 5 Fundamental attribution error, 31, 181, 185 knowledge and, 47 promotion of, 11–12, 96 Gambler’s fallacy, 184–85 recognizing, 27 Gender, 179 of social aspects of science, 80 Generalizations, 150–51 strategic, 89 Genetic fallacy, 168 valuing, 27 Genuine evidence, 310 Ignorance economy, 47 Gobbledygook, 142 Ignoring sample size, 179 Good intentions, 45–46 Illusion of control, 182 Grounds, 318 Illusion of knowing, 48 Group dynamics, 167–73, 175– 78 Incomplete claims, 105 Groupthink, 101, 170–71, 197–99, 201 Inferences, 23 Informal fallacies, 318–19 Hardheartedness, 153, 207 Informed consent, 229 Harm Informed participants, clients as, 229–35 advocating against, 39 Institutional corruption, 14 from diagnostic errors, 304 Insufficient adjustment, 185 helper-induced ( See Iatrogenic effects) Integrating diverse sources/types of helpful distinctions in avoiding, 44–52 information, 214 from psychotropic medicines, 60 Integrity, intellectual, 29 questions regarding, 207 Intellectual traits, for critical thinking, 29 Hasty generalizations, 150 Interdisciplinary teams, 171, 219–22 Helping professions Internal validity, 262 fraud and corruption in, 18–19 Interpersonal dynamics, 167–73, 175– 78 as huge businesses, 9–11 Intervention plans, 333, 335–36 influence of special interests on, 117–22 Interventions propaganda in, 96 association between outcomes and, 153–55 See also specific topics checklist for reviewing, 335–36 Hermeneutics, 372 making decisions about, 67, 69–71 Heuristics, 179, 250 newness of, 155 Hindsight bias, 180–81 tests of, 257 Homogeneity, 288 Intuitive thinking, 49–50 Humanistic thinking, 4 Human services advertising, 103–8 Journal club, 353–54 Humility, intellectual, 29 Journals, 12–14, 59, 105 Hypotheses, 317 Judgments, 4, 5, 23, 49 Justification, 45, 61 Iatrogenic effects (helper-induced harm), 4, 45–46, 207 Knowledge, 23, 372 Ignorance, 73 accurate estimates of, 37 accurate estimates of, 37 criteria for evaluating claims of, 3 activated, 11 in critical thinking, 23, 25–27

Index 417 418

Knowledge (Cont.) Meta-analysis, 210 domain-specific, 25–26 Meta-cognitive thought, 26–27 exaggerated claims of, 65 Methodological search filters, 210 ignorance and, 47 Misleading claims, 96 and illusion of knowing, 48 Mistakes, 6–7, 32, 37, 250 objective, 47 Money. See Financial issues personal, 47 Moral issues, 4, 8, 62 privileged, 62 Motivational biases, 33–34 reviewing beliefs about, 73–78 rights to, 7, 8 Natural frequencies, 296–301 rooted in culture, 62 Necessary and sufficient criteria, 318–19 scientific, social dimensions of, 52 Negative predictive value, 373 self-, 26– 27 Newness, relying on, 155 Known unknowns, 73 Newspeak, 12 “No evidence,” 310 Labeling, 8, 34, 64, 158 Nonfallacy items, 160, 173, 186 Language, 36–37, 123–25, 142, 260 Nonjustificationist epistemology, 372–73 Law of large numbers, 183 “Normal” science, 59–60 Law of small numbers, 183 Number needed to treat (NNT), 295–96, 373 Learning, 27, 37 Likelihood ratio, 372 Objective knowledge, 47 Likelihood ratio of a negative test result, 372 Objectivity, 55 Likelihood ratio of a positive test result, 372 Observation, 61 Linguistic analysis, 207 Odds ratio, 297 Logical positivism, 61, 372 Omission bias, 184 Opinions, 51 Management speak, 224 Outcomes Manner, fallacy of, 170 of EBP, evaluating, 241–47 Marketing, 10–15 intentions vs., 45–46 Medical errors, 4, 46 interventions and, 153–55 Medical fraud, 17 Overconfidence, 48, 182 Medicalization of problems, 10, 64, 113 Overinterpretation, 158 Medical journals, 13–14 Oversimplifications, 158–59 Medical social control, 8 Medication errors, 4 Palaver, 123 Medications Paradigm, 375 advertising of, 103, 105 Parsimony, 57–58 deaths from, 46 Pathological science, 53 diagnosed based on corrupt research, 16 Patterns, 57 epidemic uses of, 8 Peer reviews, 11–13, 18, 101 fraudulent prescribing of, 17–18 Personal knowledge, 47 given to children, 43 Persuasion, 34, 50 revealing of false information about, 11 Pharmaceutical companies/industry Medicine, new ideas in, 54 academic researchers and, 14, 15

418 Index 419

advertising by, 103 social construction of, 113 astroturfs created/funded by, 100 unsolvable, 59 deceptions on part of, 13 “wicked,” 5 influence of pitches by, 96 Problem solving promotion of ignorance by, 11 awareness of personal obstacles in, propaganda from, 105 361–63, 365– 67 studies conducted by or for, 266–67 barriers to, 250, 251 Phenomenology, 375 critical appraisal of arguments in, 317–25 PICO questions (patient, intervention, critical thinking integral to, 5–7, 39 comparison, and outcome), 208 ethical and moral issues in, 4 Plagiarism, 18 learning from mistakes in, 53–54 Point of view, 50 phases and common errors, 251 Political corruption, 14 in science, 59 Poor-quality services, excuses for, 371–72 Socratic questions related to, 24–25 Popularity fallacy, 171 See also Critical thinking Positive predictive value, 375 Professional organizations, 9, 13, 15–17, 19 Post hoc ergo propter hoc, 7, 160, 313 Professional Thinking Form, 137, 139–48 Postmodernism, 62, 63 Profit making.See Financial issues Post positivism, 373 Prognosis questions, 206–7 Post-test odds, 373 Proof, 50–51 Posttest probability, 373 Propaganda, 95–101 Practice/policy guidelines biased views in, 50 evidence-informed, 39–44 context hidden by, 31 quality of, 279–81, 285 delinquency prevention program, 109, 111–12 Prediction questions, 206–7 disguised advertising as, 100–101 Predictions, 180–81, 293–98, 299–300 in evidence-based practice, 44 Predictive accuracy, 373 examples of, 95–98 Predictive validity, 289 function of, 8 Predispositions, 27–28 illusion of argument in, 318 Preferences, 51–52 influence of special interests, 117–22 Premature closure, 179 language of, 123–25 Premises, 318, 320–21 misleading problem framing, 113–16 Pretest odds, 373 ploys of, 96–98 Pretest probability, 373 pseudoscience as, 20 Prevalence rate, 184, 294–95, 373 in public relations, 100 Prevention questions, 207 recognizing and avoiding influence Privileged knowledge, 62 of, 34–35 Probabilities, 264, 299–300 techniques of, 96 Problems in technological society, 8–9 beliefs about, 11 Pseudoevidence, 310 commodification of, 11 Pseudoscience, 19–20, 36, 373 factors in shaping, 8 Psychobabble, 36, 142 framing, 1, 3, 113–16 Public relations industry, 100 medicalization of, 10, 113 Purpose, 21, 22, 55

Index 419 420

Quackery, 20–21, 36 lack of truth in, 12–14 Quality filters, 210 problem framing in, 114 Quality of study rating form, 272 quality filters for locating, 210 Questions regarding causes, 311–13, 315 in critical thinking, 2, 6–8, 23, 24, 28, 205–9 rigorousness of, 57, 262 in evidence-based practice, 217–18, 237– 40 self-report measures, 287–89, 291– 92 hard, 237–40 Research question, critical appraisal obstacles to posing, 209 of, 257–67 Socratic, 24–25 Research reviews, quality types of, 206–7 of, 279–81, 283– 84 Resources for practitioners, 211–12 Race, 179 Retrospective accuracy, 376 Racism, 169 Risk assessment, 293–99, 301 Random assignment, 273 Risk questions, 206–7 Rationalizing, 45 Rival hypotheses, 260 Reasoning, 1, 6, 50 causal, 159 Sample size, 179, 262–63 circular, 168 Satisfying heuristic, 179 truth and, 44–45 Scholarship, marketing disguised as, 12–15 Reasoning-in- Practice Games Science, 52–63, 374 common practice fallacies and biases, antiscience and, 62 149–61, 163– 65 misunderstandings and misrepresentations group and interpersonal dynamics, of, 60–62 167–73, 175– 78 “normal,” 59–60 more cognitive and affective biases, 179–90 objectivity, 374 Rebuttal, 321 openness to scrutiny in, 20 Recency effects, 185 parsimony in, 57–58 Red herrings, 169 pathological, 53 Reflection, 7, 31, 37 patterns and regularities in, 57 Reflexivity, 37 relativism and, 62–63 Regression effect, overlooking, 179, 182–83 rigorousness of tests in, 57 Regularities, 57 skeptical attitude in, 58 Relative risk, 297 Scientific fraud, 18 Relative risk reduction, 295, 297–98, 374 Scientific literature, unreliability of, 13–14 Relativism, 62–63, 374 Scientific objectivity, 374 Relevant premises, 320 Scientism, 62, 374 Reliability, 13–14, 263, 287–88 Screening devices, 294 Representativeness bias, 179 Search history log, 212–13 Research Self-awareness, 37, 361–63, 365– 67 critical appraisal of, 213, 257–67 Self-criticism, 37 in EBP, 41–42 Self-esteem, 35 effectiveness studies, 269–78 Self-knowledge, 26– 27 flawed, 52–53 Self-report measures, 287–89, 291– 92 kinds of, 259 Sensitivity, 294, 374

420 Index 421

Sexism, 169 Taboos, 73 Shared decision making Teams (SDM), 229–31, 341– 42 avoiding groupthink in meetings, Significance testing, 265 197–99, 201 Simplifying strategies, 179 enhancing effectiveness Skepticism, 58, 374 of, 219–20 Skills, critical thinking, 23, 26 Technology, 8–9, 43 Slippery slope fallacy, 172 Testimonials, 151–52 Small numbers, law of, 183 Test–retest reliability, 288 Social dimensions of scientific Tests knowledge, 52 of claims, 55–56 Social injustice, 89 diagnostic, 303–5, 307– 9 Social problems industry, 9–11 of interventions, 257 Social psychological strategies, 96 predictive value of, 294–95 Social tranquilizers, 4 relativists’ view of, 62 Society rigorousness of, 57 constraints in, 63 of scientific guesses, 55, 56 technological, 8 Theories, 55, 56, 261, 375 Sociocentric biases, 7, 37 Theory-ladenness (of perception), 375 Softhearted/softheaded, 153 Thinking, 50 “So-what claim,” 105 meta-cognitive levels of, 26–27 Special interests, 117–22 normal, 6 Specificity, 294, 374 types of, 49–50 Staff meetings, 171 See also Critical thinking Standardization, 8 Thoughtfulness, culture of, 341–45 Statements of fact, 317 Tradition, relying on, 155 Statistical conclusion validity, 262 Tricks, 149 Statistical prediction, 153 True negative rate, 375 Statistical significance, 265 True positive rate, 375 Statistical testing, 265 Truth, 375 Statistics, 264–65 activated ignorance and, 11 Status quo bias, 179 credibility and, 47–48 Stereotyping, 169, 179 factors hampering search for, 52 Strategems, 149 feelings as criterion for, 46 Strategic ignorance, 89 reasoning and, 44–45 Strategies in research, 12–14 of marketing, 10–11 valuing, 31–32 simplifying, 179 widely accepted information vs., 46 Strawperson argument, 172 Style, fallacy of, 170 Uncertainties, 4–5, 42, 207 Sufficient grounds, 321 Uncritical documentation, 157 Sunk costs, 179 Unfounded authority, 155–58 Sweeping generalization, 33 Unknown knowns, 73 Systematic review, 208, 213 Unknown unknowns, 73

Index 421 422

Vague claims, 105, 124 propagandists’ appeal Vagueness, 152, 293 to, 98–99 Validity, 213, 262, 263, 288–89 Warrants, 318 Values Weasel words, 12, 105 client, 41 Wellness, 114 in critical thinking, 27–30 “Wicked” problems, 5 of marketing, 10–12 Widely accepted information, 46

422 Index