W&M ScholarWorks

Dissertations, Theses, and Masters Projects Theses, Dissertations, & Master Projects

Spring 2017

A Delphi Study Regarding How, Can, and Should Individual Psychology Demonstrate Efficacy andff E ectiveness Given Evidence Based Practice Evaluation Standards

Sterling Price Travis College of William and Mary - School of Education, [email protected]

Follow this and additional works at: https://scholarworks.wm.edu/etd

Part of the Education Commons

Recommended Citation Travis, Sterling Price, "A Delphi Study Regarding How, Can, and Should Individual Psychology Demonstrate Efficacy andff E ectiveness Given Evidence Based Practice Evaluation Standards" (2017). Dissertations, Theses, and Masters Projects. Paper 1499450001. http://doi.org/10.21220/W4P945

This Dissertation is brought to you for free and open access by the Theses, Dissertations, & Master Projects at W&M ScholarWorks. It has been accepted for inclusion in Dissertations, Theses, and Masters Projects by an authorized administrator of W&M ScholarWorks. For more information, please contact [email protected]. Running head: INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS

A DELPHI STUDY REGARDING HOW, CAN, AND SHOULD INDIVIDUAL

PSYCHOLOGY DEMONSTRATE EFFICACY AND EFFECTIVENESS GIVEN

EVIDENCE BASED PRACTICE EVALUATION STANDARDS

______

A Dissertation

Presented to

The Faculty of the School of Education

The College of William & Mary in Virginia

______

In Partial Fulfillment

Of the Requirements for the Degree

Doctor of Philosophy

______

by

Sterling P. Travis

March, 2017

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS iii

A DELPHI STUDY REGARDING HOW, CAN, AND SHOULD INDIVIDUAL

PSYCHOLOGY DEMONSTRATE EFFICACY AND EFFECTIVENESS GIVEN

EVIDENCE BASED PRACTICE EVALUATION STANDARDS

By

Sterling P. Travis

______

Approved March 2017 by

Charles (Rip) R. McAdams, Ph.D.

Chairperson of Doctoral Committee

Victoria A. Foster, Ph.D.

Jacqueline Rodriguez, Ph.D.

Richard E. Watts, Ph.D.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS iv

Dedication

This dissertation is dedicated to Karen Whitcomb, Charlie Travis, Mary Carver, and Velma

Walton. Whom each taught me to pursue everything with passion, to love others, and to enjoy

life to the fullest.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS v

TABLE OF CONTENTS

CHAPTER ONE ...... 2

General Statements on Individual Psychology ...... 3

Evidence Based Practice ...... 7

Definition of Evidence Based Practice and Empirically Supported Treatments ...... 7

History of EBP Movement ...... 8

Goal of EBP Movement ...... 10

Debate Around EBP ...... 11

Pressure to move Towards EBP ...... 12

Struggle for Contemporary Theories to Adapt to EBP ...... 14

Statement of the Problem ...... 16

Purpose of the Study ...... 19

Sample Description and Data Gathering Procedures ...... 20

Summary ...... 24

CHAPTER TWO ...... 25

Review of Selected and Current Literature ...... 25

EBP Evaluation Guidelines and Standards ...... 25

EBP General Evaluation Standards ...... 26

Treatment efficacy...... 26

Treatment effectiveness...... 28

EST Evaluation Process ...... 30

Focus on manualized treatment...... 32

Focus on outcome research...... 34 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS vi

Summary of EBP Evaluation Standards, Guidelines, and Process ...... 35

Limitations of the EST Evaluation Method ...... 36

Debate Around EBP and EST ...... 37

Limitation of Evaluation Standards and Guidelines: General ...... 38

Limitation of Applying Evaluation Standards and Guidelines to Individual Psychology .... 41

Argument for Individual Psychology’s Continued Relevance ...... 44

Summary, Trends, and Pressure ...... 47

Need for Examination of Individual Psychology’s Efficacy and Effectiveness ...... 48

Debate and Suggested Adaptations and Integration ...... 50

Summary ...... 52

CHAPTER THREE ...... 53

Methodology ...... 53

The Delphi Method ...... 53

Development of the Delphi Method ...... 54

Strength and Weakness Considerations of the Delphi Method ...... 55

Panel Selection ...... 58

Selection and Recruitment of Participants ...... 58

Data Collection ...... 59

Role of the Researcher ...... 60

Data Collection and Analysis Description by Rounds ...... 60

Initial Round Data Collection ...... 60

Initial Round Data Analysis ...... 61

Second Round Data Collection ...... 62 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS vii

Second Round Data Analysis ...... 63

Preparation for Round Three ...... 63

Third Round Data Collection ...... 64

Third Round Data Analysis ...... 64

Ethical Considerations ...... 67

Limitations ...... 67

Summary ...... 68

CHAPTER FOUR ...... 69

Results ...... 69

Review of Delphi Process ...... 70

Delphi Response Profiles by Each Round ...... 72

Round I...... 72

Round II ...... 86

Round III ...... 108

Stability Testing ...... 118

Summary of Findings for Each Component ...... 124

How Component Findings ...... 124

Efficacy...... 125

Effectiveness...... 129

Can and Should Component Findings ...... 134

Efficacy...... 135

Effectiveness...... 143

Universal Consensus ...... 153 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS viii

Summary ...... 154

CHAPTER FIVE ...... 156

Discussion of Results ...... 156

Summary of the Study ...... 157

Interpretation and Implications of Results ...... 160

The How Component ...... 161

Efficacy...... 161

Implications of how component suggestions trending towards consensus (efficacy). 162

Effectiveness...... 164

Implications of how component suggestions trending towards consensus (effectiveness)...... 165

Factors contributing to the movement away from consensus for both efficacy and effectiveness...... 167

The Can Component ...... 169

Efficacy...... 170

Implications of can component suggestions (efficacy)...... 170

Effectiveness...... 173

Implications of can component suggestions (effectiveness)...... 173

The Should Component ...... 177

Efficacy...... 177

Implications of should component suggestions (efficacy)...... 178

Effectiveness...... 181

Implications of should component suggestions (effectiveness)...... 182

Overall Summary and Implications of Themes Identified ...... 186 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS ix

Research design ...... 187

Operationalizing, standardizing, and manualization...... 188

Dissemination...... 189

Support...... 190

Limitations of this Study ...... 191

Respondents ...... 191

Implementation of the Delphi Method ...... 191

Implications for Individual Psychology ...... 193

Field of Individual Psychology ...... 193

Adlerian Organizations and Institutions ...... 194

Adlerian Researchers ...... 194

Recommendation’s for Future Research ...... 196

Summary ...... 198

References ...... 200

LIST OF APPENDICES

Appendix A ...... 214 Recruitment Email Inviting Panel Members to Participate in Round One

Appendix B ...... 216 Round One Qualtrics Questionnaire

Appendix C ...... 218 Informed Consent

Appendix D ...... 221 Email to Panel Members Regarding Round One Results

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS x

Appendix E ...... 238 Recruitment Email Inviting Panel Members to Participate in Round Two

Appendix F ...... 239 Round Two Qualtrics Questionnaire

Appendix G ...... 248 Email to Panel Members Regarding Round Two Results

Appendix H ...... 286 Recruitment Email Inviting Panel Members to Participate in Round Three

Appendix I ...... 287 Round Three Qualtrics Survey

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS xi

Acknowledgements

I would like to acknowledge the following individuals who have encouraged, challenged, and supported me in achieving my goals.

• Dr. McAdams, having you as my advisor, supervisor, and mentor while at William and Mary has been a privilege. I am grateful for the guidance that you provided me throughout this process. I appreciate the support that you gave me when times were tough, and always felt encouraged after talking with you. You taught me so much, and I have become a better clinician and educator because of you. Thank you. • Dr. Foster, thank you for all of your feedback and suggestions (and helping me discover the Delphi process). From the beginning of the doctoral program you always made me feel welcome, and I appreciate all that you have taught me. Thank you. • Dr. Rodriquez, I learned a great deal from you about the Delphi process. I appreciate the support you gave me and enthusiasm you had for my dissertation. Thank you. • Dr. Watts, having you serve on my dissertation committee was an honor. I have always appreciated your mentorship as I have developed as an Adlerian. I learned so much from you about the research process and about contributing to the field of individual psychology. Thank you. • Dr. ‘s Gressard & Brendel, thank you for all that you both have taught me while I have been at William and Mary. You each welcomed me into the William and Mary family and I appreciate your mentorship and friendship. I am a better supervisor and educator because of each of you. Thank you. • Catie, Amy, Sharon, & Pam, this experience at William and Mary would have never been the same without each of you. I am so glad that we have become such close friends and that we collaborated, supported, and challenged each other throughout this process. You each hold a special place in my heart. Thank you. • Kevin Stoltz, I would have never found individual psychology without you. I never knew I would go beyond Ole Miss, but your guidance allowed me to realize my dreams. Your mentorship and friendship has meant so much to me, and I appreciate all the support, encouragement, and feedback that you have provided me all of these years. Thank you. • My parents, I could have never achieved any of my goals without each of you. Dad, you taught me the importance of doing all things with passion and doing them the right way. Mom, you spent countless hours with me through grade school when times were hard, and have always showed me the meaning of selflessness and caring for others. I thank both of you from the bottom of my heart for all that you have given me. I love both of you. Thank you. • My sister Sarah, you have always been an inspiration to me academically and personally. I always have looked up to you, and thank you for always supporting me and encouraging me to keep going. Without you I probably wouldn’t have found my passion for counseling, and I always love our conversations about the mental health field. Thank you. • To my love Alex, I cannot thank you enough. You have unwaveringly supported me through the tough times, and celebrated with me all of the good times. You have been so patient while I have worked to achieve this goal. You have kept me grounded and have always encouraged me to believe in myself. I love you from the bottom of my heart, and appreciate all that you do for me. Thank you. Running head: INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS

LIST OF TABLES

Table 1...... 73 Round One - Revised Comprehensive List of Suggestion (Efficacy)

Table 2...... 80 Round One - Revised Comprehensive List of Suggestion (Effectiveness)

Table 3...... 87 Item Median Scores and Interquartile Ranges for the Can and Should Components (Efficacy)

Table 4...... 90 Item Median Scores and Interquartile Ranges for the Can and Should Components (Effectiveness)

Table 5...... 92 Suggestions Percentages and Frequencies for Panel Member Rankings (Efficacy)

Table 6...... 95 Item Percentages and Frequencies for Panel Member Rankings (Effectiveness)

Table 7...... 98 Round Two - Revised Comprehensive List of Suggestion (Efficacy)

Table 8...... 102 Round Two - Revised Comprehensive List of Suggestion (Effectiveness)

Table 9...... 109 Round Three - Item Median Scores and Interquartile Ranges for the Can and Should Components (Efficacy)

Table 10...... 111 Round Three - Item Median Scores and Interquartile Ranges for the Can and Should Components (Effectiveness)

Table 11...... 114 Round 3 - Item Percentages and Frequencies for Panel Member Rankings (Efficacy)

Table 12...... 115 Round 3 - Item Percentages and Frequencies for Panel Member Rankings (Effectiveness) INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS iii

Table 13...... 120 Stability Testing – Spearman Rank Correlation Coefficient for the Can and Should Components (Efficacy)

Table 14...... 122 Spearman Correlation Coefficients for the Can and Should Components (Efficacy)

Table 15...... 122 Spearman Correlation Coefficient for the Can and Should Components (Effectiveness)

Table 16...... 124 Spearman Correlation Coefficients for the Can and Should Components (Effectiveness)

Table 17...... 125 Movement Away from Consensus Between Rounds Two and Three (Efficacy)

Table 18...... 128 Suggestions that were Ranked by 50% of the Panel Members in Round Two and Three (Efficacy)

Table 19...... 129 Movement Away from Consensus Between Rounds Two and Three (Effectiveness)

Table 20...... 133 Suggestions that were Ranked by 50% of the Panel Members in Round Two and Three (Effectiveness)

Table 21...... 135 Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy) Can Component) Suggestions

Table 22...... 136 Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy Can Component) Suggestions with Spearman’s r, and Likert Ratings

Table 23...... 137 Can component suggestions interquartile range between rounds two and three (efficacy)

Table 24...... 138 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS iv

Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy Should Component) Suggestions

Table 25...... 140 Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy Should Component) Suggestions with Spearman’s r, and Likert Ratings

Table 26...... 140 Should Component Suggestions Interquartile Range Between Rounds Two and Three (Efficacy)

Table 27...... 143 Suggestions that Were Significantly Positively Correlated for both the Can and Should Components (Efficacy)

Table 28...... 143 Suggestions that were Significantly Positively Correlated for both the Can and Should Components (Efficacy) with Likert Ratings

Table 29...... 144 Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Effectiveness Can Component) Suggestions

Table 30...... 145 Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Effectiveness Can Component) Suggestions with Spearman’s rs, and Likert Ratings

Table 31...... 145 Can Component Suggestion’s Interquartile Ranges Between Rounds Two and Three (Effectiveness)

Table 32...... 147 Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Effectiveness Should Component) Suggestions

Table 33...... 149 Suggestions that were Significantly Positively Correlated for both the Can and Should Components (Efficacy) with Likert Ratings

Table 34...... 150 Should Component Suggestion’s Interquartile Ranges Between Rounds Two and Three (Effectiveness) INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS v

Table 35...... 152 Stability Testing –Suggestions that were Significantly Positively Correlated for both the Can and Should Components (Effectiveness)

Table 36...... 153 Suggestions that were Significantly Positively Correlated for both the Can and Should Components (Efficacy) with Likert Ratings

Table 37...... 154 How, Can, and Should Components, and Spearman rs for Suggestion Six

LIST OF FIGURES

Figure 1. Likert rating scale for the can component ...... 86

Figure 2. Likert rating scale for the should component...... 86

Figure 3. Example of suggestion table from the report ...... 108

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS ii

A DELPHI STUDY REGARDING HOW, CAN, AND SHOULD INDIVIDUAL

PSYCHOLOGY DEMONSTRATE EFFICACY AND EFFECTIVENESS GIVEN

EVIDENCE BASED PRACTICE EVALUATION STANDARDS

ABSTRACT

The purpose of this research was to identify and reach consensus regarding how, can, and should individual psychology demonstrate efficacy and effectiveness given the current evidence based practice (EBP) evaluation standards. A review of the current literature on individual psychology and the pressure for the mental health field to adhere to EBP evaluation standards was presented. A Delphi study was conducted within three iterative rounds in order to reach consensus for how, can, and should individual psychology demonstrate efficacy and effectiveness. A panel of Adlerian Experts initially constructed a list of suggestions for efficacy (81) and a list of suggestions for effectiveness (54), and were asked to rank order and rate all suggestions based on the how, can, and should components for each round. Frequencies (percentages) and measures of central tendency (median and interquartile range) were computed for each suggestions rankings and ratings between rounds in order to identify suggestions that trended towards or reached consensus. After three rounds and testing for stability (Wilcoxon matched-pairs signed-rank test, Spearman correlation coefficient) the Adlerian Experts indicated several suggestions that had reached or trended towards stable consensus. From each of the suggestions that reached stable consensus, four common themes emerged (research design; operationalizing, standardizing, and manualizing; dissemination; and internal and external support) and are elaborated on in this study. Limitations, future research, and implications are identified.

STERLING P. TRAVIS

COUNSELOR EDUCATION & SUPERVISION

SCHOOL OF EDUCATION

THE COLLEGE OF WILLIAM & MARY INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFECTIVENESS iii

A DELPHI STUDY REGARDING HOW, CAN, AND SHOULD INDIVIDUAL

PSYCHOLOGY DEMONSTRATE EFFICACY AND EFFECTIVENESS GIVEN

EVIDENCE BASED PRACTICE EVALUATION STANDARDS INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 2

CHAPTER ONE

Evidence Based Practice (EBP) standards and guidelines were first introduced in the mental health field in 1949, at the annual conference of the American Psychological Association

(APA) in Boulder Colorado. The Boulder Model, as it would come to be known as, led to the creation of policy requiring clinicians and practitioner trainers to adopt a scientist-practitioner framework which emphasized the training of mental health professionals be focused on developing the clinical and research skills needed to utilize and promote EBP (Crane & Hafen,

2002; Generali, Foss-Kelly, & McNamara, 2011; Thomason, 2010). Since its introduction, many professional organizations, accrediting bodies, and reimbursement agencies have pressured the field of mental health to adopt and advance EBP guidelines and standards in the training of practitioners and the application of mental health services (Thomason, 2010). Thomason, also noted that EBP guidelines and evaluation standards have been debated in the field of mental health for being too reductionistic (i.e., symptom focused, manualized, etc.), and lacking an emphasis on effectiveness. Projective trends, professional organizations, accrediting bodies, and reimbursement agencies have challenged and pressured contemporary theories of psychotherapy to provide evidence of their effectiveness and efficacy in practice and to adapt to changing trends in order to remain viable (Norcross, Pfund, & Prochaska, 2013; Thomason, 2010). Sexton

(2001) noted that, “moving towards evidence-based counseling practice […] has been, and continues to be, a struggle within counseling” (p. 499). Norcross et al. (2013) projected that theories supported by EBP (e.g., mindfulness, cognitive, integrative, multicultural) will continue to increase in usage in the next ten years, while the use of others (e.g., Jungian, transactional analysis, psychoanalytic, and individual psychology) will decrease. This projection poses a INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 3

specific challenge for those contemporary theories of counseling not currently considered as

EBP.

In the face of challenges to the contemporary viability of individual psychology, the current study investigated the perceived importance and relevance of the model in the post- modern, EBP-dominated era. The study seeks to establish consensus among experts regarding the means by which practitioners and researchers can ensure Adlerian Psychology’s alignment with today’s EBP and Empirically Supported Treatment (EST) standards.

Chapter one outlines the problem to be studied, including the influence and impact of both individual psychology and EBP on the mental health field. Additionally, the first chapter explores the debate regarding the influence of EBP on the mental health field, and will provide a justification for the study. Chapter two will provide a review of current literature to provide an understanding of EBP principles and the current evaluation standards of EST’s, and it will highlight identified flaws in the EST evaluation standards. Chapter two will also illustrate the specific challenges faced by individual psychology and provide foundational support for efforts to explore how the model may demonstrate its continued relevance and validity. Chapter three will describe the methodology utilized in the study including sampling, data collection, instrumentation, and analysis. Chapter four will report the results of the study and provide interpretation of the consensus reached for each research question. Finally, chapter Five will discuss the implications of the results from this study, address limitations, and provide recommendations for future research.

General Statements on Individual Psychology

Psychologists, counselors, social workers, parent educators and other mental health professionals who ascribe to the Adlerian philosophy or individual psychology are commonly INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 4

referred to as Adlerians. Throughout this paper individual psychology will be used to refer to

Alfred Adler’s theory, and clinicians who ascribe to individual psychology will be referred to as

Adlerians. Alfred Adler presented individual psychology as a social psychology, “seeing people as being fully functioning units who somehow have to cope with living together on a planet”

(Adler, 1927, p. 3). Adler proposed that to enable a healthy individual to maintain holistic unity and actively belong and contribute to others in society, the individual develops and utilizes social interest (1927). Social interest allows an individual to develop an awareness of personal assets and abilities and to use empathy and awareness in order to understand and contribute to the lives of others (Adler, 1927). Individual psychology proposes that all behavior is purposeful and goal driven, with the basic human motivation being to strive to belong and develop social interest

(Adler, 1931). Individual psychology does not propose that an individual must be perfect; conversely, individual psychology values imperfections and feelings of inferiority as important components of an individual’s human experience (Sperry, 2014a). Sperry (2014a) noted that healthy individuals have the ability to develop socially useful goals to compensate for feelings of inferiority, while pathological individuals believe that imperfections must be overcome in order to strive to become more perfect. Individual psychology in practice relies heavily on an individual’s personal perceptions or private logic in order to develop and understanding of the underlying purpose of an individual’s behavior.

As an individual strives to belong and feel significant, subjective perceptions guide behavior (Dinkmeyer, Pew, & Dinkmeyer, 1979). Individual psychology in practice focuses on the subjective perceptions through the exploration of client’s life style. Dinkmeyer, Pew, and

Dinkmeyer (1979) noted that an individual’s life style is primarily developed through subjective experiences, and influences taken from the family growing up (e.g., family constellation, birth INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 5

order, familial atmosphere), early life experiences, and other social experiences. The life style, then, “refers to a person’s basic orientation towards life” (Dinkmeyer et al., 1979, p. 26).

Dinkmeyer et al. remarked that an individual’s life style provides an orientation to what is right and wrong (private logic), a life plan, and fictional goals developed out of childhood perceptions of what will keep the individual safe and allow the individual to strive to overcome feelings of inferiority and to belong. Further, Dinkmeyer and colleagues (1973) explained that an individual’s life style could be broken into four main groups:

(1) The self-concept - the convictions I have about who I am; (2) the self-ideal (this is

Adler’s term) - the convictions concerning what I should be or am obliged tone in order

to have a place in the world; (3) the Weltbild (picture of the world) - the convictions

about the non self (world, people, nature, and so on) and what the world demands of me;

and (4) the ethical convictions - the personal “right/wrong” code. (p. 31)

With an awareness of how an individual’s life style develops and guides the individual, Adlerians assist clients in exploring and understanding how their life style serves them positively; moreover, Adlerians assist clients in reorienting their current life style and behaviors towards more socially useful lifestyles and behaviors (Adler, 1927).

Traditional individual psychology in practice generally follows a four-phase pattern based on the work of Adler and developed by Rudolph Dreikurs (1967). The first phase is centered around the practitioner establishing a therapeutic relationship with the client; it is centered on egalitarianism. A strong egalitarian relationship is utilized to encourage clients to take responsibility over their own goals and therapeutic experience. The second phase focuses on the clinician and client gathering relevant information through assessment of the client’s life style.

The therapist uses a semi-structured interview process that allows the client to express presenting INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 6

initial concerns, while the therapist gathers relevant information through general life style assessment; exploring early-recollections and birth order and gathering information about familial relationships, family constellation, and dreams as needed (Oberst & Stewart, 2003). The practitioner may utilize various quantifiable measures to supplement the data gathered through the interview process but favors the information obtained from the interview (Oberst & Stewart,

2003). As the clinician and client enter the third phase, the focus becomes centered on assisting the client in gaining insight into fictional goals, mistaken ideals, and otherwise socially useless behaviors (Dreikurs, 1967). In this phase the client becomes aware of how previous experiences, feelings of inferiority, fictional goals, and mistaken ideals influence life style and guide behavior.

In the third phase the practitioner utilizes Adlerian therapeutic techniques of confrontation, paradox, and spitting in the client’s soup (Adlerian term) to promote insight regarding the current purpose of the client’s behavior or ways of living (Oberst & Stewart, 2003). The fourth and final is the reorientation phase, where the Adlerian practitioner and the client work towards reorienting the client’s life style in a way that promotes more socially useful behavior (Dreikurs, 1967). The practitioner assists the client in catching himself or herself when using previously faulty ways of thinking and in living as if he or she has made changes to a more socially useful way of being.

Ultimately individual psychology is considered a strength-based therapeutic approach, focusing on the client’s positive rather than negative attributes (Oberst & Stewart, 2003).

Individual psychology is considered a psychology of use, focused on the purpose behind client’s symptoms and behaviors rather than viewing clients as possessing behaviors and symptoms

(Sperry, 2014a). Many individual psychology concepts such as holism, purposefulness of behavior, private logic, and overall appreciation of relationships and social interest have permeated many modern psychological schools of thought (Adler & Deutsch, 1959). Watts and INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 7

Pietrzak (2000) noted that individual psychology has influenced and been associated with many modern and post-modern, therapeutic models. Watts and Pietrzak specifically acknowledged the extent that individual psychology has contributed to the development of existentialism, person- centered therapy, rational emotive behavior therapy, cognitive therapy, reality therapy, and many family systems approaches. Connections have also been made to the influence that individual psychology has had in the development of post-modern schools of psychology such as social constructivism (Watts & Phillips, 2004). Mosak and Maniacci (1999) went as far as to claim that the majority of psychotherapeutic interventions and practices widely used today can attribute their roots to individual psychology. Many of today’s introductions to psychological theories textbooks even remark that individual psychology may be the most influential psychological theory ever developed (Corey, 2005; Jones-Smith, 2012). Though the profound influence that individual psychology has had over modern and post-modern theories of psychology has been noted and celebrated, individual psychology’s relevance, efficacy, and effectiveness are frequently brought into question, as the field of counseling and psychotherapy faces the post- modern era of mental health centered on providing EBP (Norcross et al., 2013).

Evidence Based Practice

Definition of Evidence Based Practice and Empirically Supported Treatments

The APA Presidential Task Force on Evidence-Based Practice defined EBP as “the integration of the best available research with clinical expertise in the context of patient (or client) characteristics, culture, and preferences” (APA Task Force, 2006, p. 273). In order to do this, EBP intends to provide clinicians and researchers with a framework to bridge the gap between science and practice (Reynolds, 2000). This has changed the face of mental health in a post-modern era, as EBP places emphasis on “promoting effective psychological practice and INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 8

enhancing public health by applying empirically supported principles of psychological assessment, case formulation, therapeutic relationship, and intervention” (APA Task Force, 2006, p. 284). The emphasis on integrating empirically supported psychological practices has encouraged contemporary theories of psychotherapy to demonstrate efficacy and effectiveness through empirical support of interventions (APA Task Force, 2006). On the basis of intervention selection within the EBP process, Empirically Supported Treatments (EST) are considered interventions that have passed the evaluation standards set by EBP and have demonstrated efficacy (Chambless & Ollendick, 2001). ESTs are identified “treatments with at least two randomized controlled clinical trials that have demonstrated their efficacy” (Thomason, 2010, p.

30). Though EBP and EST are related, it is important to clarify that ESTs are seated within EBP practice as a whole (APA Task Force, 2006). EBP emphasizes the overall process of constructing a clinical question related to a client’s presenting problems and collecting and analyzing all available research to inform clinicians as they make clinical decisions regarding interventions to be used. ESTs are one piece of the over-arching EBP process, and are interventions that have been evaluated for their empirical evidence and deemed as EST suitable to treat specific diagnosis. However, EBP moves past EST to require clinicians also to explore the evidence base regarding other “clinical activities (e.g., psychological assessment, case correlation, therapy relationship)” (APA Task Force, 2006, p. 273).

History of EBP Movement

EBP within psychology, counseling, social work, and other mental health related fields over the past six decades stemmed from the diagnostic and treatment model utilized in the medical field (Thomason, 2010; Thorn, 2007). The medical model: INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 9

…views the source of the problem as an atomistic, concrete pathogen […] located within

the “patient,” which is diagnosed using objective, quantifiable evidence from the

patient...and treated with replicable, manualizable interventions […] In our medicalized

psychology, “diagnosis” is based on objectively measurable client behaviors, and

“treatment” consists of applying replicable protocols to change target behaviors [....]

(Hunsberger, 2007, p. 614).

The medical model and the EBP movement first became influential and relevant to the mental health field at the 1949 annual conference of the American Psychological Association in Boulder

Colorado (Thomason, 2010). For the next several decades the Boulder Model continued to take shape and influence the mental health field. In 1995, the APA Division 12 Task Force on

Promotion and Dissemination of Psychological Procedures identified 18 Empirically Supported

Treatments (EST) using randomized controlled clinical experiments with specific populations, where manualized treatment was efficient in working with specific mental health disorders (APA

Task Force, 2006). The criteria that were used to evaluate the 18 EST continued to become refined until 2002a, when the APA identified two overarching dimensions of evaluation: efficacy and clinical utility (effectiveness). The APA task force required that the dimension of efficacy be an, “evaluation of the strength of evidence pertaining to establishing causal relationships between interventions and disorders under treatment” (APA Task Force, 2006, p. 272).

Specifically, the dimension of efficacy strives to evaluate the strength of the evidence supporting a specific EST effect on the disorder or symptom being treated. The effectiveness dimension evaluates the “availability of research evidence and clinical consensus regarding the generalizability, feasibility (including patient acceptability), and costs and benefits of interventions” (APA Task Force, 2006, p. 272). Ultimately, clinical utility evaluates the INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 10

feasibility and cost effective effectiveness of the EST when generalized to non-experimental samples.

As the EBP movement continued to grow, the development of EBP evaluation dimensions influenced various health care policies, insurance companies, and major funding agencies to view EBP as a necessary component of psychological practice (Thomason, 2010).

The appointment of the APA Presidential Task Force on Evidence Based Practice by the APA

President Ronald F. Levant in 2005 (APA Task Force, 2006) set out to address concerns that the

EBP movement would influence policy makers, funding organizations, insurance companies, and training programs to dictate treatment choices and restrict clients access to care. The task force was charged to influence the continued integration of science and practice while advocating for policy makers, funding organizations, insurance companies, and training programs to appropriately consider all evidence when making policies and decision regarding EBP. Further, the Presidential Task Force of 2005 was tasked with the “further development and refinement of evidence-based practice for the betterment of psychological aspects of health care as it is delivered around the world” (APA Task Force, 2006, p. 273).

Goal of EBP Movement

From its inception, the Boulder Model challenged the mental health field to emphasize and utilize the scientist-practitioner model. EBP provides clinicians with a framework to

“systematically and objectively” integrate research into clinical practice (Reynolds, 2000, p.

258). The overarching goal of EBP has been to increase accountability of practitioners, ensure the highest quality of care for clients; ensure cost-effectiveness of treatment; and advocate for policy makers, funding agencies, and insurance companies to consider all evidence in making policies related to mental health (APA Task Force, 2006). Further, EBP urges clinicians to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 11

become informed researchers that demonstrate and provide effective and empirically supported interventions (Crane & Hafen, 2002; Generali et al., 2011; Levant & Hasan, 2008; Norcross,

Hedges, & Prochaska, 2002; Thomason, 2010). Though the intention of EBP is ultimately considered beneficial, it has had a polarizing effect on much of the mental health field (APA Task

Force, 2006; Reynolds, 2000; Thomason 2010).

Debate Around EBP

Clinicians and researchers have debated and resisted the EBP movement (Wampold et al.,

2007). Chambless and Ollendick (2001) indicate that many mental health professionals believe that EBP takes a reductionistic stance by focusing too heavily on a client’s presenting symptoms, rather than appreciating the complexity of the whole individual presenting for treatment. The medical model’s influence on EBP to standardizing treatment and objectively measuring symptoms and outcomes raises the concern of clinicians that the EBP model is too reductionistic

(Thomason, 2010). Wampold and others (2007) further argued that EBP is too objective and systematic, disregarding the importance of subjectivity that psychotherapy is foundationally tied to. Silverman’s (2006) critical review of the APA Task Force guidelines found that the guidelines oversimplified the therapeutic process, bringing into question the evaluation methodology used in approving interventions as empirically supported. Others shared Silverman’s concerns, contending that the methods employed in evaluating EST and guiding EBP were limited in their ability to be generalized outside of controlled trials and that they oversimplified client needs and the overall therapeutic process (Kazdin, 2008; Reynolds, 2000; Wampold et al., 2007). The evidence supporting EST is also questioned due to the difficulty to measure the effect of therapeutic relationship, therapeutic alliance, client expectations, and other common therapeutic factors (Lilienfeld et al., 2013; Lilienfeld et al., 2014; Messer, 2004; Wampold & Bhati, 2004). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 12

Pressure to move Towards EBP

Despite resistance from some clinicians, many professional organizations within the mental health field have adopted the scientist-practitioner model, encouraging clinicians to conduct research and implement EBP. The American Psychological Association (APA), National

Association of Social Workers (NASW) and the American Counseling Association (ACA) emphasize the use of EBP in the process of ethical decision-making and clinical best practices.

The Ethical Principles of Psychologists, published by the APA require psychologists to provide services that are grounded in “established scientific and professional knowledge of the discipline” (2002b, p. 6). Similarly, the NASW code of ethics (2008) states: “social workers should base practice on recognized knowledge, including empirically based knowledge”

(Standard 4.01[c]). The ACA (2014) ethical code also requires that "Counselors use techniques/procedures/modalities that are grounded in theory and/or have an empirical or scientific foundation" (p. 11). Burker and Kazukauskas (2010) added that there is also a call for

EBP to be incorporated into rehabilitation counselor practice due to changes made in the Code of

Professional Ethics in Rehabilitation Counseling.

Accreditation standards of training programs in Psychology, Social Work, and Counseling have also been influenced by EBP. The APA Evidence-Based Practice Standards for

Accreditation (2000) as well as the Division 12 Task Force on Promotion and Dissemination of

Psychological Procedures report (1995) recommended that EST training should be required of all students in order for training programs to receive APA accreditation. The Educational Policy and

Accreditation Standards written by the Council on Social Work Education (2008), require curricula to explicitly emphasize “social workers use practice experience to inform research; employ evidence based interventions; evaluate their own practice; and use research findings to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 13

improve practice, policy, and social service delivery” (p. 5). The Council for Accreditation of

Counseling and Related Educational Programs (CACREP) emphasizes the ethical obligation of counselor educators to encourage counselors in training to utilize EBP and to become more capable consumers of research (Kaplan & Martz, 2014). In order to provide students with adequate training, counselor educator training programs are challenged to set standards to ensure instructor’s competence in performing and instructing students to use EBP (Martino, 2010). As professional organizations and ethical and accreditation standards require EBP integration, it is evident that the EBP movement is far reaching (Thomason, 2010).

EBP has also had a significant impact on the healthcare system, related to policy and program construction. Managed care facilities, insurance companies, and many local, state, and federal funding agencies have challenged clinicians to adopt EBP standards and guidelines by restricting reimbursement for services to be exclusively for clinicians who utilize EST’s (Crane

& Hafen, 2002; Levant & Hasan, 2008). Thomason (2010) predicted that EBP is winning and will continue to win the debate over psychotherapy and talk-therapy models in relation to third party reimbursement. The pressures continue to push clinicians to utilize psychological treatment that can be grounded in science, as this is the only way to ensure reimbursement for services. Thomason (2010) and other researchers emphasized that trends in research and policy demonstrate that third-party reimbursement and liability insurance will be restricted only to clinicians utilizing EBP in the future (Generali et al., 2011; Norcross et al., 2002; Thomason,

2010). Thomason (2010) predicted that clinicians who desire not to utilize EBPs will be forced to work more in the private practice sector where they can independently bill their clients.

Clinicians also are being pressured to utilize treatment modalities that can fit into briefer more cost-effective forms of treatment (Thomason, 2010). The pressure to provide briefer, more INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 14

cost efficient practice has led managed care agencies and health service policy makers to encourage briefer models of therapy (Reynolds, 2000; Thomason, 2010). Cummings and

O’donahue (2008) have charged clinicians to meet the call for medicalized treatment through striving to diagnosis and begin treatment within the first 15 minutes of meeting a client. Though this stance is extreme and plays into the polarizing debate that revolves around EBP, the field consistently demonstrates trends towards manualized treatment, symptom-focused treatment and other EBT interventions (Addis et al., 1999; Norcross et al., 2002; Norcross et al., 2013).

Norcross et al.’s (2002) projective Delphi study of psychotherapy in the year 2010 indicated that the mental health field would be prominently influenced by EBP. The most recent

Delphi study conducted by Norcross et al. (2013) projected that as the field of mental health moves towards the year 2022, EBP will continue to dominate clinical practice. The researchers reported that a panel of 70 mental health experts predicted that EBP and briefer models of psychotherapy would significantly influence the mental health field into the year 2022. They also concluded that theoretical orientations that best fit into the evaluation standards of EBP will surge and become the most commonly used theoretical models in clinical practice. Although

Norcross and other’s projections are supported by pressure from professional organizations, accrediting bodies, and healthcare systems many clinicians maintain reservations (Crane &

Hafen, 2002; Generali et al., 2011; Levant & Hasan, 2008; Martino, 2010; Norcross & Wampold,

2011; Norcross et al., 2002; Thomason, 2010).

Struggle for Contemporary Theories to Adapt to EBP

Projective trends, professional organizations, accrediting bodies, and the health care system have challenged contemporary theories of psychotherapy to provide evidence of their effectiveness and efficacy in practice or adapt in order to remain viable. Specific to the struggle INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 15

of individual psychology, in the 1999 Ansbacher Memorial Address to the North American

Society of Adlerian Psychology (NASAP), Jon Carlson (2000) questioned if individual psychology would be able to adapt to the changing field of mental health, or if it would cease to exist. Sweeney (1998) and Watts (2013) both noted that a significant struggle of individual psychology (related to the pressures of EBP) is the failure of Adler to present a clear and systematic approach to his theory. Watts (2013) suggested that Adler’s writings on individual psychology’s constructs are vague and can be difficult to understand, thus limiting the ability to develop treatment manuals and measure the effectiveness of individual psychology. Given the vague definitions of its constructs and lack of a well-defined approach to clinical practice, individual psychology also struggles to construct outcome and process measures to empirically validate the theory (Jones-Smith, 2012; Sweeney, 1998; Watts, 2013). The EBP evaluation standards’ emphasis on evaluating an intervention’s efficacy and effectiveness based on outcome evidence of symptom reduction and/or behavior change is foundationally different from the underlying framework of individual psychology. Adlerians maintain that individuals should be treated holistically by focusing treatment on the underlying issues of the whole individual, rather than focusing exclusively on treating specific symptoms of a disorder (Sperry, 2014b). From this stance, individual psychology is foundationally not concerned with clinical diagnosis and the manualization of treatment, but conversely focuses on flexing clinical interventions to meet the needs of the individual that is present in counseling. Adlerians also limit the use of assessment and outcome measures, utilizing them in a way not to objectify or classify the client, but rather using assessment to inform treatment. The lack of controlled empirical research, treatment manuals, and outcome measurements limits the ability of researchers to conclusively comment INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 16

on the interventions used in individual psychology’s efficacy and effectiveness despite the substantial support for Adlerian constructs (Sommers-Flanagan & Sommers-Flanagan, 2011).

The continued struggle of individual psychology and other contemporary theories, to demonstrate contemporary validity, has left many clinicians and researchers with negative emotions regarding EBP (APA Task Force, 2006; Thomason, 2010; Reynolds, 2000). Reynolds

(2000) noted that some clinicians feel that EBP is a form of authoritarianism that requires clinicians to abandon previously accepted theories in order to standardize the field of mental health. Carlson (2000) also described how the trends in the current era of psychotherapy have led to theories of psychotherapy losing relevance as EST becomes more of the focus. This poses a specific and unique problem to the field of mental health, as much is lost as contemporary theories of counseling such as individual psychology continue to lose relevance.

Statement of the Problem

Richard Watts (personal communication, May 15, 2015) remarked that the influence of individual psychology on modern and post-modern theories and interventions of counseling can be understood as a river with many tributaries. Individual psychology can be viewed as the main river with other theories of counseling that have developed out of individual psychology (i.e., person-centered, logotherapy, cognitive therapy, cognitive-behavioral therapy, rational emotive behavioral therapy, Adlerian play therapy, reality therapy, family systems theory, solution- focused brief therapy) make up the many tributaries emerging out of the main river. Watts

(2015) noted that if the field of mental health were to lose individual psychology, then the river that provides the depth of understanding to many approved ESTs would be lost. Specifically, if the field of mental health were to lose individual psychology, then many approved interventions under EBP standards and evaluation guidelines would lose their footing. Without individual INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 17

psychology, individuals will be treated based primarily on symptom reduction rather than based on the underlying issues that an individual is experiencing from a holistic standpoint. The statement of the problem can be summed up as follows:

1. As the mental health field continues to move towards EBP standards and evaluation

guidelines, and as the utilization of ESTs continue to become the required methodology

of psychotherapy practice, clinicians will take a reductionistic stance to providing

treatment. Clinicians will be required to provide treatment based on the ability of an

intervention to have a specific effect on a symptom of a specific diagnosis. This stance

poses a specific problem for the mental health field, as clinicians will be forced to focus

treatment on symptom reduction rather than focusing treatment on working from a

holistic standpoint to assist clients in working through underlying problems (McWilliams,

2005; Shedler, 2010, 2011).

2. As individual psychology is not currently approved under EBP evaluation standards as an

EST, individual psychology struggles significantly to demonstrate its continued

relevance. Although the influence that individual psychology has had on the

development of many EST development has been noted, individual psychology continues

to be omitted from lists of EST, and not recognized for its influence on many ESTs

(Chambless & Ollendick, 2001). Further, if the field of mental health continues to

diminish the importance of individual psychology, the foundation of many ESTs will

suffer (Watts, 2015). The lack of recognition for the influence that individual psychology

has had on many developed ESTs poses a significant threat to the future of individual

psychology. This threat is further heightened if counseling training programs, pressured

to emphasize the training of EBPs, lose interest in training practitioners in individual INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 18

psychology thus continuing to reduce the overall usage and recognition of individual

psychology.

3. Individual psychology is projected to be among the least used psychological theories

within the next ten years as the pressure from professional organizations, accrediting

bodies, and reimbursement agencies continue to require the use of EBPs (Norcross et al.,

2013). The evaluation standards of EBP have been debated by Adlerian clinicians based

on its foundational differences to approval as an EST (e.g., requirements for treatment

centered around manualization, empirically controlled research studies, and emphasis on

outcome measures, etc.); however, the pressure for individual psychology to demonstrate

its efficacy and effectiveness remains. Current methods utilized to demonstrate the

continued relevance, efficacy, and effectiveness of individual psychology have included

qualitative research, process research, integrated approach to treatment, citing the

influence of individual psychology on approved EST, and debating the evaluation

standards of EBP. These methods though do not align with the current EBP evaluation

standards’ hierarchy of evidence (outlined in detail in Chapter Two) that emphasize that

efficacy and effectiveness be demonstrated via the use of RCT (APA, 2006). As many of

these methods only reflect clinician opinion, only demonstrate an interventions

effectiveness in a single case (case study), and do not follow a treatment manual, the

evidence supporting individual psychology does not sufficiently demonstrate efficacy or

effectiveness based on the EBP evaluation standards (APA, 2006). Further, although the

arguments made by Adlerians against the EBP evaluation standards align with the current

debate in the overall field, these arguments do not remove the pressure within the field of

mental health to utilize interventions that have demonstrated efficacy and effectiveness INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 19

within the current evaluation standards. Thus, individual psychology continues to be

faced with the challenge to demonstrate its effectiveness and efficacy in a post-modern

era centered on EBP.

Purpose of the Study

The purpose of the current study was to conduct a Delphi study to explore the opinion of experts in the field of individual psychology regarding the future of individual psychology in a post-modern era of mental health driven by EBP. The results of the study will offer a consensual opinion of Adlerian Experts to inform Adlerian clinicians and researchers regarding how, can, and should individual psychology address the pressure of EBP. Inasmuch as approval as an EBP focuses on the evaluation of an interventions efficacy and effectiveness, the Delphi study sought a consensual opinion regarding how individual psychology may demonstrate efficacy and effectiveness and if individual psychology would benefit from striving to do so. Specifically, the purpose of the study was to develop consensual opinions related to the following questions:

1. How, can, and should individual psychology demonstrate efficacy given the current EBP

evaluation standards?

2. How, can, and should individual psychology demonstrate effectiveness given the current

EBP evaluation standards?

The how, can, and should components of each of the two research questions in this study were utilized to establish useful (how component), feasible (can component), and beneficial (should component) suggestions regarding how individual psychology may demonstrate efficacy and effectiveness.

The two research questions, though vernacularly similar, are uniquely different based on the definition of efficacy and effectiveness provided by the APA Task Force (2006). The INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 20

efficacy research question relates to the APA EBP evaluation standards requiring an evaluation of “the strength of evidence pertaining to establishing causal relationships between interventions and disorders under treatment” (APA Task Force, 2006, p. 272). The effectiveness research question pertains to the APA EBP evaluation criteria standards requiring an evaluation of the

“available research evidence and clinical consensus regarding the generalizability, feasibility

(including patient acceptability), and costs and benefits of interventions” (APA Task Force,

2006, p. 272).

Sample Description and Data Gathering Procedures

In order to obtain a panel of experts in individual psychology, the researcher purposefully recruited clinicians and researchers who identified as Adlerian clinicians and researchers. Adler and Ziglio (1996) suggested that the expert participants meet four “expertise” requirements: (a) knowledge and experience with the issues under investigation; (b) capacity and willingness to participate; (c) sufficient time to take part in the Delphi; and, (d) effective communication skills.

The researcher additionally required that Panel Members have engaged in scholarly activity (i.e., publication, conference presentations, workshops, etc.) related to individual psychology, and/or have served in a leadership role within an Adlerian organization. Skulmoski, Hartman, and

Krahn (2007) suggested that for a homogenous group, an Expert Panel of ten to fifteen participants should be used. Thus, the researcher initially recruited 20 potential participants for the Expert Panel to ensure that a sufficient amount of participants may provide responses throughout the Delphi study and to protect against participant drop out. Ultimately, of the 20 participants recruited, 16 participants agreed to participate at the outset of this study, and 12

Panelists completed all three rounds of the study. Traditional Delphi methodology suggests that the first research questionnaire be developed using broad, open-ended, and intentional questions INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 21

to be presented in the first round, from which a qualitative free-response paragraph is generated

(Skulmoski et al., 2007). Once participants were chosen for the Expert Panel, the researcher presented the Panel with a questionnaire that asked them to respond to two open-ended forms of the developed research questions based on their personal opinion. The Expert Panel then responded to the two open-ended research questions in a qualitative, free-response, open-ended, and paragraph form. From the qualitative full paragraph responses provided by each Panelist, the researcher conducted qualitative content analysis to reduce the qualitative paragraphs into two compiled lists of all singular suggestions to the research questions. For example, if a Panel

Member responded to the question, “How may individual psychology demonstrate efficacy given the current EBP evaluation standards?” with the following paragraph:

“Adlerians could adapt their stance on clinical diagnosis. By adapting the stance on

clinical diagnosis, Adlerians would be able to develop a treatment manual for using

individual psychology to work with clients struggling with depression or other specific

clinical diagnosis. By developing a treatment manual individual psychology could also

subject itself to a wider range of outcome research…”

The researcher then would conduct classical content analysis of this statement and construct a list of each specific suggestion from the example (e.g., (a) adapt stance on clinical diagnosis; (b) construct treatment manual for specific clinical diagnosis [depression]; (c). conduct outcome research) regarding how individual psychology may demonstrate efficacy. After completing the classical content analysis process, the researcher would construct a comprehensive list of the compiled list of suggestions. Additionally, traditional Delphi methodology suggests that the research questions be adapted for the second round to be more specific and that they be adapted INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 22

to ask Panel Members to quantify the compiled list of responses from the first round by ranking and rating the responses (Powell, 2003).

Once the research questions were adapted and the list of all responses were compiled, the researcher distributed the second round questionnaire and compiled list of responses to the Panel for a second round, where Panel Members edited or expanded on the list once viewing other

Panel Member’s responses. Additionally, Panel Members were asked to rank order the top twenty-five suggestions in terms of the perceived utility of each suggestion to address the how component of each research question (1. How may individual psychology demonstrate efficacy given the current EBP evaluation standards; 2. How may individual psychology demonstrate effectiveness given the current EBP evaluation standards). Finally, the researcher instructed

Panel Members to rate each response using two seven-point Likert scales. The first asked

Panelists to rate each response based on the can component of each research question in terms of the perceived feasibility of each response to be implemented using a seven-point Likert scale ranging from definitely can not to definitely can. The second Likert scale asked Panelists to rate each response based on the should component of each research question in terms of the perceived benefit of each response using a seven-point Likert scale ranging from Absolutely no benefit to a great deal of benefit. After the second round of responses were collected, the researcher analyzed the findings using quantitative statistical analysis to determine measures of central tendency (median and interquartile range) of each participant rank order and rating of the responses. After the second round, the researcher checked the statistical analysis for consensus.

For the how component, consensus was determined using an 80% consensus cut-off point

(Ulschak, 1983), which required that a suggestion be rank-ordered in the top twenty-five by at least 80% of the Panel Members. For the can and should components consensus was determined INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 23

using the interquartile range of Panel Members ratings, which required that Panel Members ratings of suggestion reach an interquartile range between 0 and 2 in order to demonstrate consensus. On the basis of this analysis, the researcher was able to identify how suggestions were moving towards consensus. The researcher used this analysis to remove suggestions that were not rank ordered by at least 25% of the Expert Panel; however, all suggestions that were ranked by more than 25% of the Panel were included in the third round to provide the most inclusive list of suggestions. After suggestions were removed, the researcher provided the

Expert Panel with the statistical analysis of the second round, so that the Panelists could review how their fellow Panel Members had ranked and rated the suggestions. The researcher then revised the research questions and possible outcomes list for a third round and disseminated the third round questionnaire and compiled list of responses to the Expert Panel for the third round.

The third round was conducted in the same format as the second round, with the exception of two changes. First, for the how component, the researcher asked Panelists to rank order their top ten suggestions from both revised lists rather than their top twenty-five that was used in the second round. This was done to urge the Panel towards a stronger level of consensus.

Additionally, during data analysis the researcher conducted statistical analysis for stability in order to verify that Panel Member’s rankings and ratings were not fluctuating between rounds

(Scheibe, Skutsch, & Schofer, 1975). Von der Gracht (2012) remarked that Delphi studies should not stop collecting data or consider consensus achievement until the stability of responses has been determined stable. Stability testing was used to measure the consistency of Panel

Members ranking and ratings of suggestions between successive rounds. As the current study sought to have Panel Members rank and rate suggestion, Von der Gracht (2012) suggested that two specific nonparametric statistical analyses be conducted to measure stability. Von der INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 24

Gracht (2012) suggested using the Wilcoxon matched-pairs signed-ranks test to examine the

“paired data of the same group of individuals as in a before and after situation” between rounds to determine, “whether a difference between the data of two Delphi rounds has statistical significance thereby testing for stability of the data” (p. 1532). Additionally, Von der Gracht

(2012) suggested utilizing Spearman’s rank correlation coefficient to measure the level of agreement between subsequent rounds rating and ranking (on the how, can, and should components) among Panelists for each suggestion. Once stability was demonstrated, the researcher ceased data collection and identified and reported on the findings of the consensus reached for each research question based on the measures of central tendency analysis, stability testing, and the 80% consensus cut-off point.

Summary

This chapter presented the challenges that individual psychology faces related to the pressures of the EBP movement. Specifically, this chapter noted that individual psychology is challenged by EBP’s foundational difference to individual psychology’s treatment model, the lack of individual psychology’s recognition as an influential treatment model, and individual psychology’s inability to demonstrate its efficacy and effectiveness based on EBP’s evaluation standards. An overview of the critiques of the EBP evaluation guidelines, and individual psychology’s struggles to address the pressures of EBP were provided. Finally, the Delphi method was discussed and a summary of the methodology of this study was provided. The next chapter will provide a thorough review of the relevant literature of the current study. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 25

CHAPTER TWO

Review of Selected and Current Literature

Chapter two will provide a discussion of current research to provide the foundation for the study. This chapter will describe the current evaluation standards of EBP, and will illustrate the emphasis on demonstration of efficacy and effectiveness, treatment manualization, and outcome-based evidence. The chapter will also provide evidence that the current evaluation standards may be inefficient to evaluate the efficacy and effectiveness of individual psychology.

Chapter two will conclude by highlighting that, although the evaluation standards may be insufficient, there remains a need for individual psychology to address the pressure to demonstrate its efficacy and effectiveness.

EBP Evaluation Guidelines and Standards

As previously stated, EBP’s general purpose has been to provide a systematic, and objective decision-making model to bridge the gap between research and clinical practice (APA

Task Force, 2006; Reynolds, 2000). More Specifically, EBP provides a method that

“distinguishes between research that is of direct clinical significance, and that which is

not […] provides a set of simple rules for evaluating research evidence […] and provides

a framework for making clinical decisions on the basis of research findings and of

applying research findings to individual patients” (Reynolds, 2000, p. 259).

EBP encourages clinicians to critically evaluate and integrate the best available research into clinical decision making and practice while encouraging researchers to strive to produce research that is up to date, comprehensible, and easily disseminated (Reynolds, 2000). To assist in the complex task of evaluating and producing useful research that can be integrated into clinical practice, the EBP movement establishes standards and methodologies through which the best INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 26

available research on treatment programs can be evaluated (APA Task Force, 2006; Chambless &

Hollon, 1998; Cooper, 2005; Hall, 2008; Hunseley, 2007; Kazdin, 2008; Reynolds, 2000).

EBP General Evaluation Standards

Measurement is a foundational component of the EBP process that stresses the importance of the evaluation process (Reynolds, 2000, p. 260). The APA (2002a) Criteria for

Evaluating Treatment Guidelines considered treatment efficacy and treatment utility (or treatment effectiveness) to be the strongest criteria for evaluating the evidence supporting ESTs (in APA

Task Force, 2006). In assessing an intervention’s efficacy, research outcomes supporting a specific intervention are evaluated based on the intervention’s ability to have a measurable effect on the clinical question of concern and on its outcomes being free from bias (APA Task Force,

2006; Barkham & Mellor-Clark, 2003; Reynolds, 2000). Once efficacy has been evaluated, effectiveness is evaluated to determine if the evidence is generalizable and feasible and if it can be clinically applied by clinicians to have an effect on client symptoms within the treatment setting (APA Task Force, 2006; Barkham & Mellor-Clark, 2003; Chambless & Hollon, 1998;

Reynolds, 2000).

Treatment efficacy. The APA task force (2006) regarded specific research designs as superior to others in demonstrating interventions efficacy. A commonly utilized hierarchy of evidence presented by Chan and colleagues (2010) noted that the highest level of research evidence is established through systematic review of evidence compiled from multiple randomized controlled trials. The Criteria for Evaluating Treatment Guidelines (APA, 2002a), identified that research evidence based on “sophisticated empirical methodologies, including quasi-experiments and randomized controlled experiments or logical equivalents” (p.1054) provides the most accurate evaluation of efficacy, compared to clinical opinion, and clinical INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 27

observations (Reynolds, 2000). The hierarchy presented by Chan and colleagues (2010) agreed with the APA guidelines in labeling authoritative opinion as the lowest level of evidence, and labeling non-experimental studies and non-randomized experiments as being inferior to evidence collected from randomized controlled trials. Randomized Controlled Trials (RCT), are considered the best research methodology to demonstrate treatment efficacy (APA Task Force,

2006; Barkham & Mellor-Clark, 2003; Chambless & Hollon, 1998; Reynolds, 2000). RCTs involve random assignment of clients into either a specific manualized treatment group or to a contrast group, wherein outcomes are measured to demonstrate if an intervention had an effect on specific symptoms (Barkham & Mellor-Clark, 2003; Chambless & Hollon, 1998; Clement,

2013). Through randomization and controlling treatment through treatment manuals, the internal validity of RCT can be assured (Emmelkamp et al., 2014). Chambless and Hollon (1998) stressed that, although the overall methodology of RCT research is accepted as the best method of demonstrating efficacy, an intervention is not deemed efficacious just for using a RCT design.

They suggest that the overall quality of evidence is increased through examination of a study’s description of the sample (specifically clinical diagnosis) as well as how appropriate outcome measures were selected and implemented to measure changes. They also suggest that the implementation process of the intervention as well as the research outcomes should be evaluated to assess if the reported outcomes are truly related to the specific intervention (Chambless &

Hollon, 1998). In order to better demonstrate a specific treatment’s efficacy in more than one clinical sample, Reynolds (2000) agreed with the APA guidelines that meta-analytical research compiling the research outcomes of several RCT studies is preferred over single RCT research.

Meta-analysis is believed to provide more conclusive support for a specific intervention’s efficacy by providing researchers a quantitative approach to reviewing the literature of RCT INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 28

outcomes and presenting evidence of consistent treatment efficacy when the treatment is replicated with varying populations and settings (Clement, 2013; Reynolds, 2000).

Efficacy standards and research guidelines set by the APA task force are frequently debated. The largest body of literature exploring the limitations of efficacy research is related to concerns about the ability of RCT outcome data to be generalized and demonstrate effectiveness in a non-experimental environment (Chambless & Ollendick, 2001). In Chambless and

Ollendick’s (2001) article on identified ESTs and the controversy around the EBP evaluation process, effectiveness and generalizability are noted as the strongest arguments in opposition of

EST. The internal validity that is significantly increased with RCTs does not guarantee that external validity (generalizability) and the overall effectiveness of the intervention will be ensured when the intervention is implemented in the clinical population (Emmelkamp et al.,

2014). Baez (2005), Cooper (2005), and Prendergast (2011) all addressed the limitations of RCT findings to be generalized to non-experimental populations and question if efficacy research truly can transfer into effective clinical practice.

Treatment effectiveness. Concerned with the limitations of RCT findings to be generalized and effective outside of experimental controlled settings, the APA Task Force (2006) emphasized that there is a specific need to evaluate the overall treatment effectiveness of interventions found to be efficacious when making the decision to implement an intervention in clinical settings (Chambless & Hollon, 1998). Evaluation of an intervention’s treatment effectiveness is considered complimentary to efficacy research, focusing on the generalizability, feasibility, and cost-effectiveness of the treatment (APA Task Force, 2006). Though RCT research is the primary method of demonstrating efficacy with high levels of internal validity, it is noted that controlled trials are not representative of real world experiences and do not replicate INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 29

the uncontrolled environment of clinical practice (APA Task Force, 2006; Barkham & Mellor-

Clark, 2003; Chambless & Hollon, 1998; Clement, 2013; Emmelkamp et al., 2014). Therefore,

EBP recommends that clinicians evaluate the generalizability and effectiveness of the outcome results across varying populations, clinicians, settings, and any interaction between the three

(APA Task Force, 2006; Chambless & Hollon, 1998). Additionally, in order to integrate research into practice, EBP suggests that clinicians evaluate if a specific intervention can be easily disseminated and if the client will accept the intervention and comply with treatment goals (APA

Task Force, 2006; Chambless & Hollon, 1998). Lastly, clinicians must evaluate if the benefits of utilizing the specific treatment outweigh any costs, both financially and psychologically to the client (APA Task Force, 2006; Chambless & Hollon, 1998).

As previously noted, Chambless and Ollendick (2001) suggested that more research is needed regarding the effectiveness of EST after efficacy has been demonstrated. Although, ESTs utilized with a variety of clients with panic disorders, depression, and oppositional-defiant disorders have demonstrated certain levels of effectiveness in varying populations and settings,

Chambless and Ollendick remarked that there is a dearth of research regarding outcome effectiveness. The studies that have demonstrated the level of effectiveness of specific EST with specific clinical diagnoses in clinical settings is limited, and overall effectiveness research is not viewed as important as efficacy research when evaluating an intervention’s empirical support for inclusion in a list of ESTs. Effectiveness research is considered complementary to efficacy research, and EST evaluation guidelines place more emphasis on evaluating the efficacy of an intervention. Emmelkamp and colleagues (2014) suggested that effectiveness research should be an ongoing process, stating: “Validity testing is never complete, but should be an integral and explicit part of a continuous research program - more than is current practice in experimental INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 30

psychopathology research” (p. 64). Ultimately Emmelkamp and colleagues (2014) suggested that the evaluation of the effectiveness of an intervention falls on the shoulders of researchers and clinicians who are implementing ESTs. Suggestions are also made that the hierarchy of evidence that places RCT and meta-analysis at a superior level to other forms of research (e.g., qualitative research, process research, case studies, single-subject experiments, clinical opinion, etc.) for evaluating the efficacy of an intervention may not be as strong as other research designs to demonstrate effectiveness in practice (Barkham & Mellor-Clark, 2003). There are many that argue that qualitative methods of research and other measures that fall lower on the hierarchy of evidence should be given more appreciation than is currently given for their ability to demonstrate effectiveness (Kazdin, 2008; Shallcross, 2012; Wampold, 2003).

EST Evaluation Process

Under pressure from managed care facilities and other reimbursement agencies, attempts have been made to standardize the EST evaluation process, so as to comprehensively inform mental health clinicians, clients, and health organizations of the practices in mental health that have been approved as ESTs (Prendergast, 2011). Chambless and Ollendick (2001) summarized the standards and criteria that are commonly used in identifying EST. In order to receive a distinction as a well-established EST, at least two RCT are needed, demonstrating efficacy compared to medication or a previously established treatment. Chambless and Ollendick (2001) also found that the use of treatment manuals defining characteristics of the sample (clinical diagnosis) and effects being demonstrated by separate researchers all contribute to the approval of an intervention as empirically supported. Chambless and Ollendick also reviewed the work of other researchers to identify EST and identified similar criteria for evaluating the evidence to support EST’s (2001). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 31

The Substance Abuse and Mental Health Services Administration (SAMHSA), a U.S.

Department of Health and Human Services agency granting EST status, has developed a standardized method of evaluating interventions (Tilsen & McNamee, 2015). SAMHSA maintains the National Registry of Evidence-based Programs and Practices (NREPP) to inform clinicians, clients, and health organization of the currently approved ESTs (Kazdin, 2008). In order to be considered for inclusion in the NREPP, treatment programs must meet the minimum requirements and submit an application for review. The NREPP website currently lists that in order to be considered for review, an intervention must have fully developed treatment materials and manuals as well as demonstrated and published findings from an RCT or quasi-experimental study where the intervention positively affected behavioral change (SAMSHA, 2015). Once an intervention has met the minimum requirements and its application has been accepted for review, the treatment program is evaluated based on the quality of research, and on the treatment program’s readiness for dissemination (Prendergast, 2011). The quality of research review evaluates the reported results of the specific intervention focusing on the reliability and validity of the measures used, appropriateness of data analysis, fidelity, and freedom in the results from bias or extraneous variables (SAMHSA, 2015). Reviewers then evaluate whether the intervention is ready for dissemination based on the treatment manuals and other materials needed, the training that is necessary to implement the treatment, and the potential for quality assurance or potential for effectiveness research (SAMHSA, 2015). The SAMHSA guidelines do not specifically evaluate an intervention’s current effectiveness research, but rather they evaluate if there is potential for effectiveness research to be conducted. If the evaluators feel that the intervention has satisfied the evaluation standards, the intervention is added to the NREPP. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 32

Currently, the NREPP website lists 424 interventions that have been submitted and have met all evaluation standards (SAMSHA, 2017).

Many critics of the lists of EST argue that the criteria that evaluate EST focus more on the quality of research than on the treatment effectiveness of the research (Tanenbaum, 2005).

Critics argue that the criteria used focus more on the ability of the researcher to conduct an adequate experiment than on evaluating if the evidence supports a specific intervention’s ability to produce behavioral change (Wampold, 2003). In reviewing the criteria identified by

Chambless and Ollendick (2001) and used by the NREPP (SAMSHA, 2015), there is a profound emphasis on quality of research over the effectiveness of the intervention being studied.

Chambless and Ollendick (2001) suggested that the decision-making model to grant an intervention EST status is loosely defined and poses a threat of evaluator bias. As previously mentioned, although RCTs are regarded as having the highest level of evidence of an intervention’s efficacy, the omission of effectiveness studies in the EST criteria raises a concern of whether ESTs will be efficacious and effective in clinical settings (Emmelkamp et al., 2014).

Focus on manualized treatment. Several components of EST evaluation criteria, and of

EBP in general are considered necessary to the evaluation process when evaluating the evidence supporting an intervention (Chambless & Ollendick, 2001). EST evaluation standards and guidelines specifically emphasize that interventions should be manualized, and that outcome research should be conducted (APA, 2002a; SAMSHA, 2015). In Chambless and Ollendick’s

(2001) review of the evidence and controversy around EST, they remarked that the APA task force concluded that:

Treatment manuals […] are necessary to provide an operational definition of the

intervention under study […] Failing such a description, clinicians are unable to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 33

determine the similarity of their own practices to ESTs, and educators are ignorant of

how they might train their students in ESTs. That is, in terms of generalizable

knowledge, it is meaningless to say that a treatment works without being able to say what

that treatment is. (p. 701)

Chambless and Ollendick also remarked that evaluation standards require interventions to utilize treatment manuals in order to be considered as an EST. This requirement comes from the ability of treatment manuals to increase internal validity of EST research (Addis & Krasnow, 2000).

Chambless and Hollon (1998) stressed that research where treatment manuals were not developed or utilized is of limited importance in establishing an intervention’s clinical utility and treatment efficacy.

Addis, Wade, and Hatgis (1999) stressed that treatment manualization has posed a significant challenge to many clinicians and has polarized the field. The focus of the debate has primarily focused on comparing manualized treatments to flexible treatment but has produced mixed results as to which treatment is superior (Addis et al.,1999; Chambless & Ollendick, 2001;

Stiles, Shapiro, & Elliot, 1986). The question is commonly raised as to whether manualized treatments that have been deemed ESTs will maintain their effectiveness when implemented, or whether reliability and validity of the EST will be reduced (Silverman, 1996). Proponents of treatment manualization argue that the potential benefits of treatment manualization are that they ensure internal validity (Addis & Krasnow, 2000) and increase the fidelity of EST (Moncher &

Prinz, 1991). The lack of research to adequately demonstrate if flexible treatment is equivalent or superior to manualized treatment has prompted managed care and reimbursement agencies to favor the evidence supporting the efficacy of ESTs that use manualized treatment (Chambless &

Ollendick, 2001). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 34

Focus on outcome research. Outcome research is a hallmark of the EBP movement and evaluation process (APA Task Force, 2006). Being foundationally built on measurement, EBP relies heavily on outcome research in the evaluation process (Margison et al., 2000). EBP’s emphasis on outcome measures intends to ensure that clients receive the highest level of care by identifying which interventions have been and continue to be proven with specific client symptoms (Reed & Eisden, 2005). As previously discussed, intervention efficacy evaluations focus on the evaluation of outcome data collected from RCT demonstrating a behavioral change or symptom reduction (APA Task Force, 2006; SAMSHA, 2015). EBP additionally relies on outcome measures, and clinician expertise to evaluate the evidence regarding the clinical utility of an intervention once applied to clinical settings (non-experimental) (Huppert, Fabbro, &

Barlow, 2006).

Outcome measures are a staple of the EBP movement and have become a widely accepted format of proven intervention efficacy in the public’s eye (Casey, 2012). Reed and

Eisden (2005) and Resnick (2005) suggested that the emphasis on outcome measures must be expanded beyond symptom reduction and evidence of behavior change provided from RCT, and they must provide evidence to demonstrate the effectiveness of interventions within clinical settings. Several researchers have also suggested that there is a need for more routine outcome measurement to examine the effectiveness of interventions once implemented in non- experimental settings (Halstead, Leach, & Rust, 2007; Lillienfeld et al., 2014; Margison et al.,

2000, Unsworth, Cowie, & Green, 2012). The debate continues around whether or not: (a) the

EST standards of evaluating theories and interventions should expand their criteria to better suit interventions that do not fit well with RCT methodologies, and (b) they should pay more attention to research findings of process and qualitative research and other research designs INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 35

lower on the currently used hierarchy of evidence so that interventions that adequately demonstrate efficacy and effectiveness through these research methodologies may be considered for inclusion as EST (Hubbert et al., 2005; Messer, 2004).

Summary of EBP Evaluation Standards, Guidelines, and Process

The current method of identifying theoretical orientations and interventions that are effective and efficacious is through the evaluation of research evidence demonstrating efficacy

(primarily) and treatment effectiveness of an intervention in effecting symptom or behavior change (APA Task Force, 2006). Attempts have been made to standardize the process of evaluating the evidence supporting specific interventions’ efficacy and identifying ESTs

(Chambless & Ollendick, 2001). A hierarchy of evidence emphasizes that RCTs and meta- analysis are the preferred methods of researching and demonstrating intervention efficacy over qualitative and process research (APA Task Force, 2006; Chambless & Ollendick, 2001).

Besides considering RCT evidence as the highest level of evidence, the current evaluation standards emphasize that treatment manualization (Addis et al., 1999) and outcome research

(Resnick, 2005) based on a reduction of symptoms as necessary to demonstrate the efficacy and effectiveness of an intervention (Chambless & Ollendick, 2001). Concerns are frequently raised related to the limitations of RCT research and the EST evaluation criteria for an intervention to be deemed an EST. Moreover, concerns are raised regarding the lack of evaluation of treatment effectiveness when evaluating an intervention’s evidence for inclusion as an EST. Primary concerns regarding the ability of RCT findings to be generalized and demonstrate effectiveness in non-experimental settings fuel the debate around EST relevance (Barkham & Mellor-Clark,

2003; Kazdin, 2008; Silverman, 1996; Stiles et al., 1986). Recommendations for development and appreciation of outcome studies that demonstrate the overall effectiveness of EST in clinical INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 36

settings are needed (Emmelkamp et al., 2014; Resnick, 2005; Silverman, 1996). Additionally, the criteria used to evaluate interventions has been challenged to broaden its appreciation for evidence that supports those interventions’ effectiveness that fall lower on the hierarchy of evidence (Kazdin, 2008; Shallcross, 2012; Wampold, 2003).

Limitations of the EST Evaluation Method

The evaluation standards that are used in identifying ESTs need to be subjected to evaluation of their overall accuracy in evaluating interventions (Fongy, Roth, & Higgitt, 2005).

Fongy and colleagues (2005) suggested that evaluation standards and methods should be empirically tested based on “specificity (the likelihood of falsely identifying a treatment as effective) and sensitivity (the change of misclassifying an effective treatment as ineffective)” (p.

2). It has been proposed that the current approach to evaluating efficacy and effectiveness outlined by the APA Task Force standards and guidelines and implemented by EST evaluators may be insufficient to assess the efficacy and effectiveness of individual psychology accurately, and it may also be inaccurate in viewing individual psychology as lacking a sufficient base of evidence of its clinical efficacy and effectiveness. The current criteria utilized in evaluating the efficacy and effectiveness of interventions are frequently debated and resisted based on limited emphasis on evaluating interventions’ effectiveness outside of controlled trials, and for oversimplifying client needs and the overall therapeutic process (Kazdin, 2008; Wampold et al.,

2007). These criticisms raise question as to whether the EBP evaluation standards defining ESTs are a sound measure of which interventions are effective, or if the evaluation standards merely identify interventions that can specifically demonstrate efficacy through RCT’s (Messer, 2004).

EBP and the evaluation process of establishing EST’s have instigated a major debate in the field of mental health (Thomason, 2010). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 37

Debate Around EBP and EST

Primarily, the debate has centered on the hierarchy of evidence used in evaluating interventions. The quantitative nature and use of outcome results in RCT are debated due to the lack of evidence demonstrating effectiveness when in a non-experimental setting (Kazdin, 2008;

Wampold, Goodheart, & Levant, 2007). This limitation has raised questions regarding why evaluation standards have yet to value research designs that could better demonstrate the generalizability of interventions (Hubbert et al., 2005; Messer, 2004). Kazdin (2006) has argued that qualitative research, single-case experimental design, and case studies provide a valuable amount of evidence regarding the clinical effectiveness of an intervention. Although RCT outcomes can demonstrate the clinically significant effect that an intervention has on symptom reduction (objectively), other research designs better demonstrate if the client actually experienced change (subjectively) (Kazdin, 2008). By focusing on symptom reduction, critics argue that RCT takes a reductionist stance (Kazdin, 2006; Kazdin, 2008). In a RCT, a researcher matches a specific intervention to a specific symptom or diagnosis and measures the effect the intervention has on reducing the symptoms (Emmelkamp et al., 2014). Emmelkamp and colleagues suggests that the reduction of a client to symptoms is empirically weak due to the disregard for underlying psychological mechanisms that may be causing the client’s symptoms.

Hall (2008) also questioned the generalizability of research on the effect of a specific intervention to reduce specific symptoms due to the high level of cases that present to counseling with co-morbid diagnoses. Watchel (2010) added to the debate contending that by focusing on matching singular interventions to a singular diagnosis, clients who present to counseling that do not fit the criteria for a clinical diagnosis are excluded. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 38

On the other side of the reductionist debate, critics argue that EBP’s emphasis on using treatment manuals minimize the therapeutic process and pose a significant threat to ESTs fidelity and generalizability. Treatment manuals are critiqued for attempting to standardize treatment and for threatening clinician’s ability to flex their interventions to meet the unique needs of clients (Reed & Eisman, 2005). Once the use of an EST is determined, there is no guarantee that the treatment manual will be followed explicitly, and clinicians may adapt the intervention to the specific client being treated (Prendergast, 2011; Watchel, 2010). Moncher and Prinz (1991) remarked that although the validity of an EST is increased when treatment manuals are used, treatment manuals are insufficient in guaranteeing interventions’ fidelity given the need for clinicians to adapt treatment. Critics frequently make the argument that counseling is a fluid process based on mutual interaction between a unique client and a unique counselor in a unique setting that cannot be standardized (Corsini & Wedding, 2005; Watchel, 2010).

Limitation of Evaluation Standards and Guidelines: General

The debate around EBP evaluation methods and standards is not exclusively related to

RCT. Emmelkamp and colleagues (2014) noted that although the current evaluation process focuses on evaluating interventions efficacy, little effort has been made to evaluate the empirical support for the theory from which interventions are based. Further, many researchers have noted that current evaluation standards may not be suited to evaluate interventions from specific theoretical backgrounds (Chambless & Ollendick, 2001). As the EBP movement has progressed, there has been a movement from treating whole individuals to focusing on symptom reduction

(Emmelkamp et al., 2014). The focus of EBP on evaluating interventions on the basis of their direct effect on symptom reduction is fundamentally different from many psychodynamic and humanistic interventions that seek to treat the whole individual (Chambless & Ollendick, 2001). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 39

Tanenbaum (2005) argued that evaluation standards’ adherence to a treatment manual also undermines psychodynamic and humanistic theorys’ preference to allow theory to guide interventions selection and implementation. Tanenbaum (2005) further argued this point by identifying that psychodynamic and humanistic approaches are more focused on treating underlying issues rather than focusing on treatment manuals emphasizing symptom reduction.

Thus, critics have argued that evaluation standards requiring interventions demonstrate effectiveness through RCT are not an appropriate measure of psychodynamic and humanistic interventions (Chambless & Ollendick, 2001; Fonagy et al., 2005; Tanenbaum, 2005).

Tanenbaum (2005) argued that the exclusion of process and qualitative research evidence in evaluating interventions’ effectiveness demonstrates a preference of the evidence hierarchy over evidence of effectiveness. Several researchers have noted that until EBP standards are expanded to include qualitative and process research, many interventions will be wrongly excluded from lists of EST (Fonagy et al., 2005; Huppert et al., 2006; Tanenbaum, 2005, Wachtel, 2010).

Messer (2004) pointed out that evidence to support the efficacy of some therapy models does not invalidate other theories. Interventions that have not been subjected to RCT are not reviewed within the NREPP process (Emmelkamp et al., 2014), so they commonly are not regarded as EST. Watchel (2010) raised concern with lists of EST such as the NREPP, citing that even though it is not out rightly stated, treatments that are not included on the NREPP are considered ineffective by consumers. Hundsley (2007) questioned if current EST standards are too stringent in excluding therapy models that could effectively address clients’ presenting problems but they lack the preferred and accepted research design (RCT). Watchel (2010) shared in this concern, stating: INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 40

The therapies being investigated may not be manualized, or the study may not be

restricted to a narrow segment of patients, and hence, even when the studies are

rigorously conducted RCTs, they are ignored by “EST” advocates. But they provide

robust evidence for the effectiveness of a far broader set of therapeutic approaches than

“EST” advocates acknowledge. (p. 263)

Messer (2004) argued that a lack of RCT evidence cannot invalidate interventions tied to firm theoretical support that have been practiced extensively over time such as psychodynamic, humanistic, and family systems theories.

Another critique that is commonly cited in the debate regarding EBP evaluation standards is the emphasis on outcome criteria as opposed to other common therapeutic factors that have been identified (Norcross & Lambert, 2011). Within the common factors approach, client or extratherapeutic factors account for 40% of the disparity in client outcomes, whereas 30% of the disparity comes from the therapeutic relationship, 15% is attributed to hope/expectancy/placebo factors, and the last 15% is accounted for by the therapist’s model/technique. Messer (2004) noted that compared to the specific interventions that are approved as EST, “process therapies tend to emphasize competence, skill, and personal qualities of individual therapists and their impact on therapeutic alliance more so than specific techniques or ingredients… characteristic of

ESTs” (p. 582). Messer (2004) also noted that as the medical model that influences EST practice “seeks a therapist who use techniques with demonstrable abilities to alleviate conditions,” clinicians who use the common factors approach are “interpersonally competent therapists who use a treatment approaches compatible with the client’s world view […] as expressed in a theoretical orientation” (p. 582 - 583). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 41

Limitation of Applying Evaluation Standards and Guidelines to Individual Psychology

Carlson, Watts, and Maniacci (2006) proposed that there are distinct similarities between individual psychology’s foundational principles and the common factors approach. They noted that Adlerian clinicians pay attention to the client’s extratherapeutic factors through attending to all aspects of what a client brings with them to counseling and through a desire to understand the client holistically. Further, Adlerian clinicians put significant effort into developing strong therapeutic relationships; individual psychology’s principles dictate that developing a strong therapeutic relationship is the most pivotal component of therapy. Moreover, being a therapeutic model seated in encouragement and strength focused interventions, Adlerian clinicians help clients build hope and expectancy. Lastly, although some would critique individual psychology for having a less standardized therapeutic model and techniques, individual psychology allows the counselor to adapt and integrate treatment interventions to meet the needs of the client. The foundational similarities that Carlson and others (2006) drew between individual psychology and the common factors approach, validates many of the principles of individual psychology.

Individual psychology’s emphasis on developing an awareness and appreciate for the client’s worldview or life style and its emphasis on establishing a strong therapeutic relationship aligns the treatment Adlerian clinicians provide more closely with the common factors approach in comparison to the reductionistic stance of many ESTs (Messer, 2004). The alignment of individual psychology and the common factors model also demonstrates that although individual psychology independently is not accepted as an EST, it is aligned with a model that validates individual psychology’s approach to instilling change.

Although individual psychology can argue that its alignment with the common factors approach supports its continued relevance and validity, there are still foundational differences INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 42

between individual psychology and the EBP evaluation standards that hinder its ability to become accepted as an EST (Evans & Meredith, 1991). EBP evaluation standards’ emphasis on evaluating interventions based on quantitative outcome research is in opposition to individual psychology’s research framework that has historically been seated in qualitative and process research (Carlson et al., 2006; Mosak & Maniacci, 1999; Shelley, 2000). Some researchers have proposed that individual psychology would fare better in EST evaluation if such evaluation included case studies, single-case experiments, qualitative, and process research (Kazdin, 2008).

For example, in a multi-case research study conducted by Marshall and Fitch (2006), Adlerian constructs were utilized in informing the selection of appropriate interventions in working with clients with purging behaviors. The interventions were not selected on the basis of symptomatic behavior, but, rather, based on treating the clients holistically based on the identified individual psychology constructs. Each case study presented demonstrated that the purging behavior stemmed from a deeper seated issue. The case studies presented also provided suggestions for specific Adlerian interventions to meet the needs of the clients. A case-study presented by

Carlson and Robey (2011) demonstrated individual psychology interventions and case conceptualization that are appropriate for working with families with multiple problems. The case study describes specific interventions that were utilized and demonstrates the intervention’s effectiveness based on how the family responded to the interventions. Pfefferle and Mansager

(2014) presented several case studies to demonstrate the effectiveness of utilizing the Adlerian

Family Diagnostic Process intake form. Interventions effectiveness was demonstrated through case examples of how the family responds to the interventions, and how the counselor was able to adapt the intervention to continue to meet the family’s needs. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 43

Attempts by Adlerian clinicians and researchers to develop treatment manuals have been difficult and limited (Mosak & Maniacci, 1999). The exclusionary stance taken in EBP evaluation standards and EST criteria limit the ability of individual psychology to be demonstrated in RCTs (Tanenbaum, 2005). Carlson and others (2006) suggested that the development of a treatment manual would assist Adlerian clinicians and researchers in researching the Adlerian therapeutic process, but at the current time there is no treatment manual to assess. Individual psychology is foundationally opposed to treatment manualization due to the emphasis on adapting treatment interventions to meet the needs of clients, and it avoids using assessments that operationalize a client through formal diagnosis (Mosak & Maniacci, 1999;

Sweeney, 1998). Adlerians view each client as a unique, whole, person and adapt treatment to meet the client’s unique needs, whereas EBPs focus on reductionistic diagnosis and symptom reduction that are directly in conflict with that aspect of the Adlerian view (Evans & Meredith,

1991). Further, Adlerians take a different approach to the utilization of assessments (Kalkan,

2009; Peluso et al., 2010; Shelley, 2000, Sweeney, Myers, & Stephen, 2006). While RCT outcomes can demonstrate the clinically significant effect that an intervention has on symptom reduction (objectively; Kazdin, 2006), Adlerian clinicians utilize semi-structured interviews and other assessment instruments to gain a better understanding of the client’s subjective experience and to inform intervention selection throughout the therapeutic process (Shelley, 2000, Sweeney et al., 2006).

The fundamental differences between Adlerian counseling and the EBP evaluation standards and guidelines limit the ability of EBP to evaluate the effectiveness of individual psychology accurately. Individual psychology has abandoned the medical model and made efforts to oppose reductionism, causality, and pathology in an effort to value the Adlerian INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 44

fundamental principles of holism, purposefulness of behavior, and social interest (Watts &

Pietzak, 2000; Evans & Meredith, 1991) In doing so, individual psychology has limited representation on lists of EST (Chambless & Ollendick, 2001; SAMSHA, 2015). Because the

EBP evaluation standards and guidelines have been insufficient in accurately evaluating Adlerian psychotherapy’s validity, Adlerian supporters have been faced with questions regarding the continued validity and relevance of Adlerian Psychotherapy.

Argument for Individual Psychology’s Continued Relevance

The primary method of arguing the continued relevance of individual psychology has been to draw connections to other theories and interventions that have been influenced by individual psychology principles. Adlerian’s propose that many of the textbooks commonly used in counselor training programs acknowledge the significant contributions that individual psychology had in the development of other major psychological theories (Jones-Smith, 2012).

In Jones-Smith’s (2012) textbook on counseling theories, Adler’s specific contributions to family counseling were highlighted. Specifically, Jones-Smith acknowledged Adler's influence in the development of parent education. Parent education programs such as the Systematic Training for

Effective Parenting (STEP; Dinkmeyer et al., 1979) and active parenting (Popkin, 1993) were developed out of the principles of individual psychology. Individual psychology has also been praised for its contributions to the development of play therapy (Bratton, Ray, Rhine, & Jones,

2005; Carlson et al., 2006;). Carlson and others (2006) noted that Albert Ellis and Aaron Beck both acknowledged the influence that individual psychology principles had on the development of rational emotive behavior therapy and cognitive therapy. Adler’s principles of societal influence have also been attached to Viktor Frankl’s theory of logotherapy (Frankl, 2014;

O’connell, 1972; Pytell, 2000). Moreover, many of Adler’s original concepts are reflected in the INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 45

humanistic school of thought (Ansbacher, 1990; Watts, 1996; Watts, 1998) and in social constructionism (Watts & Phillips, 2004). Watts (2000a) pointed out that the integrative nature of Adlerian psychotherapy emphasizes how individual psychology can be integrated within the brief-solution focused theoretical orientations as well. With the absence of a fixed treatment model, individual psychology has the freedom to integrate the appropriate interventions and practice technical eclecticism to meet client’s needs (Carlson et al., 2006). The connections made between individual psychology and the transtheoretical common factors approach in the previous section reflect the integration potential of individual psychology, and also show the potential of individual psychology to demonstrate its continued relevance in a postmodern era; however, limitations are still present (Carlson et al., 2006).

Sweeney (1998) cautioned that, although evidence has been supported in the literature regarding the influence individual psychology has had on other theoretical models, individual psychology still is not “fully realized” (p.27). Sweeney (1998) and Watts (2013) both found that

Adler's inability to operationally define the constructs of individual psychology significantly limits the opportunity for research to validate individual psychology’s constructs and therapeutic process. Given this limitation, research has attempted to explore the constructs of individual psychology through developing measures of certain constructs and through conducting qualitative research and case studies (Watkins & Guarnaccia, 1999). The Adlerian constructs of social interest, birth order, and early recollections have been the focal point of the much of the research relating to individual psychology (Sweeney, 1998). Several social interest scales have been developed to measure an individual’s social interest. The Social Interest Index (SII)

(Greever, Tseng, & Friedland, 1973) is commonly referred to as one of the most utilized social interest scales (Leak 2006), although its authors identified limitations in the SII due to weak INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 46

factor structure and threats to content validity. Leak (2006) developed an 11-item short form of the SII that demonstrated comparable validity to the original SII while reducing the assessments length and increasing content validity by removing weak items. Kalkan (2009) made revisions to the original SII as well, intending to investigate specific aspects of social interest’s effect on romantic relationships. Kalkan’s measure also demonstrated adequate psychometric qualities.

Leak (2006) and Kalkin’s (2009) research assisted in further exploring the construct of social interest generally, as well as in specific context (Kalkan, 2009; Leak, 2006). Advancements have also been made in research around Adlerian life style, with the development of measures such as the Basic Adlerian Scales for Interpersonal Success-Adult Form (BASIS-A) Inventory (Wheeler,

Kern, & Curlette, 1993). The BASIS-A has a demonstrated internal consistency reliability ranging from .82 to .87 and has test-retest reliability shown at the moderate level (Wheeler,

1996). Prior to the development of the BASIS-A, life style assessment was based on a semi- structured qualitative interview, which was difficult to research (Peluso et al., 2010; Wheeler,

1996). The BASIS-A provided researchers a way to more definitively define life style and conduct comparison group research in the hopes of broadening the acceptance of individual psychology (Peluso et al., 2010). Oberst and Stewart (2003) have praised the BASIS-A for its validation in comparison to other measures such as the Myers-Briggs Types Indicator and for its ability to use life style to guide treatment. Several attempts have been made to manualized and guide the process of early recollection content gathering procedures and analysis (Kern,

Belangee, & Eckstein, 2004; Powers & Griffith, 1987; Shulman & Mosak, 1988; Wingett, 2015).

Though attempts have been made to standardize and guide the investigation and analysis of early recollections, Adlerians have not settled on a singular standardized protocol, and they have not INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 47

produced research to demonstrate the efficacy and effectiveness of early recollection models in a format other than case studies.

Although the integrative and influential nature of individual psychology has been noted

(Carlson et al., 2006; Watts, 2000b; Watts, 2004), and efforts have been made to operationally define Adler’s constructs in empirically measurable ways (Kalkin, 2009; Leak, 2006; Wheeler et al., 1993), there remain significant challenges ahead (Carlson, 2000; Hartshorne, 1991).

Summary, Trends, and Pressure

The evaluation standards of EBP are currently viewed as insufficient in their ability to evaluate individual psychology’s effectiveness; however, the field is still expected to move towards EBP, and ESTs (Norcross et al., 2013; Thomason, 2010). Individual psychology is among the theoretical orientations that are projected to diminish in the coming years (Norcross et al., 2013). Given these predictions, the insufficiency of EBP evaluation standards to accurately assess the effectiveness of individual psychology poses a significant challenge. Adlerian researchers have made strides in operationally defining constructs and using empirically supported measures to assist in the therapeutic process, but little has been done to fully demonstrate individual psychology’s efficacy and effectiveness through development of treatment manuals and conducting well-designed outcome research (Carlson, 2000; Hartshorne,

1991; Sperry, 1991). If evaluation standards are unsuitable for measuring the efficacy and effectiveness of individual psychology, individual psychology will need to explore more respected ways of demonstrating efficacy and effectiveness (Carlson, 2000; Sperry, 1991;

Tanenbaum, 2005).

In the future, Adlerian clinicians will struggle to be reimbursed and receive liability insurance, as third-party payers and health policy makers continue to require that clinicians INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 48

utilize ESTs (Thomason, 2010). As reimbursement agencies continue to move towards restricting payment-by-results, the lack of Adlerian outcome research and a constructed treatment manual will put it at a disadvantage (Casey, 2012, Thomason, 2010). The argument that individual psychology gains validation from ESTs that have roots connected to individual psychology is, although well intentioned, not comparable to well-designed outcome research needed for EBP approval (Carlson, 2000; Sperry, 1991). The cited limitations of the evaluation standards of EBP do not diminish the responsibility of Adlerian researchers to conduct research that demonstrates the efficacy and effectiveness of individual psychology (Sperry, 1991).

Likewise, the lack of outcome research and treatment manualization in individual psychology and the limitations of EBPs evaluation standards to sufficiently demonstrate individual psychology’s effectiveness and efficacy do not halt the influence EBP has on the mental health field (Norcross et al., 2013; Thomason, 2010). As this poses a significant issue for Adlerian

Theory, further examination is needed to explore the ability, benefit, and means in which individual psychology may demonstrate its efficacy, effectiveness, and continued relevance in a post-modern era of counseling.

Need for Examination of Individual Psychology’s Efficacy and Effectiveness

In 2000, a special issue of the Journal of Individual Psychology published the transcript from Jon Carlson’s Ansbacher Lecture (previously described) as well as a selection of articles pertaining to the future of Adlerian counseling in the new millennium. Carlson questioned whether individual psychology would be able to adapt to the changing field of mental health or if it would cease to exist. Dreikurs-Ferguson (2000) questioned whether Alfred Adler was ahead of his time, charging that contemporary Adlerians must continue efforts to demonstrate the continued relevance of individual psychology. Dreikurs-Ferguson’s article argued that many of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 49

the core principles of individual psychology such as social interest and holism are extremely relevant to the problems that contemporary clients are presenting to counseling with. Dreikurs-

Ferguson claimed that individual psychology is useful, but it may be misunderstood due to the current trends in the field of mental health. Watts (2000b) advanced that dialogue, discussing the natural integrative nature of individual psychology. Watts claimed that the influence that individual psychology has had on certain EST provides an opportunity for Adlerian clinicians to integrate those empirically supported interventions into Adlerian counseling. Watts (2000b) and

Dreikurs-Ferguson (2000) have contended however that the historical influence that individual psychology has had on the theory and practice of psychotherapy historically may go overlooked.

They claim that clinicians and researchers should not abandon the theory, but, rather, they should creatively find ways to inform the mental health field of the continued relevance of individual psychology related to current problems. Watts (2000a) proposed that individual psychology could address the EBP’s challenge for interventions to demonstrate efficacy and effectiveness through the integrative and flexible nature of the theory, suggesting that individual psychology could continue to explore intervention integration and utilize appropriate outcome measures to demonstrate efficacy and effectiveness.

In an earlier interview, Jon Carlson advocated for individual psychology to become more integrative and voiced concern that some Adlerian clinicians and researchers are “more concerned about being Adlerian than effective” (Nystul, 1991, p. 502). Conversely, in the same special issue of the Journal of Individual Psychology (mentioned previously), Shelley (2000) analyzed and critiqued the dependence on quantitative observations emphasized in the EBP, arguing that Adlerian counseling should resist EBP’s overemphasis on quantitative research and should continue to advocate for a more qualitative emphasis. Shelley proposed that in order to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 50

meet the challenges of EBP, Adlerian clinicians and researchers should not adapt to the quantitative nature of EBP, but, rather, should develop a stronger research base of Adlerian- qualitative research. In earlier publications, Sperry (1991) and Carlson (Nystul, 1991) contended that Adlerians do not need to abandon qualitative research in preference of other forms of research, but instead advocated for Adlerian research to focus on systematically demonstrating individual psychology’s efficacy and effectiveness in a variety of settings. Sweeney and others

(2006) presented a method for integrating developmental counseling and therapy assessment with individual psychology to increase clinician’s ability to match interventions to specific client’s developmental needs. Sweeney and others found that through using developmental assessments, Adlerian clinicians could more effectively support developmental change.

Debate and Suggested Adaptations and Integration

As individual psychology has been challenged in the face of EBP to demonstrate its efficacy and effectiveness and continued relevance, there has been a polarizing debate over how

Adlerians should meet this challenge. On one side of the debate, there are those who share the sentiment of Dreikurs-Ferguson (2000) and Shelley’s (2000) that individual psychology should resist the pressure to adapt to meet the needs of EBP, and that it should do a better job of informing the public of the established evidence supporting the effectiveness of individual psychology. This side of the debate suggests that Adlerian researchers continue to conduct qualitative research due to its alignment with Adlerian principles valuing the subjective experience of the client. Further, qualitative research is supportive from this side of the debate given its emphasis on the therapeutic process of change as opposed to viewing change as a cause and effect relationship of symptom reduction favored by EBP. One of the primary suggested adaptations for EBP is to adapt the hierarchy of evidence currently used in evaluating ESTs to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 51

better appreciate qualitative and process research (Shelley, 2000). Carlson and others (2006) proposed that if qualitative, process, and case-based research were acknowledged for their contributions to EBP, individual psychology and other contemporary theories would benefit.

Hundsley (2007) also proposed that if adaptations were made to EBP evaluation standards hierarchy of evidence, fewer interventions would be excluded from lists of EST.

Conversely, there are those on the other side of the debate that view the challenges posed by EBP as an opportunity to investigate the integrative and adaptive nature of individual psychology while systematically exploring the effectiveness of individual psychology in a variety of settings. Sperry (1991) suggested that individual psychology must adapt and encourage systematic well-designed research in order to further demonstrate the validity, efficacy, and effectiveness of Adlerian psychotherapy. Similarly, Carlson and others (2006) suggested that Adlerian clinicians and researchers might benefit from developing treatment manuals to examine the overall efficacy and effectiveness of the therapeutic process of individual psychology. Watts (2000b) advocated that in order to meet the challenges of EBP, Adlerian psychotherapist should utilize the integrative nature of individual psychology to conduct research demonstrating the efficacy and effectiveness of the theory. Watts and Pietzak (2000) have provided evidence of the specific similarities between Adlerian encouragement and solution- focused brief therapy. Sweeney and others (2006) have also demonstrated the significant similarities and benefits of integrating developmental psychotherapy assessments within the

Adlerian psychotherapeutic framework. Frequently, Adlerian proponents have noted the influence that individual psychology has had on many of today’s major theoretical frameworks, and Watts (2000a) argued that Adlerian clinicians should not be afraid of integrating individual psychology with techniques from other related therapeutic models. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 52

Summary

This chapter presented literature that examined and critiqued the EBP movement and development of evaluation standards while highlighting the emphasis that those evaluation standards place on interventions demonstrating efficacy and effectiveness (APA, 2002a; APA

Task Force, 2006; Barkham & Mellor-Clark, 2003; Chambless & Hollon, 1998; Chambless &

Ollendick, 2001; Clement, 2013; Emmelkamp et al., 2014; Kazdin, 2008; Thomason, 2010;

Wampold et al., 2007). The chapter also indicated that the EBP evaluation standards are limited in their ability to evaluate the efficacy and effectiveness of individual psychology (Carlson et al.,

2006; Messer, 2004; Mosak & Maniacci, 1999; Shelley, 2000; Sweeney et al., 2006), which has lead to a polarizing debate within the field of individual psychology regarding how individual psychology can and should address the pressures of EBP (Carlson et al., 2006; Dreikurs-

Ferguson, 2000; Shelley, 2000; Sperry, 1991; Watts, 2000b; Sweeney et al., 2006; Watts &

Pietzak, 2000). Given this current debate within individual psychology related to the challenges posed by EBP as well as the current trend emphasizing movement towards EBP (Prochaska et al.,

2013), consensus needs to be reached regarding how Adlerian psychotherapy can meet today’s call for EBP in counseling treatment. The literature reviewed in this chapter supports the need for the current Delphi study to be conducted to examine how, can, and should individual psychology demonstrate efficacy and effectiveness given current EBP evaluation standards. The methodology used to conduct the Delphi study is outlined in the next chapter. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 53

CHAPTER THREE

Methodology

This chapter outlines the research methodology for the current study. The information describes the research design, the history and development of the Delphi method, panel selection, and the data collection and analysis by round. Limitations and ethical considerations regarding the study are also discussed.

For this study, the Delphi method was used to discover how Adlerian experts suggest that

Adlerian clinicians and researchers may demonstrate the efficacy and effectiveness of individual psychology given the current evaluation standards of EBP. The overriding research goal was to develop a consensual opinion from experts as to how, can, and should the efficacy and effectiveness of individual psychology be demonstrated given the current EBP evaluation standards. An Expert Panel was asked to respond to three rounds of questionnaires focused on identifying specific suggestions for how the research goal can be achieved. The specific research questions of the study were as follows:

1. How, can, and should individual psychology demonstrate efficacy given the current EBP

evaluation standards?

2. How, can, and should individual psychology demonstrate effectiveness given the current

EBP evaluation standards?

The Delphi Method

Given the debate among Adlerian clinicians and researchers and the lack of unified direction regarding how individual psychology may approach the pressures of EBP to demonstrate efficacy and effectiveness, a Delphi study was considered to be appropriate to identify a consensual opinion on the questions of how, can, and should individual psychology INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 54

demonstrate effectiveness and efficacy. The professional literature currently does not provide such an opinion, and a Delphi study could provide an opportunity for such opinion to be developed. Individual psychology is commonly critiqued for its limited research base, and a

Delphi study could lead to consensus regarding what opportunities there are to address these issues. Through the Delphi technique, an Expert Panel was drawn from a vast geographical area, thus permitting a rich and comprehensive exploration of the topic.

Development of the Delphi Method

The RAND CORPORATION first utilized the Delphi method in the early 1950’s for forecasting purposes, but the Delphi method has continued to be developed and utilized in many scientific and technological studies (Hasson, Keeney, & McKenna, 2000). The premise behind the Delphi method comes from Greek mythology, where in the Greek town of Delphi, Apollo would consult with the oracle Pythia to predict the future. The Delphi method of today does not predict the future, rely on one expert’s opinion, or reduce discussion over topics; rather the

Delphi method has developed into an appropriate method when a study seeks to achieve the following objectives:

1. To determine or develop a range of possible program alternatives;

2. To explore or expose underlying assumptions or information leading to different

judgments;

3. To seek out information, which may generate a consensus on the part of the respondent

group;

4. To correlate informed judgments on a topic spanning a wide range of disciplines, and;

5. To educate the respondent group as to the diverse and interrelated aspects of the topic

(Delbecq, Van de Ven, and Gustafson, 1975, p. 11). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 55

The Delphi method has been found to provide a reliable and generalizable method suitable for gathering multiple subjective judgments into a singular consensual opinion when dealing with complex problems (Day & Bobeva, 2005). Day and Bobeva (2005) have argued that although the Delphi method has been used since the 1950’s, it is still in its early stages of development.

Further, many researchers have noted specific strengths and weakness of the Delphi approach and stress that the Delphi method must be carried out carefully and precisely (Day & Bobeva,

2005; Hsu & Sandford, 2007; Powell, 2003).

Strength and Weakness Considerations of the Delphi Method

The Delphi method has provided opportunity for researchers in many ways. The most profound strength of the Delphi method is it’s ability to reach consensus regarding an area of uncertainty (Delbecq et al., 1975; Dawson & Barker, 1995; Murphy et al., 1998). Powell (2003) noted that in addition to reaching consensus, the Delphi method’s emphasis on the provision of feedback between individual rounds offers the opportunity to widen the knowledge base and stimulate new ideas in a motivating and educational format. The Delphi process of reaching consensus is also strengthened by having the study be conducted in an anonymous format within each round, thus limiting the potential for opinions to be influenced by dominating group dynamics (Helmer, 1983). Okoli and Pawlowski (2004) also noted the ability of the Delphi method to provide generalizable results and “identify variables of interest and generate propositions” given the participants’ level of expertise and wide range of experiences (p. 27).

Day and Bobeva (2005) have remarked that because the Delphi method is still in its developmental stages, there are some weakness and limitations that should be considered when planning and implementing such a study. The most frequently noted weakness is the extensive time commitment and effort that a Delphi study requires of the researcher and participants (Hsu INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 56

& Sanford, 2007; Powell, 2003). Hsu and Sanford (2007) specifically discussed how the time commitment and effort needed to conduct a Delphi study may lead to low response rate and potential Panel Members dropping out of the study. Further, some have questioned whether the consensus that is developed within a Delphi study actually provides a specific opinion regarding the area of uncertainty or if the opinion becomes watered down and too broad (Hsu & Sanford,

2007; Murphy et al., 1998; Powell, 2003; Sackman, 1975). Sackman (1975) remarked that the anonymous nature of the Delphi method reduces the accountability of members. The Delphi method also is vulnerable to the researcher molding the opinion of the Panel through the presentation and wording of questions as well as through the feedback provided (Hsu & Sanford,

2007). Hasson and others (2000) noted that the reliability and validity of the findings of a Delphi study could be questioned, given that there is no evidence to substantiate that if the same Delphi study was conducted with two separate Expert Panels that the consensual opinion would be the same. Lastly, Delphi research runs the risk of unintentionally or purposefully over-structuring the inquiry process, thus potentially limiting participant input (Linstone & Turoff, 1975) if the views of individual participants are overlooked in the quest for consensus (Blow & Sprenkle,

2001; Linstone & Turoff, 1975).

Given these limitations, specific considerations are given that can assist researchers in conducting Delphi research. Day and Bobeva (2005) developed a decision-making model to assist researchers in conducting Delphi research in a way that addresses its limitations. It is suggested that specific consideration be given to the preparation of the Delphi study. As the

Delphi method is based on the “aggregation of opinion,” consideration must be given to the selection of the Delphi approach to ensure that the answer to the problem being investigated would not be more “easily and fully acquired through direct measurement, experiment, or INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 57

simulation” (Day & Bobeva, 2005, p. 108). If the Delphi approach is deemed appropriate to the problem being investigated, the selection (and management) of the Expert Panel and instrument development must be rigorously planned and implemented. As noted previously, Adler and

Ziglio (1996) suggest that experts possess the following minimum qualifications: (a) knowledge and experience with the issues under investigation; (b) capacity and willingness to participate;

(c) sufficient time to participate in the Delphi; and, (d) effective communication skills.

Skulmoski and colleagues (2007) and Day and Bobeva (2005) noted that consideration must also be given to the number of experts making up the Panel. Skulmoski et al. (2007) suggested that when using a homogeneous sample, ten to fifteen experts is sufficient if the results are to be generalized to a similar homogeneous population. The researcher must also consider the timing of the study (i.e., of rounds and timing between rounds) (Day & Bobeva, 2005) as well as the mode of interaction (i.e., paper pencil, telephone, email, online survey, etc.) that will be utilized

(Skulmoski et al., 2007). As the mode of interaction is determined, Day and Bobeva (2005) have warned that instrumentation development is critical to the exploration and analysis stages of the

Delphi methodology. In addition, questions should be constructed within each round moving from broad to more specific (Skulmoski et al., 2007). Day and Bobeva (2005) have suggested that the researcher should also consider how communication between the researcher and the

Expert Panel will occur, as well as how the documentation and analysis of results will be handled prior to conducting the Delphi study. Lastly, Skulmoski and others (2007) have remarked that researchers should consider how further verification of findings from the Delphi research will be conducted in order to address its generalizability limitations. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 58

Panel Selection

Selection and Recruitment of Participants

According to Okoli & Pawlowski (2004): “A Delphi study does not depend on a statistical sample that attempts to be representative of any population. It is a group decision mechanism requiring qualified experts who have a deep understanding of the issues” (p. 20).

The crux of the Delphi method is based on consensus of opinion among members of an expert panel, and thus, the selection process is a critical component of the Delphi method (Skulmoski et al., 2007). Specific attention must be given to the criteria utilized in selecting experts for the panel as well as the number of panel members (Day & Bobeva, 2005; Hasson et al., 2000; Hsu &

Sanford, 2007; Okoli & Pawlowski, 2004; Powell, 2003; Skulmoski et al., 2007). Further, the combined expertise and experiences of the expert panel are only useful if the participants are willing and able to make contributions (Okoli & Pawlowski, 2004; Powell, 2003).

For the current Delphi study, the criteria suggested by Adler and Ziglio (1996) (discussed in the previous section) were used as a minimum requirement. The researcher additionally required that Panel Members have engaged in scholarly activity (e.g., publication, conference presentation, workshops, etc.) related to individual psychology and/or have served in a leadership role within an Adlerian organization. These participants were chosen based on the assumption that their scholarly activity and/or leadership roles qualify them as experts on individual psychology. Participants were recruited through the membership and leadership of the North American Society of Adlerian Psychology. The researcher initially recruited 20 potential participants from the NASAP membership for the Expert Panel Ultimately, of the 20 participants recruited, 16 participants completed the first round of the study, 14 Panel Members completed the second round, and 12 Panelists completed all three rounds of the study. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 59

Data Collection

The data collection procedure of traditional Delphi research follows a specific methodology applied in several rounds. The Delphi method generally provides participants the opportunity to express their opinions about a problem individually and anonymously, to receive feedback about their expressed opinions, to observe other opinions regarding the problem, to have opportunities to revise their own views, and to rank and/or rate all opinions based on specific criteria (Fish & Busby, 1996). This process is usually carried out in two to ten rounds, but, as discussed previously, data is often considered saturated and stable after three rounds

(Powell, 2003). The first round of data collection in traditional Delphi research is conducted by posing broad, open-ended, questions to the selected Expert Panel who then respond to the questions in a free-response paragraph form. The researcher pays close attention to the wording of questions, directions, type of questions, the number of questions, and the time needed to respond so as to not influence the panel members’ responses (Day & Bobeva, 2005). The data collected and analyzed from the initial round of data collection is utilized in the development of questions for subsequent rounds (Okoli & Polowski, 2004). Identical responses from the initial round are consolidated, and classical content analysis is utilized in identifying major themes and components within all responses in an effort to develop more specific and relevant questions for subsequent rounds. The second and subsequent rounds provide the Panel with increasingly refined and specific lists of Panelist responses. The Panelists are then asked: (a) to rank order all

Panelist responses in order of their perceived utility in addressing the research question and (b) to rate each response’s perceived individual utility in addressing the research question (Powell,

2003). The data collection process stops when a predetermined percentage of consensus among INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 60

the participants is reached in both: (a) panel members rank ordering and (b) ratings of responses

(Powell, 2003).

In the current study the researcher conducted three rounds of data collection. The details of the specific rounds of the proposed study are outlined in the data collection and analysis-by- rounds section below. The researcher presented the questionnaires and collected data in each round via the online software program, Qualtrics. A link to the questionnaire was sent via email to the Expert Panel Members individually in each round to ensure anonymity.

Role of the Researcher

The researcher served as the facilitator, data collector, and analyzer for the Delphi Study.

The researcher set criteria (previously described) for the Expert Panel, recruited participants to serve as Panelists, developed and analyzed questions for the individual rounds, compiled responses, analyzed data collected, and reported on the findings of the Delphi study. The researcher analyzed the results of the Delphi study through content analysis and computing measures of frequency (percentages), central tendency (median, interquartile range), and stability

(Wilcoxon matched-pairs signed-ranks test and Spearman’s rank correlation coefficient) of the

Panelist’s rank ordering and rating of responses. In order to address potential bias, the researcher kept detailed notes of the research process during the data collection and analysis of each round.

The researcher’s notes assisted the researcher in maintaining the objectivity and reliability of the results.

Data Collection and Analysis Description by Rounds

Initial Round Data Collection

The initial round of data collection addressed an absence of consensus in the literature among proponents of individual psychology regarding the how component of the two research INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 61

questions: (a) how may individual psychology demonstrate efficacy given the current EBP evaluation standards, and (b) how may individual psychology demonstrate effectiveness given the current EBP evaluation standards. The intended objective of the initial round was to provide the Expert Panel Members the opportunity to provide their candid opinions of how individual psychology can demonstrate efficacy and effectiveness given the current EBP evaluation standards. The first questionnaire (Appendix B) included two broad, open-ended, questions soliciting Panelist feedback regarding the how component of the research questions, and was disseminated to the Expert Panel Members via email linking them to the Qualtrics website.

Upon entering the site, Panelists first were instructed to complete an informed consent agreement

(Appendix A) and to review the current evaluation standards criteria for efficacy and effectiveness in order to ensure that each Panel Member was aware of what is being asked in the research question. Upon providing informed consent, the Panel Members were instructed to provide their individual (and anonymous) responses to each question in a qualitative, free- response paragraph form. The researcher sent weekly reminders for Panelists to complete the survey until 15 to 20 responses were obtained.

Initial Round Data Analysis

Using content analysis, the researcher examined the open-ended responses to identify all unique and specific suggestions that each individual Panelist made regarding the how component of the two research questions. The researcher then combined repeated suggestions into a single suggestion, remaining cautious to ensure that responses combined were only those that were identically worded or significantly similar in order to not misrepresent Panelists’ opinions and to provide the Expert Panel with the most comprehensive lists of unique suggestions possible.

Once all unique suggestions were identified, the researcher constructed a comprehensive list of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 62

those suggestions for each of the two research questions (i.e., one comprehensive list of suggestions to address how individual psychology may demonstrate efficacy given the current

EBP evaluation standards, and a second comprehensive list of suggestions to address how individual psychology may demonstrate effectiveness given the current EBP evaluation standards). Both comprehensive lists were then reviewed by impartial reviewers for mutual exclusivity and suggestion clarity. After impartial review was completed, both lists were finalized and prepared to be disseminated in the second round.

Second Round Data Collection

Prior to the second round of data collection, all Panelists were provided with the two comprehensive lists of suggestions developed in round one regarding how individual psychology may address the two research questions. Panelists were instructed to reflect on and revise their initial responses as desired after comparing them to the responses from other Panel Members.

Upon completion of that task, the researcher incorporated the revisions and distributed a new set of instructions to the Panelists via the Qualtrics website (Appendix C) and instructed Panel

Members to rank-order the top twenty-five suggestions from the two separate lists of suggestions in order of each suggestion’s perceived utility in addressing the how component of the two research questions. In addition to rank-ordering the two lists, the researcher instructed the Expert

Panel to rate each response in each list using two separate seven-point Likert scales. The first

Likert scale asked Panelists to rate each suggestion based on his or her personal belief regarding the feasibility (i.e., the can component) of the suggestion as a means of addressing its respective research question. This Likert scale ranged from definitely cannot to definitely can on a seven- point scale. The second Likert scale asked Panelists to rate each suggestion based on their personal belief regarding its perceived benefit (i.e., the should component) as a means of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 63

addressing its respective research question. This Likert scale ranged from absolutely no benefit to a great deal of benefit on a seven-point scale. Reminders were sent to Panel Members weekly until responses were collected from 10 to 15 participants.

Second Round Data Analysis

Following the second round of data collection, the researcher computed frequencies

(percentages) and measures of central tendency (median and interquartile ranges) for all participants’ rank-orderings of the two lists of suggestions as well as for their rating of suggestions on each of the two Likert scales. The researcher then utilized these findings to determine if suggestions had reached or were trending towards consensus. Consensus for this study was determined for the how component when suggestions reached 80% of consensus among all Panel Members (Ulschak, 1983). Additionally, consensus was determined for the can and should components when Panel Member’s ratings reached an interquartile range between 0 and 2. Given Powell’s (2003) remarks that measures of frequencies and central tendency are not the soundest format of measuring consensus, the researcher continued to the third round of data collection where stability testing would be conducted regardless of whether any suggestion’s rankings reached the 80% consensus goal or ratings reached an interquartile range between 0 and

2. All suggestions that were not rank ordered by at least 25% of the Panel were removed in an effort to urge the Panel towards consensus.

Preparation for Round Three

In preparation for round three, the data analysis from the second round was utilized to construct a report of the statistical findings for the Expert Panel to review. The report was provided to the Panelists via email a week before disseminating the third round instructions, thus giving each Panel Member the opportunity to review how their fellow Panel Members were INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 64

ranking and rating the suggestions. This was done in order to inform the Panel Members of how the research questions and the suggestions were trending towards consensus among Panel

Member’s rank ordering and ratings.

Third Round Data Collection

Round three research instructions were disseminated via email linking Panelists to the

Qualtrics website. The instructions provided Panelists with listings of suggestions for each of the two research questions revised as described in the previous discussion of round two analysis.

As in round two, the researcher instructed Panel Members to rank-order the suggestions in each of the two lists based on their perceived utility to address the how component of the two research questions. However, in contrast to the second round, Panel Members were instructed to only rank order their top ten suggestions from each list. This was done in an effort to further urge the

Panel towards consensus. Next, Panel Members were again instructed to conduct Likert scale ratings of each individual suggestion according to the degree to which they thought the suggestions can and should be implemented. Reminders and response requests were sent to

Panel Members weekly until responses were collected from a minimum of 10 participants.

Third Round Data Analysis

Similar to the second round of data analysis, the researcher computed measures of frequency (percentage) central tendency (median and interquartile range) for their rankings and

Likert ratings. Additionally, in the round three data analysis the researcher tested Panel

Member’s responses for stability. To test for stability of the research data, the researcher conducted nonparametric statistical analyses of Panel Members’ rank-ordering and Likert ratings utilizing the Wilcoxon matched-pairs signed-ranks test to assess whether there were significant differences between Panel Members’ common rankings and ratings of suggestion for the how, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 65

can, and should components of each research question between rounds two and three. To further verify the stability of the consensus reached between rounds two and three, the researcher also computed the Spearman’s rank correlation coefficient. This test assessed the level of association between Panel Member’s ratings and rankings of each suggestion for the how, can, and should components of each research question (Day & Bobeva, 2005; Von der Gracht, 2012).

The Wilcoxon matched-pairs signed-ranks test analyzed the “paired data of the same group of individuals” between rounds two and three to determine, “whether a difference between the data of two Delphi rounds has statistical significance” (Von der Gracht, 2012, p. 1532). That is, it tested the degree to which the difference between Panel Members’ ratings and rankings of each suggestion on the how, can, and should components of each research question differed from zero between rounds two and three. According to Kalaian and Kasim (2012), “The test provides the sum of each of the positive and negative ranks of the differences between any consecutive rounds of Delphi survey responses (e.g., ratings) with a Z statistic and its asymptotic p-value” (p.

5). Ultimately, the Wilcoxon matched-pairs signed-ranks test showed whether Panel Members’ rankings and ratings became stable and were not significantly changing between rounds. If a significant change was found between the Panel Members’ rankings and ratings for any suggestions between rounds two and three (i.e., if the difference between them is significantly different from zero), then consensus regarding that suggestion was considered unstable.

Conversely, if there was no significant change (i.e., no significant difference from zero) found between the Panel Member’s rankings and ratings between rounds two to three, the consensus regarding that suggestion was considered stable.

The Spearman rank correlation coefficient was used to compute the level of consensus among all Panel Members’ ratings and rankings of suggestions for addressing the how, can, and INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 66

should components of the two research questions. Specifically, the Spearman analysis demonstrated the level of correlation between Panel Member’s round two and three rankings of suggestions for addressing the how component of the two research questions and their Likert ratings of the can and should suggestions. The analysis produced a correlation coefficient falling between -1 and +1, with positive correlation coefficients that fall closer to +1 indicating a positive correlation (i.e., consensus) and those that fall at or below 0 indicating a lack of consensus. Suggestions demonstrating significant (α = 0.05) and positive correlations (i.e., consensus) among Panel Members in their how component rankings and their can and should component Likert ratings were considered stable. Suggestions for which the Spearman findings were not significantly positive were considered unstable.

As noted previously, The Wilcoxon and Spearman analyses were utilized to demonstrate the stability of the consensus reached by Panel Members’ responses between rounds two and three. Given Von der Gracht (2012) remarks that Delphi studies should not stop collecting data or consider consensus achievement until responses have been determined stable, the researcher utilized the stability findings to qualify suggestion consensus. Thus, for this study, the researcher required Panel Members to demonstrate consensus through both stability testing and reaching the previously described consensus goals based on percentages and interquartile range scores (for the how component: suggestions were considered to reach consensus when they had been ranked by 80% of the Panel; for the can and should components: suggestions were considered to have reached consensus once suggestions’ interquartile range was between 0 and

2). If the measures of stability, frequency, and central tendency showed that Panelists’ responses were still unstable after round three, then subsequent rounds may have been needed that would follow essentially the same protocol used in round three. However, after round three the stability INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 67

tests demonstrated that the Panelist’s rankings and ratings of several suggestions were stable and that suggestions had reached or were trending towards consensus based on frequencies and measures of central tendency. Therefore, the researcher stopped the data collection process and moved forward with the final interpretation of the data.

Ethical Considerations

Prior to initiating the study, the researcher submitted the current Delphi study for approval to the Institutional Review Board of the College of William and Mary's School of

Education, for which it was approved. As noted previously, participants were required to sign an informed consent document before being permitted to serve on the Expert Panel. They were informed of their rights, with specific attention to confidentiality and their right not to participate or voluntarily withdraw from participation if they choose. The anonymous nature of the Delphi methodology process contributed to the protection of participant confidentiality, and no identifying information was collected from participants that could link them to their responses or identify them. No known risks to participants were associated with the study.

Limitations

The current study is limited in its sampling, instrumentation, and content. The sample selection involved purposeful sampling, which limited the study’s generalizability. Inasmuch as the Delphi method requires an extensive time commitment to complete each round as well as a commitment to several rounds of data collection, sample recruitment was difficult, and there was a high potential for participant dropout. The content of the study also posed specific limitations, as the subject matter being explored and analyzed solicited opinions from a polarized debate that potentially limited the ability of participants to reach consensus. Further, the consensus that INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 68

reached may not be generalizable to other Expert Panels and the population due to the homogeneous nature of the Expert Panel.

Despite these limitations, the potential benefits of the current study warranted the need for the study to be conducted. The study potentially fills a current gap in the literature regarding how individual psychology can address challenges to its efficacy and effectiveness, as well as adds to the conversation on EBP in the mental health field. It further provides justification and direction for individual psychology to continue to provide its holistic form of treatment despite the trend in the mental health field to move towards briefer, symptom-focused treatment.

Finally, the current study may provide insight into how individual psychology can demonstrate its continued relevance in a post-modern era driven by EBP and ESTs.

Summary

This chapter outlined the overall Delphi method used in this study. The process used to recruit Panel Members and the round by round data collection and data analysis procedures were presented. Further, chapter three explained how the Wilcoxon matched-pairs signed-ranks test, and Spearman rank correlation coefficient were used to test the stability of the consensus achieved. Finally, ethical considerations and limitations of this research method were noted.

The next chapter will describe the results from each round, and report the overall stability and consensus achieved in this study for the how, can, and should components for both efficacy and effectiveness. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 69

CHAPTER FOUR

Results

The purpose of this study was to explore the opinion of Adlerian experts about the future of individual psychology in a post-modern era of mental health driven by EBP. Specifically, this study explored the consensual opinion of Adlerian experts regarding how, can, and should individual psychology demonstrate both efficacy and effectiveness. The Delphi method was selected for this study because of its ability to facilitate an anonymous discussion among experts in individual psychology who were dispersed geographically across the United States. The

Delphi method allowed for Adlerian experts to anonymously examine a topic that has been debated within the field of individual psychology for several decades without adequately being documented in the literature. By utilizing the Delphi method, several trends and potential means for individual psychology to demonstrate efficacy and effectiveness were identified and either reached consensus or demonstrated a trend towards consensus. This study effectively captures the ongoing debate within individual psychology regarding how to address the pressures of evidence-based practice.

In this chapter, the results of the study are presented, and are organized around the three rounds that were conducted. The study focused on the two following research questions:

• How, can, and should individual psychology demonstrate efficacy given the current

evidence based practice evaluation standards;

• How, can, and should individual psychology demonstrate effectiveness given the current

evidence based practice evaluation standards.

Each of the two research questions were broken down into three components, specifically the how, can, and should components respectfully. The findings of each component of the two INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 70

research questions that are presented in this chapter reflect the information collected and analyzed through the three rounds of the study.

The overall research process began in the first round, where Panel Members provided suggestions for the how component of each research question, and the researcher then compiled all suggestions into two separate categories: efficacy and effectiveness. In the two subsequent rounds each Panel Members rated each suggestion in the two separate categories based on perceived feasibility and benefit; Panel Members then rank-ordered each suggestion from both lists based on each suggestion’s perceived utility to demonstrate either efficacy or effectiveness.

The study was designed to urge the Panel’s opinions and perspectives towards consensus among their rating and rank-ordering of suggestions.

Review of Delphi Process

The three survey rounds of this study were distributed to the Panel via email invitation linking them to a Qualtrics survey. Round one of the study involved the initial exploration of the

Panel Member’s broad opinions regarding the how component of the two research questions.

Individuals were recruited to participate on the expert Panel and sent a link to complete the first round Qualtrics survey. Panelists responded to two open-ended questions (i.e., (a) how may individual psychology demonstrate efficacy given current EBP evaluation standards and (b) how may individual psychology demonstrate effectiveness given current EBP evaluation standards), resulting in two narrative responses from each Panel Member. The information collected from the initial round provided the foundation for each subsequent round of the study.

Panel Member responses to round one provided a broad range of perspectives and suggestions related to the two research questions. These open-ended responses were analyzed and interpreted using classical content analysis to condense them into lists of distinct suggestions INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 71

and to combine similar suggestions into one suggestion. Once constructed, the list of potential suggestions for efficacy and the list of potential suggestions for effectiveness were used as data for subsequent rounds.

The survey questionnaire that was developed for round two utilized the two compiled lists of suggestions for efficacy and effectiveness from the first round. Each Panel Member was sent the two compiled lists of suggestions to review a week prior to the beginning of the second round. The second round questionnaire was distributed utilizing a second email invitation to a

Qualtrics survey. Round two provided Panelists the opportunity to review all other Panel

Members’ suggestions as well as their own and to begin the process of rating and rank ordering the suggestions in each list. Responses to round two were analyzed using SPSS software to compute the median and interquartile range for each Panel Member’s rating and to compute the frequency reports (percentages) for Panel Member’s rank-ordering. The second round concluded after data analysis was completed and after suggestions with lower than 25% of the Panel ranking them in their top twenty-five were removed.

The third round mirrored the methodology of the second round while utilizing the reduced list constructed from the second round. A third email invitation was sent out to Panel

Members linking them to the third and final Qualtrics survey. Panelists were asked again to rate all suggestions in the two lists based on the perceived feasibility of each suggestion and the perceived benefit of each suggestion. Finally, Panelists were asked to rank order the top-ten suggestions from both lists. Using SPSS software, the researcher computed the median, and interquartile range of suggestion ratings from each list and computed median and frequency reports for each of the rank ordered suggestions. Additionally, the researcher computed the

Spearman rank correlation coefficient and Wilcoxon matched-pairs signed-ranks test to check INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 72

the stability of responses across rounds two and three. Round three results were utilized in constructing this final report. Although consensus was not reached for the how component, and disagreement actually widened from round two to round three, the highest frequency of rank ordered suggestions are reported as trending towards consensus. However, the stability testing that took place after round three and the interquartile ranges indicated that the can and should components were able to reach consensus around several suggestions. The following sections discuss the major findings that emerged within rounds one, two, and three. The process of reaching consensus is presented within each round; then a summary of the overall consensus and trends towards consensus is presented.

Delphi Response Profiles by Each Round

Round I

In the first round, 16 participants provided two thorough (500-word minimum) paragraph form responses to the two research questions (how may individual psychology demonstrate efficacy given the current evidence based practice evaluation standards; and how may individual psychology demonstrate effectiveness given the current evidence based practice standards).

Once all responses were recorded the researcher utilized classical content analysis to evaluate each Panel Member’s individual responses to the two research questions. Using classical content analysis, the researcher identified unique suggestions within each Panel Members paragraph responses, and grouped all similar suggestions by theme. The researcher then constructed two comprehensive lists of all suggestions that were extracted from each Panel Member’s responses

(one list of suggestions pertaining to the efficacy research question; and one list of suggestions pertaining to the effectiveness research question). After constructing each list, the researcher consolidated mutually similar suggestions into one suggestion (example: two Panel Members INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 73

offered the suggestion that individual psychology conduct more outcome studies; those suggestions were combined into one suggestion because they were the same suggestion). The researcher refined suggestions for clarity and deleted repeated suggestions. Following the researcher’s review and consolidation, the two comprehensive lists were provided to two impartial reviewers to evaluate for the clarity and mutual exclusivity of each suggestion in the two lists. After the analysis was completed, 81 suggestions for how individual psychology could demonstrate efficacy were retained (Table 1), and 54 suggestions for how individual psychology could demonstrate effectiveness were retained (Table 2).

Table 1.

Round One - Revised Comprehensive List of Suggestion (Efficacy)

Number Efficacy Suggestion 1 Individual Psychology must develop a stronger research base. 2 Individual Psychology must encourage practitioners and researchers to pool their resources and collaborate on research projects. 3 More effort must be put into recognizing supporting and incentivizing (through The North American Society of Adlerian Psychology and external funding) the research efforts (Specifically Empirical Research) of Adlerian researchers and practitioners (who publish within and outside of the Journal of Individual Psychology) 4 Individual Psychology must conduct more outcome-based research. 5 Individual psychology must focus research efforts on testing fundamental hypotheses based on the theory of individual psychology in order to develop a more solid literature base. 6 Individual Psychology must specifically define basic Adlerian constructs and the core components of the theory (lifestyle, encouragement, life- task, etc.) in an empirically testable form, and distinguish them from assessments and treatments to design empirical studies based upon those distinctions. 7 Individual Psychology must clarify if lifestyle is simply personality or broader than personality, in order to better identify the effect of treatment on lifestyle. 8 Individual Psychology must operationally define constructs and develop instrumentation that represents and measures those constructs individually, as well as the effects of treatment on those constructs. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 74

9 Individual psychology must utilize established instruments and psychometrics, which have been proven to establish efficacy and measure change, to conduct pre-/post-tests related to the efficacy of specific Adlerian interventions. (Examples of instruments: Becks Depression Inventory; Becks Anxiety Inventory; Early Recollections Rating Scale Manaster/Perryman, Millers; Session Rating Scale; Sullimans Social Interest Scale) 10 Individual Psychology must support and develop an inclusive and broad research initiative that invites scholars and practitioners to develop and be involved in generating research. 11 Individual Psychology must review, emphasize, replicate, redesign, and utilize the Adlerian empirical literature and data that is presently available regarding the efficacy and influence of Adlerian concepts, interventions, and instruments to demonstrate of the current efficacy of individual psychology. 12 Individual Psychology must utilize the quantitative evidence and support for Adlerian parent education programs that are considered EST (STEP Program, Active Parenting, Etc.) to provide evidence for selecting Adlerian treatment models for use with specific clinical populations, and to provide evidence to support that Adlerian parent education programs provide superior treatment outcomes compared to not engaging in treatment. 13 Given the influence that individual psychology has had on cognitive behavioral therapy, individual psychology can utilize the evidence supporting CBT’s efficacy and demonstrate its own efficacy by distinguish itself from CBT based on influence and effects that Adlerian techniques (lifestyle assessment) have on the therapeutic process. 14 Individual Psychology must utilize the BASIS-A as a foundation for assessing the effect of lifestyle assessment on the treatment process as a means to distinguish individual psychology from cognitive behavioral therapy. 15 Individual psychology must conduct comparative research (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma). 16 Individual Psychology could develop treatment manuals similar to the process that interpersonal psychotherapists have. 17 Individual Psychology must provide evidence to support the need for Individual Psychology to begin to conduct research regarding its efficacy. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 75

18 Adlerian graduate programs must prioritize the training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. 19 Individual psychology must explore and utilize a broad empirical literature base (non-Adlerian literature) from models outside of Adlerian psychology to support Adlerian concepts that are researched in other models (concepts such as belonging, social connectedness, social interest, and family constellation and atmosphere that are shown to be relevant aspects of clinical models from the CBT approach). 20 Individual Psychology must utilize literature-comparing BASIS-A to other instruments. 21 Individual Psychology must conduct research utilizing the BASIS-A to demonstrate clinical outcomes. 22 Individual Psychology must conduct research establishing and demonstrating that Adlerian counseling (specifically lifestyle assessment) promotes deeper understanding, encourages motivation for change, and is a powerful insight-building tool compared to standard clinical interviews based on DSM/ICD systems and or straight DBT and CBT skills. 23 Individual Psychology must conduct Adlerian based research to demonstrate the efficacy of Adlerian interventions with specific populations. 24 Individual Psychology must commission several methodology experts and/or establish an executive research planning and oversight team to establish a study design that meets the required EBP evaluation standards for efficacy, and to provide evaluation over research projects. 25 Individual psychology needs to conduct research using double blind randomized control trials. 26 Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk-therapy strategies compared to “treatment as usual”, a no-treatment group, or a waiting list group of clients who are not currently receiving care, longitudinally if possible. 27 Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). 28 Individual Psychology must establish an Adlerian research task force to seek groups/practices to implement experimental studies. 29 Individual Psychology must establish an Adlerian research task force to seek collaboration from several Adlerian experts to assemble an Adlerian treatment strategy manual 30 Individual Psychology must utilize training videos/appropriate supervision to develop treatment fidelity. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 76

31 Individual Psychology must utilize Certified Adlerian trainers (possibly The North American Society of Adlerian Psychology diplomats) to offer training and supervision to ensure treatment fidelity. 32 Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well- controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 33 Individual Psychology must manualize specific individual psychology interventions such as push-button technique, reflecting as if, three-step emotional change trick and other Adlerian approaches. 34 Individual Psychology must attempt to loosely manualize techniques to serve as an outline that clinicians can adapt based on individual cases. 35 Individual Psychology must publish outside of Adlerian-based journals. 36 Individual Psychology must increase professional development opportunities to have more training in research. At The North American Society of Adlerian Psychology and local conferences there can be specific pre-/post-conference workshops where individuals get specific training on research process and statistics. Specific strand at the conference could be offered on research, maybe through open forums on research ideas, or sharing research results. 37 Individual Psychology must encourage the Adlerian community to provide more mentorship to and Adlerian graduate programs masters and doctoral students’ conducting research. 38 When the Journal of Individual Psychology receives an empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. Therefore, the process is more encouraging and it means more publications of empirical research. 39 Individual Psychology must offer auxiliary to the conferences, specific training, workshop, or conference on research in Adlerian theory (Gestalt practitioners are doing this). 40 Individual Psychology must provide practitioners with training in systematic case study, single case time series, and single case experimental designs. 41 Individual Psychology must emphasize, support, and award the scientist/practitioner model (could establish a program similar to emerging leader program, an emerging research practitioners can be started). 42 The journal of Individual Psychology must be indexed in the Social Science Citation Index to encourage more international researchers to publish in it (SCI journals are more valued in their organizations and in their countries). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 77

43 Individual Psychology must reach out to the international community to become involved in empirical research (this is how Gestalt research programs have started increasing empirical research). 44 Individual Psychology must reach out to sister organizations, Association of Humanistic Counseling, Division 32 of APA, other postmodern and phenomenological therapies to explore empirical research options. 45 Individual Psychology must offer research grants from The North American Society of Adlerian Psychology (Clonick) and assist its members in seeking external grants. 46 Individual Psychology must encourage regional groups to establish their research circles and publish. 47 Individual Psychology could establish another journal (maybe online) focusing only on Adlerian research studies. 48 Individual Psychology must conduct meta-analysis studies of empirical literature regarding Adlerian constructs and treatment interventions. 49 Individual Psychology must develop a researcher in residence program that is funded to conduct research (possibly have Adlerian graduate programs initiate) 50 Individual Psychology must train researchers and practitioners on process based research. Such as Narrative Correspondence Method, prospective naturalistic study, etc.. 51 Individual Psychology must link researchers together to target one intervention at a time to pursue gathering enough data. 52 Individual Psychology must identify specific interventions to be extensively researched such as lifestyle assessment interpretation, use of metaphors (Kopp metaphor intervention), use of paradox, interpretation of ER’s, use of encouragement, use of stories imagery techniques such as push button, reflecting as if, interpreting BASIS-A 53 Individual Psychology should look at previous studies and methodologies used by brief dynamic theories to develop research methodologies to specify intended outcomes and evaluate the efficacy of individual psychology to meet those outcomes. 54 Treatment manuals must be developed that include matrices that track skills and techniques that should occur in each phase of treatment in order to establish treatment fidelity. 55 Treatment manuals should be developed that include case conceptualization that occurs after the second phase of treatment and before proceeding with the third and fourth phase. 56 Treatment manuals must be developed for treating broad groups (individual work with adults, group work, family work, etc.). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 78

57 Treatment manuals must be constructed for treatment with in each of the broad groups for use with specific populations, various clinical diagnosis, and problem areas within each broad group (i.e., individual work with adults who experience depression, group work with teens with anxiety, family work with step families who have experienced trauma, etc.). 58 An expert in the application of individual psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity. 59 Researchers can look at how other clinical models have established efficacy and design studies similar to those done in the published empirical literature. 60 Researchers can work to design better instruments that measure factors affected by intervention with individual psychology and find instruments used in well-designed studies of other clinical models that have already established themselves as efficacious according to the current EBP evaluations standards and use those instruments in studies measuring the efficacy of individual psychology. 61 Researchers should measure short term and long-term effects of Individual Psychology interventions. 62 Individual Psychology must collaborate with staff at a variety of college counseling centers so that counselors who are working with college students who receive counseling services in these centers are using individual psychology as the basis for their work and then use their clients to do pre-post tests control group studies. 63 Adlerian oriented theorist need to align themselves with institutions that will sponsor and support empirical research. 64 Adlerian theorists must clearly define the critical components of their techniques and the parameters of positive outcomes. 65 Adlerians must improve their research design and utilize more than one inventory to validate the theory and or treatment. 66 Individual Psychology must acquire the personnel and research sophistication to plan and implement clinical trials in Adlerian training clinics. 67 Individual Psychology must utilize research design that is consistent with established EBP criteria. 68 Individual Psychology must utilize 20-30 therapists (trainees and experienced clinicians) at more than 5 or more sites with approximately 300 clients in efficacy-based research. 69 Individual Psychology must plan a standardized intervention protocol for all sites that would be used during clinical trials. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 79

70 Individual Psychology must evaluate session-to-session monitoring of both clinical outcomes and therapeutic alliance. 71 Individual Psychology must systematically collect data from participants at multiple sites and enter in online data site, which has been specifically constructed or modified and field-tested for use Adlerian research projects. 72 Individual Psychology must statistically analyze data, tabulate results, and disseminate written reports on the efficacy of individual psychology. 73 Individual Psychology must encourage Adlerians to conduct experimental research related to the efficacy of individual psychology through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. 74 Individual Psychology must teach specific concepts of experimental research through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. Instructions that would emphasize supporting the efficacy of individual psychology would be an important aspect of the concepts taught. 75 Individual Psychology must teach the process for submitting experimental research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting experimental research about the efficacy of specific techniques of individual psychology and submitting the results to SAMSHA for inclusion as Evidenced-Based Practice on the national register. 76 A detailed discussion with Michael Popkin about the process that was enacted to have “Active Parenting” accepted, as an Evidenced-Based Practice on the national register must happen. 77 Adlerians must present their experimental research findings at non- Adlerian conferences for the purpose of interesting non-Adlerians in conducting experimental research related to the efficacy of techniques coming from individual psychology. 78 The Journal of Individual Psychology must feature an article each issue about the importance of Adlerians conducting experimental research and Meta analyses about the efficacy of individual psychology. 79 Individual Psychology needs to develop a model that can be easily disseminated. 80 Individual Psychology needs to view the challenges of demonstrating efficacy as an opportunity to re-examine its methodology and/or modify the clinical model/interventions. 81 Individual Psychology needs to spend less time justifying its views and beliefs and spend more time putting them through the ringer and admitting errors or the need for modification.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 80

Table 2.

Round One - Revised Comprehensive List of Suggestion (Effectiveness)

Number Effectiveness Suggestions 1 Individual Psychology must conduct outcome studies 2 Individual Psychology must acquire, align with, and utilize resources such as trained professionals and supportive research institutions to conduct controlled studies. 3 The North American Society of Adlerian Psychology and individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. 4 Individual Psychology must operationally define Individual Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner. 5 Individual Psychology must create instrumentation to measure Individual Psychologies constructs so outcome work can commence. 6 Individual Psychology must utilize existing instruments (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. 7 Individual Psychology needs more reliance on the scientific method, training in scientific method, and support for conducting strong empirical research. 8 Individual Psychology researchers need to go outside of IP and take steps to build a research base similar to the process used by other empirically supported treatments (cognitive therapy; behavior therapy). 9 Individual Psychology must select one intervention/technique and develop a mode for treating specific types of problems and then conduct research regarding the effectiveness of the interventions in multiple settings. 10 Individual Psychology must develop and/or utilize an existing program evaluation model (CBT and IPT have already established evaluation models) to look at inputs to identify client, clinician, and setting characteristics, while specifying treatment and alterations while measuring outcomes. 11 The North American Society Of Adlerian Psychology must provide grants and funding for researchers to build a program evaluation model. 12 Although current Adlerian literature includes demographic information and has analyzed various factors such as gender/sex, age, etc., individual psychology must conduct research regarding the extent that various factors effect Adlerian treatment outcomes. 13 Individual Psychology must conduct case studies to demonstrate the extent that demographics and other factors have on Adlerian treatment outcomes. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 81

14 Individual Psychology must develop and utilize a BASIS-A lifestyle assessment protocol to demonstrate effectiveness. 15 Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 16 Individual Psychology must utilize the published case studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. 17 Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology). 18 In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 19 Data Collection in future studies should include collecting data regarding the setting where treatment is provided (inpatient/outpatient/school), and other variables that will influence outcomes (medication, other therapeutic services being received, support system, support group, clients stage of change, etc.). 20 Adlerian fidelity measures should be created and utilized (similar to the Cognitive Therapy Rating Scale) that measures the competence of the clinician using Adlerian therapy, as a means to ensure treatment fidelity among clinicians and treatment provided in research studies. 21 Individual Psychology must set a minimum level of competency (measured via a constructed Adlerian fidelity measure) to be able to participate in specific empirical studies. 22 In order for clinicians to participate in effectiveness studies individual psychology should require clinicians to meet certification requirements including a minimum number of training hours, meeting a minimum level of competency on a developed Adlerian fidelity measure, and submission of counseling video demonstrating the use of Adlerian techniques that would be evaluated utilizing an established Adlerian therapy scale. 23 In order to quantify feasibility, individual psychology must develop an assessment protocol for the locations that are being considered for participation based on the benefits and challenges of each location where services may be offered (inpatient drug treatment center, college counseling centers, community mental health clinics, etc.). 24 Individual Psychology may utilize clinics and locations connected with Adler graduate programs to conduct research do to the readably available training and supervision that these clinics may be able to offer. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 82

25 Individual Psychology must understand that there are various methods to demonstrate effectiveness such as Seligman’s consumer report study: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. 26 Individual Psychology must develop a survey questionnaire and administer it to 20 Adlerian counselors, 20 CBT counselors, and 20 Eclectic counselors to evaluate counselor’s differential perception of their outcomes based on counselors ratings to demonstrate there is no difference between theoretical orientations related to treatment effectiveness. 27 Individual Psychology must develop a develop a survey questionnaire and administer it to current or previous clients of 20 Adlerian counselors, 20 CBT counselors, and 20 Eclectic Counselors to evaluate clients differential perception of their outcomes from treatment based on client ratings to demonstrate no difference between theoretical orientations related to treatment effectiveness. 28 Given the only real difference between accepted EBP’s and individual psychology is verbage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. 29 Individual Psychology must utilize the variety of settings where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. 30 Researchers can partner with practitioners and international researchers in order to conduct outcome studies regarding the effectiveness of individual psychology in a variety of situations and with clients and clinicians with diverse backgrounds. 31 Individual Psychology should establish regional and international research groups (with researchers and practitioners coming from diverse backgrounds) to compare and control various research studies conducted to explore the effects of therapist variables on treatment process and outcomes. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 83

32 Individual Psychology must utilize strong partnership and a mentorship process to support research efforts. 33 Individual Psychology should start a workgroup to develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual. 34 In order to establish effectiveness individual psychology must: develop treatment protocols and manuals for single interventions; Disseminate these to everyone (Adlerian or not) freely; Establish research training for Adlerians; Pair researchers with practitioners; Conduct outcome research on effectiveness; Publish the findings; and Repeat the process. 35 Adlerians already respect the influence patient variables (age, gender, sexual orientation, e.g.) have on treatment outcomes, but need to develop means to quantify this influence that maintains a respect for each person’s holistic way of being and uniqueness. 36 Individual Psychology must establish a means to quantify the influence that individual clinicians skills have on treatment rather than quantify characteristics of a clinician. 37 Individual Psychology must utilize established measures besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social). 38 Individual Psychology must utilize Miller’s session Rating Scale to demonstrate that individual psychology approach is satisfying and that clients are willing to participate in counseling sessions. 39 Individual Psychology must conduct comparative outcome studies that use several different levels of a patient variable (i.e., very complex clinical presentation, moderate clinical presentation, simple clinical presentation) receiving Adlerian treatment compared to a no treatment group, and have a large enough sample sizes that the researcher can gather information about patient variables such as gender/sex; culture; age/developmental level and in the analysis of the results group the clients accordingly. 40 Individual Psychology must conduct comparative outcome studies that measure the effect that different clinicians have on the treatment outcomes by collecting data about professional identity, clinical experience, measures of clinical skill, fidelity measures, graduate degrees held, licensure status, numbers of years using Adlerian techniques, culture/ethnic background, gender, sexual orientation, and age of clinician in the demographic information gathered, and use these in the variables used in the data analysis in order to monitor the effect of each clinicians on treatment outcomes among clients (large sample size so that the subgroups will have a large enough number of subjects to be relevant in the data analysis). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 84

41 Individual Psychology must conduct comparative studies that have specific elements altered in the administration of the treatment (with specific elements changed in each group). For instance, Adlerian play therapy with and without parent consultation; Adlerian play therapy with parent consultation compared to Adlerian play therapy with teacher consultation; Adlerian play therapy that lasts 16 sessions, compared to Adlerian play therapy that lasts 30 sessions; twice a week sessions compared to once a week sessions; etc. 42 Individual Psychology must conduct studies with populations that might have a long-term financial benefit from therapy (school children, prisoners with dual diagnoses, etc.). 43 Individual Psychology must conduct studies with clients with some kind of medical condition that might also be alleviated or dissipated by medical intervention combined with therapy (people with ulcers, people with diabetes, etc.). 44 Individual Psychology must conduct Efficiency studies (practice based Evidence studies) that are concerned with real world applications of Individual Psychologies treatment model in everyday treatment settings, and focus on session-to-session client self-comparison rather than comparing client outcomes to group means and aggregated client outcomes as used in effectiveness research (Note: Practice-Based- Evidence is the converse of the Evidence-Based-Practice model. At the present time, such studies would be eligible for listing in the National Registry of Evidence-Based Programs and Practices [SAMSHA], but not in the Research-Supported Psychological Treatments (APA-Division 12). 45 Individual Psychology must implement practice-based research at mental health agencies and private practice offices where there are Adlerian clinicians practicing. 46 Individual Psychology must have Adlerian faculty focus even more on teaching and encouraging Adlerian-oriented students the skills that they will need to conduct quantitative studies and case studies that support the clinical effectiveness of individual Psychology. 47 Individual Psychology must have the North American Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. 48 Individual Psychology must encourage Adlerians to conduct qualitative research of the clinical effectiveness of individual psychology through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 85

49 Individual Psychology must teach specific concepts of qualitative research through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. Instructions that would emphasize supporting the clinical effectiveness of individual psychology and financial feasibility of the application of Adlerian techniques would be an important aspect of the concepts taught. 50 Individual Psychology must teach the process for submitting qualitative research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting qualitative research about the clinical effectiveness of specific techniques individual psychology and submitting the results to SAMSHHA for inclusion as Evidenced-Based Practice on the national register. A preponderance of qualitative research demonstrating clinical applicability can result in acceptance. 51 Individual Psychology must have the Theory, Research, and Teaching TRT section of the North American Society for Adlerian Psychology implement a research team approach to conduct qualitative research related to the clinical effectiveness of individual psychology. This research team could be coordinated through the TRT listserve. 52 Individual Psychology must have Adlerians present their qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. 53 Individual Psychology must highlight qualitative research activities at the continental annual conference of The North American Society Of Adlerian Psychology to bring attention to the importance of these activities to support the clinical effectiveness of individual psychology. 54 Individual Psychology must have the Journal of Individual Psychology feature an article each issue about the importance of Adlerians conducting qualitative research and case studies about the clinical effectiveness of individual psychology.

After completing all data collection and analysis and constructing both comprehensive lists of suggestions, the researcher concluded the first round of the Delphi study. In preparation for the second round, both comprehensive lists of suggestions were sent out to each of the 16

Panel Members to review and provide any feedback (Appendix D). The feedback (provided by one Panelist) was exclusively to revise grammatical issues, and after incorporating feedback from the one Panelist, the researcher began the second round. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 86

Round II

Following the completion and revisions of the two comprehensive lists of suggestions from round one, all 16 Panel Members were sent a web link to the second Qualtrics survey

(Appendix F) via email (Appendix E). The second round focused on the how, can, and should components of both research questions. The Qualtrics survey for the second round asked

Panelists to rate each suggestion on two separate Likert scales and to rank order their top twenty- five suggestions. The first seven-point Likert scale (Figure 1) asked Panelists to rate each suggestion based on the perceived feasibility of the suggestion (can component).

Figure 1. Likert rating scale for the can component

4 1 2 3 5 6 7 Neither Definitely Mostly Somewhat Somewhat Mostly Definitely Can or Cannot Cannot Cannot Can Can Can Cannot

The second seven-point Likert scale (Figure 2) asked Panelists to rate each suggestion based on the perceived benefit of implementing the suggestion (should component).

Figure 2. Likert rating scale for the should component.

7 1 2 3 4 5 6 A Great Absolutely Minimally Slightly Somewhat Moderately Very Deal No Beneficial Beneficial Beneficial Beneficial Beneficial Of Benefit Benefit

After rating each suggestion, each Panel Member was asked to rank order the top twenty-five suggestions from each list based on the perceived utility of the suggestion to demonstrate efficacy or effectiveness (how component). Of the 16 original Panel Members, 14 Panel

Members returned completed results in the second round. Given that this study was designed to adjust for attrition, the loss of two Panel Members’ responses was considered normal. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 87

After Panel Members’ second round ratings and rankings had been recorded, data collection concluded, and data analysis began. The 14 remaining Panel Members rated and rank ordered all 81 suggestions provided for efficacy, and all 54 suggestions provided for effectiveness from the first round. Using SPSS statistical software, median scores and interquartile ranges of suggestions for the can and should component for efficacy were calculated and are reported in descending order in Table 3 below.

Table 3.

Item Median Scores and Interquartile Ranges for the Can and Should Components (Efficacy)

Efficacy Can Component Efficacy Should Component Suggestion Median Interquartile Suggestion Median Interquartile Number Range Number Range 35 7.0 1.0 1 7.0 0.0 59 7.0 1.0 4 7.0 1.0 76 7.0 1.0 6 7.0 2.0 9 6.5 1.0 9 7.0 1.0 38 6.5 1.0 26 7.0 2.0 45 6.5 2.0 27 7.0 2.0 52 6.5 1.0 35 7.0 1.0 1 6.0 2.0 72 6.5 2.0 2 6.0 2.0 2 6.0 2.0 4 6.0 2.0 3 6.0 2.0 5 6.0 1.0 5 6.0 2.0 6 6.0 1.0 8 6.0 2.0 8 6.0 1.0 10 6.0 2.0 11 6.0 2.0 11 6.0 2.0 12 6.0 1.0 15 6.0 2.0 14 6.0 2.0 16 6.0 3.0 16 6.0 2.0 18 6.0 2.0 17 6.0 2.0 22 6.0 1.0 18 6.0 2.0 23 6.0 1.0 19 6.0 1.0 24 6.0 1.0 20 6.0 1.0 29 6.0 3.0 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 88

21 6.0 3.0 30 6.0 3.0 23 6.0 2.0 31 6.0 4.0 24 6.0 2.0 32 6.0 3.0 26 6.0 2.0 33 6.0 3.0 27 6.0 2.0 36 6.0 3.0 28 6.0 2.0 37 6.0 2.0 29 6.0 1.0 45 6.0 1.0 30 6.0 1.0 48 6.0 2.0 31 6.0 2.0 52 6.0 3.0 32 6.0 1.0 59 6.0 1.0 33 6.0 1.0 60 6.0 2.0 34 6.0 1.0 64 6.0 2.0 36 6.0 1.0 66 6.0 2.0 37 6.0 1.0 67 6.0 1.0 39 6.0 1.0 70 6.0 3.0 40 6.0 2.0 71 6.0 3.0 41 6.0 2.0 75 6.0 2.0 42 6.0 2.0 77 6.0 3.0 43 6.0 2.0 43 5.5 2.0 48 6.0 2.0 54 5.5 3.0 51 6.0 2.0 58 5.5 4.0 53 6.0 2.0 61 5.5 2.0 54 6.0 1.0 65 5.5 2.0 55 6.0 2.0 68 5.5 1.0 56 6.0 2.0 80 5.5 3.0 57 6.0 2.0 12 5.0 2.0 60 6.0 2.0 13 5.0 5.0 61 6.0 2.0 19 5.0 2.0 64 6.0 2.0 25 5.0 2.0 65 6.0 2.0 28 5.0 3.0 67 6.0 1.0 34 5.0 3.0 70 6.0 1.0 38 5.0 5.0 72 6.0 2.0 39 5.0 2.0 73 6.0 1.0 41 5.0 2.0 74 6.0 2.0 42 5.0 3.0 75 6.0 1.0 49 5.0 2.0 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 89

77 6.0 1.0 51 5.0 2.0 78 6.0 2.0 53 5.0 1.0 80 6.0 2.0 56 5.0 3.0 81 6.0 2.0 57 5.0 2.0 3 5.5 1.0 69 5.0 2.0 10 5.5 1.0 81 5.0 2.0 22 5.5 1.0 7 4.5 2.0 44 5.5 2.0 46 4.5 2.0 58 5.5 1.0 63 4.5 2.0 7 5.0 2.0 76 4.5 3.0 13 5.0 2.0 79 4.5 3.0 15 5.0 2.0 14 4.0 4.0 25 5.0 1.0 17 4.0 4.0 46 5.0 1.0 20 4.0 3.0 47 5.0 2.0 40 4.0 2.0 49 5.0 2.0 44 4.0 2.0 50 5.0 1.0 50 4.0 1.0 63 5.0 1.0 55 4.0 3.0 66 5.0 1.0 62 4.0 2.0 68 5.0 1.0 73 4.0 3.0 69 5.0 2.0 78 4.0 3.0 71 5.0 1.0 74 3.5 3.0 79 5.0 1.0 21 3.0 3.0 62 4.0 3.0 47 2.0 3.0

For the can component, suggestions median scores ranged from 7 (definitely can) to 4 (neither can or cannot), and interquartile scores ranged from 3 to 1. The lowest interquartile ranges

(those indicating the most consensus), generally occurred among the suggestions with median

Likert ratings between 5 (somewhat can) and 7 (definitely can). Suggestions median scores for the should component ranged from 7 (a great deal of benefit) to 2 (minimally beneficial), and interquartile scores ranged from 5 to 0. It is important to note the difference in the interquartile range between ratings for the can and should component. The Panel ratings for the can INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 90

component ranged between 3 and 1 while the ratings for the should component ranged from 5 to

0. This difference shows that the suggestions with larger interquartile ranges had a wider range of variability and thus indicated less consensus among Panel ratings for those suggestions.

Median scores and interquartile ranges for suggestions from the second round for the can and should component for effectiveness were calculated and are reported in descending order in

Table 4 below.

Table 4.

Item Median Scores and Interquartile Ranges for the Can and Should Components

(Effectiveness)

Effectiveness Can Component Effectiveness Should Component Suggestion Median Interquartile Suggestion Median Interquartile Number Range Number Range 1 7.00 1 1 7.00 1 3 7.00 1 9 6.50 2 53 7.00 1 17 6.50 1 6 6.50 1 2 6.00 1 10 6.50 1 3 6.00 3 48 6.50 1 4 6.00 2 2 6.00 1 5 6.00 2 4 6.00 1 6 6.00 2 5 6.00 1 7 6.00 1 7 6.00 2 10 6.00 8 6.00 2 20 6.00 2 9 6.00 2 33 6.00 3 12 6.00 1 39 6.00 1 13 6.00 1 41 6.00 1 14 6.00 1 8 5.50 3 15 6.00 1 15 5.50 2 16 6.00 1 18 5.50 1 17 6.00 2 19 5.50 2 18 6.00 1 24 5.50 2 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 91

19 6.00 2 37 5.50 2 20 6.00 1 46 5.50 2 24 6.00 2 11 5.00 4 25 6.00 2 12 5.00 1 28 6.00 1 13 5.00 4 32 6.00 2 16 5.00 1 33 6.00 1 23 5.00 2 38 6.00 2 25 5.00 3 41 6.00 2 28 5.00 3 45 6.00 1 30 5.00 3 46 6.00 2 32 5.00 1 49 6.00 2 34 5.00 2 50 6.00 1 35 5.00 2 51 6.00 2 40 5.00 2 52 6.00 1 42 5.00 2 54 6.00 2 43 5.00 2 26 5.50 1 44 5.00 2 27 5.50 2 45 5.00 2 29 5.50 1 47 5.00 3 30 5.50 2 50 5.00 2 34 5.50 1 51 5.00 2 37 5.50 1 52 5.00 3 44 5.50 2 54 5.00 3 11 5.00 1 21 4.50 3 21 5.00 1 22 4.50 4 22 5.00 1 53 4.50 3 23 5.00 1 14 4.00 2 31 5.00 1 27 4.00 3 35 5.00 1 29 4.00 2 36 5.00 1 31 4.00 2 39 5.00 1 36 4.00 2 40 5.00 1 38 4.00 1 42 5.00 1 48 3.50 3 43 5.00 1 49 3.50 3 47 5.00 1 26 3.00 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 92

Median scores of suggestions for the can component ranged from 7 (definitely can) to 5

(somewhat can), and interquartile scores ranged from 2 to 1, with the lowest interquartile scores

(those indicating the most consensus) generally occurring among the suggestions with the highest median scores. Median scores of suggestions for the should component ranged from 7 (a great deal of benefit) to 3 (slightly beneficial), and interquartile scores ranged from 4 to 0. Once again, it is important to note the difference between the interquartile range between ratings for the can and should component. The Panel ratings for the can component ranged between 2 and 1 while the ratings for the should component ranged from 4 to 0. This difference shows that the suggestions with larger interquartile ranges had a wider range of variability and thus indicated less consensus among Panel ratings for those suggestions.

In the second round, Panel Members were also asked to rank order the top twenty-five suggestions from both lists based on their opinion of which suggestions would be the best option to demonstrate efficacy or effectiveness respectively. Percentages and frequencies of Panelists who rank ordered a specific suggestion were computed for each suggestion and are reported for efficacy (Table 5) and effectiveness (Table 6) in descending order below.

Table 5.

Suggestions Percentages and Frequencies for Panel Member Rankings (Efficacy)

Suggestion Number Frequency of Panel Percentage of Panel Members Who Ranked Item Members Who Ranked Item 35 11 79% 6 10 71% 22 10 71% 23 10 71% 1 9 64% 4 9 64% 15 9 64% INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 93

2 8 57% 8 8 57% 9 8 57% 27 8 57% 60 8 57% 3 7 50% 16 7 50% 18 7 50% 36 7 50% 38 7 50% 39 7 50% 67 7 50% 5 6 43% 11 6 43% 13 6 43% 24 6 43% 32 6 43% 33 6 43% 45 6 43% 48 6 43% 75 6 43% 19 5 36% 26 5 36% 30 5 36% 52 5 36% 58 5 36% 61 5 36% 25 4 29% 28 4 29% 43 4 29% 53 4 29% 57 4 29% 68 4 29% 69 4 29% 77 4 29% 80 4 29% INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 94

12 3 21% 14 3 21% 29 3 21% 31 3 21% 34 3 21% 37 3 21% 49 3 21% 54 3 21% 59 3 21% 63 3 21% 71 3 21% 76 3 21% 78 3 21% 81 3 21% 7 2 14% 10 2 14% 41 2 14% 42 2 14% 44 2 14% 51 2 14% 55 2 14% 64 2 14% 65 2 14% 66 2 14% 70 2 14% 72 2 14% 79 2 14% 21 1 7% 40 1 7% 50 1 7% 56 1 7% 62 1 7% 17 0 0% 20 0 0% 46 0 0% 47 0 0% INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 95

73 0 0% 74 0 0%

For efficacy, percentages ranged from 79%, indicating that more than three-quarters of the

Panelists ranked the suggestion in their top twenty-five, to 0%, indicating that no participant selected that particular suggestion within their top twenty-five. Twenty-three percent of the suggestions had percentages of 50% or greater, and an additional twenty-nine percent were ranked between 25% and 50% of the Panel Members’ top twenty-five.

Table 6.

Item Percentages and Frequencies for Panel Member Rankings (Effectiveness)

Suggestion Frequency of Panel Members Percentage of Panel Number Who Ranked Item Members Who Ranked Item 1 12 86% 50 11 79% 4 10 71% 16 10 71% 17 10 71% 24 10 71% 32 10 71% 37 10 71% 2 9 64% 20 9 64% 44 9 64% 47 9 64% 52 9 64% 3 8 57% 8 8 57% 10 8 57% 11 8 57% 30 8 57% 46 8 57% 53 8 57% INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 96

5 7 50% 6 7 50% 15 7 50% 19 7 50% 21 7 50% 28 7 50% 29 7 50% 33 7 50% 39 7 50% 45 7 50% 51 7 50% 18 6 43% 40 6 43% 41 6 43% 42 6 43% 9 5 36% 13 5 36% 25 5 36% 34 5 36% 35 5 36% 22 4 29% 26 4 29% 38 4 29% 49 4 29% 7 3 21% 14 3 21% 27 3 21% 31 3 21% 54 3 21% 12 2 14% 23 2 14% 43 2 14% 48 2 14% 36 1 7%

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 97

For effectiveness, percentages ranged from 86%, indicating that the suggestion was ranked in the top twenty-five by 12 out of the 14 Panel Members, to 7% indicating that only one Panel

Member ranked the suggestion in the top twenty-five. Fifty-seven percent of the suggestions had percentages of 50% or greater, and an additional twenty-four percent of suggestions were ranked between 25% and 50% of the Panel Members’ top twenty-five.

The final procedure in the second round determined which suggestions would be eliminated and which suggestions would be retained in both the compiled lists of suggestions for efficacy and effectiveness. For this study, the consensus cut-off was previously set at 80% agreement for suggestions ranking among all Panel Members for the how component (Ulschak,

1983). That percentage was not achieved in the second round; thus, the researcher decided to continue to the third and final round of data collection, wherein stability testing would occur regardless of the level of consensus reached. In an effort to usher the Panel further towards consensus, suggestions were eliminated that were not ranked in the top twenty-five by at least

25% of the Panelists. The decision was made to maintain suggestions that were ranked in the top twenty-five by at least 25% of the Panel in order to maintain the most inclusive list for the third and final round. For efficacy, 38 suggestions were removed from the compiled list of suggestions, and 43 suggestions were retained. For effectiveness, 10 suggestions were removed from the compiled list of suggestions, and 44 suggestions were retained. The revised lists of top twenty-five suggestions for efficacy and effectiveness are provided in Table 7 and Table 8 below.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 98

Table 7.

Round Two - Revised Comprehensive List of Suggestion (Efficacy)

Suggestion Efficacy Suggestions Number 1 Individual Psychology must publish outside of Adlerian-based journals. 2 Individual Psychology must specifically define basic Adlerian constructs and the core components of the theory (lifestyle, encouragement, life-task, etc.) in an empirically testable form, and distinguish them from assessments and treatments to design empirical studies based upon those distinctions. 3 Individual Psychology must conduct research establishing and demonstrating that Adlerian counseling (specifically lifestyle assessment) promotes deeper understanding, encourages motivation for change, and is a powerful insight-building tool compared to standard clinical interviews based on DSM/ICD systems and or straight DBT and CBT skills.

4 Individual Psychology must conduct Adlerian based research to demonstrate the efficacy of Adlerian interventions with specific populations. 5 Individual Psychology must develop a stronger research base. 6 Individual Psychology must conduct more outcome-based research. 7 Individual psychology must conduct comparative research (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma). 8 Researchers can work to design better instruments that measure factors affected by intervention with individual psychology and find instruments used in well-designed studies of other clinical models that have already established themselves as efficacious according to the current EBP evaluations standards and use those instruments in studies measuring the efficacy of individual psychology. 9 Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). 10 Individual psychology must utilize established instruments and psychometrics, which have been proven to establish efficacy and measure change, to conduct pre-/post-tests related to the efficacy of specific Adlerian interventions. (Examples of instruments: Becks Depression Inventory; Becks Anxiety Inventory; Early Recollections Rating Scale Manaster/Perryman, Millers; Session Rating Scale; Sullimans Social Interest Scale) INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 99

11 Individual Psychology must operationally define constructs and develop instrumentation that represents and measures those constructs individually, as well as the effects of treatment on those constructs. 12 Individual Psychology must encourage practitioners and researchers to pool their resources and collaborate on research projects. 13 More effort must be put into recognizing supporting and incentivizing (through The North American Society of Adlerian Psychology and external funding) the research efforts (Specifically Empirical Research) of Adlerian researchers and practitioners (who publish within and outside of the Journal of Individual Psychology) 14 Individual Psychology could develop treatment manuals similar to the process that interpersonal psychotherapists have. 15 Adlerian graduate programs must prioritize the training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. 16 Individual Psychology must increase professional development opportunities to have more training in research. At The North American Society of Adlerian Psychology and local conferences there can be specific pre-/post-conference workshops where individuals get specific training on research process and statistics. Specific strand at the conference could be offered on research, maybe through open forums on research ideas, or sharing research results. 17 When the Journal of Individual Psychology receives an empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. Therefore, the process is more encouraging and it means more publications of empirical research. 18 Individual Psychology must offer auxiliary to the conferences, specific training, workshop, or conference on research in Adlerian theory (Gestalt practitioners are doing this). 19 Individual Psychology must utilize research design that is consistent with established EBP criteria. 20 Individual psychology must focus research efforts on testing fundamental hypotheses based on the theory of individual psychology in order to develop a more solid literature base. 21 Individual Psychology must review, emphasize, replicate, redesign, and utilize the Adlerian empirical literature and data that is presently available regarding the efficacy and influence of Adlerian concepts, interventions, and instruments to demonstrate of the current efficacy of individual psychology. 22 Given the influence that individual psychology has had on cognitive behavioral therapy, individual psychology can utilize the evidence supporting CBT’s efficacy and demonstrate its own efficacy by distinguish itself from CBT based on influence and effects that Adlerian techniques (lifestyle assessment) have on the therapeutic process. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 100

23 Individual Psychology must commission several methodology experts and/or establish an executive research planning and oversight team to establish a study design that meets the required EBP evaluation standards for efficacy, and to provide evaluation over research projects. 24 Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 25 Individual Psychology must manualize specific individual psychology interventions such as push-button technique, reflecting as if, three-step emotional change trick and other Adlerian approaches. 26 Individual Psychology must offer research grants from The North American Society of Adlerian Psychology (Clonick) and assist its members in seeking external grants. 27 Individual Psychology must conduct meta-analysis studies of empirical literature regarding Adlerian constructs and treatment interventions. 28 Individual Psychology must teach the process for submitting experimental research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting experimental research about the efficacy of specific techniques of individual psychology and submitting the results to SAMSHA for inclusion as Evidenced-Based Practice on the national register. 29 Researchers should measure short term and long-term effects of Individual Psychology interventions. 30 An expert in the application of individual psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity. 31 Individual Psychology must identify specific interventions to be extensively researched such as lifestyle assessment interpretation, use of metaphors (Kopp metaphor intervention), use of paradox, interpretation of ER’s, use of encouragement, use of stories imagery techniques such as push button, reflecting as if, interpreting BASIS-A) 32 Individual Psychology must utilize training videos/appropriate supervision to develop treatment fidelity. 33 Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk-therapy strategies compared to “treatment as usual”, a no-treatment group, or a waiting list group of clients who are not currently receiving care, longitudinally if possible. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 101

34 Individual psychology must explore and utilize a broad empirical literature base (non-Adlerian literature) from models outside of Adlerian psychology to support Adlerian concepts that are researched in other models (concepts such as belonging, social connectedness, social interest, and family constellation and atmosphere that are shown to be relevant aspects of clinical models from the CBT approach). 35 Individual Psychology needs to view the challenges of demonstrating efficacy as an opportunity to re-examine its methodology and/or modify the clinical model/interventions. 36 Adlerians must present their experimental research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians in conducting experimental research related to the efficacy of techniques coming from individual psychology. 37 Individual Psychology must plan a standardized intervention protocol for all sites that would be used during clinical trials. 38 Individual Psychology must utilize 20-30 therapists (trainees and experienced clinicians) at more than 5 or more sites with approximately 300 clients in efficacy-based research. 39 Treatment manuals must be constructed for treatment with in each of the broad groups for use with specific populations, various clinical diagnosis, and problem areas within each broad group (i.e., individual work with adults who experience depression, group work with teens with anxiety, family work with step families who have experienced trauma, etc.). 40 Individual Psychology should look at previous studies and methodologies used by brief dynamic theories to develop research methodologies to specify intended outcomes and evaluate the efficacy of individual psychology to meet those outcomes. 41 Individual Psychology must reach out to the international community to become involved in empirical research (this is how Gestalt research programs have started increasing empirical research). 42 Individual Psychology must establish an Adlerian research task force to seek groups/practices to implement experimental studies. 43 Individual psychology needs to conduct research using double blind randomized control trials.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 102

Table 8.

Round Two - Revised Comprehensive List of Suggestion (Effectiveness)

Suggestion Effectiveness Suggestions Number 1 Individual Psychology must conduct outcome studies 2 Individual Psychology must teach the process for submitting qualitative research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting qualitative research about the clinical effectiveness of specific techniques individual psychology and submitting the results to SAMSHHA for inclusion as Evidenced-Based Practice on the national register. A preponderance of qualitative research demonstrating clinical applicability can result in acceptance. 3 Individual Psychology must utilize established measures besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social). 4 Individual Psychology must utilize strong partnership and a mentorship process to support research efforts. 5 Individual Psychology may utilize clinics and locations connected with Adler graduate programs to conduct research do to the readably available training and supervision that these clinics may be able to offer. 6 Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) 7 Individual Psychology must utilize the published case studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. 8 Individual Psychology must operationally define Individual Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner. 9 Individual Psychology must acquire, align with, and utilize resources such as trained professionals and supportive research institutions to conduct controlled studies. 10 Adlerian fidelity measures should be created and utilized (similar to the Cognitive Therapy Rating Scale) that measures the competence of the clinician using Adlerian therapy, as a means to ensure treatment fidelity among clinicians and treatment provided in research studies. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 103

11 Individual Psychology must conduct Efficiency studies (practice based Evidence studies) that are concerned with real world applications of Individual Psychologies treatment model in everyday treatment settings, and focus on session-to-session client self-comparison rather than comparing client outcomes to group means and aggregated client outcomes as used in effectiveness research (Note: Practice-Based- Evidence is the converse of the Evidence-Based-Practice model. At the present time, such studies would be eligible for listing in the National Registry of Evidence-Based Programs and Practices [SAMSHA], but not in the Research-Supported Psychological Treatments (APA- Division 12). 12 Individual Psychology must have the North American Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. 13 Individual Psychology must have Adlerians present their qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. 14 Individual Psychology must highlight qualitative research activities at the continental annual conference of The North American Society Of Adlerian Psychology to bring attention to the importance of these activities to support the clinical effectiveness of individual psychology. 15 Individual Psychology must have Adlerian faculty focus even more on teaching and encouraging Adlerian-oriented students the skills that they will need to conduct quantitative studies and case studies that support the clinical effectiveness of individual Psychology. 16 Researchers can partner with practitioners and international researchers in order to conduct outcome studies regarding the effectiveness of individual psychology in a variety of situations and with clients and clinicians with diverse backgrounds. 17 The North American Society Of Adlerian Psychology must provide grants and funding for researchers to build a program evaluation model. 18 Individual Psychology must develop and/or utilize an existing program evaluation model (CBT and IPT have already established evaluation models) to look at inputs to identify client, clinician, and setting characteristics, while specifying treatment and alterations while measuring outcomes. 19 Individual Psychology researchers need to go outside of IP and take steps to build a research base similar to the process used by other empirically supported treatments (cognitive therapy; behavior therapy). 20 The North American Society of Adlerian Psychology and individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. 21 Individual Psychology must create instrumentation to measure Individual Psychologies constructs so outcome work can commence. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 104

22 Individual Psychology must utilize existing instruments (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. 23 Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 24 Data Collection in future studies should include collecting data regarding the setting where treatment is provided (inpatient/outpatient/school), and other variables that will influence outcomes (medication, other therapeutic services being received, support system, support group, clients stage of change, etc.). 25 Individual Psychology must set a minimum level of competency (measured via a constructed Adlerian fidelity measure) to be able to participate in specific empirical studies. 26 Given the only real difference between accepted EBP’s and individual psychology is verbiage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. 27 Individual Psychology must utilize the variety of settings where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. 28 Individual Psychology should start a workgroup to develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual. 29 Individual Psychology must conduct comparative outcome studies that use several different levels of a patient variable (i.e., very complex clinical presentation, moderate clinical presentation, simple clinical presentation) receiving Adlerian treatment compared to a no treatment group, and have a large enough sample sizes that the researcher can gather information about patient variables such as gender/sex; culture; age/developmental level and in the analysis of the results group the clients accordingly. 30 Individual Psychology must implement practice-based research at mental health agencies and private practice offices where there are Adlerian clinicians practicing. 31 Individual Psychology must have the Theory, Research, and Teaching TRT section of the North American Society for Adlerian Psychology implement a research team approach to conduct qualitative research related to the clinical effectiveness of individual psychology. This research team could be coordinated through the TRT listserve. 32 Individual Psychology must conduct studies with populations that might have a long-term financial benefit from therapy (school children, prisoners with dual diagnoses, etc.). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 105

33 Individual Psychology must conduct comparative studies that have specific elements altered in the administration of the treatment (with specific elements changed in each group). For instance, Adlerian play therapy with and without parent consultation; Adlerian play therapy with parent consultation compared to Adlerian play therapy with teacher consultation; Adlerian play therapy that lasts 16 sessions, compared to Adlerian play therapy that lasts 30 sessions; twice a week sessions compared to once a week sessions; etc. 34 Individual Psychology must conduct comparative outcome studies that measure the effect that different clinicians have on the treatment outcomes by collecting data about professional identity, clinical experience, measures of clinical skill, fidelity measures, graduate degrees held, licensure status, numbers of years using Adlerian techniques, culture/ethnic background, gender, sexual orientation, and age of clinician in the demographic information gathered, and use these in the variables used in the data analysis in order to monitor the effect of each clinicians on treatment outcomes among clients (large sample size so that the subgroups will have a large enough number of subjects to be relevant in the data analysis). 35 In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 36 Individual Psychology must select one intervention/technique and develop a mode for treating specific types of problems and then conduct research regarding the effectiveness of the interventions in multiple settings 37 Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 38 Individual Psychology must understand that there are various methods to demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 106

39 In order to establish effectiveness individual psychology must: develop treatment protocols and manuals for single interventions; Disseminate these to everyone (Adlerian or not) freely; Establish research training for Adlerians; Pair researchers with practitioners; Conduct outcome research on effectiveness; Publish the findings; and Repeat the process. 40 Adlerians already respect the influence patient variables (age, gender, sexual orientation, e.g.) have on treatment outcomes, but need to develop means to quantify this influence that maintains a respect for each person’s holistic way of being and uniqueness. 41 Individual Psychology must teach specific concepts of qualitative research through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. Instructions that would emphasize supporting the clinical effectiveness of individual psychology and financial feasibility of the application of Adlerian techniques would be an important aspect of the concepts taught. 42 Individual Psychology must utilize Miller’s session Rating Scale to demonstrate that individual psychology approach is satisfying and that clients are willing to participate in counseling sessions. 43 Individual Psychology must develop a survey questionnaire and administer it to 20 Adlerian counselors, 20 CBT counselors, and 20 Eclectic counselors to evaluate counselor’s differential perception of their outcomes based on counselors ratings to demonstrate there is no difference between theoretical orientations related to treatment effectiveness. 44 In order for clinicians to participate in effectiveness studies individual psychology should require clinicians to meet certification requirements including a minimum number of training hours, meeting a minimum level of competency on a developed Adlerian fidelity measure, and submission of counseling video demonstrating the use of Adlerian techniques that would be evaluated utilizing an established Adlerian therapy scale.

In preparation for the third round, the two reduced lists of suggestions were emailed to the 14 Panel Members from the second round a week before the third round was to begin.

Accompanying each of the two lists of suggestions forwarded to Panel Members was a report containing the statistical findings of the second round data analysis (Appendix G). Specifically, the report contained tables for each suggestion that provided Panel Members with the following information: INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 107

• the name of the suggestion

• the rank order that the Panel Member reviewing the report had assigned the

suggestion

• the frequency and percentage of Panel Members who ranked the suggestion

within their top twenty-five

• the Likert rating the Panel Member reviewing the report had assigned the

suggestion for can component

• the median and interquartile range of all Panel Members’ Likert ratings for the

can component regarding the specific suggestion

• the unique Likert rating the Panel Member reviewing the report assigned the

suggestion for the can component

• the median and interquartile range of all Panel Members Likert for the should

component regarding the specific suggestion.

• the unique Likert rating the Panel Member reviewing the report assigned the

suggestion for the should component

An example of one of the suggestion tables is displayed in Figure 3 below.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 108

Figure 3. Example of suggestion table from the report

Effectiveness 1: Individual Psychology must conduct outcome studies Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Number Percentage Ranked the Suggestion in the Top 25 12 86%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquarti Rating* Rating* Range Rating** Rating* le Range * inserted> 4 1 2 3 5 6 7 *Can Neither Definitely Mostly Somewhat Somewhat Mostly Definitely Component: Can or Cannot Cannot Cannot Can Can Can Cannot 7 1 2 3 4 5 6 A Great ** Should Absolutely Minimally Slightly Somewhat Moderately Very Deal Component No Beneficial Beneficial Beneficial Beneficial Beneficial Of Benefit Benefit

The report was provided as a means for Panel Members to familiarize themselves with the overall trends towards consensus that were emerging and for each Panelist to reflect on their own rating and ranking of suggestions compared to the group. Panelists were asked to review the report to better understand how their fellow Panel Members rated and ranked each of the retained suggestions to the research questions and to compare their own rankings and ratings of the suggestions to those of the group.

Round III

After being provided a week to review the report from the second round of the Delphi study, instructions for the third round as well as a link to the third round Qualtrics survey INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 109

(Appendix I) were disseminated via email (Appendix H) to the 14 remaining Panel Members.

The third round of the study focused on reaching further consensus regarding the how, can, and should components of both the research questions. As in round two, Panelists were asked to conduct Likert scale ratings of individual suggestions according to their perceived feasibility

(can component) and benefit (should component). Panelists were once again instructed to rank order the suggestions in each of the two lists based on their perceived utility (how component).

In an effort to further usher the Panel towards consensus, in the third round, Panelists were asked to rank order only their top ten rather than top twenty-five suggestions. Of the remaining 14

Panel Members, 12 Panel Members completed and returned the third round instrument. The loss of two Panel Members’ responses was anticipated, and the study was designed to adjust for attrition. This limitation to generalizability is noted and will be addressed in the next chapter.

After Panel Members’ ratings and rankings of suggestions had been recorded, data collection concluded, and data analysis began. Results were computed using SPSS statistical software and are reported below. Item median scores and interquartile ranges from the third round for the can and should component for efficacy were calculated and are reported in descending order in Table 9 below.

Table 9.

Round Three - Item Median Scores and Interquartile Ranges for the Can and Should

Components (Efficacy)

Efficacy Can Component Efficacy Should Component Suggestion Median Interquartile Suggestion Median Interquartile Number Range Number Range 1 7.0 0.0 1 7.0 0.0 5 7.0 1.0 5 7.0 0.8 6 7.0 1.0 6 7.0 1.0 27 7.0 1.0 7 7.0 1.0 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 110

2 6.5 1.0 23 7.0 2.0 25 6.5 2.0 2 6.5 1.8 30 6.5 2.0 19 6.5 1.0 42 6.5 1.0 30 6.5 2.0 3 6.0 0.0 31 6.5 2.8 4 6.0 1.8 33 6.5 1.0 8 6.0 1.0 42 6.5 2.8 9 6.0 1.0 3 6.0 2.0 10 6.0 1.0 4 6.0 1.8 11 6.0 1.5 9 6.0 1.8 12 6.0 1.0 10 6.0 2.0 14 6.0 1.8 11 6.0 2.0 15 6.0 2.0 13 6.0 3.8 16 6.0 1.0 14 6.0 2.5 17 6.0 1.8 15 6.0 0.5 18 6.0 0.8 20 6.0 1.0 19 6.0 1.8 25 6.0 2.8 20 6.0 1.8 26 6.0 2.0 21 6.0 2.0 27 6.0 1.0 22 6.0 1.0 29 6.0 2.0 24 6.0 1.5 37 6.0 1.8 26 6.0 1.8 43 6.0 1.8 28 6.0 1.0 8 5.5 1.8 29 6.0 1.0 12 5.5 3.0 31 6.0 2.0 16 5.5 3.0 33 6.0 1.5 18 5.5 1.8 34 6.0 1.8 21 5.5 2.0 36 6.0 1.0 28 5.5 2.8 40 6.0 2.0 32 5.5 2.8 41 6.0 2.0 34 5.5 2.8 7 5.5 2.0 17 5.0 2.8 13 5.5 1.8 22 5.0 2.8 23 5.5 1.8 24 5.0 1.8 32 5.5 2.0 35 5.0 3.0 35 5.5 1.8 36 5.0 2.5 37 5.5 1.0 38 5.0 3.8 38 5.0 0.8 39 5.0 2.8 39 5.0 1.5 40 5.0 1.8 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 111

43 5.0 1.0 41 5.0 1.8

Median scores of suggestions for the can component ranged from 7 (definitely can) to 5

(somewhat can), and interquartile scores ranged from 2 to 0. The lowest interquartile ranges

(those indicating the most consensus) generally occurred among the suggestions with the highest median Likert ratings. Median scores of suggestions for the should component for efficacy ranged from 7 (a great deal of benefit) to 5 (moderately beneficial), and interquartile scores ranged from 3.8 to 0. As in round two, it is important to note the difference between the interquartile range between ratings for the can and should component. The Panel ratings for the can component ranged between 2 and 0, while the ratings for the should component ranged from

3.8 to 0. This difference shows that the suggestions with larger interquartile ranges had a wider range of variability and, thus, indicated less consensus among Panel ratings for those suggestions.

Median scores and interquartile ranges of suggestions from the third round for the can component for effectiveness were calculated and are reported in descending order in Table 10.

Table 10.

Round Three - Item Median Scores and Interquartile Ranges for the Can and Should

Components (Effectiveness)

Effectiveness Can Component Effectiveness Should Component Suggestion Median Interquartile Suggestion Median Interquartile Number Range Number Range 20 7.0 1.0 6 7.0 1.0 22 6.5 1.0 1 6.5 1.0 1 6.0 1.0 4 6.5 2.8 2 6.0 1.0 35 6.5 2.0 3 6.0 1.0 3 6.0 2.0 4 6.0 1.0 5 6.0 1.5 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 112

5 6.0 1.0 8 6.0 2.0 6 6.0 2.0 9 6.0 1.8 7 6.0 1.8 10 6.0 1.8 8 6.0 2.0 11 6.0 1.8 9 6.0 1.0 15 6.0 2.5 10 6.0 0.8 17 6.0 2.8 11 6.0 1.8 18 6.0 1.0 13 6.0 1.5 19 6.0 1.0 14 6.0 1.0 21 6.0 2.0 15 6.0 0.8 22 6.0 1.8 16 6.0 2.0 24 6.0 1.8 18 6.0 0.0 30 6.0 0.8 19 6.0 1.0 33 6.0 1.8 21 6.0 1.0 2 5.5 3.5 23 6.0 1.0 12 5.5 2.0 24 6.0 1.0 13 5.5 2.8 25 6.0 1.8 16 5.5 2.8 26 6.0 2.8 20 5.5 2.8 28 6.0 1.0 28 5.5 1.8 30 6.0 1.8 37 5.5 3.0 33 6.0 1.8 39 5.5 2.8 34 6.0 0.8 7 5.0 1.8 35 6.0 1.8 14 5.0 2.5 36 6.0 1.8 23 5.0 3.8 37 6.0 0.8 25 5.0 3.5 39 6.0 1.0 26 5.0 3.0 41 6.0 1.8 27 5.0 1.8 42 6.0 1.8 29 5.0 1.0 27 5.5 1.8 31 5.0 2.0 29 5.5 1.0 32 5.0 2.8 31 5.5 1.0 34 5.0 0.8 38 5.5 2.8 36 5.0 2.8 43 5.5 1.0 38 5.0 1.8 12 5.0 1.0 42 5.0 3.0 17 5.0 1.0 40 4.5 2.8 32 5.0 0.8 44 4.5 4.0 40 5.0 1.8 41 4.0 3.0 44 5.0 1.8 43 3.0 2.8

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 113

Median scores of suggestions for the can component ranged from 7 (definitely can) to 5

(somewhat can), and interquartile ranges ranged from 2.8 to 0. The lowest interquartile scores

(those indicating the most consensus) generally occurred among the suggestions with the highest median scores. Median scores of suggestions for the should component ranged from 7 (a great deal of benefit) to 3 (slightly beneficial), and interquartile scores ranged from 4 to 0.8. The difference between the interquartile range between ratings for the can and should component should again be noted. The Panel ratings for the can component ranged between 2.8 and 0 while the ratings for the should component ranged from 3.8 to 0. This difference shows that the suggestions with larger interquartile ranges had a wider range of variability and thus indicated less consensus among Panel ratings for those suggestions.

As in round two, Panelists were asked to rank order both lists of suggestions based on the perceived utility (how component) of each suggestion to demonstrate efficacy or effectiveness respectively. As noted previously, Panel Members were asked to only rank order their top ten suggestions rather than their top twenty-five. For efficacy, percentages ranged from 58%, indicating that more than half of the Panel Members ranked the suggestion in their top ten, to

0%, indicating that no Panel Members selected that particular suggestion within their top ten.

Four suggestions (9%) were included in the top ten of 50% or more of the Panel Members’ rankings, and eighteen (40%) suggestions were included between 25% and 50% of the Panel

Members’ top ten. These ranking frequencies and percentiles for efficiency are presented in

Table 11 below.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 114

Table 11.

Round 3 - Item Percentages and Frequencies for Panel Member Rankings (Efficacy)

Suggestion Frequency of Panel Members Percentage of Panel Number Who Ranked Item Members Who Ranked Item 1 7 58% 2 7 58% 4 6 50% 23 6 50% 11 5 42% 19 5 42% 25 5 42% 6 4 33% 7 4 33% 8 4 33% 10 4 33% 13 4 33% 16 4 33% 30 4 33% 5 3 25% 12 3 25% 14 3 25% 15 3 25% 17 3 25% 26 3 25% 27 3 25% 31 3 25% 3 2 17% 9 2 17% 20 2 17% 24 2 17% 33 2 17% 34 2 17% 36 2 17% 37 2 17% 42 2 17% 18 1 8% 21 1 8% INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 115

22 1 8% 28 1 8% 35 1 8% 38 1 8% 39 1 8% 40 1 8% 41 1 8% 29 0 0% 32 0 0% 43 0 0%

For effectiveness, percentages ranged from 58%, indicating that the suggestion was ranked in the top ten by more than half of the Panel, to 0% indicating that no Panel Member ranked the suggestion in their top ten. Six percent of the suggestions were included in 50% or more of the

Panel Members top ten, and an additional forty percent of the suggestions were included between

25% and 50% of the Panel Members top ten. The ranking frequencies and percentages are presented in Table 12 below.

Table 12.

Round 3 - Item Percentages and Frequencies for Panel Member Rankings (Effectiveness)

Suggestion Frequency of Panel Members Percentage of Panel Members Number Who Ranked Item Who Ranked Item 6 7 58% 11 7 58% 3 6 50% 1 5 42% 15 5 42% 21 5 42% 22 5 42% 35 5 42% 5 4 33% 8 4 33% 10 4 33% 29 4 33% INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 116

39 4 33% 2 3 25% 13 3 25% 19 3 25% 26 3 25% 28 3 25% 30 3 25% 36 3 25% 44 3 25% 4 2 17% 7 2 17% 9 2 17% 16 2 17% 17 2 17% 18 2 17% 24 2 17% 33 2 17% 34 2 17% 37 2 17% 38 2 17% 42 2 17% 12 1 8% 14 1 8% 20 1 8% 23 1 8% 27 1 8% 32 1 8% 40 1 8% 25 0 0% 31 0 0% 41 0 0% 43 0 0%

While reviewing the outcome data from the third round, the reduction in consensus that occurred regarding the how component is noteworthy. For efficacy, the highest level of consensus in the second round (78%) dropped in round three to 58%. Further, in the third round, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 117

nine percent of efficacy suggestions were ranked by at least 50% of the Panel, in contrast to twenty-three percent of suggestions that were ranked by 50% of the Panel in the second round.

Similarly, for effectiveness in round three, the highest level of consensus reached for a suggestion was 58% compared to the highest percentage of consensus in round two which was

86%. Additionally, in the third round there were six percent of suggestions that were ranked by at least 50% of the Panel compared to fifty-seven percent of the suggestions being ranked by at least 50% of the Panel in round two.

The reduction of consensus could be attributed to protocol changes or Panel Member drop-out between rounds two and three. The researcher’s decision to change the ranking scale from top twenty-five in round two to top ten in round three could also have affected Panel

Member’s rankings. The loss of two Panel Members who completed the second round but did not complete the third round could also have affected the findings. An additional factor that may have impacted consensus between rounds two and three could be attributed to the timing of the third round. The third round was conducted after the annual meeting of the North American

Society of Adlerian Psychology. At this meeting the newly elected president provided a charge to Adlerians to increase empirical research in regards to evidence based practice. Further, at this meeting a newly developed research task force was appointed to focus specifically on developing research endeavors to increase the empirical research backing for individual psychology. These limitations are noted here and will be expanded on in Chapter V.

Although the how component moved away from consensus, several suggestions trended towards consensus and were maintained and will be analyzed in a future section. Additionally, consensus was achieved for several suggestions for both the can and should component. To INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 118

more thoroughly understand the consensus reached for the can and should components, stability testing was conducted.

Stability Testing

In order to test the stability of the research data between subsequent rounds, two nonparametric statistical tests were computed to analyze changes in Panel Members’ ratings between rounds two and three. Stability testing was not computed for the how component, due to the ranking change from top twenty-five in round two to top ten in round three. The Wilcoxon matched-pairs signed-ranks test was utilized to assess whether there were significant differences between Panel Members’ average ratings of suggestions for the can and should components of each research question between rounds two and three. Further, the Spearman’s rank correlation coefficient was utilized to assess the level of association between Panel Members’ ratings and of each suggestion for the can and should components between rounds two and three. In the following section, the results of each of these analyses are reported.

The Wilcoxon matched-pairs signed-ranks test was computed using SPSS statistical software. The stability of the can, and should components of both efficacy and effectiveness were computed for the 12 Panel Members that completed all three rounds. For efficacy, the majority of suggestions for the can and should component ratings were stable from rounds two to three.

Regarding the can component for efficacy, 41 of the suggestions ratings were considered stable, and two suggestions were not considered stable at the 0.05 level of significance. The

Wilcoxon matched-pairs signed-ranks test indicated that the third round ratings for suggestion 30 were significantly higher than the second round ratings (Z = 21, p < .023). Therefore, it was found that the third round ratings for suggestion 30, an expert in the application of individual INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 119

psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity, were significantly higher than the ratings for the same suggestion from the second round. Additionally, the Wilcoxon matched-pairs signed-ranks test indicated that the third round ratings for suggestion 39 were significantly lower than the second round ratings (Z = 2.5, p < .047). These findings indicated that the third round ratings for suggestion 39, Treatment manuals must be constructed for treatment within each of the broad groups for use with specific populations, various clinical diagnosis, and problem areas within each broad group (i.e., individual work with adults who experience depression, group work with teens with anxiety, family work with step families who have experienced trauma, etc.), were significantly lower than the ratings for the same suggestion from the second round.

The should component for efficacy similarly was stable across 41 of the suggestions, and two suggestions were not considered stable at the .05 level of significance. The Wilcoxon matched-pairs signed-ranks test indicated that the third round ratings for suggestion 12 were significantly lower than the second round ratings (Z= 6, p < .046). Therefore, the findings indicated that the third round ratings for suggestion 12, Individual Psychology must have the

North American Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility, were significantly lower than the ratings for the same suggestion in round two. Additionally, the Wilcoxon matched-pairs signed-ranks test indicated that the third round ratings for suggestion 30 were significantly higher than the second round ratings (Z = 15, p < .04). These findings indicated that the third round ratings for suggestion 30, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 120

Individual Psychology must implement practice-based research at mental health agencies and private practice offices where there are Adlerian clinicians practicing, were significantly higher than the ratings for the same suggestion from the second round.

The Wilcoxon matched-pairs signed-ranks test for the how component of the efficacy research question could not be conducted due to the change in ranking scope from top twenty- five in round two to top ten in round three. For the effectiveness research question, the

Wilcoxon matched-pairs signed-ranks test indicated that both the can and should components were stable across all suggestions’ ratings at the .05 significance level. Once again, the test could not be computed for the how component of the effectiveness research question due to the ranking change between rounds two and three.

The Spearman rank correlation coefficient was computed using SPSS statistical software and was used to assess the level of association between Panel Member’s ratings of each suggestion for the can and should components between rounds two and three. The results for the can and should components of efficacy are reported in descending order in Table 13 below.

Table 13.

Stability Testing – Spearman Rank Correlation Coefficient for the Can and Should Components

(Efficacy)

Spearman Efficacy Can Component Spearman Efficacy Should Component Suggestion Spearman Sig N Suggestion Spearman Sig N Number rs Number rs 7 .868** 0.001 12 22 0.672* 0.023 12 24 .868** 0 12 38 .859** 0 12 9 .780** 0.005 12 30 .840** 0.001 12 30 .716** 0.009 12 31 .788** 0.003 12 43 .713** 0.009 12 14 .761** 0.006 12 25 .681* 0.015 12 15 .720* 0.012 12 14 .676* 0.016 12 32 .682* 0.021 12 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 121

39 .653* 0.021 12 1 .680* 0.021 12 4 0.56 0.059 12 17 .680* 0.021 12 28 0.553 0.062 12 35 .611* 0.046 12 29 0.542 0.069 12 37 .610* 0.035 12 36 0.508 0.092 12 24 .608* 0.047 12 5 0.501 0.097 12 43 .606* 0.048 12 31 0.499 0.099 12 29 0.573 0.052 12 40 0.472 0.121 12 39 0.559 0.059 12 35 0.467 0.126 12 34 0.542 0.085 12 38 0.467 0.126 12 19 0.528 0.078 12 37 0.462 0.131 12 25 0.51 0.109 12 6 0.456 0.138 12 23 0.488 0.128 12 19 0.456 0.136 12 9 0.459 0.182 12 22 0.433 0.16 12 40 0.457 0.135 12 26 0.429 0.164 12 21 0.419 0.2 12 23 0.42 0.174 12 42 0.4 0.223 12 8 0.375 0.229 12 28 0.394 0.205 12 27 0.369 0.238 12 18 0.386 0.241 12 33 0.343 0.274 12 41 0.354 0.259 12 17 0.307 0.332 12 27 0.339 0.28 12 16 0.305 0.336 12 26 0.333 0.29 12 15 0.271 0.394 12 36 0.325 0.329 12 2 0.248 0.436 12 13 0.278 0.408 12 41 0.195 0.543 12 12 0.274 0.415 12 12 0.172 0.594 12 2 0.271 0.42 12 18 0.158 0.625 12 33 0.254 0.451 12 3 0.156 0.628 12 16 0.239 0.479 12 32 0.12 0.71 12 7 0.236 0.485 12 21 0.088 0.786 12 6 0.209 0.538 12 34 0.077 0.813 12 20 0.185 0.587 12 13 0.012 0.972 12 11 0.184 0.589 12 1 0 0 12 4 0.175 0.629 12 42 0 1 12 10 0.102 0.779 12 10 -0.042 0.897 12 8 0.055 0.866 12 11 -0.422 0.171 12 3 -0.022 0.948 12 20 -0.435 0.158 12 5 -0.166 0.647 12 ** p < .01 * p<.05

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 122

Regarding the can component, eight of the 43 suggestions were found to be significantly positively correlated (α= .05), indicating that these eight suggestions were stable between rounds two and three. For the should component, 13 of the 44 suggestions were found to be significantly positively correlated (α= .05), indicating that these 13 suggestions were stable between rounds two and three.

The Spearman correlation coefficient test results indicated that four suggestions were found to be significantly positively correlated for both the can and should components of efficacy. Those results are presented in Tables 14 below.

Table 14.

Spearman Correlation Coefficients for the Can and Should Components (Efficacy)

Suggestion N Efficacy Can Component Efficacy Should Component Number Spearman rs Sig Spearman rs Sig 14 12 .676* 0.016 .761** 0.006 24 12 .868** 0 .608* 0.047 30 12 .716** 0.009 .840** 0.001 43 12 .713** 0.009 .606* 0.048

Results of Spearman Correlation Coefficient test results for the can and should components of effectiveness are reported in descending order in Table 15.

Table 15.

Spearman Correlation Coefficient for the Can and Should Components (Effectiveness)

Spearman Effectiveness Can Spearman Effectiveness Should Component Component Suggestion Spearman Sig N Suggestion Spearman Sig N Number rs Number rs 1 0.705* 0.01 12 37 .921** 0 12 8 .852** 0 12 23 .908** 0 12 6 .750** 0.005 12 26 .871** 0 12 38 .680* 0.015 12 6 .867** 0.001 12 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 123

26 .628* 0.029 12 28 .804** 0.002 12 35 .624* 0.03 12 27 .759** 0.004 12 20 .616* 0.033 12 35 .737** 0.006 12 12 .593* 0.042 12 22 .722** 0.003 12 37 0.54 0.07 12 13 .644* 0.024 12 22 0.517 0.085 12 38 .635* 0.026 12 40 0.498 0.099 12 44 .632* 0.037 12 44 0.445 0.147 12 7 .609* 0.047 12 23 0.429 0.164 12 17 .606* 0.048 12 16 0.428 0.166 12 41 0.812 0.001 12 17 0.411 0.185 12 30 0.575 0.051 12 29 0.407 0.189 12 8 0.574 0.051 12 13 0.392 0.207 12 34 0.564 0.056 12 5 0.38 0.224 12 14 0.541 0.069 12 18 0.378 0.226 12 33 0.535 0.073 12 31 0.357 0.254 12 40 0.531 0.076 12 24 0.314 0.321 12 2 0.518 0.085 12 2 0.306 0.334 12 43 0.496 0.101 12 3 0.298 0.348 12 21 0.455 0.137 12 33 0.298 0.347 12 12 0.435 0.157 12 30 0.292 0.357 12 3 0.432 0.161 12 39 0.256 0.422 12 20 0.408 0.188 12 15 0.247 0.44 12 11 0.372 0.234 12 4 0.227 0.477 12 31 0.359 0.252 12 28 0.183 0.57 12 32 0.356 0.256 12 34 0.162 0.616 12 9 0.342 0.277 12 9 0.13 0.687 12 39 0.324 0.304 12 42 0.102 0.752 12 25 0.309 0.329 12 21 0.094 0.773 12 15 0.272 0.393 12 19 0.084 0.796 12 4 0.25 0.433 12 10 0.055 0.864 12 16 0.244 0.483 12 27 0.048 0.882 12 36 0.223 0.485 12 41 0.027 0.933 12 5 0.217 0.499 12 36 0.012 0.971 12 24 0.206 0.521 12 14 -0.021 0.948 12 10 0.198 0.538 12 32 -0.033 0.92 12 18 0.154 0.632 12 11 -0.079 0.807 12 29 0.149 0.644 12 43 -0.095 0.77 12 1 0.115 0.721 12 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 124

25 -0.124 0.7 12 42 -0.178 0.58 12 7 -0.152 0.637 12 19 -0.262 0.412 12 **p<.01 *p<.05

Eight suggestions for the can component of effectiveness were found to be significantly positively correlated (α= .05), and 13 suggestions for the should component were significantly positively correlated (α= .05). The Spearman correlation coefficient results indicated that four suggestions were significantly positively correlated for both the can and should components of effectiveness. Those results are presented in Table 16 below:

Table 16.

Spearman Correlation Coefficients for the Can and Should Components (Effectiveness)

Suggestion N Efficacy Can Component Efficacy Should Component Number Spearman rs Sig Spearman rs Sig

6 12 .750** 0.005 .867** 0.001 26 12 .628* 0.029 .871** 0 35 12 .624* 0.03 .737** 0.006 38 12 .680* 0.015 .635* 0.026

Summary of Findings for Each Component

The results presented in this chapter reported the data collection and analysis that took place in this Delphi study. Although consensus generally decreased between rounds two and three, the analysis indicated several trends towards consensus for the how component and identified consensus for several of the suggestions for the can and should components.

How Component Findings

The how component struggled to reach consensus for both efficacy and effectiveness. A previously established consensus cut-off requiring 80% of the Panel to rank order a specific suggestion was not met for any of the suggestions in either efficacy or effectiveness. Further, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 125

due to the decision to change the ranking scale from top twenty-five in round two, to top ten in round three stability testing was not able to be conducted in order to validate any of the consensus trends that emerged during analysis. Although stability testing was not able to be carried out for the how component, and the 80% consensus cut-off was not met for any suggestions, suggestions that were ranked by more than 50% of the Panel Members in both rounds two and three are reported as trending towards consensus. Additionally, it is important to note that for both efficacy and effectiveness there was significant movement away from consensus between rounds two and three.

Efficacy. As noted previously, the researcher maintained suggestions that were ranked in the top twenty-five, and top ten by at least 50% of the Panel in rounds two and three. Table 17 below illustrate a significant movement away from consensus for the how component of efficacy between rounds two and three.

Table 17.

Movement Away from Consensus Between Rounds Two and Three (Efficacy)

Suggestion Round Round Two Three % % 1 - Individual Psychology must publish outside of Adlerian- 79% 58%* based journals. 2 - Individual Psychology must specifically define basic 71% 58%* Adlerian constructs and the core components of the theory (lifestyle, encouragement, life-task, etc.) in an empirically testable form, and distinguish them from assessments and treatments to design empirical studies based upon those distinctions. 3 - Individual Psychology must conduct Adlerian based 71% 50%* research to demonstrate the efficacy of Adlerian interventions with specific populations. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 126

4 - Individual Psychology must conduct research establishing 71% 17% and demonstrating that Adlerian counseling (specifically lifestyle assessment) promotes deeper understanding, encourages motivation for change, and is a powerful insight- building tool compared to standard clinical interviews based on DSM/ICD systems and or straight DBT and CBT skills. 5 - Individual Psychology must develop a stronger research 64% 25% base. 6 - Individual Psychology must conduct more outcome-based 64% 33% research.

7 - Individual psychology must conduct comparative research 64% 33% (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma).

8 - Individual Psychology must encourage practitioners and 57% 33% researchers to pool their resources and collaborate on research projects.

9 - Individual Psychology must operationally define constructs 57% 17% and develop instrumentation that represents and measures those constructs individually, as well as the effects of treatment on those constructs. 10 - Individual psychology must utilize established 57% 33% instruments and psychometrics, which have been proven to establish efficacy and measure change, to conduct pre-/post- tests related to the efficacy of specific Adlerian interventions. (Examples of instruments: Becks Depression Inventory; Becks Anxiety Inventory; Early Recollections Rating Scale Manaster/Perryman, Millers; Session Rating Scale; Sullimans Social Interest Scale) 11 - Individual Psychology must implement experimental 57% 42% studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). 12 - Researchers can work to design better instruments that 57% 25% measure factors affected by intervention with individual psychology and find instruments used in well-designed studies INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 127

of other clinical models that have already established themselves as efficacious according to the current EBP evaluations standards and use those instruments in studies measuring the efficacy of individual psychology. 13 - More effort must be put into recognizing supporting and 50% 33% incentivizing (through The North American Society of Adlerian Psychology and external funding) the research efforts (Specifically Empirical Research) of Adlerian researchers and practitioners (who publish within and outside of the Journal of Individual Psychology) 14 - Individual Psychology could develop treatment manuals 50% 25% similar to the process that interpersonal psychotherapists have. 15 - Adlerian graduate programs must prioritize the training 50% 25% of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design.

16 - Individual Psychology must increase professional 50% 33% development opportunities to have more training in research. At The North American Society of Adlerian Psychology and local conferences there can be specific pre-/post-conference workshops where individuals get specific training on research process and statistics. Specific strand at the conference could be offered on research, maybe through open forums on research ideas, or sharing research results. 17 - When the Journal of Individual Psychology receives an 50% 25% empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. Therefore, the process is more encouraging and it means more publications of empirical research. 18 - Individual Psychology must offer auxiliary to the 50% 8% conferences, specific training, workshop, or conference on research in Adlerian theory (Gestalt practitioners are doing this). 19 - Individual Psychology must utilize research design that is 50% 42% consistent with established EBP criteria. * - Indicates that the suggestion was ranked by at least 50% of the Panel in rounds two and three and was maintained.

Of the 19 suggestions ranked by 50% of the Panel in round two, only three suggestions were ranked by 50% of the Panel in round three, and the other 16 suggestions were ranked by INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 128

42% or less of the Panel in round three. This demonstrates a significant movement away from consensus between rounds two and three for the how component ranking for efficacy. The three suggestions that were ranked by 50% of the Panel in rounds two and three were maintained in the final list of suggestions for the how component for efficacy. The three suggestions that met this criterion are presented in Table 18.

Table 18.

Suggestions that were Ranked by 50% of the Panel Members in Round Two and Three (Efficacy)

Efficacy Round 2 Round 3 Rank Likert Rating Rank Likert Ratings Order Order Suggestion N % Can Shoul N % Can Should d M I M I M I M I E Q E Q E Q E Q D R D R D R D R I I I I A A A A N N N N 1 - Individual 1 79 7 1 7 1 7 58 7 0 7 0 Psychology must publish 1 % % outside of Adlerian- based journals. 2 - Individual 1 71 6 1 7 2 7 58 6.5 1 7 1 Psychology must 0 % % specifically define basic Adlerian constructs and the core components of the theory (lifestyle, encouragement, life- task, etc.) in an empirically testable form, and distinguish them from assessments and treatments to design empirical studies based upon those distinctions. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 129

4 - Individual 1 71 6 2 6 1 6 50 6 1.8 6 1.8 Psychology must 0 % % conduct Adlerian based research to demonstrate the efficacy of Adlerian interventions with specific populations.

It is important to note while reviewing these results that they are reported as trending towards consensus and not as reaching consensus, given that they did not meet the predetermined 80% consensus cut off.

Effectiveness. Similarly, for effectiveness consensus was not met at the 80% cut-off point for the how component, and stability testing could not be computed due to the change in ranking scale. Additionally, a decrease in consensus was apparent for the how component between rounds two and three. Table 19 illustrates the movement away from consensus between rounds two and three for effectiveness.

Table 19.

Movement Away from Consensus Between Rounds Two and Three (Effectiveness)

Suggestion Round Round Two Three % % 1 - Individual Psychology must conduct outcome studies 86% 42% 2 - Individual Psychology must teach the process for submitting 79% 25% qualitative research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting qualitative research about the clinical effectiveness of specific techniques individual psychology and submitting the results to SAMSHHA for inclusion as Evidenced-Based Practice on the national register. A preponderance of qualitative research demonstrating clinical applicability can result in acceptance. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 130

3 - Individual Psychology must operationally define Individual 71% 33% Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner. 4 - Individual Psychology must utilize the published case 71% 17% studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. 5 - Individual Psychology must implement quantitative studies 71% 58%* that meet the requirements of EBP (utilizing experimental double blind randomized control methodology). 6 - Individual Psychology may utilize clinics and locations 71% 33% connected with Adler graduate programs to conduct research do to the readably available training and supervision that these clinics may be able to offer. 7 - Individual Psychology must utilize strong partnership and a 71% 17% mentorship process to support research efforts. 8 - Individual Psychology must utilize established measures 71% 50%* besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social). 9 - Individual Psychology must acquire, align with, and utilize 64% 17% resources such as trained professionals and supportive research institutions to conduct controlled studies. 10 - Adlerian fidelity measures should be created and utilized 64% 33% (similar to the Cognitive Therapy Rating Scale) that measures the competence of the clinician using Adlerian therapy, as a means to ensure treatment fidelity among clinicians and treatment provided in research studies. 11 - Individual Psychology must conduct Efficiency studies 64% 58%* (practice based Evidence studies) that are concerned with real world applications of Individual Psychologies treatment model in everyday treatment settings, and focus on session-to-session client self-comparison rather than comparing client outcomes to group means and aggregated client outcomes as used in effectiveness research (Note: Practice-Based-Evidence is the converse of the Evidence-Based-Practice model. At the present time, such studies would be eligible for listing in the National Registry of Evidence-Based Programs and Practices [SAMSHA], but not in the Research-Supported Psychological Treatments (APA-Division 12). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 131

12 - Individual Psychology must have the North American 64% 8% Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. 13 - Individual Psychology must have Adlerians present their 64% 25% qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. 14 - The North American Society of Adlerian Psychology and 57% 8% individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. 15 - Individual Psychology researchers need to go outside of IP 57% 25% and take steps to build a research base similar to the process used by other empirically supported treatments (cognitive therapy; behavior therapy). 16 - Individual Psychology must develop and/or utilize an 57% 17% existing program evaluation model (CBT and IPT have already established evaluation models) to look at inputs to identify client, clinician, and setting characteristics, while specifying treatment and alterations while measuring outcomes. 17 - The North American Society Of Adlerian Psychology must 57% 17% provide grants and funding for researchers to build a program evaluation model. 18 - Researchers can partner with practitioners and 57% 17% international researchers in order to conduct outcome studies regarding the effectiveness of individual psychology in a variety of situations and with clients and clinicians with diverse backgrounds. 19 - Individual Psychology must have Adlerian faculty focus 57% 42% even more on teaching and encouraging Adlerian-oriented students the skills that they will need to conduct quantitative studies and case studies that support the clinical effectiveness of individual Psychology. 20 - Individual Psychology must highlight qualitative research 57% 8% activities at the continental annual conference of The North American Society Of Adlerian Psychology to bring attention to the importance of these activities to support the clinical effectiveness of individual psychology. 21 - Individual Psychology must create instrumentation to 50% 42% measure Individual Psychologies constructs so outcome work can commence. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 132

22 - Individual Psychology must utilize existing instruments 50% 42% (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. 23 - Individual Psychology must continue to support and 50% 8% explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 24 - Data Collection in future studies should include collecting 50% 17% data regarding the setting where treatment is provided (inpatient/outpatient/school), and other variables that will influence outcomes (medication, other therapeutic services being received, support system, support group, clients stage of change, etc.).

25 - Individual Psychology must set a minimum level of 50% 0% competency (measured via a constructed Adlerian fidelity measure) to be able to participate in specific empirical studies. 26 - Given the only real difference between accepted EBP’s and 50% 25% individual psychology is verbage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. 27 - Individual Psychology must utilize the variety of settings 50% 8% where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. 28 - Individual Psychology should start a workgroup to develop 50% 25% an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual. 29 - Individual Psychology must conduct comparative outcome 50% 33% studies that use several different levels of a patient variable (i.e., very complex clinical presentation, moderate clinical presentation, simple clinical presentation) receiving Adlerian treatment compared to a no treatment group, and have a large enough sample sizes that the researcher can gather information about patient variables such as gender/sex; culture; age/developmental level and in the analysis of the results group the clients accordingly. 30 - Individual Psychology must implement practice-based 50% 25% research at mental health agencies and private practice offices where there are Adlerian clinicians practicing. 31 - Individual Psychology must have the Theory, Research, 50% 0% and Teaching TRT section of the North American Society for INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 133

Adlerian Psychology implement a research team approach to conduct qualitative research related to the clinical effectiveness of individual psychology. This research team could be coordinated through the TRT listserve. * - Indicates that the suggestion was ranked by at least 50% of the Panel in rounds two and three and was maintained.

Similar to efficacy, the majority (28) of the 31 suggestion ranked by 50% of the Panel in round two for effectiveness were ranked by 42% or less of the Panel in round three. Ultimately, three suggestions were identified as trending towards consensus using the same 50% cut-off criteria for consensus that was utilized for efficacy. The three suggestions that were ranked by at least

50% of the Panel in both rounds two and three and are presented in Table 20.

Table 20.

Suggestions that were Ranked by 50% of the Panel Members in Round Two and Three

(Effectiveness)

Effectiveness Round 2 Round 3 Rank Likert Rating Rank Likert Ratings Order Order Suggestion N % Can Shoul N % Can Shoul d d M I M I M I M I E Q E Q E Q E Q D R D R D R D R I I I I A A A A N N N N 11 - Individual 9 64 5.5 2 5 2 7 58 6 1.8 6 1.8 Psychology must conduct % % Efficiency studies (practice based Evidence studies) that are concerned with real world applications of Individual Psychologies treatment model in everyday treatment settings, and focus on session-to-session client INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 134

self-comparison rather than comparing client outcomes to group means and aggregated client outcomes as used in effectiveness research (Note: Practice-Based- Evidence is the converse of the Evidence-Based- Practice model. At the present time, such studies would be eligible for listing in the National Registry of Evidence- Based Programs and Practices [SAMSHA], but not in the Research- Supported Psychological Treatments (APA- Division 12). 6 - Individual Psychology 1 71 6 2 6.5 1 7 58 6 2 7 1 must implement 0 % % quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) 3 - Individual Psychology 1 71 5.5 1 5.5 2 6 50 6 1 6 2 must utilize established 0 % % measures besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social).

It is important to note while reviewing the final results for the how component of effectiveness that the results are reported as trending towards consensus and not as reaching consensus.

Can and Should Component Findings

The findings from the Wilcoxon matched-pairs signed-ranks test and Spearman rank correlation coefficient stability tests for the can and should components revealed that several INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 135

suggestions for both efficacy and effectiveness were stable. The Wilcoxon matched-pairs signed-rank test demonstrated the stability of rankings across the two rounds by identifying statistical differences between Panel Member’s ratings between rounds two and three. The

Spearman rank correlation coefficient identified suggestions that Likert ratings were significantly and positively correlated between rounds two and three. However, this study required that suggestions reach consensus based on both the interquartile range and proof of stability, so suggestions that reached consensus solely based on the stability testing were not considered as reaching consensus.

Efficacy. For efficacy, eight suggestions were found to have significant positive correlations for the can component, and they are presented in Table 21 and 22. The eight suggestions are presented in descending order based on the interquartile range of the final rounds.

Table 21.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy

Can Component) Suggestions

Number Suggestion

9 Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). 43 Individual psychology needs to conduct research using double blind randomized control trials. 24 Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 136

39 Treatment manuals must be constructed for treatment with in each of the broad groups for use with specific populations, various clinical diagnosis, and problem areas within each broad group (i.e., individual work with adults who experience depression, group work with teens with anxiety, family work with step families who have experienced trauma, etc.). 14 Individual Psychology could develop treatment manuals similar to the process that interpersonal psychotherapists have. 7 Individual psychology must conduct comparative research (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma). 30 An expert in the application of individual psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity. 25 Individual Psychology must manualize specific individual psychology interventions such as push-button technique, reflecting as if, three-step emotional change trick and other Adlerian approaches.

Table 22.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy

Can Component) Suggestions with Spearman’s r, and Likert Ratings

Spearman Efficacy Can Component Round 2 Round 3 Suggestion Spearman Sig N Likert Rating Likert Rating Number rs (can) (can) Median IQR Median IQR 9 .780** 0.005 12 6 2 6 1 43 .713** 0.009 12 5 1 5 1 24 .868** 0 12 6 1 6 1.5 39* .653* 0.021 12 6 2 5 1.5 14 .676* 0.016 12 6 2 6 1.8 7 .868** 0.001 12 5 2 5.5 2 30* .716** 0.009 12 5.5 1 6.5 2 25 .681* 0.015 12 6 1 6.5 2 * Wilcoxon matched-pairs signed-ranks test found the Likert ratings for suggestions 30 and 39 were statistically different between rounds two and three for the efficacy can component (unstable).

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 137

These eight suggestions were positively correlated (indicating stability between Likert ratings for rounds two and three) using the Spearman rank correlation. However, the Wilcoxon matched-pairs signed-ranks test found that suggestions 30 and 39 were significantly different and unstable. Therefore, suggestions 30 and 39 were removed, and the six other suggestions were maintained. An examination of the interquartile range of the Panel Members’ final rating identified the strength of consensus around the six suggestions that were maintained. Table 23 presents the interquartile ranges for the six maintained suggestions in descending order of strength based on their third round interquartile range.

Table 23.

Can component suggestions interquartile range between rounds two and three (efficacy)

Suggestion Round Round Two Three IQR IQR 9- Individual Psychology must implement experimental 2 1 studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). 43 - Individual psychology needs to conduct research using 1 1 double blind randomized control trials. 24 - Individual Psychology must conduct a series of single 1 1.5 case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 14- Individual Psychology could develop treatment manuals 2 1.8 similar to the process that interpersonal psychotherapists have. 7- Individual psychology must conduct comparative research 2 2 (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 138

other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma). 25 - Individual Psychology must manualize specific 1 2 individual psychology interventions such as push-button technique, reflecting as if, three-step emotional change trick and other Adlerian approaches.

For the can component of efficacy, no suggestions had a final interquartile range of 0, two suggestions had an interquartile range of 1 (indicating a strong consensus), and four suggestions had an interquartile range between 1 and 2 (indicating minimal consensus). Thus, all six suggestions that were stable also reached consensus based on their interquartile range. These six suggestions were maintained as the final list of suggestions for the efficacy can component.

For the should component of efficacy, 13 suggestions were found to have significant positive correlations, and they are presented below in Table 24 and 25. The 13 suggestions are presented in descending order based on the final rounds interquartile range.

Table 24.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy

Should Component) Suggestions

Number Suggestion

1 Individual Psychology must publish outside of Adlerian-based journals. 15 Adlerian graduate programs must prioritize the training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. 37 Individual Psychology must plan a standardized intervention protocol for all sites that would be used during clinical trials. 24 Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 139

concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 43 Individual psychology needs to conduct research using double blind randomized control trials. 30 An expert in the application of individual psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity. 14 Individual Psychology could develop treatment manuals similar to the process that interpersonal psychotherapists have. 22 Given the influence that individual psychology has had on cognitive behavioral therapy, individual psychology can utilize the evidence supporting CBT’s efficacy and demonstrate its own efficacy by distinguish itself from CBT based on influence and effects that Adlerian techniques (lifestyle assessment) have on the therapeutic process. 31 Individual Psychology must identify specific interventions to be extensively researched such as lifestyle assessment interpretation, use of metaphors (Kopp metaphor intervention), use of paradox, interpretation of ER’s, use of encouragement, use of stories imagery techniques such as push button, reflecting as if, interpreting BASIS-A) 32 Individual Psychology must utilize training videos/appropriate supervision to develop treatment fidelity. 17 When the Journal of Individual Psychology receives an empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. Therefore, the process is more encouraging and it means more publications of empirical research. 35 Individual Psychology needs to view the challenges of demonstrating efficacy as an opportunity to re-examine its methodology and/or modify the clinical model/interventions. 38 Individual Psychology must utilize 20-30 therapists (trainees and experienced clinicians) at more than 5 or more sites with approximately 300 clients in efficacy-based research.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 140

Table 25.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated (Efficacy

Should Component) Suggestions with Spearman’s r, and Likert Ratings

Spearman Efficacy Should Component Round 2 Round 3 Suggestion Spearman Sig N Likert Rating Likert Rating Number rs (can) (can) Median IQR Median IQR 1 .680* 0.021 12 7 1 7 0 15 .720* 0.012 12 6 2 6 .5 37 .610* 0.035 12 5 2 6.0 1.8 24 .608* 0.047 12 6 3 5.0 1.8 43 .606* 0.048 12 5 2 6.0 1.8 30* .840** 0.001 12 5.5 4 6.5 2.0 14 .761** 0.006 12 6 3 6 2.5 22 0.672* 0.023 12 5 5 5 2.8 31 .788** 0.003 12 6 3 6.5 2.8 32 .682* 0.021 12 6 3 5.5 2.8 17 .680* 0.021 12 5 5 5 2.8 35 .611* 0.046 12 5.5 3 5 3 38 .859** 0 12 5.5 1 5 3.8 * Wilcoxon matched-pairs signed-ranks test found the Likert ratings for suggestions 30 were statistically different between rounds two and three for the efficacy should component (unstable).

These 13 suggestions were significant and positively correlated (indicating stability between

Likert ratings for rounds two and three) based on the Spearman rank correlation. The Wilcoxon matched-pairs signed-ranks test found that the Likert scale rating for suggestion 30 between rounds two and three again was significantly different and unstable. Therefore, suggestion 30 was not maintained, leaving 12 suggestions for the should component of efficacy. The 12 suggestions were evaluated using the same method used from the can component, and five suggestions were found to be stable and reached consensus based on the strength of their interquartile range (Table 26).

Table 26. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 141

Should Component Suggestions Interquartile Range Between Rounds Two and Three

(Efficacy)

Suggestion Round Round Two Three IQR IQR 1 - Individual Psychology must publish outside of Adlerian- 1 0 based journals. 15 - Adlerian graduate programs must prioritize the 2 .5 training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. 37- Individual Psychology must plan a standardized 2 1.8 intervention protocol for all sites that would be used during clinical trials. 24 - Individual Psychology must conduct a series of single 3 1.8 case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 43- Individual psychology needs to conduct research using 2 1.8 double blind randomized control trials. 14 - Individual Psychology could develop treatment 3 2.5** manuals similar to the process that interpersonal psychotherapists have. 22 - Given the influence that individual psychology has had 5 2.8** on cognitive behavioral therapy, individual psychology can utilize the evidence supporting CBT’s efficacy and demonstrate its own efficacy by distinguish itself from CBT based on influence and effects that Adlerian techniques (lifestyle assessment) have on the therapeutic process. 31 - Individual Psychology must identify specific 3 2.8** interventions to be extensively researched such as lifestyle assessment interpretation, use of metaphors (Kopp metaphor intervention), use of paradox, interpretation of ER’s, use of encouragement, use of stories imagery techniques such as push button, reflecting as if, interpreting BASIS-A) INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 142

32 - Individual Psychology must utilize training 3 2.8** videos/appropriate supervision to develop treatment fidelity. 17 - When the Journal of Individual Psychology receives an 5 2.8** empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. Therefore, the process is more encouraging and it means more publications of empirical research. 35 - Individual Psychology needs to view the challenges of 3 3** demonstrating efficacy as an opportunity to re-examine its methodology and/or modify the clinical model/interventions. 38 - Individual Psychology must utilize 20-30 therapists 1 3.8** (trainees and experienced clinicians) at more than 5 or more sites with approximately 300 clients in efficacy-based research. ** - Indicates that this suggestion’s interquartile range was greater than 2 and therefore did not reach consensus.

Two suggestions had an interquartile range at or below 1 which indicated a strong level of consensus; three suggestions had an interquartile range between 1and 2 indicating minimal consensus and seven suggestions had an interquartile range greater than 2 and were considered as not reaching consensus. Therefore, five suggestions for the should component of efficacy that were stable also reached consensus based on the interquartile range of each suggestion in round three, and thus were maintained and validated as reaching consensus.

In reviewing the findings from the can and should components for efficacy, it was identified that two unique efficacy suggestions were stable and validated as reaching consensus for both the can and should components. Thus these two suggestions were considered as both feasible and beneficial based on their interquartile ranges and stability testing. The two suggestions are reported in Table 27 and Table 28 reports the average Likert ratings and interquartile ranges for both suggestions from the second and third round for the can and should INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 143

components. Both tables are presented in descending order based on the final rounds interquartile range.

Table 27.

Suggestions that Were Significantly Positively Correlated for both the Can and Should

Components (Efficacy)

Number Suggestion

24 Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 43 Individual psychology needs to conduct research using double blind randomized control trials.

Table 28.

Suggestions that were Significantly Positively Correlated for both the Can and Should

Components (Efficacy) with Likert Ratings

Efficacy Second Round Likert Rating Third Round Likert Ratings Can Should Can Should Media IQR Media IQR Median IQR Median IQR n n 24 6 1 6 3 6 1.5 5 1.8 43 5 1 5 2 5 1 6 1.8

Effectiveness. Similar to efficacy, the Spearman rank correlation coefficient identified eight suggestions for the can component of effectiveness that had significant positive correlations, and they are presented below in Tables 29. Additionally, Table 30 reports the average Likert ratings, interquartile ranges, and Spearman rs for the eight can component INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 144

suggestion for both the second and third round. The eight suggestions are presented in descending order based on the final rounds interquartile range.

Table 29.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated

(Effectiveness Can Component) Suggestions

Number Suggestion 1 Individual Psychology must conduct outcome studies 20 The North American Society of Adlerian Psychology and individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. 12 Individual Psychology must have the North American Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. 35 In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 8 Individual Psychology must operationally define Individual Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner.

6 Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) 38 Individual Psychology must understand that there are various methods to demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 145

26 Given the only real difference between accepted EBP’s and individual psychology is verbiage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP.

Table 30.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated

(Effectiveness Can Component) Suggestions with Spearman’s rs, and Likert Ratings

Spearman Effectiveness Can Component Round 2 Round 3 Suggestion Spearman Sig N Likert Rating Likert Rating Number rs (can) (can) Median IQR Median IQR 1 0.705* 0.01 12 7 1 6 1 20 .616* 0.033 12 7 1 7 1 12 .593* 0.042 12 5 1 5 1 35 .624* 0.03 12 6 1 6 1.8 8 .852** 0 12 6 1 6 2 6 .750** 0.005 12 6 2 6 2 38 .680* 0.015 12 6 2 5.5 2.8 26 .628* 0.029 12 6 1 6 2.8

The eight suggestions were considered stable based on their significant positive Spearman rank correlation. The Wilcoxon matched-pairs signed-rank test identified no significant differences for any of the suggestions for the can component of effectiveness. Therefore, the eight suggestions were maintained, and their interquartile ranges were evaluated for consensus (Table

31).

Table 31.

Can Component Suggestion’s Interquartile Ranges Between Rounds Two and Three

(Effectiveness)

Suggestion Number Round Round Two Three IQR IQR INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 146

1 - Individual Psychology must conduct outcome studies 1 1 20 - The North American Society of Adlerian Psychology and 1 1 individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. 12 - Individual Psychology must have the North American 1 1 Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. 35 - In future experimental studies individual psychology will 1 1.8 need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 8 - Individual Psychology must operationally define 1 2 Individual Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner. 6 - Individual Psychology must implement quantitative studies 2 2 that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) 38 - Individual Psychology must understand that there are 2 2.8* various methods to demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. 26 - Given the only real difference between accepted EBP’s 1 2.8* and individual psychology is verbiage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 147

* - Indicates that this suggestion’s interquartile range was greater than 2 and therefore did not reach consensus.

Of the eight suggestions identified as stable, three suggestions interquartile range were between 0 and 1, which indicated a strong level of consensus; three suggestions had an interquartile range between 1 and 2, indicating minimal consensus; and two suggestions had an interquartile range larger than 2. The two suggestions that were larger than two were not considered as reaching consensus and were removed. Therefore, six suggestions ultimately reached consensus as feasible options for individual psychology to demonstrate effectiveness.

The six suggestions that reached consensus (i.e., that had an interquartile range between 0 and 2) did so for both the second and third round. Thus, six suggestions were maintained and validated as reaching consensus for this study.

For the should component of effectiveness, 13 suggestions were found to have significant positive correlations, and they are presented in Tables 32. Further, Table 33 reports the average

Likert ratings, interquartile ranges, and Spearman rs for the 13 should component suggestion for both the second and third round. The 13 suggestions are presented in descending order based on the final rounds interquartile range.

Table 32.

Spearman Rank Correlation Coefficient, Significantly Positively Correlated

(Effectiveness Should Component) Suggestions

Number Suggestion

6 Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) 28 Individual Psychology should start a workgroup to develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 148

27 Individual Psychology must utilize the variety of settings where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. 22 Individual Psychology must utilize existing instruments (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. 38 Individual Psychology must understand that there are various methods to demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. 7 Individual Psychology must utilize the published case studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. 35 In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 13 Individual Psychology must have Adlerians present their qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. 17 The North American Society Of Adlerian Psychology must provide grants and funding for researchers to build a program evaluation model. 37 Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 26 Given the only real difference between accepted EBP’s and individual psychology is verbiage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 149

23 Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 44 In order for clinicians to participate in effectiveness studies individual psychology should require clinicians to meet certification requirements including a minimum number of training hours, meeting a minimum level of competency on a developed Adlerian fidelity measure, and submission of counseling video demonstrating the use of Adlerian techniques that would be evaluated utilizing an established Adlerian therapy scale.

Table 33.

Suggestions that were Significantly Positively Correlated for both the Can and Should

Components (Efficacy) with Likert Ratings

Spearman Effectiveness Should Round 2 Round 3 Component Suggestion Spearman Sig N Likert Rating Likert Rating Number rs (can) (can) Median IQR Median IQR 6 .867** 0.001 12 6.5 1 7 1 28 .804** 0.002 12 6 3 5.5 1.8 27 .759** 0.004 12 4 2 5 1.8 22 .722** 0.003 12 6 2 6 1.8 38 .635* 0.026 12 5 3 5 1.8 7 .609* 0.047 12 5 1 5 1.8 35 .737** 0.006 12 5.5 1 6.5 2 13 .644* 0.024 12 5 3 5.5 2.8 17 .606* 0.048 12 5 4 6 2.8 37* .921** 0 12 5.5 2 5.5 3 26 .871** 0 12 5 3 5 3 23* .908** 0 12 5.5 2 5 3.8 44 .632* 0.037 12 4.5 4 4.5 4 * - Suggestions 37 and 23 are the same suggestion that was presented twice in the effectiveness should component Qualtrics survey.

The 13 suggestions ratings for the should component were significantly positively correlated per the Spearman rank correlation calculation. The Wilcoxon matched-pairs signed-rank test again did not identify any unstable suggestions for the effectiveness should component. Therefore, all INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 150

13 suggestions then were evaluated based on their interquartile range to examine the strength of consensus. The results of that evaluation are presented in Table 34.

Table 34.

Should Component Suggestion’s Interquartile Ranges Between Rounds Two and Three

(Effectiveness)

Suggestion Number Round Round Two Three IQR IQR 6 - Individual Psychology must implement quantitative 1 1 studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology)

28 - Individual Psychology should start a workgroup to 3 1.8 develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual.

27 - Individual Psychology must utilize the variety of settings 2 1.8 where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. 22 - Individual Psychology must utilize existing instruments 2 1.8 (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. 38 - Individual Psychology must understand that there are 3 1.8 various methods to demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short- term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 151

better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. 7- Individual Psychology must utilize the published case 1 1.8 studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. 35 - In future experimental studies individual psychology 1 2 will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 13 - Individual Psychology must have Adlerians present 3 2.8* their qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. 17 - The North American Society Of Adlerian Psychology 4 2.8* must provide grants and funding for researchers to build a program evaluation model. 37 - Individual Psychology must continue to support and 2 3* explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 26 - Given the only real difference between accepted EBP’s 3 3* and individual psychology is verbiage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. 23 - Individual Psychology must continue to support and 2 3.8* explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 44 - In order for clinicians to participate in effectiveness 4 4* studies individual psychology should require clinicians to meet certification requirements including a minimum number of training hours, meeting a minimum level of competency on a developed Adlerian fidelity measure, and submission of counseling video demonstrating the use of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 152

Adlerian techniques that would be evaluated utilizing an established Adlerian therapy scale. * - Indicates that this suggestion’s interquartile range was greater than 2 and therefore did not reach consensus.

Of the 13 suggestions identified as stable, one suggestion’s interquartile range was 1, which indicated a strong level of consensus; six suggestions had an interquartile range between 1 and 2, indicating minimal consensus; and six suggestions had an interquartile range larger than 2 indicating no consensus. Therefore, seven suggestions ultimately reached consensus as a beneficial option for individual psychology to demonstrate effectiveness. The seven suggestions were maintained and validated as reaching consensus.

In reviewing the significant findings from the can and should suggestions for effectiveness, two effectiveness suggestions were identified as stable and reaching consensus for both the can and should components. Consequently, those two suggestions were considered as both feasible and beneficial based on their interquartile range and stability testing. The two suggestions are presented in descending order based on the final rounds interquartile range in

Tables 35, and Table 36 presents the second and third round Likert ratings and interquartile ranges of both suggestions below.

Table 35.

Stability Testing –Suggestions that were Significantly Positively Correlated for both the

Can and Should Components (Effectiveness)

Suggestion Effectiveness Suggestions Number 6 Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) 35 In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 153

Table 36.

Suggestions that were Significantly Positively Correlated for both the Can and Should

Components (Efficacy) with Likert Ratings

Effectiveness Second Round Likert Rating Third Round Likert Ratings Can Should Can Should Media IQR Media IQR Media IQR Media IQR n n n n 6 6 2 6.5 1 6 2 7 1 35 6 1 5.5 1 6 1.8 6.5 2

Universal Consensus

For efficacy, no suggestion reached consensus across each of the how, can, and should components. However, for effectiveness one suggestion reached consensus across each of the how, can, and should components and passed both stability tests. Suggestion 6, individual

Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double-blind randomized control methodology), was identified as trending towards consensus for the how component; it also reached consensus for both the can and should components and passed both of the stability tests. Table 37, below indicates the trend and consensus that was reached for effectiveness suggestion six.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 154

Table 37.

How, Can, and Should Components, and Spearman rs for Suggestion Six

Effectiveness - Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) How Component Round 2 Round 3 Frequency % Frequency % 10 71% 7 58%

Can Component Should Component

Spearman rs Sig Spearman rs Sig .750** .005 .867** .001 Likert Rating Likert Rating Round 2 Round 3 Round 2 Round 3 Median IQR Median IQR Median IQR Median IQR 6 2 6 2 6.5 1 7 1 ** - significant at α < .01

The findings indicated that of all of the suggestions, suggestion 6 was the only suggestion to achieve a level of consensus across all three components of the effectiveness research question.

Summary

This chapter presented the findings of this study, the consensus reached for the can and should components, and the trends towards consensus for the how component. In the next chapter, a thorough examination and interpretation of these results will be presented. Further, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 155

Chapter V will offer implications of this research, limitations of this study, and recommendations for future research.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 156

CHAPTER FIVE

Discussion of Results

This study investigated the opinions of Adlerian experts about the future of individual psychology in a post-modern era of mental health driven by EBP. A review of the literature revealed that as the pressure to utilize EBP continues to rise, several fields of mental health, including individual psychology, must identify ways to demonstrate their efficacy and effectiveness in order to remain relevant (Prochaska et al., 2013; Sperry, 1991). The study sought to address this issue by focusing on two specific research questions:

• How, can, and should individual psychology demonstrate efficacy given the current

evidence based practice evaluation standards;

• How, can, and should individual psychology demonstrate effectiveness given the current

evidence based practice evaluation standards.

This chapter begins with a summary of the study and its results. Interpretation and implications of the findings are discussed in reference to existing Adlerian literature, and how individual psychology may address the pressures of EBP. Specifically, implications and interpretation of the findings are discussed regarding the three components of the two research questions: (a) how, (b) can, and (c) should individual psychology demonstrate efficacy given the current evidence based practice evaluation standards; and (d) how, (e) can, and (f) should individual psychology demonstrate effectiveness given the current evidence based practice evaluation standards. Implications will be offered for the field of individual psychology,

Adlerian organizations, and Adlerian researchers. Finally, the chapter will discuss the limitations of the study, recommend future research efforts, and provide a final summary. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 157

Summary of the Study

The purpose of this study was to explore the opinions of Adlerian experts regarding the future of individual psychology in a post-modern era of mental health. An Expert Panel was created, and they responded to three individual rounds of Qualtrics questionnaires. Through this process the Expert Panel identified and reached consensus for several suggestions regarding how, can, and should individual psychology demonstrate efficacy and effectiveness.

The debate over how mental health professionals should address the pressures to meet the standards set by EBP is an issue many contemporary theories of psychology, including individual psychology, continue to struggle with (Chambless & Ollendick, 2001; Emmelkamp, et al., 2014; Fonagy et al., 2005; Tanenbaum, 2005). This issue poses a significant threat to individual psychology which has not to date conducted the needed empirical research to be recognized as an Empirically Supported Treatment (EST). Theories of counseling that do not meet EBP standards may struggle to remain relevant in years to come (Prochaska et al., 2013).

After extensively reviewing the literature, it appears that this study is the first to use the

Delphi method to specifically examine how a particular theory of counseling may address the pressures of EBP. Further, this study is the first to call upon experts in a specific theory to seek consensus regarding how, can, and should efficacy and effectiveness be demonstrated. Lastly, in the field of individual psychology, this study is the first to empirically demonstrate the debate amongst Adlerians regarding EBP and to offer consensual suggestions for how, can, and should individual psychology demonstrate efficacy and effectiveness. Previous literature in the field of individual psychology has offered little to no empirical suggestions regarding how to address

EBP. Previous Adlerian research has focused on offering critiques of EBP, reviews of previous literature, case studies, interviews, or written theoretical pieces regarding the pressures of EBP INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 158

(Carlson & Robey, 2011; Carlson et al., 2006; Shelley, 2000; Sweeney, 1998; Watts, 2013).

Although previous literature has acknowledged the debate among Adlerians regarding EBP

(Dreikurs-Ferguson; 2000; Shelley, 2000; Sperry, 1991; Watts, 2000b), there has yet to be a study that specifically examines the debate within the field. Therefore, this study fills a specific gap in the literature by offering a thorough and comprehensive examination of the EBP evaluation standards, clarifies the debate amongst Adlerians, and identifies and offers suggestions for how, can, and should individual psychology demonstrate efficacy and effectiveness.

The Delphi method was selected for this study based on its ability to facilitate anonymous collection and response analysis of Adlerian experts from geographically and experientially diverse backgrounds. The Delphi method allowed the researcher to empirically examine this pressing issue that has received little prior research attention in the field of individual psychology. Through using the Delphi method, the study identified several suggestions that are useful, feasible, and beneficial for how, can, and should individual psychology demonstrate both efficacy and effectiveness. Sixteen Adlerian Experts initially agreed to complete the Delphi study and 12 Experts completed all three subsequent rounds (this attrition will be further examined in the limitations section of this chapter). Potential panel members’ scholarly activity

(e.g., publications, conference presentations, workshops, etc.) and/or experience in leadership roles within an Adlerian organization were used to indicate and qualify expertise and influence within the field of individual psychology. To be invited to participate in the Expert Panel,

Adlerians who had engaged in scholarly activity related to individual psychology and/or had served in a leadership role within an Adlerian organization were identified. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 159

The Delphi method in this study asked the Expert Panel to complete three rounds of questionnaires via a Qualtrics website. The first round of the Delphi study consisted of open- ended questions related to the how component for both efficacy and effectiveness. Panel members’ free responses were analyzed through classical content analysis, and two comprehensive lists (one for efficacy and one for effectiveness) of suggestions were constructed and utilized in subsequent rounds two and three. The two comprehensive lists of suggestions represented a variety of opinions (both in support or critical of EBP evaluation standards) related to how individual psychology can demonstrate efficacy and effectiveness.

The second and third rounds of the Delphi study utilized quantitative data collection and asked Expert Panel members to rate and rank-order the suggestion from both comprehensive lists. In the second round Panel Members were asked to rate each suggestion from both comprehensive lists using two Likert scales (one pertaining to the can component and the other relating to the should component). In the second round, Panel Members were also asked to rank order the top twenty-five suggestions from each of the two comprehensive lists. All suggestions that were ranked by at least 25% of the Panel were maintained for the third and final round of the study.

The final round asked Panel Members to use the same two Likert scales from the previous round to rate each suggestion from both of the revised lists of suggestions. Panel members were again asked to rank order the suggestions, but in the third round Panel Members were asked to only rank order their top ten suggestions from each of the two lists. After the third round, stability testing was computed for the can and should components (stability testing could not be computed for the how component due to the change in ranking scale between rounds two and three). For the how component, a predetermined consensus cut-off was set requiring INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 160

suggestions to reach 80% consensus (80% of Panel Members rank ordering the suggestion) in order to be maintained. No suggestions for efficacy or effectiveness reached the 80% consensus cut-off for the how component. For the purpose of this study, suggestions that demonstrated a trend towards consensus (ranked ordered by at least 50% of the Panel in rounds two and three) were maintained and reported. Stability testing for the can and should component was used to identify suggestions that were found to be positively correlated (Spearman rank correlation) and not significantly different (Wilcoxon matched-pairs signed-ranks test), and those suggestions were maintained and compared to their interquartile rankings. Ultimately, suggestions that were stable and had an interquartile range between 0-2 were maintained and reported as reaching consensus for the can and should components. Throughout the study, Panel Members’ identities remained anonymous. From the time the first recruitment email was sent to the final data analysis, the entire study took approximately eleven months to complete.

Interpretation and Implications of Results

In this section, a more in-depth interpretation of the results is presented, and implications of the results are offered related to how, can, and should individual psychology demonstrate efficacy and effectiveness. Each of the findings will be interpreted based on the process of reaching consensus across the three rounds for each of the three components. Additionally, for the suggestions that were maintained for each component, an interpretation of the strength and level of consensus reached will be presented. This interpretation will provide a more thorough understanding of the suggestions that ultimately were deemed as consensual. Finally, implications of the findings from the study will be offered related to existing Adlerian literature and how individual psychology may address the pressures of EBP (APA Task Force, 2006; INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 161

Dreikurs-Ferguson, 2000; Shelley, 2000; Sperry, 1991; Sweeney 1998; Watkins & Garnaccia,

1999; Watts, 2013).

The How Component

The how component was investigated in terms of which suggestions were most useful for demonstrating efficacy or effectiveness. Analysis of the findings indicated that at the end of three rounds, the Expert Panel was unable to reach consensus. As previously stated, no suggestions for the how component reached the 80% cut-off; however, suggestions that were ranked by at least 50% of the Panel in both rounds two and three were reported as trending towards consensus and were maintained for this final interpretation. Three suggestions were maintained for efficacy and three suggestions were maintained for effectiveness.

Efficacy. Although consensus was not achieved for the how component for efficacy, the movement that occurred between rounds two and three was important. Between rounds two and three there were substantial changes in the ranking of suggestions. In the final round, only four suggestions were ranked by 50% or more of the Panel, compared to nineteen suggestions that were ranked by more than 50% of the Panel in round two. Of the four suggestions that were ranked by at least 50% of the Panel in round three, one suggestion was only ranked by 43% of the Panel in round two.

A unique observation between rounds two and three was the substantial reduction in ranking by the Panel on the following suggestion:

Individual Psychology must conduct research establishing and demonstrating that

Adlerian counseling (specifically lifestyle assessment) promotes deeper understanding,

encourages motivation for change, and is a powerful insight-building tool compared to INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 162

standard clinical interviews based on DSM/ICD systems and or straight DBT and CBT

skills.

This suggestion dropped in consensus from 71% of the Panel ranking it in their top twenty-five in round two to 17% of the Panel ranking it in their top ten in round three. Although 15 suggestions moved away from consensus, the three suggestions that were ranked by at least 50% of the Panel in rounds two and three consistently trended towards consensus in both rounds; that is, they were consistently perceived by the Panel as being a useful means for how individual psychology could demonstrate efficacy.

Implications of how component suggestions trending towards consensus (efficacy).

The following suggestion indicated the importance of individual psychology disseminating its literature to a wider audience: Individual psychology must publish outside of Adlerian-based journals. Many Adlerians believe that a viable option for enabling individual psychology to demonstrate its continued relevance is to publish existing literature in non-Adlerian based major journals and other publications (Dreikurs-Ferguson, 2000; Shelley, 2000). This would assist

Adlerians in spreading information regarding the continued relevance, influence, and impact that

Adlerian ideals have had on the modern mental health field. Some Adlerians have been critical of Adlerian clinicians and researchers exclusively publishing their research in Adlerian journals

(Bitter, 2016; Sperry 2000). They argue that only publishing in Adlerian journals has limited the spread of Adlerian ideals and research, because most Adlerian journals are primarily read and cited by other Adlerians. This study indicated a perceived Panel Member view that if Adlerians would publish more outside of Adlerian-based journals, individual psychology would better be able to demonstrate its efficacy and continued relevance to non-Adlerians. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 163

In previous literature, a major critique of individual psychology has been related to

Alfred Adler and his followers’ neglect to operationally define the major constructs in empirically measurable terms (Corey, 2005; Jones-Smith, 2012; Sweeney, 1998; Watts 2013).

Further, Adlerians have historically struggled to differentiate between the use of assessments as treatment tools versus using assessments as outcome measures (Bitter, 2016). The Expert

Panel’s agreement in this study on the following suggestion points to on means by which

Adlerians could demonstrate efficacy:

Individual Psychology must specifically define basic Adlerian constructs and the core

components of the theory (lifestyle, encouragement, life-task, etc.) in an empirically

testable form, and distinguish them from assessments and treatments to design empirical

studies based upon those distinctions.

This suggestion was ranked by the majority of the Expert Panel in their top five suggestions in round three, indicating that this suggestion was a primary consideration. The suggestion proposes that individual psychology must address the theory’s vagueness and do a better job of operationally defining its constructs and major components. It implies that if operationally defined constructs were more clearly articulated, individual psychology would be better suited to conduct empirical outcome research to better demonstrate its efficacy. Some Adlerians have suggested that operationalizing the major tenants of individual psychology would result in the loss of important aspects from the theory (Shelley, 2000). However, this study suggests that empirically operationalizing the major tenants of the theory in measurable terms is needed to demonstrate individual psychology’s efficacy according to the Expert Panel.

Finally, the Expert Panel trended towards consensus regarding the suggestion: Individual psychology must conduct Adlerian-based research to demonstrate the efficacy of Adlerian INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 164

interventions with specific populations. This finding is in congruence with the EBP and EST evaluation standards that require efficacy research to demonstrate the effect of an intervention on a designated population in order to be recognized as an EST (APA, 2002a; APA Task Force,

2006; SAMSHA, 2016). Ultimately, this study suggests that individual psychology must identify specific populations (sample populations, clinical diagnosis, etc.) and conduct research to demonstrate the effect of Adlerian interventions on change. The Expert Panel agreed that individual psychology must identify populations that Adlerian interventions would be expected to have a positive impact on, and that individual psychology must conduct empirical research in order to demonstrate individual psychology’s efficacy with those populations. Eventually, individual psychology will need to replicate such impact studies in order to meet EBP evaluation standards (Chambless & Ollendick, 2001), but in order to do so, individual psychology must first identify the appropriate populations to study.

Effectiveness. As with efficacy, consensus was not achieved for the how component for effectiveness after the third round, but the trending movement that occurred between rounds two and three was important. Several suggestions that were ranked by a large percentage of Panel

Members in round two dropped below 50% in round three. For example, the following suggestion reached the 80% consensus cut-off in round two (86%), but dropped to 42% of the

Panel ranking it in their top ten in round three: Individual Psychology must conduct outcome studies. Other than the three suggestions that were ranked by 50% or more in rounds two and three, all other suggestions ranked by 50% of the Expert Panel in round two dropped in consensus between rounds two and three for the how component of effectiveness. Compared to efficacy, it was surprising that of the thirty-one suggestions that were ranked by 50% or more in round two, only three suggestions were rated by at least 50% in round three. The three INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 165

suggestions that were ranked by at least 50% of the Panel in rounds two and three consistently trended towards consensus in both rounds. Therefore, the three suggestions are reported as trending towards consensus for the how component of effectiveness.

Implications of how component suggestions trending towards consensus

(effectiveness). The study indicated that individual psychology could demonstrate its effectiveness through engaging in practice-based evidence research in the following suggestion:

Individual Psychology must conduct Efficiency studies (practice based Evidence studies)

that are concerned with real world applications of Individual Psychologies treatment

model in everyday treatment settings, and focus on session-to-session client self-

comparison rather than comparing client outcomes to group means and aggregated

client outcomes as used in effectiveness research (Note: Practice-Based-Evidence is the

converse of the Evidence-Based-Practice model. At the present time, such studies would

be eligible for listing in the National Registry of Evidence-Based Programs and Practices

[SAMSHA], but not in the Research-Supported Psychological Treatments (APA-Division

12).

Given that practice-based evidence research is currently an accepted means to demonstrate effectiveness, and that it is an accepted form of research for some EST registries (SAMSHA,

2016), the Panel tended to agree that individual psychology could use practice-based evidence research to demonstrate its effectiveness. Many Adlerians believe that individual psychology was never intended to be a measurement based, operationalized, and reductionistic form of psychotherapy (Dreikurs-Ferguson, 2000; Shelley, 2000); thus, it is not surprising that the Expert

Panel supported this suggestion. Further, many researchers who have been critical of the EBP movement have advocated for the usefulness of practice-based evidence in demonstrating the INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 166

effectiveness of psychological interventions (Barkham & Mellor-Clark, 2003; Clement, 2013;

Nevo & Slonim-Nevo, 2011; Westfall, Mold, & Fangan, 2007). Practice-based evidence in isolation is currently not a means to become recognized in the APA-Division 12 (2006) registry of Research-Supported Psychological Treatments, but practice-based evidence research would be an acceptable form of research to demonstrate effectiveness for the National Registry of

Evidence-based Programs and Practices (NREPP) developed by SAMSHA (2016). Further, practice-based evidence would help individual psychology demonstrate its usefulness and relevance in a variety of settings and clinical populations. It offers opportunities for researchers opposed to some research methodologies (randomized controlled trials, outcome studies, etc.) and for clinicians to be able to conduct meaningful research to demonstrate individual psychology’s effectiveness.

The second trending suggestion ranked by at least 50% of the Panel in rounds two and three for effectiveness was: Individual psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double-blind randomized control methodology).

The suggestion supports one of the most emphasized empirical forms of research required by

EBP—randomized controlled trials (RCT) (APA Task-Force, 2006; Barkham & Mellor-Clark,

2003; Chambless & Ollendick, 2001; Chan et al., 2010; Reynolds, 2000). Although randomized controlled trials are more closely associated with efficacy-based research (APA Task Force,

2006), more than half of the Panel perceived that individual psychology would be better able to demonstrate its effectiveness if RCT were conducted. The ranking of this suggestion by 50% of the Panel in both rounds demonstrates a trend towards wider acceptance of certain forms of quantitative research that previously had been resisted within individual psychology. The Panel found that if individual psychology wishes to become recognized as an EST, studies of this INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 167

nature will need to be conducted (APA Task Force, 2006; Chambless & Ollendick, 2001;

SAMSHA, 2016).

The third trending suggestion indicated a need to move away from exclusively measuring change through client reports: Individual Psychology must utilize established measures besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social). In order to meet the evaluation standards of EBP, the Panel suggested that individual psychology has to develop and utilize established outcome measures (APA Task Force, 2006). Compared to the objective focus of EBP (Kazdin, 2006), the attempts to develop and utilize assessments in individual psychology have typically focused on gaining a better understanding of clients’ subjective experience (Shelley, 2000; Sweeney et al., 2006). Individual psychology has previously utilized assessments as clinical tools to guide treatment (Leak, 2006; Kalkan, 2009;

Wheeler et al., 1993), but the findings of this study suggest that more emphasis needs to be placed on measuring outcomes, specifically with regard to the Adlerian life-tasks (intimacy, work, social).

Factors contributing to the movement away from consensus for both efficacy and effectiveness. Inasmuch as movement away from consensus was not an intended outcome of this study, the decrease in consensus for the efficacy and effectiveness of the how component requires further interpretation. Several possible explanations for the decrease in consensus for so many suggestions between rounds two and three could have been related to the methodology of the Delphi study. First, the movement away from consensus could have been due to the change in ranking level between rounds two and three from top twenty-five to top ten. This shift could have contributed to the reduction in consensus between rounds two and three due to the number of suggestions that each Panel Member was able to rank. In round two, Panel Members were INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 168

able to rank-order 25 suggestions compared to 10 suggestions in round three. Therefore, in round three Panel Members were unable to assign rankings to 15 suggestions that they had previously been able to rank. This reduced the opportunity for Panel Members to rank order specific suggestions in round three, thus reducing the frequency and percentage of Panel

Members who ranked suggestions. Another contributing factor may have been the raw rank order position that each Panel Member assigned to each suggestion for the second and third round. The previously identified suggestion for the how component of efficacy dropped in consensus from 71% in round two to 17% in round three. In round two, 71% (10 Panel

Members) ranked the suggestion in their top 25. In reviewing the distribution of the 10 Panel

Member's who ranked that suggestion in their top 25, three Panel Members ranked the suggestion in the top ten, five Panel Member's ranked the suggestion between 10 and 20, and two

Panel Members ranked the suggestion between 20 and 25. It stands to reason that the seven

Panel Members who ranked the suggestion in their top twenty-five but ranked the suggestion outside of their top ten in round two would not likely have been expected to increase their ranking of that suggestion in round three to their top ten. As a final methodological explanation for the reduction in consensus, the report of the findings from round two that was sent to Panel

Members before round three could have influenced Panel Members to change their rankings based on the average rankings of each suggestion emerging within the group.

Environmental and extraneous factors may also have contributed to the reduction in consensus. The timing of this study coincided with several developing trends in the field of

Adlerian psychology. Specifically, between rounds two and three the North American Society of

Adlerian Psychology (NASAP) held its annual conference in Minneapolis Minnesota. At this conference, the newly elected president of the Society, Dr. Jim Bitter, made a call in his INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 169

presidential remarks for NASAP to increase and develop empirical research supporting individual psychology in order to address EBP (2016, May). Additionally, Dr. Bitter appointed a presidential task force on Adlerian research to develop and conduct empirical research studies, develop treatment manualization, and construct outcome instruments in an effort for Adlerian interventions to become recognized as EST. As many of the Panel Members were at this conference, Dr. Bitter’ remarks and the work of the presidential task force on Adlerian research could have influenced Panel Member's responses in round three. An additional extraneous factor that could have impacted the reduction in consensus between rounds two and three was the attrition in the subject pool from 14 members to 12 members. The impact on consensus of the loss of two Panel Members’ responses in the third round will remain unknown, however its potential for having an impact on the final data must be considered.

The Can Component

The can component was investigated as a part of this study in order to identify and reach consensus regarding which suggestions would be feasible for individual psychology to implement in order to demonstrate efficacy or effectiveness. Analysis of the study’s findings indicated that at the end of three rounds, the Expert Panel was able to reach stability and consensus among Likert ratings between rounds. After round three, six suggestions for efficacy and eight suggestions for effectiveness were maintained based on stability testing. However, this study required suggestions to demonstrate consensus based on stability testing and interquartile range. The six suggestions for the efficacy can component all had an interquartile range between

0-2 and therefore all six suggestions were maintained. Two of the eight suggestions for the effectiveness can component though had an interquartile range higher than two and were eliminated, thus only six suggestions were maintained. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 170

Efficacy. The same Panel suggestions that were identified and rank ordered for the how component of efficacy were rated by each Panel Member in rounds two and three for the can component. After three rounds, Likert ratings of eight efficacy suggestions for the can component were positively correlated based on the Spearman rank correlation. Two suggestions were eliminated based on the Wilcoxon matched-pairs signed-ranks test, indicating that their

Likert ratings were significantly different between rounds two and three. Of the six remaining suggestions, interquartile ranges demonstrated that all six suggestions had reached consensus.

Implications of can component suggestions (efficacy). While analyzing the six can component suggestions for efficacy, two themes emerged. Each suggestion could be grouped into suggestions related to either a theme of research design or suggestions related to a theme of standardization, manualization, and operationalization. Both of these major themes are consistent with the overall evaluation standards of efficacy research in EBP (APA Task Force,

2006; SAMSHA, 2016). Suggestions 7, 9, 24, and 43 each related to research designs that were deemed as feasible for individual psychology to utilize in demonstrating efficacy:

• Suggestion 7: Individual psychology must conduct comparative research (preferably

longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian

interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.)

and/or no treatment, in working with specific populations and specific problem areas

(individuals experiencing depression; Groups working with anger issues; Families

recovering from trauma).

• Suggestion 9: Individual Psychology must implement experimental studies to examine the

efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for

major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 171

social anxiety, autism, PTSD, and other common clinical presentations (even chronic

health considerations).

• Suggestion 24: Individual Psychology must conduct a series of single case design

experiments such as multiple baseline single case design to quantitatively evaluate both

treatment process and treatment outcomes in a well-controlled application of an

Adlerian-based therapy model, and conduct follow-up or concurrent research, following

an initial single case design, to replicate the same study in a different research lab with a

different principle investigator.

• Suggestion 43: Individual psychology needs to conduct research using double blind

randomized control trials.

Suggestions 14, and 25 referred to potentially feasible opportunities to develop treatment manuals to demonstrate individual psychology’s efficacy:

• Suggestion 14: Individual Psychology could develop treatment manuals similar to the

process that interpersonal psychotherapists have.

• Suggestion 25: Individual Psychology must manualize specific individual psychology

interventions such as push-button technique, reflecting as if, three-step emotional change

trick and other Adlerian approaches.

It is also important to note that each of the six suggestions that reached consensus for the can component for efficacy reached a median rating of between 5 (somewhat can) and 6.5 (between mostly can and definitely can). This suggests that the Panel perceived that each of these suggestions were at least somewhat feasible for individual psychology to implement in order to demonstrate individual psychology’s efficacy. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 172

This study indicated that individual psychology can feasibly develop research studies with more rigorous and empirical research design. Randomized controlled trials, single-case design experiments, and comparative research designs were each supported as potential research that can be conducted. Several of these research designs are approved forms of research that demonstrate efficacy based on the EBP evaluation standards (APA Task Force, 2006; SAMSHA,

2016), and they are considered more empirically rigorous forms of research design than the more common approaches that have currently been taken by individual psychology (Chan, et al.,

2010). These findings are important in that they support the feasibility of individual psychology conducting more empirical research aimed at demonstrating outcome and effect on change.

Several of the suggestions grouped into this theme also emphasized the importance of developing research to investigate the effect of Adlerian interventions on “specific populations,”

“clinical diagnosis,” and “problem areas” with “broad mental health groups” (family counseling, group counseling, and individual counseling). Chambless and Hollon (1998) have noted the importance of specifically describing the population within a study and the treatment setting in order to increase the generalizability and quality of the results. The Panel perceived that it is somewhat to mostly feasible for individual psychology to conduct randomized controlled trials, single-case design experiments, and comparative research with specified samples.

As treatment manualization is required for many forms of high-quality experimental research and emphasized in EBP evaluation standards (APA Task Force, 2006; Chambless and

Hollon, 1998; Chan et al., 2010), it is notable that the Panel in this study perceived that treatment manualization in several forms is feasible. Chambless and Ollendick (2001) noted that

“treatment manuals […] are necessary to provide an operational definition of the intervention under study” in reference to EST evaluation standards (p. 701). The suggestions offered in this INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 173

study indicate that individual psychology could develop treatment manuals for several Adlerian interventions. Treatment manualization not only would increase the internal validity of research studies (Addis & Krasnow, 2000) but would also assist in generalizing the application of

Adlerian interventions (Chambless & Ollendick, 2001). Panelists in this study agreed that it would be feasible for individual psychology to develop treatment manuals similar to those developed by interpersonal psychotherapy. Utilizing the process that EST-approved psychodynamic approaches have used to develop treatment manuals could be a potential guide to assist individual psychology in developing its own treatment manuals. Each of the suggestions relating to manualization were considered as somewhat to mostly feasible.

Effectiveness. Similar to efficacy, effectiveness assessment of the can component utilized the same suggestions that were identified and rank ordered for the how component of effectiveness. The Spearman correlation identified eight suggestions that Likert ratings between rounds two and three were positively correlated, and the Wilcoxon matched-pairs signed ranks test did not identify any suggestion Likert ratings that were significantly different between rounds two and three; thus, no suggestions were eliminated from the eight suggestions. Of the eight suggestions identified as stable, two suggestions had an interquartile range larger than two; therefore, six suggestions ultimately reached consensus as feasible options for ways that individual psychology can demonstrate effectiveness.

Implications of can component suggestions (effectiveness). Three themes emerged from the six can component suggestions for effectiveness that were maintained after analysis.

Each suggestion could be placed into a thematic group of related to research design, dissemination, and or support from Adlerian organizations and institutions. Suggestions 1, 6, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 174

and 20 each emphasize that several research designs are feasible for individual psychology to utilize to demonstrate effectiveness:

• Suggestion 1: Individual Psychology must conduct outcome studies.

• Suggestion 6: Individual Psychology must implement quantitative studies that meet the

requirements of EBP (utilizing experimental double blind randomized control

methodology).

• Suggestion 20: The North American Society of Adlerian Psychology and individual

psychology need to emphasize, highlight, and support the need for empirical support

through outcome research.

Suggestions 8 and 35 related to the dissemination of an operationally defined clinical model and treatment manual:

• Suggestion 8: Individual Psychology must operationally define Individual Psychologies

constructs compared to constructs from other disciplines, and do a better job of

presenting the Adlerian clinical model in a concrete and defined manner.

• Suggestion 35: In future experimental studies individual psychology will need to include

a process where clinicians will be trained with a manualized version of Adlerian

strategies (once developed).

Finally, suggestions 20 and 35 emphasized that Adlerian organizations and institutions offering support for research efforts is feasible and would contribute to demonstrating effectiveness:

• Suggestion 20: The North American Society of Adlerian Psychology and individual

psychology need to emphasize, highlight, and support the need for empirical support

through outcome research. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 175

• Suggestion 35: In future experimental studies individual psychology will need to include

a process where clinicians will be trained with a manualized version of Adlerian

strategies (once developed).

It is also important to note that each of the six suggestions that reached consensus for the can component for effectiveness reached a median rating of between 5.5 (between somewhat can and mostly can) and 7 (definitely can), as this suggests that these six suggestions are at least somewhat feasible options for individual psychology to demonstrate effectiveness.

Outcome research must be conducted in order to demonstrate the clinical effectiveness of individual psychology (Chambless & Hollon, 1998). The Expert Panel believed that it would be feasible for individual psychology to conduct outcome research as well as experimental double- blind randomized controlled trials. Inasmuch as effectiveness research receives less attention regarding the evaluation of EST’s compared to efficacy research (Chambless & Ollendick,

2001), outcome research seems to be a feasible starting point for demonstrating individual psychology’s effectiveness. In order to be recognized as an effective EST, empirical findings must be evaluated for generalizability, feasibility of intervention, and cost effectiveness of treatment (APA Task Force, 2006). Whereas, conducting various forms of rigorous empirical research would be feasible for individual psychology, Adlerians will have to conduct the research in a manner that would demonstrate effectiveness. Each of the Panel suggestions associated with research design were rated as mostly to definitely feasible.

Related to effectiveness, EBP evaluation standards examine the readiness of all evidence and supporting materials (treatment manuals, assessments, etc.) to be disseminated (APA Task

Force, 2006; Chambless & Hollon, 1998; Reynolds, 2000). This study found that once developed, individual psychology could disseminate training protocols in order to train clinicians INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 176

regarding the use of treatment manuals. Further, the Panel indicated that individual psychology could feasibly develop operational definitions of the major constructs, and that it could operationalize the clinical model. The Panel found that individual psychology’s dissemination of treatment manuals and operationally defined explanations of the major tenants and clinical model would assist in demonstrating the overall effectiveness and generalizability of Adlerian interventions. Ultimately, the process of developing treatment manuals and operationally defining the constructs of individual psychology will be time-consuming. However, it is important that the Panel perceived it to be feasible for individual psychology to develop them in a way that can be disseminated so that effectiveness research finding can be generalized and replicated. The Panel considered each of the suggestions related to dissemination to be definitely feasible.

Finally, this study indicated that Adlerian organizations such as the North American

Society of Adlerian Psychology (NASAP) need to support, emphasize, and highlight the need for future research in order to demonstrate individual psychology’s effectiveness. One drawback of pursuing research studies that would be approved by EBP evaluation standards is the expense associated with conducting such studies (Emmelkamp et al., 2014; Guy et al., 2012; Minas &

Jorm, 2010). The Expert Panel agreed that NASAP could assist in funding and supporting new quantitative and approved qualitative research efforts to demonstrate individual psychology’s effectiveness. Quantitative research would be needed in order to meet the evaluation standards of EBP (APA Task Force, 2006), and although qualitative research is not recognized as a strong source of research by EBP evaluation standards, qualitative research would be useful in demonstrating the utility of clinical application, financial feasibility, and overall generalizability of individual psychology (Chan et al., 2010; Chambless & Hollon, 1998; Reynolds, 2000). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 177

Additionally, the Panel supported the need for individual psychology to increase its encouragement and support of Adlerian research efforts. They agreed that as individual psychology develops research initiatives to meet the criteria of EBP, Adlerians and the organizations they are members of will need to support those initiatives. The suggestions related to the need for Adlerian support of research efforts was perceived by the Panel as moderately to definitely feasible.

The Should Component

The should component was investigated in order to identify and reach consensus regarding which suggestions would be beneficial in demonstrating the efficacy or effectiveness of individual psychology. Analysis of the findings from Chapter IV indicated that the Expert

Panel was able to reach stable consensus around the Likert ratings between rounds. After round three, 12 suggestions were identified for efficacy as stable, and 13 suggestions were identified for effectiveness as stable. After examining the interquartile range of each stable suggestion, five suggestions reached consensus for efficacy, and seven suggestions reached consensus for effectiveness.

Efficacy. As with the can component, the should component utilized the same suggestions identified and rank-ordered for the how component of efficacy and each Panel

Member rated each suggestion in rounds two and three. After the third round, and computing both stability tests, 12 suggestions were identified as stable. However, interquartile range analysis indicated that of the 12 suggestions identified as stable, seven suggestions had an interquartile range greater than two; therefore, six suggestions finally reached consensus for the should component of efficacy. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 178

Implications of should component suggestions (efficacy). The five suggestions that reached consensus for the should component of efficacy referenced four of the previously indicated themes that emerged in this study. Suggestion 1 further supported the theme of dissemination as a potentially beneficial means to demonstrating individual psychology’s efficacy:

• Suggestion 1: Individual Psychology must publish outside of Adlerian-based journals.

Suggestion 15 contributed to the theme of Adlerian institutions and organizations supporting research efforts would be beneficial in demonstrating individual psychologies efficacy:

• Suggestion 15: Adlerian graduate programs must prioritize the training of its students in

conducting empirical research, and encourage students to conduct and publish research

utilizing empirical design.

Suggestion 37 added to the emerging theme of treatment manualization as a potential benefit:

• Suggestion 37: Individual Psychology must plan a standardized intervention protocol for

all sites that would be used during clinical trials.

Lastly, suggestions 24 and 43 further contributed to the emerging trend of potential research designs that may be beneficial in demonstrating individual’s psychologies efficacy:

• Suggestion 24: Individual Psychology must conduct a series of single case design

experiments such as multiple baseline single case design to quantitatively evaluate both

treatment process and treatment outcomes in a well-controlled application of an

Adlerian-based therapy model, and conduct follow-up or concurrent research, following

an initial single case design, to replicate the same study in a different research lab with a

different principle investigator. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 179

• Suggestion 43: Individual psychology needs to conduct research using double blind

randomized control trials.

Each of the five suggestions that reached consensus for the should component for efficacy reached a median rating of between five (moderately beneficial) and seven (a great deal of benefit). This indicates that the Panel perceived that each of these suggestions would at least moderately benefit individual psychology in demonstrating efficacy.

The Expert Panel found that individual psychology would benefit from “publishing in more non-Adlerian journals.” This suggestion, that also trended towards consensus for the how component of efficacy, would benefit the field of individual psychology by allowing the field to disseminate any new and existing literature regarding individual psychology’s efficacy to a broader readership (Sperry, 2000). The suggestion is specifically important in relation to any new research that is conducted demonstrating efficacy empirically, as EBP evaluation standards require interventions seeking recognition as an EST to be prepared to disseminate their findings

(APA Task Force, 2006; SAMSHA, 2016). Further, this study indicated that individual psychology also should disseminate non-empirical research. This study suggests that Adlerians opposed to empirical research methods should conduct and disseminate research that utilizes alternative research methods to support individual psychology’s continued relevance (Dreikurs-

Ferguson, 2000; Shelley, 2000). This is important, as some Adlerians remain critical of the research emphasis required by EBP evaluation standards, and argue that individual psychology could demonstrate its efficacy based on previous non-empirical research demonstrating the efficacy of individual psychology (Shelley, 2000). The Expert Panel agreed that publishing in non-Adlerian journals would provide a great deal of benefit to individual psychology. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 180

Suggestion 15 indicated a need for Adlerian institutions to provide more training and overall support for student researchers conducting and publishing empirical research: Adlerian graduate programs must prioritize the training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. This suggestion contributes to the theme that support is needed from Adlerian organizations and institutions for individual psychology to demonstrate its efficacy adequately. As previously indicated with regard to the can component of effectiveness, financial and overall support is needed to encourage experimental research. Given that many forms of EBP approved efficacy research require extensive knowledge regarding the research methodology and implementation

(Emmelkamp et al., 2014), the findings of this study indicate that individual psychology would benefit from Adlerian graduate schools providing training and support to graduate students in conducting and publishing empirical research to demonstrate individual psychology’s efficacy.

This would benefit the field of individual psychology in developing more empirical researchers to contribute to the demonstration of its efficacy. The Panelists in this study indicated that gaining support from Adlerian graduate institutions would be very beneficial for individual psychology in demonstrating efficacy.

Corresponding with the fact that treatment manualization is a contributing factor to both efficacy research (intervention protocol, adherence to treatment manuals, RCT’s, etc.) and effectiveness research (dissemination of treatment manuals, generalizability of intervention effect on outcomes, treatment fidelity, etc.), this study found that developing standardized treatment protocols would benefit individual psychology in demonstrating efficacy (APA Task

Force, 2006; SAMSHA, 2016). EBP approved efficacy research requires that all clinicians participating in efficacy research adhere to a treatment manual and a standardized intervention INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 181

protocol (Chambless & Ollendick, 2001). The Panelists in the current study agreed with the suggestion that individual psychology would benefit if it would develop standardized intervention protocols in coordination with treatment manualization. They concluded that this suggestion would be very beneficial to individual psychology.

Finally, the Panelists in this study agreed that more rigorous forms of research design are both feasible and beneficial options for individual psychology to demonstrate efficacy (Chan et al., 2010). This is important, given that RCT and single case design experiments are considered to be necessary for demonstrating efficacy based on EBP and EST evaluation standards (APA

Task Force, 2006; SAMSHA, 2016). Further, as noted in Suggestion 24, Panelists agreed that individual psychology would benefit from replicating research:

Individual Psychology must conduct a series of single case design experiments such as

multiple baseline single case design to quantitatively evaluate both treatment process and

treatment outcomes in a well-controlled application of an Adlerian-based therapy model,

and conduct follow-up or concurrent research, following an initial single case design, to

replicate the same study in a different research lab with a different principle investigator.

By replicating research, individual psychology would be able to better demonstrate its efficacy through attempting to replicate the same study in different research labs with different principle investigators. Such replication would also bring individual psychology in line with the EBP evaluation standards’ hierarchy of research (Chan et al., 2010). The Expert Panel indicated that these suggestions would be moderately and very beneficial to individual psychology.

Effectiveness. The effectiveness should component, like the efficacy should component, focused on identifying and reaching consensus around suggestions that were perceived as beneficial for individual psychology to use to demonstrate effectiveness. The same suggestions INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 182

that were identified and rank ordered for the how component of effectiveness were utilized for the should component. After three rounds 13 suggestions were found to be stable. Of the 13 suggestions, seven suggestions had an interquartile range equal or less than two and, thus, achieved consensus.

Implications of should component suggestions (effectiveness). The seven suggestions regarding the should component for effectiveness fell into the three thematic groups of research design, treatment manualization, and disseminating existing Adlerian literature. Suggestions 6,

22, 27, and 38 each emphasize that specific research design would benefit individual psychology in demonstrating effectiveness:

• Suggestion 6: Individual Psychology must implement quantitative studies that meet the

requirements of EBP (utilizing experimental double blind randomized control

methodology).

• Suggestion 22 - Individual Psychology must utilize existing instruments (BDI-II, STAI,

Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic

restructuring and other individual psychology interventions have on treatment outcomes.

• Suggestion 27: Individual Psychology must utilize the variety of settings where Adlerians

are represented and the outcome measures available to conduct research in various

settings with various client groups.

• Suggestion 38: Individual Psychology must understand that there are various methods to

demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract

of Seligman's summary of the research: Consumer Reports (1995, November) published

an article which concluded that patients benefited very substantially from psychotherapy,

that long-term treatment did considerably better than short-term treatment, and that INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 183

psychotherapy alone did not differ in effectiveness from medication plus psychotherapy.

Furthermore, no specific modality of psychotherapy did better than any other for any

disorder; psychologists, psychiatrists, and social workers did not differ in their

effectiveness as treaters; and all did better than marriage counselors and long-term

family doctoring. Patients whose length of therapy or choice of therapist was limited by

insurance or managed care did worse. The methodological virtues and drawbacks of this

large-scale survey are examined and contrasted with the more traditional efficacy study,

in which patients are randomized into a manualized, fixed duration treatment or into

control groups.

Suggestions 28 and 35 contributed to the emerging theme regarding the benefit of treatment manualization:

• Suggestion 28: Individual Psychology should start a workgroup to develop an easily

disseminated treatment manual and start recruiting research practitioners to field-test

the treatment manual.

• Suggestion 35: In future experimental studies individual psychology will need to include

a process where clinicians will be trained with a manualized version of Adlerian

strategies (once developed).

Finally, suggestions 7 emphasized that individual psychology would benefit from utilizing existing Adlerian literature to demonstrate effectiveness:

• Suggestion 7: Individual Psychology must utilize the published case studies that

exemplify the use of Adlerian strategies used with clients from various cultures and a

variety of clinical presentations (examining patient variables that influence outcomes that INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 184

are controlled for in the data analysis phase) to demonstrate Individual Psychologies

effectiveness.

It is important to note that each of the six suggestions that reached consensus for the can component for effectiveness reached a median rating of between five (moderately beneficial) and seven (a great deal of benefit). This indicates that the Panel found each of these suggestions to be at least moderately beneficial.

Regarding research design, the Expert Panel found that it would be beneficial for individual psychology to conduct RCT and outcome studies with various clinical groups in a variety of clinical settings in order to demonstrate effectiveness. This study indicated that the field of individual psychology would improve the generalizability of its research if a better job was done specifying the clinical groups and describing the clinical settings in which research is being conducted (APA Task Force, 2006). Additionally, the Expert Panelists in this study agreed that individual psychology would benefit from exploring alternative research designs in addition to designs aimed at demonstrating EBP effectiveness. This finding is congruent with

Adlerian literature reporting that many Adlerians believe that EBP effectiveness research does not accurately demonstrate the effectiveness of interventions in real-world settings (Emmelkamp et al., 2014; Shelley, 2000). Conversely, some Adlerians believe that alternative research designs such as case studies and client outcome satisfaction surveys are more representative of the effectiveness of Adlerian interventions in real-world settings (Shelley, 2000). The general sentiment seems to be that Adlerians should explore all possible methods to demonstrate its effectiveness including practice-based evidence, case studies, interviews, and client outcome satisfaction studies. Additionally, the Panel agreed on the suggestion that Adlerians should:

Utilize existing instruments (BDI-II, STAI, etc.) to demonstrate the effects of lifestyle analysis, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 185

encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. If individual psychology used established measures, Adlerians could demonstrate the effectiveness of common Adlerian interventions. The Panel rated the research design suggestions from moderately beneficial to a great deal of benefit.

Suggestion 28, Individual Psychology should start a workgroup to develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual, and Suggestion 35, In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed) contributed to the emerging theme of treatment manualization.

Findings with regard to the should component of effectiveness indicated that developing a treatment manual that could be readily disseminated and developing training for clinicians on how to utilize treatment manuals once developed would benefit individual psychology. These findings relate to the EBP evaluation standards for effectiveness regarding the ease and readiness of an intervention to be disseminated that are required to be accepted as an EST (APA Task

Force, 2006; Chambless & Hollon, 1998; SAMSHA, 2006). Each of these suggestions were rated by the Panel as being moderately beneficial to individual psychology.

Lastly, the Panel indicated that individual psychology would benefit from utilizing existing literature and assessments to demonstrate individual psychology’s effectiveness. These findings provide support for Adlerians who emphasize the need for individual psychology to utilize the prescribed methods outlined by EBP (Dreikurs-Ferguson, 2000; Shelley, 2000).

Similar to Seligman's (1995) remarks that indicated individual psychology would benefit from exploring all research methods to demonstrate effectiveness (suggestion 38), the Panelists in this study agreed that individual psychology would benefit from demonstrating its effectiveness in INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 186

both EBP approved ways and in utilizing other research designs. Given the debate amongst

Adlerians regarding how individual psychology should address EBP evaluation standards

(Dreikurs-Ferguson, 2000; Shelley, 2000), the consensual finding in this study that individual psychology should do a better job of disseminating existing support of individual psychology offers important direction for the field. The Panelists agreed that individual psychology would be able to demonstrate the clinical application of Adlerian interventions if it used previously published case studies. Panelists indicated that if individual psychology utilized existing literature it would be moderately to very beneficial.

Overall Summary and Implications of Themes Identified

The established (can and should components) and trended (how component) consensus achieved in this study can assist Adlerians in their ongoing efforts to demonstrate the efficacy and effectiveness of individual psychology. Amidst the controversies around developing EBP and EST’s have been debated in the field of individual psychology (Carlson et al., 2006;

Dreikurs-Ferguson, 2000; Nystul, 1991; Shelley, 2000; Sperry, 1991; Watts, 2000a; Watts

2000b), this study provided support and encouraging steps forward for Adlerians on both sides of the debate. The suggestions that reached or trended towards consensus in this study provide individual psychology with a guide based on empirically supported suggestions for how, can, and should Adlerians on both sides of the debate demonstrate efficacy and effectiveness based on

EBP evaluation standards. All suggestions that reached or trended towards consensus relate to four main themes that are supported in the EBP research (APA Task Force, 2006; SAMSHA,

2016): research design; operationalizing, manualization, and standardizing; dissemination; and support. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 187

Research design. In order for any theory or intervention to demonstrate efficacy or effectiveness, EBP evaluation standards require substantiating research evidence that is collected using specific research designs (APA, 2002a; APA Task Force, 2006). For efficacy, interventions must provide evidence from “a sophisticated empirical methodology, including quasi-experiments and randomized controlled experiments” (APA, 2002a, p. 1054). The Expert

Panel in this study agreed that individual psychology would benefit from, double-blind randomized controlled trial and single-case design (multiple-baseline/ pre and post-test) studies to demonstrate its efficacy. They also agreed that such studies could be feasibly conducted.

Based on the APA Treatment Evaluation Guidelines (2002a) both of these research methodologies would be appropriate methodologies for use by individual psychology to demonstrate efficacy. It is of particular importance to note that the participants in this study indicated that RCT and single-case design research would be feasible and beneficial, given that individual psychology has not typically conducted this type of research. Additionally, several suggestions indicated that there is a need for individual psychology to encourage more empirical outcome-based research design. It was suggested that individual psychology could utilize existing instruments that are validated as strong measures of change. Utilizing existing measures could help to facilitate individual psychology's development of outcome research instruments by eliminating the need to start from scratch. Participants in this study also agreed that individual psychology could conduct comparative research, comparing individual psychology to other treatment modalities and/or no treatment groups to demonstrate individual psychology’s efficacy. According to EBP evaluation standards, if individual psychology were to conduct comparative research to demonstrate efficacy, it would need to be done in coordination with INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 188

RCT or single-case design and based on the hierarchy of evidence that is supported by EBP

(Chan et al., 2010).

To determine effectiveness, APA Treatment Evaluation Guidelines (APA, 2002a) indicate that evidence must be evaluated on the basis of its generalizability, feasibility, and cost- effectiveness. Related to generalizability, the Panelists in this study agreed that individual psychology can and should conduct empirical research with specific populations and various clinical DSM-5 diagnoses in order to demonstrate the effectiveness of its interventions on promoting change. By defining the specific population receiving treatment, individual psychology could demonstrate the effect that Adlerian interventions have on behavior change and symptom reduction within those specific populations. The Panelists in this study also indicated that individual psychology should explore other research designs for demonstrating effectiveness. For instance, practice-based evidence, which is currently supported by SAMSHA

(2016) evaluation standards but not by the APA evaluation guidelines (2002a), was indicated as a potential research design that individual psychology could utilize. Given that many Adlerians are resistant to the operational and standardized nature of RCT and other forms of efficacy research, practice-based evidence may provide an alternative method to demonstrating effectiveness.

Operationalizing, standardizing, and manualization. Quasi and experimental research studies require some level of operationalizing, standardizing, and manualizing treatment

(Chambless & Hollon, 1998). The Expert Panel in this study agreed that individual psychology must operationally define its constructs and critical components in an empirically testable form.

This is important, given that individual psychology will need to have empirical evidence to demonstrate efficacy and effectiveness. Additionally, in order to conduct efficacy and INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 189

effectiveness research that meets the requirements of EBP evaluation standards, a standardized intervention protocol must be followed (APA, 2002a). One key component in the EBP movement that is required by EST evaluation standards is the development of and adherence to a treatment manual (Chambless & Ollendick, 2001). Panelists in this study agreed that individual psychology can and should develop treatment manuals in order to conduct efficacy and effectiveness research. They suggested that individual psychology should develop treatment manuals based on specific clinical groups (clinical diagnosis and based on broad intervention groups (individual, family, and group counseling). By constructing treatment manuals based on broad intervention groups and specific populations, individual psychology would be able to articulate and empirically demonstrate the generalizability of its findings thus contributing to the demonstration of its efficacy (APA, 2002a). The Panelists in this study further agreed that several Adlerian interventions would be appropriate starting points for Adlerian researchers to focus on in developing treatment manuals (e.g., push button technique, reflecting as if, three step emotional change trick, etc.). Developing and adhering to treatment manuals would contribute to individual psychology’s demonstrating efficacy and effectiveness as defined in the current EBP evaluation standards (APA, 2002a; Chambless and Ollendick, 2001). Finally, Panelists suggested that individual psychology could look to psychodynamic theories that have successfully developed EST-recognized treatment manuals (e.g., interpersonal psychotherapy) in order to replicate the process of formulating a treatment manual.

Dissemination. The Expert Panel in this study agreed that individual psychology must improve its dissemination of the evidence supporting individual psychology’s efficacy and effectiveness. One issue that Adlerians have identified as problematic has been that individual psychology publishes the majority of its literature in Adlerian-journals (Bitter, 2016; Sperry, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 190

1991). This study’s Panel agreed that individual psychology must publish more literature in non-

Adlerian based publications. If Adlerians were to publish in more non-Adlerian based journals, they would be able to disseminate evidence supporting individual psychology’s efficacy and effectiveness to a larger demographic. This would be particularly important if individual psychology indeed begins to conduct more empirical experimental outcome studies, because individual psychology could disseminate empirical findings related to the efficacy and effectiveness of individual psychology to non-Adlerians. On a related note, the Panelists also agreed that individual psychology would benefit from disseminating the existing qualitative evidence that supports individual psychology’s efficacy and effectiveness to a broader audience.

Even if not recognized by EST evaluation standards, Panelists felt that the dissemination of qualitative research such as case studies could contribute to the demonstration of individual psychology’s effectiveness.

Support. Lastly, the Expert Panel in this study agreed that in order for individual psychology to demonstrate its efficacy and effectiveness, it needs support from formal Adlerian organizations and institutions. Much of the research that is required for demonstrating EBP is expensive (Emmelkamp et al., 2014; Guy, et al., 2012; Minas & Jorm, 2010) and requires significant training in order to be implemented (Emmelkamp et al., 2014). Panelists in this study agreed that NASAP should highlight and emphasize the need for empirical research to be conducted. NASAP support would be helpful in encouraging Adlerian researchers to acknowledge the need for individual psychology to conduct research related to proving its efficacy and effectiveness. Further, the Expert Panel agreed that NASAP should assist Adlerian researchers financially through grants and other funding sources, so that more empirical research can be feasibly conducted. It was also agreed that Adlerian training institutions should support INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 191

graduate students by encouraging and training them to conduct and publish efficacy and effectiveness research related to individual psychology.

Limitations of this Study

Although efforts were made to ensure that the results of this study were reliable, several limitations must be considered when interpreting the results. Specifically, limitations were associated with the respondents of this study, and with the implementation of the Delphi Method.

Respondents

Respondents were identified for inclusion in the Expert Panel based on their publications in the area of individual psychology and their involvement in Adlerian leadership. However, other criteria could have been utilized in identifying and assembling the Expert Panel. For example, the researcher could additionally have required that respondents demonstrate their knowledge of EBP and EST evaluation standards. If this requirement had been included, the responses could have been different as a result of varied levels of Panelist knowledge and comfort with the EPB assessment proves. The small Panel size and attrition are also noted as limitations. Although the Panel achieved the established minimum numbers of Panel Members in each round, the inclusion of more Panel Members might have resulted in different findings resulting from a broader scope of individual perspectives. Additionally, the attrition between rounds raises question as to how different the responses would have been if those Panel

Members who dropped out had completed the study.

Implementation of the Delphi Method

The implementation of this Delphi study followed an accepted revised version of the

Delphi method protocol. However, the uniqueness of the study’s rating and ranking scales presented several limitations that must be considered. The change in the ranking scale between INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 192

rounds two and three from top twenty-five to top ten posed a potentially significant limitation for the how component. If this had not been changed, and a predetermined ranking scale was maintained between rounds two and three, the data for the how component could have been different. As noted previously, it is unlikely that Panel Members who ranked a suggestion in their top twenty-five, but ranked the suggestion outside of their top ten in round two, would have increased their ranking of that suggestion in round three to be included in their top ten.

Additionally, the study concluded after the third round; however, the responses might have changed if the study had continued to subsequent rounds. For example, if subsequent rounds had been conducted, suggestions for the how component (that in this study moved away from consensus) might eventually have reached the 80% consensus cut-off point, and the Panel may have achieved stronger consensus and stability for the can and should components. Finally, the timing of the study may have affected Panelists responses. As previously discussed, between the second and third round many Panelists attended the North American Society of Adlerian

Psychology’s (NASAP) annual conference. The Expert Panel may have been influenced by the remarks of the society president, Dr. Bitter, regarding the need for individual psychology to increase and develop empirical research to address EBP (2016, May). Additionally, at this conference, a presidential task force on Adlerian research was developed to demonstrate individual psychology's efficacy and effectiveness. The presidential remarks and the development of the Adlerian research task force may have influenced Panelists to reconsider their perspective regarding how individual psychology can demonstrate efficacy and effectiveness. These extraneous variables may have affected the Panelists rankings and ratings between rounds two and three. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 193

Implications for Individual Psychology

Despite its limitations, the current study has important implications in relation to how, can, and should individual psychology demonstrate efficacy and effectiveness given EBP evaluation standards. Specifically, the findings of this study have implications for the field of individual psychology, Adlerian organizations and universities, and Adlerian researchers.

Field of Individual Psychology

Past research and discussion in the field of individual psychology related to EBP has led to a divisive debate among Adlerian researchers and clinicians, and no prior research has examined the effects of this debate on the field of individual psychology. The debate has led to a divided direction in the field regarding how to address the current trends towards EBP in modern mental health. Those trends pose a significant challenge to individual psychology that will have to demonstrate its efficacy and effectiveness in order to remain relevant in the post-modern era

(Prochaska et al., 2013). The current study examined the debate over future directions for the field of individual psychology through the sharing of perspectives from all its various sides in order to provide empirically supported suggestions as to how, can, and should the field demonstrate efficacy and effectiveness. The Expert Panel agreed that there is a need for research to be conducted if individual psychology wishes to become recognized as an EST and offered suggestions for conducting such research that they considered to be both feasible and beneficial.

Additionally, they offered suggestions for how individual psychology can utilize alternative research methods to demonstrate efficacy and effectiveness. The Panel’s suggestions offer a hopeful alternative to the oppositional view that many Adlerian practitioners and researchers have taken with regard to the relevance of outcome research and operationalizing the theory through treatment manualization and standardizing the clinical process in individual psychology. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 194

Adlerian Organizations and Institutions

The findings of this study also have relevant implications for Adlerian organizations and institutions such as the North American Society of Adlerian Psychology (NASAP), the

International Committee of Adlerian Summer Schools and Institutes (ICASSI), the International

Association of Individual Psychology (IAIP), Adler University, and the Adler Graduate School.

The Panel agreed that individual psychology will require assistance from Adlerian organizations and institutions if it is to be able to demonstrate its efficacy and effectiveness in the age of EBT.

These organizations and institutions will need to provide encouragement and financial support related to EBT-related research initiatives. It is hoped that the findings of this study may help to enlighten these organizations and institutions as to the nature and scope of the support that is needed. Funding sources will need to be established to assist with the expenses associated with efficacy and effectiveness research, and academic institutions will need to focus on training graduate students to conduct and publish that research. Finally, it is hoped that the findings of this study may provide useful information and direction for the future action that may be taken by the presidential research task force that was appointed in May 2016 to “demonstrate and strengthen the efficacy and effectiveness of informed Adlerian practice through ongoing research and collaboration” (mission statement).

Adlerian Researchers

The findings of this study have specific implications for Adlerian researchers. The

Expert Panel found that several research designs would be feasible and beneficial for individual psychology to use to demonstrate efficacy and effectiveness. It is important that the Expert

Panel reached consensus regarding the need for individual psychology to conduct double-blind

RCTs, as this is the most widely accepted form of research to demonstrate efficacy (APA, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 195

2002a). Further, both single case design and comparative research were each identified as being beneficial and feasible. The findings suggest that Adlerian researchers should work together to conduct RCTs, single case design, and comparative research to demonstrate individual psychology's efficacy and effectiveness. Additionally, several suggestions indicated that developing treatment manuals and operationally defining the major constructs of individual psychology would be feasible and beneficial. Adlerian researchers can start by developing treatment manuals for commonly utilized interventions such as the “push button technique”,

“reflecting as if”, and the “three step emotional change trick”. To operationally define the major constructs of individual psychology, Adlerian researchers potentially can conduct meta-analysis or utilize the Delphi method to establish a consensual definition of the major constructs.

Additionally, this study suggested that Adlerian researchers can use alternative methods to demonstrate efficacy and effectiveness. Although practice-based evidence research and qualitative research are not approved by the APA (2002a) EBP evaluation standards, Adlerian researchers can conduct research using these methodologies to contribute and support the findings from approved efficacy and effectiveness research. In doing so, Adlerian researchers opposed to more rigorous forms of research design can provide supportive evidence regarding the efficacy and effectiveness of their interventions. Finally, the findings of this study indicates that Adlerian researchers have a profound role to play in conducting this research, and it is hoped, that the findings will provide guidance and support for future research endeavors of

Adlerian researchers as they strive to demonstrate individual psychology's efficacy and effectiveness. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 196

Recommendation’s for Future Research

There is a need for continued research, given the paucity of Adlerian research examining

EBP evaluation standards. This study provides consensual suggestions regarding how, can, and should individual psychology demonstrate efficacy and effectiveness, and provides a framework to examine individual psychology's relevance in the face of EBP. The Delphi method was a useful tool in assisting the Panelists in navigating a complex topic and reaching consensus for several suggestions. As several limitations were identified in the application of this study, there are several recommendations for future research that could assist future researchers in extending and replicating this research:

• Larger Expert Panel – Although this study maintained an acceptable number of Panel

members in each round (Skulmoski et al., 2007), future research would benefit from

recruiting a larger Expert Panel of different Adlerian experts. If the study was replicated

with a larger Expert Panel, it might address the issues of attrition that were present in this

study. Additionally, a larger Panel of different Adlerian experts would help to validate

the suggestions that reached consensus in this study, and could potentially provide an

opportunity for new suggestions and themes to emerge.

• Different Qualifications for Experts – As indicated in the limitations section, expansions

of this research would potentially benefit from setting more stringent eligibility

requirements for inclusion in the Expert Panel. If this study were replicated, the research

would benefit from the ensurance that Panelists were knowledgeable of both individual

psychology and EBP evaluation standards.

• Incentivize Completion – In this study Panel Members were not offered any incentives to

complete each round or the entire Delphi study. If this study was replicated, providing INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 197

incentives to Panelists to complete the Delphi study in its entirety could potentially limit

attrition. However, the incentives would have to be selected with caution so as to not

influence the responses of Panelists.

• Conduct a Pilot Study Before the First Round – Although this study involved an impartial

review of all suggestions after round one, no pilot study was conducted to validate the

suggestions. Some Delphi research suggests that preliminary round be conducted which

allow the Panel Members to consolidate responses and provide feedback. Replications of

this study could benefit from providing a preliminary round of the study allowing the

Expert Panel to remove or re-word suggestions for clarity. Further, if provided, a pilot

study could enable Panel Members to consolidate suggestions into common themes or

suggestions to reduce the overall number of suggestions.

• Utilize Same Ranking Scale Throughout Study – In this study the ranking scale for the

how component changed from top twenty-five to top ten between rounds two and three.

This presented issues for the data analysis and stability testing of the how component in

particular. Replications of this study could potentially avoid these problems by using a

common ranking scale consistently throughout the study. If a common ranking scale was

utilized for the how component stability testing would be able to be conducted for

suggestions that reached or trended towards consensus.

• Conduct Subsequent Rounds – Although consensus was achieved for several suggestions

for the can and should components, this study concluded without reaching consensus for

the how component. Ultimately, agreement for the how component was moving away

from consensus between rounds two and three, and there is no evidence that consensus

would have eventually been achieved. That being said, replications of this study could INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 198

conduct subsequent rounds to better ensure accuracy in the determination (or rejection) of

consensus.

• Conduct Follow-Up Qualitative Interviews with Panelists - This study did not conduct

any follow-up interviews with Panel Members to validate responses. By adding a follow-

up interview, Panelists could have added qualitative feedback to further validate the

suggestions that reached consensus. Replications of this study could benefit from

conducting qualitative interviews with panel Members following data collection and

analysis to corroborate the findings.

Summary

Due to the growing pressure to adhere to EBP evaluation standards in mental health

(Prochaska et al., 2011) this study sought to identify and reach consensus regarding how, can, and should individual psychology demonstrate efficacy and effectiveness given current EBP evaluation standards. Through using the Delphi method in this study, a panel of Adlerian

Experts were able to identify several suggestions that reached or trended towards consensus for each of the three components for both efficacy and effectiveness. For efficacy, three suggestions trended towards consensus for the how component, eight suggestions reached consensus for the can component, and five suggestions reached consensus for the should component. For effectiveness, three suggestions trended towards consensus for the how component, six suggestions reached consensus for the can component, and seven suggestions reached consensus for the should component. From each of the suggestions that reached stable consensus, four common themes emerged. This study indicated that individual psychology will have to explore several empirical research designs (i.e., RCT, single-case design, comparative research, etc.) and alternative research methods (i.e., approved qualitative research, case studies, practice-based INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 199

evidence, etc.) to demonstrate efficacy and effectiveness. Additionally, this study indicated that individual psychology will need to standardize treatment protocols, operationalize major constructs, and develop treatment manuals in order to meet EBP evaluation standards. If individual psychology seeks to demonstrate its efficacy and effectiveness, Adlerians will have to improve their efforts to disseminate both existing literature and newly developed empirical research. Finally, the findings of this study indicated that individual psychology will need the support of Adlerian organizations and institutions in order to take steps to demonstrate efficacy and effectiveness.

The results of this study indicate that individual psychology would benefit from and could feasibly implement the suggestions offered by its Expert Panelists to address the pressures of EBP evaluation standards. In addition to having specific implications to the overall field of individual psychology, the findings have specific relevance to Adlerian organizations and institutions, as well as Adlerian researchers. Although the rendering of individual psychology as an EBP is currently a hotly debated topic among Adlerian practitioners and researchers, empirical study of the topic is in its infancy, and there is a need for future research. It is hoped that this study has established a foundational framework for continued research into how individual psychology can and should demonstrate efficacy and effectiveness given the current

EBP evaluation standards.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 200

References

Addis, M., & Krasnow, A. (2000). A national survey of practicing psychologists’ attitudes

toward psychotherapy treatment manuals. Journal of Consulting and Clinical

Psychology, 68(2), 331–339. http://doi.org/10.1037/0022-006X.68.2.331

Addis, M. E., Wade, W. A., & Hatgis, C. (1999). Barriers to dissemination of evidence based

practices: Addressing practitioners' concerns about manual based psychotherapies.

Clinical Psychology: Science and Practice, 6(4), 430-441.

Adler, A. (1927). Understanding life. New York: Greenberg.

Adler, A. (1931). What life should mean to you. Minnesota: Hazleton.

Adler, K. A., & Deutsch, D. (Eds.). (1959). Essays in individual psychology. New York: Grove

Press.

Adler, M. & Ziglio. E. (1996). Gazing into the oracle: The Delphi Method and its application to

social policy and public health. London: Jessica Kingsley Publishers.

Ansbacher, H. L. (1990). Hypnosis for diagnostic purpose: Adlerian understanding. Individual

Psychology: The Journal of Adlerian Theory, Research & Practice, 46(4), 516— 521.

APA Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in

psychology. American Psychologist, 61, 271-285.

American Counseling Association. (2014). Code of ethics. Virginia: Author

American Psychological Association. (2000). Guidelines and principles for accreditation of

programs in professional psychology. Washington D.C.: Author.

American Psychological Association. (2002a). Criteria for evaluating treatment guidelines.

American Psychologist, 57(12), 1052-1059.

American Psychological Association. (2002b). Ethical principles of psychologists and code of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 201

conduct. American psychologist, 57(12), 1060-1073.

Baez, T. (2005). Evidenced-based practice for anxiety disorders in college mental health. Journal

of College Student Psychotherapy, 20(1), 33–48.

Barkham, M., & Mellor-Clark, J. (2003). Bridging evidence-based practice and practice-based

evidence: Developing a rigorous and relevant knowledge for the psychological therapies.

Clinical Psychology & Psychotherapy, 10(6), 319-327.

Blow, A. J., & Sprenkle, D. H. (2001). Common factors across theories of marriage and family

therapy: A modified Delphi study. Journal of Marital and Family Therapy, 27(3), 385-

401.

Bratton, S. C., Ray, D., Rhine, T., & Jones, L. (2005). The efficacy of play therapy with children:

A meta-analytic review of treatment outcomes. Professional Psychology: Research and

Practice, 36(4), 376-390.

Burker, E. J., & Kazukauskas, K. A. (2010). Code of ethics for rehabilitation educators and

counselors: A call for evidence-based practice. Rehabilitation Education, 24(3-4), 101-

111.

Carlson, J. (2000). The 1999 Heinz L. and Rowena R. Ansbacher memorial address - individual

psychology in the year 2000 and beyond: Astronaut or dinosaur? Headline or footnote?

The Journal of Individual Psychology, 56(1), 3-13.

Carlson, J. D., & Robey, P. A. (2011). An integrative adlerian approach to family counseling.

Journal of Individual Psychology, 67(3), 232-244.

Carlson, J., Watts, R. E., & Maniacci, M. (2006). Adlerian therapy: Theory and practice.

Washington D.C.: American Psychological Association.

Casey, A. (2012). Engaging meaningfully with outcome measurement. British Journal of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 202

Psychotherapy, 28(4), 516–530.

Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of

Consulting and Clinical Psychology, 66(1), 7–18.

Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions:

Controversies and evidence. Annual Review of Psychology, 52(1), 685-716.

Chan, F., Bezyak, J., Ramirez, M. R., Chiu, C.-Y., Sung, C., & Fujikawa, M. (2010). Concepts,

challenges, barriers, and opportunities Related to Evidence-Based Practice in

rehabilitation counseling. Rehabilitation Education, 24(3/4), 179–190.

Clement, P. (2013). Practice-based evidence: 45 years of psychotherapy's effectiveness in a

private practice. American Journal of Psychotherapy, 67(1), 23-46.

Cooper, S. (2005). Evidenced-based psychotherapy in college mental health: Common concerns

and implications for practice and research. Journal of College Student Psychotherapy,

20(1), 33–48.

Corsini, R. J., & Wedding, D. (2005). Current psychotherapies. In current psychotherapies 6th

ed., ed. Raymond J. Corsini and Danny Wedding, . 1–17. California: Wadsworth /

Thomson Learning.

Corey, G. (2005). Theory and practice of counseling and psychotherapy (7th ed.). California:

Brooks/Cole.

Council on Social Work Education. (2008). Education policy and accreditation standards.

Virginia: Council on Social Work Education.

Crane, D. R., & Hafen Jr., M. (2002). Meeting the needs of evidence-based practice in family

therapy: Developing the scientist-practitioner model. Journal of Family Therapy, 24(2),

113–124. Retrieved from http://www.blackwell-synergy.com/links/doi/10.1111/1467- INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 203

6427.00206.

Cummings, N., & O’Donahue, W. (2008). Eleven blunders that are crippling psychotherapy.

New York: Routledge.

Dawson, S., & Barker, J. (1995). Hospice and palliative care: A Delphi survey of occupational

therapists roles and training needs. Australian Occupational Therapy Journal, 42, l 19-

27.

Day, J., & Bobeva, M. (2005). A generic toolkit for the successful management of Delphi

studies. Electronic Journal of Business Research Methods, 3(2), 103–116.

Delbecq A.L., Van de Ven A.H. & Gustafson D.H. (1975). Group techniques for program

planning: a guide to nominal and del- phi processes. Illinois: Scott, Foresman and Co..

Dinkmeyer, D. C., Pew, W. L., & Dinkmeyer, D. C., (1979). Adlerian counseling and

psychotherapy. California: Wadsworth.

Dreikurs, R (1967). Psychodynamics, psychotherapy, and counseling. Chicago: Alfred Adler

Institute.

Dreikurs-Ferguson, E. (2000). Individual Psychology Is Ahead of Its Time. Journal of Individual

Psychology, 56(1), 14-19.

Emmelkamp, P. M., David, D., Beckers, T., Muris, P., Cuijpers, P., Lutz, W., ... & Vervliet, B.

(2014). Advancing psychotherapy and evidence-based psychological interventions.

International Journal of Methods in Psychiatric Research, 23(S1), 58-91.

Evans, T. D., & Meredith, C. W. (1991). How far can you go and still be Adlerian? Special

Issue: “On beyond Adler.” Journal of Adlerian Theory, Research and Practice, 47(4),

541–547.

Fish, L. S. & Busby, D. M. (1996). The Delphi method. In D. H. Sprenkle and S. M. Moon INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 204

(Eds.), Research methods in family therapy (pp. 469-484). New York: Guilford.

Fonagy, P., Roth, A., & Higgitt, A. (2005). Psychodynamic psychotherapies: Evidence–based

practice and clinical wisdom. Bulletin of Menninger Clinic, 69(1), 1–59.

Frankl, V. E. (2014). The will to meaning: Foundations and applications of logotherapy. New

York: Penguin.

Generali, M. M., Foss-Kelly, L. L., & McNamara, K. (2011). Barriers to evidence-based

counseling practices: A counselor educator training model. In American Association of

Counselor Education and Supervision Conference. Nashville, TN.

Greever, K. B., Tseng, M. S., & Friedland, B. U. (1973). Development of the social interest

index. Journal of Consulting and Clinical Psychology, 41(3), 454-458.

Guy, A., Thomas, R., Stephenson, S., & Loewenthal, D. (2012). NICE under scrutiny The impact

of the National Institute for Health and Clinical Excellence guidelines on the provision of

psychotherapy in UK. The Journal of Psychological Therapies in Primary Care, 1(1-2),

77-102.

Hall, J. C. (2008). A practitioner’s application and deconstruction of evidence-based practice.

Families in Society: The Journal of Contemporary Social Services, 89(3), 385–393.

http://doi.org/10.1606/1044-3894.3764

Halstead, J. E., Leach, C., & Rust, J. (2007). The development of a brief distress measure for the

evaluation of psychotherapy and counseling (sPaCE). Psychotherapy Research, 17(6),

656–672.

Hartshorne, T. S. (1991). The evolution of psychotherapy: Where are the Adlerians?. Individual

Psychology: Journal of Adlerian Theory, Research & Practice, 7(3), 321-326.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 205

Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey

technique. Journal of Advanced Nursing, 32(4), 1008–1015.

Helmer, O. (1983). Looking forward: A guide to futures research. California: Sage.

Hsu, C., & Sandford, B. (2007). The Delphi technique: making sense of consensus. Practical

Assessment, Research & Evaluation, 12(10), 1–8.

Hunsberger, P. H. (2007). Reestablishing clinical psychology's subjective core. The American

psychologist, 62(6), 614-615.

Hundsley, J. (2007). Addressing key challenges in evidence-based practice in psychology.

Professional Psychology: Research and Practice, 38(2), 113-121.

Huppert, J. D., Fabbro, A., & Barlow, D. H. (2006). Evidence-based practice and psychological

treatments. In C. D. Goodheart, A. E. Kazdin, & R. J. Sternberg (Eds.), Evidence-based

psychotherapy: Where practice and research meet (pp. 131–152). Washington, DC:

American Psychological Association.

Jones-Smith, E. (2012). Theories of counseling and psychotherapy: An integrative approach.

California: Sage Publications.

Kalkan, M. (2009). Adlerian Social Interest Scale-Romantic Relationship Form (ASIS-RR):

Scale development and psychometric properties. Individual Differences Research, 7(1),

40-48.

Kalaian, S. A., & Kasim, R. M. (2012). Terminating sequential Delphi survey data collection.

Practical Assessment, Research & Evaluation, 17(5), 1-10.

Kaplan, D., & Martz, E. (2014). Raising the bar for counselor educators. Counseling Today,

57(5), 26-27. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 206

Kern, R. M., Belangee, S. E., & Eckstein, D. (2004). Early recollections: A guide for

practitioners. Journal of Individual Psychology, 60(2), 132-140.

Kazdin, A. E. (2006). Assessment and evaluation in clinical practice. In C. D. Goodheart, A. E.

Kazdin, & R. J. Sternberg (Eds.), Evidence-based psychotherapy: Where practice and

research meet (pp. 153–177). Washington, DC: American Psychological Association.

Kazdin, A. E. (2008). Evidence-based treatment and practice: New opportunities to bridge

clinical research and practice, enhance the knowledge base, and improve patient care.

American psychologist, 63(3), 146.

Leak, G. K. (2006). Development and validation of a revised measure of Adlerian social interest.

Social Behavior and Personality: an international journal, 34(4), 443-450.

Levant, R.F. & Hasan, N.T. (2008). Evidence-based practice in psychology. Professional

Psychology: Research and Practice, 39(6), 658-662

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2013). Why many

clinical psychologists are resistant to evidence-based practice: Root causes and

constructive remedies. Clinical Psychology Review, 33(7), 883-900.

Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2014). Why

ineffective psychotherapies appear to work: A taxonomy of causes of spurious

therapeutic effectiveness. Perspectives on Psychological Science, 9(4), 355-387.

Linstone, H. A., & Turoff, M. (1975). The Delphi method: Techniques and applications (pp. 3-

12). Massachusetts: Addison-Wesley Publishing Company.

Margison, F. R., Barkham, M., Evans, C., McGrath, G., Clark, J. M., Audin, K., & Connell, J.

(2000). Measurement and psychotherapy. Evidence-based practice and practice-based

evidence. The British Journal of Psychiatry, 177(2), 123-130. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 207

Marshall, J. L., & Fitch, T. J. (2006). Adlerian perspectives on purging behavior. Journal of

Individual Psychology, 62(3) 301-311.

Martino, S. (2010). Strategies for training counselors in evidence-based treatments. Addiction

Science & Clinical Practice, 5(2), 30-39.

McWilliams, N. (2005). Preserving our humanity as therapists. Psychotherapy: Theory,

Research, Practice, Training, 42(2), 139-151.

Messer, S. B. (2004). Evidence-based practice: Beyond empirically supported treatments.

Professional Psychology: Research and Practice, 35(6), 580-588.

Minas, H., & Jorm, A. F. (2010). Where there is no evidence: Use of expert consensus methods

to fill the evidence gap in low-income countries and cultural minorities. International

Journal of Mental Health Systems, 4(1), 33-38.

Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology

Review, 11(3), 247-266.

Mosak, H., & Maniacci, M. (1999). A primer of Adlerian psychology. Philadelphia: Brunner-

Mazel.

Murphy, M. K., Black, N., Lamping D. L., McKee, C. M., Sanderson, C. F. B., Askham, J., &

Marteau, T. (1998). Consensus development methods and their use in clinical guideline

development. Health Technology Assessment, 2(3), 1-88.

NASW Delegate Assembly (2008). Code of Ethics of the National Association of Social

Workers. Retrieved from http://www.socialworkers.org/pubs/code/code.asp

Nevo, I., & Slonim-Nevo, V. (2011). The myth of evidence-based practice: Towards evidence-

informed practice. British Journal of Social Work, 41(6), 1176-1197.

Norcross, J. C., Hedges, M., & Prochaska, J. O. (2002). The face of 2010: A Delphi poll on the INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 208

future of psychotherapy. Professional Psychology: Research and Practice, 33(3), 316-

323.

Norcross, J. C., & Lambert, M. J. (2011). Evidence-based therapy relationships. In J. C. Norcross

(Eds.), Psychotherapy relationships that work: Evidence- based responsiveness (2nd ed.,

pp. 3-21). North Carolina: Oxford University Press.

Norcross, J. C., Pfund, R. A., & Prochaska, J. O. (2013). Psychotherapy in 2022: A Delphi poll

on its future. Professional Psychology: Research and Practice, 44(5), 363-370.

Norcross, J. C., & Wampold, B. E. (2011). Evidence-based therapy relationships: Research

conclusions and clinical practices. Psychotherapy, 48(1), 98-102.

Nystul, M. (1991). An Interview with Jon Carlson. Individual Psychology, 47(4), 498–503.

Oberst, U. E., & Stewart, A. E. (2003). Adlerian psychotherapy. An advanced approach to

individual psychology. New York: Brunner-Routledge.

Okoli, C. & Pawlowski, S. (2004) The Delphi method as a research too: an example, design

considerations and applications. Information & Management, 42(1), 15-29.

Peluso, P. R., Stoltz, K. B., Belangee, S., Frey, M. R., & Pelus, J. P. (2010). A confirmatory

factor analysis of a measure of the Adlerian lifestyle, the BASIS-A Inventory. Journal of

Individual Psychology, 66(2,), 152-165.

Pfefferlé, J., & Mansager, E. (2014). Applying the classical Adlerian family diagnostic process.

The Journal of Individual Psychology, 70(4), 332-377.

Popkin, M. (1993). Active parenting today. Atlanta: Active Parenting.

Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing,

41(4), 376–382.

Powers, R. L., & Griffith, J. (1987). Understanding life-style: The psycho-clarity process. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 209

Chicago: The Americas Institute of Adlerian Studies.

Prendergast, M. L. (2011). Issues in defining and applying evidence-based practices criteria for

treatment of criminal-justice involved clients. Journal of Psychoactive Drugs, 43(sup1),

10-18.

Pytell, T. (2000). The missing pieces of the puzzle: a reflection on the odd career of Viktor

Frankl. Journal of Contemporary history, 35(2), 281-306.

Reed, G. M., & Eisman, E. (2006). Uses and misuses of evidence: Managed care, treatment

guidelines, and outcomes measurement in professional practice. In Goodheart CD,

Kazdin AE, Sternberg RJ, (Eds.), Evidence-based psychotherapy: Where practice and

research meet (pp. 13-35). Washington, DC: American Psychological Association.

Resnick, J. L. (2005). Evidence-based practice for treatment of eating disorders. Journal of

College Student Psychotherapy, 20(1), 33–48.

Reynolds, S. (2000). Evidence based practice and psychotherapy research. Journal of Mental

Health, 9(3), 257-266.

Sackman H. (1975) Delphi critique. Massachusetts: Lexington Books.

Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A.

Linstone, & M. Turoff (Eds.), The Delphi method: Techniques and applications (pp. 262-

287). Massachusetts: Addison-Wesley Publishing Company.

Sexton, T. (2001). Evidence-based counseling intervention programs: Practicing “best practices.”

In D. C. Locke, J. E., Meyers, & E. H. Herr (Eds.), The handbook of counseling (pp.499-

512). California: Sage.

Shallcross, L. (2012). Proof positive. Counseling Today, 55(3), 28-37.

Shedler, J. (2010). The efficacy of psychodynamic psychotherapy. American Psychologist, 65(2), INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 210

98-109.

Shedler, J. (2011). Science or ideology. American Psychologist, 66(2), 152-154.

Shelley, C. (2000). Epistemology and methods in individual psychology: Toward a fusion of

horizons with hermeneutics. Journal of Individual Psychology, 56(1) 59-73.

Shulman, B. H., & Mosak, H. H. (1988). Manual for life assessment. Indiana: Accelerated

Development.

Skulmoski, G., Hartman, F., & Krahn, J. (2007). The Delphi method for graduate research.

Journal of Information Technology Education: Research, 6(1), 1-21.

Silverman, W. H. (1996). Cookbooks, manuals, and paint-by-numbers: Psychotherapy in the

90's. Psychotherapy: Theory, Research, Practice, Training, 33(2), 207-215.

Sommers-Flanagan, J., & Sommers-Flanagan, R. (2011). Counseling and psychotherapy theories

in context and practice: Skills, strategies, and techniques. John Wiley & Sons.

Sperry, L. (1991). An alternative future for individual psychology: A challenging agenda for

NASAP. Journal of Adlerian Theory, Research & Practice, 47(4), 548–553.

Sperry, L. (2014a). Diagnosis, case conceptualization, culture, and treatment. In L. Sperry, J.

Carlson, J. Duba Sauerheber, & J. Sperry (Eds.), Psychopathology and psychotherapy:

DSM-5 diagnosis, case conceptualization, and treatment (3rd ed., pp.1-14). New York:

Routledge.

Sperry, L. (2014b). Adlerian case conceptualization. In L. Sperry, J. Carlson, J. Duba

Sauerheber, & J. Sperry (Eds.), Psychopathology and psychotherapy: DSM-5 diagnosis,

case conceptualization, and treatment (3rd ed., pp.15-26). New York: Routledge.

Stiles, W. B., Shapiro, D. A., & Elliott, R. (1986). Are all psychotherapies equivalent?. American

psychologist, 41(2), 165. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 211

Substance Abuse and Mental Health Services Administration [SAMHSA]. (2017). SAMHSA’s

national registry of evidence-based programs and practices. Retrieved February 25th,

2017, from http:// www.nrepp.samhsa.gov/.

Sweeney, T. J. (1998). Adlerian counseling: A practitioner’s approach. Philadelphia: Taylor &

Francis.

Sweeney, T. J., Myers, J. E., & Stephan, J. B. (2006). Integrating developmental counseling and

therapy assessment with Adlerian early recollections. Journal of Individual Psychology,

62(3), 251-269.

Tanenbaum, S. J. (2005). Evidence-based practice as mental health policy: Three controversies

and a caveat. Health Affairs, 24(1), 163-173.

Task Force on Promotion and Dissemination of Psychological Procedures. (1995). Training in

and dissemination of empirically validated psychological treatments: Report and

recommendations of the Task Force on Promotion and Dissemination of Psychological

Procedures of Division 12 (Clinical Psychology) of the American Psychological

Association. Clinical Psychologist, 48, 3-23.

Thomason, T. C. (2010). The trend Toward evidence-based practice and the future of

psychotheraphy. American Journal of Psychotherapy, 64(1), 29-38.

Tilsen, J., & McNamee, S. (2015). Feedback informed treatment: Evidence-based practice meets

social construction. Family Process, 54(1), 124–137.

Ulschak, F. L. (1983). Human resource development: The theory and practice of need

assessment. Virginia: Reston Publishing Company, Inc.

Unsworth, G., Cowie, H., & Green, A. (2012). Therapists’ and clients’ perceptions of routine

outcome measurement in the NHS: A qualitative study. Counseling and Psychotherapy INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 212

Research, 12(1), 71-80.

Von der Gracht, H. A. (2012). Consensus measurement in Delphi studies review and

implications for future quality assurance. Technological Forecasting and Social Change,

79(8), 1525–1536.

Wachtel, P. L. (2010). Beyond “ESTs”: Problematic assumptions in the pursuit of evidence-

based practice. Psychoanalytic Psychology, 27(3), 251-272.

Wampold, B. E. (2003). Bashing positivism and reversing a medical model under the guise of

evidence. The Counseling Psychologist, 31(5), 539-545.

Wampold, B. E., Goodheart, C. D., & Levant, R. F. (2007). Clarification and elaboration on

evidence-based practice in psychology. American Psychologist, 62(6), 616-618.

Wampold, B. E., & Bhati, K. S. (2004). Attending to the omissions: A historical examination of

evidence-based practice movements. Professional Psychology: Research and Practice,

35(6), 563-570.

Watkins Jr, C. E., & Guarnaccia, C. A. (1999). Introduction: The future of psychotherapy

training: psychodynamic, experiential, and eclectic perspectives. Journal of Clinical

Psychology, 55(4), 381-383.

Watts, R. E. (1996). Social interest and the core conditions: Could it be that Adler influenced

Rogers?. The Journal of Humanistic Education and Development, 34(4), 165-170.

Watts, R. E. (1998). The remarkable parallel between Rogers's core conditions and Adler's social

interest. Journal of Individual Psychology, 54, 4-9.

Watts, R. E. (2000a). Adlerian counseling: A viable approach for contemporary practice. Journal

of Professional Counseling, Practice, Theory, & Research, 28(1), 11-18.

Watts, R. E. (2000b). Entering the new millennium: Is individual psychology still relevant?. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 213

Journal of Individual Psychology, 56(1), 21-30.

Watts, R. E. (2013). Adlerian counseling. In B. Irby, G. Brown, R. Lara-Alecio, & S. Jackson

(Eds.), The handbook of educational theories (pp.459-472). North Carolina: Information

Age Publishing.

Watts, R. E., & Phillips, K. A. (2004, July). Adlerian psychology and psychotherapy: A

relational constructivist approach. Paper presented at the Biennial Conference of the

North American Personal Construct Network, Vancouver, BC, Canada.

Watts, R. E., & Pietrzak, D. (2000). Adlerian “Encouragement” and the therapeutic process of

solution-focused brief therapy. Journal of Counseling & Development, 78(4), 442-447.

Westfall, J. M., Mold, J., & Fagnan, L. (2007). Practice-based research - “Blue Highways” on

the NIH roadmap. The Journal of the American Medical Association, 297(4), 403-406.

Wheeler, M. S. (1996). Using the BASIS-A Inventory: Examples from a clinical setting.

Individual Psychology, 52, 104-118.

Wheeler, M. S., Kern, R. M., & Curlette, W. L. (1993). Basic Adlerian Scales for Interpersonal

Success-Adult Form (BASIS-A) Inventory. North Carolina: TRT Associates.

Wingett, W. (2015). Early Recollections as Vehicles for Change. presentation at the annual

conference of the North American Society of Adlerian Psychology, Philadelphia, Pa.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 214

Appendix A

Recruitment Email Inviting Panel Members to Participate in Round One

The College Of WILLIAM & MARY

School of Education Sterling P. Travis Counselor Education Doctoral Candidate Williamsburg, VA Phone: (434) 989-8952

Email: [email protected] Letter of Invitation Dear

You have been identified as an expert in the field of individual psychology, and your participation is requested in a groundbreaking study to discover how individual psychology can best demonstrate its contemporary relevance in a professional environment that is increasingly being shaped by mandates for compliance with standards of Evidence Based Practice. Your contribution will help address a major issue that we in individual psychology have faced as the field of mental health has moved into the post-modern era.

Specifically, I am asking you to participate in my dissertation study entitled: A Delphi Study Regarding How, Can, and Should Individual Psychology Demonstrate Efficacy and Effectiveness Given Evidence Based Practice Evaluation Standards, that is being conducted through the College of William and Mary. The specific purpose of this research is to develop consensus among Adlerian experts regarding how, can, and should the efficacy and effectiveness of individual psychology be demonstrated in light of current Evidence Based Practice standards.

According to the current Evidence Based Practice standards, efficacy refers to whether proposed beneficial effects of an intervention can be demonstrated scientifically (e.g., through treatment manualization and standardization, random assignment; random control trials, empirical outcome data, internal validity, etc.). Effectiveness, refers to whether an intervention is generalizable and feasible (practically and financially) for implementation with various populations, settings, and clinicians.

You are being invited to participate in this study, because you have been involved in scholarly activity related to individual psychology and/or have served in a leadership role in an Adlerian organization. Your efforts will assist in filling a current and significant gap in the literature regarding the means by which individual psychology can address current challenges to demonstrate its continued relevance in the post-modern era.

The Delphi study will take place in approximately three rounds, with the first round requiring your response to two open-ended questions regarding the means by which you believe individual psychology can demonstrate efficacy and effectiveness as an Evidence Based Treatment (i.e., the INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 215

how component). Subsequent rounds will require you to rank-order two lists of all participant suggestions developed from the first round and then to rate individual suggestions based on both their perceived feasibility for implementation (the can component) and the perceived benefit of their implementation (the should component). The Delphi method allows your expert opinion to build on that of other experts in the field to generate consensus of information.

Please begin your participation in the study by clicking on the following link: Thank-you in advance for your willingness to collaborate with other experts in order to further advance the evolution of individual psychology in the 21st century. If you have any questions or concerns regarding this research, please contact me at 434-989-8952 or via e- mail at [email protected].

Sincerely, Sterling P. Travis, M.Ed. College of William and Mary Doctoral Candidate

THIS PROJECT WAS FOUND TO COMPLY WITH APPROPRIATE ETHICAL STANDARDS AND WAS EXEMPTED FROM THE NEED FOR FORMAL REVIEW BY THE COLLEGE OF WILLIAM AND MARY PROTECTION OF HUMAN SUBJECTS COMMITTEE (Phone 757- 221-3966) ON 2016-02-20 AND EXPIRES ON 2017-02-20.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 216

Appendix B

Round One Qualtrics Questionnaire

Round One Directions: In the text box at the bottom of the page, please answer the first of two research question in narrative paragraph form:

In what ways can individual psychology demonstrate efficacy given the current EBP evaluation standards?

Efficacy refers to whether proposed beneficial effects of an intervention can be demonstrated scientifically (e.g., through treatment manualization and standardization, random assignment; random control trials, empirical outcome data, internal validity, etc.). In order to demonstrate efficacy as an Evidence Based Treatment, clinical models must satisfy the five established Efficacy Assessment Criteria (Hollon, Miller, & Robinson, 2002):

1. The (EBP evaluation guidelines require a clinical model to be grounded in and based on careful consideration of a broad base of relevant empirical literature. 2. EBP evaluation guidelines require the research methodology supporting a clinical model to demonstrate the highest level of rigor and sophistication (i.e., meta analysis and randomized controlled trials as opposed to qualitative research, clinical opinion, case studies, etc.). 3. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence that its treatment outcomes are superior to treatment outcomes of other comparable clinical models and to treatment outcomes from not engaging in treatment. 4. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence to support its selection for use with specific patients. 5. EBP evaluation guidelines require the intended treatment outcomes of a clinical model to be specified, and the actual treatment outcomes to be quantitatively evaluated in relation to (or “against the influence of”) variables within the specific treatment context. (i.e., treatment goals; measures of life functioning; attrition; long-term/indirect consequences of treatment; negative consequences; client satisfaction; clinical significance; and methods).

Please be as thorough and comprehensive as possible (a minimum of 500 words) in providing specific suggestions regarding how you believe individual psychology is already meeting each of the five criteria or how it can do so in the future.

Research Question 1: How can individual psychology demonstrate efficacy given the current EBP evaluation standards?

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 217

Directions:

In the text box at the bottom of the page, please answer the second of two research questions in narrative paragraph form:

In what ways can individual psychology demonstrate effectiveness given the current EBP evaluation standards?

Effectiveness refers to whether an intervention is generalizable and feasible (practically and financially) for implementation with various populations, settings, and clinicians. In order to demonstrate effectiveness as an Evidence Based Treatment, clinical models must satisfy the five Effectiveness Assessment Criteria that have been established for Evidence Based Treatments (Hollon, Miller, & Robinson, 2002):

1. EBP evaluation guidelines require a clinical model to be able to quantify the influence that patient variables may have on its treatment outcomes (i.e., complexity of clinical presentation; culture; gender/sex; age/developmental level; etc.). 2. EBP evaluation guidelines require a clinical model to be able to quantify the effect that different clinician’s will have on its treatment outcomes (i.e.; clinical skill; experience; culture/ethnic background; gender; etc.). 3. EBP evaluation guidelines require a clinical model to be able to quantify the influence that the treatment setting may have on its treatment outcomes (i.e.; home; school; day treatment; clinic; etc.). 4. EBP evaluation guidelines require a clinical model to be able to quantify the influence of alterations in its administration on treatment outcomes (i.e.; deviation from protocol; time frame; delivery method; etc.). 5. EBP evaluation guidelines require a clinical model to be able to quantify its feasibility (i.e., clients choice/willingness/and ability to participate in the intervention) and its benefit relative to cost (i.e.; financial cost to client/clinician; prevention of future disorders; medical costs; etc.) for those providing and receiving treatment.

Please be as thorough and comprehensive as possible (a minimum of 500 words) in providing specific suggestions regarding how you believe individual psychology is already meeting each of the five criteria or how it can do so in the future.

Research Question 1: How can individual psychology demonstrate effectiveness given the current EBP evaluation standards?

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 218

Appendix C

Informed Consent

Informed Consent

Project Title: You are invited to participate in a research project as part of a doctoral dissertation conducted by Sterling P. Travis entitled “A Delphi Study Regarding How, Can, and Should Individual Psychology Demonstrate Efficacy and Effectiveness Given Evidence Based Practice Evaluation Standards.” The study will be conducted through the College of William and Mary, School of Education under the direction of Charles R. McAdams, III, Ph.D., Chairperson.

Delphi Research: The Delphi technique strives to obtain consensus on the opinions of experts, termed panel members, through a series of structured questionnaires. During the process, panel member ideas for resolving a named problem are ranked and rated multiple times until a desired level of consensus is reached among panelists regarding the best solution to the problem The Delphi is therefore an iterative multi-stage process designed to transform individual opinion into group consensus.

Purpose of the Research and Methodology: The research regarding how individual psychology can address the challenges to demonstrate efficacy and effectiveness given the current Evidence Based Practices evaluation standards is limited (Carlson, 2000; Norcross, Pfund, & Prochaska, 2013; Sperry, 2014a). The purpose of this study is to discover how Adlerian experts suggest that Adlerian clinicians and researchers demonstrate the efficacy and effectiveness of individual psychology given the current evaluation standards of EBP. The overriding research goal is to develop a consensual opinion as to how, can, and should the efficacy and effectiveness of individual psychology be demonstrated given the current EBP evaluation standards. This study asks you as an expert to respond to approximately three rounds of questionnaires focused on identifying specific suggestions for how the research goal can be achieved. The specific research questions of the study are as follows:

1.How/can/should individual psychology demonstrate efficacy given the current EBP evaluation standards 2.How/can/should individual psychology demonstrate effectiveness given the current EBP evaluation standards

The research will utilize the Delphi method that will take place in approximately three rounds, with the first round including open-ended questions where you will provide a response to the how component of the two previously mentioned research questions. The subsequent rounds will ask you to rank-order two constructed lists of all suggestions from the first round regarding the how component of each research question, and to rate each suggestion based on its perceived ability to be implemented (can component) and perceived benefit (should component). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 219

Responsibility of Panel Members: The researcher is requesting that you, as an established expert in the field of individual psychology, provide your opinion regarding how, can, and should individual psychology demonstrate efficacy and effectiveness given the current Evidence Based Practice evaluation standards. The researcher is inviting you to participate as a Delphi panel member. Your participation will require you to participate in approximately three rounds of data collection via a Qualtrics survey. The initial round of data collection will involve you answering a brief questionnaire related to how individual psychology may demonstrate efficacy and effectiveness given the current Evidence Based Practice Evaluation Standards. The initial round should take approximately 30 minutes to complete. In subsequent rounds, you will receive a comprehensive list of all panel members’ suggestions to address the research questions. In each subsequent round you will be asked to rank order each suggestion based on the suggestions perceived utility in addressing the how component of each of the two research questions. Additionally, you will rate each suggestion based on each suggestion perceived feasibility and benefit as a means of addressing its respective research question on a seven point Likert scale. This process will continue until a group consensus is achieved or three Delphi rounds have been completed. In order to allow timely conclusion of the study we will respectfully request a response time of two weeks for completion of each round.

Risks and Procedures: If you choose to participate in this study, all reasonable efforts will be taken to decrease any risk to you. The chief risk to you in participating in this study is the time commitment necessary for Delphi research. As Delphi research requires panel members to participate in several rounds of data collection (e.g., presenting individual opinion; reviewing comprehensive lists of suggestions; rating/ranking suggestions; etc.), the researcher understands that you as a panel member have many commitments and will provide panel members with sufficient time to complete each round of data collection. Further, although your expertise and participation is a valuable asset to this research, your participation is completely voluntary and you may terminate participation at any time with out any penalty. If you experience distress of any kind you may contact Charles R. McAdams III ([757] 221-2338) who will provide referral services if needed.

Confidentiality and Anonymity: Individual privacy will be maintained for all participants in the written material resulting from this study. All responses received in the study will be strictly confidential, and your identity will not be divulged. Direct quotes may be used as part of the study report, but these will not be traceable back to you. Only the principal investigator and the dissertation chairperson will have access to the names of the participants.

Voluntary Participation: Although your participation in the study is highly desired, your participation is entirely voluntary. You may withdraw from the study at any time, and there will be no negative consequences as a result of your withdrawal. You have the right to refuse to answer any question (s) for any reason.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 220

Questions or Concerns: If you experience distress, have concerns or complaints about how you were treated during this study please contact Charles R. McAdams, III, Ph.D., Chairperson, (757-221-2338).

THIS PROJECT WAS FOUND TO COMPLY WITH APPROPRIATE ETHICAL STANDARDS AND WAS EXEMPTED FROM THE NEED FOR FORMAL REVIEW BY THE COLLEGE OF WILLIAM AND MARY PROTECTION OF HUMAN SUBJECTS COMMITTEE (Phone 757-221-3966) ON 2016-02-20 AND EXPIRES ON 2017-02-20.

I am aware that I must be at least 18 years of age to participate in this project.

I am aware that I may report dissatisfaction with any aspect of this study to the College of William & Mary Protection of Human Subjects Committee by telephone (757-221-3966).

I agree to participate in this study and have read all the information provided on this form. My acknowledgment below confirms that my participation in this project is voluntary, and that I have received a copy of this consent form.

[] I agree to participate in this study and met the criteria for participation

[] I do not agree to participate in this study

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 221

Appendix D

Email to Panel Members Regarding Round One Results Dr. , First, I would like to thank you again for participating in my dissertation research. I enjoyed reading your responses to the first round and I am excited to have your continued presence on the expert panel as the second round begins. The opinions, suggestions, and expertise that you provided have been, and will continue to be, instrumental in this study. I appreciate your patience through out the data collection of the first round as the panel was constructed, and as each panel member provided their responses. That being said, the data collection and analysis have concluded for the first round, and I am excited to share with you the results of the first round as we will begin the second round next week (Week of July, 25-29th). In the following two PDF attachments, I have provided the two comprehensive lists of suggestions compiled from all panel member responses to the research questions. The comprehensive lists of suggestions for how Individual Psychology may demonstrate efficacy given the current evidence based practice evaluation standards consists of eighty-one (81) suggestions. The comprehensive list of suggestions for how Individual Psychology may demonstrate effectiveness given the current evidence based practice evaluation standards consists of fifty-four (54) suggestions. These compiled lists were constructed utilizing qualitative content analysis of each panel member’s suggestions to the research questions. Repeated suggestions, and suggestions deemed similar were combined. Given that suggestions were combined I am asking you to review both of the compiled lists I have attached to ensure that the list comprehensively represents the suggestions that you offered. Upon review:

• If you feel the compiled list comprehensively represents your suggestions you are asked to review the entire list in preparation of the second round of data collection where you will be asked to rate each suggestions based on feasibility and perceived benefit, as well as to select and rank order your top twenty-five suggestions. • If you feel that the compiled list does not comprehensively represent the suggestions that you offered, you are asked to offer revisions and/or additions to the list through email. o Please send revisions/additions by email to [email protected]. *In your email please indicate: § If you are sending a revision or addition § The list in which you desire a revision and/or addition to be added (efficacy, effectiveness) § The original suggestions number (if making a revision) § Your revision/addition.

I am excited to begin the second round of data collection and look forward to your continued contributions. I intend to send out the link to the second round of data collection next week (July, 25-29). Please email me with any revisions and/or additions to the comprehensive lists if you have them, and if you have any questions or concerns. Thanks, Sterling INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 222

Efficacy Expert Panel Review of Comprehensive List (Efficacy)

Directions:

The comprehensive list of all suggestions provided by each expert panel member regarding how Individual Psychology may demonstrate efficacy given the current evidence based practice evaluation standards is provided below. The list was constructed by conducting qualitative content analysis of each individual panel member’s response to the research question. Repeated suggestions and suggestions deemed similar were combined resulting in a comprehensive list of eighty-one (81) suggestions.

Given that suggestions were combined each panel member is asked to review the compiled list to ensure that the list comprehensively represents the suggestions that they offered. Upon review:

• If the compiled list is comprehensively representative, panel members are asked to review the list in preparation of the second round of data collection.

• If panel members feel that the compiled list does not comprehensively represent the suggestions they offered, they are asked to offer revisions and/or additions to the list through email.

o Please send revisions/additions by email to [email protected]. For revisions: include the suggestions number and your revision to be offered to the compiled list.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 223

Prompt Provided to Each Panel Member Efficacy refers to whether proposed beneficial effects of an intervention can be demonstrated scientifically (e.g., through treatment manualization and standardization, random assignment; random control trials, empirical outcome data, internal validity, etc.). In order to demonstrate efficacy as an Evidence Based Treatment, clinical models must satisfy the five established Efficacy Assessment Criteria (Hollon, Miller, & Robinson, 2002):

1. The (EBP evaluation guidelines require a clinical model to be grounded in and based on careful consideration of a broad base of relevant empirical literature. 2. EBP evaluation guidelines require the research methodology supporting a clinical model to demonstrate the highest level of rigor and sophistication (i.e., meta analysis and randomized controlled trials as opposed to qualitative research, clinical opinion, case studies, etc.). 3. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence that its treatment outcomes are superior to treatment outcomes of other comparable clinical models and to treatment outcomes from not engaging in treatment. 4. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence to support its selection for use with specific patients. 5. EBP evaluation guidelines require the intended treatment outcomes of a clinical model to be specified, and the actual treatment outcomes to be quantitatively evaluated in relation to (or “against the influence of”) variables within the specific treatment context. (i.e., treatment goals; measures of life functioning; attrition; long-term/indirect consequences of treatment; negative consequences; client satisfaction; clinical significance; and methods). Please be as thorough and comprehensive as possible (a minimum of 500 words) in providing specific suggestions regarding how you believe individual psychology is already meeting each of the five criteria or how it can do so in the future.

Research Question 1: How can individual psychology demonstrate efficacy given the current EBP evaluation standards? INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 224

Comprehensive list of all Efficacy suggestions 1. Individual Psychology must develop a stronger research base. 2. Individual Psychology must encourage practitioners and researchers to pool their resources and collaborate on research projects. 3. More effort must be put into recognizing supporting and incentivizing (through The North American Society of Adlerian Psychology and external funding) the research efforts (Specifically Empirical Research) of Adlerian researchers and practitioners (who publish within and outside of the Journal of Individual Psychology) 4. Individual Psychology must conduct more outcome-based research. 5. Individual psychology must focus research efforts on testing fundamental hypotheses based on the theory of individual psychology in order to develop a more solid literature base. 6. Individual Psychology must specifically define basic Adlerian constructs and the core components of the theory (lifestyle, encouragement, life-task, etc.) in an empirically testable form, and distinguish them from assessments and treatments to design empirical studies based upon those distinctions. 7. Individual Psychology must clarify if lifestyle is simply personality or broader than personality, in order to better identify the effect of treatment on lifestyle. 8. Individual Psychology must operationally define constructs and develop instrumentation that represents and measures those constructs individually, as well as the effects of treatment on those constructs. 9. Individual psychology must utilize established instruments and psychometrics, which have been proven to establish efficacy and measure change, to conduct pre-/post-tests related to the efficacy of specific Adlerian interventions. (Examples of instruments: Becks Depression Inventory; Becks Anxiety Inventory; Early Recollections Rating Scale Manaster/Perryman, Millers; Session Rating Scale; Sullimans Social Interest Scale) 10. Individual Psychology must support and develop an inclusive and broad research initiative that invites scholars and practitioners to develop and be involved in generating research. 11. Individual Psychology must review, emphasize, replicate, redesign, and utilize the Adlerian empirical literature and data that is presently available regarding the efficacy and influence of Adlerian concepts, interventions, and instruments to demonstrate of the current efficacy of individual psychology. 12. Individual Psychology must utilize the quantitative evidence and support for Adlerian parent education programs that are considered EST (STEP Program, Active Parenting, Etc.) to provide evidence for selecting Adlerian treatment models for use with specific clinical populations, and to provide evidence to support that Adlerian parent education programs provide superior treatment outcomes compared to not engaging in treatment. 13. Given the influence that individual psychology has had on cognitive behavioral therapy, individual psychology can utilize the evidence supporting CBT’s efficacy and INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 225

demonstrate its own efficacy by distinguish itself from CBT based on influence and effects that Adlerian techniques (lifestyle assessment) have on the therapeutic process. 14. Individual Psychology must utilize the BASIS-A as a foundation for assessing the effect of lifestyle assessment on the treatment process as a means to distinguish individual psychology from cognitive behavioral therapy. 15. Individual psychology must conduct comparative research (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma). 16. Individual Psychology could develop treatment manuals similar to the process that interpersonal psychotherapists have. 17. Individual Psychology must provide evidence to support the need for Individual Psychology to begin to conduct research regarding its efficacy. 18. Adlerian graduate programs must prioritize the training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. 19. Individual psychology must explore and utilize a broad empirical literature base (non- Adlerian literature) from models outside of Adlerian psychology to support Adlerian concepts that are researched in other models (concepts such as belonging, social connectedness, social interest, and family constellation and atmosphere that are shown to be relevant aspects of clinical models from the CBT approach). 20. Individual Psychology must utilize literature-comparing BASIS-A to other instruments. 21. Individual Psychology must conduct research utilizing the BASIS-A to demonstrate clinical outcomes. 22. Individual Psychology must conduct research establishing and demonstrating that Adlerian counseling (specifically lifestyle assessment) promotes deeper understanding, encourages motivation for change, and is a powerful insight-building tool compared to standard clinical interviews based on DSM/ICD systems and or straight DBT and CBT skills. 23. Individual Psychology must conduct Adlerian based research to demonstrate the efficacy of Adlerian interventions with specific populations. 24. Individual Psychology must commission several methodology experts and/or establish an executive research planning and oversight team to establish a study design that meets the required EBP evaluation standards for efficacy, and to provide evaluation over research projects. 25. Individual psychology needs to conduct research using double blind randomized control trials. 26. Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk-therapy strategies compared to “treatment as usual”, a no-treatment group, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 226

or a waiting list group of clients who are not currently receiving care, longitudinally if possible. 27. Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). 28. Individual Psychology must establish an Adlerian research task force to seek groups/practices to implement experimental studies. 29. Individual Psychology must establish an Adlerian research task force to seek collaboration from several Adlerian experts to assemble an Adlerian treatment strategy manual 30. Individual Psychology must utilize training videos/appropriate supervision to develop treatment fidelity. 31. Individual Psychology must utilize Certified Adlerian trainers (possibly The North American Society of Adlerian Psychology diplomats) to offer training and supervision to ensure treatment fidelity. 32. Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. 33. Individual Psychology must manualize specific individual psychology interventions such as push-button technique, reflecting as if, three-step emotional change trick and other Adlerian approaches. 34. Individual Psychology must attempt to loosely manualize techniques to serve as an outline that clinicians can adapt based on individual cases. 35. Individual Psychology must publish outside of Adlerian-based journals. 36. Individual Psychology must increase professional development opportunities to have more training in research. At The North American Society of Adlerian Psychology and local conferences there can be specific pre-/post-conference workshops where individuals get specific training on research process and statistics. Specific strand at the conference could be offered on research, maybe through open forums on research ideas, or sharing research results. 37. Individual Psychology must encourage the Adlerian community to provide more mentorship to and Adlerian graduate programs masters and doctoral students’ conducting research. 38. When the Journal of Individual Psychology receives an empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 227

Therefore, the process is more encouraging and it means more publications of empirical research. 39. Individual Psychology must offer auxiliary to the conferences, specific training, workshop, or conference on research in Adlerian theory (Gestalt practitioners are doing this). 40. Individual Psychology must provide practitioners with training in systematic case study, single case time series, and single case experimental designs. 41. Individual Psychology must emphasize, support, and award the scientist/practitioner model (could establish a program similar to emerging leader program, an emerging research practitioners can be started). 42. The journal of Individual Psychology must be indexed in the Social Science Citation Index to encourage more international researchers to publish in it (SCI journals are more valued in their organizations and in their countries). 43. Individual Psychology must reach out to the international community to become involved in empirical research (this is how Gestalt research programs have started increasing empirical research). 44. Individual Psychology must reach out to sister organizations, Association of Humanistic Counseling, Division 32 of APA, other postmodern and phenomenological therapies to explore empirical research options. 45. Individual Psychology must offer research grants from The North American Society of Adlerian Psychology (Clonick) and assist its members in seeking external grants. 46. Individual Psychology must encourage regional groups to establish their research circles and publish. 47. Individual Psychology could establish another journal (maybe online) focusing only on Adlerian research studies. 48. Individual Psychology must conduct meta-analysis studies of empirical literature regarding Adlerian constructs and treatment interventions. 49. Individual Psychology must develop a researcher in residence program that is funded to conduct research (possibly have Adlerian graduate programs initiate) 50. Individual Psychology must train researchers and practitioners on process based research. Such as Narrative Correspondence Method, prospective naturalistic study, etc.. 51. Individual Psychology must link researchers together to target one intervention at a time to pursue gathering enough data. 52. Individual Psychology must identify specific interventions to be extensively researched such as lifestyle assessment interpretation, use of metaphors (Kopp metaphor intervention), use of paradox, interpretation of ER’s, use of encouragement, use of stories imagery techniques such as push button, reflecting as if, interpreting BASIS-A 53. Individual Psychology should look at previous studies and methodologies used by brief dynamic theories to develop research methodologies to specify intended outcomes and evaluate the efficacy of individual psychology to meet those outcomes. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 228

54. Treatment manuals must be developed that include matrices that track skills and techniques that should occur in each phase of treatment in order to establish treatment fidelity. 55. Treatment manuals should be developed that include case conceptualization that occurs after the second phase of treatment and before proceeding with the third and fourth phase. 56. Treatment manuals must be developed for treating broad groups (individual work with adults, group work, family work, etc.). 57. Treatment manuals must be constructed for treatment with in each of the broad groups for use with specific populations, various clinical diagnosis, and problem areas within each broad group (i.e., individual work with adults who experience depression, group work with teens with anxiety, family work with step families who have experienced trauma, etc.). 58. An expert in the application of individual psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity. 59. Researchers can look at how other clinical models have established efficacy and design studies similar to those done in the published empirical literature. 60. Researchers can work to design better instruments that measure factors affected by intervention with individual psychology and find instruments used in well-designed studies of other clinical models that have already established themselves as efficacious according to the current EBP evaluations standards and use those instruments in studies measuring the efficacy of individual psychology. 61. Researchers should measure short term and long-term effects of Individual Psychology interventions. 62. Individual Psychology must collaborate with staff at a variety of college counseling centers so that counselors who are working with college students who receive counseling services in these centers are using individual psychology as the basis for their work and then use their clients to do pre-post tests control group studies. 63. Adlerian oriented theorist need to align themselves with institutions that will sponsor and support empirical research. 64. Adlerian theorists must clearly define the critical components of their techniques and the parameters of positive outcomes. 65. Adlerians must improve their research design and utilize more than one inventory to validate the theory and or treatment. 66. Individual Psychology must acquire the personnel and research sophistication to plan and implement clinical trials in Adlerian training clinics. 67. Individual Psychology must utilize research design that is consistent with established EBP criteria. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 229

68. Individual Psychology must utilize 20-30 therapists (trainees and experienced clinicians) at more than 5 or more sites with approximately 300 clients in efficacy-based research. 69. Individual Psychology must plan a standardized intervention protocol for all sites that would be used during clinical trials. 70. Individual Psychology must evaluate session-to-session monitoring of both clinical outcomes and therapeutic alliance. 71. Individual Psychology must systematically collect data from participants at multiple sites and enter in online data site, which has been specifically constructed or modified and field-tested for use Adlerian research projects. 72. Individual Psychology must statistically analyze data, tabulate results, and disseminate written reports on the efficacy of individual psychology. 73. Individual Psychology must encourage Adlerians to conduct experimental research related to the efficacy of individual psychology through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. 74. Individual Psychology must teach specific concepts of experimental research through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. Instructions that would emphasize supporting the efficacy of individual psychology would be an important aspect of the concepts taught. 75. Individual Psychology must teach the process for submitting experimental research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting experimental research about the efficacy of specific techniques of individual psychology and submitting the results to SAMSHA for inclusion as Evidenced-Based Practice on the national register. 76. A detailed discussion with Michael Popkin about the process that was enacted to have “Active Parenting” accepted, as an Evidenced-Based Practice on the national register must happen. 77. Adlerians must present their experimental research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians in conducting experimental research related to the efficacy of techniques coming from individual psychology. 78. The Journal of Individual Psychology must feature an article each issue about the importance of Adlerians conducting experimental research and Meta analyses about the efficacy of individual psychology. 79. Individual Psychology needs to develop a model that can be easily disseminated. 80. Individual Psychology needs to view the challenges of demonstrating efficacy as an opportunity to re-examine its methodology and/or modify the clinical model/interventions. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 230

81. Individual Psychology needs to spend less time justifying its views and beliefs and spend more time putting them through the ringer and admitting errors or the need for modification.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 231

Effectiveness Expert Panel Review of Comprehensive List (Effectiveness)

Directions:

The comprehensive list of all suggestions provided by each expert panel member regarding how Individual Psychology may demonstrate Effectiveness given the current evidence based practice evaluation standards is provided below. The list was constructed by conducting qualitative content analysis of each individual panel member’s response to the research question. Repeated suggestions and suggestions deemed similar were combined resulting in a comprehensive list of fifty-four (54) suggestions.

Given that suggestions were combined each panel member is asked to review the compiled list to ensure that the list comprehensively represents the suggestions that they offered. Upon review:

• If the compiled list is comprehensively representative panel members are asked to review the list in preparation of the second round of data collection.

• If panel members feel that the compiled list does not comprehensively represent the suggestions they offered, they are asked to offer revisions and/or additions to the list through email.

o Please send revisions/additions by email to [email protected]. For revisions: include the suggestions number and your revision to be offered to the compiled list.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 232

Prompt Provided to Each Panel Member Effectiveness refers to whether an intervention is generalizable and feasible (practically and financially) for implementation with various populations, settings, and clinicians. In order to demonstrate effectiveness as an Evidence Based Treatment, clinical models must satisfy the five Effectiveness Assessment Criteria that have been established for Evidence Based Treatments (Hollon, Miller, & Robinson, 2002):

1. EBP evaluation guidelines require a clinical model to be able to quantify the influence that patient variables may have on its treatment outcomes (i.e., complexity of clinical presentation; culture; gender/sex; age/developmental level; etc.).

2. EBP evaluation guidelines require a clinical model to be able to quantify the effect that different clinician’s will have on its treatment outcomes (i.e.; clinical skill; experience; culture/ethnic background; gender; etc.).

3. EBP evaluation guidelines require a clinical model to be able to quantify the influence that the treatment setting may have on its treatment outcomes (i.e.; home; school; day treatment; clinic; etc.).

4. EBP evaluation guidelines require a clinical model to be able to quantify the influence of alterations in its administration on treatment outcomes (i.e.; deviation from protocol; time frame; delivery method; etc.).

5. EBP evaluation guidelines require a clinical model to be able to quantify its feasibility (i.e., clients choice/willingness/and ability to participate in the intervention) and its benefit relative to cost (i.e.; financial cost to client/clinician; prevention of future disorders; medical costs; etc.) for those providing and receiving treatment. Please be as thorough and comprehensive as possible (a minimum of 500 words) in providing specific suggestions regarding how you believe individual psychology is already meeting each of the five criteria or how it can do so in the future.

Research Question 1: How can individual psychology demonstrate effectiveness given the current EBP evaluation standards?

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 233

Comprehensive list of All Effectiveness suggestions 1. Individual Psychology must conduct outcome studies 2. Individual Psychology must acquire, align with, and utilize resources such as trained professionals and supportive research institutions to conduct controlled studies. 3. The North American Society of Adlerian Psychology and individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. 4. Individual Psychology must operationally define Individual Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner. 5. Individual Psychology must create instrumentation to measure Individual Psychologies constructs so outcome work can commence. 6. Individual Psychology must utilize existing instruments (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. 7. Individual Psychology needs more reliance on the scientific method, training in scientific method, and support for conducting strong empirical research. 8. Individual Psychology researchers need to go outside of IP and take steps to build a research base similar to the process used by other empirically supported treatments (cognitive therapy; behavior therapy). 9. Individual Psychology must select one intervention/technique and develop a mode for treating specific types of problems and then conduct research regarding the effectiveness of the interventions in multiple settings. 10. Individual Psychology must develop and/or utilize an existing program evaluation model (CBT and IPT have already established evaluation models) to look at inputs to identify client, clinician, and setting characteristics, while specifying treatment and alterations while measuring outcomes. 11. The North American Society Of Adlerian Psychology must provide grants and funding for researchers to build a program evaluation model. 12. Although current Adlerian literature includes demographic information and has analyzed various factors such as gender/sex, age, etc., individual psychology must conduct research regarding the extent that various factors effect Adlerian treatment outcomes. 13. Individual Psychology must conduct case studies to demonstrate the extent that demographics and other factors have on Adlerian treatment outcomes. 14. Individual Psychology must develop and utilize a BASIS-A lifestyle assessment protocol to demonstrate effectiveness. 15. Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. 16. Individual Psychology must utilize the published case studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. 17. Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 234

18. In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). 19. Data Collection in future studies should include collecting data regarding the setting where treatment is provided (inpatient/outpatient/school), and other variables that will influence outcomes (medication, other therapeutic services being received, support system, support group, clients stage of change, etc.). 20. Adlerian fidelity measures should be created and utilized (similar to the Cognitive Therapy Rating Scale) that measures the competence of the clinician using Adlerian therapy, as a means to ensure treatment fidelity among clinicians and treatment provided in research studies. 21. Individual Psychology must set a minimum level of competency (measured via a constructed Adlerian fidelity measure) to be able to participate in specific empirical studies. 22. In order for clinicians to participate in effectiveness studies individual psychology should require clinicians to meet certification requirements including a minimum number of training hours, meeting a minimum level of competency on a developed Adlerian fidelity measure, and submission of counseling video demonstrating the use of Adlerian techniques that would be evaluated utilizing an established Adlerian therapy scale. 23. In order to quantify feasibility, individual psychology must develop an assessment protocol for the locations that are being considered for participation based on the benefits and challenges of each location where services may be offered (inpatient drug treatment center, college counseling centers, community mental health clinics, etc.). 24. Individual Psychology may utilize clinics and locations connected with Adler graduate programs to conduct research do to the readably available training and supervision that these clinics may be able to offer. 25. Individual Psychology must understand that there are various methods to demonstrate effectiveness such as Seligman’s consumer report study: a. Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. 26. Individual Psychology must develop a survey questionnaire and administer it to 20 Adlerian counselors, 20 CBT counselors, and 20 Eclectic counselors to evaluate counselor’s differential perception of their outcomes based on counselors ratings to demonstrate there is no difference between theoretical orientations related to treatment effectiveness. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 235

27. Individual Psychology must develop a develop a survey questionnaire and administer it to current or previous clients of 20 Adlerian counselors, 20 CBT counselors, and 20 Eclectic Counselors to evaluate clients differential perception of their outcomes from treatment based on client ratings to demonstrate no difference between theoretical orientations related to treatment effectiveness. 28. Given the only real difference between accepted EBP’s and individual psychology is verbage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. 29. Individual Psychology must utilize the variety of settings where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. 30. Researchers can partner with practitioners and international researchers in order to conduct outcome studies regarding the effectiveness of individual psychology in a variety of situations and with clients and clinicians with diverse backgrounds. 31. Individual Psychology should establish regional and international research groups (with researchers and practitioners coming from diverse backgrounds) to compare and control various research studies conducted to explore the effects of therapist variables on treatment process and outcomes. 32. Individual Psychology must utilize strong partnership and a mentorship process to support research efforts. 33. Individual Psychology should start a workgroup to develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual. 34. In order to establish effectiveness individual psychology must: develop treatment protocols and manuals for single interventions; Disseminate these to everyone (Adlerian or not) freely; Establish research training for Adlerians; Pair researchers with practitioners; Conduct outcome research on effectiveness; Publish the findings; and Repeat the process. 35. Adlerians already respect the influence patient variables (age, gender, sexual orientation, e.g.) have on treatment outcomes, but need to develop means to quantify this influence that maintains a respect for each person’s holistic way of being and uniqueness. 36. Individual Psychology must establish a means to quantify the influence that individual clinician’s skills have on treatment rather than quantify characteristics of a clinician. 37. Individual Psychology must utilize established measures besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social). 38. Individual Psychology must utilize Miller’s session Rating Scale to demonstrate that individual psychology approach is satisfying and that clients are willing to participate in counseling sessions. 39. Individual Psychology must conduct comparative outcome studies that use several different levels of a patient variable (i.e., very complex clinical presentation, moderate clinical presentation, simple clinical presentation) receiving Adlerian treatment compared to a no treatment group, and have a large enough sample sizes that the researcher can gather information about patient variables such as gender/sex; culture; age/developmental level and in the analysis of the results group the clients accordingly. INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 236

40. Individual Psychology must conduct comparative outcome studies that measure the effect that different clinicians have on the treatment outcomes by collecting data about professional identity, clinical experience, measures of clinical skill, fidelity measures, graduate degrees held, licensure status, numbers of years using Adlerian techniques, culture/ethnic background, gender, sexual orientation, and age of clinician in the demographic information gathered, and use these in the variables used in the data analysis in order to monitor the effect of each clinicians on treatment outcomes among clients (large sample size so that the subgroups will have a large enough number of subjects to be relevant in the data analysis). 41. Individual Psychology must conduct comparative studies that have specific elements altered in the administration of the treatment (with specific elements changed in each group). For instance, Adlerian play therapy with and without parent consultation; Adlerian play therapy with parent consultation compared to Adlerian play therapy with teacher consultation; Adlerian play therapy that lasts 16 sessions, compared to Adlerian play therapy that lasts 30 sessions; twice a week sessions compared to once a week sessions; etc. 42. Individual Psychology must conduct studies with populations that might have a long-term financial benefit from therapy (school children, prisoners with dual diagnoses, etc.). 43. Individual Psychology must conduct studies with clients with some kind of medical condition that might also be alleviated or dissipated by medical intervention combined with therapy (people with ulcers, people with diabetes, etc.). 44. Individual Psychology must conduct Efficiency studies (practice based Evidence studies) that are concerned with real world applications of Individual Psychologies treatment model in everyday treatment settings, and focus on session-to-session client self- comparison rather than comparing client outcomes to group means and aggregated client outcomes as used in effectiveness research (Note: Practice-Based-Evidence is the converse of the Evidence-Based-Practice model. At the present time, such studies would be eligible for listing in the National Registry of Evidence-Based Programs and Practices [SAMSHA], but not in the Research-Supported Psychological Treatments (APA-Division 12). 45. Individual Psychology must implement practice-based research at mental health agencies and private practice offices where there are Adlerian clinicians practicing. 46. Individual Psychology must have Adlerian faculty focus even more on teaching and encouraging Adlerian-oriented students the skills that they will need to conduct quantitative studies and case studies that support the clinical effectiveness of individual Psychology. 47. Individual Psychology must have the North American Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. 48. Individual Psychology must encourage Adlerians to conduct qualitative research of the clinical effectiveness of individual psychology through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. 49. Individual Psychology must teach specific concepts of qualitative research through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 237

Research, and Teaching section of the North American Society for Adlerian Psychology. Instructions that would emphasize supporting the clinical effectiveness of individual psychology and financial feasibility of the application of Adlerian techniques would be an important aspect of the concepts taught. 50. Individual Psychology must teach the process for submitting qualitative research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting qualitative research about the clinical effectiveness of specific techniques individual psychology and submitting the results to SAMSHHA for inclusion as Evidenced-Based Practice on the national register. A preponderance of qualitative research demonstrating clinical applicability can result in acceptance. 51. Individual Psychology must have the Theory, Research, and Teaching TRT section of the North American Society for Adlerian Psychology implement a research team approach to conduct qualitative research related to the clinical effectiveness of individual psychology. This research team could be coordinated through the TRT listserve. 52. Individual Psychology must have Adlerians present their qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. 53. Individual Psychology must highlight qualitative research activities at the continental annual conference of The North American Society Of Adlerian Psychology to bring attention to the importance of these activities to support the clinical effectiveness of individual psychology. 54. Individual Psychology must have the Journal of Individual Psychology feature an article each issue about the importance of Adlerians conducting qualitative research and case studies about the clinical effectiveness of individual psychology.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 238

Appendix E

Recruitment Email Inviting Panel Members to Participate in Round Two

,

Thank you for reviewing both the lists and providing me with your feedback. I have received many thoughtful responses from our diverse panel of experts, and I am excited for this second round to get underway. I am attaching the link for the second round at the bottom of this email. In this round I will ask you to rate all suggestions in both the efficacy and effectiveness lists. I will ask that you rate each suggestion based on the perceived feasibility of the suggestion to be implemented, and also to rate each suggestion based on the perceived benefit of the suggestion if implemented. Lastly, I will ask you to select and rank order your “top twenty-five” suggestions from each list. All of these directions are provided in more detail in the Qualtrics survey.

I truly appreciate how much effort you are putting into this study, and I am excited to see how each panel member’s ratings and rank-ordering of each suggestion effect both comprehensive lists. In my previous email I noted that I would like the second round to be completed by all panel members in two weeks in order to maintain my procedural timeline. I am asking all panel members to complete the second round by Tuesday, August 23rd. If there are any concerns with this date, or if you have any questions regarding the second round please contact me.

Thank you again, and I look forward to this exciting round as the panel continues to work towards consensus.

Link:

Thanks,

Sterling

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 239

Appendix F

Round Two Qualtrics Questionnaire

Round Two Second Round Questionnaire: The second round of the study will attempt to identify points of consensus around how, can, and should Individual Psychology address increasing environmental demands for evidence based practice. It is hoped that the time and effort you put into completing this critical round will provide useful guidance for future directions in Individual Psychology research and practice. Two compiled lists of all suggestions from the first round were provided previously for your review and possible revision. In this second round of this study I ask that, for each of the two, now revised, lists, (a) you rate the suggestions based on their perceived feasibility and benefit, and (b) you select and rank-order what you perceive to be the "top twenty-five" (25) suggestions.

Specifically, I am asking that you rate all suggestions from both comprehensive lists using two seven-point Likert scales. The first Likert rating will ask you to rate each suggestion based on the can component of each research question in terms of the perceived feasibility of each suggestion to be implemented (seven point Likert scale ranging from definitely can not to definitely can). The second Likert scale will ask you to rate each suggestion based on the should component of each research question in terms of the perceived benefit of each suggestion as a means to addressing how Individual Psychology may demonstrate efficacy/effectiveness given the evidence based practice evaluation standards (seven point Likert scale ranging from Absolutely no benefit to great deal of benefit).

After rating each list, I then ask that you select from each comprehensive list the "top twenty- five" suggestions that you believe best address the how component (how individual psychology may demonstrate efficacy/effectiveness given the current evidence based practice evaluation standards). Finally, I ask that you rank-order the "top twenty-five" list that you selected from most to least based on the perceived utility of each suggestion to address the how component of each research question. . Please type your name in the box below (your name will only be used to pair your responses across individual rounds of the Delphi study; and your name will remain confidential and your identity will not be divulged).

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 240

Evaluation Criteria Provided to Each Panel Member During the First Round

Below I have provided you with the the Assessment Criteria (Hollon, Miller, & Robinson, 2002) for both Efficacy and Effectiveness that was used while completing the first round of this study.

Efficacy Evaluation Criteria Efficacy refers to whether proposed beneficial effects of an intervention can be demonstrated scientifically (e.g., through treatment manualization and standardization, random assignment; random control trials, empirical outcome data, internal validity, etc.). In order to demonstrate efficacy as an Evidence Based Treatment, clinical models must satisfy the five established Efficacy Assessment Criteria (Hollon, Miller, & Robinson, 2002):

1. The (EBP evaluation guidelines require a clinical model to be grounded in and based on careful consideration of a broad base of relevant empirical literature. 2. EBP evaluation guidelines require the research methodology supporting a clinical model to demonstrate the highest level of rigor and sophistication (i.e., meta analysis and randomized controlled trials as opposed to qualitative research, clinical opinion, case studies, etc.). 3. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence that its treatment outcomes are superior to treatment outcomes of other comparable clinical models and to treatment outcomes from not engaging in treatment. 4. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence to support its selection for use with specific patients. 5. EBP evaluation guidelines require the intended treatment outcomes of a clinical model to be specified, and the actual treatment outcomes to be quantitatively evaluated in relation to (or “against the influence of”) variables within the specific treatment context. (i.e., treatment goals; measures of life functioning; attrition; long-term/indirect consequences of treatment; negative consequences; client satisfaction; clinical significance; and methods).

Effectiveness Evaluation Criteria Effectiveness refers to whether an intervention is generalizable and feasible (practically and financially) for implementation with various populations, settings, and clinicians. In order to demonstrate effectiveness as an Evidence Based Treatment, clinical models must satisfy the five Effectiveness Assessment Criteria that have been established for Evidence Based Treatments (Hollon, Miller, & Robinson, 2002):

1. EBP evaluation guidelines require a clinical model to be able to quantify the influence that patient variables may have on its treatment outcomes (i.e., complexity of clinical presentation; culture; gender/sex; age/developmental level; etc.). 2. EBP evaluation guidelines require a clinical model to be able to quantify the effect that different clinician’s will have on its treatment outcomes (i.e.; clinical skill; experience; culture/ethnic background; gender; etc.). 3. EBP evaluation guidelines require a clinical model to be able to quantify the influence that the treatment setting may have on its treatment outcomes (i.e.; home; school; day treatment; clinic; etc.). INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 241

4. EBP evaluation guidelines require a clinical model to be able to quantify the influence of alterations in its administration on treatment outcomes (i.e.; deviation from protocol; time frame; delivery method; etc.). 5. EBP evaluation guidelines require a clinical model to be able to quantify its feasibility (i.e., client’s choice/willingness/and ability to participate in the intervention) and its benefit relative to cost (i.e.; financial cost to client/clinician; prevention of future disorders; medical costs; etc.) for those providing and receiving treatment.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 242

Efficacy Section Directions: In the following section on efficacy, you will first be presented with the comprehensive list of suggestions regarding efficacy, that I will ask you to rate using the two provided Likert scales. I then will ask you to select from the comprehensive list of suggestions regarding efficacy the "top twenty-five" suggestions that you believe best address the how component. Lastly, I will ask you to rank-order the "top twenty-five" efficacy suggestions you selected from most to least based on your personal belief regarding the perceived utility of each suggestion to address how individual psychology may demonstrate efficacy based on the current evidence based practice evaluation standards.

Round Two Efficacy Can & Should Component (Likert Scale Rating)

Please rate each suggestion utilizing the two Likert scales provided below:

For the Can Component, please base your rating on your personal belief regarding the perceived feasibility of each suggestion to be implemented by Individual Psychology, as a means of addressing how Individual Psychology may demonstrate efficacy given the current evidence based practice evaluation standards. The can component is rated on a seven point Likert scale ranging from Definitely Cannot to Definitely Can.

For the Should Component, please base your rating on your personal belief regarding the perceived benefit of each suggestion, as a means of addressing how individual psychology may demonstrate efficacy given the current evidence based practice evaluation standards. The should component is rated on a seven point Likert scale ranging from: Absolutely No Benefit to A Great Deal of Benefit.

The two Likert scales are presented side by side next to each suggestion. I ask that you rate each suggestion on both Likert scales.

[Panel Members were presented with all 81 efficacy suggestions, and asked to rate each suggestion using the two provided seven–point Likert rating scale]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 243

Round Two Efficacy How Component Selection of "Top Twenty-Five"

I ask that you select from this comprehensive list of efficacy suggestions the "top twenty- five" suggestions that you believe best address the how component (how individual psychology may demonstrate efficacy given the current evidence based practice evaluation standards).

To select the “top twenty-five” suggestions from the comprehensive list please click directly on the suggestion you are wishing to select (it will become highlighted).

Note: The order in which you select each suggestion will not be analyzed. ***I suggested that you use a pen and paper to keep track of the number of suggestions that you have selected as there is no digital counter incorporated into the survey software***

[Panel Members were presented with all 81 efficacy suggestions, and asked to select 25 suggestions to be rank ordered in the next section]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 244

Round Two Efficacy How Component Rank-Ordering of “Top Twenty-Five”

Presented below is the list of the “top twenty-five” suggestions that you personally selected form the comprehensive list of all suggestions related to efficacy. Please rank-order this list from most to least based on your personal belief regarding the perceived utility of each suggestion to address how individual psychology may demonstrate efficacy based on the current evidence based practice evaluation standards.

To rank-order each suggestion, please use the text box to the left of each suggestion and indicate the rank-order position you wish to assign to each suggestion. Please use numerical rankings (1,2,3...23,24,25), with a ranking of 1 representing the suggestion that you believe has the most perceived utility, and 25 representing the suggestion you believe has the least perceived utility. I ask that you rank-order every suggestion in your “top twenty-five.”

[The 25 efficacy suggestions selected for the “top twenty-five” in the previous section were presented to each panel for them to rank order]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 245

Effectiveness Section Directions: In the following section on effectiveness, you will first be presented with the comprehensive list of suggestions regarding effectiveness, that I will ask you to rate using the two provided Likert scales. I then will ask you to select from the comprehensive list of suggestions regarding effectiveness the "top twenty-five" suggestions that you believe best address the how component. Lastly, I will ask you to rank-order the "top twenty-five" effectiveness suggestions you selected from most to least based on your personal belief regarding the perceived utility of each suggestion to address how individual psychology may demonstrate effectiveness based on the current evidence based practice evaluation standards.

Round Two Effectiveness Can & Should Component (Likert Scale Rating)

Please rate each suggestion utilizing the two Likert scales provided below:

For the Can Component, please base your rating on your personal belief regarding the perceived feasibility of each suggestion to be implemented by Individual Psychology, as a means of addressing how Individual Psychology may demonstrate effectiveness given the current evidence based practice evaluation standards. The can component is rated on a seven point Likert scale ranging from Definitely Cannot to Definitely Can.

For the Should Component, please base your rating on your personal belief regarding the perceived benefit of each suggestion, as a means of addressing how individual psychology may demonstrate effectiveness given the current evidence based practice evaluation standards. The should component is rated on a seven point Likert scale ranging from: Absolutely No Benefit to A Great Deal of Benefit.

The two Likert scales are presented side by side next to each suggestion. I ask that you rate each suggestion on both Likert scales.

[Panel Members were presented with all 54 effectiveness suggestions, and asked to rate each suggestion using the two provided seven–point Likert rating scale]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 246

Round Two Effectiveness How Component Selection of "Top Twenty-Five" I ask that you select from this comprehensive list of effectiveness suggestions the "top twenty- five" suggestions that you believe best address the how component (how individual psychology may demonstrate effectiveness given the current evidence based practice evaluation standards).

To select the “top twenty-five” suggestions please click directly on the suggestion you are wishing to select (it will become highlighted).

Note: The order in which you select each suggestion will not be analyzed. ***I suggested that you use a pen and paper to keep track of the number of suggestions that you have selected as there is no digital counter incorporated into the survey software***

[Panel Members were presented with all 54 effectiveness suggestions, and asked to select 25 suggestions to be rank ordered in the next section]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 247

Round Two Effectiveness How Component Rank-Ordering of “Top Twenty-Five” Presented below is the list of the “top twenty-five” suggestions that you personally selected form the comprehensive list of all suggestions related to effectiveness. Please rank-order this list from most to least based on your personal belief regarding the perceived utility of each suggestion to address how individual psychology may demonstrate effectiveness based on the current evidence based practice evaluation standards.

To rank-order each suggestion, please use the text box to the left of each suggestion and indicate the rank-order position you wish to assign to each suggestion. Please use numerical rankings (1,2,3...23,24,25), with a ranking of 1 representing the suggestion that you believe has the most perceived utility, and 25 representing the suggestion you believe has the least perceived utility. Please rank-order every suggestion in your “top twenty-five.”

[The 25 effectiveness suggestions selected for the “top twenty-five” in the previous section were presented to each panel for them to rank order]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 248

Appendix G

Email to Panel Members Regarding Round Two Results

The responses from the second round of the study, A Delphi Study Regarding How, Can, and Should Individual Psychology Demonstrate Efficacy and Effectiveness Given the Current Evidence Based Practice Evaluation Standards, have been received and analyzed. Thanks so much for continuing to share your expertise.

Below you will find the results of the analysis of the second round. Each list of suggestions for both efficacy and effectiveness have been reduced to include the suggestions that you and at least 25% of the panel selected as the “top twenty-five” suggestions to demonstrate efficacy and effectiveness. In total there are 43 suggestions that were maintained for efficacy (Originally 81), and 44 suggestions maintained for effectiveness (originally 54). Each list is presented in descending order from the suggestion with the highest percentage of the panel that selected and ranked it, to the suggestion with the lowest percentage of the panel that selected and ranked it. Each suggestion is accompanied by a chart that illustrates the outcomes from the analysis of the second round. For each suggestion you will be presented in the chart with: • The name of the suggestion; • The personal rank that you assigned to the specific suggestion; • The frequency and percentage of panel members who selected the suggestion; • The Likert rating you assigned to the specific suggestion for the can component; • The median and interquartile range of all the Likert ratings for the can component regarding the specific suggestion; • The Likert rating you assigned to the specific suggestion for the should component; and • The median and interquartile range of all the Likert ratings for the should component regarding the specific suggestion.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 249

An example of the chart for each suggestion is provided below:

Effectiveness 1: Individual Psychology must conduct outcome studies Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Number Percentage Ranked the Suggestion in the Top 25 12 86%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 7 1 4 1 2 3 5 6 7 *Can Neither Definitely Mostly Somewhat Somewhat Mostly Definitely Component: Can or Cannot Cannot Cannot Can Can Can Cannot 7 1 2 3 4 5 6 A Great ** Should Absolutely Minimally Slightly Somewhat Moderately Very Deal Component No Beneficial Beneficial Beneficial Beneficial Beneficial Of Benefit Benefit

The second attachment is an expanded table that does not include your specific rankings and ratings of each item, but breaks the ratings down based on the percentage of the panel that ranked each suggestion between 1 and 5, 6 and 10, 11 and 15, 16 and 20, and 21 and 25. You may use this second table if you would like to see a more detailed break down of the panels rankings. You are not asked to respond to this email; however, please take some time to review the rankings and ratings of the overall panel, and prepare yourself to once again rank order and rate each of these suggestions in our third and final round. As you will see in the findings below, your commitment and efforts thus far have allowed us to begin to identify potential opportunities for individual psychology to address the evaluation standards of evidence based practice. I genuinely appreciate your continued efforts and am excited to see the results from our third and final round, as I am confident that the effort that you put in will have a lasting effect on the future of Adlerian research and practice. Thanks in advance for your continued participation. The third and final round will be sent out on Monday October 24th 2016, which will ask you to offer your final ranking and rating of our list of suggestions.

Sincerely, Sterling P. Travis INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 250

Efficacy Suggestions: 43 total Suggestions Maintained

Efficacy 1 - Individual Psychology must publish outside of Adlerian-based journals. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 11 79%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 7 1

Efficacy 2 - Individual Psychology must specifically define basic Adlerian constructs and the core components of the theory (lifestyle, encouragement, life- task, etc.) in an empirically testable form, and distinguish them from assessments and treatments to design empirical studies based upon those distinctions. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 7 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 251

Efficacy 3 - Individual Psychology must conduct research establishing and demonstrating that Adlerian counseling (specifically lifestyle assessment) promotes deeper understanding, encourages motivation for change, and is a powerful insight-building tool compared to standard clinical interviews based on DSM/ICD systems and or straight DBT and CBT skills. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 1

Efficacy 4 - Individual Psychology must conduct Adlerian based research to demonstrate the efficacy of Adlerian interventions with specific populations. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 1

Efficacy 5 - Individual Psychology must develop a stronger research base. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 7 0

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 252

Efficacy 6 - Individual Psychology must conduct more outcome-based research. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency PErcentage Ranked the Suggestion in the Top 25 9 64%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 7 1

Efficacy 7 - Individual psychology must conduct comparative research (preferably longitudinal with pre- and post-tests) regarding the efficacy of specified Adlerian interventions compared to other treatment modalities (CBT, Reality, Brief Dynamic, etc.) and/or no treatment, in working with specific populations and specific problem areas (individuals experiencing depression; Groups working with anger issues; Families recovering from trauma). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Likert Scale Rating Should Component Likert Scale Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 2 6 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 253

Efficacy 8 - Researchers can work to design better instruments that measure factors affected by intervention with individual psychology and find instruments used in well-designed studies of other clinical models that have already established themselves as efficacious according to the current EBP evaluations standards and use those instruments in studies measuring the efficacy of individual psychology. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 6 1

Efficacy 9 - Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk therapy strategies among clients meeting DSM-5 criteria for major depressive disorder, generalized anxiety disorder, substance abuse disorder, OCD, social anxiety, autism, PTSD, and other common clinical presentations (even chronic health considerations). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 7 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 254

Efficacy 10 - Individual psychology must utilize established instruments and psychometrics, which have been proven to establish efficacy and measure change, to conduct pre-/post-tests related to the efficacy of specific Adlerian interventions. (Examples of instruments: Becks Depression Inventory; Becks Anxiety Inventory; Early Recollections Rating Scale Manaster/Perryman, Millers; Session Rating Scale; Sullimans Social Interest Scale) Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 7 1

Efficacy 11 - Individual Psychology must operationally define constructs and develop instrumentation that represents and measures those constructs individually, as well as the effects of treatment on those constructs. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 2

Efficacy 12 - Individual Psychology must encourage practitioners and researchers to pool their resources and collaborate on research projects. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 2 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 255

Efficacy 13 - More effort must be put into recognizing supporting and incentivizing (through The North American Society of Adlerian Psychology and external funding) the research efforts (Specifically Empirical Research) of Adlerian researchers and practitioners (who publish within and outside of the Journal of Individual Psychology) Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 2 6 2

Efficacy 14 - Individual Psychology could develop treatment manuals similar to the process that interpersonal psychotherapists have. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 3

Efficacy 15 - Adlerian graduate programs must prioritize the training of its students in conducting empirical research, and encourage students to conduct and publish research utilizing empirical design. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 256

Efficacy 16 - Individual Psychology must increase professional development opportunities to have more training in research. At The North American Society of Adlerian Psychology and local conferences there can be specific pre-/post- conference workshops where individuals get specific training on research process and statistics. Specific strand at the conference could be offered on research, maybe through open forums on research ideas, or sharing research results. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 3

Efficacy 17 - When the Journal of Individual Psychology receives an empirical study (especially from a junior member) instead of rejecting the manuscript or having it go through the regular review process; the author can be paired with an established scholar for mentorship. Therefore, the process is more encouraging and it means more publications of empirical research. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range

7 2 5 5

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 257

Efficacy 18 - Individual Psychology must offer auxiliary to the conferences, specific training, workshop, or conference on research in Adlerian theory (Gestalt practitioners are doing this). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5 2

Efficacy 19 - Individual Psychology must utilize research design that is consistent with established EBP criteria. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 6 2

Efficacy 20 - Individual psychology must focus research efforts on testing fundamental hypotheses based on the Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range

6 2 6 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 258

Efficacy 21 - Individual Psychology must review, emphasize, replicate, redesign, and utilize the Adlerian empirical literature and data that is presently available regarding the efficacy and influence of Adlerian concepts, interventions, and instruments to demonstrate of the current efficacy of individual psychology. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Frequency Percentage Who Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 2

Efficacy 22 - Given the influence that individual psychology has had on cognitive behavioral therapy, individual psychology can utilize the evidence supporting CBT’s efficacy and demonstrate its own efficacy by distinguish itself from CBT based on influence and effects that Adlerian techniques (lifestyle assessment) have on the therapeutic process. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 2 5 5

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 259

Efficacy 23- Individual Psychology must commission several methodology experts and/or establish an executive research planning and oversight team to establish a study design that meets the required EBP evaluation standards for efficacy, and to provide evaluation over research projects. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 1

Efficacy 24 - Individual Psychology must conduct a series of single case design experiments such as multiple baseline single case design to quantitatively evaluate both treatment process and treatment outcomes in a well-controlled application of an Adlerian-based therapy model, and conduct follow-up or concurrent research, following an initial single case design, to replicate the same study in a different research lab with a different principle investigator. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 260

Efficacy 25 - Individual Psychology must manualize specific individual psychology interventions such as push-button technique, reflecting as if, three- step emotional change trick and other Adlerian approaches. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 3

Efficacy 26 - Individual Psychology must offer research grants from The North American Society of Adlerian Psychology (Clonick k ) and assist its members in seeking external grants. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 2 4 2

Efficacy 27 - Individual Psychology must conduct meta-analysis studies of empirical literature regarding Adlerian constructs and treatment interventions. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 2 2 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 261

Efficacy 28 - Individual Psychology must teach the process for submitting experimental research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting experimental research about the efficacy of specific techniques of individual psychology and submitting the results to SAMSHA for inclusion as Evidenced- Based Practice on the national register. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 3.5 3

Efficacy 29 - Researchers should measure short term and long-term effects of Individual Psychology interventions. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 2

Efficacy 30 - An expert in the application of individual psychology who is familiar with, once developed, the treatment manual and the accepted application of clinical practice with a specified population should train clinicians in utilizing manualized Adlerian treatment in order to ensure treatment fidelity. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 262

6 2 5 2

Efficacy 31 - Individual Psychology must identify specific interventions to be extensively researched such as lifestyle assessment interpretation, use of metaphors (Kopp metaphor intervention), use of paradox, interpretation of ER’s, use of encouragement, use of stories imagery techniques such as push button, reflecting as if, interpreting BASIS-A Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentag Ranked the Suggestion in the Top 25 e 5 36%

Can Component Rating Should Component Rating Persona Median Interquartile Personal Median Interquarti l Rating* Range Rating** Rating** le Range Rating* 6 2 5 2

Efficacy 32 - Individual Psychology must utilize training videos/appropriate supervision to develop treatment fidelity. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 263

Efficacy 33 - Individual Psychology must implement experimental studies to examine the efficacy of Adlerian talk-therapy strategies compared to “treatment as usual”, a no-treatment group, or a waiting list group of clients who are not currently receiving care, longitudinally if possible. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 7 2

Efficacy 34 - Individual psychology must explore and utilize a broad empirical literature base (non-Adlerian literature) from models outside of Adlerian psychology to support Adlerian concepts that are researched in other models (concepts such as belonging, social connectedness, social interest, and family constellation and atmosphere that are shown to be relevant aspects of clinical models from the CBT approach). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 264

Efficacy 35 - Individual Psychology needs to view the challenges of demonstrating efficacy as an opportunity to re-examine its methodology and/or modify the clinical model/interventions. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 4.5 3

Efficacy 36 - Adlerians must present their experimental research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians in conducting experimental research related to the efficacy of techniques coming from individual psychology. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 4.5 3

Efficacy 37 - Individual Psychology must plan a standardized intervention protocol for all sites that would be used during clinical trials. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 5.5 0

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 265

Efficacy 38 - Individual Psychology must utilize 20-30 therapists (trainees and experienced clinicians) at more than 5 or more sites with approximately 300 clients in efficacy-based research. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 1

Efficacy 39 - Treatment manuals must be constructed for treatment with in each of the broad groups for use with specific populations, various clinical diagnosis, and problem areas within each broad group (i.e., individual work with adults who experience depression, group work with teens with anxiety, family work with step families who have experienced trauma, etc.). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 266

Efficacy 40 - Individual Psychology should look at previous studies and methodologies used by brief dynamic theories to develop research methodologies to specify intended outcomes and evaluate the efficacy of individual psychology to meet those outcomes. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6.5 1 6 3

Efficacy 41 - Individual Psychology must reach out to the international community to become involved in empirical research (this is how Gestalt research programs have started increasing empirical research). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5 3

Efficacy 42 - Individual Psychology must establish an Adlerian research task force to seek groups/practices to implement experimental studies. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 267

Efficacy 43- Individual psychology needs to conduct research using double blind randomized control trials. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 2 5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 268

Effectiveness Suggestions: 44 Total Suggestions Maintained

Effectiveness 1: Individual Psychology must conduct outcome studies Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 12 86%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 7 1

Effectiveness 2 - Individual Psychology must teach the process for submitting qualitative research to SAMSHA at the annual conference of North American Society for Adlerian Psychology. This would emphasize conducting qualitative research about the clinical effectiveness of specific techniques individual psychology and submitting the results to SAMSHHA for inclusion as Evidenced- Based Practice on the national register. A preponderance of qualitative research demonstrating clinical applicability can result in acceptance. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 11 79%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 269

Effectiveness 3 - Individual Psychology must utilize established measures besides client reports to measure changes in Adlerian life tasks (intimacy, work, and social). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 1 5.5 2

Effectiveness 4 - Individual Psychology must utilize strong partnership and a mentorship process to support research efforts. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5 1

Effectiveness 5 - Individual Psychology may utilize clinics and locations connected with Adler graduate programs to conduct research do to the readably available training and supervision that these clinics may be able to offer. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5.5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 270

Effectiveness 6 - Individual Psychology must implement quantitative studies that meet the requirements of EBP (utilizing experimental double blind randomized control methodology) Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6.5 1

Effectiveness 7 - Individual Psychology must utilize the published case studies that exemplify the use of Adlerian strategies used with clients from various cultures and a variety of clinical presentations (examining patient variables that influence outcomes that are controlled for in the data analysis phase) to demonstrate Individual Psychologies effectiveness. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5 1

Effectiveness 8 - Individual Psychology must operationally define Individual Psychologies constructs compared to constructs from other disciplines, and do a better job of presenting the Adlerian clinical model in a concrete and defined manner. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 10 71%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 2 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 271

Effectiveness 9 - Individual Psychology must acquire, align with, and utilize resources such as trained professionals and supportive research institutions to conduct controlled studies. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 1

Effectiveness 10 - Adlerian fidelity measures should be created and utilized (similar to the Cognitive Therapy Rating Scale) that measures the competence of the clinician using Adlerian therapy, as a means to ensure treatment fidelity among clinicians and treatment provided in research studies. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 272

Effectiveness 11 - Individual Psychology must conduct Efficiency studies (practice based Evidence studies) that are concerned with real world applications of Individual Psychologies treatment model in everyday treatment settings, and focus on session-to-session client self-comparison rather than comparing client outcomes to group means and aggregated client outcomes as used in effectiveness research (Note: Practice-Based-Evidence is the converse of the Evidence-Based-Practice model. At the present time, such studies would be eligible for listing in the National Registry of Evidence-Based Programs and Practices [SAMSHA], but not in the Research-Supported Psychological Treatments (APA-Division 12). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 2 5 2

Effectiveness 12 - Individual Psychology must have the North American Society for Adlerian Psychology financially support qualitative research efforts through their Clonick grants or other available funding sources in order to demonstrate clinical application and financial feasibility. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 5 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 273

Effectiveness 13 - Individual Psychology must have Adlerians present their qualitative research findings at non-Adlerian conferences for the purpose of interesting non-Adlerians to conduct qualitative research on the clinical effectiveness of techniques coming from individual psychology. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 9 64%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5 3

Effectiveness 14 - Individual Psychology must highlight qualitative research activities at the continental annual conference of The North American Society Of Adlerian Psychology to bring attention to the importance of these activities to support the clinical effectiveness of individual psychology. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 4.5 3

Effectiveness 15 - Individual Psychology must have Adlerian faculty focus even more on teaching and encouraging Adlerian-oriented students the skills that they will need to conduct quantitative studies and case studies that support the clinical effectiveness of individual Psychology. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 274

6 2 5.5 2

Effectiveness 16 - Researchers can partner with practitioners and international researchers in order to conduct outcome studies regarding the effectiveness of individual psychology in a variety of situations and with clients and clinicians with diverse backgrounds. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 2 5 3

Effectiveness 17 - The North American Society Of Adlerian Psychology must provide grants and funding for researchers to build a program evaluation model. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 5 4

Effectiveness 18 - Individual Psychology must develop and/or utilize an existing program evaluation model (CBT and IPT have already established evaluation models) to look at inputs to identify client, clinician, and setting characteristics, while specifying treatment and alterations while measuring outcomes. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 275

6.5 1 6 0

Effectiveness 19 - Individual Psychology researchers need to go outside of IP and take steps to build a research base similar to the process used by other empirically supported treatments (cognitive therapy; behavior therapy). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5.5 3

Effectiveness 20 - The North American Society of Adlerian Psychology and individual psychology need to emphasize, highlight, and support the need for empirical support through outcome research. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 8 57%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 7 1 6 3

Effectiveness 21 - Individual Psychology must create instrumentation to measure Individual Psychologies constructs so outcome work can commence. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 276

Effectiveness 22 - Individual Psychology must utilize existing instruments (BDI-II, STAI, Etc.) to demonstrate the effects of lifestyle analysis, encouragement, private logic restructuring and other individual psychology interventions have on treatment outcomes. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6.5 1 6 2

Effectiveness 23 - Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5.5 2

Effectiveness 24 - Data Collection in future studies should include collecting data regarding the setting where treatment is provided (inpatient/outpatient/school), and other variables that will influence outcomes (medication, other therapeutic services being received, support system, support group, clients stage of change, etc.). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 277

6 2 5.5 2 Effectiveness 25 - Individual Psychology must set a minimum level of competency (measured via a constructed Adlerian fidelity measure) to be able to participate in specific empirical studies. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who 7 50% Ranked the Suggestion in the Top 25

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 4.5 3

Effectiveness 26 - Given the only real difference between accepted EBP’s and individual psychology is verbiage, individual psychology may conduct correlational research using both EBP models and individual psychology concepts to make direct links between the two models, and thus establish Adlerian concepts, techniques, etc. as an EBP. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5 3

Effectiveness 27 - Individual Psychology must utilize the variety of settings where Adlerians are represented and the outcome measures available to conduct research in various settings with various client groups. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 1 4 2 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 278

Effectiveness 28 - Individual Psychology should start a workgroup to develop an easily disseminated treatment manual and start recruiting research practitioners to field-test the treatment manual. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 6 3

Effectiveness 29 - Individual Psychology must conduct comparative outcome studies that use several different levels of a patient variable (i.e., very complex clinical presentation, moderate clinical presentation, simple clinical presentation) receiving Adlerian treatment compared to a no treatment group, and have a large enough sample sizes that the researcher can gather information about patient variables such as gender/sex; culture; age/developmental level and in the analysis of the results group the clients accordingly. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 6 1

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 279

Effectiveness 30 - Individual Psychology must implement practice-based research at mental health agencies and private practice offices where there are Adlerian clinicians practicing. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5 2

Effectiveness 31 - Individual Psychology must have the Theory, Research, and Teaching TRT section of the North American Society for Adlerian Psychology implement a research team approach to conduct qualitative research related to the clinical effectiveness of individual psychology. This research team could be coordinated through the TRT listserve. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 4 2 5 2

Effectiveness 32 - Individual Psychology must conduct studies with populations that might have a long-term financial benefit from therapy (school children, prisoners with dual diagnoses, etc.). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 5 2 INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 280

Effectiveness 33 - Individual Psychology must conduct comparative studies that have specific elements altered in the administration of the treatment (with specific elements changed in each group). For instance, Adlerian play therapy with and without parent consultation; Adlerian play therapy with parent consultation compared to Adlerian play therapy with teacher consultation; Adlerian play therapy that lasts 16 sessions, compared to Adlerian play therapy that lasts 30 sessions; twice a week sessions compared to once a week sessions; etc. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6 1

Effectiveness 34 - Individual Psychology must conduct comparative outcome studies that measure the effect that different clinicians have on the treatment outcomes by collecting data about professional identity, clinical experience, measures of clinical skill, fidelity measures, graduate degrees held, licensure status, numbers of years using Adlerian techniques, culture/ethnic background, gender, sexual orientation, and age of clinician in the demographic information gathered, and use these in the variables used in the data analysis in order to monitor the effect of each clinicians on treatment outcomes among clients (large sample size so that the subgroups will have a large enough number of subjects to be relevant in the data analysis). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency PErcentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 281

Effectiveness 35- In future experimental studies individual psychology will need to include a process where clinicians will be trained with a manualized version of Adlerian strategies (once developed). Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 6 43%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5.5 1

Effectiveness 36 - Individual Psychology must select one intervention/technique and develop a mode for treating specific types of problems and then conduct research regarding the effectiveness of the interventions in multiple settings Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 6.5 2

Effectiveness 37 - Individual Psychology must continue to support and explore research efforts to establish Richard Watts’ “Reflecting As If” technique as an approved EBP. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 7 50%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 1 5.5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 282

Effectiveness 38 - Individual Psychology must understand that there are various methods to demonstrate effectiveness such as Seligman’s consumer report study: Here's an abstract of Seligman's summary of the research: Consumer Reports (1995, November) published an article which concluded that patients benefited very substantially from psychotherapy, that long-term treatment did considerably better than short-term treatment, and that psychotherapy alone did not differ in effectiveness from medication plus psychotherapy. Furthermore, no specific modality of psychotherapy did better than any other for any disorder; psychologists, psychiatrists, and social workers did not differ in their effectiveness as treaters; and all did better than marriage counselors and long-term family doctoring. Patients whose length of therapy or choice of therapist was limited by insurance or managed care did worse. The methodological virtues and drawbacks of this large-scale survey are examined and contrasted with the more traditional efficacy study, in which patients are randomized into a manualized, fixed duration treatment or into control groups. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 5 3

Effectiveness 39 - In order to establish effectiveness individual psychology must: develop treatment protocols and manuals for single interventions; Disseminate these to everyone (Adlerian or not) freely; Establish research training for Adlerians; Pair researchers with practitioners; Conduct outcome research on effectiveness; Publish the findings; and Repeat the process. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 1 5 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 283

Effectiveness 40 - Adlerians already respect the influence patient variables (age, gender, sexual orientation, e.g.) have on treatment outcomes, but need to develop means to quantify this influence that maintains a respect for each person’s holistic way of being and uniqueness. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 5 36%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 5 2

Effectiveness 41 - Individual Psychology must teach specific concepts of qualitative research through the monthly TAP Talks (Teaching Adlerian Psychology) that are sponsored by the Theory, Research, and Teaching section of the North American Society for Adlerian Psychology. Instructions that would emphasize supporting the clinical effectiveness of individual psychology and financial feasibility of the application of Adlerian techniques would be an important aspect of the concepts taught. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 3.5 3

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 284

Effectiveness 42 - Individual Psychology must utilize Miller’s session Rating Scale to demonstrate that individual psychology approach is satisfying and that clients are willing to participate in counseling sessions. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 6 2 4 1

Effectiveness 43 - Individual Psychology must develop a survey questionnaire and administer it to 20 Adlerian counselors, 20 CBT counselors, and 20 Eclectic counselors to evaluate counselor’s differential perception of their outcomes based on counselors ratings to demonstrate there is no difference between theoretical orientations related to treatment effectiveness. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5.5 1 3 2

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 285

Effectiveness 44 - In order for clinicians to participate in effectiveness studies individual psychology should require clinicians to meet certification requirements including a minimum number of training hours, meeting a minimum level of competency on a developed Adlerian fidelity measure, and submission of counseling video demonstrating the use of Adlerian techniques that would be evaluated utilizing an established Adlerian therapy scale. Personal Rank That You Assigned to This Item:

Number & Percentage of the 14 Panel Members Who Frequency Percentage Ranked the Suggestion in the Top 25 4 29%

Can Component Rating Should Component Rating Personal Median Interquartile Personal Median Interquartile Rating* Rating* Range Rating** Rating** Range 5 1 4.5 4

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 286

Appendix H

Recruitment Email Inviting Panel Members to Participate in Round Three

Dear ,

First, let me extend my sincere gratitude to you for your time, trouble, and continued support through out this process. Due to your efforts and those of the whole expert panel, we are nearing consensus and are able to begin our third and final round of the Delphi study regarding how, can, and should individual psychology demonstrate efficacy and effectiveness given the current evidence based practice evaluation standards. Hopefully over the last week you have been able to review the results from the second round and familiarize yourself with the two newly revised lists of suggestions, and are prepared to re-rate and re-rank these lists.

In this third and final round, similar to the second round I am asking that you again rate each suggestion based on the perceived feasibility and perceived benefit of each suggestion utilizing the two previously developed Likert scales. In addition, I am asking that you again select and rank order your “top-twenty-five” suggestions from each list. These directions are provided in more detail in the Qualtrics survey.

I truly appreciate how much effort you have put in to this study so far, and I am excited to review the data from the third and final round. I am asking that all panel members complete the last round within two weeks so that I may begin the final round of data analysis by Wednesday November 9th. If there are any concerns with this date, or if you have any questions regarding the third round please contact me.

We are in the home stretch, and I am so thankful for your participation, and I am encouraged to know that we are laying the groundwork for future research efforts in Individual Psychology. I look forward to this culminating round as the panel continues to work towards and finalize consensus.

With Great Appreciation,

Sterling

Click or Copy the Following Link:

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 287

Appendix I

Round Three Qualtrics Survey

Round Three Third Round Questionnaire: This third and final round of the study will further attempt to identify points of consensus around how, can, and should individual psychology address increasing environmental demands for evidence based practice. Your continued efforts and time that you put into completing this final will provide useful guidance for future directions in individual psychology research and practice. Based on the ratings and rankings that you provided in the second round, both the efficacy and effectiveness lists of suggestions have been revised, and were previously provided for you to review. If you have not done so already I would suggest looking over the revised list that was previously sent to familiarize yourself with the new list and to see how your personal ratings and rankings compare to the averages of the overall panel.

In this third and final round I ask that, for each of the two revised lists, (a) you once again rate the suggestions based on their perceived feasibility and benefit, and (b) you select and rank-order what you perceive to be the "top ten" (10) suggestions.

Specifically, I am asking that you rate all suggestions from both comprehensive lists using the two seven-point Likert scales used in the second round. The first Likert rating will ask you to rate each suggestion based on the can component of each research question in terms of the perceived feasibility of each suggestion to be implemented (seven point Likert scale ranging from definitely can not to definitely can). The Likert scale will ask you to rate each suggestion based on the should component of each research question in terms of the perceived benefit of each suggestion as a means to addressing how Individual Psychology may demonstrate efficacy/effectiveness given the evidence based practice evaluation standards (seven point Likert scale ranging from Absolutely no benefit to great deal of benefit).

After rating each list, I then ask that you select from each comprehensive list the "top ten" suggestions that you believe best address the how component (how individual psychology may demonstrate efficacy/effectiveness given the current evidence based practice evaluation standards). Finally, I ask that you rank-order the "top ten" list that you selected from most to least based on the perceived utility of each suggestion to address the how component of each research question.

Please type your name in the box below (your name will only be used to pair your responses across individual rounds of the Delphi study; and your name will remain confidential and your identity will not be divulged).

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 288

Evaluation Criteria Provided to Each Panel Member During the First

Below I have provided you with the the Assessment Criteria (Hollon, Miller, & Robinson, 2002) for both Efficacy and Effectiveness that was used while completing the first round of this study.

Efficacy Evaluation Criteria Efficacy refers to whether proposed beneficial effects of an intervention can be demonstrated scientifically (e.g., through treatment manualization and standardization, random assignment; random control trials, empirical outcome data, internal validity, etc.). In order to demonstrate efficacy as an Evidence Based Treatment, clinical models must satisfy the five established Efficacy Assessment Criteria (Hollon, Miller, & Robinson, 2002):

1. The (EBP evaluation guidelines require a clinical model to be ged in and based on careful consideration of a broad base of relevant empirical literature. 2. EBP evaluation guidelines require the research methodology supporting a clinical model to demonstrate the highest level of rigor and sophistication (i.e., meta analysis and randomized controlled trials as opposed to qualitative research, clinical opinion, case studies, etc.). 3. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence that its treatment outcomes are superior to treatment outcomes of other comparable clinical models and to treatment outcomes from not engaging in treatment. 4. EBP evaluation guidelines require a clinical model to be able to provide quantitative evidence to support its selection for use with specific patients. 5. EBP evaluation guidelines require the intended treatment outcomes of a clinical model to be specified, and the actual treatment outcomes to be quantitatively evaluated in relation to (or “against the influence of”) variables within the specific treatment context. (i.e., treatment goals; measures of life functioning; attrition; long-term/indirect consequences of treatment; negative consequences; client satisfaction; clinical significance; and methods).

Effectiveness Evaluation Criteria Effectiveness refers to whether an intervention is generalizable and feasible (practically and financially) for implementation with various populations, settings, and clinicians. In order to demonstrate effectiveness as an Evidence Based Treatment, clinical models must satisfy the five Effectiveness Assessment Criteria that have been established for Evidence Based Treatments (Hollon, Miller, & Robinson, 2002):

1. EBP evaluation guidelines require a clinical model to be able to quantify the influence that patient variables may have on its treatment outcomes (i.e., complexity of clinical presentation; culture; gender/sex; age/developmental level; etc.). 2. EBP evaluation guidelines require a clinical model to be able to quantify the effect that different clinician’s will have on its treatment outcomes (i.e.; clinical skill; experience; culture/ethnic backg; gender; etc.). 3. EBP evaluation guidelines require a clinical model to be able to quantify the influence that the treatment setting may have on its treatment outcomes (i.e.; home; school; day treatment; clinic; etc.). 4. EBP evaluation guidelines require a clinical model to be able to quantify the influence of INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 289

alterations in its administration on treatment outcomes (i.e.; deviation from protocol; time frame; delivery method; etc.). 5. EBP evaluation guidelines require a clinical model to be able to quantify its feasibility (i.e., clients choice/willingness/and ability to participate in the intervention) and its benefit relative to cost (i.e.; financial cost to client/clinician; prevention of future disorders; medical costs; etc.) for those providing and receiving treatment.

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 290

Round Three Efficacy Can & Should Component (Likert Scale Rating)

Please rate each suggestion utilizing the two Likert scales provided below:

For the Can Component, please base your rating on your personal belief regarding the perceived feasibility of each suggestion to be implemented by Individual Psychology, as a means of addressing how Individual Psychology may demonstrate efficacy given the current evidence based practice evaluation standards. The can component is rated on a seven point Likert scale ranging from Definitely Cannot to Definitely Can.

For the Should Component, please base your rating on your personal belief regarding the perceived benefit of each suggestion, as a means of addressing how individual psychology may demonstrate efficacy given the current evidence based practice evaluation standards. The should component is rated on a seven point Likert scale ranging from: Absolutely No Benefit to A Great Deal of Benefit.

The two Likert scales are presented side by side next to each suggestion. I ask that you rate each suggestion on both Likert scales.

[Panel Members were presented with all 43 efficacy suggestions, and asked to rate each suggestion using the two provided seven–point Likert rating scale]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 291

Round Three Efficacy How Component Selection of "Top Ten"

I ask that you select from this comprehensive list of efficacy suggestions the "top ten" suggestions that you believe best address the how component (how individual psychology may demonstrate efficacy given the current evidence based practice evaluation standards).

To select the “top ten” suggestions from the comprehensive list please click directly on the suggestion you are wishing to select (it will become highlighted).

Note: The order in which you select each suggestion will not be analyzed. ***I suggested that you use a pen and paper to keep track of the number of suggestions that you have selected as there is no digital counter incorporated into the survey software***

[Panel Members were presented with all 43 effectiveness suggestions, and asked to select 10 suggestions to be rank ordered in the next section]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 292

Round Three Efficacy How Component Rank-Ordering of “Top Ten”

Presented below is the list of the “top ten” suggestions that you personally selected form the comprehensive list of all suggestions related to efficacy. Please rank-order this list from most to least based on your personal belief regarding the perceived utility of each suggestion to address how individual psychology may demonstrate efficacy based on the current evidence based practice evaluation standards.

To rank-order each suggestion, please use the text box to the left of each suggestion and indicate the rank-order position you wish to assign to each suggestion. Please use numerical rankings (1,2,3...8,9,10), with a ranking of 1 representing the suggestion that you believe has the most perceived utility, and 10 representing the suggestion you believe has the least perceived utility. I ask that you rank-order every suggestion in your “top ten.”

[The 10 efficacy suggestions selected for the “top ten” in the previous section were presented to each panel for them to rank order]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 293

Round Three Effectiveness Can & Should Component (Likert Scale Rating)

Please rate each suggestion utilizing the two Likert scales provided below:

For the Can Component, please base your rating on your personal belief regarding the perceived feasibility of each suggestion to be implemented by Individual Psychology, as a means of addressing how Individual Psychology may demonstrate effectiveness given the current evidence based practice evaluation standards. The can component is rated on a seven point Likert scale ranging from Definitely Cannot to Definitely Can.

For the Should Component, please base your rating on your personal belief regarding the perceived benefit of each suggestion, as a means of addressing how individual psychology may demonstrate effectiveness given the current evidence based practice evaluation standards. The should component is rated on a seven point Likert scale ranging from: Absolutely No Benefit to A Great Deal of Benefit.

The two Likert scales are presented side by side next to each suggestion. I ask that you rate each suggestion on both Likert scales.

[Panel Members were presented with all 44 effectiveness suggestions, and asked to rate each suggestion using the two provided seven–point Likert rating scale]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 294

Round Three Effectiveness How Component Selection of "Top Ten"

I ask that you select from this comprehensive list of effectiveness suggestions the "top ten" suggestions that you believe best address the how component (how individual psychology may demonstrate effectiveness given the current evidence based practice evaluation standards).

To select the “top ten” suggestions please click directly on the suggestion you are wishing to select (it will become highlighted).

Note: The order in which you select each suggestion will not be analyzed. ***I suggested that you use a pen and paper to keep track of the number of suggestions that you have selected as there is no digital counter incorporated into the survey software***

[Panel Members were presented with all 44 effectiveness suggestions, and asked to select 10 suggestions to be rank ordered in the next section]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 295

Round Three Effectiveness How Component Rank-Ordering of “Top Ten” Presented below is the list of the “top ten” suggestions that you personally selected form the comprehensive list of all suggestions related to effectiveness. Please rank-order this list from most to least based on your personal belief regarding the perceived utility of each suggestion to address how individual psychology may demonstrate effectiveness based on the current evidence based practice evaluation standards.

To rank-order each suggestion, please use the text box to the left of each suggestion and indicate the rank-order position you wish to assign to each suggestion. Please use numerical rankings (1,2,3...8,9,10), with a ranking of 1 representing the suggestion that you believe has the most perceived utility, and 10 representing the suggestion you believe has the least perceived utility. Please rank-order every suggestion in your “top ten.”

[The 10 suggestions selected for the “top ten” by each panel member were presented to each panel for them to rank order]

INDIVIDUAL PSYCHOLOGY’S EFFICACY AND EFFEECTIVENESS 296

Vitae Sterling P. Travis Birthplace: Statesville, NC March 28th, 1989

EDUCATION 2017 Ph.D. Counselor Education & Supervision, The College of William & Mary Cognate: College Student Development Dissertation: A Delphi Study Regarding How, Can, And Should Individual Psychology Demonstrate Efficacy and Effectiveness Given Evidence Based Practice Evaluation Standards 2013 M.Ed. Community Counseling, The University of Mississippi Masters Research: Development and analysis of an Adlerian eating disorder assessment Awarded: Outstanding Masters Student in Research Counselor Education 2011 B.S. Psychology, James Madison University

CURRENT PROFESSIONAL POSITIONS Clinical Director, Veritas Collaborative Richmond Virginia

PUBLICATIONS Greene, C. A., Williams, A. E., Harris, P. N., Travis, S. P., & Kim, S. Y. (2016). Unfolding Case-Based Practicum Curriculum Infusing Crisis, Trauma, and Disaster Preparation. Counselor Education and Supervision, 55: 216–232.

Booth, N. R., Travis, S. P., Borzumato-Gainey, C., & Degges-White, S. (2013). Social involvement: Helping students find their place in campus life. In C. Borzumato-Gainey & S. Degges-White (Eds.). College student mental health counseling: A developmental approach, Thousand Oaks, CA: Sage.