The Dodo Bird Verdict-Controversial, Inevitable and Important: A

Clinical Psychology and Psychotherapy Clin. Psychol. Psychother. 16, 510–522 (2009) Published online 2 September 2009 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/cpp.648 Assessment The Dodo Bird Verdict—Controversial, Inevitable and Important: A Commentary on 30 Years of Meta-Analyses Rick Budd1 and Ian Hughes2* 1 Ludlow Street Healthcare, Llanbedr Court, Newport, Wales, UK 2 Psychology Department, Cardiff and Vale NHS Trust, Cardiff, Wales, UK

In this article, the assertion that different psychological therapies are of broadly similar effi cacy—often called the ‘Dodo Bird Verdict’—is contrasted with the alternative view that there are specifi c therapies that are more effective than others for particular diagnoses. We note that, despite thirty years of meta-analytic reviews tending to support the fi nding of therapy equivalence, this view is still controversial and has not been accepted by many within the psychological therapy community; we explore this from a theory of science perspective. It is further argued that the equivalence of ostensibly different therapies is an inevitable consequence of the methodology that has domi- nated this fi eld of investigation; namely, randomised controlled trials [RCTs]. The implicit assumptions of RCTs are analysed and it is argued that what we know about psychological therapy indicates that it is not appropriate to treat ‘type of therapy’ and ‘diagnosis’ as if they were independent variables in an experimental design. It is noted that one logical consequence of this is that we would not expect RCTs to be capable of isolating effects that are specifi c to ‘type of therapy’ and ‘diagnosis’. Rather, RCTs would only be expected to be capable of identifying the non-specifi c effects of covariates, such as those of therapist allegiance. It is further suggested that those non- specifi c effects that have been identifi ed via meta-analysis are not trivial fi ndings, but rather characterise important features of psychological therapy. Copyright © 2009 John Wiley & Sons, Ltd.

Keywords: Meta-Analysis; Dodo Bird Verdict; Psychotherapy, CBT THE HISTORICAL CONTEXT was reported by Smith and Glass in 1977. At that time, meta-analysis was still a new and relatively The ﬁ rst meta-analysis of studies examining the controversial methodology (e.g., Gallo, 1978) that efﬁ cacy of different types of psychological therapy was just starting to be used as a way of summarizing a rapidly expanding literature. Rather * Correspondence to: Ian Hughes, Psychology Department, than subjectively appraise the support that differ- Whitchurch Hospital, Cardiff, Wales, UK. CF14 7XB. ent studies provided for different psychological E-mail: [email protected] therapies, as had been done until that time (e.g.,

Luborsky, Singer, & Luborsky, 1975), Smith and Cucherat, & Blackburn, 1998; Svartberg and Stiles, Glass (1977) turned to meta-analysis with the hope 1991). On the contrary, Leichsenring (2001) found that this methodology would provide an objective short-term psychodynamic therapy to be of similar method for summarizing the literature. The now effi cacy to CBT for depression, and Ekers, Richards, familiar goal of their meta-analytic review was to and Gilbody (2008) have conducted a meta-analysis aggregate the sizes of the effects found across dif- that suggested that purely behavioural treatments ferent studies and thereby draw objective conclu- were at least as effective as cognitive–behavioural sions about which psychological therapies were approaches in depression. Moreover, re-analysing most effective. Gloaguen et al.’s (1998) data, Wampold, Minami, With improvements in methodology, meta- Baskin and Tierney (2002) concluded that all the analysis has ceased to be regarded as a contro- other psychological therapies that they classifi ed as versial methodology and has become the standard bona fi de were of similar effi cacy to CBT in treating tool for reviewing and summarizing the research depression. A more recent meta-analysis (Cuijpers, literature. Moreover, there have been signifi - van Straten, Andersson, & Van Oppen, 2008) came cant developments both in methodology, and in to similar overall conclusions, although hinting the specifi city of the questions being asked (e.g., at a possible slight advantage for Interpersonal Benish, Imel, & Wampold, 2008; Siev & Chamb- Therapy (IPT) over other approaches, in the treat- less, 2007). However, while the potential utility of ment of mild–moderate severity of depression. this methodology is now accepted by most, the Certainly, no clear pattern of superiority for any conclusions that can reasonably be drawn from the one treatment modality has yet emerged. literature continue to be controversial and hotly In the area of anxiety, the only meta-analysis debated (e.g., Holmes, 2002; Tarrier, 2002). that has clearly found CBT to be superior to other treatments is that reported by Siev and Chambless (2007), with these authors reporting THE DODO BIRD VERDICT AS that although Applied Relaxation (AR) was as CONTROVERSIAL effective as CBT for Generalized Anxiety Disor- der (GAD), CBT was more effective than AR for The controversy that Smith and Glass (1977) fi rst Panic Disorder. A major review by Hofmann and attempted to solve via the use of meta-analysis was Smits (2008) found CBT to be broadly effi cacious that raised by Luborsky et al.’s (1975) qualitative across the spectrum of anxiety disorders, but the (non-meta-analytic) review of the literature, from effect sizes in the Intention-to-treat (ITT) studies which they concluded that the then extant outcome included were very modest, and Ost (2008) has literature provided no clear support for the supe- further noted no clear evidence of signifi cantly riority of any one type of psychological therapy improved outcome rates over approximately four over any other. Quoting the Dodo Bird in Lewis decades of CBT in anxiety. Norton and Price Carroll’s Alice in Wonderland, they asked whether (2007), reporting a meta-analysis that summa- the conclusion to be drawn was that ‘everyone has rized 108 treatment studies across all the anxiety won and all must have prizes’. However, to some disorders, showed that behavioural therapy (BT) authorities the superiority of behavioural and cog- was as effi cacious as cognitive therapy (CT) and nitive–behavioural techniques was so evident from CBT, while Benish et al.’s (2008) meta-analysis existing qualitative literature reviews that Eysenck in post-traumatic stress disorder (PTSD) also (1978) was prompted to brand meta-analysis, and showed broad equivalence of outcomes across the conclusions that Smith and Glass (1977) had modalities. come to, as an ‘exercise in mega-silliness’ (emphasis These fi ndings have led to both reviews of the added). meta-analytic literature (e.g., Butler, Chapman, While the subsequent meta-analysis of Shapiro Forman, & Beck, 2006) and to meta-analyses of and Shapiro (1982) provided further support for the meta-analyses (e.g., Grissom, 1996). While the assertion that most psychological therapies some authors have come to the conclusion that were of broadly equivalent effi cacy, this debate ‘the Dodo Bird Verdict is alive and well – mostly’ has continued to the present day. (Luborsky et al., 2002, p. 2), others suggest that In depression, some authors have reported meta- ‘the dodo bird is extinct’ (Beutler, 2002, p. 30). In analyses showing cognitive–behavioural therapy the current paper we address the question of why (CBT) to be superior to other forms of psychologi- this debate has persisted, and we ask what we can cal therapy for depression (Gloaguen, Cottraux, learn from this debate.

OBJECTIVE SCIENCE VERSUS SCIENCE studies may in fact be as high as 0.85. While this AS DISCOURSE might surprise those who view science as merely an objective accumulation of facts, it would come To understand why this debate has lasted for over as no surprise to social psychologists who long 30 years, and has not been resolved despite the ago argued that social infl uence processes make numerous meta-analyses that have been conducted such biases an inherent part of all human activity, over that time, it is helpful to refl ect on the nature scientifi c activity included (Farr, 1976). of scientifi c debate. It is often thought that science Thus, rather than view science as an objective advances by collecting data and objectively evaluat- accumulation of facts, we might better view science ing them, with agreed knowledge emerging from as a discourse that occurs in a public domain these data (Charlton, 2000). From this standpoint, it between proponents of rival theoretical perspec- is easy to understand the hopes of the early meta- tives. As Charlton (2000) notes: analyses, i.e., that aggregating effect sizes across studies would enable a decision to be made about Alternative observations, experiments and which interventions are most effective, and that this interpretations emerge from different directions. would then act as a basis to promote their wide- Disagreement and dissent (whether over funda- spread adoption and discourage the use of other, mentals or minutiae) is the normal state of affairs less effective therapies (Charlton, 2000). However, within any active subject within science. Despite the hostile responses (e.g., Eysenck, 1978) that the disagreement, some scientifi c theories are accepted early meta-analyses (e.g., Smith & Glass, 1977) gen- and built-upon, while others are . . . either ignored erated, and the unresolved debate that continues to or rejected (p. 15). this day (e.g., Holmes, 2002; Tarrier, 2002) indicate that consensus has still not emerged from the data. Therefore, to understand why a consensus has not But perhaps this continuing controversy is not as emerged out of the meta-analytic literature we fi rst surprising as it might appear at fi rst sight. As Charl- need to clarify the nature of the rival theoretical ton (2000) notes: ‘. . . science is not an undiscrimi- perspectives this debate has centred on. nating process of relentless, cumulative, impartial observation’, (p. 25), but rather ‘. . . science is in the questions asked and the means by which they THE RIVAL PERSPECTIVES AND THE are addressed’ (p. 25), with this being very clearly RESULTING SCIENTIFIC DISCOURSE demonstrated in the meta-analytic literature. Over 25 years ago, Shapiro and Shapiro (1982) noted that The theoretical perspectives underpinning this the outcome literature contained a preponderance debate can be broadly classifi ed into two rival of studies examining cognitive–behavioural inter- positions. The fi rst, which is often adopted by ventions and that ‘. . . the few instances of dynamic researchers aligned to a traditional psychothera- and humanistic therapy were interpretable as straw peutic perspective (e.g., Leichsenring, 2001; Lubor- man treatments, not expected to yield good results sky et al., 2002; Shapiro & Shapiro, 1982; Wampold, by the investigators’ (p. 597). Similarly, Wampold 2001), is that most psychological therapies are of et al., (2002) reported that, although their meta- approximately equal effi cacy. Researchers aligned analysis initially appeared to indicate that CBT was to this perspective have tended to argue that the superior to other talking therapies for depression, apparent equivalence of different psychological when ‘straw man’ interventions were removed therapies is due to important common factors that from the meta-analysis CBT was found to be no all therapies share. Hence, they argue that it is more effi cacious than other bona fi de therapies. (See such elements as creating the therapeutic alliance, also Benish et al., 2008 in relation to PTSD). remoralizing the patient, instilling hope, normal- The related fi nding that researchers’ therapeutic izing distressing experiences and giving the client allegiance has a signifi cant effect on the outcome of a meaningful explanation of her/his diffi culties, studies examining the effi cacy of different types of which account for most of the change that occurs therapy has been well documented across a range in any psychological therapy. of diagnoses (e.g., Berman, Miller, & Massman, The second, rival perspective, which is generally 1985; Robinson, Berman, & Neimeyer, 1990), with adopted by researchers aligned to CBT (e.g., Butler Luborsky et al. (1999) suggesting that the corre- et al., 2006; Tarrier, 2002), is that there are specifi c, lation between the researchers’ therapeutic alle- usually broadly cognitive–behavioural, interven- giance and the results of comparative outcome tions that are effective for particular disorders,

Copyright © 2009 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. 16, 510–522 (2009) DOI: 10.1002/cpp The Dodo Bird Verdict: A Commentary 513 with all other interventions being little more than models. Thus, for proponents of CBT, demonstrat- inert placebos (e.g., Chambless & Hollon, 1998). ing the apparent equivalence of diverse psycho- Within this perspective, non-specifi c common ele- logical therapies—including CBT—would appear ments of therapy have generally been seen as, at also to undermine the causal models of CBT. In best, necessary but not suffi cient elements in the contrast, other psychotherapeutic approaches are process of change. (In effect, this regards such less likely to be based on a specifi c model of the non-specifi c elements of therapy as little different aetiology of the condition they are intended to in kind from extra-therapeutic factors such as the treat (e.g., IPT), or they are based on complex availability of a comfortable consulting room in models that do not relate directly to standard which both therapist and client may be seated or diagnostic categories; see, for example, Strupp and the client being greeted by a friendly reception- Binder’s (1985) or Malan’s (1963) brief psychody- ist.) As Westen, Novotny, and Thompson-Brenner namic therapies. Therefore, for researchers coming (2004) have noted, this perspective has recently from a psychotherapeutic perspective, the therapy moved from being a purely theoretical perspec- equivalence fi nding—or Dodo Bird Verdict—does tive to being one that is now closely aligned with not present a signifi cant challenge to their models the development of treatment guidelines and of therapy. protocols that emphasize the use of Evidence- Perhaps more importantly from a CBT perspec- Supported Therapies (ESTs) to the exclusion of tive, however, demonstrating that all psychologi- other (untested) interventions. It has, thus, moved cal therapies are broadly equivalent in terms of to centre stage in the development of mental outcome could be taken as evidence that CBT lacks health policy in the UK and the USA. any specifi c therapeutic ingredients and in reality As numerous authors have noted (e.g., Holmes, provides little more than a structure for develop- 2002; Luborsky, McLellan, Diguer, Woody, & ing a therapeutic alliance and a treatment ratio- Seligman, 1997; Shapiro & Shapiro, 1982), CBT has nale that engenders expectations of change in the for many years tended to dominate the research lit- client. Such a proposition presents a particular erature. Ever since Eysenck’s (1952) groundbreaking challenge to those who are aligned with CBT, since literature review, behavioural, and more recently cog- the goal of over 50 years of research has not only nitive–behavioural, approaches have generally been been to develop specifi c interventions for specifi c viewed as evidence-based interventions founded on conditions, but also to refi ne and improve these scientifi c principles, while more traditional ‘psycho- interventions on the basis of developments in our therapeutic’ interventions have been viewed at best understanding of the psychological processes that as not being evidence based, and at worst as being are believed to underpin these conditions. By con- founded on unscientifi c concepts. trast, meta-analytic fi ndings of therapeutic equiva- Possibly as a result of the dominance of CBT lence do not present such a major problem for those within the research literature, most supporters aligned with the traditional psychotherapeutic of other treatment approaches have been content perspective, since these authors are typically con- to demonstrate that these approaches are as effec- cerned with facilitating individual change through tive as CBT (e.g., Leichsenring, 2001; Leichsenring, promoting insight, clarifying emotions and resolv- Rabung, & Leibing, 2004). Researchers aligned ing interpersonal problems within the context of a to CBT have been keen to demonstrate that it is healing therapeutic relationship, rather than with more effective than traditional psychotherapeutic developing specifi c interventions to treat specifi c approaches, and that it is as effective as pharmaco- conditions (see, for example, Lambert, Bergin, & therapy (e.g., Rachman & Wilson, 2008). Although Garfi eld, 2004). serious doubts have recently been raised about the effectiveness of medication in mental health care (Kirsch et al., 2008), given the dominance of THE HIDDEN ASSUMPTIONS OF RCTs the medical model the fact that pharmacotherapy continues to be the benchmark against which CBT Randomized Controlled Trials (RCTs) have long compares itself is perhaps not surprising. been considered to provide the ‘gold standard’ The desire of CBT proponents to demonstrate that for testing the effi cacy of new drugs, such that cognitive–behavioural approaches are superior to for psychological therapies to challenge the domi- other psychological interventions is intensifi ed by nance of pharmacotherapy, it was necessary for the fact that the therapeutic interventions in this researchers to use RCTs in order to compare the tradition have been explicitly derived from causal effi cacy of these two treatment modalities (Westen

Copyright © 2009 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. 16, 510–522 (2009) DOI: 10.1002/cpp 514 R. Budd and I. Hughes et al., 2004). Meta-analysis has risen to the chal- THE PROBLEM OF TREATING lenge of summarizing this literature, having ‘DIAGNOSIS’ AS AN INDEPENDENT convincingly demonstrated that pharmacotherapy VARIABLE and psychological therapy are of broadly equivalent effi cacy (e.g., Gloaguen et al., 1998; Gould, One of the principal criteria used for including Buckminster, Pollack, Otto, & Yap, 1997; Gould, participants in RCTs is that all participants have Otto, & Pollack, 1995). While this fi nding has been the same diagnosis, with any who present with co- widely applauded by researchers aligned to CBT morbid axis I or axis II diagnoses being excluded (Rachman & Wilson, 2008), as Beutler (2000) has from the study in order ostensibly to maximize the noted, it has been more controversial in psychia- homogeneity of the group on which the treatment try. The success of this enterprise has resulted in is being tested. Westen et al. (2004) have cogently RCTs also becoming the dominant methodology argued that this selection criterion severely limits for exploring a second question, namely, what are the generalizability of RCTs to clinical populations, the active ingredients of the psychological thera- since large percentages of patients in routine clini- pies? However, while psychological therapies, as cal practice typically do have co-morbid axis I and/ a whole domain, have stood up to the ‘gold stan- or axis II diagnoses (Morrison, Bradley, & Westen, dard’ test of RCTs (or at least have proved to be 2003; Thompson-Brenner & Westen, 2005a). Nev- as effective as medication), this methodology has ertheless, restrictive selection criterion constitute a not proved capable of mining these therapies for core feature of most RCTs as their principal aim is further gold. That is to say, it has not been possible to explore what intervention is most effective for to use RCTs to determine which of the various which tightly defi ned disorder; or ‘what works for psychological therapies is most effi cacious, or whom’ as some have termed this question (Roth to determine which particular elements of these & Fonagy, 1996). To this end one goal of recent therapies account for their effi cacy. meta-analyses has been to examine the interaction Westen et al. (2004) have argued that RCT meth- between the independent variables ‘diagnosis’ and odology contains a number of assumptions that ‘type of treatment’ (Norton & Price, 2007; Siev & are violated when this methodology is applied Chambless, 2007). to psychological interventions. This results in Even with these restrictive exclusion criteria the fi ndings generated by these RCTs not being removing obvious co-morbidity, however, the generalizable to routine clinical practice. In the resulting ‘homogeneity’ of the now clinically unrep- current paper it is argued that three of the false resentative participants in the RCTs is likely to be assumptions identifi ed by Westen et al. (2004) in more apparent than real. As Westen et al. (2004) fact account for the Dodo Bird Verdict. These three noted, the assumption that even a tightly-defi ned false assumptions are as follows: ‘most patients diagnosis can be used to classify participants into have one primary problem or can be treated as if meaningful groups, and thereby address the ques- they do’ (p. 634); ‘psychological symptoms can be tion of whether treatment X or Y works best for understood and treated in isolation from person- type of patient Z, is itself likely to be seriously ality dispositions’ (p. 636); and ‘. . . the elements mistaken. As these authors and many others (e.g., of effi cacious treatment are dissociable and hence Craske & Waters, 2005; Watson, O’Hara, & Stuart, subject to dismantling’ (p. 640). These assump- 2008) have argued, there is in fact little reason to tions will be described briefl y below. We then believe that the Diagnostic and Statistical Manual consider how they may account for the Dodo Bird of Mental Disorders (DSM) diagnostic categories Verdict. are valid, psychologically meaningful constructs The present paper also goes beyond the argu- as they have neither been logically derived from ments of Westen et al. (2004) to suggest that the any psychological model, nor derived empirically core problem lies in the level of analysis that (e.g., via latent class analysis, factor analysis, etc.). is used in psychological treatment research. We Rather, the diagnostic categories, and the criteria argue that the active ingredients of different psy- that need to be met for a patient to be given a par- chological treatments are located at the level of ticular diagnosis, have been agreed by committee individual therapist–client interactions, which, with little consideration having been given to issues most importantly, are reciprocal in nature. This of construct validity. As many diagnoses consist is in contrast to locating the active ingredients of groups of overlapping symptoms which have at the level of different psychological treatment high incidences of co-morbidity (Brown, Camp- packages. bell, Lehman, Grisham, & Mancill, 2001; Mineka,

Watson, & Clark, 1998; Zimmerman, McGlinchey, of perfectionism and the quality of the therapeu- Chelminski, & Young, 2008), they would not be tic alliance in predicting outcome (Blatt, Zuroff, expected, on a priori grounds, to be conceptu- Quinlan, & Pilkonis, 1996). Thus, if personality ally distinct entities. More importantly, however, factors that are likely to affect the outcome of an when the construct validity of these diagnostic intervention are not assessed, and their effects are categories has been explored empirically they have not partialled out, this factor is likely to confound been found not to be supported by the data (e.g., the results of the RCT. Krueger, 1999; Zimmerman & Chelminski, 2003). Thus, if patients with different diagnoses present with overlapping symptoms, and the symptoms of THE PROBLEM OF TREATING ‘TYPE those who have the same diagnosis have different OF THERAPY’ AS AN INDEPENDENT (unknown) aetiologies, it would not be expected VARIABLE that there should be specifi c psychological interventions that are most effective for particular In experimental designs it is axiomatic that the diagnoses. independent variables are clearly defi ned, and can Most signifi cantly in this regard, structural equa- be adequately operationalized, so that causal effects tion modelling and other related techniques have on the dependent variable(s) can be examined. As suggested that anxiety and depression are best Westen et al. (2004) note, ‘type of psychological described by a triarchic structure (Olino, Klein, therapy’ is almost unique among independent Lewinsohn, Rohde, & Seeley, 2008; Watson et al., variables in that the experimental manipulation (a 2008). This suggests that although the diagnoses of type of therapy) is an intervention provided by Major Depressive Disorder (MDD) and GAD each different therapists over a long time period, with have symptoms that are unique to them, they also the experimental manipulation often being pro- share common features, which include personal- vided over as many as 20 sessions. Maintaining the ity traits that underlie these presenting problems purity of such a complex experimental manipula- (Ormel & Wohlfarth, 1991; Ormel, Oldehinkel, & tion therefore presents a signifi cant methodologi- Vollebergh, 2004) and which may account for the cal challenge, even when the different therapies large proportion of patients who have alternating are set out in detailed treatment manuals and their episodes of GAD and MDD over time (Kessler integrity is monitored via the analysis of audio/ et al., 2008). video recordings of sessions. A further problem with treating diagnosis as In this regard it is instructive to consider the an independent variable is the assumption that NIMH Treatment of Depression Collaborative treatment can be provided to patients in different Research Program (Elkin et al., 1989), as this is often diagnostic groups independently of the patients’ considered to be one of the most methodologically personalities. While participants are typically sound comparative effi cacy outcome studies that excluded from RCTs if they have clear axis II per- have been conducted to date. For example, using sonality disorders, no attention is paid to partici- process analysis to examine the two psychologi- pants’ personality characteristics if these are within cal therapies (CBT and IPT) that were examined the ‘normal’ range. But as Westen and Thompson- in the NIMH study, Ablon and Jones (1999, 2002) Brenner (2004) note, the assumption that patients concluded that these two therapies shared many can be treated independently of their personal- common components and that the IPT interven- ity is not supported either by clinical experience tion may have had more in common with a CBT (Morrison et al., 2003; Thompson-Brenner & Westen, ideal therapy prototype than it had with an IPT 2005b) or by the research data, which suggest that ideal therapy prototype (at least across two of the certain personality traits function as underlying risk study sites). Reporting on the Sheffi eld Psycho- factors for experiencing episodes of anxiety and/or therapy Studies, Stiles and his colleagues (Stiles & depression (Brown, 2007; Ormel et al., 2004). Shapiro, 1994; Stiles, Shapiro, & Firth-Cozens, 1988) Interestingly, in the National Institute of Mental noted that although the cognitive–behavioural and Health (NIMH) Treatment of Depression Col- interpersonal–psychodynamic therapies used in laborative Research Program, patients’ levels of these studies were clearly distinguishable in terms perfectionism were found to moderate the effec- of therapist response mode, there was nonetheless tiveness of the treatments and, while this personal- considerable overlap in terms of therapist behav- ity trait did not interact with the type of treatment iour across both treatment types. While this could provided, there was an interaction between level be viewed simply as a failure to ensure treatment

fi delity, it is interesting to note that in routine clini- client in various doses (number of sessions), with cal practice a similar fi nding has been noted. That different drugs (different types of psychological is to say, while the behaviour of therapists aligned therapy) being indicated for different conditions. to either CBT or psychotherapeutic perspectives is This metaphor underpins the use of experimental distinguishable, there nonetheless remains signifi - designs that treat type of therapy and diagnosis cant overlap in the responses provided by thera- as if they were independent variables, with the pists from these different orientations (e.g., Ablon aim of discovering ESTs that are diagnosis spe- & Jones, 1998; Goldfried, Raue, & Castonguay, cifi c. The dominance of the drug metaphor in 1998; Trijsburg, Trent, & Perry, 2004). therapy outcome research refl ects the hegemony Most critically, however, the central problem of the medical model and has prompted the wide- in maintaining the purity of different therapeu- spread acceptance of RCTs as the ‘gold standard’ tic interventions rests on a feature that is central for testing the effi cacy of psychological therapies. to, and inherent in all, psychological therapies. While the dominance of this approach to outcome That is, all psychological therapies, whatever their research has been successful to the extent that it theoretical orientation, consist of a social interac- has resulted in the effi cacy of psychological thera- tion (usually dyadic) between therapist and client pies becoming widely accepted by governments where both infl uence each other’s behaviour (Stiles, and other health care funding organizations (see Honos-Webb, & Michael, 1998). Thus the goal of for example Rachman & Wilson’s, 2008 favourable maintaining the ‘purity’ of a psychological therapy discussion of UK government mental health initia- (experimental manipulation) across clients, even tives), it has at the same time resulted in the Dodo when supported by a treatment manual and valid- Bird Verdict. The reason for this lies in the fact that ity checks, is by its very nature fl awed. As Hardy, the central assumptions of RCTs (and, hence, the Stiles, Barkham, and Startup (1998) have shown in drug metaphor) are violated in the case of psycho- the Sheffi eld Psychotherapy Studies, clients infl u- logical therapies. ence the behaviour of therapists in subtle ways As has been argued above, diagnostic catego- despite rigorous attempts to manualize interven- ries do not consist of groups of clients who have tions, with this being an inherent feature of all distinct problems, with the same psychological human interactions (Farr, 1976), not just of psy- processes being responsible for the cause and/or chological therapy. maintenance of ‘the problem’ that characterizes Paradoxically, this phenomenon is as explicitly each diagnosis. Rather, they consist of people pre- acknowledged in CBT as it is in the interpersonal– senting with fairly heterogeneous and overlapping psychodynamic therapies, with CBT defi ning itself problems (symptoms) with different (generally as a collaborative enterprise that occurs between unknown) psychological processes causing and/ client and therapist (Beck, 1976). So it is acknowl- or maintaining these problems. As such, even if it edged that the therapist must respond to the were possible to ensure that different psychologi- client in a fl exible, collaborative manner in order cal therapies were ‘pure’ treatments that contained to develop a therapeutic alliance, which is known unique components that did not overlap in any to be central to the effectiveness of all psychologi- way, we would not expect the same intervention cal interventions (Horvath & Bedi, 2002; Martin, to be effective for every client within a diagnostic Garske, & Davis, 2000), including CBT (Wadding- group. Rather we would expect the same inter- ton, 2002). However, despite this fundamental vention to be differentially effective for different feature of therapy (and all other human interac- clients with the same diagnosis, this depending tions) being acknowledged in clinical practice, it is upon the psychological and social factors that typically sidestepped by RCTs since it presents a are maintaining that client’s presenting problems serious challenge to their experimental validity. (symptoms). One extension of this analysis is to offer a broadly based, non-specifi c therapeutic THE MISLEADING DRUG METAPHOR package to a diagnostically mixed patient group, AS A CAUSE OF THE DODO with the expectation that on average any given BIRD VERDICT patient will be able to benefi t from enough of the package to enable them to make worthwhile prog- Stiles and Shapiro (1989) have noted that a common ress. This has been done, for instance, in a recent and often implicit assumption underlying much study by McEvoy and Nathan (2007), with appar- outcome research is that psychological therapy ently promising results: 10 2-hour CBT groups for can be likened to a drug that can be given to a a diagnostically heterogeneous sample of patients

Copyright © 2009 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. 16, 510–522 (2009) DOI: 10.1002/cpp The Dodo Bird Verdict: A Commentary 517 produced effect sizes at least as good as those that therapy) on which RCTs are based are not capable would have been expected from diagnosis-specifi c of directly and uniquely manipulating the psy- interventions. Although these particular authors chological processes that account for the symptom were using a CBT approach, the overall implica- change that occurs in therapy. This point is freely tion of their study, that non-specifi c interventions acknowledged by Hofmann (2008) in response to are as useful as are more specifi c interventions, Longmore and Worrell’s (2007) criticism of cogni- seems broadly compatible with the wider psycho- tive therapy on the grounds that component studies logical therapy equivalence fi ndings cited above. A do not indicate the superiority of interventions that logical extension of this approach is the Internet- directly target cognitions over purely behavioural based CBT packages now being developed, which interventions that do not do so. In this regard, are also not diagnosis specifi c. Hofmann (2008) accepts that cognitions can change Moreover, as has been argued above, different in a therapeutically helpful manner without the psychological therapies are not well-controlled need to explicitly target them in treatment. Hence, experimental manipulations, but rather consist of for example, a client’s catastrophic cognitions a range of overlapping interventions, which, while about the consequences of a panic attack might broadly distinguishable from each other, share change simply in response to them being invali- many common components. They are provided by dated by exposure without the need for them to different therapists, some of whom are more effec- have been directly challenged via setting up the tive than others, with the client’s understanding exposure sessions as behavioural experiments. Or, and interpretation of the therapist’s intervention, as a more extreme example, the whole atmosphere and its therapeutic impact, depending upon both of the clinic as a friendly, accepting and ‘normal’ the context in which it was provided and upon environment in which the client is valued might idiosyncratic features of the client. Worst of all, begin to challenge patients’ negative self-concept from a purely experimental perspective, the nature even before they get to see their therapist! of the therapy (experimental manipulation) that Ablon and Jones (1998) have similarly argued each client (experimental participant) receives is that ostensibly different types of verbal response infl uenced in subtle and uncontrollable ways by from the therapist (e.g., a traditional psychoana- the client (experimental participant). lyst offering an interpretation, a CBT practitioner presenting a CBT formulation etc.) can have essen- tially the same therapeutic function, depending on THE DODO BIRD VERDICT how the therapist’s intervention is interpreted AS INEVITABLE and understood. This is consistent with the long known phenomenon that clients’ and therapists’ Since the independent variables (diagnosis and reports of what were the signifi cant events that type of therapy) that are submitted to meta- occurred in therapy can often diverge quite mark- analysis can neither be adequately controlled nor edly (e.g., Llewelyn, 1988): it is not necessarily operationalized, we would not expect any meta- what the therapist thought s/he was doing in analysis of RCTs to be capable of successfully par- therapy that was the active ingredient, but rather titioning any possible treatment-specifi c variance how the client understood/interpreted the thera- in outcome to these two independent variables. pist’s intervention. It is even more unlikely to identify any interaction effect. That is to say, we would not expect this methodology to be capable of isolating which WHAT HAVE WE LEARNED? THE type of therapy is most effective for which diag- DODO BIRD VERDICT AS IMPORTANT nosis. Thus, the repeated failure to fi nd any clear evidence that any one type of therapy is superior Given that core assumptions underlying RCTs to any other, for any given diagnosis (e.g., Benish are violated in the case of psychological therapies et al., 2008), is a direct consequence of the fact that (Westen et al., 2004) so as to make it impossible key assumptions underpinning RCTs, namely that to partition any possible treatment specifi c vari- the experimental manipulations are clearly defi n- ance in outcome to either type of therapy and/or able and operationalizable, do not apply to psy- the interaction between type of therapy and diag- chological therapy. nosis, it is to be expected that most of what we A fi nal reason for the Dodo Bird Verdict is that have learned from over 30 years of meta-analytic the experimental manipulations (different types of reviews of RCTs concerns those effects that are

Copyright © 2009 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. 16, 510–522 (2009) DOI: 10.1002/cpp 518 R. Budd and I. Hughes common across different interventions and diag- Second, the literature clearly demonstrates that noses. Although this could be seen as a ‘failure’, the therapeutic alliance is central to the effective- it is worth noting that those effects that have been ness of all types of psychological therapy (Horvath isolated shed considerable light on the process of & Bedi, 2002; Martin et al., 2000). This implies therapy, even though these fi ndings are not spe- that one important goal for further research is to cifi c to any individual therapeutic intervention or explore what factors promote the therapeutic alli- diagnosis. In summary, meta-analytic reviews of ance and, in particular, to focus on ways of pro- the outcome literature have demonstrated three moting a strong alliance in those clients whose effects that are of both theoretical and practical personality may hinder this process (Blatt et al., importance. 1996). Moreover, at a clinical level it suggests that it First, the literature clearly demonstrates that the may be of value for therapists routinely to monitor therapist’s allegiance to a particular intervention is clients’ progress in therapy (e.g., Lambert, 2007), highly predictive of the effi cacy of that interven- and the quality of the therapeutic alliance, in order tion, with Luborsky et al. (1999) and Wampold to enable them promptly to address events that (2001) presenting data which indicate that this may disrupt therapy. effect is substantially larger than any other effect Third, the literature clearly demonstrates that (either specifi c or general) that has so far been iso- some therapists are more effective than others lated from RCTs. Rather than regard this merely as (Crits-Christoph et al., 1991; Luborsky et al., 1997), a failure of ‘scientifi c objectivity’, it is possible to with this effect surprisingly having been demon- argue that it in fact highlights an important basic strated both for clinicians providing psychological therapeutic processes. In this regard, Wampold therapies (Kim, Wampold, & Bolt, 2006) and for and his colleagues (Benish et al., 2008; Wampold those providing pharmacotherapy (McKay, Imel, et al., 1997; Wampold et al., 2002) have shown, & Wampold, 2006). At a research level this sug- across a range of diagnoses, that therapies with gests, as Westen et al. (2004) propose, that sys- a clear treatment rationale, and which therapists tematically exploring expert clinicians’ routine themselves believe to be effective, are more effec- practice might provide useful hypotheses about tive than those that are intended to be placebo the nature of successful therapy which can then interventions. Moreover, they have demonstrated be tested using more traditional research designs. that this result holds, regardless of the rationale With regard to clinical practice, this fi nding sug- underpinning the interventions that are intended gests that there may be value in trying to construct to be therapeutic. more systematic methods for coaching and men- At a research level, Berman et al. (1985) showed toring therapists than is the case with many current over 20 years ago that this effect has the potential clinical supervision arrangements. to bias meta-analytic reviews of the literature and, as a result, researcher allegiance needs to be partialled out in meta-analyses in order to control for CONCLUSIONS AND IMPLICATIONS the effect this may have on the differential outcome FOR FUTURE RESEARCH of different therapies. At a clinical level this fi nding implies that an EST is likely to be less effective if it As has been argued above, the failure of RCTs is administered by a therapist who is not convinced (and the meta-analyses of RCTs) to partition out of the value of that therapy than if it is adminis- any specifi c effects of psychological therapy to dif- tered by one who is. This runs counter to the drug ferent types of interventions (even when diagnosis metaphor of psychological therapy, which sug- has been controlled for) is attributable to the fact gests that therapeutic effectiveness rests mainly on that it is exceptionally diffi cult to operationalize ensuring that therapists only use ESTs and closely and control this independent variable. That is to adhere to the related treatment manuals. Rather, say that despite rigorous attempts at manualiza- it may be more appropriate for therapists to use tion the evidence suggests that it is not possible to evidence-based techniques with their clients, in the ensure that different therapies do not share such a context of an individual case formulation (which the large proportion of active therapeutic ingredients therapist of necessity has allegiance to), than for as to dilute the power of RCTs to isolate treatment- therapists slavishly to follow treatment manuals specifi c effects. Moreover, and perhaps even more that, as Westen et al. (2004) have cogently argued, signifi cantly, it is impossible to ensure that different may not be appropriate for many of the clients who interventions do not affect the same psychological are treated in routine clinical practice. mechanisms of change, thus similarly limiting the

Copyright © 2009 John Wiley & Sons, Ltd. Clin. Psychol. Psychother. 16, 510–522 (2009) DOI: 10.1002/cpp The Dodo Bird Verdict: A Commentary 519 power of RCTs to isolate treatment specifi c effects. broader environmental context); and, that it also When these limitations are combined with the fact ignores the therapeutic relationship. Low mood, that diagnoses are overlapping, ‘fuzzy’ concepts, for instance, is a very common feature of ‘depres- which do not adequately control for psychological sion’. In order to ameliorate it, we would advocate factors (e.g., sub-clinical personality features, etc.) a detailed analysis of its maintaining factors on that are likely to interact with the type of interven- an individual basis, and then the application of tion in determining its effi cacy, it is not surprising an appropriate technique (e.g., increasing activity that the Dodo Bird Verdict has survived for more levels or resolving relationship confl icts or explor- than 30 years. In short, its survival rests on the fact ing the validity of negative automatic thoughts). that while RCTs (and the meta-analysis of data But, importantly, this needs to be done within obtained from RCTs) are capable of demonstrating the context of the therapeutic process, that is to that psychological therapy as an aggregate is effec- say, at an appropriate time, and in a way that is tive, they are not capable of identifying the specifi c most likely to promote change within the client. elements of therapy that are effective for different (A similar general approach is much more fully individuals. described in the important chapter by Norcross From the research perspective the clarion call and Beutler [2008]). is clear. RCTs do not represent an objective ‘gold Most importantly we need to reject the medical standard’ of research, but rather, are simply one model, cease to view therapy as being like a drug of many research methodologies, which, like all that is given to clients and, instead, view it for what methodologies, have their strengths and weak- it is, a social infl uence process that occurs (most nesses. When used to test causal relationships typically) between two people. between a dependent variable and a set of independent variables that can be adequately defi ned and operationalized, there is no alternative to con- structing an RCT. However, for addressing other REFERENCES research questions, such as ‘how does psychologi- Ablon, J.S., & Jones, E.E. (1998). How expert clinicians’ cal therapy help different people change?’ differ- prototype of an ideal correlate with outcome in psy- ent methodologies could be more appropriate (e.g., chodynamic and cognitive-behavioural therapy. Psy- Pachankis & Goldfried, 2007). chotherapy Research, 8, 71–83. Moreover, there is a need to move away from Ablon, J.S., & Jones, E.E. (1999). Psychotherapy process designing psychological interventions targeted in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of at an invalid psychiatric nosology, to developing Consulting and Clinical Psychology, 67, 64–75. interventions directed at alleviating distressing Ablon, J.S., & Jones, E.E. (2002). Validity of controlled experiences (e.g., low mood, intrusive memories) clinical trials of psychotherapy: Findings from the or which are directed at changing problematical NIMH Treatment of Depression Collaborative Research behaviours (e.g., avoidance, rumination). These Program. American Journal of Psychiatry, 159, 775– interventions need to be developed in light of our 783. Beck, A.T. (1976). Cognitive therapy and the emotional understanding of the psychosocial processes that disorders. New York: International Universities Press. maintain them. How and when they are used in Benish, S.G., Imel, Z.E., & Wampold, B.E. (2008). The therapy needs to be informed by a full understand- relative effi cacy of bona fi de psychotherapies for treating of the individual therapy process. ing post-traumatic stress disorder: A meta-analysis To illustrate our favoured approach, let us con- of direct comparisons. Clinical Psychology Review, 28, sider ‘depression’. Typically, within the present, 746–758. diagnostically driven approach, patients diagnosed Berman, J.S., Miller, R.C., & Massman, P.J. (1985). Cog- nitive Therapy Versus Systematic Desensitization: Is as having ‘depression’ are offered a package— One Treatment Superior? Psychological Bulletin, 97, often of around 20 sessions—of CBT treatment, 451–461. which comprises a number of almost invariant ele- Beutler, L.E. (2000). David and Goliath: When empiri- ments (e.g., activity scheduling, thought diaries, cal and clinical standards of practice meet. American thought challenging). We consider that such an Psychologist, 55, 997–1007. approach ignores the variability and complexity Beutler, L.E. (2002). The dodo bird is extinct. Clinical Psychology: Science and Practice, 9, 30–34. of the symptoms of people diagnosed as having Blatt, S.J., Zuroff, D.C., Quinlan, D.M., & Pilkonis, P.A. ‘depression’, and that it also ignores the psycho- (1996). Interpersonal factors in brief treatment of social factors maintaining the symptoms (e.g., depression: further analyses of the National Institute family relationships, employment problems, the of Mental Health Treatment of Depression Collabora-

tive Research Program. Journal of Consulting and Clini- Gould, R.A., Buckminster, S., Pollack, M.H., Otto, M.W., cal Psychology, 64, 162–171. & Yap, L. (1997). Cognitive-behavioural and pharma- Brown, T.A. (2007). Temporal course and structural cological treatment or social phobia: A meta-analysis. relationships among dimensions of temperament and Clinical Psychology: Science and Practice, 4, 291–306. DSM–IV anxiety and mood disorder constructs. Journal Gould, R.A., Otto, M.W., & Pollack, M.H. (1995). A meta- of Abnormal Psychology, 116, 313–328. analysis of treatment outcomes for panic disorder. Brown, T.A., Campbell, L.A., Lehman, C.L., Grisham, J.R., Clinical Psychology Review, 15, 819–884. & Mancill, R.B. (2001). Current and lifetime comorbid- Grissom, R.J. (1996). The magic number 7 + –2: Meta- ity of the DSM–IV anxiety and mood disorders in a analysis of the probability of superior outcome in large clinical sample. Journal of Abnormal Psychology, comparisons involving therapy, placebo and control. 110, 585–599. Journal of Consulting and Clinical Psychology, 64, 973– Butler, A.C., Chapman, J.E., Forman, E.M., & Beck, A.T. 982. (2006). The empirical status of cognitive-behavioural Hardy, G.E., Stiles, W.B., Barkham, M., & Startup, M. therapy: A review of meta-analyses. Clinical Psychology (1998). Therapist responsiveness to client interpersonal Review, 26, 17–31. styles during time-limited treatments for depression. Chambless, D.L., & Hollon, S.D. (1998). Defi ning empir- Journal of Consulting and Clinical Psychology, 66, 304– ically supported therapies. Journal of Consulting and 312. Clinical Psychology, 66, 7–18. Hofmann, S.G. (2008). Common misconceptions about Charlton, B.G. (2000). The new management of scien- cognitive mediation of treatment change: A commen- tifi c knowledge: a change in direction with profound tary to Longmore and Worrell (2007). Clinical Psychol- implications. In A. Miles, J.R. Hampton, and B. ogy Review, 28, 67–70. Hurwitz (Eds), NICE, CHI and the NHS reforms: enabling Hofmann, S.G., & Smits, J.A.J. (2008). Cognitive- excellence or imposing control? (pp. 13–32). London: behavioral therapy for adult anxiety disorders: A Aesculapius Medical Press. meta-analysis of randomized placebo-controlled trials. Craske, M.G., & Waters, A.M. (2005). Panic disorder, Journal of Clinical Psychiatry, 69, 621–632. phobias and Generalized Anxiety Disorder. Annual Holmes, J. (2002). All you need is cognitive behaviour Review of Clinical Psychology, 1, 197–225. therapy? British Medical Journal, 324, 288–289. Crits-Christoph, P., Baranackie, K., Kurcias, J.S., Horvath, A.O., & Bedi, R.P. (2002). The alliance. In J.C. Beck, A., Carroll, K., & Perry, K. (1991). Meta- Norcross (Ed.), Psychotherapy relationships that work: analysis of therapist effects in psychotherapy out- Therapist contributions and responsiveness to patients (pp. come studies. Psycotherapy Research, 1, 81–91. 37–69). New York: Oxford University Press. Cuijpers, P., van Straten, A., Andersson, G., & Van Oppen, Kessler, R.C., Gruber, M., Hettema, J.M., Hwang, I., P. (2008). Psychotherapy for depression in adults: A Sampson, N., & Yonkers, K.A. (2008). Co-morbid meta-analysis of comparative outcome studies. Journal major depression and generalized anxiety disorders of Consulting and Clinical Psychology, 76, 909–922. in the National Comorbidity Survey follow-up. Psy- Ekers, D., Richards, D., & Gilbody, S. (2008). A meta- chological Medicine, 38, 365–374. analysis of randomized trials of behavioural treatment Kim, D.A., Wampold, B.E., & Bolt, D.M. (2006). Therapist of depression. Psychological Medicine, 38, 611–623. effects in psychotherapy: A random effects modelling Elkin, I., Shea, T., Watkins, J.T., Imber, S.D., Sotsky, of the NIMH TDCRP data. Psychotherapy Research, 16, S.M., & Collier, J.F. (1989). National Institute of 161–172. Mental Health Treatment of Depression Collaborative Kirsch, I., Brett, J.D., Huedo-Medina, T.B., Scoboria, A., Research Program. Archives of General Psychiatry, 46, Moore, T.J., & Johnson, B.T. (2008). Initial severity 971–982. and antidepressant benefi ts: A meta-analysis of data Eysenck, H.J. (1952). The effects of psychotherapy: an submitted to the Food and Drug Administration. PLos evaluation. Journal of Consulting Psychology, 16, 319– Msed, 5(2), e45. 324. Krueger, R.F. (1999). The structure of common mental Eysenck, H.J. (1978). An exercise in mega-silliness. disorders. Archives of General Psychiatry, 56, 921–926. American Psychologist, 33, 517. Lambert, M. (2007). Presidential address: What we have Farr, R.M. (1976). Experimentation: A social psycho- learned from a decade of research aimed at improving logical perspective. British Journal of Social and Clinical psychotherapy outcome in routine care. Psychotherapy Psychology, 15, 225–238. Research, 17, 1–14. Gallo, P.S. (1978). Meta-analysis – a mixed metaphor. Lambert, M., Bergin, A., & Garfi eld, S. (2004). Handbook American Psychologist, 33, 515–517. of psychotherapy and behaviour change (5th ed.). New Gloaguen, V., Cottraux, J., Cucherat, M., & Blackburn, York: Wiley. I.M. (1998). A meta-analysis of the effects of cogni- Leichsenring, F. (2001). Comparative effects of short- tive therapy in depressed patients. Journal of Affective term psychodynamic psychotherapy and cognitive- Disorders, 49, 59–72. behavioural therapy in depression: A meta-analytic Goldfried, M., Raue, P.J., & Castonguay, L.G. (1998). approach. Clinical Psychology Review, 21, 401–419. The therapeutic focus in signifi cant sessions of master Leichsenring, F., Rabung, S., & Leibing, E. (2004). The therapists: A comparison of cognitive–behavioural and effi cacy of short-term psychodynamic psychotherapy psychodynamic–interpersonal interventions. Journal of in specifi c psychiatric disorders: a meta-analysis. Consulting and Clinical Psychology, 66, 803–810. Archives of General Psychiatry, 61, 1208–1216.

Llewelyn, S.P. (1988). Psychological therapy as viewed Ormel, J., Oldehinkel, A.J., & Vollebergh, W. (2004). Vul- by clients and therapists. British Journal of Clinical Psy- nerability before, during, and after a major depressive chology, 27, 223–237. episode: A 3-wave population-based study. Archives of Longmore, R.J., & Worrell, M. (2007). Do we need to General Psychiatry, 61, 990996. challenge thoughts in cognitive behaviour therapy? Ost, L-G (2008). Cognitive behaviour therapy for anxiety Clinical Psychology Review, 27, 173–187. disorders: 40 years of progress. Nordic Journal of Psy- Luborsky, L., Singer, B., & Luborsky, L. (1975). Compara- chiatry, 62, 5–10. tive studies of psychotherapy: Is it true that ‘everyone Pachankis, J.E., & Goldfried, M.R. (2007). On the next has won and all must have prizes?’ Archives of General generation of process research. Clinical Psychology Psychiatry, 32, 995–1008. Review, 27, 760–768. Luborsky, L., McLellan, A.T., Diguer, L., Woody, G., & Rachman, S.J., & Wilson, G.T. (2008). Expansion in the Seligman, D.A. (1997). The psychotherapist matters: provision of psychological treatment in the United comparison of outcomes across twenty-two therapists Kingdom. Behaviour Research and Therapy, 46, 293–295. and seven patient samples. Clinical Psychology: Science Robinson, L.A., Berman, J.S., & Neimeyer, R.A. (1990). and Practice, 4, 53–65. Psychotherapy for the treatment of depression: a com- Luborsky, L., Diguer, L., Seligman, D.A., Rosenthal, R., prehensive review of controlled outcome research. Johnson, S., Halperin, G., Bishop, M., & Schweizer, E. Psychological Bulletin, 108, 30–49. (1999). The researcher’s own therapeutic allegiances: Roth, A., & Fonagy, P. (1996). What works for whom. New A ‘wild card’ in comparisons of treatment effi cacy. York: Guilford Press. Clinical Psychology: Science and Practice, 6, 49–62. Shapiro, D.A., & Shapiro, D. (1982). Meta-analysis of Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T.P., comparative therapy outcome studies: A replica- Berman, J.S., Levitt, J.T. Seligman, D.A., & Krause, E.D. tion and refi nement. Psychological Bulletin, 92, 581– (2002). The dodo bird verdict is alive and well – mostly. 604. Clinical Psychology: Science and Practice, 9, 2–12. Siev, J., & Chambless, D.L. (2007). Specifi city of treatment Malan, DH (1963). A study of brief psychotherapy. London: effects: Cognitive therapy and relaxation for general- Tavistock Press. ized anxiety and panic. Journal of Consulting and Clini- Martin, D.J., Garske, J.P., & Davis, K.M. (2000). Relation cal Psychology, 75, 513–522. of the therapeutic alliance with outcome and other Smith, M.L., & Glass, G.V. (1977). Meta-analysis of psy- variables: A meta-analytic review. Journal of Consulting chotherapy outcome studies. American Psychologist, 32, and Clinical Psychology, 68, 438–450. 752–760. McEvoy, P.M., & Nathan, P. (2007). Effectiveness of Stiles, W.B., & Shapiro, D.A. (1989). Abuse of the drug cognitive behavioural therapy for diagnostically het- metaphor in psychotherapy process-outcome research. erogeneous groups. Journal of Consulting and Clinical Clinical Psychology Review, 28, 1–16. Psychology, 75, 344–350. Stiles, W.B., & Shapiro, D.A. (1994). Disabuse of the McKay, K.M., Imel, Z.E., & Wampold, B.E. (2006). Psy- drug metaphor: Psychotherapy process-outcome cor- chiatrist effects in the psychopharmacological treat- relations. Journal of Consulting and Clinical Psychology, ment of depression. Journal of Affective Disorders, 92, 62, 942–948. 287–290. Stiles, W.B., Shapiro, D.A., & Firth-Cozens, J.A. (1988). Mineka, S., Watson, D., & Clark, L.A. (1998). Comorbid- Verbal response mode use in contrasting psychothera- ity of anxiety and unipolar mood disorders. Annual pies: A within-subjects comparison. Journal of Consult- Review of Psychology, 49, 377–412. ing and Clinical Psychology, 56, 727–733. Morrison, K.H., Bradley, R., & Westen, D. (2003). Stiles, W.B., Honos-Webb, L., & Michael, M. (1998). The external validity of controlled clinical trials of Responsiveness in psychotherapy. Clinical Psychology: psychotherapy for depression and anxiety: A natu- Science and Practice, 4, 439–458. ralistic study. Psychology and Psychotherapy: Theory, Strupp, HH, & Binder, JL (1985). Psychotherapy in a new Research and Practice, 76, 109–132. key: A guide to time limited dynamic psychotherapy. New Norcross, J.C., & Beutler, L.E. (2008) Integrative psycho- York: Basic Books. therapies. In: R.J. Corsini & D. Wedding (Eds) Current Svartberg, M., & Stiles, T.C. (1991). Comparative effects Psychotherapies (8th ed., pp. 481–511) Belmont, CA: of short-term psychodynamic psychotherapy: A meta- Brooks/Cole. analysis. Journal of Consulting and Clinical Psychology, Norton, P.J., & Price, E.C. (2007). A meta-analytic review 59, 704–714. of adult cognitive-behavioural treatment outcome Tarrier, N. (2002). Yes, cognitive behaviour therapy across the anxiety disorders. Journal of Nervous and may be all you need. British Medical Journal, 324, 291– Mental Diseases, 195, 521–531. 292. Olino, T.M., Klein, D.N., Lewinsohn, P.M., Rohde, P., & Thompson-Brenner, H., & Westen, D. (2005a). A natu- Seeley, J.R. (2008). Longitudinal associations between ralistic study of psychotherapy for bulimia nervosa, depressive and anxiety disorders: a comparison of two part 1: comorbidity and therapeutic outcome. Journal trait models. Psychological Medicine, 38, 353–363. of Nervous and Mental Diseases, 193, 573–584. Ormel, J., & Wohlfarth, T. (1991). How neuroticism, long- Thompson-Brenner, H., & Westen, D. (2005b). A natural- term diffi culties, and life situation change infl uence istic study of psychotherapy for bulimia nervosa, part psychological distress: A longitudinal model. Journal 2 therapeutic interventions in the community. Journal of Personality and Social Psychology, 60, 744–755. of Nervous and Mental Diseases, 193, 585–595.

Trijsburg, W.R., Trent, S.T., & Perry, C.J. (2004). An Watson, D., O’Hara, M.W., & Stuart, S. (2008). Hierar- empirical study of the differences in interventions chical structures of affect and psychopathology and between psychodynamic therapy and cognitive- their implications for the classifi cation of emotional behavioural therapy for recurrent major depression. disorders. Anxiety and. Anxiety and Depression, 25, Canadian Journal of Psychoanalysis, 12, 325–345. 282–288. Waddington, L. (2002). The therapy relationship in cog- Westen, D., Novotny, C., & Thompson-Brenner, H. nitive therapy: A review. Behavioural and Cognitive (2004). The empirical status of empirically supported Psychotherapy, 30, 179–191. psychotherapies: Assumptions, fi ndings, and report- Wampold, B.E. (2001). The great psychotherapy debate: ing in controlled clinical trials. Psychological Bulletin, models, methods and fi ndings. Mahwah, NJ: Erlbaum. 30, 631–663. Wampold, B.E., Mondin, G.W., Moody, M., Stich, F., Zimmerman, M., & Chelminski, I. (2003). Generalized Benson, K., & Ahn, H.N. (1997). A meta-analysis of anxiety disorder in patients with major depression: Is outcome studies comparing bona fi de psychothera- DSM-IV’s hierarchy correct? American Journal of Psy- pies: empirically ‘all must have prizes’. Psychological chiatry, 160, 504–512. Bulletin, 122, 203–215. Zimmerman, M., McGlinchey, J.B., Chelminski, I., & Wampold, B., Minami, T., Baskin, T., & Tierney, S. (2002). Young, D. (2008). Diagnostic co-morbidity in 2300 A meta-(re)analysis of the effects of cognitive therapy psychiatric out-patients presenting for treatment versus ‘other therapies’ for depression. Journal of Affec- evaluated with a semi-structured diagnostic interview. tive Disorders, 68, 159–165. Psychological Medicine, 38(38), 199–210.