How do you solve a problem like adherence? Carla Walton and Nick Bendit Centre for Psychotherapy , Hunter New England Mental Health Service & University of Newcastle Mike Startup University of Newcastle Contact: [email protected] 1 Overview Aspects of treatment fidelity Our trial: RCT of 2 psychotherapies for Borderline Personality Disorder Implementing treatment fidelity in our trial Challenges associated with implementation of fidelity Discussion 2 Treatment fidelity / integrity The degree to which a treatment is implemented as intended & its ability to be differentiated from other approaches (Perepletchikova & Kazdin, 2005) 3 Why is treatment fidelity important? Need to know that treatment was delivered as intended. – i.e., that the treatment, the Independent Variable, was successfully manipulated. If treatment fidelity is not assessed, we can’t make any firm conclusions about intervention effects as we don’t know if the outcome was to do with the intervention or alternate factors. 4 Components of treatment fidelity 1. Therapist adherence 2. Therapist competence 3. Treatment differentiation 5 Therapist adherence The extent to which a therapist implements interventions prescribed by the treatment manual Assumes the existence of detailed and specific treatment intervention guidelines / manual Includes treatment behaviours that are unique and essential to the treatment Assessed using rating scales specific to the intervention guidelines / manual 6 Therapist competence “the extent to which the therapists conducting the interventions took the relevant aspects of the therapeutic context into account and responded to those contextual variables appropriately” (Waltz et al, 1993) it is context dependent and requires the therapist to understand when and how to intervene and when not to do so Needs to be independently verified 7 Treatment Differentiation Whether treatments under investigation differ from each other and can be distinguished from one another in their implementation Particularly important where the same therapists are delivering both arms of treatment. – Need to show that therapists aren’t combining therapies Previous studies, e.g., NIMH Treatment of Depression Collaborative Research Program used a scale that includes subscales for each of the treatment arms 8 How frequently is it implemented? Despite recognition of methodological necessity… Few studies adequately implement treatment fidelity procedures. In a review of RCTs of psychosocial interventions in 6 influential psychological and psychiatric journals, only 3.5% addressed treatment fidelity (Perepletchikova et al, 2007) No widely accepted methodology & little consistency across studies. 9 Recommendations for fidelity Establishing – Use of treatment manuals, therapist training & supervision Assessing – Using direct observation & psychometrically sound measures Evaluating - ensuring accuracy of obtained data, training of raters, assessing inter rater reliability Reporting 10 Our trial Randomised Clinical Trial comparing 2 psychotherapies in the treatment of Borderline Personality Disorder: – Dialectical Behaviour Therapy Conversational Model (CM) – locally developed in Australia: psychodynamic psychotherapy for disorders of self, in particular BPD (Meares and Stevenson, 2000). Background: – Need for studies comparing DBT to other treatments – Set in a real world, naturalistic setting – Behavioural change vs self 11 Dialectical Behaviour Therapy (DBT) Originally developed as an adapted form of CBT for women with BPD who are suicidal and/or self-harming. Currently treatment with most evidence for treatment of BPD Several RCTs showing significant changes in: – Suicidal and self-mutilating behaviours – Decrease in service utilisation – Improvement in quality of life No change in level of misery 12 Conversational Model Adapted form of psychoanalytic therapy developed by Robert Hobson, modified by Russell Meares 1 wait list controlled study & 1 replication. Showed significant changes in: – Deliberate self-harm – Significant reductions in hopelessness and depression – Improvements in social functioning Gains were maintained at 5 yrs 13 Hypotheses 1. DBT will be more effective at reducing deliberate self-harm 2. Conversational Model will be more effective at reducing depression 14 Treatment In DBT treatment arm: – Weekly individual therapy (1 hour) – Weekly skills training group (2.5 hours) – access to therapist by phone during working hours or on-call phone 8:30am-10:00pm daily, 7 days per week. In CM treatment arm: – Twice weekly individual therapy (1 hour each session) – Access to therapist by phone during working hours 15 Design Persons referred to Centre for Psychotherapy and meeting inclusion criteria, invited to participate in the trial. If they consent, randomly allocated to DBT or CM. Both groups receive 14 months of therapy. Assessed at baseline, mid-treatment (7 months), post- treatment (14 months), 1 year, 2 year and 5 year follow-up. Mostly same therapists providing both treatments. Recruitment began January 2007. 16 Status to date... Potential persons referred to service (n=76) Eligible (n = 65): Meet diagnostic criteria for BPD Ineligible (n =11) Min of 3 self-harm episodes in past year Randomised Between age of 18-65 Would not consent for randomisation or Had prior experience of DBT Dialectical Behaviour Therapy (n = 33) Conversational Model (n=32) Mid-treatment ax (7 mths) (n=21) Mid-treatment ax (7 mths) (n=20) Post-treatment ax (14 mths) (n=9) Post-treatment ax (14 mths) (n=9) F/U at 1 year post-treatment (26 mths) (n=3) F/U at 1 year post-treatment (26 mths) (n=2) F/U at 2 year post-treatment (38 mths) (n=0) F/U at 2 year post-treatment (38 mths) (n=0) F/U at 5 year post-treatment (74 mths) (n=0) F/U at 5 year post-treatment (74 mths) (n=0) 17 Implementing fidelity in current trial Fidelity is being established through: – use of treatment manuals – therapist training – therapist supervision. All sessions are being audiotaped and vast majority of participants have consented to this. Our issue is now the assessment and evaluation of fidelity. 18 Assessment of adherence in DBT A DBT adherence rating scale exists Developed by Marsha Linehan (treatment developer) and colleagues Only allowed to use it if you are an accredited rater and very few of these in the world at present. 19 Assessment of adherence in CM Existing rating scale used in a previous trial for a shorter version of CM. Would need to be modified to reflect treatment differences in long-term CM. 20 Therapist competence No existing therapist competence scales for either DBT or Conversational Model. 21 Treatment differentiation Critical for our trial to assess: – DBT bleeding into CM – CM bleeding into DBT Could establish this by rating a small sample from each trial using the adherence scale for the other treatment to (hopefully) show low adherence. 22 Challenges to fidelity in this trial Minimal funding. Lack of access to developed DBT scale: – Cost to have sessions rated expensive ($200 per session) – Expensive to be trained as accredited raters. – Lack of credibility for our adherence if we don’t use the Uni of Washington developed adherence scale Lack of availability of suitable CM scale 23 Challenges to fidelity in this trial Adherence in a study of long-term therapy For Conversational Model: – 14 months of treatment x twice weekly sessions =112 sessions per therapist / client dyad – 5% of sessions in a total sample (n=60) = 336 sessions to code For DBT: 168 individual sessions & 24 sessions of group therapy. 24 Solutions so far…. Applied for 3 grants.... but not received funding. Collaboration with an accredited DBT rater who will rate some of the sessions for a separate study.... But this will only be for a portion of the sessions Recruited a doctoral student to develop adherence scale for the CM treatment and to rate some of the tapes....but feasibly, he would only be able to rate a portion of the sessions. 25 Problems looking for solutions Which rating scales do we use? – Existing scales with high cost to either rate or be trained vs developing our own with less credibility Quantity of sessions that need to be rated – 5% equates to ~ 500 sessions (about 1000 hrs of rating) – but utility of doing less? how do we solve the problem of adherence? 26 How do you solve a problem like adherence? (sung to ‘how do you solve a problem like Maria’) How do you solve a problem like adherence? How do you rate more than 500 hours? When you don’t have an available scale And you can’t afford to pay How do we show we’re doing what we say Oh, how do we solve a problem like adherence? I’m counting on you to help me find a way. 27.