How do you solve a problem like adherence?
Carla Walton and Nick Bendit Centre for Psychotherapy , Hunter New England Mental Health Service & University of Newcastle Mike Startup
University of Newcastle
Contact: [email protected] 1 Overview
Aspects of treatment fidelity
Our trial: RCT of 2 psychotherapies for Borderline Personality Disorder
Implementing treatment fidelity in our trial
Challenges associated with implementation of fidelity
Discussion
2 Treatment fidelity / integrity
The degree to which a treatment is implemented as intended & its ability to be differentiated from other approaches (Perepletchikova & Kazdin, 2005)
3 Why is treatment fidelity important?
Need to know that treatment was delivered as intended. – i.e., that the treatment, the Independent Variable, was successfully manipulated.
If treatment fidelity is not assessed, we can’t make any firm conclusions about intervention effects as we don’t know if the outcome was to do with the intervention or alternate factors.
4 Components of treatment fidelity
1. Therapist adherence
2. Therapist competence
3. Treatment differentiation
5 Therapist adherence
The extent to which a therapist implements interventions prescribed by the treatment manual
Assumes the existence of detailed and specific treatment intervention guidelines / manual
Includes treatment behaviours that are unique and essential to the treatment
Assessed using rating scales specific to the intervention guidelines / manual
6 Therapist competence
“the extent to which the therapists conducting the interventions took the relevant aspects of the therapeutic context into account and responded to those contextual variables appropriately” (Waltz et al, 1993)
it is context dependent and requires the therapist to understand when and how to intervene and when not to do so
Needs to be independently verified
7 Treatment Differentiation
Whether treatments under investigation differ from each other and can be distinguished from one another in their implementation
Particularly important where the same therapists are delivering both arms of treatment. – Need to show that therapists aren’t combining therapies
Previous studies, e.g., NIMH Treatment of Depression Collaborative Research Program used a scale that includes subscales for each of the treatment arms 8 How frequently is it implemented?
Despite recognition of methodological necessity…
Few studies adequately implement treatment fidelity procedures.
In a review of RCTs of psychosocial interventions in 6 influential psychological and psychiatric journals, only 3.5% addressed treatment fidelity (Perepletchikova et al, 2007)
No widely accepted methodology & little consistency across studies.
9 Recommendations for fidelity
Establishing – Use of treatment manuals, therapist training & supervision
Assessing – Using direct observation & psychometrically sound measures
Evaluating
- ensuring accuracy of obtained data, training of raters, assessing inter rater reliability
Reporting 10 Our trial
Randomised Clinical Trial comparing 2 psychotherapies in the treatment of Borderline Personality Disorder: – Dialectical Behaviour Therapy Conversational Model (CM) – locally developed in Australia: psychodynamic psychotherapy for disorders of self, in particular BPD (Meares and Stevenson, 2000). Background: – Need for studies comparing DBT to other treatments – Set in a real world, naturalistic setting – Behavioural change vs self
11 Dialectical Behaviour Therapy (DBT)
Originally developed as an adapted form of CBT for women with BPD who are suicidal and/or self-harming.
Currently treatment with most evidence for treatment of BPD
Several RCTs showing significant changes in: – Suicidal and self-mutilating behaviours – Decrease in service utilisation – Improvement in quality of life
No change in level of misery
12 Conversational Model
Adapted form of psychoanalytic therapy developed by Robert Hobson, modified by Russell Meares
1 wait list controlled study & 1 replication. Showed significant changes in: – Deliberate self-harm – Significant reductions in hopelessness and depression – Improvements in social functioning
Gains were maintained at 5 yrs
13 Hypotheses
1. DBT will be more effective at reducing deliberate self-harm
2. Conversational Model will be more effective at reducing depression
14 Treatment
In DBT treatment arm: – Weekly individual therapy (1 hour) – Weekly skills training group (2.5 hours) – access to therapist by phone during working hours or on-call phone 8:30am-10:00pm daily, 7 days per week.
In CM treatment arm: – Twice weekly individual therapy (1 hour each session) – Access to therapist by phone during working hours
15 Design
Persons referred to Centre for Psychotherapy and meeting inclusion criteria, invited to participate in the trial.
If they consent, randomly allocated to DBT or CM.
Both groups receive 14 months of therapy.
Assessed at baseline, mid-treatment (7 months), post- treatment (14 months), 1 year, 2 year and 5 year follow-up.
Mostly same therapists providing both treatments.
Recruitment began January 2007.
16 Status to date...
Potential persons referred to service (n=76)
Eligible (n = 65):
Meet diagnostic criteria for BPD Ineligible (n =11) Min of 3 self-harm episodes in past year Randomised Between age of 18-65 Would not consent for randomisation or Had prior experience of DBT
Dialectical Behaviour Therapy (n = 33) Conversational Model (n=32)
Mid-treatment ax (7 mths) (n=21) Mid-treatment ax (7 mths) (n=20)
Post-treatment ax (14 mths) (n=9) Post-treatment ax (14 mths) (n=9)
F/U at 1 year post-treatment (26 mths) (n=3) F/U at 1 year post-treatment (26 mths) (n=2)
F/U at 2 year post-treatment (38 mths) (n=0) F/U at 2 year post-treatment (38 mths) (n=0)
F/U at 5 year post-treatment (74 mths) (n=0) F/U at 5 year post-treatment (74 mths) (n=0)
17 Implementing fidelity in current trial
Fidelity is being established through: – use of treatment manuals – therapist training – therapist supervision.
All sessions are being audiotaped and vast majority of participants have consented to this.
Our issue is now the assessment and evaluation of fidelity.
18 Assessment of adherence in DBT
A DBT adherence rating scale exists
Developed by Marsha Linehan (treatment developer) and colleagues
Only allowed to use it if you are an accredited rater and very few of these in the world at present.
19 Assessment of adherence in CM
Existing rating scale used in a previous trial for a shorter version of CM.
Would need to be modified to reflect treatment differences in long-term CM.
20 Therapist competence
No existing therapist competence scales for either DBT or Conversational Model.
21 Treatment differentiation
Critical for our trial to assess: – DBT bleeding into CM – CM bleeding into DBT
Could establish this by rating a small sample from each trial using the adherence scale for the other treatment to (hopefully) show low adherence.
22 Challenges to fidelity in this trial
Minimal funding.
Lack of access to developed DBT scale: – Cost to have sessions rated expensive ($200 per session) – Expensive to be trained as accredited raters. – Lack of credibility for our adherence if we don’t use the Uni of Washington developed adherence scale
Lack of availability of suitable CM scale
23 Challenges to fidelity in this trial
Adherence in a study of long-term therapy
For Conversational Model: – 14 months of treatment x twice weekly sessions =112 sessions per therapist / client dyad – 5% of sessions in a total sample (n=60) = 336 sessions to code
For DBT: 168 individual sessions & 24 sessions of group therapy.
24 Solutions so far….
Applied for 3 grants.... but not received funding.
Collaboration with an accredited DBT rater who will rate some of the sessions for a separate study.... But this will only be for a portion of the sessions
Recruited a doctoral student to develop adherence scale for the CM treatment and to rate some of the tapes....but feasibly, he would only be able to rate a portion of the sessions.
25 Problems looking for solutions
Which rating scales do we use? – Existing scales with high cost to either rate or be trained vs developing our own with less credibility
Quantity of sessions that need to be rated – 5% equates to ~ 500 sessions (about 1000 hrs of rating) – but utility of doing less?
how do we solve the problem of adherence?
26 How do you solve a problem like adherence? (sung to ‘how do you solve a problem like Maria’)
How do you solve a problem like adherence?
How do you rate more than 500 hours?
When you don’t have an available scale
And you can’t afford to pay
How do we show we’re doing what we say
Oh, how do we solve a problem like adherence?
I’m counting on you to help me find a way.
27