Learning the lessons: a multi- method evaluation of dedicated community-based services for people with personality disorder Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO) November, 2007 prepared by Dr Mike Crawford Imperial College London Queen's Printer and Controller of HMSO 2007 Page 1 Deborah Rutter Imperial College London Kathryn Price Imperial College London Tim Weaver Imperial College London Mili Josson Imperial College London Peter Tyrer Imperial College London Sarah Gibson Mental Health Foundation Sarah Gillespie Mental Health Foundation Alison Faulkner Mental Health Foundation Iain Ryrie Mental Health Foundation Kamaldeep Dhillon Mental Health Foundation Anthony Bateman University College London Peter Fonagy University College London Bethany Taylor University College London Paul Moran Kings College London Judith Beckett Service user researcher Helen Blackwell Service user researcher Colin Burbridge Queen's Printer and Controller of HMSO 2007 Page 2 Service user researcher Tina Coldham Service user researcher Dorothy Gould Service user researcher Susan Imlack Service user researcher Sheree Parfoot Service user researcher Kay Sheldon Service user researcher Angela Sweeney Service user researcher Emma Taylor Service user researcher Address for correspondence Dr Mike Crawford Reader in Mental Health Services Research Department of Psychological Medicine Imperial College London Claybrook Centre 37 Claybrook Road London W6 8LN E-mail: [email protected] Queen's Printer and Controller of HMSO 2007 Page 3 Contents Contents .....................................................................4 Glossary .....................................................................8 Acronyms ...................................................................9 Identity of pilot services and study participants.......10 Acknowledgements ..................................................11 Project Advisory Group Members.................................................11 Delphi Panel Members ...............................................................12 Executive Summary ..................................................13 Background..............................................................................13 Aims .......................................................................................13 Methods ..................................................................................14 Findings...................................................................................15 Conclusions..............................................................................18 Disclaimer................................................................................19 1 Background ...........................................................20 1.1 Current service provision......................................................20 1.2 Service user views...............................................................21 1.3 National service developments ..............................................21 1.4 Lessons learned ..................................................................22 1.5 Study aims .........................................................................22 2 Study methods ......................................................24 2.1 Overview............................................................................24 2.2 Study setting ......................................................................25 2.3 Project Advisory Group.........................................................25 2.4 Organisational evaluation .....................................................25 2.4.1 Data collection ...........................................................25 2.4.2 Data analysis .............................................................27 2.5 User-led qualitative evaluation ..............................................28 2.5.1 Recruitment, training and supervision of service user researchers ...............................................................28 2.5.2 Sampling and recruitment strategies .............................28 2.5.3 Interview schedules and topic guides ............................29 2.5.4 Data collection methods ..............................................29 2.5.6 Data analysis and validation.........................................30 2.6 Cohort study.......................................................................31 Queen's Printer and Controller of HMSO 2007 Page 4 2.6.1 Study questionnaires...................................................31 2.6.2 Data collection methods and follow-up ..........................32 2.6.3 Sample size ...............................................................33 2.6.4 Data analysis .............................................................33 2.7 Delphi study .......................................................................33 2.7.1 Development of the Delphi questionnaire.......................34 2.7.2 Selecting members of the Delphi panel ..........................34 2.7.3 Data collection and analysis .........................................35 2.8 Ethics ................................................................................35 3 Case studies ..........................................................37 3.1 Data collection ....................................................................37 3.2 Summary of service provision ...............................................37 4 Overarching themes ..............................................48 4.1 Service providers.................................................................48 4.1.1 Organisation of services...............................................48 4.1.2 Delivery of services .....................................................52 4.1.3 The service user’s journey ...........................................58 4.1.4 Managing crises and self-harm .....................................65 4.1.5 Staffing the pilots .......................................................69 4.1.6 Leadership and management .......................................78 4.1.7 Involvement of service users and carers ........................82 4.1.8 Working with other local services ..................................86 4.1.9 Issues in setting up and running new services ................91 4.2 Referrers and commissioners ................................................94 4.2.1 Referrers ...................................................................94 4.2.2 Commissioners .........................................................102 4.3 Service users and carers .................................................... 108 4.3.1 The ‘coming in’ process .............................................109 4.3.2 The service model.....................................................114 4.3.3 Relationships with staff..............................................121 4.3.4 Relationships with other service users .........................123 4.3.5 Service-user involvement ..........................................126 4.3.6 Outcomes ................................................................128 4.3.8 External factors ........................................................133 4.3.9 Carers .....................................................................133 4.3.10 Endings .................................................................135 4.3.11 Suggestions for service improvements .......................137 5 Cohort study........................................................140 5.1 Demographic and clinical characteristics............................... 140 5.2 Assessment and service provision........................................ 141 5.3 Minimum dataset............................................................... 142 5.4 Additional data.................................................................. 143 5.5 Self-referral ...................................................................... 143 Queen's Printer and Controller of HMSO 2007 Page 5 5.6 Ending contact with services ............................................... 144 6 Delphi study ........................................................153 6.1 Response rate...................................................153 6.2 Consensus items ............................................................... 153 6.3 Ranking items................................................................... 155 7 Discussion ...........................................................158 7.1 Overview of study findings.................................................. 158 7.1.1 Case studies and overarching themes..........................158 7.1.2 Cohort study ............................................................159 7.1.3 Delphi study.............................................................162 7.2 Study strengths and limitations........................................... 163 7.2.1 Timing of the study...................................................163 7.2.2 Response bias ..........................................................164 7.2.3 Study samples .........................................................164 7.2.4 Cohort data..............................................................165 7.2.5 Delphi study.............................................................165 7.3 Areas of convergence......................................................... 166 7.3.1 Organisation and delivery of direct services..................166 7.3.2. Personal attributes of staff working in