BMJ Open Is Committed to Open Peer Review. As Part of This Commitment We Make the Peer Review History of Every Article We Publish Publicly Available

BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from

BMJ Open is committed to open peer review. As part of this commitment we make the peer review history of every article we publish publicly available.

When an article is published we post the peer reviewers’ comments and the authors’ responses online. We also post the versions of the paper that were used during peer review. These are the versions that the peer review comments apply to.

The versions of the paper that follow are the versions that were submitted during the peer review process. They are not the versions of record or the final published versions. They should not be cited or distributed as the published version of this manuscript.

BMJ Open is an open access journal and the full, final, typeset and author-corrected version of record of the manuscript is available on our site with no access controls, subscription charges or pay-per-view fees (http://bmjopen.bmj.com).

If you have any questions on BMJ Open’s open peer review process please email [email protected] http://bmjopen.bmj.com/ on September 29, 2021 by guest. Protected copyright. BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from

Building motivation to participate in a Quality Improvement Collaborative; a qualitative participatory evaluation

Journal: BMJ Open ManuscriptFor ID peerbmjopen-2017-020930 review only Article Type: Research

Date Submitted by the Author: 05-Dec-2017

Complete List of Authors: Lalani, Mirza; Primary Care and Population Health Hall, Kate; UCLPartners Skrypak, Mirek; UCLPartners Laing, C; Royal Free Hospital, London, UCL Centre for Nephrology Welch, John; University College London Hospitals NHS Foundation Trust, Critical Care Department Toohey, Peter; UCLPartners Seaholme, Sarah; UCLPartners Weijburg, Thomas; UCLPartners Eyre, Laura; University College London Research Department of Primary Care and Population Health Marshall, Martin; University College London, Primary Care and Population Health http://bmjopen.bmj.com/ Primary Subject Health services research Heading:

Secondary Subject Heading: Qualitative research

Quality in health care < HEALTH SERVICES ADMINISTRATION & Keywords: MANAGEMENT, QUALITATIVE RESEARCH, HEALTH SERVICES ADMINISTRATION & MANAGEMENT

on September 29, 2021 by guest. Protected copyright.

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 1 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 4 Building motivation to participate in a Quality Improvement 5 6 Collaborative; a qualitative participatory evaluation 7 Mirza Lalani,1* Kate Hall,2 Mirek Skrypak,2 Chris Laing,3 John Welch,4 Peter Toohey,2 Sarah Seaholme,2 8 Thomas Weijburg,2 Laura Eyre,1 Martin Marshall1 9 10 11 1 Research Department of Primary Care and Population Health, University College London, London, UK. 12 2 13 UCLPartners, London, UK. 3 14 The Royal Free London NHS Foundation Trust, London, UK. 4 15 University College London Hospitals NHS Foundation Trust, London, UK. 16 For peer review only 17 *Corresponding author: Mirza Lalani, Research Department of Primary Care and Population Health, Upper 18 19 Third Floor, UCL Medical School (Royal Free Campus), Rowland Hill Street, London, NW3 2PF; 20 [email protected]; 07388 220242. 21 22 23 Word count: 3981. 24 25 Keywords: Quality Improvement Collaboratives; Participatory Evaluation 26 27 28 29 30 31 32 http://bmjopen.bmj.com/ 33 34 35 36 37 38 39

40 on September 29, 2021 by guest. Protected copyright. 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 1 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 2 of 13 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 ABSTRACT 4 Objectives: This study explores the barriers and facilitators that impact on the motivation of 5 practitioners to participate in a Quality Improvement Collaborative. 6 7 Design: A qualitative and formative evaluation using a participative approach, the Researcher-in- 8 Residence model which embraces the concept of ‘co-producing’ knowledge between researchers 9 and practitioners using a range of research methods such as participant observation, interviews and 10 documentary analysis. The design, creation and application of newly generated evidence is 11 12 facilitated by the researcher through negotiation and compromise with team members. 13 Participants: Senior and middle managers, doctors and nurses. 14 15 Setting: Two hospitals in South East England participating in a Patient Safety Improvement 16 Collaborative andFor the facilitator peer (host) of the review collaborative, based only in central London. 17 18 Results: The evaluation has revealed facilitators and barriers to motivation categorised under two 19 main themes; inherent motivation and factors that influence motivation – inter and intra- 20 organisational features as well as external factors. Facilitators included collaborative ‘champions,’ 21 individuals who drove the quality improvement agenda at a local level, raising awareness and 22 inspiring colleagues. The collaborative itself acted as a facilitator, promoting shared learning as well 23 24 as building motivation for participation. A key barrier was the lack of board engagement in the 25 participating NHS organisations which may have affected motivation amongst frontline staff. 26 27 Conclusions: Collaboratives maybe an important way of engaging practitioners in quality 28 improvement initiatives. This study highlights that despite a challenging healthcare environment in 29 the UK, there remains motivation amongst individuals to participate in quality improvement 30 programmes as they recognise that improvement approaches may facilitate positive change in local 31 clinical processes and systems. Collaboratives can harness this individual motivation to facilitate 32 spread and adoption of improvement methodology and build engagement across their membership. http://bmjopen.bmj.com/ 33 34 35 36 STRENGTHS AND LIMITATIONS OF THIS STUDY 37 38 • There are relatively few studies in which a researcher has been embedded in a Quality 39 Improvement Collaborative as is the case in this study.

40 • The evaluation generated evidence that was mobilised by the researcher as the Patient on September 29, 2021 by guest. Protected copyright. 41 Safety Collaborative progressed, contributing to the development of the programme and to 42 meeting its objectives. 43 • The study develops our understanding of the barriers and facilitators that affect the 44 motivation of clinicians and managers to engage with quality improvement initiatives. 45 46 • A limitation of the study is that the researcher was only embedded in two hospital trusts, 47 raising questions of generalisability, nevertheless, we believe the concepts generated are 48 likely to be transferrable to the rest of the Improvement Collaborative and other similar 49 initiatives. 50 51 52 53 54 55 56 57 58 2 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 3 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 BACKGROUND 4 Quality Improvement Collaboratives in healthcare bring together groups of health professionals, 5 managers and support staff either within an organisation or from several organisations, to work on a 6 common purpose, with the goal of improving health services.[1] Improvement Collaboratives have 7 8 become increasingly popular in the UK, with the NHS promoting them as a mechanism for change 9 across the healthcare system involving different types of organisations including GP surgeries, 10 hospitals and care homes.[2-4] Improvement Collaboratives frequently follow a Breakthrough Series 11 approach supporting organisations to close the gap between good care and usual practice in a short 12 time period (6-18 months).[5] During this time, teams meet to share learning on a topic (learning 13 sessions), to understand how to make improvements, to implement and test these improvements 14 (action periods), and to share their progress and results with the rest of the collaborative.[6] 15 Collaboratives provide an infrastructure for an inter-organisational support network from which 16 For peer review only 17 members can address common barriers and learn from others’ successes and challenges.[7] The 18 success of Improvement Collaboratives is determined by the influence of their inter-organisational 19 (shared learning) and intra-organisational features (culture, resources, leadership etc.).[8] 20 21 The evidence for the effectiveness of Improvement Collaboratives as interventions for improving 22 health outcomes is growing but is not as yet compelling. Indeed, collaboratives may have a greater 23 impact on changes to professional behaviour and care processes than on care quality or health 24 service outcomes.[9] This is in part due to the heterogeneity of contexts within which collaboratives 25 operate.[10] Additionally, research studies have focussed on demonstrating ‘what’ impact an 26 Improvement Collaborative has had, overlooking ‘why,’ ‘how’ and ‘what works for whom in what 27 context,’ with relatively few qualitative evaluations that can provide a rationale for their impact on 28 29 outcomes.[11] 30 Studies have explored factors influencing clinician engagement in quality improvement but have not 31 considered the role of motivation to participate, especially within a challenging healthcare 32 http://bmjopen.bmj.com/ 33 environment with workforce problems such as low morale, increasing numbers of vacant clinical 34 posts and issues with staff retention.[12-14] Quality improvement initiatives often rely upon the 35 inherent motivation of practitioners to provide high quality care for their patients.[15] Carter et al 36 [16] proposed that concepts such as ‘collaborative advantage’ emerging within a collaborative may 37 affect motivation, instilling change through promoting competition amongst teams who may strive 38 to attain the same level of advancement of others. Mixed empirical evidence for the effectiveness of 39 Improvement Collaboratives coupled with minimal understanding of the motivation of practitioners 40 on September 29, 2021 by guest. Protected copyright. 41 to participate presents an important challenge to proponents of quality improvement approaches in 42 healthcare. 43

40 Study design on September 29, 2021 by guest. Protected copyright. 41 The evaluation was undertaken in a series of iterative stages of participation; data collection, 42 analysis, interpretation and dissemination of emerging findings, with the application of evidence to 43 influence the development of the programme. Qualitative methods were used to generate and 44 45 analyse the data. 46 Data collection 47 48 Participant observation involved attendance at trust and Patient Safety Collaborative meetings as 49 well as collaborative learning sessions, totalling approximately 100 hours. As the evaluation 50 progressed the researcher became an active participant in meetings, providing input and facilitating 51 discussion. Field notes at meetings were recorded. Pertinent points arising from these observations 52 were communicated to the teams to facilitate discussion at future meetings and to enable them to 53 determine how to use the findings in relation to the progress of the collaborative. Semi-structured 54 interviews (n=15) were held with team members of the two hospital trusts and the Patient Safety 55 Collaborative team. An additional two participants from within the collaborative were identified by 56 57 58 4 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 5 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 snowball sampling.[21] The interview guide considered the existing Improvement Collaborative 4 literature in its design and was further informed in its development by data from participant 5 observation and documentary review as well as input from members of the evaluation committee. 6 The guide was adapted iteratively using an inductive approach allowing the exploration of new 7 emergent themes. Interviews ceased once thematic saturation was reached. A review of documents 8 pertaining to the collaborative at trust level and the Patient Safety Collaborative such as meeting 9 10 notes was also undertaken. Documents were carefully scrutinised and emerging relevant themes 11 were mapped to the thematic framework discussed below. 12 13 Data analysis 14 The first level of qualitative data analysis was undertaken using a thematic framework approach.[22] 15 The framework was developed iteratively to capture emerging themes from the data. Components 16 of the analysis planFor such as thepeer coding framework review were shared with only evaluation committee members, 17 which facilitated their understanding of how their perceptions of the collaborative influenced their 18 engagement with it. Co-interpretation with the committee of emerging themes was also conducted. 19 20 21 22 RESULTS 23 Two main categories of themes centred on the core concept of motivation emerged; 1) inherent 24 motivation - relating to personal drivers and 2) factors that influence motivation – inter and intra- 25 organisational features common to Improvement Collaboratives as well as external factors. 26 27 Inherent motivation (personal drivers) 28 Inherent motivation at the level of the individual was integral to participation in the collaborative. 29 30 Clinicians mentioned using the collaborative as an opportunity for career and professional 31 advancement. Patient safety teams focussed on recruiting junior doctors, who were required to

32 document evidence of quality improvement work in their career portfolios. Additionally, some of the http://bmjopen.bmj.com/ 33 interviewees mentioned that witnessing the positive effect of improvement interventions provided a 34 sense of personal achievement which acted as an enabler in motivating individuals. 35 36 Motivation was also shaped by individual attitudes and perceptions such as the extent of 37 understanding of the purpose and value of quality improvement. Some clinicians suggested that 38 quality improvement was not a priority and was regarded as secondary to the need to deliver 39 routine care. Additionally, interviewees mentioned a degree of confusion amongst some of their 40 staff about their understanding of the definition of quality improvement. Quality improvement was on September 29, 2021 by guest. Protected copyright. 41 also described by some as burdensome and unnecessary as it did not appear to directly benefit 42 43 patients. Some interviewees mentioned using improvement methodology to focus on marginal 44 gains, ensuring that new processes were more beneficial to patient care with a preference for 45 demonstrating small scale changes at a ward level. 46 47 ‘It's about allowing people to try things out in a small environment…..your tests of change 48 through quality improvement are based on evidence but it’s small scale steps of evidence, 49 and if you can change something for just one person that must be a good thing.’ Manager, 50 partner trust 51 52 As the collaborative progressed, a few ‘champions’ identified by the Patient Safety Collaborative 53 team, emerged. These champions were similar to change agents, driving the patient safety agenda in 54 their teams and departments. Their motivation to shape the local level patient safety strategy and 55 the sharing of this narrative with the collaborative led to the spread and adoption of a champion’s 56 57 58 5 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 6 of 13 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 model in other teams. Some champions emerged in trusts where there were significant internal and 4 external pressures. 5 6 ' And then organisations that aren’t necessarily in the good organisation space, they’ve really 7 risen to the challenge and become involved, partly thanks to these individuals.' Patient Safety 8 Collaborative team member 9 10 11 Factors affecting motivation 12 Intra-organisational features 13 14 The most frequently mentioned intra-organisational features that were potential barriers and 15 facilitators to motivation were; conflicting priorities, engagement and clinical ownership, and 16 support and culture.For peer review only 17 18 Conflicting priorities 19 At the time of this programme, patient safety teams in trusts from across the collaborative were 20 allocating a significant proportion of their time to undertaking Commissioning for Quality and 21 Innovation (CQUIN) data collection to meet financially incentivised targets. Interviewees felt that 22 CQUIN detracted them from improving safety in the organisation, which ironically was the premise 23 of the project. 24 25 ‘….because our team spend so much time collecting data; reviewing patient records; trying to 26 find the information, they aren’t out there using information to make a difference; through 27 teaching; training; coaching…… because, ultimately, CQUIN schemes earn this organisation 28 about one and a half million a year. ‘ Manager, partner trust 29 30 Engagement and clinical ownership 31 This was a key barrier to motivation at trust level. Interviewees mentioned insufficient support from 32 some frontline clinicians, which was seen as limiting the embedding of quality improvement http://bmjopen.bmj.com/ 33 processes and affecting the motivation of team members. This was compounded by mainly senior 34 35 staff undertaking improvement work with junior nurses in particular, neither aware or involved. 36 These issues were thought to reflect a broader challenge of embedding an improvement culture 37 within organisations. 38 39 ‘…more could be done to raise awareness among the junior nurses about quality

40 improvement. It seems to be something the consultants and senior nurses do. I think we on September 29, 2021 by guest. Protected copyright. 41 should all be doing it or at least be aware of it or how will the whole organisation improve?’ 42 Nurse, partner trust 43 44 In addition, one interviewee claimed that their organisation, despite previously having a good 45 reputation for safety improvement, was now prioritising efficiency to ensure compliance with 46 targets such as reducing Accident and Emergency waiting times. This efficiency drive was led by the 47 Chief Executive Officer whose focus was on the operational aspects of the everyday running of the 48 hospital. 49 50 ‘The organisation is so operational. People should be thinking a bit more strategically and 51 there should be discussions and planning as a joint effort. But actually the people at those 52 levels, including the Chief Executive, spend a lot of time running the operations of the 53 hospital.’ Senior Manager, partner trust 54 55 56 57 58 6 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 7 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 Support and culture 4 It was suggested that with fewer resources, individuals responsible for implementing quality 5 improvement projects were less motivated and that this may have affected their willingness to 6 participate in the collaborative. Moreover, medical directors in some trusts did not involve their 7 clinicians in discussions about joining the collaborative and initially, some of these teams struggled 8 to engage. Even so, as some interviewees mentioned, motivation could not be created, it had to 9 exist within organisational culture. One of the collaborative organisations had a self-professed 10 11 culture of improvement formed through a top down approach resulting in a directorate of quality 12 and a well-resourced patient safety team. Yet, even in this organisation, some expressed concerns 13 about the extent of resource prioritisation for patient safety. 14 15 Interviewer: ‘Why can't that (the funding gap) be filled by the Board with resources from 16 within the organisation?’For peer review only 17 18 Participant: ’absolutely could be. But it's about showing that there's some return on 19 investment. You know, Ophthalmology is a department that earns money. Maternity is a 20 department that earns money. Patient Safety isn't a department that earns money. We save 21 money. It's quite difficult.’ Senior manager, partner trust 22 23 24 Inter-organisational features 25 Interviewees were generally positive about participating in the collaborative identifying inter- 26 organisational learning and sharing as key motivating factors to ongoing participation. Through 27 28 observation of learning sessions, it was apparent that teams were proud of their achievements and 29 showcasing these successes was an important incentive. 30 ‘…we do not have many resources, but we have embraced learning and driven the patient 31 safety work in our trust. I am sure others can do the same and if they come to appreciate 32 http://bmjopen.bmj.com/ 33 that from seeing our achievements, that can only be a positive thing.’ Nurse, partner trust 34 Some interviewees suggested that the reputation of the collaborative was a significant reason for 35 36 trusts to join. The Patient Safety Collaborative team members described their perceived successes 37 and challenges relevant to ongoing motivation of the collaborative participants. Raising awareness 38 of AKI and sepsis and information sharing across the collaborative through the use of emails, social 39 media, webinars, teleconferences and learning sessions was viewed as a key success. Also

40 mentioned, was facilitating the understanding of the benefits of sustained quality improvement on September 29, 2021 by guest. Protected copyright. 41 approaches on health outcomes. 42 43 Sustaining engagement with the collaborative was seen as a significant challenge and was thought to 44 be as a result of intra-organisational issues such as staff turnover or dwindling resources. Patient 45 Safety Collaborative team members also identified two key learnings; 1) in some cases, they had not 46 fully explored and understood the factors affecting individual trust engagement and 2) they had not 47 48 assessed the readiness of some organisations to effectively participate in the collaborative. 49 Nonetheless, even if reasons for less engagement were known, the Patient Safety Collaborative 50 team struggled to achieve consensus on how best to support the relevant trust teams to overcome 51 these factors, potentially affecting their motivation. 52 53 'It just highlights how you can’t just brainwash an MD at a meeting and then rely on them for 54 people to start coming and devolving time, you know. So actually the Trusts that weren’t 55 performing or coming or engaging, I think that was a reflection of us.' Patient Safety 56 Collaborative team member 57 58 7 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 8 of 13 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 External factors 4 Two emerging sub-themes relating to the health system (context and system pressures) that may 5 have affected motivation were identified as external factors. The context within which the 6 collaborative operated was similar in all trusts. For example, imminent or recent inspections by the 7 national regulator, the Care Quality Commission (CQC) were frequently cited as a reason for joining 8 9 the collaborative. There was also heightened awareness around sepsis due to a national campaign 10 advocating for improving its recognition and treatment and significant changes to National Institute 11 of Clinical Excellence guidelines.[23 24] Such high profile developments coupled with increased 12 media scrutiny of several avoidable deaths,[25] were seen by interviewees as potential drivers for 13 participation in the collaborative. Relevant to health systems pressures, interviewees from the trust 14 teams expressed an ambivalence to minimal resource provision for patient safety, acknowledging 15 that working within tight parameters was the norm, yet, participating in quality improvement was 16 still possible. For peer review only 17 18 ‘You have to look at what resources you've got, and then how you can do what you need to 19 do within that..in this climate, and in this organisation, I think we should be realistic, we have 20 21 to work differently to achieve the outcomes that we need to achieve.’ Manager, partner trust 22 23 24 DISCUSSION 25 The study has revealed three principal findings that provide important lessons for quality 26 improvement programmes in the NHS. Firstly, individuals can act as change agents, driving the 27 quality improvement agenda at a local level. Secondly, inadequate board engagement at trust level 28 29 may affect motivation amongst team members. Finally, Improvement Collaboratives may have an 30 important role in the UK healthcare system, as they promote shared learning and the formation of 31 networks that are established on the premise of mutual experience and a common purpose.

32 Following the completion of this collaborative programme, a deteriorating patient community of http://bmjopen.bmj.com/ 33 practice was established as well as a focus on paediatric sepsis across London. This is one such 34 example of the potential legacy of Improvement Collaboratives. 35 36 A strength of this study was that the participatory approach to the evaluation made a positive 37 contribution to the progression of the collaborative in terms of: the provision of operational support; 38 mobilisation of current knowledge (from the academic literature) as well as sharing newly generated 39 knowledge to assist the programme in meeting its objectives; and the observation and sharing of 40 on September 29, 2021 by guest. Protected copyright. information to connect different components of the programme. An in-depth focus on just two trust 41 42 teams was a trade-off for more generalisable findings. Nonetheless, one of these teams was 43 representative of most of the collaborative members in terms of its intra-organisational features. 44 Moreover, the in-depth participatory approach to the evaluation enabled the researcher to generate 45 findings and insights on the motivation of individuals within a patient safety team which may be 46 transferable to similar settings in NHS trusts. An additional limitation was that of the in-residence 47 model whereby sometimes it was challenging for the researcher to maintain objectivity with fellow 48 team members as participants in the evaluation. This was minimised by the researcher regularly 49 50 discussing findings with independent academic colleagues. 51 Most studies in this field have used a summative approach to evaluate the effectiveness of an 52 Improvement Collaborative as an intervention to improve patient outcomes. In contrast, this study 53 54 provides a novel approach, using a qualitative and formative evaluation with a researcher embedded 55 in the host organisation of an Improvement Collaborative. This approach has revealed some key 56 aspects of the ‘black box’ that exists in terms of understanding of ‘how’ and ‘why’ a collaborative is 57 58 8 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 9 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 effective through focussing on member motivation at the level of the individual and organisation. 4 Yet, it has not established the effect of motivation within this Patient Safety Collaborative on patient 5 outcomes for AKI and/or sepsis. 6 7 Three key aspects of embedded research identified from a recent review of the role of embedded 8 research in quality improvement programmes [26] have been identified as relevant to this 9 evaluation; (1) the researcher became a key member of the Patient Safety Collaborative team 10 playing an operational and evaluative role, (2) important working relationships with staff were 11 developed and (3) knowledge was co-produced through partnership working with the evaluation 12 committee. These aspects were especially pertinent to an embedded researcher working as a 13 14 member of an organisation in an Improvement Collaborative whereby the implementation of the 15 improvement methodology and its perceived impact required ‘real-time’ feedback from 16 collaborative membersFor to enable peer the host teamreview to adapt and improve only its processes, directed toward 17 encouraging the teams to learn and share from each other. 18 19 This study has important implications for clinicians and managers implementing quality 20 improvement programmes in NHS organisations as well for those agencies involved in hosting such 21 initiatives. At the individual level, career advancement and personal achievement provide 22 organisations a focus for incentivising clinician participation in quality improvement as is the case 23 with junior doctors in this evaluation whose recruitment to participate in such initiatives may 24 enhance organisational systems and develop medical professionalism.[27] The individual perception 25 26 of the term ‘quality improvement’ also influenced motivation and caused confusion amongst some, 27 acting as a potential barrier to motivation. The evaluation suggests that reframing quality 28 improvement for clinicians, as a series of marginal gains, whereby the immediate discernible and 29 small-scale benefits are seen as acceptable, may encourage participation in improvement 30 programmes. It is rare to achieve the dramatic scale of improvement seen in the much cited 31 ‘Matching Michigan’ study.[28] 32 http://bmjopen.bmj.com/ 33 The findings also highlight important external factors (over which the collaborative teams had little 34 control) that influence motivation and the ability of teams to engage, potentially negatively affecting 35 other members of the collaborative network.[29] Research has suggested that existing 36 organisational culture can be superseded by sufficiently empowered managers and clinicians, similar 37 38 to the champions of the collaborative, who emerged from trusts where an improvement culture was 39 not apparent.[30] Armenakis et al suggest that to institutionalise permanent change, change agents

40 require credibility within their organisation.[31] In this Improvement Collaborative, ‘champions’ on September 29, 2021 by guest. Protected copyright. 41 were senior clinical staff who had demonstrated a desire to lead an improvement programme. The 42 Patient Safety Collaborative team successfully harnessed the inherent motivation of these 43 individuals resulting in spread and adoption of several ‘champions’ inspired approaches to patient 44 safety across the collaborative. 45 46 The ‘all share, all learn’ approach of the Patient Safety Collaborative team has revealed some of the 47 potential benefits of a non-hierarchical network, such as promoting togetherness amongst teams, 48 providing reassurance, trust and a common purpose [32], which is especially pertinent at a time 49 50 when pervasive health system pressures are affecting workforce motivation.[33] The study findings 51 suggest that the collaborative was integral in raising awareness and facilitating the improvement of 52 local level clinical processes for AKI and sepsis. The reputation of the Patient Safety Collaborative 53 host organisation was identified as a means of gaining recognition with trust boards, the CQC and 54 other clinicians. Hence, there is a potential gap for Improvement Collaboratives to bridge in the 55 healthcare system in England, in building capability and capacity and motivating individuals to 56 57 58 9 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 10 of 13 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 participate in quality improvement, especially in organisations that have neither the resource or 4 volition. 5 6 The NHS trusts in this Improvement Collaborative demonstrated different levels of readiness despite 7 facing similar challenges relating to their intra-organisational features. This was in part associated 8 with the engagement of trust boards, which demonstrated an initial interest in participating in the 9 collaborative by signing a contract agreement with the host organisation, but, in some cases this 10 failed to develop into a sustained commitment and may have been overtaken by other priorities. 11 Using a bottom up approach, involving frontline clinicians from the outset, may facilitate the 12 continued engagement of organisations in an Improvement Collaborative. Additionally, other intra- 13 14 organisational features such as a lack of support and resources are prominent in this evaluation and 15 affect motivation. Nevertheless, some interviewees mentioned that carrying out quality 16 improvement workFor in addition peer to providing reviewroutine care within existing only resource parameters was 17 acceptable and achievable. A recent King’s fund report suggested that trusts and individuals should 18 be pursing approaches to continuously improve the quality of care despite a lack of resources.[34] 19 20 The host organisation in this Improvement Collaborative is one of several Academic Health Science 21 Networks across England involved in building improvement capacity and capability within their 22 partnership of NHS trusts. This raises a question of whether it is the responsibility of individual NHS 23 organisations themselves to allocate resources for improvement initiatives or whether this is the 24 role of an external organisation. The host organisation’s role is to develop programme aims, disease 25 26 specific measures and tools as well as to facilitate and provide technical support. Yet, in this study 27 they often acted as a proxy quality improvement team for some of the NHS organisations. This is a 28 significant challenge for external organisations as they try to find a specific place within a healthcare 29 system whilst acting as enablers for their partners. This study should initiate further discussion and 30 examination of the role of external organisations such as Academic Health Science Networks that 31 may provide significant and meaningful support for quality improvement to individual NHS trusts. 32 http://bmjopen.bmj.com/ 33 34 35 FUNDING STATEMENT 36 This work was supported by NHSEngland, grant number 533957. NHSEngland provided the funds to 37 UCLPartners to host a Patient Safety Collaborative of which some were allocated to undertaking this 38 evaluation. 39 40 on September 29, 2021 by guest. Protected copyright. 41 42 COMPETING INTERESTS 43 No competing interests are declared. 44 45 46 47 ETHICS APPROVAL 48 Ethics approval was obtained from the University College London research ethics committee. 49 Reference number 16/0007. 50 51 52 53 AUTHOR CONTRIBUTION 54 ML, MM, KH and MS conceived and planned the overall study. ML, MM and MS wrote the initial 55 study protocol. ML, MM, KH, MS, CL, JW, LE and PT co-designed aspects of the study including the 56 strategy for sampling of the two trusts for participation in the evaluation and reviewing tools for 57 58 10 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 11 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 data collection. ML undertook recruitment, data collection and analysis. ML, MM, KH, MS, CL, JW, 4 TW, SS, PT and LE formed the overall membership of the evaluation steering committee. All authors 5 co-interpreted the study findings. ML drafted the manuscript and revised it in response to comments 6 from all co-authors. All authors read and approved the final version of the manuscript. 7 8 9 10 ACKNOWLEDGEMENTS 11 The authors would like to thank all of the study participants and in particular the two trusts that 12 allowed the researcher to be embedded within their patient safety teams as well as the contribution 13 of several of the Patient Safety Collaborative team members through the course of the evaluation. 14 The authors would also like to acknowledge the contribution of Professor Roger Jones who provided 15 guidance on some aspects of the structure and framing of the manuscript. 16 For peer review only 17 18 19 DATA SHARING STATEMENT 20 No additional data are available. 21 22 23

24 25 26 27 28 29

30 31

32 http://bmjopen.bmj.com/ 33 34 35 36

37 38 39

40 on September 29, 2021 by guest. Protected copyright. 41 42 43

44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 11 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 12 of 13 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 REFERENCES 4 1. Øvretveit J, Bate P, Cleary P, et al. Quality collaboratives: lessons from research. Quality and safety 5 in health care 2002;11(4):345-51 6 2. Power M, Tyrrell PJ, Rudd AG, et al. Did a quality improvement collaborative make stroke care 7 better? A cluster randomised trial. Implementation science 2014;9 doi: 10.1186/1748-5908- 8 9-40[published Online First: Epub Date]|. 9 3. Dawda P, Jenkins R, Varnam R. Quality improvement in general practice: The King's Fund, 2010. 10 4. Working together to make all our lives better: Collaborative care in residential homes: The Health 11 Foundation, 2017. 12 13 5. Power M, Tyrrell PJ, Rudd AG, et al. Did a quality improvement collaborative make stroke care 14 better? A cluster randomized trial. Implement Sci 2014;9:40 15 6. The Breakthrough Series: IHI's collaborative model for achieving breakthrough improvement. 16 InnovationFor Series: Institute peer for Healthcare review Improvement, only2003. 17 7. Nadeem E, Olin SS, Hill LC, et al. Understanding the components of quality improvement 18 collaboratives: a systematic literature review. Milbank Quarterly 2013;91(2):354-94 19 8. Nembhard IM. Learning and Improving in Quality Improvement Collaboratives: Which 20 Collaborative Features Do Participants Value Most? Health Services Research 2009;44(2 Pt 21 1):359-78 doi: 10.1111/j.1475-6773.2008.00923.x[published Online First: Epub Date]|. 22 9. De Silva D. Improvement collaboratives in health care. London, UK: The Health Foundation 2014. 23 10. Schouten LM, Hulscher ME, van Everdingen JJ, et al. Evidence for the impact of quality 24 improvement collaboratives: systematic review. Bmj 2008;336(7659):1491-94 25 11. Howe C, Randall K, Chalkley S, et al. Supporting improvement in a quality collaborative. Br J 26 Healthc Manag 2013;19:434-42 27 12. Ling T. How Do You Get Clinicians Involved in Quality Improvement?: An Evaluation of the Health 28 Foundation's Engaging with Quality Initiative: A Programme of Work to Support Clinicians to 29 Drive Forward Quality: Final Report: The Health Foundation, 2010. 30 13. Davies H, Powell A, Rushmer R. Why don't clinicians engage with quality improvement? Journal 31 of Health Services Research & Policy 2007;12(3):129-30 doi: 32 http://bmjopen.bmj.com/ 10.1258/135581907781543139[published Online First: Epub Date]|. 33 34 14. Iacobucci G. Future gaps in workforce pose an impending crisis for the NHS, report warns: British 35 Medical Journal Publishing Group, 2013. 36 15. Dixon-Woods M, McNicol S, Martin GP. Ten challenges in improving quality in health care: 37 lessons from the health foundation’s programme evaluations and relevant literature. BMJ 38 Quality & Safety 2012;21 doi: 10.1136/bmjqs-2011-000760[published Online First: Epub 39 Date]|.

40 16. Carter P, Ozieranski P, McNicol S, et al. How collaborative are quality improvement on September 29, 2021 by guest. Protected copyright. 41 collaboratives: a qualitative study in stroke care. Implementation Science 2014;9(1):1-11 doi: 42 10.1186/1748-5908-9-32[published Online First: Epub Date]|. 43 17. Berwick D. A Promise to Learn – a Commitment to Act: Improving the Safety of Patients in 44 England. London, England: NHS, 2013. 45 18. Patient Safety: The UCLPartners Patient Safety Programme. Secondary Patient Safety: The 46 UCLPartners Patient Safety Programme 2017. http://uclpartners.com/what-we-do/patient- 47 safety/. 48 19. Eyre L, Farrelly M, Marshall M. What can a participatory approach to evaluation contribute to the 49 field of integrated care? BMJ Qual Saf 2016 doi: 10.1136/bmjqs-2016-005777[published 50 Online First: Epub Date]|. 51 20. Marshall M, Pagel C, French C, et al. Moving improvement research closer to practice: the 52 Researcher-in-Residence model. BMJ quality & safety 2014:bmjqs-2013-002779 53 54 21. Atkinson R, Flint J. Social Research Update. 55 56 57 58 12 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 13 of 13 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 22. Gale NK, Heath G, Cameron E, et al. Using the framework method for the analysis of qualitative 4 data in multi-disciplinary health research. BMC Med Res Methodol 2013;13:117 doi: 5 10.1186/1471-2288-13-117[published Online First: Epub Date]|. 6 23. Freitag A, Constanti M, O'flynn N, et al. Suspected sepsis: summary of NICE guidance. BMJ: 7 British Medical Journal 2016;354 8 24. Wise J. Sepsis should be treated within one hour, says NICE. BMJ 2017;356 doi: 9 10.1136/bmj.j1257[published Online First: Epub Date]|. 10 25. Rawlinson K. Spotting sepsis 'could prevent 37,000 deaths a year. The Guardian 2016 15th 11 December 2016. 12 26. Vindrola-Padros C, Pape T, Utley M, et al. The role of embedded research in quality 13 improvement: a narrative review. BMJ Qual Saf 2017;26(1):70-80 14 27. Involving junior doctors in quality improvement London, UK: The Health Foundation 2011. 15 28. Bion J, Richardson A, Hibbert P, et al. ‘Matching Michigan’: a 2-year stepped interventional 16 programmeFor to minimise peer central venous review catheter-blood streamonly infections in intensive care 17 units in England. BMJ Quality & Safety 2012 doi: 10.1136/bmjqs-2012- 18 19 001325[published Online First: Epub Date]|. 20 29. Aveling EL, Martin G, Herbert G, et al. Optimising the community-based approach to healthcare 21 improvement: Comparative case studies of the clinical community model in practice. Soc Sci 22 Med 2017;173:96-103 doi: 10.1016/j.socscimed.2016.11.026[published Online First: Epub 23 Date]|. 24 30. Phung VH, Essam N, Asghar Z, et al. Exploration of contextual factors in a successful quality 25 improvement collaborative in English ambulance services: cross-sectional survey. Journal of 26 evaluation in clinical practice 2016;22(1):77-85 doi: 10.1111/jep.12438[published Online 27 First: Epub Date]|. 28 31. Armenakis AA, Harris SG, Feild HS. Making change permanent A model for institutionalizing 29 change interventions. Research in organizational change and development: Emerald Group 30 Publishing Limited, 2000:97-128. 31 32. Health leaders' panel survey 6: footprints, financing and staff morale: Nuffield Trust, 2015. 32 33. Huxham C, Vangen S. Managing to collaborate: The theory and practice of collaborative http://bmjopen.bmj.com/ 33 advantage. Abingdon: Routledge, 2013. 34 34. Limb M. Doctors must pursue quality improvement despite challenges. BMJ careers 2015 35 36 37 38 39

40 on September 29, 2021 by guest. Protected copyright. 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 13 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from

Building motivation to participate in a Quality Improvement Collaborative in NHS hospital trusts in south-east England; a qualitative participatory evaluation

ForJournal: peerBMJ Open review only Manuscript ID bmjopen-2017-020930.R1

Article Type: Research

Date Submitted by the Author: 08-Feb-2018

Primary Subject Health services research Heading:

Secondary Subject Heading: Qualitative research

Quality in health care < HEALTH SERVICES ADMINISTRATION & Keywords: MANAGEMENT, QUALITATIVE RESEARCH, HEALTH SERVICES ADMINISTRATION & MANAGEMENT on September 29, 2021 by guest. Protected copyright.

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 1 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 4 Building motivation to participate in a Quality Improvement 5 6 Collaborative in NHS hospital trusts in south-east England; a 7 qualitative participatory evaluation 8 1* 2 2 3 4 2 2 9 Mirza Lalani, Kate Hall, Mirek Skrypak, Chris Laing, John Welch, Peter Toohey, Sarah Seaholme, 2 1 1 10 Thomas Weijburg, Laura Eyre, Martin Marshall 11 12 13 1 Research Department of Primary Care and Population Health, University College London, London, UK. 14 2 UCLPartners, London, UK. 15 3 The Royal Free London NHS Foundation Trust, London, UK. 16 4 University CollegeFor London Hospitals peer NHS Foundation review Trust, London, UK. only 17

18 19 *Corresponding author: Mirza Lalani MPharm MSc, Research Department of Primary Care and Population 20 Health, Upper Third Floor, UCL Medical School (Royal Free Campus), Rowland Hill Street, London, NW3 2PF; 21 [email protected]; 07388 220242. 22 23 24 25 Word count: 4412. 26 Keywords: Quality Improvement Collaboratives; Participatory Evaluation 27 28 29 30 31 32 http://bmjopen.bmj.com/ 33 34 35 36 37 38 39 40 on September 29, 2021 by guest. Protected copyright. 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 1 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 2 of 15 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 ABSTRACT 4 Objectives: This study explores the barriers and facilitators that impact on the motivation of 5 practitioners to participate in a Quality Improvement Collaborative. 6 7 Design: A qualitative and formative evaluation using a participatory approach, the Researcher-in- 8 Residence model which embraces the concept of ‘co-producing’ knowledge between researchers 9 and practitioners using a range of research methods such as participant observation, interviews and 10 documentary analysis. The design, creation and application of newly generated evidence is 11 12 facilitated by the researcher through negotiation and compromise with team members. 13 Participants: Senior and middle managers, doctors and nurses. 14 15 Setting: Two hospitals in south-east England participating in a Patient Safety Improvement 16 Collaborative andFor the facilitator peer (host) of the review collaborative, based only in central London. 17 18 Results: The evaluation has revealed facilitators and barriers to motivation categorised under two 19 main themes; inherent motivation and factors that influence motivation – inter and intra- 20 organisational features as well as external factors. Facilitators included collaborative ‘champions,’ 21 individuals who drove the quality improvement agenda at a local level, raising awareness and 22 inspiring colleagues. The collaborative itself acted as a facilitator, promoting shared learning as well 23 24 as building motivation for participation. A key barrier was the lack of board engagement in the 25 participating NHS organisations which may have affected motivation amongst frontline staff. 26 27 Conclusions: Collaboratives maybe an important way of engaging practitioners in quality 28 improvement initiatives. This study highlights that despite a challenging healthcare environment in 29 the UK, there remains motivation amongst individuals to participate in quality improvement 30 programmes as they recognise that improvement approaches may facilitate positive change in local 31 clinical processes and systems. Collaboratives can harness this individual motivation to facilitate 32 spread and adoption of improvement methodology and build engagement across their membership. http://bmjopen.bmj.com/ 33 34 35 36 STRENGTHS AND LIMITATIONS OF THIS STUDY 37 38 • The use of the in-residence model and a formative and qualitative approach to evaluation is 39 novel in the context of a Quality Improvement Collaborative. This approach develops our

40 understanding of the factors affecting motivation of clinicians and managers to engage with on September 29, 2021 by guest. Protected copyright. 41 quality improvement initiatives. 42 • A limitation of the study is that the researcher was only embedded in two hospital trusts, 43 raising questions of generalisability, nevertheless, we believe the concepts generated are 44 likely to be transferrable to the rest of the Improvement Collaborative and other similar 45 46 initiatives. 47 • Maintaining objectivity in participatory research can be challenging as the researcher’s 48 fellow team members are also participants in the evaluation. This was minimised by the 49 researcher regularly discussing findings with independent academic colleagues to obtain 50 different perspectives whilst also recognising the importance of including individuals familiar 51 with the programme, providing a context for findings and enabling learning through the 52 evaluation process. 53 54 55 56 57 58 2 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 3 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 BACKGROUND 4 Quality Improvement Collaboratives in healthcare bring together groups of health professionals, 5 managers and support staff either within an organisation or from several organisations, to work on a 6 common purpose, with the goal of improving health services.[1] Improvement Collaboratives have 7 8 become increasingly popular in the UK, with the NHS promoting them as a mechanism for change 9 across the healthcare system involving several types of organisations including GP surgeries, 10 hospitals and care homes.[2-4] Improvement Collaboratives frequently follow a Breakthrough Series 11 approach supporting organisations to close the gap between good care and usual practice in a short 12 time period (6-18 months).[5] During this time, teams meet to share learning on a topic (learning 13 sessions), to understand how to make improvements, to implement and test these improvements, 14 and to share their progress and results with the rest of the collaborative.[6] Collaboratives provide 15 an infrastructure for an inter-organisational support network from which members can address 16 For peer review only 17 common barriers and learn from others’ successes and challenges.[7] The success of Improvement 18 Collaboratives is determined by the influence of their inter-organisational (shared learning) and 19 intra-organisational features (culture, resources, leadership etc.).[8] 20 21 The evidence for the effectiveness of Improvement Collaboratives as interventions for improving 22 health outcomes is growing but is not yet compelling. Indeed, collaboratives may have a greater 23 impact on changes to professional behaviour and care processes than on care quality or health 24 service outcomes.[9] This is in part due to the heterogeneity of contexts within which collaboratives 25 operate.[10] Additionally, research studies have focussed on demonstrating ‘what’ impact an 26 Improvement Collaborative has had, overlooking ‘why,’ ‘how’ and ‘what works for whom in what 27 context,’ with relatively few qualitative evaluations that can provide a rationale for their impact on 28 29 outcomes.[11] 30 Studies have explored factors influencing clinician engagement in quality improvement but have not 31 considered the role of motivation to participate, especially within a challenging healthcare 32 http://bmjopen.bmj.com/ 33 environment with workforce problems such as low morale, increasing numbers of vacant clinical 34 posts and issues with staff retention.[12-14] Quality improvement initiatives often rely upon the 35 inherent motivation of practitioners to provide high quality care for their patients.[15] Carter et al 36 [16] proposed that concepts such as ‘collaborative advantage’ emerging within a collaborative may 37 affect motivation, instilling change through promoting competition amongst teams who may strive 38 to attain the same level of advancement of others. Mixed empirical evidence for the effectiveness of 39 Improvement Collaboratives coupled with minimal understanding of the motivation of practitioners 40 on September 29, 2021 by guest. Protected copyright. 41 to participate presents an important challenge to proponents of quality improvement approaches in 42 healthcare. 43

44 45 A participatory evaluation of a Quality Improvement Collaborative 46 47 Patient Safety Collaboratives 48 In 2014 NHS England commissioned 15 Patient Safety Collaboratives hosted by Academic Health 49 Science Networks to delivery safety improvement in response to Professor Don Berwick’s report A 50 Promise to Learn – a commitment to act.[17] This evaluation centres on an Academic Health Science 51 Network with a partnership of NHS organisations (hospital trusts) in south-east England, which 52 identified sepsis and acute kidney injury (AKI) as key priority areas for patient safety because they 53 54 were responsible for the greatest number of avoidable deaths in hospitalised patients.[18] Using a 55 Breakthrough Series approach, the Patient Safety Collaborative team aimed to improve patient 56 outcomes through improving clinical process measures to enable rapid detection and treatment, 57 58 3 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 4 of 15 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 using up to date and evidenced based guidelines as well as building improvement capacity and 4 capability in the participating hospital trust. As the host organisation, the Patient Safety 5 Collaborative Team was responsible for the day to day operation of the collaborative. 6 7 A participatory approach; the Researcher in Residence model 8 Participatory approaches have the potential to close the gap between research and practice in 9 Improvement Collaboratives by adopting a research-based approach to addressing the challenge of 10 motivation of the participating teams and the individuals within them. Participatory approaches 11 involve partnership with stakeholders to solve practical problems and a sustained commitment by 12 13 researchers to continually collaborate.[19] The in-residence model is an emerging model of 14 participatory research which embraces the concept of ‘co-creating’ knowledge between researchers 15 and practitioners, using a range of approaches.[20] In this study, the researcher (ML) was embedded 16 in the collaborative,For acting aspeer an interface betweenreview the emerging only evidence from the evaluation and 17 its application to collaborative processes, co-creating knowledge through participation, with 18 research expertise communicated to and negotiated with the collaborative participants. The 19 participatory approach was enhanced by the formation of an evaluation steering committee 20 21 comprising the Patient Safety Collaborative team members, academic colleagues and clinical leads 22 from hospital trusts in the collaborative. This committee co-designed and co-interpreted findings 23 from the study, generating evidence that could potentially optimise motivation within the 24 collaborative. This study explores the barriers and facilitators to motivation that may affect 25 participation in an Improvement Collaborative using an illustrative case study, and describes a 26 practical approach to participatory evaluation. 27 28 29 30 METHODS 31 Subjects and setting 32 http://bmjopen.bmj.com/ The collaborative ran from September 2016 to June 2017 with the evaluation undertaken between 33 34 January 2016 to April 2017. Three teams participated in the evaluation; two were represented by 35 patient safety teams from hospital trusts from the total collaborative membership of 23 teams and 36 the third was the Patient Safety Collaborative programme team. The two teams representing the 37 hospital trusts were purposively selected through discussions with the evaluation steering 38 committee, based on their perceived contrasting maturity in terms of quality improvement 39 capability and capacity. One team was well-resourced with a specific department dedicated to 40 on September 29, 2021 by guest. Protected copyright. quality improvement in its hospital, whilst the second was viewed as similar to all other teams in the 41 42 collaborative (limited resources for patient safety and minimal improvement expertise). 43 Overall, 15 individual semi-structured interviews were conducted. An initial 13 interviews were held 44 with team members of the two hospital trusts and the Patient Safety Collaborative team who were 45 46 purposively selected based on their role in patient safety within their organisation. Using snowball 47 sampling, a further two participants from the other collaborative teams were identified and 48 interviewed. [21] Interview participants included 6 senior and middle managers, 3 doctors (2 49 consultants and 1 registrar), 5 senior nurses (NHS band 7 and above) and 1 junior nurse (NHS band 50 5). There was no consistent patient representation in the teams and hence this group were not 51 included in this study. As a result of the in-residence approach, all participants were known to the 52 researcher and were aware of the purpose and aims of the study. Interview participants were 53 54 approached in person or by email and there were no refusals to participate. Written informed 55 consent was obtained from each participant prior to interview. 56 57 58 4 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 5 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 Study design 4 The evaluation was undertaken in a series of iterative stages of participation; data collection, 5 analysis, interpretation and dissemination of emerging findings, with the application of evidence to 6 influence the development of the programme. Qualitative methods were used to generate and 7 analyse the data. 8 9 Data collection 10 Participant observation was undertaken by ML and involved attendance at trust and Patient Safety 11 12 Collaborative meetings as well as collaborative learning sessions, totalling approximately 100 hours. 13 As the evaluation progressed the researcher became an active participant in meetings, providing 14 input and facilitating discussion. Field notes at meetings were recorded. Pertinent points arising 15 from these observations were communicated to the teams to facilitate discussion at future meetings 16 and to enable themFor to determine peer how to use review the findings in relation only to the progress of the 17 collaborative. 18 19 Interviews were conducted by ML, a researcher with experience of conducting health service 20 evaluations using qualitative methods. Interviews were held at the participant’s workplace in a 21 private meeting room and lasted between 45-60 minutes. Interviews were audio-recorded and 22 transcribed verbatim. No repeat interviews were carried out. The interview guide was based upon 23 24 the existing Improvement Collaborative literature and was informed in its development by 25 participant observation data and documentary review as well as input from members of the 26 evaluation committee. The guide was adapted iteratively using an inductive approach allowing the 27 exploration of new emergent themes. Interviews ceased once thematic saturation was reached. A 28 review of documents pertaining to the collaborative at trust level and those produced by the Patient 29 Safety Collaborative team such as meeting notes was also undertaken. Documents were carefully 30 scrutinised and emerging relevant themes were mapped to the thematic framework discussed 31 below. 32 http://bmjopen.bmj.com/ 33 Data analysis 34 Data was managed using NVivo version 11.0. ML conducted qualitative analysis using a thematic 35 36 framework approach to code the data and identify patterns and themes.[22] The framework was 37 developed iteratively to capture emerging themes from the data and was also informed by field 38 notes from participant observation. Components of the analysis plan such as the thematic 39 framework were discussed with evaluation committee members, which facilitated their

40 understanding of how their perceptions of the collaborative influenced their engagement with it. Co- on September 29, 2021 by guest. Protected copyright. 41 interpretation of emerging themes with the evaluation committee was also conducted and 42 alternative interpretations developing from these discussions were included in the analysis. 43 44 45 46 RESULTS 47 Two main categories of themes centred on the core concept of motivation emerged; 1) inherent 48 motivation - relating to personal drivers and 2) factors that influence motivation – inter and intra- 49 50 organisational features common to Improvement Collaboratives as well as external factors. 51 Inherent motivation (personal drivers) 52 53 Inherent motivation at the level of the individual was integral to participation in the collaborative. 54 Clinicians mentioned using the collaborative as an opportunity for career and professional 55 advancement. From collaborative learning sessions, it was observed that several of the patient 56 safety teams focussed on recruiting junior doctors, who were required to document evidence of 57 58 5 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 6 of 15 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 quality improvement activities in their career portfolios. Additionally, some of the interviewees 4 mentioned that witnessing the positive effect of improvement interventions provided a sense of 5 personal achievement which acted as an enabler in motivating individuals. 6 7 Motivation was also shaped by individual attitudes and perceptions such as the extent of 8 understanding of the purpose and value of quality improvement. Some clinicians suggested that 9 quality improvement was not a priority and was regarded as secondary to the need to deliver 10 routine care. Additionally, interviewees mentioned a degree of confusion amongst some of their 11 staff about their understanding of the definition of quality improvement. Some described quality 12 improvement as burdensome and unnecessary as it did not appear to directly benefit patients. 13 14 Interviewees also mentioned using improvement methodology to focus on marginal gains, ensuring 15 that new processes were beneficial to patient care with a preference for demonstrating small scale 16 changes at a wardFor level. peer review only 17 18 ‘It's about allowing people to try things out in a small environment…..your tests of change 19 through quality improvement are based on small scale steps of evidence, and if you can 20 change something for just one person that must be a good thing.’ Manager, partner trust 21 22 As the collaborative progressed, a few ‘champions’ identified by the Patient Safety Collaborative 23 team emerged. From learning sessions these champions were observed to be similar to change 24 agents, driving the patient safety agenda in their teams and departments. Their motivation to shape 25 the local level patient safety strategy and the sharing of this narrative with the collaborative led to 26 the spread and adoption of a champion’s model in other teams. Some champions emerged in trusts 27 where there were significant internal and external pressures. 28 29 ' And then organisations that aren’t necessarily in the good organisation space, they’ve really 30 risen to the challenge and become involved, partly thanks to these individuals.' Patient Safety 31 Collaborative team member 32 http://bmjopen.bmj.com/ 33 34 35 Factors affecting motivation 36 Intra-organisational features 37 The most frequently mentioned or observed intra-organisational features that were potential 38 barriers and facilitators to motivation were; conflicting priorities, engagement and clinical 39 ownership, and support and culture. 40 on September 29, 2021 by guest. Protected copyright. 41 Conflicting priorities 42 At the time of this programme, patient safety teams in trusts from across the collaborative were 43 allocating a significant proportion of their time to undertaking Commissioning for Quality and 44 Innovation (CQUIN) data collection to meet financially incentivised targets. Interviewees felt that 45 46 CQUIN detracted them from improving safety in the organisation, which ironically was the premise 47 of the project. 48 ‘….because our team spend so much time collecting data; reviewing patient records; trying to 49 50 find information, they aren’t using information to make a difference; through teaching; 51 training; coaching…… because, ultimately, CQUIN schemes earn this organisation about one 52 and a half million a year.’ Manager, partner trust 53 54 Engagement and clinical ownership 55 Engagement and clinical ownership were key barriers to motivation at trust level. Interviewees 56 mentioned insufficient support from some frontline clinicians, which was seen as limiting the 57 58 6 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 7 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 embedding of quality improvement processes and affecting the motivation of team members. The 4 lack of support was compounded by mainly senior staff undertaking improvement work with junior 5 nurses in particular, neither aware or involved. These issues were thought to reflect a broader 6 challenge of embedding an improvement culture within organisations. 7 8 ‘…more could be done to raise awareness among the junior nurses about quality 9 improvement. It seems to be something the consultants and senior nurses do. I think we 10 should all be doing it or at least be aware of it or how will the whole organisation improve?’ 11 Nurse, partner trust 12 13 In addition, one interviewee claimed that their organisation, despite previously having a good 14 reputation for safety improvement, was now prioritising efficiency to ensure compliance with 15 targets such as reducing Accident and Emergency waiting times. This efficiency drive was led by the 16 Chief Executive OfficerFor whose peer focus was on reviewthe operational aspects only of the everyday running of the 17 18 hospital. 19 ‘The organisation is so operational. People should be thinking a bit more strategically and 20 there should be discussions and planning as a joint effort. But actually the people at those 21 22 levels, including the Chief Executive, spend a lot of time running the operations of the 23 hospital.’ Senior Manager, partner trust 24

25 26 Support and culture 27 It was suggested that with fewer resources of time, financial support and staff with training in 28 quality improvement, individuals responsible for implementing improvement projects were less 29 30 motivated and that this may have affected their willingness to participate in the collaborative. 31 Moreover, medical directors in some trusts did not involve clinicians in discussions about joining the

32 collaborative and initially, some of these teams struggled to engage. Even so, as some interviewees http://bmjopen.bmj.com/ 33 mentioned, motivation could not be created, it had to exist within organisational culture. One of the 34 collaborative organisations had a self-professed culture of improvement formed through a top down 35 approach resulting in a directorate of quality and a well-resourced patient safety team. Yet, even in 36 this organisation, some expressed concerns about the extent of resource prioritisation for patient 37 safety. 38 39 Interviewer: ‘Why can't the funding gap be filled by the Board with resources from within the 40 on September 29, 2021 by guest. Protected copyright. organisation?’ 41 42 Participant: ’Absolutely could be. But it's about showing that there's some return on 43 investment. You know, Ophthalmology is a department that earns money. Maternity is a 44 department that earns money. Patient Safety isn't a department that earns money. We save 45 46 money. It's quite difficult.’ Senior manager, partner trust 47 48 49 Inter-organisational features 50 Interviewees were generally positive about participating in the collaborative identifying inter- 51 organisational learning and sharing as key motivating factors to ongoing participation. Through 52 observation of learning sessions, it was apparent that teams were proud of their achievements and 53 showcasing these successes was an important incentive. 54 55 56 57 58 7 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 8 of 15 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 ‘…we do not have many resources, but we have embraced learning and driven the patient 4 safety work in our trust. I am sure others can do the same and if they come to appreciate 5 that from seeing our achievements, that can only be a positive thing.’ Nurse, partner trust 6 7 Some interviewees suggested that the reputation of the collaborative was a significant reason for 8 trusts to join. The Patient Safety Collaborative team members described their perceived successes 9 and challenges relevant to ongoing motivation of the collaborative participants. Raising awareness 10 of AKI and sepsis and information sharing across the collaborative through the use of emails, social 11 media, webinars, teleconferences and learning sessions was viewed as a key success. Also 12 mentioned, was facilitating the understanding of the benefits of sustained quality improvement 13 14 approaches on health outcomes. 15 Sustaining engagement with the collaborative was viewed as a significant challenge and was thought 16 to be because of intra-organisationalFor peer issues reviewsuch as staff turnover only or dwindling resources. Patient 17 18 Safety Collaborative team members also identified two key learnings; 1) in some cases, they had not 19 fully explored and understood the factors affecting individual trust engagement and 2) they had not 20 assessed the readiness of some organisations to effectively participate in the collaborative. 21 Nonetheless, through observations it was apparent that even if reasons for less engagement were 22 known, the Patient Safety Collaborative team struggled to achieve consensus on how best to 23 support the relevant trust teams to overcome these factors, potentially affecting their motivation. 24 25 ‘The team are unsure of the best strategy to deal with less engaged trusts. There are 26 suggestions that discussions should be held with the MD who maybe able to facilitate 27 engagement. Others feel that this could be counter-productive and that teams should be 28 gently encouraged to participate and supported where possible to do so.’ ML field notes. 29 30 'It just highlights how you can’t just brainwash an MD at a meeting and then rely on them for 31 people to start coming and devolving time. So actually the Trusts that weren’t performing or 32 http://bmjopen.bmj.com/ 33 coming or engaging, I think that was a reflection of us.' Patient Safety Collaborative team 34 member 35 External factors 36 37 Two emerging sub-themes relating to the health system (context and system pressures) that may 38 have affected motivation were identified as external factors. The context within which the 39 collaborative operated was similar in all trusts. For example, imminent or recent inspections by the

40 national regulator, the Care Quality Commission (CQC) were frequently cited as a reason for joining on September 29, 2021 by guest. Protected copyright. 41 the collaborative. There was also heightened awareness around sepsis due to a national campaign 42 advocating for improving its recognition and treatment and significant changes to National Institute 43 of Clinical Excellence guidelines.[23 24] Such high profile developments coupled with increased 44 media scrutiny of several avoidable deaths,[25] were seen by interviewees as potential drivers for 45 46 participation in the collaborative. Relevant to health system pressures, interviewees from the trusts 47 expressed an ambivalence to minimal resource provision for patient safety, acknowledging that 48 working within tight parameters was the norm within the context of the current healthcare 49 environment in the UK, yet, participating in quality improvement was still possible. 50 51 ‘You have to look at what resources you've got, and then how you can do what you need to 52 within that….in this climate, and in this organisation, I think we should be realistic, we have 53 to work differently to achieve outcomes.’ Doctor, partner trust 54 55 56 57 58 8 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 9 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 DISCUSSION 4 The study has revealed three principal findings that provide important lessons for quality 5 improvement programmes in the NHS. Firstly, individuals can act as change agents, driving the 6 quality improvement agenda at a local level. Secondly, inadequate board (medical director) 7 8 engagement at trust level may affect motivation amongst team members. Finally, Improvement 9 Collaboratives may have a key role in the UK healthcare system, as they promote shared learning 10 and the formation of networks that are established on the premise of mutual experience and a 11 common purpose. Following the completion of this collaborative programme, a deteriorating patient 12 community of practice was formed as well as a focus on paediatric sepsis across the partner trusts in 13 London. This is one such example of the potential legacy of Improvement Collaboratives. 14 15 A strength of this study was that the participatory approach to the evaluation made a positive 16 contribution to theFor progression peer of the collaborative review in terms of: theonly provision of operational support; 17 mobilisation of current knowledge (from the academic literature) as well as sharing newly generated 18 knowledge to assist the programme in meeting its objectives; and the sharing of information to 19 connect different components of the programme. An in-depth focus on two trust teams was a trade- 20 21 off for more generalisable findings. Nonetheless, one of these teams was representative of most of 22 the collaborative members in terms of its intra-organisational features. Moreover, the in-depth 23 participatory approach enabled the researcher to generate findings and insights on the motivation 24 of individuals within a patient safety team which may be transferable to similar settings in NHS 25 trusts. An additional limitation was that of the in-residence model whereby sometimes it was 26 challenging for the researcher to maintain objectivity with fellow team members as participants in 27 the evaluation. This was minimised by the researcher regularly discussing findings with independent 28 29 academic colleagues. Furthermore, participatory approaches acknowledge the importance of 30 including individuals familiar with a programme or service, providing a context for findings and 31 enabling learning through the process of evaluation.[26]

32 http://bmjopen.bmj.com/ 33 Most studies in this field have used a summative approach to evaluate the effectiveness of an 34 Improvement Collaborative as an intervention to improve patient outcomes. In contrast, this study 35 provides a novel approach, using a qualitative and formative evaluation with a researcher embedded 36 an Improvement Collaborative. This approach has revealed some key aspects of the ‘black box’ that 37 exists in terms of understanding of ‘how’ and ‘why’ a collaborative is effective through focussing on 38 motivation at the level of the individual and organisation. Yet, it has not established the effect of 39 motivation within this Patient Safety Collaborative on patient outcomes for AKI and/or sepsis. 40 on September 29, 2021 by guest. Protected copyright. 41 Three key aspects of embedded research identified from a recent review of the role of embedded 42 research in quality improvement programmes [27] have been identified as relevant to this 43 evaluation; (1) the researcher became a key member of the Patient Safety Collaborative team 44 45 playing an operational and evaluative role, (2) important working relationships with staff were 46 developed and (3) knowledge was co-produced through partnership working with the evaluation 47 committee. These aspects were especially pertinent to an embedded researcher working as a 48 member of an organisation in an Improvement Collaborative whereby the implementation of the 49 improvement methodology and its perceived impact required ‘real-time’ feedback from 50 collaborative members to enable the host team to adapt and improve its processes, directed toward 51 encouraging the teams to learn and share from each other. In this study, we have highlighted some 52 53 of the experiences of an in-residence researcher in a service or programme that have also been 54 described elsewhere.[28] However, it is challenging to fully assess and attribute change to the role of 55 the researcher on the progress of this collaborative and its outcomes. There is scope for further 56 exploration of this aspect in future studies using the Researcher in Residence model. 57 58 9 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 10 of 15 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 This study has important implications for clinicians and managers planning and implementing quality 4 improvement programmes in NHS organisations as well for those agencies involved in hosting such 5 initiatives. At the individual level, career advancement and personal achievement provide 6 organisations with a focus for incentivising clinician participation in quality improvement as is the 7 case with junior doctors in this evaluation whose recruitment to such initiatives may enhance 8 organisational systems and develop medical professionalism.[29] The individual perception of the 9 10 term ‘quality improvement’ also influenced motivation and caused confusion amongst some, acting 11 as a potential barrier to motivation. The evaluation suggests that reframing quality improvement for 12 clinicians, as a series of marginal gains, whereby the immediate discernible and small-scale benefits 13 are perceived as acceptable, may encourage participation in improvement programmes. It is rare to 14 achieve the dramatic scale of improvement seen in the much cited ‘Matching Michigan’ study.[30] 15 16 The findings also highlightFor important peer external review factors (over which only the collaborative teams had little 17 control) that influence motivation and the ability of teams to engage, potentially negatively affecting 18 other members of the collaborative network.[31] Research has suggested that existing 19 organisational culture can be superseded by sufficiently empowered managers and clinicians, similar 20 to the champions of this collaborative, who emerged from trusts where an improvement culture was 21 22 not apparent.[32] Armenakis et al suggest that to institutionalise permanent change, change agents 23 require credibility within their organisation.[33] In this collaborative, ‘champions’ were senior clinical 24 staff who had demonstrated a desire to lead an improvement programme. The Patient Safety 25 Collaborative team successfully harnessed the inherent motivation of these individuals resulting in 26 spread and adoption of several ‘champions’ inspired approaches to patient safety across the 27 collaborative. 28 29 The ‘all share, all learn’ approach of the Patient Safety Collaborative team has revealed some of the 30 potential benefits of a non-hierarchical network, such as promoting togetherness amongst teams, 31 providing reassurance, trust and a common purpose [34], which is especially pertinent at a time 32 http://bmjopen.bmj.com/ when pervasive health system pressures are affecting workforce motivation.[35] The study findings 33 34 suggest that the collaborative was integral in raising awareness and facilitating the improvement of 35 local level clinical processes for AKI and sepsis. The reputation of the Patient Safety Collaborative 36 host organisation was identified as a means of gaining recognition with trust boards, the CQC and 37 other clinicians. Hence, there is a potential gap for Improvement Collaboratives to bridge in the 38 healthcare system in England, in building capability and capacity and motivating individuals to 39 participate in quality improvement, especially in organisations that have neither the resource or 40 volition. on September 29, 2021 by guest. Protected copyright. 41 42 The NHS trusts in this Improvement Collaborative demonstrated different levels of readiness despite 43 facing similar challenges relating to their intra-organisational features. This was in part associated 44 with the engagement of trust boards, which demonstrated an initial interest in participating in the 45 collaborative by signing a contract agreement with the host organisation, but, in some cases this 46 47 failed to develop into a sustained commitment and may have been overtaken by other priorities. 48 Using a bottom up approach, involving frontline clinicians from the outset, may facilitate the 49 continued engagement of organisations in an Improvement Collaborative. Additionally, other intra- 50 organisational features such as a lack of support and resources are prominent in this evaluation and 51 affect motivation. Nevertheless, some interviewees mentioned that carrying out quality 52 improvement work in addition to providing routine care within existing resource parameters was 53 acceptable and achievable. A recent King’s fund report suggested that trusts and individuals should 54 55 be pursing approaches to continuously improve the quality of care despite a lack of resources.[36] 56 57 58 10 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 11 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 The host organisation in this Improvement Collaborative is one of several Academic Health Science 4 Networks across England involved in building improvement capacity and capability within their 5 partnership of NHS trusts. This raises a question of whether it is the responsibility of individual NHS 6 organisations themselves to allocate resources for improvement initiatives or whether it is the role 7 of an external organisation. The host organisation’s role is to develop programme aims, disease 8 specific measures and tools as well as to facilitate and provide technical support but in this study, 9 10 they acted as a proxy quality improvement team for some of the NHS organisations. This is a 11 significant challenge for external organisations as they try to find a role within a healthcare system 12 whilst acting as enablers for their partners. This study should initiate further discussion and 13 examination of the role of organisations such as Academic Health Science Networks that may 14 provide significant support for quality improvement to individual NHS trusts. 15 16 For peer review only 17 18 FUNDING STATEMENT 19 This work was supported by NHSEngland, grant number 533957. NHSEngland provided the funds to 20 UCLPartners to host a Patient Safety Collaborative of which some were allocated to undertaking this 21 evaluation. 22 23 24 25 COMPETING INTERESTS 26 No competing interests are declared. 27 28 29 30 ETHICS APPROVAL 31 Ethics approval was obtained from the University College London research ethics committee.

32 Reference number 16/0007. http://bmjopen.bmj.com/ 33 34 35 36 AUTHOR CONTRIBUTION 37 ML, MM, KH and MS conceived and planned the overall study. ML, MM and MS wrote the initial 38 study protocol. ML, MM, KH, MS, CL, JW, LE and PT co-designed aspects of the study including the 39 strategy for sampling of the two trusts for participation in the evaluation and reviewing tools for

40 data collection. ML undertook recruitment, data collection and analysis. ML, MM, KH, MS, CL, JW, on September 29, 2021 by guest. Protected copyright. 41 TW, SS, PT and LE formed the overall membership of the evaluation steering committee. All authors 42 co-interpreted the study findings. ML drafted the manuscript and revised it in response to comments 43 44 from all co-authors. All authors read and approved the final version of the manuscript. 45 46 47 ACKNOWLEDGEMENTS 48 The authors would like to thank all of the study participants and in particular the two trusts that 49 allowed the researcher to be embedded within their patient safety teams as well as the contribution 50 of several of the Patient Safety Collaborative team members through the course of the evaluation. 51 52 The authors would also like to acknowledge the contribution of Professor Roger Jones who provided 53 guidance on some aspects of the structure and framing of the manuscript. 54 55 56 57 58 11 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 12 of 15 BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 4 DATA SHARING STATEMENT 5 6 No additional data are available. 7 8 9 REFERENCES 10 1. Øvretveit J, Bate P, Cleary P, et al. Quality collaboratives: lessons from research. Quality and safety 11 12 in health care 2002;11(4):345-51 13 2. Power M, Tyrrell PJ, Rudd AG, et al. Did a quality improvement collaborative make stroke care 14 better? A cluster randomised trial. Implementation science 2014;9 doi: 10.1186/1748-5908- 15 9-40[published Online First: Epub Date]|. 16 3. Dawda P, JenkinsFor R, Varnam peer R. Quality improvement review in general only practice: The King's Fund, 2010. 17 4. Working together to make all our lives better: Collaborative care in residential homes: The Health 18 Foundation, 2017. 19 5. Power M, Tyrrell PJ, Rudd AG, et al. Did a quality improvement collaborative make stroke care 20 better? A cluster randomized trial. Implement Sci 2014;9:40 21 6. The Breakthrough Series: IHI's collaborative model for achieving breakthrough improvement. 22 Innovation Series: Institute for Healthcare Improvement, 2003. 23 7. Nadeem E, Olin SS, Hill LC, et al. Understanding the components of quality improvement 24 collaboratives: a systematic literature review. Milbank Quarterly 2013;91(2):354-94 25 8. Nembhard IM. Learning and Improving in Quality Improvement Collaboratives: Which 26 Collaborative Features Do Participants Value Most? Health Services Research 2009;44(2 Pt 27 1):359-78 doi: 10.1111/j.1475-6773.2008.00923.x[published Online First: Epub Date]|. 28 9. De Silva D. Improvement collaboratives in health care. London, UK: The Health Foundation 2014. 29 10. Schouten LM, Hulscher ME, van Everdingen JJ, et al. Evidence for the impact of quality 30 improvement collaboratives: systematic review. Bmj 2008;336(7659):1491-94 31 11. Howe C, Randall K, Chalkley S, et al. Supporting improvement in a quality collaborative. Br J 32 http://bmjopen.bmj.com/ 33 Healthc Manag 2013;19:434-42 34 12. Ling T. How Do You Get Clinicians Involved in Quality Improvement?: An Evaluation of the Health 35 Foundation's Engaging with Quality Initiative: A Programme of Work to Support Clinicians to 36 Drive Forward Quality: Final Report: The Health Foundation, 2010. 37 13. Davies H, Powell A, Rushmer R. Why don't clinicians engage with quality improvement? Journal 38 of Health Services Research & Policy 2007;12(3):129-30 doi: 39 10.1258/135581907781543139[published Online First: Epub Date]|.

40 14. Iacobucci G. Future gaps in workforce pose an impending crisis for the NHS, report warns: British on September 29, 2021 by guest. Protected copyright. 41 Medical Journal Publishing Group, 2013. 42 15. Dixon-Woods M, McNicol S, Martin GP. Ten challenges in improving quality in health care: 43 lessons from the health foundation’s programme evaluations and relevant literature. BMJ 44 Quality & Safety 2012;21 doi: 10.1136/bmjqs-2011-000760[published Online First: Epub 45 Date]|. 46 16. Carter P, Ozieranski P, McNicol S, et al. How collaborative are quality improvement 47 collaboratives: a qualitative study in stroke care. Implementation Science 2014;9(1):1-11 doi: 48 10.1186/1748-5908-9-32[published Online First: Epub Date]|. 49 17. Berwick D. A Promise to Learn – a Commitment to Act: Improving the Safety of Patients in 50 England. London, England: NHS, 2013. 51 18. Patient Safety: The UCLPartners Patient Safety Programme. Secondary Patient Safety: The 52 53 UCLPartners Patient Safety Programme 2017. http://uclpartners.com/what-we-do/patient- 54 safety/. 55 56 57 58 12 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 13 of 15 BMJ Open BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 2 3 19. Eyre L, Farrelly M, Marshall M. What can a participatory approach to evaluation contribute to the 4 field of integrated care? BMJ Qual Saf 2016 doi: 10.1136/bmjqs-2016-005777[published 5 Online First: Epub Date]|. 6 20. Marshall M, Pagel C, French C, et al. Moving improvement research closer to practice: the 7 Researcher-in-Residence model. BMJ quality & safety 2014:bmjqs-2013-002779 8 21. Atkinson R, Flint J. Social Research Update. 9 22. Gale NK, Heath G, Cameron E, et al. Using the framework method for the analysis of qualitative 10 data in multi-disciplinary health research. BMC Med Res Methodol 2013;13:117 doi: 11 10.1186/1471-2288-13-117[published Online First: Epub Date]|. 12 23. Freitag A, Constanti M, O'flynn N, et al. Suspected sepsis: summary of NICE guidance. BMJ: 13 British Medical Journal 2016;354 14 24. Wise J. Sepsis should be treated within one hour, says NICE. BMJ 2017;356 doi: 15 10.1136/bmj.j1257[published Online First: Epub Date]|. 16 25. Rawlinson K. SpottingFor sepsis peer 'could prevent review 37,000 deaths a year.only The Guardian 2016 15th 17 December 2016. 18 19 26. Griffin S, Glover SH, Williams AW, et al. Participatory Evaluation of Community-Based HPV and 20 Cervical Cancer Prevention and Control Efforts. Journal of the South Carolina Medical 21 Association (1975) 2009;105(7):309-17 22 27. Vindrola-Padros C, Pape T, Utley M, et al. The role of embedded research in quality 23 improvement: a narrative review. BMJ Qual Saf 2017;26(1):70-80 24 28. Marshall M, Eyre L, Lalani M, et al. Increasing the impact of health services research on service 25 improvement: the researcher-in-residence model. Journal of the Royal Society of Medicine 26 2016;109(6):220-25 27 29. Involving junior doctors in quality improvement London, UK: The Health Foundation 2011. 28 30. Bion J, Richardson A, Hibbert P, et al. ‘Matching Michigan’: a 2-year stepped interventional 29 programme to minimise central venous catheter-blood stream infections in intensive care 30 units in England. BMJ Quality & Safety 2012 doi: 10.1136/bmjqs-2012- 31 001325[published Online First: Epub Date]|. 32 31. Aveling EL, Martin G, Herbert G, et al. Optimising the community-based approach to healthcare http://bmjopen.bmj.com/ 33 improvement: Comparative case studies of the clinical community model in practice. Soc Sci 34 Med 2017;173:96-103 doi: 10.1016/j.socscimed.2016.11.026[published Online First: Epub 35 Date]|. 36 32. Phung VH, Essam N, Asghar Z, et al. Exploration of contextual factors in a successful quality 37 improvement collaborative in English ambulance services: cross-sectional survey. Journal of 38 39 evaluation in clinical practice 2016;22(1):77-85 doi: 10.1111/jep.12438[published Online First: Epub Date]|. 40 on September 29, 2021 by guest. Protected copyright. 41 33. Armenakis AA, Harris SG, Feild HS. Making change permanent A model for institutionalizing 42 change interventions. Research in organizational change and development: Emerald Group 43 Publishing Limited, 2000:97-128. 44 34. Health leaders' panel survey 6: footprints, financing and staff morale: Nuffield Trust, 2015. 45 35. Huxham C, Vangen S. Managing to collaborate: The theory and practice of collaborative 46 advantage. Abingdon: Routledge, 2013. 47 36. Limb M. Doctors must pursue quality improvement despite challenges. BMJ careers 2015 48 49 50 51 52 53 54 55 56 57 58 13 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml BMJ Open Page 14 of 15

COREQ (COnsolidated criteria for REporting Qualitative research) Checklist BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1

2 3 A checklist of items that should be included in reports of qualitative research. You must report the page number in your manuscript 4 where you consider each of the items listed in this checklist. If you have not included this information, either revise your manuscript 5 accordingly before submitting or note N/A. 6 7 Topic Item No. Guide Questions/Description Reported on 8 Page No. 9 10 Domain 1: Research team 11 and reﬂexivity 12 Personal characteristics 13 Interviewer/facilitator 1 Which author/s conducted the interview or focus group? 14 Credentials 2 What were the researcher’s credentials? E.g. PhD, MD 15 16 Occupation For3 Whatpeer was their occupationreview at the time only of the study? 17 Gender 4 Was the researcher male or female? 18 Experience and training 5 What experience or training did the researcher have? 19 Relationship with 20 participants 21 22 Relationship established 6 Was a relationship established prior to study commencement? 23 Participant knowledge of 7 What did the participants know about the researcher? e.g. personal 24 the interviewer goals, reasons for doing the research 25 Interviewer characteristics 8 What characteristics were reported about the inter viewer/facilitator? 26 e.g. Bias, assumptions, reasons and interests in the research topic 27 28 Domain 2: Study design 29 Theoretical framework 30 Methodological orientation 9 What methodological orientation was stated to underpin the study? e.g. 31 and Theory grounded theory, discourse analysis, ethnography, phenomenology, 32 http://bmjopen.bmj.com/ content analysis 33 34 Participant selection 35 Sampling 10 How were participants selected? e.g. purposive, convenience, 36 consecutive, snowball 37 Method of approach 11 How were participants approached? e.g. face-to-face, telephone, mail, 38 email 39

40 Sample size 12 How many participants were in the study? on September 29, 2021 by guest. Protected copyright. 41 Non-participation 13 How many people refused to participate or dropped out? Reasons? 42 Setting 43 Setting of data collection 14 Where was the data collected? e.g. home, clinic, workplace 44 Presence of non- 15 Was anyone else present besides the participants and researchers? 45 46 participants 47 Description of sample 16 What are the important characteristics of the sample? e.g. demographic 48 data, date 49 Data collection 50 Interview guide 17 Were questions, prompts, guides provided by the authors? Was it pilot 51 52 tested? 53 Repeat interviews 18 Were repeat inter views carried out? If yes, how many? 54 Audio/visual recording 19 Did the research use audio or visual recording to collect the data? 55 Field notes 20 Were ﬁeld notes made during and/or after the inter view or focus group? 56 Duration 21 What was the duration of the inter views or focus group? 57 58 Data saturation 22 Was data saturation discussed? 59 Transcripts returned 23 Were transcripts returned to participants for comment and/or 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml Page 15 of 15 BMJ Open

Topic Item No. Guide Questions/Description Reported on BMJ Open: first published as 10.1136/bmjopen-2017-020930 on 7 April 2018. Downloaded from 1 Page No. 2 3 correction? 4 Domain 3: analysis and 5 findings 6 Data analysis 7 Number of data coders 24 How many data coders coded the data? 8 9 Description of the coding 25 Did authors provide a description of the coding tree? 10 tree 11 Derivation of themes 26 Were themes identified in advance or derived from the data? 12 Software 27 What software, if applicable, was used to manage the data? 13 Participant checking 28 Did participants provide feedback on the findings? 14 15 Reporting 16 Quotations presented For29 Werepeer participant review quotations presented only to illustrate the themes/findings? 17 Was each quotation identified? e.g. participant number 18 Data and findings consistent 30 Was there consistency between the data presented and the findings? 19 Clarity of major themes 31 Were major themes clearly presented in the findings? 20 21 Clarity of minor themes 32 Is there a description of diverse cases or discussion of minor themes? 22 23 Developed from: Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist 24 for interviews and focus groups. International Journal for Quality in Health Care. 2007. Volume 19, Number 6: pp. 349 – 357 25

26 27 Once you have completed this checklist, please save a copy and upload it as part of your submission. DO NOT include this 28 checklist as part of the main manuscript document. It must be uploaded as a separate file. 29 30 31

32 http://bmjopen.bmj.com/ 33 34 35 36 37 38 39

40 on September 29, 2021 by guest. Protected copyright. 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml