Local Ownership Opportunity and Risk Diagnosis

NPfIT NLOP Programme NLOP Document Record ID Key Sub- LOORD Programme Prog. Director David Flory Status Final Owner Carol Clarke Version V1.01 Author Linda Lloyd, Version 18 June 2008 Project Lead Date & Andrew Luce

Local Ownership Opportunity and Risk Diagnosis (LOORD) “supporting the NHS to deliver better quality and safer services using IT”

Local Ownership Opportunity and Risk Diagnosis Final Report

Product: NLOP Version: 1.01

Amendment History:

Version Date Amendment History V.0.01 First Draft for Review with Linda Lloyd and John Willshere V0.02 Second Draft for Review to Linda Lloyd, John Willshere, Carol Clarke, Peter Magee and Paul Mansell V0.03 5 May 2008 Third Draft incorporating Linda Lloyd material for Review to Linda Lloyd, John Willshere, Carol Clarke, Peter Magee and Paul Mansell V0.04 6 May 2008 Updated Graphs and added table of figures V0.05 6 May 2008 Updated Graphs and table of figures V0.06 6 May 2008 Updated Graphs and added table of figures V0.07 7 May 2008 Revised document following review 6 May V0.08 7 May 2008 Main Report and Annexes separated into two documents, recommendations restructured V0.09 10 May 2008 Text modified and Executive Summary written Linda Lloyd & Jana Dale V0.10 13 May 2008 Slides Typing Errors Modified V0.11 14 May 2008 Altered to incorporate modifications requested by Linda Lloyd V0.12 20 May 2008 Incorporates Linda Lloyd and Jana Dale inputs V0.13 23 May 2008 Presentation changes following a discussion with Bob Alexander V1.00 2 June 2008 Final Report incorporating comments from Carol Clarke V1.01 6 June 2008 Case studies added by Linda Lloyd to Final Released Version Reviewers: This document must be reviewed by the following;

Name Signature Title / Responsibility Date Version Carol Clarke Director of Stakeholder June 2nd, 2008 V 1.00 Engagement & CRS Paul Mansell Director, Moorhouse June 6th, 2008 V 1.01 Consulting Approvals: This document must be approved by the following:

Name Signature Title / Responsibility Date Version Carol Clarke Director of Stakeholder June 2nd, 2008 V1.00 Engagement & CRS

Distribution: Approvers, reviewers Document Status: This is a controlled document. Whilst this document may be printed, the electronic version maintained in FileCM is the controlled copy. Any printed copies of the document are not controlled.

Table of Contents: 1 Executive Summary 3 1.1 Key Messages 3 1.2 Background 3 1.3 Purpose 3 1.4 Principles of NLOP Transition Impacting Local Ownership 3 1.5 Method 3 1.6 Report Structure 3 1.7 Summary Findings from the Trusts 3 1.8 Progress on Risks Identified within TAR 3 1.9 Correlation with Health Informatics Findings and Summary Care Record Early Adopters Evaluation 3 1.10 Conclusions 3 1.11 Main Recommendations 3 2 Method and Approach 3 2.1 Methodology 3 3 Context 3 3.1 Current Deployment Status by Local PfIT 3 3.2 London Programme for IT Update 3 3.2 Southern Programme for IT update 3 3.3 North Midland and East Programme for IT Update 3 4 Scores and Analysis 3 4.1 Discussion summary - principal observations and analysis 3 4.2 Overall Confidence Ratings 3 4.3 Confidence Ratings By Trust Category 3 5 Dimension Analysis 3 5.1 Governance 3 5.2 Planning 3 5.3 Capacity and Capability 3

5.4 Performance Management 3 5.5 Finance 3 5.6 Deployment 3 5.7 Service Implementation 3 5.8 Service Management 3 5.9 Commercial 3 5.10 Clinical 3 5.11 Benefits 3 6 Conclusion and Recommendations 3 6.1 The Local Level 3 6.2 The SHA Level 3 6.3 Recommendations and Observations: Local Programme Level 3 6.4 The National Level 3

1 Executive Summary

1.1 Key Messages There were a number of common themes running through all the project dimensions as well as the types of organisations that participated in the project:

. Awareness of NLOP – all the organisations were aware of the National Programme for IT (NPfIT) Local Ownership Project (NLOP), but the extent of understanding of what it meant in practice varied. The penetration of NLOP principles below the SHA level remains subject to local variations and interpretation. The majority of risks identified in the TAR report remain. . Concerns about credibility and low confidence – currently acute secondary care CRS products are seen as very weak, and the LSPs’ ability to deliver to timescales on these has not been credible. Similarly there are serious doubts about the deployment approach with many parallel or only slightly staggered projects. Trusts want to see clear, successful demonstrators before committing to the Programme. Conversely some MH and community products have been implemented very successfully, with much benefit gained and more to come as some organisations belatedly redesign care and exploit benefits across Trust boundaries. . Insufficient Communication is a common theme running through all aspects and dimensions of this review. This issue may have been expressed in a number of different ways, e.g. lack of transparency, lack of clarity, being secretive and similar. The comments often referred to areas where there appeared to be no commercial or other internal reasons for not making information available to the Trusts. . Commitment to the vision – all Trusts interviewed expressed their continued commitment to the vision of patient-centred life long electronic health record shared across local health communities (LHCs). This commitment was re-stated despite local frustrations with lack of progress with systems implementation, business change and benefits delivery to date. . Lead Role of PCTs – the leadership role PCTs perform in local health communities in terms of the development of IM&T strategy and planning appears to be directly related to the maturity and stability of the NHS organisations and their working relationships. Although all the PCTs were still consolidating following the last re-organisation, the CEOs were consciously developing their SRO roles and integrating IM&T into the business planning cycle. Relationships with local foundations trusts were not presenting any specific problems. The positive developments to date indicate that local health communities offer the best chance of success for the implementation and exploitation of IM&T. . Capability and capacity – the majority of the Trusts have made some progress in developing their capability to use IM&T successfully. The largest gaps are in the area of mainstreaming, business change and benefits management. These are also identified as areas where support would be most useful. The Trusts also feel that more advantage should be taken of pooling resource and creating resource mobility.

1.2 Background The National Programme for IT (NPfIT) Local Ownership Project was implemented in the middle of 2007. It was primarily focused on transferring accountability and responsibility for the planning and delivery of NPfIT products and solutions from NHS CFH and as such had the following objectives: . strengthen local governance and ownership, so that the SHAs and PCTs are enabled to drive the LPfIT in an appropriate direction that achieves the right balance between national imperatives and local needs . reinforce the value and benefits that can be derived from NPfIT . implement the recommendations of the NAO Report in 2006

. follow the direction provided by Sir Ian Carruthers’ letter (‘NPfIT – Responsibilities and Information Governance’ dated 21 Aug 2006) . implement the recommendations of the NHS Management Board.

NLOP Transition Assurance Review (TAR) was carried out in 2007and considered the transfer of ownership from NHS Connecting for Health to the ten SHAs. The review reported ten risks. These were based on self assessed scores obtained from the ten SHAs and follow-up interviews. The review did not seek to go below the SHA level. The LOORD project was commissioned in February 2008 by the Director of Finance, Performance and Operations on behalf of the NHS Management Board to obtain an evidenced base view of the extent to which NLOP has penetrated down from the SHA level into the wider NHS at the local level and how risks are perceived by NHS organisations (Trusts)1.

1.3 Purpose The objectives of the LOORD project have been to:

. measure the degree to which the principles of Local Ownership of the Local Programmes for IT have penetrated beyond the SHA level to individual NHS organisations . establish an evidenced based understanding of the degree of confidence Trusts have in the current arrangements . identify areas of good practice that could potentially be replicated more widely . identify areas where central support might be provided at the national, SHA or local level to help resolve the issues identified through the project.

1.4 Principles of NLOP Transition Impacting Local Ownership Amongst the objectives that NLOP transition was seeking to achieve, the following are relevant at local level:

1 For the purposes of this document the word Trust is used to cover all four types of NHS front line organisations that in the project – Primary, Acute, Mental Health and Ambulance Trusts. None of the Care Trusts took part in the project.

Relevant NLOP Principle LOORD Team Commentary

Safe transition: NLOP transition does not seem to have worked fully. However it to 'pressure-test' NLOP transition against the then is necessary to identify the extent current risk profile of the NPfIT implementation of real, substantive risk to the programme programme and to differentiate risks from issues. The identified risks are: Financial – costs are duplicated and prolonged as roll-outs slip behind published schedules and Trust have to bear programme costs Resource consumption – An increased demand for resources in excess of initial estimates which diverts important and scare resources away from other important initiatives. Clinical – weak products and governance arrangements could combine to create clinical risk and loss of commitment leading to impaired benefits realisation

Core Functions: The majority of Trusts are not satisfied in this area – they do not feel that they design the to focus on the core functions of Deployment and solution but merely configure it. The first Design, Build and Test (DBT) engagement event is to talk about deployment not design and they dislike the “one size fits all” approach.

Ownership:to shift ownership to Results are uneven but there are pockets of good the SHAs with the expectation practice that are worth investigating to see if they that they then establish clear can be replicated more widely. ownership and accountabilities at the local level

Senior Stakeholders: This list does not include Local SROs and this is consistent with the Trusts’ issue that LSPs and to provide assurance to the NHS CfH do not see them as the customer senior stakeholders: SHA CEOs, NLOP Regional SROs, NHS CEO, NHS CFH

Risk Mitigation: Risks remain substantially unresolved - see section 1.8 below to identify, quantify, and qualify the risks to successful programme delivery caused as a result of the NLOP transition

1.5 Method The project team collected information and interviewed senior executives, very often including the CEO, from a representative sample of 37 NHS Trusts. Trusts scored themselves against a set of confidence statements covering the following dimensions:

. Governance . Planning . Capability and Capacity . Performance Management . Deployment . Service Implementation . Service Management . Commercial . Clinical . Benefits

Based on these scores, the team facilitated peer challenge sessions of comparable Trusts or where this was not possible interviewed the NHS organisations individually. The project field work was carried out between 12 March and 1 May 2008

1.6 Report Structure The report presents findings from the self assessments and subsequent interviews. The report then draws conclusions and makes high level recommendations to be considered by the Project sponsor and, if appropriate, by the NHS Management Board. It will be shared with SHA CEOs in their SRO capacity under NLOP. SHA CIOs have committed to disseminating and supplementing these recommendations with their own early action plans, developed for local dialogue. In addition, mapping with the recommendations from the Health Informatics Review is to be carried out.

1.7 Summary Findings from the Trusts The primary focus of the project was on the effects of NLOP transition. The Trusts’ focus, however, has moved on to the longer term and they are concerned about how they implement and transition to ‘business as usual’. The findings reflect this position and are summarised here against the four project objectives.

Objective 1 Measure the degree to which the principles of Local Ownership of the Local PIT programmes have penetrated beyond the SHA level to individual NHS organisations

. Perceptions as to the effectiveness of NLOP transition vary both within and between the Local Programmes for IT. There is still a need for greater clarity as there is not yet a consistent understanding amongst Trusts about how arrangements have changed post-NLOP and how these changes affect them. . Some Trusts are unaware of any resource shift from the centre to the front line. . The Trusts feel that neither the LSPs nor NHS CFH consider or treat them as the customer.

Objective 2

Establish an evidenced based understanding of the degree of confidence Trusts have in the current arrangements

. The Trusts feel that weaknesses of acute CRS products combined with confusion about governance could combine into a substantial clinical risk. . The Trusts feel very strongly that they lack a role in the contract and any levers to control the delivery of the contract both during deployment and in live running; local influence and control over deployment and service management is lacking). . Many Trusts did not treat the National Programme deployments as part of their mainstream project portfolio leading to unclear investment objectives and weak benefits management. . The adoption of a mass roll-out approach is seen as deeply flawed as the products need a large degree of development. Trusts believe that this approach should be abandoned until such time when the solutions are stable and can be rolled out across multiple parallel sites. . The delays occurring as a result of remedial work and fixes to the products are resulting in ever shifting timescales and a lack of credibility of the DIP and the Development Plan. . Trusts challenge the timetable for rollouts as the capacity and capability of both the LSPs and the LHCs to support the proposed rhythm of delivery appear questionable, even if the CRS products were fit for purpose and ready for use.

Objective 3 Identify areas of good practice that could potentially be replicated more widely

. Some NHS organisations show markedly greater confidence in local arrangements and ownership than the average, particularly those where there was a cohesive local health community covering more than one PCT, the local Acute Trusts, Mental Health Trust and local authority, even independent contractors and the charitable sector.

Case study: Worcestershire PCT and the single Worcester County ICT Services (WHICTS) The PCT Chief Executive has been involved with the National Programme since its inception. He chairs the LHC Information and Communications Technology (ICT) Programme Board and is a member of the West Midlands Programme Board chaired by the SHA CEO. Creating and maintaining good governance both within ICT and between ICT and service organisations is seen as critical to achieving the Programme’s outcomes. A structure now exists (Project Boards through Organisational Programme Board to County ICT Programme Board) that streamlines decisions and direction at the right level throughout the strategic, programme and project life cycles. This also supports and is guided by reference groups (Primary Care, Clinical Reference Group, and Information Governance Forum) in the county, providing a critical path for service leaders to ensure alignment and influence the strategic and operational direction of travel of ICT Services and Projects, reducing ‘silo’ planning. Service Level Agreements (SLA) have been established between WHICTS and the three Trusts as well as between WHICTS and primary care general practitioners. The SLAs are formally agreed by the County ICT Programme Board. The LHC is also developing a portfolio management approach and expanding its pool of project managers who can be used both in IT and non-IT projects. The overall emphasis is therefore on a benefits led approach to planning and implementation that covers the whole business and IM&T agenda, not solely the National Programme solutions. Risk is managed at programme and project level. Risk logs are maintained at a project level using OGC evidence based risk tracking, allowing Project Board and assurance roles to have ready access to them. Significant risk is then taken through organisational risk registers, if appropriate.

. There are notable examples of effective working within the National Programme, where Trusts have negotiated specific arrangements to the solution and/or variations to the original contract; some Trusts have brokered local implementations of additional products/solutions (‘top-ups’) to bring forward interim solutions and to fund new requirements within that interim product  Two Mental Health Trusts, Tees, Esk & Weare Valley and Leeds, are receiving PARIS as an interim solution ahead of Lorenzo through the National Programme:

Case Study: Tees, Esk and Weare Valley Mental Health Trust The quest for a shared electronic patient record began in the Trust’s predecessor, with a MH services strategy which put electronic records at its heart, and generated an IT business case in 2003, as NPfIT started up. The Trust became an early CRS adopter for Mental Health. November 2005 saw their CRS work cease, due to lack of development of the product, despite the project running for a year, with 500+ users trained and being under the national spotlight as one of the first in mental health. As a result of good intelligence sharing, the North East Mental Health Forum made a convincing case and gained agreement with CFH to offer an alternative MH solution in Trusts with unsupported systems. Following lengthy multi party negotiations which survived a change in the main contractor from Accenture to CSC, and creation of a new, merged NHS Trust, contracts

between CFH, CSC, In4tek and Digica, were signed in December 2007. In terms of people, clinical leadership in the the trust fully supported the decisions and at times although frustrated by the delays, were focused on the real delivarbles and fully backed and support our work and stance with NHS CFH. It is hard to articulate how invaluable this was and how this gave us strength and confidence to continue down this route The Trust is now just weeks away from the first 600 users going live on the PARIS system. Over the next 9 months, in excess of 200 change workshops will be undertaken to ensure that the real benefits the system can facilitate are delivered, through enhancing our processes and ways of working, over 3000 users will be trained and the system will be available at over 100 sites. Only through aligning with the key business objectives, has the project stayed at the forefront of the organisations business. It has survived the test of time and change of organisations. Now the pressure is really on to make it deliver what it set out to - Improvements in care for patients and a modern work environment for staff’

 An Acute Trust that has introduced both an additional gateway prior to go-live and a three part payment to replace the single DVC payment. The impacts of this on payments to the supplier have been handled by NHS CFH.

Case Study: Taunton & Somerset NHS Foundation Trust In general, Trusts see the deployment process as too rigid and consequently perceive that they are placed under a lot of pressure to sign off milestones and stick to unachievable dates. However, Taunton and Somerset NHS Foundation Trust took ownership and control of the project and introduced strong local project management discipline. This resulted in diligent scrutiny of project milestones and insistence on good quality deliverables before allowing the project to progress to a next stage. Additional gateway reviews were introduced at key points e.g. a gateway review eight weeks before go-live meant that the Trust insisted on the testing phase being completed before proceeding to the final deployment phase. Although this sounds like common sense and simply adherence to good project management there are examples in different organisations were these disciplines were not adhered to, resulting in problems further down the line. After the go-live, the Trust agreed a partial (three part) sign-off and introduced more meaningful local sign-off criteria, departing from the norm of either signing or rejecting completely at the end of 45 day deployment verification period. This was beneficial both to the Trust and the supplier. The impact of this on payments to the supplier were handled by NHS CfH.

 An increasing number of Trusts see information as a key business resource and are incorporating IM&T planning into their business planning cycle . Some Trusts have a rigorous approach to benefits management that applies to all investments in that organisation, including IM&T projects . There is evidence of a developing programme and project management culture and an increasing recognition that project management is a professional skill that needs be used to manage both IM&T and other projects . There is some evidence of local pooling of specialist resources, e.g. trainers

Objective 4 Identify areas where central support might be provided at the national, SHA or local level to help resolve the issues identified through the project.

. Improving the handling of issues both in terms of the speed and the way in which they are handled . Providing Trusts with informed access to i.e. understanding elements of the contract over which they do have control, and direct dialogue with the LSPs . Making core product implementation timelines more realistic and extending them as necessary . Providing greater flexibility and consistency of approach when it comes to Trusts using interim solutions . Enabling experience and good practice to be shared and communicated….

Case study: Collaborative Lessons Learned (LL) Process Capturing and sharing of Lessons Learned had been very much ad-hoc across London PfIT, with a formalised process only at post go-live stage. The project team identified the need for a formal joined-up approach to the capture, sharing and acting upon Lessons Learned and good practice. NHS CFH Capacity & Capability Programme became the sponsors of a Lessons Learned pilot project, including Barts and the London NHS Trust. Project’s achievements: 1) Performed a rigorous lessons learned review, with NHS CFH helping collection, as part of their final go-live planning activities (2) Helped to design a project lifecycle LL process & go-live lessons learned capture process which resulted in some 150 lessons being collected during their go-live (3) Transferred knowledge of lessons learned directly to the next implementation site, to influence their final go-live planning An evaluation report will be published in late July. The following diagram describes a continuous lessons learned process:

Project Team can Local Deployment Manager (LDM)/ browse/search LL .Project Manager (PM)facilitate review of Repository as required LL from previous projects with Project and recommend/take Team – typically at start of project but Appropriate actions are actions where necessary may repeat at key stages planned to avoid pitfalls and to apply good practice

Observe Lessons throughout the project Disseminate or Good practice Collect

Routine LL reports can Reuse be generated to inform Project Team of LL from elsewhere Lessons Learned Collect Lessons Repository Applying Knowledge or Good practice Project Team capture LL in LL Log weekly – especially Workstream Leads

LDM uploads LL Log to populate repository weekly Store Review for Applicability, etc. Verify

Trust designated lessons Organisational learned Co -ordinator review / Improvement validate workstream logs and Opportunities LDM/PM identify and consolidate into weekly Project agree improvement LL Log actions and record in LL Log

. Developing a credible rapid-replication strategy and creating pools of experience to support roll-out, reduce risk and increase confidence . Celebrating and recognising successful implementations done with, not to, the NHS. . Helping maintain a focus on delivering the National Programme, which being a long term programme by its nature is constantly being displaced by immediate and short term performance pressures. . Developing capability building offers in leadership for IT, enabled change, benefits, programme, and project management competencies designed with and readily accessible by the NHS.

1.8 Progress on Risks Identified within TAR We revisited the Top 10 TAR risks to see whether they had disappeared, were significantly reduced, or substantially the same. These TAR risks were seen as the key interim transitional risks and we wanted to see if they manifested themselves at the local level and we also sought to identify whether new risks had emerged.

Ref: TAR Risk LOORD Risk Assessment LOORD Dimension

01 Ref GOV04: There is a lack of A perceived lack of clarity as to what the Governance clarity regarding the commercial governance arrangements are and how they framework and no Scheme of work. Rectifying this is regarded as a priority Delegation exists. SHAs ability to issue. Whilst Trusts want NLOP to work we make informed decisions will be found some evidence that a lack of clarity as compromised leading to eroded to their liabilities and obligations can reduce influence and lack of engagement. confidence in the principle of accepting ownership and accountability at the local level.

02 Ref. GOV06: Failure to realise the We found a variable degree of acceptance of Governance full potential of the NPfIT Business responsibility or accountability. Where this Case due to lack of understanding was questioned it was associated with of ownership and accountability. concern about lack of leverage and inadequacy of performance measures. This is not universal and Trusts have made recommendations or found ways to overcome this. We found that where local ownership has been taken up progress has been made. Deals have been brokered and Trusts have broken through the log jam.

03 Ref. PER05: Foundation Trusts The risk that Foundation Trusts might declare Governance (FTs) will procure outside the UDI on masse is not considered a real “risk” programme, which will reduce despite the Newcastle example. Trusts seem deployments and devalue the to understand the conceptual benefit of Business Case. existing within a national framework – and see the risk that this solution may fail as the more serious risk for which they may soon need to make contingency plans. It is reasonable to expect that where there is a mature relationship between PCTs and FTs on the healthcare agenda then the IM&T agenda follows.

04 Ref. FIN03: New requirements The risk that new requirements cannot be Finance cannot be easily met, as there is no easily met remains – the existing contractual provision for these future cost framework stifles local initiatives to pressures. Implementation costs supplement or extend core programme become greater than the NHS can components in order to generate cost savings afford as there is no contingency in or improve service delivery. the programme. Some Trusts have had the opportunity to broker local top-ups to bring forward interim solutions and to fund new requirements within that interim product and this should be publicised.

05 Ref. PER09: NHS CFH The risk of un-coordinated activity remains Performance performance is not effectively with significant criticism around the lack of a Management managed leading to un-coordinated collaborative approach and clarity about issue activity. (NHS CFH accountability escalation and resolution. for specific responsibilities to the local NHS is not clear) From a Trust perspective the visibility of and feedback from SHA / CfH has not improved and the general point about an underlying weakness in communications remains: that a continuing lack of transparency creates the risk that priorities are not aligned resulting in disconnected action and loss of focus.

06 Ref. COM04: That LSP priorities Trusts express widespread discomfort with Service are not always aligned with those of the payment system and the 45 day “all or Management the SHA and that inappropriate nothing” Deployment Verification Period sign- payments are made to LSPs. off, considering that this exposes them to financial, operational and hence clinical risk. However, one Trust has achieved two modifications that provide a significant enhancement of the contractual arrangements in this area: this establishes a precedent that can be followed.

07 Ref. RDBT05: Insufficient numbers The risk remains that insufficient numbers of Clinical of NHS staff are interested / NHS staff are interested / available in the available in the numbers and skills numbers and skills mix required to support mix required to support design design plans. plans. It can be difficult to engage where delays / confidence in the For Clinicians to contribute effectively to the product are encountered strategic service redesign work that has to happen 18-24 months ahead of deployment they need to be freed off from short term pressures in order to do this and very few Trusts are considering this.

There is a general feeling that the lack of functionality discourages Trusts from seeking to maintain the engagement of the Clinical community and that they will wait to do this until such time as valid clinical elements exist with the product solutions.

08 Ref COM03: Without NHS CFH Trusts believe that their inadequate Commercial support, including access to the understanding of and lack of access to the contract, the SHA is unable to Contract impedes decision making. There was provide adequate commercial frequent reference to the lack of access to a advice and support to Trusts / resource that has a proper understanding of PCTs. This hampers decision- the contract and can provide them with making guidance. SHA commercial support is considered either to be inadequate or not understood in terms of:

. What is available . When it is available . How it may be accessed

09 Ref SM03: The Release This risk that ineffectiveness in the Release Deployment Management process is not Management process leads to a decrease in effective leading to a decrease in confidence of LSP / NPfIT products remains - confidence of LSP / NPfIT products Trusts are severe in their comments on the Planning and Deployment process.

10 Ref. HR01: Risk of losing staff and There have been variations in the how SHAs Capability and not being able to recruit into key have deployed resources transferred from Capacity positions. NHSCfH. Some Trusts do not consider that there has been any direct benefit to them or reduction in their risks as a result of this transfer.

11 The failure to manage communications in a Planning strategic manner with a structured positive feedback mechanism creates programme implementation and management risk. The team encountered a number of examples of both good practice and poor practice.

12 There is a general view that the contractual Service SLA Performance Measures use Management inappropriate metrics which is leading to operational disruption and in the future would create clinical risk if left unchanged.

13 A strongly voiced concern relating to the Capacity and capability and capacity dimension is the risk Capability associated the projected rate of implementation. The view that it is simply not achievable on the grounds that it is just too much for either the LSPs or the Trusts to handle. It is considered that repeated public promotion of unrealistic deployment schedules will risk destroying the programme due to implementation failure.

14 There is a strongly voiced concern from those Capacity and that have been through implementation that Capability Trusts will significantly underestimate the effort required in terms of time, money and workforce and that experience sharing, structured workforce planning and shared resources will be required in order to mitigate this risk

1.9 Correlation with Health Informatics Findings and Summary Care Record Early Adopters Evaluation Two LOORD team members worked directly on the Health Informatics review workstreams, looking at governance and successful CRS implementation. They have identified that there is a great deal of synergy and complementarity between the two projects and suggest that full cross referencing of remedial proposals would be a positive contribution towards ensuring a solid, single, joined-up approach to issues to be addressed at national level. With regard to the independent evaluation of SCR, whilst NLOP was not their main focus, a side observation which very much mirrored what the LOORD team found was reported; “we found numerous examples of confusion and miscommunication between PCTs and CFH over “national” versus “local” responsibility for particular parts of the programme.” and “the tension between centre and the periphery should be addressed proactively rather than left to resolve over time.”

1.10 Conclusions Overall, the Trusts that participated in the review were generally positive in their attitude towards the aims and objectives of their local Programmes for IT and the overall National Programme. Their recommendations work in the direction of the removal of perceived weaknesses and drawbacks. This will allow them to move to a point where they can approach deployments with a reasonable degree of confidence. They want to share experience and they want the whole IM&T agenda to become a normal part of everyday business. The distance between where they believe they are and where they would like to be is not great and bridging the gap would play an important part building the conditions for the long term success of the Programme.

1.11 Main Recommendations The following sections contain a summary of the key recommendations that have been made by participating Trusts (unless otherwise stated). They relate to the level at which it is felt action may best be brought to bear – the local Trust level, the SHA Level, the Local Programme level and the National level.

Local Level

. Ensure adequate executive representation for IM&T at Trust Board level . Ensure an experienced Non-Executive Director takes on a Trust Informatics Champion role . Ensure that all IM&T projects, whether they are a part of NPfIT or local, are grounded in properly worked up business cases . Ensure that local service improvement plans include associated IMT components

SHA Level

. Focus on a realistic implementation approach to a credible acute secondary CRS system . Communicate to Trusts the SHA IM&T strategy and investment plans . Share with local Trusts the basis on which the LSP is performance managed and performance data on progress against plan and benefits realisation . Explain SHA team roles, the level of support they can offer to local projects and how Trusts may access this

Local Programme for IT Level

. Ensure that there is a transparent and simplified escalation path and route to problem resolution . Clarify for the NHS how governance issues relating to the Local PfIT work . Improve overall planning within the Programme . Enable experience and resource sharing at the Programme level . Ensure performance measures and SLAs measure whether operational performance meets user acceptability levels . Improve the approach to familiarisation and training

National Level

. Strengthen and clarify governance arrangements and communications . Establish governance arrangements for Mental Health and Ambulance Trusts that suit their multiple PCT relationships . Ensure that the priority ranking used by the National Service Centre corresponds to the front line delivery impact . CfH focus more on acting as a standards setting body that sets specifications, defines interfaces, accredits providers and negotiates framework contracts

LOORD Team Recommendations & Observations . Clarify the rules and procedures for procuring an interim system and buying outside the NPfIT catalogue

. Create the framework through which Trusts may establish transparent and predictable arrangements with suppliers . Identify those performance measures that allow an application to be contract compliant but non-functioning in practical usage terms and, if necessary, change the contracts to ensure that all performance measures are “fit for purpose” . Establish a mechanism for tracking local level priorities for new applications and validating that understanding with local owners. . National capability building support needs to be widely communicated, with feedback loop to ensure resources remain relevant and responsive.

2 Method and Approach

2.1 Methodology

2.1.1 Approach

The methodology adopted is described below. The basic concept is that of a pure fact finding exercise designed to help the Trusts – one where there is no right or wrong answer – providing a straight reflection of their current state of confidence and their perception of where they are. To assist this exercise the team developed a set of Confidence Statements (see Annex 2) for the Trusts to score. Scores ran from 1 (No Confidence) to 6 (Fully Confident) with the score of 4 (Reasonably Confident) being the lowest acceptable level of confidence for the LOORD risk analysis. The option of 0 (Non Applicable) was offered but was accepted only where the reasons why the statement did not apply could be justified.. The LOORD design group composed of Carol Clarke, Linda Lloyd, Johann Zahra, Nick Fitch, Andrew Luce, Peter Magee and Paul Mansell. The Dimensions used were: Governance Service Implementation Planning Service Management Capability and Commercial Capacity Performance Clinical Management Finance Benefits Deployment . Within the 11 Dimensions above, the Team developed and refined a list of 75 statements for Trusts to score. This Confidence Statement was tested at a pilot Trust, amended and a set of guidance notes developed to aid the Trusts in its completion.. A letter explaining the whole process was sent to participating Trusts together with the Confidence Statement and the Guidance Notes The key intention was that the statements should be completed by the Trust under the aegis and with the direct involvement of the Trust CEO acting in his or her capacity as the SRO for the local implementation. The project documentation for participating Trusts was sent to the CEO as SRO. By and large, the exercise was successful in obtaining CEO participation. Following the successful experience of the TAR project the LOORD team retained the concept of peer reviews whereby Trusts scores and the issues driving the scores would be discussed at interviews comprising of representatives of two Trusts and the LOORD Team. The principle retained was that the pairs would be drawn from the same category of Trust within the same Local PfIT but from different SHA areas. This worked well for the PCTs and the Acute Trusts but worked less well for the Mental Health and Ambulance Trusts due to the size of their areas. In practise there were just 9 interviews involving 18 paired Trusts and 19 single Trust interviews, as we had difficulty in finding matching diary dates that suited both sets of Trust participants. Trusts were asked to complete and return their Confidence Statements. Their data was loaded into the PAD (Project Assurance Diagnostic) tool and Spider diagram analysis was performed. Interview guidance was prepared and sent to Trusts in advance of the interviews

The LOORD interview team comprised Carol Clarke, Linda Lloyd, John Willshere, Kathy Mason, Jana Dale, Paul Charnley, James Hawkins, Peter Magee, Manpreet Pujara, and Andrew Luce. All were briefed by the Project Lead, Linda Lloyd. Wherever possible two LOORD team members attended interviews but this was not always possible and 16 interviews were conducted by a single LOORD team member. Post interview report summaries were then developed with the emphasis on collating comments by dimension, although frequently comments cut across more than one dimension. As a general courtesy Interview reports were shared with the participating Trusts. The LOORD project ran from 13th February to 13th May 2008; the last interview occurred on May 1. 2.1.2 Participating Organisations

A total of 37 Trusts have participated in the LOORD project. The splits by category and SHA are as follows Table 1 Trust Type Sample Sizes Table 2 SHA Sample Sizes

Grouping Sample Size Grouping Sample Size

PCT 12 North West SHA 4 Mental Health Trust 7 North East SHA 3 Acute Trust 15 Yorkshire and Humber SHA 4 Ambulance Trust 3 West Midlands SHA 4 Total Trusts 37 East Midlands SHA 3 East of England SHA 3 South Central SHA 5 South East Coast SHA 5 South West SHA 3 London SHA 3 Total Trusts 37

The split by LSP and by Foundation Trust status is as follows: Table 3 PfIT/LSP Sample Sizes Table 4 Foundation Trust Status Sample Sizes

Applying for Sample FT Status / LSP Size Type of Trust No. FT Will Apply

BT 3 PCT 12 0 0

Fujitsu 13 Mental Health Trust 7 3 4

CSC 21 Acute Trust 15 6 8

Total Trusts 37 Ambulance Trust 3 0 2

Total Trusts 37 11 12

2.1.3 NHS Engagement with LOORD Process

The LOORD project took place at the same time as NHS organisations also had to complete LHC IM&T Plans and go through SHA scrutiny on these. Given that we were asking them to make a significant additional time commitment we adopted a highly flexible approach. Whilst a Project Plan, processes and a timeline were set out and shared with SHAs and NHS organisations, the Team was flexible in brokering how and when participants took part. Response times for self-assessed Confidence returns and the timing and format for follow-on peer to peer meetings were all negotiable within the overall time limit. The overriding objective was to help volunteers stay in the process despite other pressures. Strategic Health Authority Involvement In line with NLOP routes for communication, a letter went to SHA CEOs as Senior Responsible Owners, confirming that LOORD was to follow on from the Transition Assurance Review and asking for SHAs’ subsequent cooperation in planning and implementing this. A letter was sent directly to SHA CIOs, with SHA CEOs copied in, explaining more fully LOORD’s objectives and the process to be adopted, and a sample participant invitation letter was attached that they could use to make the initial engagement contact with Trust CEOs The following actions on the part of the CIOs were requested: 1. Name a contact for the SHA in terms of liaison and administrative support to this work.

2. Inform the LOORD team if they or their deputy wished to take part in the interview process

3. Identify for which Acute Trust and PCT they wished to participate in the review and secure their CEO’s support for taking part as volunteers

4. Liaise with their peer SHA to obtain 3 further participants including Mental Health, Care or Ambulance Trusts.

SHA CIOs were not able to release staff to take part as interviewers or as ad hoc observers. How Trusts were contacted by SHAs in response to this request to seek volunteers clearly conveyed its importance or otherwise to potential participants. This caused protracted debate with some over several weeks, hence delay in obtaining results. Eventually 36 organisations were identified as potential participants in the process but one was put forward beyond the deadline for closedown.

We reviewed the number and type of these organisations and CFH staff engaged directly with Trusts to gain a further 7 potential participants to achieve the balance required for sample validity. 5 Organisations dropped out of the process for a variety of reasons: 3 PCTs, one Acute and 1 Ambulance Trust. The final sample of 37 was nevertheless a representative balance of organisational type and geographic spread, with the exception of London being under-represented. This is illustrated in Section 2.1.2 Table.3. Subsequently SHAs CIOs have expressed the wish to take ownership of and disseminate the findings once these are made available, for local discussion. They will be supplementing them with proposals for PfIT, and SHA level improvement plans Responsiveness of Participating Organisations Despite competing year-end work pressures, most Trusts showed commitment to work with us and contribute to the LOORD findings. Over 80% of respondents of the self-scored Confidence returns added very comprehensive commentaries. The follow on peer challenge and single organisation interviews elicited further honest, insightful and constructive feedback and examples of good practice. 4 of the 9 peer reviews occurred by Videoconference, a process with which many were unfamiliar but found effective. CEOs were in attendance at a majority of meetings along with their Lead Director and/ or Head of IM&T. They demonstrated that they were very keen to consider the current situation, forthcoming challenges and what would help speed system maturity to exploit fully the NLOP devolution. Several organisations decided that, through the LOORD process, they had derived significant value from having established a connection with a similar Trust elsewhere in the country and determined that they would maintain that exchange of knowledge into the future. In almost all instances the CEO’s involvement was visible, in varying degrees, Bradford PCT found that the exercise stimulated an appetite for a deeper local community review, and their LHC went on to use the CfH LHC IM&T Self Assessment (LISA) tool in assessing capability leading to joint ownership of a prioritised Improvement plan. .

3 Context

3.1 Current Deployment Status by Local PfIT

This section describes the state of deployment in each of the three Local PfITs

3.2 London Programme for IT Update

. 20 out of the 31 London PCTs have installed and are using the new community systems, RIO; a further 9 will go live in the current year.

. Six out the 10 mental health trusts in London also use RiO for their electronic patient record system, with a further 1 to go live this year.

. PACS fully deployed

. Three hospital trusts live with the Cerner Millennium LC0 acute electronic patient record system with 5 more planned to go live with LC1.

. Over 60% of London GPs can now send electronic prescriptions to pharmacies, and 60% of pharmacies can receive them.

. A total of 75 GP surgeries have now installed the new electronic patient record systems.

. Three trusts and their local social services have installed the electronic Single Assessment Process – improving ease of access to information on elderly people.

. In 2008/9 the mental health and community health electronic patient record systems will be upgraded to allow access to patient demographic information via the national Spine.

. A national pilot that will allow mental health, primary and secondary care teams to share information on adults in their area with social care teams is set to go-live in Newham during 2008.

3.2 Southern Programme for IT update

. The Southern Programme for IT covers the South West, South Central and South East Coast SHAs. The three SHA Chief Executives form the SPfIT Management Board with overall responsibility for the delivery of the Programme.

. Fujitsu Services has been the LSP for the south of England, delivering the Cerner Millennium system. Cerner were subcontracted to Fujitsu Services in June 2005. The contract with Fujitsu was terminated very shortly after LOORD interviews with SPfIT interviews were completed.

. The roll-out of the Picture Archiving Communication System (PACS) was completed in March 2007, ahead of schedule.

. As at April 2008, 10 Acute and 7 Primary Care Trusts have gone live with the foundation release (known as R0) of the NHS CRS.

. As the Programme has progressed a series of maintenance and upgrade releases have been deployed to live sites, with subsequent go-lives benefiting from the experience of existing users. The latest maintenance upgrade in development is release 08.

. A further 9 Trusts due to go live with Cerner Millennium.

3.3 North Midland and East Programme for IT Update

. 718 GP solutions have been installed to date along with 175 upgrades. There are 206 planned deployments for 2008/09.

. There are 49 PCTs live with TPP Community and 11 are planned for 2008/09.

. Currently 33 PCTs have TPP Child Health installed and 19 deployments are planned for 2008/09.

. A range of additional TPP modules have been successfully piloted and are in various stages of roll out; for example: Diabetes, Drug dependency rehabilitation, out-of hours and hospices. Primary care information is starting to be shared across a wider constituency, such as with A&E Departments.

. The roll-out of the Picture Archiving Communication System (PACS) has been fully completed.

. There are 20 Acute Trusts and 13 Mental Health Trusts who have taken an LSP system to date. Contract reset with CSC is due for completion in June, when a realistic OIP will set the context for a deployment refresh.

. There have been deployments to 35 Prisons and 24 have been planned for 2008/09.

. 3,423 GPs can now send electronic prescriptions to pharmacies, and 4,648 pharmacies can receive them.

4 Scores and Analysis

4.1 Discussion summary - principal observations and analysis

Trusts treated this exercise with an appropriate degree of attention and seriousness. The quality of the reflection and consideration shown is demonstrated both in the Confidence Ratings and in the Interview reports. Trusts did understand that the object of the exercise was to address ownership and governance issues relating to the programme rather than to discuss product weaknesses. These ownership and governance issues do include, however, comments on issues relating to the way in which perceived product and LSP shortcomings are being managed. 4.1.1 The summary findings from NHS organisations interviewed are that:

. Perceptions as to the effectiveness of NLOP transition varies both within and between Local PfITs indicate that there is still a need for greater clarity

. Trusts need Local PfIT implementation to be part of business as usual and want to simplify the operating mechanisms so as to facilitate this approach

. Some NHS organisations showed markedly greater confidence in local arrangements and ownership than the average – particularly those where there is a cohesive Local Health Community covering PCT(s), the local Acute Trusts and where the Mental Health Trust is also involved.

. There are notable examples of effective work-arounds where Trusts have negotiated specific arrangements to the solution and/or variations to the original contract: two Mental Health Trusts are receiving PARIS through NMEPfIT.

. Trusts feel very strongly that they lack a role in the contract and access to control levers. They feel that this results in their not being considered to be the customer. However, one Acute Trust introduced both an additional Gateway prior to Go Live and a three part payment to replace the single DVC payment. Trusts want improvements made to the manner and the speed with which issues are handled and they want informed access to the LSPs on issues which they can and should legitimately steer.

. Planning, Deployment, Service Implementation and Service Management are all areas concern, leading to NHS Organisations scoring Benefits planning and realisation as relatively weak

. The application of a mass roll-out model as if the systems were Commercial Off The Shelf (COTS) products when, clearly, they are not is seen as badly undermining credibility. Trusts believe this model should cease until such time as the solutions are fit for purpose and can be rolled out across multiple, parallel implementation sites

. A consequence of the above is that the credibility of the DIP, OIP and the Development Plan is weakened

. Trusts also challenge the timetable for rollouts as they question the Capacity and Capability of both the LSPs and the LHCs to support the proposed rhythm of delivery, even if all CRS products were fully fit for purpose and ready for use

. If core product implementation timelines are to be more realistic and are extended, there should be greater flexibility and consistency of approach when it comes to Trusts using interim solutions

. Implementation experience should be shared and good practice communicated – there is widespread support for the notion of creating pools of experience to accelerate roll-out on a 1:2:4:8 basis.

. NHS culture is to laud and reward heroic short term effort over that of successful, complex programmed implementation without noise in the system.

. Delivering the National Programme is “long term” and is constantly being displaced by the real and immediate. Reference to IM&T in the Operating Framework should continue, driving this Programme into the performance management mainstream..

4.2 Overall Confidence Ratings

The spider diagram below shows the average statement of confidence returns across all 37 Trusts Confidence scores indicate: Score 1 = No Confidence, Score 2 = Vey Low Confidence, Score 3 = Lacking Adequate Confidence, Score 4 (threshold score) = Reasonable Confidence, Score 5 = High Confidence, Score 6 = Totally Confident. Only the Financial Dimension exceeds the threshold score of 4. Whilst none average below 3, Benefits, Commercial and Deployment are furthest from the 4 mark. The Clinical score is slightly misleading because many Trusts stated that they had largely disengaged in this area on account of the current lack of clinical functionality in the products, hence were intentionally inactive. PCTs in the NME region scored this higher than other Trusts and this may be correlated to expressed satisfaction with the rollout of their GP and Community product. Overall the gap is not huge, however all but one dimension fall into 3; “lacking adequate confidence” score, however there are significant differences between sectors.

Average across all trusts

Governance 6 Benefits 5 Planning 4 Clinical 3 Capability & Capacity 2 1 - Commercial Performance Management

Service Management Finance

Service Implementation Deployment

Acceptable Confidence Rating Average

Figure 5 Average of all Trusts across all domains

We then looked at how this average varied across the different Trust types and this is demonstrated in the chart below

PCT Average Acute Trust Average Mental Health Trust Average Ambulance Trust Average Average

Figure 6 Average by Trust type, by domain, against overall average The PCTs are the only group that scored above 4 on Governance and scored 4 on Planning. This seems partly to be due to the role of the PCTs under NLOP. We encountered 8 examples where the Local Health Communities have organised themselves tightly into a cohesive planning unit and Trusts involved in such LHC’s returned markedly stronger confidence ratings in the Governance area than other Trusts, showing a greater ability to adapt, to develop interim solutions and work around issues without losing faith in the general philosophy and long term benefits of their LPfIT. Mental Health and Ambulance Trust scores reflect the lack of fit of their multi-PCT, Multi – LHC relationships to the Governance arrangements around LHCs.. Ambulance Trusts demonstrated much stronger links to the SHAs than other Trusts, which is probably due to the size of their geographic coverage and the number of Trusts of all types with which they have to work. Mental Health Trusts are generally unhappy with progress on their solutions, which is not surprising given that these do not meet their statutory obligations and that they see the difficulty in resolving this as a Governance and Commercial issue not as a Product issue. This said, those Mental Health Trusts involved in cohesive LHCs seemed less worried about the situation, basically because, it would seem, they feel that they will be supported in finding an interim solution and two Trusts in the North have negotiated such solutions within the Programme. We then looked at the variation between the three PfITs. The London Trust sample of only three is too small to be considered representative however those three Trusts scored consistently ahead of Trusts in the other two PfITs. In the NMEPfIT the issues around Lorenzo seem to pull down scores in the Planning, Capability and Capacity Dimensions whilst the poor scores from the Southern PfIT in the Deployment, Service Implementation and Management and the Commercial Dimensions would seem to reflect the impact on confidence of the hiatus and now cessation in the Fujitsu contract.

Southern PfIT London PfIT NME PfIT Average

Figure 7 Average by PfIT, by domain, against overall average

4.3 Confidence Ratings By Trust Category We have looked at the averages according to Trust type. The caveat here is that there are only small numbers of Mental Health (7) and Ambulance Trusts (3) included in the sample populations. We use the Spider Diagrams to compare confidence ratings against the average as well as the level 4 threshold. We use Histograms to show the Range of responses within each dimension. 4.3.1 PCTs

The PCTs’ responses show that in the key Governance dimension they are, in general, reasonably confident with the NLOP arrangements. This does seem to reflect the effectiveness of the integrated LHC approach where Trusts seems to display much more confidence in their ability to manage around the situation and find interim solutions whilst remaining within the wider programme, plus the satisfaction in NMEPfIT PCT provider arms with the GP and Community products. PCTs are very confident in the Financial Dimension but recognise weaknesses in the Benefits Dimension related to capability, capacity and planning.. Other Dimensions run close to the threshold 4 level and in the commercial / service management and implementation areas are returning a more confident score than the average for all Trusts.

PCT Average against All Trust Average

Governance 6 Benefits 5 Planning 4 3 Clinical Capability & Capacity 2 1 - Commercial Performance Management

All trust Average PCT Average

Figure 8 PCT average plotted against overall average

The Range Analysis demonstrates that there are quite wide ranges in the responses but the lowest PCT scores do not seem to be dragging the overall average PCT scores down to any significant extent.

PCT Range and Average

5.50 5.29 5.17 5.25 5.00 5.00 4.75 4.78 4.75 4.63 4.50

4.00 3.50 3.50 3.20 3.00 2.75 2.67 2.70 2.50 2.25 2.00 1.80

- Governance Planning Capability & Performance Finance Deployment Service Service Commercial Clinical Benefits Capacity Management ImplementationManagement

PCT Minimum PCT Maximum PCT Average All Trust Average

Figure 9 PCT range plotted against PCT Average and Overall Average 4.3.2 Acute Trusts

Acute Trusts show scores in excess of 4 for both Finance and Capability and Capacity, where their familiarity with Prince2 and MSP methodologies is reflected in the returns. They show less confidence than PCTs in areas such as the Commercial and Deployment dimensions. Given that Acute Trusts represent nearly half the overall sample population it is not surprising that its average is generally close to the overall average. The Acute Trusts display wide ranges in their responses. The lowest overall score is for the Commercial Dimension and this reflects their view that they consider that they cannot impact the contract, do not have access to any levers that would allow them to exert real influence over the LSPs and / or Connecting for Health, and are not seen as the client by the LSP, the Supplier or CfH. One Trust, however, had managed to work through the contractual arrangements and had negotiated local arrangements for both an additional gateway review 8 weeks prior to Go Live and for making three partial DVC payments to the LSP. In subsequent interviews, other Trusts expressed interest in replicating these modifications.

Acute Trust Average against All Trust Average

All trust Average Acute Trust Average

Figure 10 Acute Trust averages plotted against overall average

Acute Trust Range and Average

5.50 5.50 5.50 5.40 5.30 5.30 5.14 5.17 4.83 4.63

4.00 3.50

3.10 3.00 2.83 2.67 2.45 2.30 2.00

1.33 1.00 1.00

Acute Trust Min Acute Trust Max Acute Trust Average All Trust Average

Figure 11 Acute Trust range plotted against Acute Trust average and overall Average

4.3.3 Mental Health Trusts

The seven Mental Health Trusts display a confidence profile that is similar to, but score less well than, the all Trust average. The Mental Health trusts in the South declared a very clear lack of confidence in the Deployment, Service Implementation & Management and Commercial dimensions. The Clinical Dimension is over 4 but is largely untested – Trusts are stating they are confident that they can handle this. The distinction was clearly drawn between any inadequacy of the product (not in scope) and the inadequacy of the contract management and implementation arrangements to handle their concerns. They feel that because they have been included in the Programme relatively late in the day the Governance arrangements are not really designed to cater for them properly, allow their views to be heard and recognised and their specific requirements met. However in the NME region two Mental Health Trusts have obtained interim solutions that have been funded within the Programme.

Mental Health Trust Average against All Trust Average

Governance 6

Benefits 5 Planning 4 3 Clinical Capability & Capacity 2 1 - Commercial Performance Management

All trust Average Mental Health Trust Average

Figure 12 Mental Health Trust averages plotted against overall average Once again there is marked variability in the response range without the extreme low scores pulling the average right down..

Mental Health Trust Range and Average

5.00 5.00 4.86 4.60 4.50 4.50 4.50 4.50 4.33 4.25 4.17

4.00 3.50 3.50

2.88 2.88 2.80 2.67

2.00 1.90 1.75

1.29 1.10

MH Trust Min MH Trust Max MH Average All Trust Average

Figure 13 Mental Health ranges plotted against Mental Health Average and overall average

4.3.4 Ambulance Trusts

The three participating Ambulance Trusts score least well on the Dimensions where they rely on their own resources or on their interaction with the LSP and return higher than average ratings for Dimensions where they have a strong interaction with the SHAs.

Ambulance Trust Average against All Trust Average

All trust Average Ambulance Trust Average

Figure 14 Ambulance Trust average plotted against overall average The small number of Trusts in the sample drives the variability around the all-Trust average. The poor scores in Planning and Capability and Capacity partly reflect a current lack of experience in IT management. Another factor is the problem with handling the large number of partner Trusts in different LHCs and the capacity of others to handle them: “We are last on everyone’s agenda”. There is no identified PCT for them to deal with so they tend to go to the SHAs, who are very supportive. Their perception is that commissioners do not understand them and are not taking into account the information impacts of changes in the settings of care.

Ambulance Trust Range and Average

5.00 5.00

4.40 4.25 4.33 4.00 4.00 4.00 3.75 3.67 3.40 3.40 3.33 3.13 3.14 3.00 3.00 3.00

2.50 2.38 2.25 2.00

Ambulance Trust Min Ambulance Trust Max Ambulance Trust Average All Trust Average

Figure 15 Ambulance Trust range plotted against Ambulance Trust average and overall average

5 Dimension Analysis

The following discussion of the Dimensions surveyed is a summary of the more complete analysis and reportage to be found in Annex 1. Level 4- Reasonably Confident was the benchmark standard. This analysis has been drawn from the self-scored returns and the interview reports. Whilst it is presented by Dimension, discussions were free-ranging and observations could apply to multiple domains.

5.1 Governance

Governance Statements

- 01:Roles and 03:Internal 04:Local 05:Impact on 06:Foundation 07:Impact in LHCs 08:SRO Benefits responsibilities Governance Governance requirements Trusts

Overall Average PCT Mental Health Acute Ambulance

Figure 16 Governance statements averages plotted by Trust Type The average is under 4 and both PCTs and Acute Trusts hit the mark despite the following concerns that were frequently expressed:

. A basic lack of clarity surrounding the new arrangements

. A lack of adequate understanding of the contract and the legitimate role local SROs in Trusts can exercise within the contract

. The apparent lack of mechanisms whereby Trusts can exert any leverage on suppliers and the LSPs

. The distinct and continuing feeling that Trusts have that they are not considered to be the client by the other parties – the LSP, the Supplier, CfH.

Trusts need to know what can they trade, what they can negotiate and how they can feed requirements into a transparent process that allows them to see when they will get a decision and what that decision is likely to be; one organisation had been seeking an answer for over a year..

The Mental Health and Ambulance Trusts are not well integrated into the new arrangements as they cover very large areas and have to deal with multiple PCTs and Acute Trusts. We observed that Trusts returned reasonable to high confidence scores when they are operating within well organised multi-Trust LHCs with clear SRO leadership

5.2 Planning

Planning Statements

- 01:Enabler of change 02:Business Plan Priority 03:Business Plan Local 04:PCT is leading planning deployment

Figure 17 Planning statement averages by Trust Type The scores are well under the 4 threshold level and Ambulance Trusts stated that they feel that they are “some years away from full involvement”. A common theme is that “it is not clear how much in the future CfH procured systems will provide the functionality to enable this change”. Yet there are positive statements:

. a PCT said “Yes –our Service Reform Lead is engaged in the governance of the programme. This provides an opportunity to align thinking across organisations”.

. a Mental Health Trust stated that “ISIP methodologies to align enablers with strategic service priorities and use change management disciplines are well embedded”.

Local PfIT is seen as a key Business Plan priority as it does score over 4 but it has been significantly diluted by the perceived inability to provide a viable product in some areas. As a result Trusts regard IM&T as a strategic priority but are expressing caution about NPfIT in this regard because of delays and concerns about functionality. Only 9 participants stated specifically that they have established full business cases for the National programme; 25% of the sample which, it could be speculated from observing their degree of cooperation with LOORD, might be skewed towards higher performers. Others expressed the view that this is not necessary as it is a mandatory programme and, most worryingly saying “because there

© Crown Copyright 2018 Page 38 of 52 0355790dd16c99d2d15ee8a8adb91472.doc 2nd June 2008 is no cost to us”. It is the LOORD Team’s view, however, that the result of there being no Board endorsed business case supported by a suitably structured approach is that this greatly amplifies risk in managing the programme. There is a lack of confidence in linkage between whole systems reform planning and the Local PfIT. This seems to come down to the lack of functionality, the relative immaturity of some PCTs in their development into a leadership role in this area and the lack of recognition in some quarters of the PCT in the SRO role.

5.3 Capacity and Capability

Capability and Capacity Statements

- 01:Fully funded02:Recruitment 03:Business 04:Gap filling 05:PRINCE 06:Individual 07:Available 08:SHA 09:SRO LHC Programme and retention Change plan and MSP roles and SHA Support Support is has Capability capability responsibilities adequate Programme Capability

Figure 18 Capability and Capacity statement averages by Trust type The Acute Trusts score markedly better than the other Trusts and are above 4 for items 3-6 above. A typical reservation is that of funding, especially where Trusts have staffed up for deployment either last year or this year and their start dates have slipped. Three concerns arising from interviews were:

. Money; infrastructure resources and backfill training

. Lack of complex programme management capacity and capability such as high end portfolio programme management attending to strategic alignment of IM&T with services and system reform.

. The desirability of being able to align the acquisition of such resources to the planning process in terms of quality, quantity and timing. The LOORD team observed that this is largely due to the perceived unreliability of planning timelines.

. The LOORD team felt that understanding and fulfilment of the SRO role was overstated by some CEOs.

5.4 Performance Management

Performance Management Statements

- 01:Board understands 02:Executive held to 03:Mechanism for 04:Integrated Delivery 05:Outcome based 08:SRO PCT and delivery account planning Plans tracking Local PfIT objectives responsibilities

Figure 19 Performance Management statements averages by Trust type Most Trusts felt that there is a need to strengthen Board scrutiny of IM&T Strategy and improve integrated delivery planning. Some reported Integrated Service Improvement Programme (ISIP) as their tool of choice here. A significant number of Trusts have identified discomfort hence reluctance to scrutinise on IM&T issues at Board level as an area needing improvement. One Trust that comes from an area with a high density of IT companies suggested that many Trusts may not have sufficient “high-IT literacy” Non-Executive Directors on their boards. A counter view was that one can always hire these skills in on an as-needs basis to advise the Board in relation to specific events from an independent perspective.

5.5 Finance

Finance Statements

- 01:Financial Plan Accommodating 02:Empowered Financial Decision Making

Figure 20 Finance statements averages by Trust type The Financial Dimension was the one that collected the strongest average scores. Concerns in this area relate mainly to alignment of planning cycles to ensure that financial plans properly reflect the costs of implementation and operation of the programme. In light of Barts and the London Trust’s and others experience, the LOORD team felt that organisations with little if any experience of deploying real time systems that require 24/7 back up may have understated ongoing financial costs. On the other side of the balance sheet, defining potential wider organisational and health system benefits potential was observed to be very limited, with few exceptions. In fact only 25% of organisations had completed a local business case for National Programme products.

5.6 Deployment

Deployment Statements

- 01:Deployment Sign Up 02:DIP Plans 03:LSP has deployed

Figure 21 Deployment statements averages by Trust Type The PCTS are more confident with Deployment than other Trusts. Acute Trusts and PCTs seem to be clear about what they are getting even if they do not like it. The issues are about timing, slippages and clarity of costs and benefits. Acute Trusts stated that they had grossly underestimated both the cost of implementation and the headcount impact of operation – quotes ranged form 40 – 60+ additional FTEs. Trusts displayed considerable vehemence about unrealistic plans: one Trust quoted 18 slippages through the process and another quoted 4 slippages since October 2007.

5.7 Service Implementation

Service Implementation Statements

- 01:LSP Business 02:LSP 03:Train the Trainer 04:Training needs 05:Training QA 06:Training QA 07:Lessons Change Capability shortcomings and plan process Review Learned Sharing resolution

Figure 22 Service Implementation statements averages by Trust Type Whilst there are criticisms and comments for Acute Trusts and PCTs four of the 7 measures are either meeting or are very close to the threshold confidence level. Business Change and Lessons Learned capability come out generally weaker across all sectors When it comes to confidence in the LSP responses seem to be somewhat dependent on the team or persons supplied, the credibility of the LSP, whether Trusts have deployed applications to date or not, and how good that experience was. The Mental Health and Ambulance Trusts display the impact of having to lag behind PCTs and Acute Trusts In relation to training, there is a need to ensure that the Training environment fits the product and for training to be Role based rather than task based. Generally, however, Trusts are comfortable in this area, in some cases identifying that there are improvements to be made in Training Quality Assurance. They see this as an internal matter. The area where the scores are relatively weak is in the communicating and sharing of experience and lessons learned and Trusts believe that major benefits could be obtained through improvements in this area. Trusts that have made the investment and gained the experience have expressed willingness to share their experience and approaches with other Trusts. Barts and The London NHS Trust have just recently implemented one of the most ambitious implementations of an acute secondary care CRS to date, and share here two of the 150 lessons they learned from the experience;

1. They had to do a significant amount of programming themselves – interfaces, test scripts and reports – and that Trusts need to ensure that this built into Programme resources and Plans

2. They underestimated the impact on their clerical and administrative processes; Millennium is in a real time environment so transactions need to be recorded “right first time” and a user simply cannot go back and change things after the event; Trusts need to expect to have to completely rebuild business processes upfront and help their staff make a difficult transition to the mind set and culture that is needed in order to work with these new processes.

They recommend setting up a mechanism whereby experience and material (which could include things like processes and investments in interfaces, reports and test scripts) may be gathered and shared across the wider NHS community. The LOORD Team regard this within the remit of Local PFITs, potentially with some CFH technical and capability support e.g. Espace collaboration suite. Lessons learned developments in London are reported as good practice at p 10.in this Report.

5.8 Service Management

Service Management Statements

- 01:SM Escalation 02:Local Control of 03:LSP SLAs 04:SHA dispute 05:Service Introduction 06:Go-live confidence Service Management resolution role

Figure 23 Service Management statements averages by Trust type The main weaknesses here lie in the areas of control over service management, Service Performance and SLAs, and the resolution of disputes. Trusts complain about getting little or no feedback on issues and no visibility as to time or path to resolution. They want predictability and transparency with set escalation points. Trusts are also very concerned about the nature of contractual Performance measures and SLAs. It is contested that a service may be contractually compliant but effectively inoperable for end users, disrupting frontline services. In the same vein Trusts suggest that the priority ranking used by the National Service Centre should correspond to the front line delivery impact.

5.9 Commercial

Commercial Statements

- 01:Local delivery timescales 02:Local CfH Capability 03:Contractual Variations 04:Commercial Expertise

Figure 24 Commercial statements - averages by Trust Type This is an area where, in general, Trusts are not happy, except for the PCTs who scored above 4 in the Planning area. Dissatisfaction was expressed that First contact with the LSP (Event 1) is to discuss deployment and not to discuss requirements. Trusts want to understand the elements of the contract that affect them, the normal basics – upgrades, enhancements, training, SLAs, procedures for scheduling downtime, outage management . They need to engage with a credible implementation and development deadline, feel confident that they will receive adequate and timely project support, be able to share information, enjoy simple and rapid mechanisms for addressing shortcomings / concerns and be able to access adequate commercial support from their SHA. Trusts perceive that there is a barrier that prevents them having effective communications either with CfH or the LSPs. They consider that by the time issues are discussed between CfH and the LSPs important elements of contextual understanding are lost or diluted The Multi Layer Barrier

C L F S H P

Organisational Layers

Local PfIT SHA

NHS Trust . Figure 25 Commercial Statements - Representation of the Multi Layer Barrier

5.10 Clinical

Clinical Statements

- 01:Clinical Engagement 02:Clinical engagement 03:Clinical involvement in 04:Clinical design involvement Mechanism Benefits Realisation

Figure 26 Clinical statements averages by Trust Type The scores revealed that, despite apparent vocal criticism, confidence levels are relatively high and that this is not seen as a serious area of risk. The situation is rather more that of disengagement and a feeling that it is best not to seek too much clinical involvement until relevant functionality improves. In many instances the arrangements are untested to date.

5.11 Benefits

Benefits Statements

- 01:Expected 02:Benefits 03:CEO Guidance 04:Benefits 05:Benefits Base 06:Benefits Base 07:Post- 08:LHC Benefits 09:LHC Benefits 10:LHC Benefits Benefits Information on Benefits Realisation Path lining Capability lining Capacity implementation Review Plans Review Capability Review Capacity Access Realisation Benefits Reviews

Figure 27 Benefits statements averages by Trust Type Benefits is a relatively weak area, with low confidence and relatively little commentary included in the responses. The LOORD team observes that one of the issues may be that NHS Trusts do not have the experience or culture of measuring benefits objectively over a long period. If benefits are predicted in a Business Case then use of the installed system is often assumed to deliver the Business Case benefits and they are not subsequently measured in an objective, methodical manner. Trusts with ISIP Benefits Realisation Planning methodology embedded in their change programmes are clearer on this. Some organisations have gone through, are going through or about to go through deployment without a business case and have no baseline to measure benefits against. This impairs their ability to focus on exploiting wider organisational and health system benefits and they are not properly equipped in terms of skills or capacity to do this. Few PCTs stated they had the skills to hold providers to account for benefits realisation and there were consistently low scores on LHC wide benefits management.

6 Conclusion and Recommendations

We concluded that Trusts generally have a positive attitude towards the aims and objectives of their Local Programmes and the overall National Programme. Our analysis of their recommendations show that they want to work to remove perceived weaknesses and drawbacks and allow them to move to a point where they can approach deployments with a reasonable degree of confidence. They want to share experience and they want the whole, integrated IM&T Agenda to become a normal part of everyday business. Bridging the gap between where they believe they are and where they believe they would like to be could play an important part in building the conditions for the long term success of the Programme. The following sections represent a record of the key recommendations made by participating Trusts, unless otherwise stated. They relate to the level at which it is felt action may best be brought to bear – the local Trust level, the SHA Level, the local Programme level and the National level. The majority relate to the SHA and the local Programme level. Some recommendations are inevitably coloured by misconceptions of the parameters around NLOP devolution. 6.1 The Local Level

. Strengthen programme governance by ensuring all Trusts identify, support and develop a member of the Board to take the role of Informatics champion .

6.2 The SHA Level

. Ensure all SHAs shift their focus from pressure to go live to pressure to go live with well tested systems

. Ensure SHAs communicate to Trusts their IM&T strategy and investment plan, together with the corresponding business case and benefits analysis, showing how it fits to the parallel strategic plan for services

. Ensure SHAs share with local Trusts the basis on which the LSP is performance managed and performance data on progress against plan and benefits realisation

. Recommendations relating to SHA Support

Explain SHA team roles and Explain clearly the mechanism the capacity and capability of through which Trusts may access the level of support they can this support offer to local projects

Assess the level of support Provide training and guidance in needed to meet the agreed Benefits planning and support to roll-out plan and ensure it is bridge any identified gaps in adequate benefits management

Set up agreed risk pooling and delay penalty arrangements

6.3 Recommendations and Observations: Local Programme Level

. Recommendations relating to Improved issue resolution

Ensure there is a transparent Establish set times for providing and simplified escalation path responses and resolving issues with

and route to problem resolution automatic escalation at the end of the period

Simplify the mechanisms for Ensure that the decision making addressing shortcomings process is transparent

. Recommendations relating to Clarifying Governance of the Local PfIT

Establish very clear Create the role of Contract Liaison mechanisms and governance Officer – the person who knows the arrangements covering how contract and can act directly with Trusts and LHCs may engage the LSPs at the appropriate level with their LSP - clarify the and be in direct contact with the responsibilities of different Trusts roles - what they can and cannot do

End the uncertainty around Ensure that the LSP arrangements access to the contract and incorporate the flexibility to adapt for provide simple, clear guidance the future on its provisions

. Recommendations relating to Improving Overall Planning within the Programme

Align the Financial and Deployment Planning Ensure that the DIPs and OIPs are cycles so that Deployment resources are credible properly reflected in the Financial Plan

Make the DIP and OIP Establish the development road planning process more map at the level of detailed transparent functional specification needed in order to set realistic timescales

The LSP should ensure The LSPs should move away everyone understands the road planning LPfIT wide roll-out as if the map solutions can follow the Commercial Off The Shelf (COTS) deployment model whilst the solutions are still under development and until rapid replication strategies have been developed.

Develop a credible and agreed Cease planning deployment slots rapid replication strategy. until a successful, demonstrable implementation pilot has been completed, lessons learned and experience shared, thereby creating the basis for a general roll-out.

. Recommendations relating to Enabling Experience and Resource Sharing at the Programme level

Share information so that Fund implementation teams for next Trusts going into Deployment wave implementations so as to can understand the true level develop experience and meet the

of investment to expect needs of the future rapid replication strategy, once it has been agreed. “In a scarce market, pool resources and integrate workforce demands”

Agree and communicate a Provide Trusts with planning tools and a Capacity standardised approach to and Capability self assessment form resource sharing and support across SHAs in the same LPfIT

Provide a framework for Provide Trusts with Benefits Benefits Realisation, including measurement and documentation a Benefits rationale for each templates Module.

. Modify Performance Measures and SLAs so that they are fit for purpose and measure whether operational performance meets user acceptability levels

. Recommendations relating to Improving the Approach to Familiarisation and Training

Provide early access to Ensure that the Training environment is the same as systems so Trusts can the version that is to be deployed familiarise themselves with the solution and analyse its impact on the LHC

Provide the Training Ensure that training is Role based Environment on site so that rather than Task based clinicians can familiarise themselves locally

6.4 The National Level

6.4.1 Trust Recommendations: National Level

. Recommendations relating to strengthening and clarifying governance arrangements

All SHAs establish governance Clarify the situation relating to the arrangements for Mental Health staff cost of delays due to and Ambulance Trusts that suit deployment slippage – their multiple PCT relationships particularly where members of staff are funded or provided by the SHA.

Establish a mechanism under Ensure equity of relationship local control to charge penalties between the Trusts and the LSPs or delay payments for non- - end the situation whereby the delivery and/or inadequate LSP can penalise Trusts for delay functionality. but Trusts cannot recover costs they incur because of LSP delays

Improve CfH communication so Establish positive bi-directional Trusts know both how to engage feedback mechanisms so that

and with whom they can engage there is effective message communication and validated understanding of priorities and issues at all levels.

. Ensure that the priority ranking used by the National Service Centre corresponds to the front line delivery impact

. Establish work under a standards body to set specifications, define interfaces, accredit providers, negotiate framework contracts

6.4.2 LOORD Team Recommendations & Observations: National Level

The comments all relate to the National Level

. Recommendations relating to Governance

Explore the finding that LHCs Consider publicising and covering a variety of Trusts recommending replication of the and operating as such to modifications Trusts achieved that agreed principles and acting introduce in concert show better . A Trust controlled Gateway review 8 weeks governance ratings than prior to Go Live others and examine whether this can be readily replicated and, if so, how best this could . Splitting the single DVC payment into 3 part- be done payments

As part of good practice, Issue guidance on what should ensure that every Trust happen if the Business Case were establishes an NPfIT not to prove favourable implementation business case

Issue guidance on Trusts Clarify the Rules and procedures for rights to delay deployment or procuring an interim system and take on interim solutions in buying outside the NPfIT catalogue, order to resolve situations drawing on Tees and Leeds Trusts where the Business Case experience. counter-indicates the current LPfIT solutions

Issue good practice guidance Help NHS to access evidence based on the representation of change and programme management IM&T at Trust Executive competencies, including SRO Board level. development, where national delivery makes sense

. Recommendations relating to Commercial Issues

Create the framework Identify those performance through which Trusts may measures that allow an establish transparent and application to be contract predictable arrangements compliant but non-functioning with suppliers in practical usage terms and take action to ensure that all . Establish the basis for any discussion of

requirements and entry into negotiation performance measures correspond to the need to . Define contractual discretion meet user acceptability levels transparently: what can/ can’t be done

. Clarify contract change mechanisms for the Trusts as they are very complex

. Generate funding by creating conditions where Trusts can, on a local basis, invest to save

Consider creating specification of a “Fit for Purpose” SLA set covering all the classic items, including Performance Measures, the scheduling of downtime, rights to upgrades, upgrade training etc.

. Ensure all Strategic and Service Improvement plans include associated IM&T components

. As with London PfIT, support other PfITs in creating processes and support mechanisms for identifying and sharing implementation experience and good practice

. Establish a mechanism for gathering intelligence on local level priorities for new applications and validating that understanding with local owners.

. Recognise links to the Health Informatics Review, WS 7&10 in particular, and Summary Care Record Evaluation. Commit to coordinated action at national level and back SHA -lead action at PfIIT, SHA and LHC level.

. Map LOORD conclusions and recommendations across the Health Informatics Review, for consistency on outcomes that address the issue of how best to create robust and transparent governance arrangements for SHAs and LHCs and look to amalgamating similar activities that need to be addressed at the same level.

Acknowledgement

The LOORD team would like to thank all the NHS CEOs and their senior teams who provided considered responses and cleared space amongst their busy diary commitments to take part in interviews which helped to bring their views, good practice and proposals into this report.