Evaluation of the National Infarct Angioplasty Project Pilots

Evaluation of the National Infarct Angioplasty Project Pilots (NIAPP): summary of proposed research.

Background, aims and objectives. The Department of Health is currently funding a number of pilot sites as part of the National Infarct Angioplasty Project Pilots to test the feasibility of implementing a countrywide primary angioplasty service for patients with acute ST-elevation mycocardial infarction (STEMI). We aim to evaluate the pilot sites and address the five elements of the brief produced by the NHS Service Delivery and Organisation (SDO) R&D Programme. The specific objectives will be:  To gain an in depth understanding of patient and carer experience of receiving care at the pilot sites, and measure differences in satisfaction between the pilot sites and sites providing standard, thrombolysis-based care.  To assess the workforce implications of setting up an angioplasty service at the pilot sites, and measure the effect of providing the service upon the angioplasty team, support staff, and related staff groups.  To describe the models of service delivery established at the pilot sites in terms of their setting (geography, population, transport and communications), structure (hospitals, referral networks, transfer and access points) and components (staff, facilities, and equipment).  To explore implementation and feasibility issues by describing the processes involved in establishing primary angioplasty (particularly the development of teams and leadership roles), identifying facilitating factors and barriers to implementation, and assessing the implications of establishing primary angioplasty for cardiology and other services.  To compare the costs and outcomes of providing angioplasty and thrombolysis, and estimate the incremental cost-effectiveness of a comprehensive primary angioplasty service, compared to standard, thrombolysis-based care, for the different organisational service models adopted in the pilot sites. Plan of investigation

We plan to compare the pilot (intervention) sites to four control sites that have not formally established a primary angioplasty service. Evaluation will take a whole-system approach. We anticipate that even pilot sites that successfully implement 24-hour primary angioplasty will still use thrombolysis for at least a residual proportion of patients. Likewise, control sites that do not develop a primary angioplasty service may still use angioplasty opportunistically for some patients arriving during working hours. Our evaluation will compare the overall effect on all patients with AMI at each site, rather than focussing upon those who received the intended or principal service.

Patient and carer experience

We propose to use a phased approach of developing an instrument specific to the patient and carer experience of primary angioplasty. We will use the instrument to describe experiences in a range of pilot and control sites. Phase 1 will involve two sites with designated special responsibility for the evaluation of the patient and carer experience. Qualitative interviews post-discharge will be based on a maximum-diversity sample of patients and carers attending the pilot sites to include different ages, genders, arrival routes and living distance from pilot sites. We will interview between 15 and 25 people in their own homes using the critical incident technique to determine positive and negative aspects of the process, including issues such as communication and quality of care. We will also use these interviews to assess the feasibility of using a postal questionnaire in phase 2, specifically addressing the issue of identification of carers. The interviews will be used alongside other published instruments to develop a NIAPP- specific instrument to measure patient and carer satisfaction. A generic questionnaire will also be developed for patients in the context of emergency cardiac care. Questionnaires will be reviewed and piloted by the project consumer group. Phase 2 will involve a questionnaire developed from the interviews as well as other generic instruments for assessing patient satisfaction. We will send a postal questionnaire one week after hospital admission to every patient with ST-elevation AMI, and their main carer, attending four of the pilot sites over a nine-month period, ensuring that the samples are suitably screened for severe adverse events. The patient questionnaire will include the NIAP-specific and generic instruments, the carer questionnaire will include the NIAP-specific instrument only. We expect a minimum of 75 patients from each site in this time period, totalling at least 300 patients, and 200 carers (given that not all patients may have carers). We will aim for a response rate of 75% with two reminders. To provide data for comparison, we will send questionnaires to 300 patients with ST- elevation AMI in four control hospitals. We will only post the generic instrument to patients at control sites for direct comparison with pilot-site patients. We will aim to select the four pilot sites with the highest proportion of patients receiving angioplasty so that this aspect of the evaluation maximises the contrast between intervention and control care. This sample size will have 80% power to detect a 0.75 point change on each 5-point patient satisfaction question (alpha=0.05). Patients will be identified by staff within the cardiology department at the hospitals. Patients will need to consent to their postal details being sent to researchers at the University of Sheffield. Questionnaires and reminders will then be administered by researchers at the University. Phase 3 will involve up to 20 qualitative interviews with patients and carers from the two original pilot sites to expand upon results from the survey and identify possible solutions to issues identified. Patients will be identified from the questionnaire respondents.

Workforce implications, implementation and feasibility

We will use a mixture of quantitative and qualitative methods to address the questions outlined in the brief. Staff involved with providing angioplasty services will be identified by prior contact with the Trust. Six of the pilot sites will be selected to take part in the workforce assessment. It is anticipated that specific material will be produced for this study. These will be developed on sites working closely with the teams who are collecting economic data to produce a holistic model of investigation, and will involve close collaboration with researchers working at two other sites to develop the patient and carer satisfaction evaluation. Once piloted these materials and methods will be developed, edited and then used with the other intervention sites. We will administer self-complete questionnaires to staff who work with patients that have treatment for AMI. We will send the questionnaire to relevant staff by internal mail suggesting completion during work time. The questionnaire will examine the changing aspects of their work role, flexible working and managing a 24 hour service, support and training, teamworking and work factors (e.g., autonomy, role clarity, role conflict, influence in decision-making, peer and leader support). Questionnaire data will be complemented by the use of focus groups and semi- structured interviews with key stakeholders to examine the impact of the angioplasty service on the workforce and explore the wider implications of providing an emergency angioplasty service, such as cardiologists not being available for the general medical rota and changes to emergency referral patterns. Additional focus groups will examine the impact of angioplasty on other associated workforces (e.g. radiography, ambulance services and the emergency department). These focus groups and interviews will also explore issues of implementing the service and factors perceived to have facilitated or opposed effective delivery. Observational data will be collected, and where this is not possible it will be collected retrospectively through discussion of cases after treatment. A one-day site visit will be used as the principal vehicle to arrange and conduct interviews and focus groups. We will try to arrange this around an elective PCI session to minimize any inconvenience to staff. The visit will be supplemented, as necessary by telephone interviews, if staff are unavailable at the time of the visit. The visit will also be used to undertake the specialist teamworking interviews and ethnographic observation (or where not possible retrospective case discussion) that will address the implementation and feasibility issues. Teamworking interviews will examine the exact nature of the service delivery team, how team members work together and how their specific roles integrate. The model for the site visit will be piloted within the first six months of the project (phase one) to enable any practical adjustments to be made. It is anticipated data will be collected on 6 sites during the period September 2006 to March 2007. Data will be analysed using quantitative and qualitative methods. This material will be presented in an anonymous manner and integrated into a final report looking at various aspects of pilot project evaluation. A summary of this report and access to the final document will be made available to all who take part in the study

Description of models of service delivery

We will use data from routine sources (NIAPP, MINAP, BCIS, CCAD, HES), a telephone survey and site visits to describe the models of service delivery developed by the pilot sites. We will seek information on the following parameters: Service setting: urban or rural; area covered; catchment population (number, age and gender, coronary heart disease morbidity and mortality rates, ethnicity and social deprivation); transport and communications networks. Service overview: participating hospitals; teaching status; referral networks between hospitals; ambulance services; other prehospital care; access route to the service. Service infrastructure: staff numbers, grades, profession and specialties; interventional cardiology skills; rostering and on-call arrangements; facilities and equipment; bed availability (general, coronary care and intensive care); cardiac surgery services. Service activity levels: number of AMI treated (prehospital thrombolysis, in-hospital thrombolysis and angioplasty), numbers of emergency and elective admissions, revascularisation procedures, and cardiac operations performed.

Economic evaluation

Mark Sculpher and colleagues at the University of York have already developed a cost- effectiveness model to compare angioplasty to thrombolysis. It considers cost- effectiveness from the perspective of the NHS and uses data from the literature and routine sources to provide an estimate of the incremental cost per QALY. The analysis also looks at the variation in cost-effectiveness according to the time delay to angioplasty. The value of the model for the pilot evaluation is that it can be used to explore how the cost-effectiveness of primary angioplasty might vary between the different types of service configuration seen in the pilot centres. In particular, it can use centre-specific data on costs and times to reperfusion to assess variation in cost- effectiveness across different models of service delivery. We will increase the robustness and scope of the cost-effectiveness analysis by undertaking a microcosting study to measure the true costs of providing angioplasty in different models of service delivery and check the validity and completeness of MINAP data. Working alongside the workforce and feasibility evaluations, we will identify factors that determine the true cost of providing angioplasty. We will collect resource use data from four intervention and four control sites for 60 patients at each site, giving a total of 240 patients in each arm. At each of these sites, data will be collected on staff costs, equipment, consumables, drugs and procedures associated with the intervention (angioplasty or thrombolysis). Care will be taken to stratify patients by day and night time admission in order to capture any differences that may arise due to the timing of the admission. The resource use data will be valued using local unit costs where available or national unit costs. We will use, as far as possible, centre-specific cost data, estimated in collaboration with local finance departments using a standardised protocol. Staffing costs will be assessed in conjunction with cath lab management. The staffing arrangements will be costed according to whether procedures take place in or out of hours and the specific trust arrangements regarding out of hours staff including sensitivity analyses that assume different costs of knock on cancellations from out of hours work. It is anticipated that much of the data can be collected retrospectively using routinely collected data (MINAP / NIAPP / BCIS datasets). Where required, we will supplement this with detailed examination of patient and ambulance notes. Data will be gathered on these patients individual characteristics, including time to needle/balloon, for use as covariates in the cost analysis. We also intend to ask staff to complete a short questionnaire immediately after completing a sample of PCIs in order to record staff involved and their perception of knock on effects the following day. An additional adjustment that must be included is to identify the proportion of patients that are referred as ST- elevation AMI and therefore may trigger the out of hours team but are subsequently found not to be “true” cases. All the new data collected will be synthesized with the data in the existing cost- effectiveness model and an incremental cost per QALY will be estimated. Probabilistic sensitivity analysis will be used to address the uncertainty in the input parameters of the cost-effectiveness analysis. The decision uncertainty will be described using a cost- effectiveness acceptability curve. We will develop a regression model using the patient-level cost data and MINAP data to determine how much of the variation in cost can be explained by differences in the models of service delivery used, having controlled for case mix. Findings from the workforce and feasibility evaluations will be used to explain how different models of service delivery may be associated with different costs. The different costs for each potential model of service delivery will then be used in the cost-effectiveness model to explore the relative cost-effectiveness of the different approaches to service delivery. Summary of research elements and timescales.

Element of No. sites What is involved for each site? Timescales research involved Patient & Carer 2-3 Recruiting around 10 patients and May-August evaluation: intervention their carers at each site for a face- 2006 phase 1 sites to-face interview. (development interviews) Patient & Carer 4 intervention Recruiting @75 patients and their October evaluation: sites and 4 carers (where possible) in each site /November phase 2 (postal control sites to be sent a questionnaire. 2006 – July questionnaires) Checking patient status for any 2007 patients to whom we send a reminder Patient & Carer Any of the Patients and carers will be recruited November evaluation: sites involved from the questionnaire study. Staff 2006-July phase 3 in phase 2. may be asked to check the status of 2007 (exploratory a patient before us contacting them. interviews) Description of All A researcher will undertake site March 2006- service models intervention visits and telephone questionnaire January 2007 and control with PI/key staff. sites We will require access to routine data sources (MINAP, CCAD, NIAPP etc) Workforce 6 intervention Staff working with STEMI patients September evaluation: staff sites (in variety of roles) will be asked to 2006- April survey complete a questionnaire around 2007 workforce issues. Workforce, 6 intervention Some of the above staff and other September implementation sites staff in roles affected by PPCI may 2006- April and feasibility: be asked to participate in focus 2007 focus groups & groups or semi-structured interviews interviews to discuss teamworking, implementation and feasibility issues. Economic 4 intervention A health economist will discuss August 2006 evaluation: sites and 4 economic issues with finance and – June 2007 identification of control sites cardiology staff. cost data We will require access to routine data and patient notes for 60 patients at each site. Economic 4 intervention For a subset of patients, staff will be August 2006 evaluation: sites and 4 asked to complete a short survey – June 2007 survey of staff control sites about time spent on PCI procedures time. and consequences of on-call.