Evaluation of the Oak Processionary Moth Control Programme Final Report
19 February 2016
This page is intentionally blank
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Evaluation of the Oak Processionary Moth Control Programme Final Report
A report submitted by ICF International in association with Centre for Ecology & Hydrology Date: 19 February 2016 Job Number 30300452
Andrew Jarvis ICF International Watling House 33 Cannon Street London EC4M 5SB T +44 (0)20 3096 4800 F +44 (0)20 3368 6960 www.icfi.com
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Document Control
Document Title Evaluation of the Oak Processionary Moth Control Programme – Final Report (r)
Job No. 30300452
Prepared by Mavourneen Conway, Andrew Jarvis, David McNeil, Naazia Ebrahim (ICF) Michael Pocock, Colin Harrower, John Redhead (CEH)
Checked by Andrew Jarvis
Date 19 February 2016
This report is the copyright of Defra and has been prepared by ICF Consulting Services Ltd under contract to Defra. The contents of this report may not be reproduced in whole or in part, nor passed to any other organisation or person without the specific prior written permission of Defra. ICF has used reasonable skill and care in checking the accuracy and completeness of information supplied by the client or third parties in the course of this project under which the report was produced. ICF is however unable to warrant either the accuracy or completeness of such information supplied by the client or third parties, nor that it is fit for any purpose. ICF does not accept responsibility for any legal, commercial or other consequences that may arise directly or indirectly as a result of the use by ICF of inaccurate or incomplete information supplied by the client or third parties in the course of this project or its inclusion in this project or its inclusion in this report.
19 February 2016
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Contents
Executive summary ...... i Glossary ...... vi
1 Introduction 1 1.1 Purpose ...... 1 1.2 Structure of the report ...... 2 2 The OPM control programme 3 2.1 OPM control between 2006 and 2012 ...... 3 2.2 OPM control under the OPMCP – 2013 to 2015 ...... 4 3 The evaluation approach 13 3.1 Evaluation questions ...... 13 3.2 Counterfactual ...... 13 3.3 Evaluation methods ...... 13 4 Impact evaluation findings 19 4.1 Introduction ...... 19 4.2 Policy level ...... 21 4.3 Social impacts ...... 26 4.4 Environmental impacts ...... 32 5 Process evaluation findings 38 5.1 Introduction ...... 38 5.2 Programme level governance and management ...... 45 5.3 Operational delivery ...... 55 5.4 Implementation of control measures ...... 63 5.5 Data gathering and data management ...... 73 5.6 Awareness and engagement ...... 78 5.7 Research and analysis ...... 87 5.8 Capabilities ...... 92 6 Learning evaluation findings 97 6.1 Introduction ...... 97 6.2 Learning evaluation questions ...... 101 7 Conclusions and recommendations 112 7.1 Conclusions ...... 112 7.2 Recommendations ...... 113 Annex 1 References ...... 116 Annex 2 Life cycle of OPM and available control options ...... 119 Annex 3 OPMCP Schematics ...... 122 Annex 4 Full list of evaluation questions ...... 128 Annex 5 Further details on the different methodologies used for the evaluation .. 131 Annex 6 Stakeholders consulted ...... 137 Annex 7 Interview questions ...... 139 Annex 8 Questions asked in the request for written responses...... 142
19 February 2016
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 9 Full method and results of the modelling exercise ...... 146 Annex 10 Landowner survey screening questions ...... 170 Annex 11 Annual observed extent of OPM ...... 174 Annex 12 Calculating the OPM rate of spread (2012 – 2015) ...... 179 Annex 13 Number of infested trees ...... 184 Annex 14 Additional evidence on OPM health impacts ...... 186 Annex 15 List of Advisory Group Members ...... 187 Annex 16 Scale of control activities carried out by outbreak area as has been reported to the OPMCP ...... 188 Annex 17 Communications activity ...... 189 Annex 18 Overview of OPM research and monitoring activity ...... 190
19 February 2016
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Executive summary
Introduction An established colony of the oak processionary moth (OPM), which is not native to the UK, was discovered in the Kew area of London in 2006. Initial control efforts (administered under plant health legislation) had limited success and in 2011 the UK Government policy objective for the outbreak, which had spread across south west London, shifted from eradication to containment. Other, smaller outbreaks were discovered in Pangbourne (Berkshire), the Croydon/Bromley area and on the site of the 2012 Olympic Park. In March 2013 Defra ministers approved a pilot programme, the ‘Oak Processionary Moth Control Programme’ (OPMCP), that would develop and test an alternative approach to tackling the OPM problem. The OPMCP applied a mix of complementary interventions and established new kinds of partnership and governance. The most significant change was to provide public funding for the control of OPM. Previously landowners had to pay for the control themselves. The pilot programme also supported enhanced levels of OPM survey and surveillance, and used communications campaigns and active stakeholder engagement to support its objectives. The case for Defra’s investment was based on the threat posed by OPM to oak tree health and the risk that contact with the toxin contained in OPM larval hairs poses to human and animal health. Analysis showed that restricting the spread of the moth would yield significant benefits. This evaluation has considered the 2013-2015 programming period of the OPMCP, during which it spent almost £4.1 million. The evaluation comprises an appraisal of the pilot programme’s impacts, an analysis of what worked well and what not so well in process terms, and the identification of lessons that can be learned from the programme experience. It is informed by: consultation with stakeholders via interviews and written submissions; an online survey of landowners involved with the pilot programme; desk-based research and analysis; modelling work to estimate the of OPM’s spread in the absence of the OPMCP; and estimation – using GIS – of the additional human population that would have been at risk from OPM if it was left uncontrolled. A framework for the evaluation is provided by a programme intervention logic and accompanying narrative.
Impacts The scale and efficiency of the system for application of controls built by the pilot programme will have reduced the abundance of OPM within the known outbreak area as compared to the counterfactual of controls being applied by landowners. It is also likely that the OPMCP contributed to slowing the spread of OPM, though the available evidence does not make it possible to distinguish programme effects from other contributory factors. The observed rate of spread of the south west London outbreak for 2013 and 2014 (at 0.9 - 1.6 km/year) is below the estimated rate of unconstrained spread and within reach of the current goal of limiting spread to less than 1km per year. The observed rate of spread for 2015 (1.1 – 2.4 km/year) is greater than in the previous two years, but still below the rate of spread expected in the absence of the OPMCP (5 km / year). The development of firm estimates of the rate of spread of the outbreak is made more difficult by the discovery of new OPM colonies that may predate the pilot programme, and by some gaps in the data. Evidence from other sites indicates that other outbreaks have been contained but have not yet been eradicated. Data gathered during 2015 suggest that the area affected in Croydon / Bromley has reduced in size. A small number of nests have been found in the vicinity of the Olympic Park. No new nests have been found in Pangbourne, though male moths (which disperse much more widely than the females) have been found in pheromone traps set to collect surveillance data on OPM distribution. The identification of male moths in pheromone traps across the entire 2015 surveillance area and the discovery of additional, well-established OPM colonies, such as in Guildford, suggests that OPM was more widely dispersed than was originally appreciated when the OPMCP began. It is estimated that 3.5 million fewer people are living in the presence of OPM than would be the case if the south west London and Croydon/Bromley outbreak had spread at the projected unconstrained rate
19 February 2016 i
Evaluation of the Oak Processionary Moth Control Programme - Final Report
of 5km/year. Research evidence suggests that between 0.5% and 5% of people living in an area where OPM is present might seek medical assistance in any given year. This implies that the slower rate of spread delivers a significant public health benefit. It is also reasonable to assume that the control of the OPM population within the outbreak area reduced the risk to public health – the moth is less abundant than would otherwise be the case. Even allowing for under-reporting, the scarcity of records of health incidents linked to OPM (fewer than 20 thus far) nonetheless indicates that there is more to do to understand the health impacts of OPM in a UK context. Suppressing OPM abundance is likely to have reduced the risk to OPM-related impacts on public access and recreation. The pilot programme may have had an indirect positive effect by sustaining public confidence in the safety of green spaces. There are some anecdotal reports of local impacts, but no systematic evidence of OPM or the control programme having significant impacts on public access and recreation. Consultees thought the control programme will have had an impact on OPM abundance that would be positive for tree health. Estimation of the scale of the net impact on oak tree health has not been possible due to data limitations. The larger scale of spraying of oaks achieved by the pilot programme will have had negative biodiversity impacts. Equally, less effective control the OPM population would be expected to have led to larger scale defoliation of individual trees and more rapid spread of OPM to other oak trees. Prior to 2013 landowners used a variety of contractors with varying levels of experience whereas the programme used experienced contractors working to set standards under a framework contract. The OPMCP also specified use of the most selective of the available control substances and so ensured that there was little use of broad spectrum insecticides. It is likely that the provision of government funding for control has reduced incentives for some landowners to fell infected oak trees to avoid control costs. The net impact on biodiversity is therefore difficult to determine.
Process evaluation The OPMCP programme model has, overall, worked as envisaged. There was evolution from year to year in many aspects of the programme design but the basic architecture has proven to be sound. Many elements have worked well. The OPMCP has successfully demonstrated a collaborative approach to tackling a novel tree pest in heavily populated areas. Stakeholder engagement at both strategic level, via a high level Advisory Group, and on an operational level, has proved effective. Landowners have been well supported and mostly responded positively to the communications and the availability of public funding for OPM control. Strategic and operational management arrangements were effective. The Project Board brought together policy, operational, science and other interests. Operational coordination and management arrangements were effective. Information systems were established that enabled data on OPM presence to be collected and acted upon quickly. The control contractors’ efficacy and capacity improved. The efforts made to ensure in-year and year-to-year learning and improvement were successful. The pilot programme connected to sources of scientific advice and used new research evidence as it refined its approach. The OPM’s life cycle provides two short periods within which controls can be applied – larvae can be sprayed in spring and the communal larval nests removed prior to the adult moths dispersing. Prior to the OPMCP the identification and reporting of OPM on an oak tree would see landowners - prompted by Statutory Plant Health Notices – being obliged to locate a contractor, arrange a visit and pay for the spraying of larvae or removal of OPM nests. Issues with reporting, delays and incentives meant that this did not provide timely and effective control. The OPMCP established a much more responsive and comprehensive control system for the application of controls. Over the three years more than 90,000 trees at 663 sites were sprayed. Almost 21,000 OPM nests were removed from 573 sites. There is some evidence that public funding of controls encouraged landowners to report OPM sightings. Public funding of surveys led to more nests being identified. Use of a smartphone application by surveyors facilitated more accurate and more timely recording of survey data than
19 February 2016 ii
Evaluation of the Oak Processionary Moth Control Programme - Final Report
before the OPMCP was in place. An OPMCP database generated real-time management information that enabled the programme manager to closely direct the application of controls and make best use of the limited time available to enact effective treatment. Contractors with the necessary specialist skills were retained via a framework agreement and there was close coordination and cooperation between the programme manager and contractors. An enhanced surveillance programme generated new information that could be used to estimate the distribution of OPM. Overall, the evidence suggests that OPMCP improved the cost effectiveness of control for the average tree, but by less than was expected. The large number of sites treated that contained ten or fewer oak trees limited the scope for economies of scale. The commitment to regular learning and review ensured that there was year-on-year improvement but the evaluation also identified some areas where programme planning and delivery worked less well. The goal of containment (rather than eradication) of the main outbreak had not registered with all consultees and expectations were not always aligned between stakeholders and those managing the pilot programme. The pilot programme’s considerable efforts to engage local authorities at an executive level were largely unsuccessful, despite significant effort, and there was not full buy-in from public health interests. Even though schoolchildren were a potentially important vulnerable the communication campaign did not reach schools; this was due, in part, to the challenges experienced in engaging local authorities. The timing of the annual budget approval process affected programme planning and delivery. This had a particularly significant impact in 2015 when the OPMCP stopped funding controls in a core area of the London outbreak in order to concentrate available resources on a buffer zone near the estimated limit of the outbreak. It was noted for instance that the announcement came too late for some large stakeholders to make budget provision to compensate. The longer term implications of reduced control at the outbreak core for the overall OPM control strategy are still to be determined. Other issues identified include, in the early years, a lack of administrative support adding to the pressure on programme staff at the busiest periods. Opportunities were sometimes missed to provide more, better and quicker information to landowners about the OPMCP and the control activities being carried out on their site. Of particular relevance to this evaluation is that the planning and inception phase did not consider and provide for the collection of the data that would be needed to determine whether the pilot programme was achieving its objectives, assess its impacts and evaluate whether the experience gained was consistent with the theory and assumptions that underpinned the pilot programme’s design.
Learning points Given the OPMCP was designed as a pilot, the identification and dissemination of points of learning is one of the most important outputs of the pilot programme. The OPMCP provides valuable lessons for similar programmes, especially those operating in an urban environment under very close public scrutiny, with multiple stakeholders and added complexities in establishing site ownership and securing access. The OPMCP results demonstrate the need for good surveillance systems to detect novel pests early and for prompt and effective action to eradicate pests that are discovered before they can become established. Once the outbreak is established the pest is likely to be costly and very difficult, perhaps impossible, to eradicate. Effectively tackling tree pests in an urban environment means engaging with multiple, diverse stakeholder interests at both strategic and operational levels. The OPMCP has demonstrated that the right governance structures and communications support can create an enabling environment for programme delivery. With central coordination, survey and surveillance programmes, good data management systems, responsive and skilled contractors and the right incentives on stakeholders it is possible to build a control regime that is more responsive and effective than relying on statutory measures to instigate action by landowners.
19 February 2016 iii
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Evaluation needs to be considered at the programme planning and resourcing stage so that the collection of the data needed to address the selected evaluation questions can be accommodated can be accommodated alongside the operational design and delivery requirements. Without this upfront investment there will be less evidence on impact, on what works, and fewer learning points at the end of the programme. In preparing larger scale control programmes (as distinct from the rapid response mounted to newly identified, small scale incursions) it is important to provide enough time in the inception phase for planning, preparation and the acquisition of the capabilities to implement it (e.g. staffing, contracting and procurement processes), even if means that the ambitions for year one have to be scaled back. Contingency planning arrangements now in place for plant health pest and disease outbreaks do now take this point into account. If annual budget decisions fall within (rather than before) the annual programme planning period there is a risk that delays or unanticipated budget adjustments will disrupt the programme. Continuity of personnel in key roles retains knowledge and builds capability within the programme. Similarly, contracting mechanisms that engage suppliers for several years and which encourage collaboration and feedback can accelerate the development of control capabilities. The OPMCP has shown how building formal ‘learning loops’ into programme design can accelerate the rate of improvement and learning. A more structured approach to identification of new relevant science, e.g. a six monthly stock-take on new research and its implications, might have helped to ensure that relevant evidence was picked up more systematically. The OPMCP experience also provides a reminder that technical solutions (e.g. species-specific pheromone traps) need to be proven in context before full scale deployment. The OPMCP experience also shows it is important to be realistic about what is achievable, and to manage stakeholder expectations accordingly. It is helpful if pilot programme objectives do not get conflated with strategic policy goals. A pilot programme set up to test and learn from approaches that it is hoped will deliver progress towards those policy goals needs to be judged in those terms.
Conclusions The pilot programme has bought time for development and consideration of options, a stated objective. There is evidence that it has slowed the rate of spread of OPM in parts of the London outbreak area and contained the outbreak in Croydon. It is not yet clear whether the new OPM locations identified in 2015 are extensions of the south west London outbreak or separate events. The policy goal of eradicating OPM in areas beyond the main outbreak in south west London had not been achieved as of January 2016. The outbreak in Croydon is contained but not eliminated. Though no new nests have been found in Pangbourne, the presence of a small number of males in pheromone traps is suggestive of a residual population in the area. Small numbers of nests have been found in the Olympic Park area. In 2015 new outbreaks were identified in other locations, such as Guildford. Expert opinion suggests that without the OPMCP the OPM’s range would have extended to cover an area where around 3.5 million more people would have been exposed to OPM than is currently the case1. The OPMCP has also reduced OPM population density within the outbreak areas, which should also have reduced public and other health risks. An OPMCP objective was to develop novel techniques in survey, control and stakeholder engagement to aid future OPM and other pest & disease control, especially in urban areas. The OPMCP has developed an integrated sighting-survey-control system that has improved the speed and efficacy of controls. It has developed approaches to stakeholder engagement and communication that are novel for urban tree health protection. It has run a large scale pheromone trapping surveillance project. In most instances it is the combination of measures taken, or the operating context, that is innovative, rather than there being fundamental shifts in technology or technique.
1 See text for caveats and assumptions.
19 February 2016 iv
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A further programme objective was to retain core capacity and skills as a contingency for quick future redeployment. The pilot programme has helped to sustain capacity in FC and in its supplier base. The transferability of skills and systems to other contexts should be explored further before the programme closes.
Recommendations It is recommended that 2016 is used to: ■ Ensure that all the benefits of the OPMCP investment are captured and embedded in the operational and decision-making practices of FC and other programme partners. A ‘benefits realisation’ plan should be prepared and implemented before the programme ends to ensure that the opportunities for learning and transfer of capabilities to FC, FR and Defra are fully exploited. Components of this plan could include process notes documenting key aspects of programme operational practice (governance, surveys, communications, etc.) and events focused on (i) strategic, (ii) operational lessons for FC, FR and Defra. ■ Gather additional evidence that would help to support decisions on future OPM policy. There are opportunities to tackle the gaps identified in this report through modified data collection strategies and new experimental designs. There would also be value in testing interventions that would be needed to support a shift from a comprehensive to a selective, risk-based OPM management strategy, and researching the management practice, technology and risk-mitigation measures currently used elsewhere in northern Europe. ■ Review OPM policy and develop supporting measures for 2017 onwards. The evaluation confirms that a review of the OPM strategy and policy, based on the OPMCP experience, is warranted. This would need to begin early enough in 2016 for plans and resources to be in place for implementation early in 2017, but should also be flexible enough to accommodate evidence gathered in 2016. An enhanced monitoring scheme is required if Defra and FC are to be able to evaluate the impact of the programme on OPM spread. The OPM monitoring scheme should be subject to formal assessment prior to implementation and cost-benefit analysis should be used to decide how many sites should be monitored given the resource available. Looking beyond OPM and 2016, it is recommended that: ■ Evaluation and learning is built into the initial design, planning and budgeting of future programmes. An evaluation plan should be in place before the programme is launched and the programme’s data strategy should be consistent with the expectations of that plan. It should make the programme theory explicit. ■ Programmes should have monitoring that specifically supports evaluation against core objectives. ■ The lessons of OPM are incorporated into future review of plant health strategy, notably the value of early detection of novel pests and a rapid, effective response.
19 February 2016 v
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Glossary
CEH Centre for Ecology & Hydrology Defra Department for Environment, Food and Rural Affairs EQ Evaluation question OPM Oak processionary moth OPMCP Oak processionary moth control programme EU European Union FC Forestry Commission FR Forest Research FTE Full time equivalent SPHN Statutory Plant Health Notice PHE Public Health England THMP Tree Health Management Plan
Acknowledgements ICF and CEH would like to thank all those who gave their time to help the evaluation. Particular thanks are due to Andrew Hoppit, OPMCP programme manager, for providing information and advice with good humour during numerous consultations, and to Dr Jake Morris, Defra project officer, for steering the process and helping to ensure the full and active engagement of policy and programme interests. The analysis of programme data and modelling of OPM spread would not have been possible without the assistance of Dr Julia Branson at the University of Southampton, for which we are grateful.
19 February 2016 vi
Evaluation of the Oak Processionary Moth Control Programme - Final Report
1 Introduction
The oak processionary moth (OPM) is a native species of central and southern Europe. Its range has been expanding northwards through Europe and it is now well established in Germany, northern France, Belgium and the Netherlands. In 2006 colonies of larvae were discovered in parts of London. The larvae (caterpillars) of OPM feed on the foliage of many species of oaks. As well as being a defoliator of oak trees, OPM also poses a risk to human and animal health. The larvae are covered in hairs that contain a toxin. Contact with this toxin can result in irritation of the skin and respiratory tract. Since OPM can be found on urban trees, along forest edges and in amenity woodlands, the risk of public exposure is potentially significant. Efforts to control OPM in the UK began after its discovery in 2006. Between 2006 and 2011, UK Government policy was to eradicate all OPM outbreaks. In 2011, the policy objective for south-west London was changed from eradication to containment. This change was based on expert advice that the moth was now established within the original outbreak area. Eradication continued to be the policy goal for outbreaks outside south-west London (e.g. Bromley / Croydon, Olympic Park). In March 2013 a three-year pilot programme for enhanced control, hereafter the ‘Oak Processionary Moth Control Programme’ (OPMCP), was launched. The pilot programme provided: enhanced management capacity; additional investment in surveys, surveillance, research and communication actions; and public funding of control measures (i.e. the spraying of trees infested with caterpillars and host trees near infestations, and physical removal of OPM nests). Prior to 2013, landowners with trees containing OPM were served with statutory notices and were responsible for organising and funding control measures. In 2015 Defra and the Forestry Commission instructed ICF International, working with the support of the Centre for Ecology and Hydrology, to evaluate the OPMCP. The evaluation fulfils an objective set out in the 2014 Tree Health Management Plan (THMP)2 to design and carry out an evaluation of the OPMCP to inform management of the pest in the future, and assess the effectiveness of management options. This report presents the findings of the evaluation. It is set in the context of: ■ Defra’s wider strategic approach to improving the rigour of its use of evaluation and finding responses to the challenges of evaluating complex policies; ■ Defra’s increased investment in plant and tree health; and, ■ Recognition that the programme, being a pilot, is an example of decision-making in the face of uncertainty. The evaluation took a particular interest in the learning that took place during the evolution of the programme. The scope of the evaluation is limited to the pilot OPM control programme and therefore covers the 2013-2015 period only. The programme of control measures that was implemented between 2006 and 2012 is not within the scope of the evaluation, although it does provide some useful contextual information.
1.1 Purpose This report presents the findings of the evaluation, comprising: ■ An impact evaluation, assessing what impacts the OPMCP has had;
2 Tree Health Management Plan. Defra, 2014. Accessed on 23/12/2015 at https://www.gov.uk/government/publications/tree-health-management-plan
19 February 2016 1
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ A process evaluation, assessing what has worked well and what not so well in the implementation of the OPMCP; and, ■ An evaluation for learning objectives, assessing what lessons can be learned from the pilot programme. The evaluation focused on determining the effectiveness of the OPMCP in relation to supporting the THMP’s objectives to control / eradicate OPM3 and to enhance tree pest outbreak preparedness and contingency planning. It did not cover other aspects of the THMP.
1.2 Structure of the report This report is structured as follows: ■ Section 2 briefly describes the policy background to the OPMCP and the OPMCP itself. ■ Section 3 provides a brief overview of the evaluation’s methodology, including the evaluation questions and the challenges and limitations of the methods and data used; ■ Section 4 presents the findings of the impact evaluation; ■ Section 5 presents the findings of the process evaluation; ■ Section 6 presents the findings of the learning evaluation; ■ Section 6 concludes and presents the key recommendations of the evaluation. ■ Annexes provide additional detail and information to support the analysis in the content of the main report
3 Government policy on OPM as stated in the THMP: “To contain the outbreak in south-west London and seek to eradicate outbreaks where they occur outside the outlying sites of infestation”. See Defra (2014) Tree Health Management Plan. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/307299/pb14167-tree-health- management-plan.pdf
19 February 2016 2
Evaluation of the Oak Processionary Moth Control Programme - Final Report
2 The OPM control programme
This section provides a brief overview of how and why the OPMCP was conceived, and how it has developed since its inception.
2.1 OPM control between 2006 and 2012 A planned, strategic response to the threat posed by OPM to tree, human and animal health began in 2006 after the discovery of the moth in south-west London. The policy goal at this time was to eradicate all OPM outbreaks in the UK. The Forestry Commission (FC) led the response. Provisions4 available under the Plant Health (Forestry) Order 20055 enabled and empowered FC inspectors to act by issuing landowners with Statutory Plant Health Notices (SPHNs) where OPM was found. SPHNs put landowners under an obligation to have control measures carried out on infested trees, and provided the FC with enforcement powers (e.g. legal action) if this action was not carried out. Plant health inspectors from the FC were responsible for surveying for OPM, advising landowners on necessary control measures and issuing SPHNs. There was no compensation available to landowners to cover the cost of control measures if intervention was required. It became clear that there were a number of difficulties with this approach. There were considerable challenges in detecting OPM and, once an infestation was detected, establishing ownership of the tree was often difficult. Unless ownership could be established a SPHN could not be served and action by the FC would be delayed. As enforcement of SPHNs was uncommon, landowners did not always see a need to comply. Landowners’ liability for the cost of control measures also contributed to a lack of compliance. This problem was exacerbated by the fact that the cost of controls, especially for individual trees, was high as contractors were quoting on a case by case basis. These issues were compounded by the lack of a consistent approach to control across the contractors active in the market (e.g. different pesticides being used to spray trees). Over time it became clear that OPM had become established within the original outbreak area of south-west London and in 2011 a decision was made to change government policy. Instead of aiming for full eradication everywhere, it was determined that in south-west London the policy would move to one of containment. The policy of eradication was retained for other areas. The policy of containment meant that the original outbreak area was split into a ‘core’ and ‘buffer’ zone, with the aim to focus more of the available government resources on surveying and the control of OPM within the outer ‘buffer’ zone to limit the spread of OPM. Maps showing the core and buffer zone are provided in Annex 11. In 2012 a new Advisory Group was also established. This brought FC staff together with representatives of a variety of different stakeholder interests with the aim of exchanging information, best practice and coordinating activities amongst those affected by, or dealing with, OPM. In 2008 the UK submitted a proposal to the European Commission for protected zone status for all areas outside of the core outbreak area and a buffer zone for OPM. This status requires that the UK presents evidence that OPM is absent in the protected zone area, and that efforts are being made to eradicate the pest or to contain it from spreading into the protected zone. In turn, it requires that all oak trees supplied to the protected zone must be shown to be free from OPM.6
4 General provisions were adopted to March 2008 when specific national measures were introduced 5 The Plant Health (Great Britain) Order, as amended, implements the EC plant health regime. Further details are available at http://www.forestry.gov.uk/forestry/INFD-9QKGJR 6 From 2008 all oak trees coming into the UK Protected Zone from another European Union Member State were required to be accompanied by an official statement, or 'plant passport', confirming that the plants were free from
19 February 2016 3
Evaluation of the Oak Processionary Moth Control Programme - Final Report
2.2 OPM control under the OPMCP – 2013 to 2015 In 2013 an impact assessment on OPM policy determined that, “without a change in policy to ensure future government intervention, the pest [would] spread outward at an accelerated rate causing damage to local landscapes and recreation areas, and putting humans and animals at risk of respiratory problems and skin rashes from the moths' toxic hairs.” It stated that government intervention was, “necessary to ensure early detection and co-ordinated destruction of nests, to reduce the rate of spread and impacts. Leaving intervention to individual landowners [would] do little to contain the spread.”7 The impact assessment estimated that implementing a programme to better control OPM (based on the programme reducing the spread by 2.6km per year on average), would result in “reduced harm to human health and reduced impacts on recreation, landscape and biodiversity relative to the baseline. These benefits total £47m, £18m, £2m, and £25m, respectively, discounted total for 40 years”. A proposal, supported by the impact assessment, led to approval for a three-year pilot programme to provide enhanced control of OPM. It was estimated that this would require a total of £5 million over the three years, but funding had to be applied for every year. In total, just under £4.1 million was spent under the OPMCP. The 2014 Tree Health Management Plan (THMP) identified OPM as one of three tree pests and diseases scheduled for priority action. Government policy on OPM, as stated in the THMP, was to: ■ contain the outbreak in south-west London; and ■ eradicate outbreaks where they occur outside this core area of infestation, including sites at Pangbourne and Bromley in South East London. The OPMCP was identified in the THMP as the main programme of work for delivering the surveillance and control of OPM, alongside the continued work at the EU level to designate the whole of the UK (except for the outbreak areas in London and Berkshire) as an EU protected zone (see above). The objectives of the OPMCP were therefore to: ■ Support the delivery of Government policy on OPM, thereby avoiding increased future control costs and buying time to enable new solutions to be developed; ■ Reduce the impacts in the current infested area (e.g. human health); ■ Develop novel techniques in survey, control and stakeholder engagement to aid future OPM and other pest & disease control, especially in urban areas; and, ■ Retain core capacity and skills as a contingency for quick future redeployment. The OPMCP was designed as an enhanced programme of work involving increased survey work, improved control methods, enhanced sector collaboration and stakeholder engagement. It was hoped that the OPMCP would develop an understanding of how to tackle OPM and other similar pests. The budget provided for a five to ten fold increase in activity, and funding from the Forestry Commission specifically for: ■ Direct action (prophylactic spraying and nest removal at infested sites, with public agencies covering the costs of survey and treatments costs in designated areas); and ■ Enhanced stakeholder communications and inputs, and technical, administrative and scientific support.
OPM. Moreover, all oak trees moving from the infested areas in London into the protected zone had to be accompanied by an official statement, or 'plant passport', confirming that the plants were free from OPM. 7 Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment.
19 February 2016 4
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The pilot programme’s design was informed by an understanding of OPM biology. The OPM has a single generation per year. Gregarious larvae form communal pupal nests from which the adults emerge during July and August and then disperse to lay over-wintering eggs. The life cycle provides two opportunities for control – the spraying of infested trees with insecticide in spring to destroy the larvae before nests are established and then the physical removal of nests in the period before the adults emerge, typically in early to mid-summer. Distribution can be assessed through surveys in summer (to identify nests for immediate removal) and winter (to identify pupal nests from which adults have emerged during the previous generation). Pheromone traps can be used in summer to trap male moths for surveillance purposes but as the males disperse over much longer distances than the females, this gives an incomplete picture of the functional distribution of the species as defined by the breeding females whose eggs develop into the next generation of larvae. More detail on when, and what type of controls can be used during the life cycle of OPM is given in Annex 2.
2.2.1 Understanding the OPMCP The design of the evaluation was informed by the development of a logic model that articulated what the pilot programme was trying to do, what activities were being undertaken, and what the desired results were. It provided the theoretical framework that was used to understand whether the OPMCP had achieved its intended impacts. The process of developing the logic model identified that the OPMCP had five main elements: ■ Oversight, governance and operational management (e.g. the OPM Project Board, Advisory Group); ■ Direct action through the implementation of control measures, including prophylactic spraying and manual nest removal; ■ Surveying and surveillance efforts, including data storage and management; ■ Communications and awareness raising, including stakeholder engagement; and, ■ Scientific research and analysis. These are discussed in more detail below. The overarching, high level logic model is shown in Figure 2.1. Detailed schematics representing the wider OPMCP governance and operational institutional framework, and the OPMCP’s operational delivery are provided in Annex 3. 2.2.1.1 Oversight, governance and operational management The Project Board is the main governance body of the OPMCP. The Project Board makes key strategic decisions on the direction and execution of the OPMCP. It was established in 2014 as a forum for discussing and deciding on key operational aspects of the OPMCP. Its members are Defra staff with policy expertise and Forestry Commission staff covering the different operational elements of the OPMCP (e.g. overarching policy, organising controls, communications, research, data management). Its purpose8 is to: ■ oversee all aspects of the OPM control programme in order to deliver the outputs and outcomes desired by the approved and funded Defra project proposal; and ■ coordinate the activities of all parties to maximise the opportunities and minimise the risks. The Project Board was also established to provide the programme manager with additional support and guidance, when needed. It usually met monthly via teleconference. Meetings typically lasted for about two hours.
8 Oak Processionary Moth (2014) Project Board Terms of Reference
19 February 2016 5
Evaluation of the Oak Processionary Moth Control Programme - Final Report
There was regular liaison between the Project Board and the stakeholder Advisory Group, as well as Defra’s Tree Health Policy Group and the London Tree Officers Association (LTOA). The Advisory Group is a consultative forum that brought together representatives of the OPMCP programme management together with senior representatives of a wide range of stakeholder interests (including landowners, policy makers, budget holders, the health sector, and service providers (e.g. contractors and surveyors)) to discuss issues of mutual interest. A list of members is provided in Annex 15. Its aims9 were to: ■ Advise / inform policy makers, decision makers & research on strategic issues; ■ Facilitate surveillance and control across all stakeholders so its effectiveness is maximised; ■ Coordinate OPM communication. Meetings were generally held three times a year, although more were held in the first year when the OPMCP had just begun. Meetings usually lasted about two hours. Day to day operational management is the responsibility of the programme manager, with the help of support staff who lead on the different operational elements (e.g. communications, research, data collection and management). The budget for the OPMCP had to be estimated and approved annually. Budgets were set for each operational element, although funding could be transferred as needed from one budget line to another over the course of the year. Annual reports were prepared for 2013 and 2014. These summarised the previous year’s activities and presented proposals for the following year. Annual communication plans were also meant to be prepared. These would specify the communications activities and materials are needed for that year. There was insufficient time to prepare a plan for 2013. No formal plan was published for 2015, although the communications needs of the OPMCP were laid out in other programme documentation. The OPMCP included regular opportunities for review and improvement through, for instance, the Project Board meetings and, in particular, through annual ‘lessons learned’ workshops held with stakeholders. 2.2.1.2 Implementation of control measures There are two main options available to treat trees infested by OPM – spraying them with pesticides and manual removal of nests. Pesticide spraying can be done from the ground or air. No aerial spraying has been conducted under the OPMCP but it has been previously used in Pangbourne. The FC’s recommended approach is two rounds of pesticide spraying in the spring, with the first and second sprays roughly ten days apart. Nests are formed and become visible after the window for spraying has closed. Trees are sprayed when OPM has been seen the year before, and when OPM caterpillars are spotted on the tree. Manual nest removal takes place in early to mid-summer once nests are formed and before the adult moths have flown. A second spraying the following spring is recommended. OPMCP funded both pesticide spraying and manual nest removal. The FC plans and coordinates control efforts. Controls are targeted based on the results of the previous year’s survey and reported OPM sightings. Job sheets are issued for sites that need to be sprayed using the information in the OPMCP database. Once a site survey is complete a job sheet for manual nest removal is usually raised within 24 working hours of the infestation’s location being uploaded onto the survey website. In 2013 and 2014 job sheets were sent to the treatment contractors each day as infested sites were found. In
9 Oak Processionary Moth Advisory Group: Terms of Reference
19 February 2016 6
Evaluation of the Oak Processionary Moth Control Programme - Final Report
2015, at the request of the contractors, batches of job sheets were sent to the contractors two or three times a week to enable them to better manage the administrative burden. Three approved contractors are used for the work, which is organised by the FC (not landowners). The three control companies are contracted through a FC framework contract. Contractors are required to report to the FC about what controls have been carried out, and where. Contractors should carry out control work within 72 hours of receiving instructions for a job. Job sheets are issued electronically. Prior to the OPMCP, surveyors would post an SPHN to a landowner if OPM had been found. This would normally take at least five working days. Landowners would be given contact details for qualified contractors, though they were under no obligation to use them. Control work under the OPMCP faced a number of challenges: ■ A short period of opportunity for carrying out effective controls: there is roughly a six week window in which pesticides can be sprayed effectively; ■ Weather conditions (e.g. high wind) can make spraying difficult, and can further reduce the time available to carry out controls; ■ Access to sites can be difficult, often because the site owner is not known, the landowner has not allowed access, or because it is difficult to get the necessary equipment on site; ■ Uncertainty about what funding is available to cover what areas has made forward planning more difficult (see Section 0 and 5.3.1); ■ The practicalities of determining where to spray and of getting pesticides into dense canopy; and, ■ The OPMCP could only call upon the services of the three control contractors who had enough experience and expertise at the beginning of the OPMCP to be awarded a framework contract for the control work. ■ The control work relies on accurate and comprehensive information gathered through the survey and surveillance work. However data collection on OPM also faces a host of practical and logistical issues (see below) which have potential implications for the effective and efficient implementation of controls. 2.2.1.3 Data gathering and data management Surveys are conducted in the summer to identify live nests and in the winter to identify the site of old OPM nests which might have been missed in the summer survey. Winter surveys were conducted in 2014 and 2015, but not 2013. Some surveys were conducted in the core zone up until the end of 2014, after which survey efforts were focused entirely on the buffer zone. However, in 2015 information on the location and extent of OPM was provided to the FC by some core zone landowners (e.g. Royal Parks, Transport for London, Network Rail). The areas surveyed in London changed over the pilot programme’s lifetime. In 2014, fewer surveys were conducted in the north east quadrant of the core London outbreak whilst in 2015 the survey effort was more focused on the east and south east. In 2013 a smartphone application was deployed for surveyors to use in the field to increase the speed with which data could be collected, submitted and processed. It was not until 2015 that all FC surveyor inputs were collected using the field app. Up until then, some paper records were still being used. Identifying OPM infestations and nests is not straightforward. Surveyors face a number of logistical and practical challenges in gathering the information and data necessary to determine where OPM is located. There is, for instance, no systematic or comprehensive information on the location of all potential host trees. Once a host tree which could be infested with OPM is identified, the surveyors can face difficulties in gaining permission to access the site, often because it is unclear who owns it. If permission is granted, there can be additional challenges in gaining physical access to the tree in question. Weather
19 February 2016 7
Evaluation of the Oak Processionary Moth Control Programme - Final Report
conditions, a difficult physical environment and the nature of the trees being surveyed (which are often quite tall, with a dense canopy) can make spotting OPM difficult. The caterpillars and egg plagues are small, and the nests can be inconspicuous. Surveyors are contracted through a general FC plant health survey framework contract rather than one specific to the OPMCP. In 2014 the framework contract terms were revised to include specific requirements for surveys conducted to identify OPM. Data are collected from contractors on the control measures applied, including when, where, and what treatment was carried out, and the cost of the work. Data from the survey were stored in an MS Access database developed, maintained and managed by GeoData at Southampton University. The database was intended to be a temporary solution to be used in 2013 whilst a national plant health database was developed. It was envisaged that the new national database would be in use for the 2014 season. Delays in the introduction of the national system have meant that the Access database was not replaced and continued to be used for the duration of the OPMCP. The database was used to coordinate the work of the OPMCP, such as through the production of job sheets to plan and direct control efforts. Sites vary in their characteristics, e.g. size, the number of affected trees, access and information. 2.2.1.4 Communications, awareness raising and stakeholder engagement Awareness raising was a key element of the OPMCP programme model. It was hoped that as a result of stakeholders being better informed about OPM, its risks, their obligations and the action being taken under the OPMCP: ■ Landowners would be more willing to support control work, provide contractors with access to their site and act quickly to deal with OPM infestations; ■ Knowledge of where OPM is located would increase as landowners would be more willing to provide surveyors with access to their sites, and also would be more willing to report sightings of OPM; ■ There would be better reporting of health incidents and a reduction in health incidents as more people avoid OPM caterpillars and nests; These changes would in turn lead to more effective targeting of OPM controls, and a relative reduction in human and animal health impacts where OPM was present. The OPMCP involved communication activities that included: ■ Posters, leaflets and banners; ■ Signs for contractor vehicles; ■ Press releases; ■ Communications targeting specific stakeholder groups through bespoke articles in specialist media, such as the arboriculture sector and the animal and veterinary sector; ■ The use of social media (e.g. Twitter); ■ Online guidance and information; and, ■ Email updates to affected and interested stakeholders. 2.2.1.5 Research and analysis The OPMCP commissioned and paid for research into the population distribution and dynamics of OPM, OPM natural predators and the efficacy of different control measures. A significant amount of additional research which is not being directly funded by the OPMCP is also being undertaken. A full list of relevant research work is provided in Annex 18.
2.2.2 The OPMCP as a pilot programme The OPMCP was established as a pilot programme. The OPMCP spanned three years, over which time a large amount changed as adjustments were made year-on-year. The major adjustments are described below and in Table 2.1.
19 February 2016 8
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The policy shift from eradication to containment in London led to the outbreak in south-west London being divided into a ‘core’ zone and a ‘buffer’ zone for operational purposes. The core zone was the area where OPM had become an established pest. The aim was to concentrate control and survey efforts on the buffer zone, a ring around an effective ‘core zone’, and thereby contain the further spread of the moth. The boundaries defined for the core and buffer zone have been adjusted each year based on the results of survey and surveillance work. The Forestry Commission continued to fund surveys and control measures in the core zone in 2013 and 2014 to supplement the efforts in the buffer zone. It was hoped that continued action funded by the FC in the core zone would contribute to limiting the spread of OPM by potentially reducing some of the spread from the core to the buffer zone. At the end of 2014 it was agreed by ministers that strict containment in south-west London would be unachievable. This was based on evidence on the limitations of detection and the effectiveness of controls (e.g. ground spraying of pesticides is not 100% effective), and that even with unlimited resources a degree of annual spread was inevitable. The policy shifted from strict containment to containment being defined as slowing the rate of spread to less than 1km per year. This figure was defined based on the impact assessment’s analysis that limiting spread to this rate would deliver the greatest benefits relative to the cost of intervention. The shift also led to a change in how the core and buffer zone were treated for operational purposes. In 2015 no control actions in the core zone were funded by the OPMCP. FC funding for surveys, trapping, spraying and nest removal in the core zone was withdrawn and OPMCP efforts were focused entirely on the 12km wide buffer zone around the core and beyond. Some landowners in the core zone continued with control activities without FC funding. Also in 2014, the whole of the UK, excluding the outbreak areas in Pangbourne, London and Croydon / Bromley, was designated an EU protected zone under Council Directive 2000/29/EC..10 The protected zone is subject to annual review by the EU Standing Committee on Plant Health to ensure it remains robust. Table 2.1 Development in the OPMCP over time
2013 2014 2015 Policy goal / strategy Containment (no further Containment (no further Slowing the rate of spread) in south-west spread) in south-west spread in south-west London, eradication London, eradication London11, eradication elsewhere elsewhere elsewhere Governance / Advisory Group already Project Board Full time administration management in place (established in established support became 2012) available to aid OPMCP Membership of the operations Advisory Group expanded to include NGO interests Known areas of South-west London South-west London South-west London infestation Croydon / Bromley Croydon / Bromley Croydon / Bromley [Pangbourne]12 Olympic Park Olympic Park [Pangbourne]13 Hampstead Heath Guildford
10 See EC (2015) Protected Zones. Available from: http://ec.europa.eu/food/plant/plant_health_biosecurity/protected_zones/index_en.htm 11 Defined as slowing the rate of spread to <1km per year. 12 No identified infested nests, but precautionary spraying carried out 13 No identified infested nests, but precautionary spraying carried out. Male moths caught in pheromone traps
19 February 2016 9
Evaluation of the Oak Processionary Moth Control Programme - Final Report
2013 2014 2015 [Pangbourne]14 Use of SPHNs15 SPHNs were issued to all landowners for sites In London only where OPM was thought to occur (e.g. based on landowners in the buffer survey data from previous years) in the core or zone were issued with buffer zone, and elsewhere. SPHNs were also SPHNs where OPM issued for sites with host trees within 50m of the was previously found on infestation. sites, plus sites with host trees within 50m of these. Landowners in other outbreak areas were also issued with SPHNs. FC funding of controls London: Funding for controls was allocated by London: Funding for the FC on a case by case basis to landowners in controls was allocated both core and buffer zone. Funding was provided on a case by case basis for controls applied to infested trees and for host but restricted to trees within 50m of affected trees, and manual landowners in the buffer nest removal. zone only. Funding was provided for spraying of Croydon / Bromley: Funding was provided for infested trees (including spraying infested trees, plus host trees within those known to be 50m of affected trees, and manual nest removal. infested in 2013 and 2014) and host trees within 50m of affected trees, and manual nest removal.
Croydon / Bromley: Funding was provided for spraying all infested trees, plus host trees within 50m of these trees, and manual nest removal FC funded surveying In core (to a limited In core (to a limited In buffer zone in extent) and buffer zone extent) and buffer zone London only, and in in London, and in other in London, and in other other outbreak areas. outbreak areas. In 2013 outbreak areas. Fewer Reliance on landowners all oak trees in Bromley surveys conducted in to notify FC of OPM / Croydon were the north east quadrant infestations in core surveyed. of the core London zone in London. The outbreak. In Bromley / survey effort in the core Croydon only oak trees London zone was within a 100m buffer of restricted to a smaller a previously-infested number of sites, with tree were surveyed. fewer surveys being conducted in the north west / west of the outbreak. In Bromley / Croydon only oak trees within 100m of a previously-infested tree were surveyed.
14 No identified infested nests, but male moths caught in pheromone traps 15 SPHNs could only be issued where landowners were successfully identified. This was not always possible.
19 February 2016 10
Evaluation of the Oak Processionary Moth Control Programme - Final Report
2013 2014 2015 The 250 radius around pheromone traps in which five or more moths were caught was also surveyed.
No winter survey Winter survey Winter survey conducted conducted. conducted Surveillance Research into the use Use of pheromone traps Extensive pheromone of pheromone traps in increased (~75 in trapping programme the core zone in Pangbourne, 200 in (850 traps in total London, +50 traps London and across London, deployed in Croydon/Bromley) Croydon/Bromley, Pangbourne Pangbourne and corridors) Data management New field application Field application used, Reliance on field used for surveyors to but use of some written application / electronic allow for real-time data records persisted. data inputs only from input. Some written FC surveyors. Non-FC records still used. surveyors provided additional data in other formats. Communications Limited funding OPM targeted OPM communications available for OPM communications expanded and became targeted increased. more targeted, including communications. use of social media, weekly updates and targeting of specific stakeholder groups Protected zone status Not yet in place In place In place
19 February 2016 11
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure 2.1 High-level OPMCP logic model
Source: ICF International
19 February 2016 12
Evaluation of the Oak Processionary Moth Control Programme - Final Report
3 The evaluation approach
The evaluation design and approach was developed by following good practice guidance16 on the structuring of an evaluation. This involved the production of a structured evaluation design17 that defined the scope of the evaluation, a plan for what would be evaluated, when this would happen, and how the evaluation would be implemented. This was delivered to Defra and the FC as an evaluation design report in July 2015 and was accepted by the evaluation’s Advisory Group18. All elements of the evaluation approach were developed and discussed with input from the Advisory Group.
3.1 Evaluation questions Evaluation questions are used to test whether an intervention has achieved its aims and objectives. In the current context evaluation questions were used to explore whether the OPMCP has achieved the planned outcomes and impacts, and whether it has worked in the manner anticipated. A set of evaluation questions and sub-questions related to the key elements of the OPMCP was developed in consultation with the evaluation Advisory Group. The evaluation was not able to cover all of the evaluation questions, either because the data and information were unavailable or because other questions had to take priority in the design of the research tools and collection of information. Results for the omitted questions are therefore not presented in the sections below. An overview of the impact, process and learning questions, alongside the summary statements in response to these questions is provided at the beginning of each of the relevant sections. The full list of evaluation questions, alongside those questions that had to be excluded for the reasons stated above are given in Annex 4.
3.2 Counterfactual The counterfactual for the evaluation is a continuation of the status quo ex ante in which landowners continue to be served with Statutory Plant Health Notices (SPHNs) and are required to fund their own control measures. There are, therefore, ad hoc controls being implemented by landowners in the counterfactual scenario against which the OPMCP case is compared - the counterfactual is not a scenario of “no control”.
3.3 Evaluation methods The evaluation had five core research elements: ■ Consultation with stakeholders, involving semi-structured interviews with core stakeholders; and written responses to key evaluation questions gathered from additional stakeholders.
■ An online survey of landowners who have been involved with the pilot programme; ■ Desk-based research and analysis; and,
16 HM Treasury (2011), The Magenta Book: Guidance for evaluation, April, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220542/magenta_book_combined. pdf. 17 HM Treasury (2011), The Magenta Book: Guidance for evaluation, April, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220542/magenta_book_combined. pdf. 18 The evaluation Advisory Group comprised representatives of Defra (Andrea Deol, Jake Morris, Amanda Mitchell, Jemilah Vanderpump, Jonathan Bonas) and the FC (Andrew Hoppit, Andy Hall, Craig Harrison).
19 February 2016 13
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ Quantitative modelling to provide a counterfactual, by predicting the rate of OPM’s spread in the absence of the OPMCP. ■ GIS mapping of the human population in the area colonised by OPM The sections below provide more information on each of these elements. Additional detail is provided in Annex 5. Where results from the stakeholder consultation and landowner survey are reported in the text that follows, “consultees” refer to those stakeholders who were interviewed and from whom written responses were collected. “Respondents” refer to landowners responding to the online landowner survey.
3.3.1 Stakeholder consultation – interviews with core stakeholders The purpose of the semi-structured interview programme was to gather evidence that would: ■ Inform answers to the full suite of process and learning evaluation questions; ■ Inform the evaluation’s answer to the question “Did the OPMCP impact on preparedness and contingency planning for other similar plant pests, and for plant pathogens / diseases more generally?” (EQ2); and, ■ Supplement the data gathered for the other impact evaluation questions with expert judgements, especially where data or other sources of information are scarce/limited. Semi-structured interviews were conducted with 20 consultees. A list of the stakeholders that were interviewed is provided in Annex 4. The list of interview questions is supplied at Annex 7.
3.3.2 Stakeholder consultation – written responses gathered from additional stakeholders The purpose of collecting written responses to certain evaluation questions from additional stakeholders (i.e. people other than those who were interviewed) was to: ■ Expand the body of evidence available to inform answers to some of the impact, process and learning evaluation questions; ■ Gather views from additional stakeholders from a wider range of organisations, including organisations on the OPM Advisory Board that are not directly involved in the main control programme; and ■ Ensure that there was a greater level of buy-in and involvement from other stakeholders in the evaluation process. The design phase identified a number of additional stakeholders whose input would add value and help increase confidence in the findings, but who were not considered essential to the evaluation. The evaluation resources also did not allow for these stakeholders to be interviewed so they were instead asked to submit written responses to a core set of priority questions (Annex 8). Nine responses were received (Annex A6.3 shows who was contacted and who responded).
3.3.3 Online survey of landowners The design phase of the evaluation identified that a survey of landowners affected by OPM or the OPMCP would be useful to gauge views on process and effectiveness, and levels of engagement. It would provide an alternative, third party view on the issue of stakeholder engagement, and how effective the communications under the OPMCP have been in improving access to, and the implementation of, control activities. The survey would also examine the impact that the availability or absence of funding had on the willingness of landowners to carry out control activities.
19 February 2016 14
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Surveys were developed to address selected evaluation questions. Separate online surveys were developed for landowners representing single sites, and those responsible for multiple sites. Landowners responsible for multiple sites were able to select multiple responses where situations across sites differed (e.g. control measures might have been paid for by the Forestry Commission on one site, but not on another). Details on the type of landowners responding to the survey, and their experience with the OPM and the OPMCP are provided in 0. In total, 447 landowners were contacted. One hundred responses to the single site survey were received and 21 responses to the multi-site survey, providing an overall response rate of 27%. More details on the administration of the survey is provide in Annex 5. Quantitative figures and charts provided in the text that reference the landowner survey are for the survey of single site representatives only. Percentages quoted are based on the full sample of people responding to the survey (i.e. 100 responses), not the number of landowners answering a specific question (this is quoted in brackets as N=X). The survey included ‘routed’ questions which were tailored to landowners who had selected specific responses to previous questions. Not all landowners were therefore asked the same questions, and therefore the number of landowners answering some questions varied from the total number of respondents. Respondents were also given the opportunity to provide comments in a free text box at the end of the survey. The number of respondents to the multi-site survey (at 21) was too small for quantitative analysis to be appropriate. Care is needed in extrapolating results beyond the sample of landowners who responded to the survey. The responses of landowners who replied to the survey are not necessarily representative of the entire population of landowners that have engaged with the OPMCP. For example, responses may be more likely to be supplied by landowners who have had a more positive experience, been more heavily engaged, or had controls funded. With the data available to the evaluation such response biases cannot readily be detected and controlled for. The evaluation team had hoped to provide more detailed analyses of the responses, by disaggregating the responses by the type of funding provided19 and the type of ownership.20 However, due to the small sample sizes, it was not possible to conduct this level of analysis, as the extent to which sampling bias was influencing the results was unclear. Also, due to resource constraints, the analysis of the survey results is limited to descriptive analysis. The analysis does not include any weighting of the results. It is not, therefore, possible to account for any implicit bias in the results based on the type of stakeholder providing the response. Consequently, care is needed in extrapolating from the findings presented here to landowners as a whole.
3.3.4 Desk based research and analysis The desk-based research and analysis contributed indicators of programme activity in the pilot programme that are helpful to the description of what has occurred, such as: ■ Reports of OPM nests; ■ Number of controls applied; ■ Location of controls; ■ Survey effort and results (extent of the outbreak)
19 i.e. whether landowners have always had their control measures funded by the FC, have always had to pay for control measures to be carried out, or whether funding had either recently been withdrawn or made available; 20 i.e. whether the site was owned by a private homeowner; private company, a public body or a charity
19 February 2016 15
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Desk research was expected to support the responses to specific evaluation questions (see Annex 5. There were three elements to the desk research: ■ A review of programme documentation (e.g. annual reports, Project Board and Advisory Group meeting minutes, outputs from the lessons learned stakeholder workshops) and other related information (e.g. financial information on annual spend; scientific research). ■ Further investigation of the experience of other EU countries with OPM, involving interviews with Mr. Luc Crevecoeur (Provinciaal Natuurcentrum, Belgium), Mr. Henry Kuppen (Terra Nostra, Netherlands), and Mrs. Marianne Klug (Nordrhein-Westfalen Chamber of Agriculture, Germany) and additional desk research on the impacts of OPM in Belgium, the Netherlands and Germany. ■ An analysis of data from the OPMCP database relating to surveys and control measures carried out across the different sites in the OPM infested areas and beyond. Caveats to the use of this data are presented in the box below.
Caveats to the use of data from the OPMCP database ■ The OPMCP data strategy, the data that were collected and the database that held them, were designed to meet operational needs, rather than to provide information that would be useful for programme monitoring, evaluation or research purposes. Large quantities of data are available but they are not well suited to answering many of the evaluation’s questions about programme impact. Data have been extracted and analysed so as to make the best possible contribution to the evaluation but results are not as clear as they might otherwise have been. The kind of data being collected also varied from year to year, which made comparison over time and interpretation of some of the key variables more difficult as other factors are likely to be influencing the observed trends.
■ The data presented relate to the observed, rather than actual, extent of the OPM infestation and impacts of the OPMCP.
■ More specific caveats to the data are: – The extent and distribution of the survey effort changed from year-to-year, therefore some of the apparent change in the scale of infestation is likely to be related to the fact that a larger area was surveyed. – The core area was not systematically surveyed in 2015 and therefore the infestation figures for 2015 are likely to be underestimated. – Reports on the number of trees / sites surveyed do not include sites that the surveyor could not get access to. osts and treatment data only relate to the sites that were funded by the FC control programme. – The number of nests and trees cited are the numbers reported by the control contractors – these will differ from the numbers reported by the surveyors (for example, once a surveyor has found a nest s/he will move onto the next tree without spending a lot of time counting nests, whilst the contractor will count the number of nests). – Changes in the funding between years means that some sites were funded by the FC in one year and then not in other years. – There were changes in the way that costs were allocated between years which makes comparison more difficult – in 2013 and 2014 all administrative costs were charged to a single ‘virtual site’, in 2015 costs were allocated to each individual site. – Different contractors charged different rates for control work, therefore the cost of treating a single site in a single year is dependent on who did the work, and the total costs vary depending on the relative allocation of sites to contractors. – The variability in the physical environment of each site also affects cost, e.g. sites that were hard to access, or needed pre-spraying clearance tended to be more expensive. – There are some inconsistencies in the contractor data. For example, there are some records that trees have been removed from sites, but no associated costs. There may be some inconsistencies, therefore, between total number of sites treated between queries. – The variability of the contractors, sites and data processing suggests that no firm conclusions can be drawn about the cost effectiveness of control from the data.
19 February 2016 16
Evaluation of the Oak Processionary Moth Control Programme - Final Report
– It was not possible to reliably compare 2014 treatment and 2015 infestations because sites within the core zone treated in 2014 were not surveyed in 2015. As such, it is not known whether the 2014 treatment was successful, or whether these sites remained infested.
3.3.5 Quantitative modelling of the spread of OPM The impact of the OPMCP on the spread of OPM cannot be determined by comparing the change in distribution of the moth under the programme’s control regime with that of identical outbreak without the same controls – there is no alternative London to provide that reference case. This problem was resolved by constructing a counterfactual, i.e. by modelling what would have happened under different scenarios, including a scenario in which there was no OPMCP. The modelling work involved: ■ Collating the available site level information and using this to calculate demographic parameters (survival with and without control, and distance-dependent colonisation); observation parameters (detection probability); and sites controlled (proportion of sites controlled). Parameters were calculated individually from specific subsets of the data. ■ Constructing a simulation model with the calculated parameters and using this to predict the spread of OPM from 2012 onwards. The sensitivity of the simulation model to variation in the parameters was then tested. The modelling was intended to support the evaluation’s response to the question, “Has the OPMCP supported the delivery of the policy goal of containment of the oak processionary moth within the main outbreak area and eradication elsewhere?” (EQ1) by providing evidence on the following indicators: ■ The difference in rate of spread of the Bromley outbreak, comparing the observed and modelled current situation to the modelled counterfactual; ■ The difference in the rate of spread of the south-west London outbreak, within selected geographical segments, comparing the observed and modelled current situation to the modelled counterfactual. Evidence from Pangbourne was brought to bear on EQ1 without the use of modelling. It ultimately proved impossible to use the model to construct a counterfactual as had been hoped due to various limitations of the data (discussed in detail below). Although the data were sufficient to parameterise the necessary variables, goodness-of-fit tests and the sensitivity analysis suggested that the model was particularly sensitive to the estimate of effective dispersal (i.e. colonisation of new sites). This meant it was not possible to use the current model to robustly construct a counterfactual and predict the potential spread of OPM in the absence of the OPMCP. The model was instead used to estimate the sensitivity of the outputs to the parameter estimates and to make recommendations for future modelling work. The parameters in the model were altered one-by-one and the impact on the predicted rates of spread assessed. A brief overview of the approach is described in Annex 5, with further technical detail provided in Annex 9. Recommendations and further research needs identified from the modelling exercise are discussed in Section A9.5. In addition to constructing a counterfactual, the model was also used to develop a systematic and repeatable methodology to quantify observed spread of OPM (rate of spread, area covered and human population affected).
19 February 2016 17
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Caveats to the modelling work ■ The data on OPM infestation were collected for operational reasons to support the OPMCP, rather than for monitoring of distribution and prevalence. These data were subject to potential bias (towards well-surveyed regions) and small sample size. ■ Parameters were based on ‘sites’ not ‘trees’, but sites are defined operationally (e.g. based on land ownership) rather than ecologically. Some large sites (e.g. Richmond Park) are probably better represented as multiple sites in the model. The local density of ‘sites’ appears to have a large influence on the modelled results and varied across the region of infestation (e.g. the Croydon region contains one-third of the sites in only 2% of the total area of infestation). ■ The parameters were estimated from subsets of the available data. Investing in a combined model to simultaneously estimate the parameters could lead to more rigorous estimates of these parameters, especially effective dispersal (distance-dependent colonisation). ■ The distribution of oak trees is not known well enough to support the prediction of OPM spread beyond the currently surveyed area. ■ Control is currently applied randomly in the model, whereas systematic application of control to a site could be more effective than random control. ■ There is no effect of site size in the model because resource limitations did not allow this to be considered. ■ The model does not account for OPM abundance in a site because the data were not suitable for estimating abundance and it was beyond the remit of the project. All our conclusions are based on spread (site colonisation and extinction) even though reducing abundance of OPM at a site may be a valuable outcome.
3.3.6 GIS mapping of the population in the area colonised by the oak processionary moth The GIS mapping exercise addressed the evaluation question (EQ3) about the programme’s impacts on public, occupational and animal health, focusing in particular on the public health impacts. The human population in the area known to be currently colonised by the OPM, and in the area predicted to have been colonised in the absence of the OPMCP was estimated using GIS mapping. The latter was calculated based on expert judgement that OPM would have spread about 5km per year in the absence of a coordinated control programme, where treatment was left to the ad hoc and spontaneous action of landowners.21
21 Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment.
19 February 2016 18
Evaluation of the Oak Processionary Moth Control Programme - Final Report
4 Impact evaluation findings
4.1 Introduction The sections below present the findings for the impact evaluation questions, specifically on the: ■ Policy impacts: – EQ1: Has the OPMCP supported the delivery of the policy goal of containment of the oak processionary moth within the main outbreak area and eradication elsewhere? – EQ2: What impacts has the OPMCP had on preparedness and contingency planning for similar plant pests, and for plant pathogens / diseases more generally?
■ Social impacts: – EQ3: What impact has the OPMCP had on public, occupational and animal health (i.e. through the diagnosis and treatment of OPM related conditions)? – EQ4: What impact has the OPMCP had on public recreation? – EQ5: What impact has the OPMCP had on the level of public confidence that OPM is being adequately addressed?
■ Environmental impacts – EQ9: What impact has the OPMCP had on oak tree health? – EQ10: What impact has the OPMCP had on biodiversity? An overview of the summary statements for each of the evaluation questions is given below. The analysis and supporting evidence is provided in the detailed responses in the sections that follow. Table 4.1 Summary statements for the impact evaluation questions
EQ# Evaluation Summary statement question Impact evaluation questions Policy impacts EQ1 Has the The OPMCP supported the delivery of government policy to contain OPM within OPMCP the main outbreak area of south-west London and to eradicate it elsewhere. supported the Overall, the evidence suggests that OPM has spread less fast than predicted in delivery of the the counterfactual scenario of no FC funding of controls. The OPMCP is also policy goal of likely to have made a significant contribution to reducing the size of the outbreak containment of area in Croydon / Bromley, and no new OPM nests have been found in the oak Pangbourne. For 2014 and 2015 estimated rates of spread for south-west processionary London have been higher than the goal of 1km per year and full eradication in moth within the other areas of infestation has not yet been achieved, suggesting that more main outbreak would need to be done to deliver the current policy on OPM. area and eradication elsewhere EQ2 What impacts The OPMCP has demonstrated the need for early detection of new pests and a has the quick response once an infestation is identified. This is influencing institutional OPMCP had responses to new pests. There is potential for the OPMCP to further enhance on preparedness and contingency planning on the basis of what has been learned preparedness through the pilot programme, including how to prepare for the application of a and holistic approach of the kind adopted by OPMCP. contingency planning for similar plant
19 February 2016 19
Evaluation of the Oak Processionary Moth Control Programme - Final Report
pests, and for plant pathogens / diseases more generally? Social impacts EQ3 What impact The number of people living in an area with OPM has increased by 1.3 million to has the 1.8 million between 2012 and 2015, based on what is known about OPM OPMCP had distribution. on public, occupational It is estimated that in 2015 3.5 million fewer people were living in an area with and animal OPM than would have been the case under the counterfactual scenario (i.e. no health (i.e. FC funding for controls, with landowners independently implementing ad hoc through the treatment measures). The OPMCP is likely to have contributed to this diagnosis and reduction, thought the size of that contribution cannot be quantified. treatment of OPM related The probability is that the control programme has reduced the health impacts conditions)? resulting from OPM by reducing the size of the OPM population compared to what it would have been in the absence of coordinated control measures, (i.e. its abundance) and its distribution. It is, however, difficult to quantify the scale of this impact as evidence of health impacts in all areas is lacking. Consultees felt that limiting the population density of OPM would have a greater impact on health impacts than limiting its spread. No data are available on the changes in abundance of OPM within the outbreak areas. There is no systematic evidence of animal health impacts, and only very limited anecdotal evidence could be identified. EQ4 What impact Consultees felt that the OPMCP had limited the impacts of OPM on public has the recreation and public access by suppressing abundance. Anecdotal evidence OPMCP had suggests that an abundance of OPM can lead to restrictions on the use of some on public public spaces at certain times of the year, though there is no systematic recreation? evidence of significant impacts of OPM on public access and recreation. The OPMCP may also have had an indirect positive effect by sustaining public confidence in the safety of green spaces. Consultees suggested that the measured approach taken to raising awareness of the health impacts of OPM in OPMCP communications has helped to ensure that there has been minimal impact on people’s use of green spaces. No data are available on the effect of the OPMCP on the population density of OPM so it is not possible to determine what direct impacts on public access may have been avoided due to the control programme. Biodiversity impacts EQ9 What impact The available evidence suggests that impacts of OPM on oak trees in the UK has the have been limited during the pilot programme period. How much of this can be OPMCP had attributed to the efforts of the OPMCP is unclear. Consultees varied in both their on oak tree views on the severity of the potential problem and the tree health benefits of the health? pilot programme. Some consultees indicated that the OPMCP had reduced the potential impact on oak trees by keeping OPM numbers and spread in check. Others were less sure of the impact the OPMCP has had on oak tree health. Overall, however, there is insufficient evidence to determine the impact of the OPMCP on oak tree health as there is very little systematic evidence on the scale of the impact that OPM has on oak tree health and no data are available on the population density of OPM, either before or since the OPMCP. EQ10 What impact The use of pesticide spraying under the OPMCP can, and most likely has, has the affected biodiversity however it is not clear what mix of species have been OPMCP had affected, to what extent, and for how long these impacts have lasted. Biodiversity on could have been more affected by the impacts of OPM (as a result of on oak tree biodiversity? defoliation and a potential decline in oak tree health) if the spread and abundance of OPM had increased in the absence of the OPMCP, however the scale of these potentially avoided impacts is unknown. The control agent applied
19 February 2016 20
Evaluation of the Oak Processionary Moth Control Programme - Final Report
by the OPMCP operatives is less damaging to non-target species than the alternatives that were sometimes used by landowners before the pilot programme started. The programme may also, by funding control measures, have led to fewer oaks being felled by landowners looking to avoid recurrent costs of OPM control. Overall it is not possible to say whether the OPMCP has had a net negative or net positive impact on biodiversity.
4.2 Policy level
4.2.1 EQ1. Has the OPMCP supported the delivery of the policy goal of containment of the oak processionary moth within the main outbreak area and eradication elsewhere? Summary statement: The OPMCP supported the delivery of government policy to contain OPM within the main outbreak area of south-west London and to eradicate it elsewhere. Overall, the evidence suggests that OPM has spread less fast than predicted in the counterfactual scenario of no FC funding of controls. The OPMCP is also likely to have made a significant contribution to reducing the size of the outbreak area in Croydon / Bromley, and no new OPM nests have been found in Pangbourne. For 2014 and 2015 estimated rates of spread for south-west London have been higher than the goal of 1km per year and full eradication in other areas of infestation has not yet been achieved, suggesting that more would need to be done to deliver the current policy on OPM. Analysis: Government policy on OPM is to contain the pest within the main outbreak area of south- west London and to eradicate it elsewhere. In 2015, looking beyond the south-west London infestation, there were known populations in Pangbourne, Croydon / Bromley, the Olympic site in east London and Guildford (identified in 2015). In 2014, the goal of containment for the main outbreak area was refined to limiting the spread of OPM to less than 1km per year. The OPMCP was established to support the delivery of government policy on OPM. It was hoped that this pilot programme would buy time until the pest could be better understood and an appropriate and more permanent solution for the future could be identified. The evidence available to answer this question consists of: - Maps of the known distribution and extent of the OPM population (see Annex 11); - Estimates of rate of spread based on data collected during the OPMCP; - Stakeholders’ views and expert opinion; and, - Expert judgement on the predicted rate of spread in the counterfactual scenario (the model was unable to provide a constructed counterfactual predicting the rate of spread and distribution of OPM in the absence of the OPMCP). ■ Containment in south-west London Since the end of 2014, the policy goal in south-west London has been to limit the spread of OPM to less than 1km per year. It is very difficult to determine with confidence whether this policy goal has been met, given the variation in survey data collected between 2013 and 2015. Survey efforts have been dictated by operational needs, and the same area has not been surveyed consistently over time. This has meant that calculating a consistent and meaningful rate of spread is challenging. The evidence suggests the spread recorded under the OPMCP has been less than projected under the counterfactual, although the spread has been higher than the goal of 1km per year. Maps showing changes to the known distribution of OPM over time are provided in Annex 11. Measures of spread, including the increase in geographical extent (i.e. increase in total km2 over time) and rates of spread (km spread per year) have been calculated using both a
19 February 2016 21
Evaluation of the Oak Processionary Moth Control Programme - Final Report
convex hull and alpha hull method. These are shown for each of the three years in Table 4.2. More detail of the three methods is provided in Annex 12. The analysis indicates that the geographical range of OPM has increased considerably since 2013. The proportionate increase was slightly smaller in 2015 than 2014 (a near 40% increase in its geographical extent compared to an approximate 50% increase in 2014, under both methods). The convex hull method suggests a rate of spread of OPM close to the 1km target rate in each year, up until 2015. The rate of spread is somewhat higher if calculated using the alpha hull method. Both methods suggest that the spread of OPM was contained to less than 1km per year in 2013. For 2015 there is more variation between the results generated by the two methods – a spread of over 2km per year on the alpha hull method and a rate of spread close to 1km per year on the convex hull method. This discrepancy is a result of the differences in how the two methods treat outliers in the survey data.22 Which of these methods provides a better estimate of spread depends on whether these outlying records are considered to be the edge of the current distribution (in which case the alpha hull method is likely to be more accurate) or if they are assumed to be isolated incidents and not part of the actual spread (in which case the convex hull method is more appropriate). From the survey data gathered in 2015 (see Annex 11) it is not possible to determine one way or the other which method provides the better estimate. It is not possible to determine how many of the new sites of infestation identified in 2015 had been present before but not spotted as opposed to being new points of infestation that would indicate OPM was spreading more quickly. Figures of spread have been calculated on the assumption that the new outbreaks identified in 2015 (e.g. in east London and Guildford) are not linked to the main outbreak area, as their origin is yet to be determined. The rates of spread also do not include other outbreak areas such as Croydon / Bromley. Table 4.2 Calculated rates of OPM spread for the south-west London outbreak
2013 2014 2015 Increase in geographical extent (total km2 (% increase on previous year)) Using the convex hull method 31.8 (17% ) 109.1 (50% ) 141.1 (43% ) Using the alpha hull method 44.0 (33% ) 95.0 (54% ) 116.0 (43% ) Rate of spread (km / year) Using the convex hull method 0.9 1.0 1.1 Using the alpha hull method 0.9 1.6 2.4 The estimated rates of spread under the OPMCP are considerably lower than the anticipated spread of OPM under the counterfactual scenario. Forest Research (FR) has estimated that in the absence of FC funded control and with some landowners independently implementing their own controls, OPM’s rate of spread would have been 5km per year between 2013 and 2017.23 This figure assumes that 10% of nests would be independently destroyed through
22 The convex hull method provides a good measure of spread if OPM is spreading evenly. However, isolated individual records beyond the current distribution have little impact on this estimate of spread. The alpha hull method gives a very similar result if OPM is spreading evenly. However, its estimates diverge from the convex hull method when there are isolated records beyond the current distribution. If those records are further than the value of alpha used in the alpha hull approach then they are treated as isolated islands. However if they are within the distance of alpha, then the method extends the current area of distribution to those points. The reason that this discrepancy becomes pronounced in 2015 is because there are many isolated records of OPM infestation to the east of the west London outbreak. 23 Based on Forest Research estimate and evidence since 2006, as quoted and used in Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment. Predicted rates of spread, where the only intervention that occurs is landowners choosing to independently implement control measures on their own property, are 5km/yr for 2013-2017, 7.5km/yr for 2018-2022 and 10km/yr thereafter. The maximum is based on European experience of OPM & US experience of gypsy moth. It is assumed that 10% of nests would be independently destroyed through spontaneous action by landowners.
19 February 2016 22
Evaluation of the Oak Processionary Moth Control Programme - Final Report
spontaneous action by landowners. FR estimated that the rate of spread would increase over time to 10km per year by 2022. The difference in the observed extent of OPM between 2012 and 2015, and the predicted rate of spread under the counterfactual is shown in Figure 4.1. This is based on an assumption that the spread will be the same in all directions. It was hoped that the rate of spread of OPM in the absence of the OPMCP could be constructed but due to data limitations this was not possible (see Section 3.3.5). Figure 4.1 Observed and estimated rate of spread of OPM under the OPMCP and the counterfactual
The numbers in red indicate the modelled annual rate of spread in km (i.e. the line labelled 10 shows the 30km buffer for a rate of spread of 10km yr-1 for three years). Source: CEH analysis The observed extent of OPM in 2012 (prior to the OPMCP being initiated) The observed extent of OPM in 2015 since the OPMCP (incl. new outbreaks) The predicted spread of OPM under different rates of spread since 2012, including the estimated rate of 5km under the counterfactual The relative influence of the pilot programme and of other factors, such as climate, on the rate of spread cannot readily be determined from the data. The cold winter of 2013/14 is likely to have naturally suppressed the OPM population going into 2014. More resource were dedicated to surveying in the buffer zone and beyond in 2015 than in previous years. Almost all of the consultees believed that the OPMCP has ensured that the spread of OPM has been lower than it would have been in the absence of the control programme. However, many noted that it was difficult to quantify the extent of the impact.
19 February 2016 23
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“OPM is highly likely to have spread further and faster [without the OPMCP]…The details of this spread and how and when it spread is however not wholly clear.” The difficulties of detecting OPM and the lack of knowledge about the full extent of OPM’s distribution repeatedly came up in discussions with consultees. Several noted that not knowing the outer extent of the OPM infestation has meant that the impact of the OPMCP on OPM distribution is unclear. A few felt that, given this lack of knowledge, it was not possible to say whether the OPMCP has been successful in limiting the spread of OPM. “In London the problem cannot be deemed as contained as the outer limit is not known.” “It’s difficult because we only know if it’s there if we’ve looked for it, and we don’t know where it was to start with before this process began… It’s difficult to say [if the OPMCP has been successful], it might have been. If we’d have been able to survey everywhere we would have known. Because we haven’t been able to, we don’t.” ■ Eradication elsewhere The policy goal of eradication applies to areas of OPM infestation outside south-west London. In the period under consideration, this included Croydon / Bromley, the Olympic site and Pangbourne. Additional areas of infestation were only identified late in 2015 and so were not subject to any controls during the pilot programme period. The outbreak in Pangbourne was detected and addressed before the OPMCP started. The OPMCP has monitored the situation in Pangbourne and carried out precautionary, rather than reactive, control measures. No new nests or sites of infestation have been found in Pangbourne since the OPMCP started, suggesting that control efforts implemented prior to the OPMCP were successful in significantly reducing the OPM population. The pheromone trapping programme has, however, captured male moths in the Pangbourne area. This could suggest a residual OPM population even though no nests have been detected. Alternatively the male moths may have arrived from elsewhere; they are able to fly significant distances. No breeding population and no female moths or nests have been found but it cannot yet be determined with certainty whether eradication in Pangbourne has been achieved. The OPMCP appears to have had a considerable impact on the extent of the OPM’s local range in the Croydon/Bromley area. Eradication has not been possible to date, suggesting that more would need to be done to meet this policy goal. No systematic data were collected on the scale of the outbreak when the outbreak was originally reported, but anecdotal evidence suggests that it was significant. “When the outbreak was reported, we think it had been there for at least 3 years. The site was about 250 acres, with over 5,000 oak trees. I think we counted at least 4,000 nests to begin with until we just stopped counting. After two years of concerted effort, we had knocked those numbers down by a huge amount.” The outbreak in Croydon / Bromley was detected much later than the one in Pangbourne. This meant that OPM had had much more time to become established and the area of infestation was much larger by the time control measures were applied. Control measures were restricted to ground-based spraying, whereas in Pangbourne aerial spraying was used (ground spraying is less effective than aerial spraying). Deltamethrin, a more effective (but also broader spectrum) pesticide, was also used in Pangbourne while Bt and dimilin were used in Croydon / Bromley. Access to some sites in Croydon / Bromley was also more restricted than to those in Pangbourne. All these factors contributed to making eradication in Croydon / Bromley much more difficult than in Pangbourne. The number of infested trees and identified nests over the programme period is shown in Table 4.3. The number of infested trees has reduced from 667 in 2013 to 289 in 2015, but the number of identified nests increased over the same period. The number of OPM nests
19 February 2016 24
Evaluation of the Oak Processionary Moth Control Programme - Final Report
identified is likely to be an understated.24 The scale of the survey effort changed from year- to-year so some of the apparent change in infestation is likely to be related to the fact that a larger area was surveyed. Overall, the Croydon outbreak seems to be have been contained more successfully than that in south-west London. The area of infestation reduced in 2015 (Table 4.4). The extent to which other factors aside from the OPMCP (e.g. weather, natural population dynamics) may have contributed to this observed change is unclear. Table 4.3 Number of infested trees and OPM nests identified in Croydon / Bromley
2013 2014 2015 # of infested trees 667 349 289 # of OPM nests25 236 431 451 Table 4.4 Extent of the known distribution of OPM in other outbreak areas over time (total km2)
Increase in total km2 2013 2014 2015 Croydon / Bromley 1.66 9.52 -2.37 Olympic site26 Not calculated 1.26 An OPM outbreak was identified at the Olympic site in 2014. Eradication efforts have not been fully successful (OPM was detected in 2014 and 2015) but the number of nests found was very small (two nests were found in a single tree in 2014, and eight nests were found in eight different trees in 2015). It is unclear why eradication at this site has been more difficult. Factors may include the restrictions on what control methods can be used in the area and problems of detection. It also appears that re-infestation might have occurred not because treatment was unsuccessful but because new, infested trees were planted on the site, despite FC advice being given on the matter. Annex 13 provides details of the number of infested sites, trees and nests found between 2012 and 2015 for the different sites of infestation, including London, Croydon/Bromley, the Olympic site and Pangbourne.
4.2.2 EQ2. What impacts has the OPMCP had on preparedness and contingency planning for similar plant pests, and for plant pathogens / diseases more generally? Summary statement: The OPMCP has demonstrated the need for early detection of new pests and a quick response once an infestation is identified. This is influencing institutional responses to new pests. There is potential for the OPMCP to further enhance preparedness and contingency planning on the basis of what has been learned through the pilot programme, including how to prepare for the application of a holistic approach of the kind adopted by the OPMCP. Analysis: Through its investment and experience in management structures, control and survey work, data management and research, it was hoped that the OPMCP would develop FC capabilities in pest control management, governance and programme management,
24 Based on values reported by the surveyors, not control contractors. Once surveyors have identified that a tree is infested, they tend not to spend time counting exactly how many nests are present. The level of underestimation per site will depend on the number of trees on a site and the number of nests in a tree. 25 Ibid. 26 The area in 2014 was not calculated as there was only one site. The increase in 2015 is therefore assumed to be from 0km2 in 2014.
19 February 2016 25
Evaluation of the Oak Processionary Moth Control Programme - Final Report
research, stakeholder engagement and pest control that could enhance its capacity to monitor and manage other pest outbreaks. The evidence points to the OPMCP having had an influence on the nature of institutional responses to new pests. There is also the potential for the lessons learned through the OPMCP to ensure that the FC and/or Defra are better equipped to deal with outbreaks of other pests and diseases, and that capacity or capabilities for doing are enhanced, although no direct evidence of this was identified during the research period. Given that the pilot programme was still on-going at the time of the research, this is not surprising. The evaluation did find evidence that the OPMCP had supported the case for prompt intervention in cases of outbreaks of new pests, including a recent outbreak of the chestnut gall wasp. When prompted on the OPMCP’s impacts on preparedness and contingency planning consultees made reference to the way it had: ■ demonstrated the importance of responding to an infestation swiftly and with sufficient resources; and, ■ highlighted the need for early detection. “The control programme has given us an impetus to say, we have to get in there and deal with it quickly. For instance, recently an outbreak of chestnut gall wasp was dealt with really quickly on a small scale. [The OPMCP] has removed the ‘stop and wait and see’ kind of approach before we are able to decide what to do.” Some consultees noted that the OPMCP has provided lessons in how to create an effective management infrastructure and processes for stakeholder engagement that can be applied in other areas of pest and disease management. Other ways in which OPMCP has helped improve preparedness and contingency planning included raising plant health up the political agenda, evidencing the importance of using scientific research and expert knowledge and demonstrating the need for a coordinated, landscape scale approach. “[The OPMCP] has considered the whole thing, in the round. It has looked at treatment, control, communications, an external advisory body, an internal steering group. The infrastructure is in place to know what’s worked and what hasn’t. You can use that experience in other situations.” “It provides a good model for how to engage and bring relevant players on board.” “To me it has highlighted the importance of expert scientific advice, without which, management tools are a blunt instrument.” Some consultees were unsure about the OPMCP’s contribution to contingency planning and preparedness, or did not think the OPMCP has had any impact. “I’m not sure it’s gone either way. We tend to deal with these things on a case by case basis. I wouldn’t think that the experience with [OPM] makes us any more or less prepared to deal with other outbreaks.”
4.3 Social impacts
4.3.1 EQ3. What impact has the OPMCP had on public, occupational and animal health? Summary statement: The number of people living in an area with OPM has increased by 1.3 million to 1.8 million between 2012 and 2015, based on what is known about OPM distribution. It is estimated that in 2015 3.5 million fewer people were living in an area with OPM than would have been the case under the counterfactual scenario (i.e. no FC funding for controls, with landowners independently implementing ad hoc treatment measures). The OPMCP is
19 February 2016 26
Evaluation of the Oak Processionary Moth Control Programme - Final Report
likely to have contributed to this reduction, thought the size of that contribution cannot be quantified. The probability is that the control programme has reduced the health impacts resulting from OPM by reducing the size of the OPM population compared to what it would have been in the absence of coordinated control measures, (i.e. its abundance) and its distribution. It is, however, difficult to quantify the scale of this impact as evidence of health impacts in all areas is lacking. Consultees felt that limiting the population density of OPM would have a greater impact on health impacts than limiting its spread. No data are available on the changes in abundance of OPM within the outbreak areas. There is no systematic evidence of animal health impacts, and only very limited anecdotal evidence could be identified. Analysis: Concern about the potential public health impacts of OPM was a central component of the OPMCP investment case. Very little was known at the time about the potential extent and scale of the health impact, although there was information available on the type of symptoms caused by OPM. There is now a better understanding of the scale and nature of the risk to human health, thanks to a systematic review of the evidence by PHE and to more experience being gained with OPM and qualitative, anecdotal observations of its impacts. However, there is still little hard evidence on the issue, and no specific research or investigations have been done to explore this issue. The available evidence is limited to a small amount of research conducted in mainland Europe, reports to the FC of OPM health impacts, and anecdotal evidence. For the purposes of this evaluation, discussions were held with experts in mainland Europe to identify further information on the health impacts of OPM but this yielded little additional evidence. Overall, it seems likely that the OPMCP has had a positive impact on reducing health impacts by reducing the size of the OPM population as compared to what it would have been in the absence of coordinated control measures, both in terms of its distribution and in terms of its abundance. Most of the stakeholders consulted were confident that the OPMCP has reduced the density of OPM populations in the areas of infestation. As such, health impacts are likely to be lower than they would have been in the absence of a coordinated control response. It is, however, difficult to quantify the scale of this benefit given the lack of evidence that can be confidently applied to the UK context. Consultees generally felt that limiting the population density of OPM would have had a greater impact on health impacts than limiting its geographical spread. However, no data are available on the changes in abundance of OPM within the outbreak areas. “The programme has reduced the numbers of moths and therefore reduced the potential harm to the public.” “That’s a success of the control programme that is hard to report – when you see the spread you’re not seeing the impact of limiting the density and that’s likely to be having a bigger impact on reducing human health impacts than limiting spread.“ The evidence suggests that for most people exposure to OPM larvae results in relatively minor, if unpleasant, health impacts.27 Much of the evidence on the occurrence of OPM- related public health impacts is anecdotal, both for the UK and in mainland Europe (see below). In the UK, localised health impacts have been recorded in some schools and several
27 Contact with OPM hairs may cause skin, eye and respiratory irritation (Gottschling et al., 2007). The impacts only last a short while and are self-limiting. Extreme reactions seem to only occur in a small percentage of the population, but sensitivity may increase rapidly with repeated exposure (Townsend, 2009). It is unclear what makes some people more sensitive to OPM than others. However, arboriculturalists and others who work in and amongst oak trees are more at risk of exposure and therefore also have a greater risk of becoming sensitised to OPM hairs.
19 February 2016 27
Evaluation of the Oak Processionary Moth Control Programme - Final Report
individuals have reported health impacts to the FC.28 Additional evidence on health impacts is provided in Annex 14. Substantive evidence on the scale of health impacts in mainland Europe is scarce, though some information is available. Moran et al. (2015) estimated the annual costs of OPM health impacts in the Netherlands at €25,000 to €333,000 for GP visits, €72,000 to €960,000 for work absences, and €2,000 to €29,000 for hospital admissions - €99,000 to €1,322,000 in total. Spending on the control programme in 2011 was between €6.6m and €9.6m. The UK Government’s impact assessment of an OPM control programme estimated that under the counterfactual scenario (i.e. no FC funding for control, where landowners continue to independently implement ad hoc treatment), the human health costs in the UK due to GP visits and lost work time would be around £290m in total, undiscounted, over 40 years (£126m discounted). In 2015 the total number of people living within an OPM infested area in south-west London and Croydon / Bromley was 1.8 million, an increase from 1.0 million in 2014, 0.7 million in 2013 and 0.5 million in 2012. Under the counterfactual scenario (i.e. no FC funding for controls, with landowners independently implementing ad hoc treatment measures), it is estimated that 5.3 million people would have been living in an area infested with OPM by 2015, which is 3.5 million more than what is estimated with the OPMCP having been in place. This is based on an assumed spread of 5km per year, compared to the current estimated rates of spread calculated using the alpha hull method (see Section 4.2.1). It is unclear what proportion of people living in an OPM infested area and therefore exposed to OPM would develop health symptoms, as no systematic evidence on this could be identified. The government’s impact assessment estimates that 0.48% of the population living within an infested area will develop symptoms. This is based on a study of GP visits due to OPM in Holland (Jans, 2008), where OPM is a widespread problem. This estimated that around 80,000 people (0.48% of the Dutch human population) were affected each year. Using this same figure (0.48%), an estimated 8,700 people would have experienced symptoms as a result of OPM exposure in 2015. Under the counterfactual scenario, the number of people affected with symptoms would have almost quadrupled, to 25,400, an increase of almost 19,000 cases. PHE conducted a systematic review of health impacts in 2015 (PHE, 2015) which finds an even higher proportion of people exposed to OPM are likely to present with symptoms. Of the 258 papers identified, only 13 papers referred to health effects specifically related to OPM. The two largest epidemiological studies conducted in mainland Europe found that in areas where OPM was present, approximately 5-7% of the ‘exposed’ population reported experiencing health complaints that they attributed to OPM. It is unclear what level of control was being carried out in these areas, what the OPM or human population density of these areas was, and how the abundance of OPM within these areas might compare to that of the colonised areas of London. It seems possible that the 5-7% figure is a significant overestimate of the current incident rate being experienced in the UK, even taking into account that a significant proportion of health effects in the UK are likely to go unrecognised and/or unreported. This figure has therefore not been applied to the analysis for this report, Nonetheless, the estimate provides some indication of the level of health impacts that might be expected if less comprehensive controls were being carried out, and if the population density of OPM was allowed to increase (i.e. if the situation in the UK were closer to that in mainland Europe).
28 In 2015 there were 17 reports of health impacts made to the FC, compared to 11 in 2014.
19 February 2016 28
Evaluation of the Oak Processionary Moth Control Programme - Final Report
To date, fewer than twenty known health incidents per year have been linked to OPM in the UK (some affected more than one person).29 This is considerably less than the 6,700 cases estimated above. It is unclear whether this reflects that OPM has so far not lead to many health impacts in the UK, or that people do not feel the need to report the health effects. OPM-related health effects are known to be under-reported. Very few people are likely to recognise that a rash could have been caused by OPM. Even if they did know the rash was caused by OPM, few people are likely go to their GP or report the event to the FC. Most people experiencing an OPM-related rash are likely to present to their local pharmacist, if they do anything about it at all (ADAS, 2014)30. Evidence from the landowner survey suggests that people who develop symptoms do not recognise that OPM is the cause, and don’t appear to feel a need to report the symptoms to health professionals. The scale of under-reporting is unclear. Many consultees thought under-reporting was likely to be a significant issue. “I suspect that I might have had OPM in the previous year(s) already as I have been suffering from symptomatic skin rashes every time I have been in the garden, but I had never heard about OPM before I saw a nest.” The lack of more widespread health reports could also be a result of the OPMCP potentially helping to suppress the OPM population. However, anecdotal evidence from sites which have been heavily infested with OPM have not seen significant reports of health impacts. When the outbreak in Croydon / Bromley at the Bethlem Royal Hospital site was first identified, for instance, trees were described as “dripping” with OPM nests and yet no health impacts had been identified.31 Similarly, a previous study (ADAS, 2014) found that there was no significant relationship between the presence of OPM in Richmond Park, where the pest was widespread, and human health effects, which had been minimal. It appears therefore that there is no clear or straightforward link between the presence of OPM and resulting health impacts. The PHE report also notes that exposure to OPM does not automatically result in symptoms, and that it is unclear why some people react to the presence of OPM and others don’t.32 Only a small proportion of single site representatives responding to the landowner survey thought that health impacts on their site had been very significant (7% for occupational health impacts, and 11% for general public health impacts). The majority of survey respondents did not note any health impacts Figure 4.2).
29 Based on reports of OPM health incidents made to the FC. Eleven incidents were reported to the Forestry Commission from June to July 2014, and 17 from March to August 2015 (though one of the latter was a pre- emptive warning rather than a reaction). Several of these incidents affected multiple people. In 2014, for example, all the residents in a block of eight flats appeared to have suffered health issues ranging from rashes to an increase in asthma attacks, and multiple children in at least four sites were hospitalized or affected by rashes. In 2015, incidents affected multiple children on at least two occasions. Most of the remaining incidents in both 2014 and 2015 appeared to involve reactions that included rashes and chest infections, some of them severe. Others occurred multiple times and/or had been persistent over several weeks. They affected people who had been in close proximity to trees, such children playing in parks, and arboricultural workers. In one case, a horse was also affected. 30 ADAS (2014) Improved methods for the early detection of oak processionary moth (TH0101) 31 The outbreak was reported by an arboriculturalist who thought the FC should be informed of the presence of nests in the trees, not as a result of any health impacts. 32 There some indication that OPM hairs can be released in response to a perceived threat, so it may be that disturbance or proximity could also be a factor in affecting why some people are affected whilst others are not (i.e. depending on the moth’s reaction).
19 February 2016 29
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure 4.2 Health impacts experienced by landowners “Please indicate how significant the following impacts of OPM have been on the site you are responsible for” (N=95)
Occupational health impacts 10 51 21 6 7
General health impacts 5 57 16 6 11
Animal health impacts 22 58 12 21
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Don't know / Not applicable No impact Not very significant Somewhat significant Very significant
Most consultees did not think the human health impacts of OPM were significant. A few consultees pointed out that the relative risk to human health posed by OPM compared to other pests is low: “[OPM] is just another pest that can cause irritation rather than something more serious.” “When we do a risk assessment for surveying of OPM, the biggest problem is Lyme’s disease, not the impacts resulting from surveying the caterpillar. We protected ourselves more against Lyme’s disease than OPM.” Generally it was believed that health impacts would only be significant in localised cases and/or only if the OPM population increased substantially. Previous research found the same, suggesting that views on the significance of the health impacts of OPM haven’t changed (ADAS, 2014). It is only recently that the PHE’s systematic review provided supporting evidence to this effect. However, evidence from the landowner survey suggests that some landowners do see OPM as a significant public health issue. “From all the evidence I have seen this could be a significant issue, and certainly from discussions with people dealing with the problem on the ground, individual reactions (particularly following repeat exposure), appear quite serious” “The human health impact is potentially very serious.” Several consultees thought that this perception of health impacts could have knock-on effects on the use of public spaces, which could potentially be a much more important issue (see section below on public recreation, 4.3.2). Some noted that individual high profile cases of human health impacts could have significant public relations or reputational impacts. “[OPM] is not going to affect lots of people ever year, but on a local basis it could be quite important” A few consultees recognised the uncertainty and the relative risk posed by OPM as compared to other pests: “I think it’s really hard to know. I just don’t know.” “I’m not sure there would have been significant health implications for humans or animals if we had done nothing” A small number of consultees noted that, even if health impacts were not serious, landowners still had a duty of care, especially with regards to schools and areas where children could potentially be affected.
19 February 2016 30
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“But there is also a duty of care, landowners have to make sure places are safe for people to go to.” Information on animal health impacts was scarce, and anecdotal or speculative. Hence no conclusions can be drawn on whether the OPMCP has reduced such impacts. The limited evidence available suggests that animal health impacts are either not recognised as being caused by OPM, and/or are not widespread or common enough to be reported.
4.3.2 EQ4. What impact has the OPMCP had on public access and recreation? Summary statement: Consultees felt that the OPMCP had limited the impacts of OPM on public recreation and public access by suppressing abundance. Anecdotal evidence suggests that an abundance of OPM can lead to restrictions on the use of some public spaces at certain times of the year, though there is no systematic evidence of significant impacts of OPM on public access and recreation. The OPMCP may also have had an indirect positive effect by sustaining public confidence in the safety of green spaces. Consultees suggested that the measured approach taken to raising awareness of the health impacts of OPM in OPMCP communications has helped to ensure that there has been minimal impact on people’s use of green spaces. No data are available on the effect of the OPMCP on the population density of OPM so it is not possible to determine what direct impacts on public access may have been avoided due to the control programme. Analysis: The programme theory identified that a considerable increase in the spatial distribution and, especially in the abundance, of OPM could lead to restrictions of public access and the use of affected spaces. The evaluation also identified that OPM could have an additional indirect impact on the use of public spaces, if people avoid green spaces due to concerns about the perceived health impacts of OPM. By limiting the spread and population density of OPM, it was expected that these negative impacts would be avoided or at least reduced as a result of the OPMCP. Although the evaluation did not identify any systematic evidence of OPM infestations leading to restrictions on public access, several consultees felt that these impacts would have been more widespread in the absence of the control programme. It was expected that these impacts would be localised, such as specific events being cancelled or specific areas of interest being closed off to the public. Some indicated that such impacts would be temporary but one thought they could be more permanent given that OPM hairs, and their effects, persist in the environment. “If the population were allowed to grow [in the absence of the OPMCP], I believe there is potentially a high impact as areas become no go zones and sites are closed as owners become risk averse and this may be the most cost effective way [for landowners] of discharging their duty of care.” “It is not the case that closure need only take place for the active season…the risk of exposure from old nests and nests on the ground remains “live” for years to come” Some consultees felt that the potential indirect impact of OPM on the use of public spaces was more serious than the direct impact of OPM causing site closures or access restrictions. They noted that the perceived risk of serious OPM-related health impacts could lead people to actively avoid using green spaces. In this case it is not the health impacts themselves, but rather a fear of the consequences to health based on people’s perceptions which would affect the use of areas where OPM could be present. One consultee noted that the measured approach taken to raising awareness of the health impacts of OPM in OPMCP communications has helped to ensure that the fear of OPM health effects has had a minimal impact on people’s use of green spaces.
19 February 2016 31
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“As awareness of OPM grows, if not carefully managed, it may encourage/foster a more negative attitude towards [the use of] green space” “In some ways the fear of [an OPM-related] health problem may actually become the most serious issue, as people stop using public space because of the possibility of harm” A few consultees suggested that this impact was likely to be limited and that more research would be needed on the issue before anything definitive could be said about OPM’s impacts on people’s attitudes towards, and actual use of, green spaces. “We don’t know what the evidence is. If you look at something like Lyme’s disease, are people not going out in the outdoors as a result of that? I don’t know.” “I think we’re a long way away from being able to say this is the effect that [OPM] would have on people’s decisions [to use green spaces]” Data from mainland Europe provide some evidence of OPM impacts on public recreation. For instance, visits to an OPM-infested park in Germany fell, leading to lower revenues. Farms and recreation areas in OPM-infested areas in Belgium saw significant falls in visitor numbers, forcing some businesses to consider closing (Klug, 2015; Crevecoeur, 2015). The majority of landowners responding to the online survey indicated that OPM had either no impact, or no significant impacts on public access and recreation on their sites. Only 15% indicated that impacts on public access and recreation had been either somewhat or very significant. Figure 4.3 Impacts of OPM on public access and recreation seen by landowners “Please indicate how significant the impact of OPM on public access and recreation has been on the site you are responsible for” (N=95)
Total 15 44 21 11 4
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Don't know / Not applicable No impact Not very significant Somewhat significant Very significant
4.4 Environmental impacts
4.4.1 EQ9. What impact has the OPMCP had on oak tree health? Summary statement: The available evidence suggests that impacts of OPM on oak trees in the UK have been limited during the pilot programme period. How much of this can be attributed to the efforts of the OPMCP is unclear. Consultees varied in both their views on the severity of the potential problem and the tree health benefits of the pilot programme. Some consultees indicated that the OPMCP had reduced the potential impact on oak trees by keeping OPM numbers and spread in check. Others were less sure of the impact the OPMCP has had on oak tree health. Overall, however, there is insufficient evidence to determine the impact of the OPMCP on oak tree health as there is very little systematic evidence on the scale of the impact that OPM has on oak tree health and no data are available on the population density of OPM, either before or since the OPMCP.
19 February 2016 32
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Analysis: The OPMCP was expected to reduce the likelihood and scale of oak tree defoliation caused by OPM and therefore contribute to continued oak tree health. However, no data are available linking OPM numbers to different levels of defoliation, or on the impact that this has on oak tree health. The available information is anecdotal and inconsistent. It was hoped that more precise evidence could be obtained through discussions with experts in mainland Europe, but no systematic data could be identified. Whilst it is therefore clear that OPM can contribute to oak decline (Thomas, 2008), the potential scale of this impact is uncertain. In the limited research identified, there appears to be a general consensus that defoliation and occasionally significant tree weakening can occur and, especially in conjunction with other stresses, this may cause the death of the tree (Crevecoeur, 2015; Klug, 2015; Kuppen, 2015). What is unclear is how often, and how easily, this happens. At present however consultees generally did not feel that OPM’s impact on oak tree health was significant. Moreover, anecdotal evidence suggests that where defoliation had occurred in the UK as a result of OPM, trees had recovered and no permanent effects had been seen. “At present I don’t believe there is direct evidence of a decline in oak health due to OPM.” “The experience we’ve had so far [regarding impacts on oak tree health] is limited. There’ve been some cases of more severe defoliation but oak trees regularly recover and don’t appear to suffer.” “We’ve seen heavy defoliation at Bethlem Royal Hospital – but most trees have recovered.” Consultees often noted that the impacts on oak tree health could be much more serious for vulnerable and “veteran trees” that often have high heritage value. Consultees recognised that OPM alone was unlikely to causes oak death, but that in combination with other stresses (e.g. other plant pests and diseases) and in a context of general oak decline, the impacts could be substantial and could lead to the deaths of some oaks. “It’s an additional stress factor for England’s oak tree population that are already under pressure.” “What we have got is the heritage impact. … We’re going to end up losing the most iconic trees because they won’t survive.” Only two landowners (one private company and one public body) responding to the online survey indicated that they had observed significant oak tree health impacts. The majority of landowners and consultees had not seen any impact, or did not think that the impact had been very significant. This small amount of evidence suggests that to date the impacts of OPM on oak trees in the UK have been limited, although how much of this can be attributed to the efforts of the OPMCP is unclear. Figure 4.4 Impacts of OPM on oak tree health seen by landowners “Please indicate how significant the impact of OPM on oak tree health has been on the site you are responsible for” (N=95)
Total 13 39 32 9 2
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Don't know / Not applicable No impact Not very significant Somewhat significant Very significant
19 February 2016 33
Evaluation of the Oak Processionary Moth Control Programme - Final Report
It is likely that the population density of OPM will have a bigger impact on the scale of potential defoliation, and therefore the resulting impact on oak tree health, than whether OPM is present or not (i.e. the distribution of OPM). The link between the OPMCP and its impact on OPM population numbers, rather than its spread, is therefore likely to be the biggest determinant factor of the OPMCP’s contribution to oak tree health. Several consultees, for instance, indicated that if OPM populations were allowed to reach plague proportions the impact on oak tree health could be significant. “It could be locally serious, if allowed to reach plague stage” “The level of defoliation depends on the caterpillar populations, plague infestations can occur leading to whole tree defoliation which if repeated can kill weakened trees” No data are available on the abundance of OPM in the UK, or how this might have been affected by the OPMCP. It is therefore difficult to assess what impact the OPMCP has had on oak tree health. However, many consultees felt that the OPMCP has mitigated this impact by limiting the rise in OPM numbers, and that without the control programme, the OPM population would have risen to a point where there would have been significant impacts on oak tree health as a result. “The control programme has prevented the population level of OPM reaching a point where these impacts become serious” One consultee had evidence that impacts can quickly escalate in the absence of control, suggesting that the continued control work under the OPMCP has had a potentially important impact on limiting the scale of potential defoliation. “For the first time in 2015, we have experienced extensive crown defoliation on oak trees where nest numbers are very high. ….It is our experience that in areas such as these which the moth favours, no intervention for more than a year results in a rapid rise in nest numbers with accompanying increase in crown defoliation and subsequent impact on tree health.” Other consultees however felt that if left uncontrolled, OPM would not necessarily cause significant problems as natural processes would help to keep OPM numbers in check. “We’re starting to see some natural predators. Actually I think OPM would become part and parcel of oak tree ecosystems and would be kept in check with checks and balances.” “Obviously any defoliating insect [like OPM] will have some impact, but no more so than any other foliar feeder” Other consultees did not feel able to predict the impact of OPM on tree health in the absence of the control programme. “It’s hard to know what would happen if you did leave it alone how it would spread, and how often you would have a year when population levels get really high. We don’t know how often you’d have a year where defoliation would be high enough to have impact.”
4.4.2 EQ10. What impact has the OPMCP had on biodiversity? Summary statement: The use of pesticide spraying under the OPMCP can, and most likely has, affected biodiversity however it is not clear what mix of species have been affected, to what extent, and for how long these impacts have lasted. Biodiversity could have been more affected by the impacts of OPM (as a result of on oak tree defoliation and a potential decline in oak tree health) if the spread and abundance of OPM had increased in the absence of the OPMCP, however the scale of these potentially avoided impacts is unknown. The control agent
19 February 2016 34
Evaluation of the Oak Processionary Moth Control Programme - Final Report
applied by the OPMCP operatives is less damaging to non-target species than the alternatives that were sometimes used by landowners before the pilot programme started. The programme may also, by funding control measures, have led to fewer oaks being felled by landowners looking to avoid recurrent costs of OPM control. Overall it is not possible to say whether the OPMCP has had a net negative or net positive impact on biodiversity. Analysis: When the OPMCP was initiated, it was unclear what the overall impacts on biodiversity would be. It was expected that the use of pesticides might have some impact on biodiversity, although the scale of this was unknown. The OPMCP uses Bt (Bacillus thurengiensis) in the spraying operations, although dimilin and deltamethrin (wider spectrum insecticides) can also be used for treating OPM. It was also expected that the OPMCP would deliver some benefits to biodiversity through contributing to oak tree health. Little evidence is available on either the impacts of the OPMCP spraying programme or the contribution that the OPMCP has made to oak tree health (see Section 4.4.1) and therefore biodiversity. It was hoped that discussions with experts in mainland Europe would provide additional evidence but no systematic data were available. There is little conclusive evidence on the impacts on biodiversity of the pesticide spraying applied under the OPMCP as very little research has been done on this issue. The research that has been done has been limited both in its geographical and temporal scope, so it is difficult to draw meaningful conclusions for the long term effects of the whole OPMCP. This was also noted by one consultee. There is therefore no systematic evidence to indicate what the overall impact on biodiversity would have been from OPM in the absence of the control programme. “There’s just not a lot of evidence on the issue of biodiversity really. The nature of the studies haven’t been based particularly on the strongest methodologies in the world. They’ve provided some insights but aren’t definitive.” Many consultees were confident that pesticide spraying would have, and has had, a negative impact on biodiversity. The limited evidence in the scientific literature supports these views. For instance, Freed and Reeve (2014) tested matched stands of oaks treated for OPM in the same park in the UK, and found a “substantial reduction” in taxonomic and numerical abundance of non-target Lepidoptera species in stands which had been sprayed with Bt (the same pesticide used in the OPMCP) as compared to non-sprayed stands. These effects persisted for at least a year after spraying, though the authors noted that further studies were needed to determine any longer term impact. They reviewed other UK and EU studies and concluded that in combination these indicated a significant impact on a range of non-target Lepidoptera. The research also suggested that a change in the Lepidoptera community in the shorter term that could have indirect impacts on related predators, parasitoids etc. Ecosulis (2014) found that the Bt pesticide used to control OPM affects all Lepidoptera larvae. The study also found that population productivity of two tit species was generally higher at the control site than the sprayed site, although clear conclusions were not possible due to a general lack of bird activity. One landowner felt that impacts on biodiversity could be particularly serious given that controls were continuously needed year on year, as the infestation would never be eradicated and would require constant management. “We’re aware that spraying would have a "huge impact on biodiversity" as it would affect many larvae of non-target species, and so impact up the food chain; we’re very concerned that expenditure and effect on biodiversity would be annual, year on year, as infestation would never be eradicated, only managed / reduced…trying to continue to fight will have massive impact on the environment and biodiversity”
19 February 2016 35
Evaluation of the Oak Processionary Moth Control Programme - Final Report
However most consultees believed that the impacts are temporary and/or limited in their geographic extent given the scale of the spraying activity to date. This is largely due to the fact that Bt is the most commonly used control agent under the OPMCP, and is much more targeted than the chemicals that could be used as alternatives (e.g. dimilin or deltamethrin). Use of deltamethrin and dimilin was more common before the OPMCP. At that time there was little stopping landowners from choosing these chemicals, which are more effective but also more damaging to non-target species. Whilst dimilin is much more targeted than deltamethrin, it is possible that in the absence of the OPMCP, the use of these more damaging pesticides would have been higher as landowners selected controls for themselves. Where broad-spectrum insecticides are used in a more indiscriminate way, the negative impacts on biodiversity are likely to be much greater. “Locally on the trees sprayed, there is an impact on other biodiversity – it will reduce numbers of other non-target species. But the effects are temporary and short lived. Those species will re-colonise very quickly” “Using Bt is very targeted, and actually applied at quite a tight time of year and what it targets is quite narrow. I’m not convinced it’s that harmful. It definitely has an impact but in relative terms its low.” “Where control has been applied only in a small local area, neither positive or negative effects are likely because populations of other invertebrates are able to recover more rapidly” Some of the scientific evidence in this area suggests that it is possible that species do recover and that the impacts are therefore limited in their temporal scope. In Belgium, for instance, Bt spraying was found to cause a significant decrease in Lepidoptera variety and prevalence, as well as in wider biodiversity in test areas (Crevecoeur, 2015). Lepidoptera species appeared to recover a few years later but biodiversity remained depressed in at least one case (Manderino, Crist, and Haynes, 2014). None of the research identified in this area continued monitoring the impacts on biodiversity for long enough to observe the effects over a significant amount of time. Other studies have also had trouble reaching conclusions with much certainty for other reasons. For example, Townsend (2014) tested moth taxa at control and sprayed sites in Berkshire, UK and found 56.5% more abundance and 10.8% more taxa at the control site than at the sprayed site. The data did not suggest that uncommon (‘Nationally Scarce’) species had been affected by the treatment. However, the authors note that the difference could not be conclusively attributed to spraying due to lack of corroborative and replicating evidence. Some consultees noted that the scale and extent of the impact on biodiversity could rapidly increase if a greater area had to be sprayed because of OPM. “The impacts on biodiversity will depend on the scale. The more you spray, the bigger the area, the bigger impact you’ll have. If there is a big increase in spraying then there’ll be a bigger impact on other things” A few consultees thought that the impacts on biodiversity from the pesticide spraying, while negative, are likely to be less severe than the biodiversity impacts which would have occurred if OPM had not been controlled under the OPMCP. “Spraying has a significant negative effect on the abundance and richness of non- target Lepidoptera….however, our experience also shows that without constant management of some kind, OPM can reach proportions that have the potential to negatively affect tree health which, if long term, would have an even worse effect on biodiversity.” “Spraying will have had some damage to biodiversity. But we have to ask what would have happened if we had done no control. The moth species itself could have a detrimental effect on other species. So for example this year, we’ve seen
19 February 2016 36
Evaluation of the Oak Processionary Moth Control Programme - Final Report
complete defoliation of a tree for the first time. This is a large oak tree with all of its oak leaves gone, that’s a major food source gone for other native species” Research into the control of the gypsy moth does suggest that it is possible for the impacts of a pest species on biodiversity to be greater than the impacts of control. For instance, Manderino, Crist, and Haynes (2014) found that populations and diversity of Geometridae species moths actually improved in the years after spraying with Bt to control gypsy moth populations, and that there were no long-term negative effects of Bt application on overall forest Lepidoptera abundance and diversity. For Geometridae in particular, this was likely due to the fact that spraying reduced the gypsy moth population and thereby also reduced defoliation, preserving - at least partially - a source of food for the Geometridae. This finding was corroborated by the fact that sites defoliated by gypsy moths but not sprayed continued to show lower populations and diversity of Geometridae moths. There is some evidence that the an OPM control regime where landowners have to fund the cost of controls themselves can lead to unintended impacts on biodiversity as landowners fell oak trees to avoid the ongoing costs of OPM management. The scale of this problem is unknown, and there are no data on how these numbers have changed over the programme period.33 Four respondents to the landowner survey had felled affected oak trees for reasons that included future potential costs and the use of pesticides associated with OPM management. One landowner wanted to remove an affected oak but was not able to do so because it was subject to a Tree Preservation Order. A small number of other respondents noted that continued OPM management cost would mean landowners would consider felling affected trees in the future. Overall the evidence suggests that felling of oak trees due to OPM is currently a relatively rare occurrence, although continued and rising management costs could lead more landowners considering this as a course of action. “I believe we had no infestation this year so no treatment has been required but a repeat may result [in] us having to fell the trees which would be a shame” “We are also aware of private owners of oak trees planning to have them felled rather than face the year on year cost of OPM management” Overall, there is not enough evidence to say whether the OPMCP has had a net negative or net positive impact on biodiversity. Several consultees noted this as well, and felt that more research needed to be done before anything conclusive could be said on the issue. “I suspect [the impact on biodiversity is] budget neutral – I don’t know. A lot of research needs to be done on this” “We just don’t know the long term effect of defoliation rather than spraying.”
33 Either comparing before and after funding was made available through the OPMCP, or comparing what happened in the core zone between 2014 (when some funding was available) and 2015 (when funding wasn’t).
19 February 2016 37
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5 Process evaluation findings
5.1 Introduction This section presents responses to the process evaluation questions. These cover the following elements: ■ Programme level governance and management; ■ Operational delivery; ■ The implementation of control measures; ■ Data gathering and data management; ■ Awareness raising and stakeholder engagement; ■ Research and analysis; and, ■ Capabilities An overview of the summary statements for each of the evaluation questions is given below. The analysis and supporting evidence is provided in the detailed responses in the sections that follow.
19 February 2016 38
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 5.1 Summary statements for the process evaluation questions
Process evaluation questions Programme level governance and management EQ11 How effective has the The OPM Project Board has provided an effective forum for discussion and decision making on the operational delivery of the OPM Project Board been OPMCP. Almost all the stakeholders consulted thought it had been very effective. The fact that the Project Board membership in steering the OPMCP? included a mix of people with policy, operational and scientific expertise was seen as particularly useful. EQ12 How effective have the The Advisory Group has been an effective mechanism for supporting communication and collaboration between the OPMCP OPM Advisory Group and external stakeholders such as contractors, landowners and NGOs. Most consultees felt that the Advisory Group was a been in supporting the useful forum and helped to support the OPMCP, mostly by raising awareness of OPM amongst wider stakeholders and work of the OPMCP? ensuring that the issue of OPM management was high on people’s agendas. Some consultees noted various issues relating to the Advisory Group’s membership. EQ13 How effective have Programme level management arrangements have worked well within the constraints imposed by factors beyond the OPMCP’s programme level control, such as a requirement for approval of plans and budgets on an annual basis. This requirement resulted in delays and management some sub-optimal outcomes (such as core area landowners receiving notice in 2014 of the withdrawal of control funding too arrangements been? late in their budgetary cycles to be able to adjust their plans). Having a dedicated and consistent programme manager in place contributed significantly to the pilot programme’s effectiveness. . Annual reporting by the programme could be more consistent and structured. EQ14 Have OPMCP budgeting The main budgetary challenge the pilot programme faced related to the alignment of the budgeting cycle with the cycle of arrangements worked as annual planning, communication and delivery. The pilot programme experienced difficulties in keeping stakeholders, including intended? contractors, sufficiently informed and in ensuring that funding was available in time for control measures to be implemented. Nonetheless, the process of getting funding in place appears to have become more efficient over time. Day-to-day budgetary planning largely worked well, though again there was year to year learning. Spending on specific activities generally came in close to planned budgets. A considerable underspend in 2013 was due to funding being made available too late in the year for the money to be spent in time. Budget management was sometimes made difficult by unexpected needs and as circumstances changes (e.g. the need for survey work around a new outbreak area near Guildford in 2015). As the pilot programme gained experience and prices for services became more established it became possible to budget line items with more confidence. EQ15 Did the programme 'Capacity' is here considered to include staff time, staff expertise and funding whilst the ‘challenge’ refers to the pilot provide capacity programme’s objectives (containment of the London outbreak, and eradication elsewhere). The decisions made in 2012/2013 appropriate to the scale on policy and on capacity allocation were made with less information than is now available, particularly on the scale of the and nature of the outbreak and dispersal of the oak processionary moth. It is now apparent that OPM is more widely distributed across the challenge? London area than was understood at the time. This has implications for the effort required for surveying and control. These issues are reflected in consultees’ responses to the evaluation question. The main conclusions are that: - the overall capacity allocated was appropriate to the challenge as it was defined and understood at that time; - doing the same thing again, with the benefit of hindsight, programme managers would have allocated capacity differently across the pilot programme’s activities;
19 February 2016 39
Evaluation of the Oak Processionary Moth Control Programme - Final Report
- more effort would have been given to surveillance, communications, research and administrative support in the early stages of the pilot programme. Providing ‘surge capacity’ to support the core programme team during the periods of peak workload that occur at certain points in the year would have relieved some of the constraints that were experienced; - the pilot programme has developed new capacity in areas such as information management and the targeting of controls. There have also been increases in the capacity of the pilot programme’s supply chain to conduct surveillance, surveys and controls. There are few contractors with the capacity to apply controls in the manner required by the pilot programme, although this is increasing; - increasing pressures on the pilot programme’s budget have prompted changes to its design (e.g. withdrawal of funding for controls in the core area) whose consequences for the core programme objectives are not yet fully understood. Operational delivery on the ground EQ16 Has the OPMCP day-to- Day to day operational management worked very well. The OPMCP benefitted greatly from having an effective programme day operational manager. That the programme model was delivered and improved over time owed much to having a manager with a mix of management function technical, management and engagement skills who was already familiar with the institutional and policy context. The continuity worked as expected? associated with having the same person being in post for the duration helped with the accumulation of knowledge within the What has contributed to programme. Consultees’ comments about potential improvements focused on structure and communications. Several of the this result? pilot programme’s components relied on single points of contact which risked the creation of bottlenecks. Feedback suggests some issues with communications with landowners, e.g. in relation to SPHNs and on response times. EQ19 Did public financing of The availability of public funding for OPM controls allowed the FC to establish a centralised, coordinated approach which controls, both in terms of enabled much more control work to be carried out in the short period of time in which spraying is effective. Qualitative evidence eradication and suggests the timeliness and efficacy of controls increased very significantly. The evidence suggests that public funding, by containment, improve addressing landowners’ fear of the financial implications of OPM, has made landowners more willing to give contractors access the timeliness and to their sites for survey and control purposes, and more willing to report sightings of OPM. It is highly likely that without OPMCP efficacy of controls as funding, fewer controls would have been carried out and that they would have been applied in a more ad hoc manner that intended? would have reduced their overall effectiveness. Rates of re-infestation from sites where no controls were applied are likely to have been higher. As no data were collected on the proportion of landowners carrying out control work prior to 2013, no comparative quantitative evidence is available to determine the extent to which OPMCP funding has improved the number and effectiveness of controls being carried out. Implementation of control measures EQ17 What worked well, and The organisation and application of control measures has worked well, especially when the many challenges are taken into what not so well in the account (e.g. the very short period in which pesticide spraying is effective, issues of access and weather conditions, uncertainty implementation of about funding). Over the programme period, more than 90,000 trees on almost 700 sites were sprayed and almost 21,000 control measures? Has OPM nests removed. The OPMCP, by providing a single centre of operations, has enabled adoption of a much more consistent the implementation of and methodical approach. Many consultees thought the implementation of controls had worked as well as could be expected. control measures Notwithstanding the successes, some areas for improvements were identified. Communications between the contractors and worked as expected? the FC on progress, and communication with landowners on when and what controls were being carried out could both have been better. Greater oversight of contractors and their work would have provided additional assurance that the controls were
19 February 2016 40
Evaluation of the Oak Processionary Moth Control Programme - Final Report
being applied and to the appropriate standard. There are also some open questions about which pesticides and which methods of application are most appropriate and useful. EQ18 Has the OPMCP Overall, the available evidence suggests that the OPMCP improved the cost effectiveness of controls, but by less than was improved the cost expected. One contributory factor is that for all three years, most of the treated sites only included ten or fewer oak trees which effectiveness of control were sprayed, so economies of scale were harder to achieve (average treatment costs drop significantly as the number of trees activities as expected? treated per site rises). Detailed information on control costs was only collected from 2013 onwards. The few data available from the prior period suggest that costs prior to 2012 ranged from £370 to just over £1,000 per tree, but were lower in larger woodlands and parks (at £50-£100 per tree), due to economies of scale and the use of volunteer labour before the OPMCP started. The impact assessment for the OPMCP assumed treatment costs of £100 per tree. Data from the OPMCP suggest that the actual average costs have been higher than this estimate. Programme cost data need to be treated with some caution but they indicate an average cost per tree for pesticide spraying of £120 - £180 over the three years. The average cost of manual nest removal fell steadily over the three year period from of £138 to £95 per nest. The average combined cost per site for both spraying and manual nest removal under the OPMCP is less than what landowners reported paying for their own controls but this result must be treated with caution as the evidence base is small. Data gathering and data management EQ20 Have the OPMCP The pilot programme built processes and systems for data capture and management that improved over time and ultimately processes for data served operational needs as intended. Many stakeholders regarded this as having evolved into one of the most successful gathering and aspects of the programme. The arrangements now enable the programme management to react promptly to new information management worked as and to deploy contractors where they would be most useful, thereby making the most of the brief period in which spraying is intended? effective. There were year on year improvements in processes. Survey and surveillance plans were developed and implemented. Data from sightings, surveys, controls and surveillance were recorded and used to inform the pilot programme. A mobile app facilitated more timely and accurate capture of data. An outsourced management service for OPM site and surveillance data was generally able to serve programme management needs, though the database was not configured to facilitate direct access by programme and other FC staff. Consultations indicate scope for further improvement in data gathering, such as gathering information on observed tree health and the abundance of OPM. EQ21 How effective have data Data management systems have been a crucial aid to programme delivery. Data supported operational decisions on management systems communications, control measures, and on survey and surveillance efforts, and greatly facilitated liaison and engagement with been in supporting the stakeholders. The maps showing the results of the OPM survey - where OPM had been found and where it was absent - were programme delivery? especially useful. The systems developed have enabled the pilot programme to turn raw survey data into actionable intelligence, and the despatch of control contractors, far more quickly than was the case previously. Consultees saw efficiency and effectiveness gains on offer from the further development of existing systems, in particular providing live access to programme data, overlaid on digital maps, to field operatives. Better provision of data on matters such as site access and contact details should enable contractors to operate more efficiently. Greater use of ‘live’ data would also be useful, such as the provision of on-line maps in addition to the use and distribution of static PDF maps. The strategy has been responsive - the pilot programme has increased its investment in surveillance, the results of which are raising new questions about the
19 February 2016 41
Evaluation of the Oak Processionary Moth Control Programme - Final Report
distribution of OPM. But the data collected are not well suited to monitoring, evaluation, research or longer-term strategic planning purposes. Had these questions been factored into the strategy at the start it would be possible to say more now about the pilot programme’s impact in relation to its core objectives.
Awareness raising and stakeholder engagement EQ22 Has the OPMCP raised The OPMCP’s communications efforts have expanded and improved significantly over the three years of the pilot programme. stakeholders’ level of Evidence suggests that the pilot programme raised awareness of OPM and related issues among landowners (especially those awareness of OPM, its with sites infested with OPM), arboriculturalists and tree health professionals, some local authorities, health professionals and risks, Government other target groups. The extent to which the OPMCP has raised public awareness of OPM and the programme is less clear. policy, their obligations and the action being taken as intended? EQ23 Has the OPMCP Some consultees suggested the OPMCP actions had increased the reporting of OPM sightings. This is difficult to validate with improved the reporting of the available evidence, but there were examples in 2015 of new colonies being brought to the FC’s attention via independent OPM sightings and reports. health incidents as There is also now a means of health professionals coding health incidents as OPM-related in the UK's health data systems. expected? Consultations suggest that a small number of such incidents have been recorded. It is not clear whether and to what extent the lack of evidence of large scale OPM-related health impacts is evidence of these impacts being uncommon, or whether other factors are at play (e.g. under-reporting). EQ24 How effective has the The pilot programme has been effective in engaging with landowners, arboriculturalists and tree health professionals, OPMCP been in contractors, a small number of local authorities in London and NGO representatives. There was consistently positive feedback engaging and promoting from stakeholders on the OPMCP’s efforts to develop a collaborative approach. The pilot programme has had less success in collaboration with engaging the other local authorities despite a concerted effort to do so. It has also not achieved the level of collaboration with stakeholders? the health sector that was hoped for, an outcome attributed to the comparatively low priority given to OPM as a public health issue by the health sector. Research and analysis EQ26 Have investments in There has been a strong focus on continuous improvement. The OPMCP has changed and aspects of its delivery improved research (e.g. on OPM significantly over the three years. The OPMCP management approach has been based on an annual cycle of planning, population distribution implementation and taking stock of lessons learnt, for instance through annual lessons learned workshops with wider and dynamics, control stakeholders. This has worked well and has resulted in continuous improvements to the programme, for example with regard to measures and communications. Scientific input and expert advice have played key roles in the OPMCP and its improvement over time. Links surveillance) contributed have been fostered between the programme and researchers; the programme managers have maintained access to scientific to the programme as expertise, and used the advice to adjust the programme. Most of the input has been informal and the programme has generally envisaged? relied on Forest Research to find and filter relevant evidence. The primary research commissioned through the OPMCP is still
19 February 2016 42
Evaluation of the Oak Processionary Moth Control Programme - Final Report
EQ27 How effective has the on-going and so has not yet made a direct contribution to OPMCP actions. Other research which has already been completed OPMCP been in has had a greater impact on the OPMCP’s delivery. There was an agreement amongst stakeholders that the research ensuring that component of the programme was important. The lack of interim results that might inform programme delivery was a frustration programme delivery is for some. The measured pace of research review and publication processes did not always meet the evidence needs of the informed by current programme. A more systematic approach to reviewing available scientific evidence, gaps and priorities would have helped to scientific knowledge demonstrate that investments were focused on priority areas and that actions were feasible. (including the results of research and analysis) and lessons learnt from within the programme? Capabilities EQ29 Has the OPMCP Much has been learned from the OPMCP on the collection and management of data related to OPM. Some of this experience improved the ability to and the lessons learned appear to be useful and transferable to other areas, particularly with regard to the need for a quick and collect and manage data responsive data management system which can be used to direct and inform operations, for instance when and where to treat on other plant pests / a pest if there are reported sightings. The OPMCP also illustrates the benefits of using an electronic, web-based field pathogens / diseases as application for collecting data. Experience with OPM data management systems has already been reviewed in scoping the expected? potential for a data management system on Chalara. The outsourcing of OPM data management services provided freedom and flexibility to implement changes, and has provided insights that can inform future systems. However a few consultees felt that this approach limited the OPMCP’s contribution to the improvement of FC in-house capability in tree pest data management. Some consultees were sceptical about how much of the OPMCP experience could be transferred to other plant pests, pathogens and diseases. Some consultees thought the outsourcing arrangements had impeded access to the data, but the accessibility constraints owed more to the data being held for longer than expected (due to delays in delivery of the replacement FC system) on a software platform that was sub-optimal than to the contracting model used to procure it. EQ30 Has the OPMCP Capability to control OPM has clearly improved over the three years. There is greatly enhanced capability to apply controls improved the capability where they are needed, when they are needed. There is now a better understanding of what is required for delivering a large- to control OPM and scale spraying programme, including the necessary underlying systems and collaborative relationships that this requires. More similar plant pests / is known about OPM and how to control it effectively, more stakeholders are aware of OPM and engaged in the process of pathogens / diseases as managing it, and the systems for data collection and control work are more efficient and effective. Consultees also noted that intended? the number of contractors with the necessary expertise is growing. The extent to which the OPMCP has improved capability to control other plant pests, pathogens and diseases is a more complex question. Elements of, and lessons from, the OPMCP on establishing an effective control programme are transferable, including the importance of communications and establishing collaborative relationships with stakeholders, governance arrangements, the learning and improvement systems, control contractor management, and data capture. The transferability of lessons learnt in application of control measures depends on whether other pests would respond to similar measures. EQ31 Has the OPMCP led to The OPMCP has developed and improved techniques that are likely to aid future OPM control, and might contribute to the the development of new control of other pests and diseases. In some cases it is the combination of a set of existing technologies and processes in techniques to aid future
19 February 2016 43
Evaluation of the Oak Processionary Moth Control Programme - Final Report
OPM and other pest / which they are used that is innovative in a UK tree health context rather than the technologies being especially innovative. It disease control as has: intended? - Developed and demonstrated a successful approach using pheromone trapping (though the application of the results to aid OPM control remains work in progress, as does the transferability of the knowledge to other pests); - Led to improvements in ground spraying techniques and a better understanding of their limitations and effectiveness; - Developed and adopted an in-field web-based application for data collection that has improved future control of OPM by enabling a rapid response to new infestations. This learning could also be applied to other pest and disease control; - Developed useful techniques for stakeholder engagement, and has established working relationships with stakeholder groups that are novel in a UK tree health context. The relationships and approaches should aid the control of future pests and diseases, particularly if they also affect urban environments.
19 February 2016 44
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5.2 Programme level governance and management
5.2.1 EQ 11. How effective has the OPM Project Board been in steering the OPMCP? Summary statement: The OPM Project Board has provided an effective forum for discussion and decision making on the operational delivery of the OPMCP. All the stakeholders consulted thought it had been very effective. The fact that the Project Board membership included a mix of people with policy, operational and scientific expertise was seen as particularly useful. Analysis: The OPMCP Project Board was established in 2014 as a forum for discussing and deciding on key operational aspects of the OPMCP. Its members were Defra staff with policy expertise and Forestry Commission staff covering the different policy and operational elements of the OPMCP (e.g. organising controls, communications, research, data management). Its purpose34 was to: ■ oversee all aspects of the OPM control programme in order to deliver the outputs and outcomes desired by the approved and funded Defra project proposal; and, ■ coordinate the activities of all parties to maximise the opportunities and minimise the risks. The Project Board was also established to provide the programme manager with additional support and guidance, when needed. It usually met monthly via teleconference. Meetings typically lasted for about 2 hours. Almost all of the consultees felt that the Project Board had been effective and useful. “[The Project Board] functioned well as a forum and a decision making group” “The Project Board has worked well and has been an important mechanism for keeping people up to date.” Before the Board was established there was a greater reliance on the programme manager to steer and deliver the pilot programme. The Board has enabled a wider consensus to be reached on how the delivery of the OPMCP could support the Government’s policy goal. The Project Board has kept communication channels between the operational and policy interests open and active. This ensured that policymakers could be responsive and take account of the challenges experienced on the ground. Interviews indicate that the Project Board was particularly beneficial in ensuring that emerging scientific evidence was used to inform and improve the pilot programme’s operational delivery. Examples include evidence on the biodiversity impacts of control measures on other species and baseline evidence on the distribution of OPM in affected areas. The Board did not make decisions on what research is conducted or commissioned. “[Through the Project Board] we’ve been able to come up with various options and discuss the pros and cons of each. For me, the main benefit has been the communications aspect of it… Having constant discussions with people on the ground, who can tell you regularly about the logistical aspects of the problem has been really useful from a policy point of view - just understanding the limitations of what you can do. If we hadn’t had such regular contact it would’ve been more difficult to develop realistic options.”
34 Oak Processionary Moth (2014) Project Board Terms of Reference
19 February 2016 45
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“[The Project Board] provided the rigour to at least have regular monitoring on progress, operational updates, decisions and future steps and stuff like that. It was an effective way to coordinate the Defra family response.” Analysis of Project Board minutes indicates that meetings have generally been well attended. The one qualification to consultees’ general consensus on the success of the Project Board related to the challenges of having a larger group of people and a greater diversity of interests and expertise shaping the direction of the pilot programme. A small number of stakeholders noted that this could reduce the influence of subject matter expertise on decision-making, and lead to discussions being less closely focused on technical issues or informed by scientific input. On the other hand, having more people involved from different backgrounds and perspectives has ensured that there is a greater level of consensus about what the programme can and should be able to deliver, and has increased the level of buy-in and support for the OPMCP’s achievements. “[Due to] greater involvement of different FC and Defra groups…the programme has become less flexible to adapt and change to changing circumstances, the more people you have to persuade and the more people have to change their views… there aren’t that many specialists and [expert] advice has been diluted… [For instance,] scientific input is much more removed from management and strategy of the whole programme than it used to be.” The only other area where the effectiveness of the Project Board was questioned was in getting local authorities engaged in the issue of OPM management. Decisions were made by the Project Board about how to increase engagement with local authorities but these attempts were not successful. Several stakeholders recognised that there were understandable reasons for this being a challenge (see section below on stakeholder engagement, 5.6.3). “The main failing of the Project Board has been in not getting local authorities to take responsibility.”
5.2.2 EQ12. How effective have the OPM Advisory Group been in supporting the work of the OPMCP? Summary statement: The Advisory Group has been an effective mechanism for supporting communication and collaboration between the OPMCP and external stakeholders such as contractors, landowners and NGOs. Most consultees felt that the Advisory Group was a useful forum and helped to support the OPMCP, mostly by raising awareness of OPM amongst wider stakeholders and ensuring that the issue of OPM management was high on people’s agendas. Some consultees noted various issues relating to the Advisory Group’s membership. Analysis: The Advisory Group is a consultative forum that bring representatives of the OPMCP programme management together with senior representatives of a wide range of stakeholder interests (including landowners, policy makers, budget holders, the health sector, and service providers (e.g. contractors and surveyors)) to discuss issues of mutual interest. A list of members is provided in Annex 15. Its aims35 are to: ■ Advise / inform policy makers, decision makers & research on strategic issues; ■ Facilitate surveillance and control across all stakeholders so its effectiveness is maximised; ■ Coordinate OPM communication.
35 Oak Processionary Moth Advisory Group: Terms of Reference
19 February 2016 46
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Meetings were generally held three times a year (more were held in the first year when the OPMCP had just begun), and last about two hours. The minutes of the 2013 lessons learnt meeting36 record that the OPM Advisory Group resulted in increased collaboration between landowners and authorities and offered the opportunity to coordinate spraying programmes. This potential was realised with the pheromone trapping programme and coordinated spraying programmes in south-west London. Overall consultees seemed to think that the Advisory Group has worked well and that it has effectively supported the OPMCP. “[The Advisory Group] has felt like a governance structure that has worked well and that has had to deal with a range of political and practical issues and achieved action and forward movement around the issue successfully” The Advisory Group’s main impact beyond the core programme has been to raise awareness and influence key stakeholder groups and organisations. “I feel that the OPM Advisory Group has raised awareness of OPM” “Government agencies can’t lobby ministers, whereas the Advisory Group can ‘put the boot in’ and get their attention. The key benefits really have therefore been in terms of spreading the message about OPM.” “The group helps make other organisations more aware of some of the problems and more realistic in their criticism [of the programme].” The Advisory Group has also given the OPMCP operational team access to different views and experiences which the Project Board could then take into account to adjust, if necessary, the management and strategy of the pilot programme. “We discuss at the Project Board meetings what the Advisory Group have said. We’ve found it really helpful….The group brings a different level of expertise and a different perspective.” “Politically it’s an important thing…It brings people in and gives them ownership of the issues.” “The FC need to be commended, they’ve been willing to be engaged with and be challenged by us….. All the way through they’ve listened to what we have said and adapted what they were planning to do.” Consultees generally acknowledged that the different kinds of expertise and experience of stakeholders on the Advisory Group has strengthened the effectiveness of the OPMCP. “The OPM Advisory Group has been a very effective and consensus based group that has taken views from a wide range of partners and steered the approach to OPM on the back of this input.” Several consultees noted that effective leadership has been an important part of the Advisory Group’s success. “[The chair] has been fantastic, and has been a real force” A few consultees noted that there was not the same level of trust and transparency between the Project Board (i.e. Defra and the FC) and the Advisory Group in the early stages of the pilot programme as there is now. Information on the programme and its progress was not as accessible to Advisory Group members as they would have liked. This affected the quality of the input Defra and the FC received from the Advisory Group. Consultations suggest that arrangements are now working much better.
36 Forestry Commission (2013) Minutes of: OPMCP Lessons Learned Meeting, Friends House, 17th September 2013.
19 February 2016 47
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“The group was less effective for a while because we weren’t able to share information with them, as that level of trust wasn’t yet established. That limited the usefulness of [the Advisory Group] in terms of feedback and exacerbated some of the communication issues. That changed [in 2015]. Through the summer of 2014 the group had real frustrations as the group ended up being in the dark... It took some sorting out … It took about a year to establish that trust again with people.” The main issue to emerge from consultations about the Advisory Group was its composition. Consultees reflected on the challenge of finding an appropriate balance between different interest groups. Examples include: ■ Providing adequate representation of biodiversity interests. Butterfly Conservation, an NGO, is now a member of the Advisory Group as a representative of multiple environmental NGOs. “Until recently, the Advisory Group membership was skewed….we weren’t considering the environmental consequences of this.” “There is now more engagement from environmental organisations. It does enrich the discussion.” ■ Tackling the relative lack of engagement by local authorities. “Sometimes it’s very hard to engage local authorities, especially at the moment when they have no money and no staff.” “The main failing is that we’ve not managed to get local authorities to take responsibility. That has been the big failing.” ■ Securing more involvement from Public Health England37, with potentially a missed opportunity to involve the Health and Safety Executive to represent occupational health interests. “Engagement with PHE hasn’t been great. It took a while to get them on board, they haven’t really done much.” “PHE are not really interested in occupational health, that’s seen as the responsibility of the Health and Safety Exec. We didn’t appreciate that until recently. It may be that we need to get HSE representatives onto the Advisory Group.” ■ Representation of contractors and landowners. “There are also vested interests in the group (e.g. those involved in the delivery of control measures), although their technical expertise is undoubtedly helpful. “ “The Advisory Group is a mixed bag really. They all have their own vested interests. Contractors and landowners for instance have been really useful as have provided a really practical realistic point of view, but we have to be a bit careful about their responses because they’re talking from their own perspective. Similarly, the environmental interests are critical of some of the aspects, which is very helpful but they don’t always look at the bigger picture.” “I’m not sure that the constitution of the group is as it should be. It’s probably more pro-control given some of the vested interests.” A small number of stakeholders suggested that the effectiveness of the Advisory Group has been lessened by the inclusion of non-technical experts who lack detailed knowledge of some OPM issues. One noted that this can result in misinformation being disseminated to other stakeholders, and unrealistic expectations being sustained. Overall, however,
37 Since the Advisory Group was established PHE has taken a number of actions related to OPM, including organising meetings to brief the wider PHE team in Southwest London.
19 February 2016 48
Evaluation of the Oak Processionary Moth Control Programme - Final Report
stakeholders generally acknowledged the need to find a balance between ensuring a representative group and detailed knowledge of aspects of the control programme.
5.2.3 EQ13. How effective have the programme level management arrangements been? Summary statement: Programme level management arrangements have worked well within the constraints imposed by factors beyond the OPMCP’s control, such as a requirement for approval of plans and budgets on an annual basis. This requirement resulted in delays and some sub- optimal outcomes (such as core area landowners receiving notice in 2014 of the withdrawal of control funding too late in their budgetary cycles to be able to adjust their plans). Having a dedicated and consistent programme manager in place contributed significantly to the pilot programme’s effectiveness. Annual reporting by the programme could be more consistent and structured. The issue of budgetary planning is discussed under EQ14. Analysis: Programme level management arrangements of the OPMCP are described in detail in Section 2.2.1. The key elements are: ■ Annual and continuous review processes; ■ Annual reporting; ■ Budgetary planning (discussed in Section 5.2.4); ■ Meetings and actions of the Project Board (discussed in 5.2.1); and, ■ Liaison with the Advisory Group (discussed in Section 5.2.2) Very few consultees raised any issues with the programme level management arrangements, despite the difficulties inherent in managing a programme of this complexity. The main exception to this was in regard to budgetary planning, discussed under EQ14. “It’s all been very useful. The annual report and lessons learned notes have been useful for getting feedback and that over the 3 years has helped modify processes. The system has improved an awful lot since it started.” “I feel the management arrangements have been very good for what is a difficult task involving so many stakeholders both public bodies and private” Interviews highlighted the importance of having had a dedicated and consistent programme manager in place. Given the number of elements and stakeholders to coordinate and manage, it was felt that having this single point of contact driving the process was critical and significantly contributed to the pilot programme’s effectiveness. Some consultees observed that the programme manager could usefully have had a greater level of administrative support; it was only in time for the 2015 season that a dedicated administrator was put in place. “I have a huge amount of respect for [the programme manager]. He knows what’s going on everywhere all the time, so he can be really responsive… he’s a real key element to making this all work.” The pilot programme also benefitted significantly from continuity of many of the core team, particularly the programme manager who started working on OPM in 2012 and has remained in post for the duration of the programme. This was crucial to retain ‘programme memory’ and learning from year to year. A small number of stakeholders noted staff turnover elsewhere resulted in loss of efficiency and difficulties in programme management. “The approach to pest outbreaks diseases is very short term, there’s too much staff and resource turnover so planning ahead is difficult.” “There are situations where we’re working with people in Defra who are new to the subject. Quite frequently that means having to spend time bringing people up to
19 February 2016 49
Evaluation of the Oak Processionary Moth Control Programme - Final Report
speed but quite often they don’t understand the subject matter because they’re new in the post. That can be a drain on resource as it’s not very efficient” One issue appears to be a lack of consistent or formalised reporting procedures. Annual reports and updates varied significantly in content and structure for each of the three years. Reports mentioned in planning documents were not always prepared and published (e.g. Annual Communication Plans). “There has been no formalised reporting process – that could be better.” “The report was quite a detailed for the first year. We had a very detailed control plan, with a summary of what happened that year. Since then, we’ve presumed a lot of the stuff was in there and didn’t need that again. We’ve been quite brief for 2014 and haven’t thought about 2015 yet…. I wonder whether we should be doing a more thorough report of 2015 and the three year programme as a whole.” Rather than there being a three year plan covering the pilot programme as a whole, an OPMCP plan had to be submitted by the Defra and the FC for approval every year. This caused delays to the programme and affected its continuity as the FC and Defra had to stop and wait for approval before being able to move forward. “One of the greatest challenges we’ve had with the whole programme has been the need for year to year approval. It means there has been no long term view of objectives and what we’re going to be enacting.” “There are complex things we’re having to go through every year, with ministerial meetings and so on to flag what we’re proposing to do. That’s been the same every year, and that’s been very frustrating. It’s a time consuming process.” Changes in ministers during the life of the pilot programme meant that additional time had to be spent briefing new ministers and explaining the rationale for the policy goal of containment rather than eradication. Although the need for yearly plans to be developed and approved has meant that the implementation of the OPMCP could be tailored and responsive to each year’s changing circumstances, the delays caused by this process could have a significant effect on landowners. For instance, several consultees thought insufficient notice was given to landowners in the core zone that funding was to be withdrawn in 2015. This information was provided too late for many landowners, particularly larger landowners, to plan for control measures to be funded as budgets had already been set by the time they had been informed. “In the core zone, a lot of people didn’t realise they weren’t going to get funding [in 2015]. When information was passed down, it was very late in the day. Some of the big landowners had set their budgets already so weren’t able to fund any control measures. It’s not that they were necessarily unwilling to implement controls, it’s just that their budgets were already set.” A few consultees noted that such issues meant it would have been beneficial to have a five year programme plan that allowed for more forward planning and better communication with stakeholders. “If the programme was set for 5 years we could really have communicated with people. At the minute, if people ask me what is going to happen next year, I have to say I don’t know. It’s not very good, there’s no consistency, no stability.” “Overall it worked, but if we had a longer control programme, for example, 5 years with funding in place for that time, that would work more effectively.”
19 February 2016 50
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5.2.4 EQ14. Have the OPMCP budgeting arrangements worked? Summary statement: The main budgetary challenge the pilot programme faced related to the alignment of the budgeting cycle with the cycle of annual planning, communication and delivery. The pilot programme experienced difficulties in keeping stakeholders, including contractors, sufficiently informed and in ensuring that funding was available in time for control measures to be implemented. Nonetheless, the process of getting funding in place appears to have become more efficient over time. Day-to-day budgetary planning largely worked well, though again there was year to year learning. Spending on specific activities generally came in close to planned budgets. A considerable underspend in 2013 was due to funding being made available too late in the year for the money to be spent. Budget management was sometimes made difficult by unexpected needs and as circumstances changed (e.g. the need for survey work around a new outbreak area in Guildford in 2015). As the pilot programme gained experience and prices for services became more established it became possible to budget line items with more confidence. Analysis: The OPMCP was funded by Defra and the budget administered by the FC. Control measures were funded either by the FC through the pilot programme or landowners funded the measures themselves (see Section 5.4). It was originally estimated that an enhanced control programme would require a total of £5 million over three years. Funding for the OPMCP had to be applied for every year. In total, just under £4.1 million was spent under the OPMCP. The annual budget fell from £2m to £1.2m over the life of the pilot programme even as the area in which OPM had been found, and the surveillance, area expanded. Actual annual expenditure figures were closer to £1.5 million. “..the money available has been of a similar amount every year, but not taking into account area we’re covering will get larger. We’re having to spread ourselves thinner.” “My one issue is that we’ve had same amount of money over 3 years, which meant it’s been a case of moving money around to different things to make things work” Stakeholders thought the budget that had been set was managed well. Comparison of data on actual expenditure and projected spend for 2013/14 and 2014/15 show that most line items came in close to budget, though there was significant variance in some cases. “The budgetary planning has worked pretty well in terms of day to day issues” The programme budget for 2013/2014 was set at £2 million. Almost every aspect of the OPMCP came in under budget. Total spending was £963,000 less than budgeted (Table 5.2). A substantial share of this under-spend related to the fact that funding became available too late in the day for activities to be organised and carried out in time. Fees paid to an outsourced provider of data services slightly exceeded the budget38. In 2014, the pilot programme’s budget was reduced to approximately £1.5 million. Total expenditure was about £1.6 (an over-spend of ~6%). Almost every item of expenditure came in over-budget, except for the cost of materials and supplies, and aerial spraying. Communication costs exceeded the budget by 48% and survey costs were 14% higher than expected.
38 OPMCP Projected and Forecast expenditure, 2013-2014.
19 February 2016 51
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 5.2 Budgeted and actual expenditure under the OPMCP for 2013 - 2015
2013 2014 201539 Activity Budget Actual % change Budget Actual % change Budget Actual % change Surveying 430,000 330,438 -23% 460,000 522,543 14% 624,500 915,252 47%
Communications 10,000 6,721 -33% 25,000 39,385 58% 25,000 25,000 0%
Database 70,000 81,067 16% 70,000 70,000 0% 70,000 70,000 0% management
Research 75,000 67,128 -10% 144,000 144,000 0% 79,000 112,083 42%
Materials / supplies 10,000 6,929 -31% 25,000 9,148 -63%
Control activities 1,415,000 551,723 -61% 756,000 794,022 5% 393,000 321,045 -18%
Aerial spraying 20,000 3,500 -83%
Total 2,000,000 1,037,077 -48% 1,485,000 1,580,380 6% 1,225,000 1,452,528 19%
Table 5.3 Change in budgeted expenditure over time
2013 -2014 2014-2015 £ % change £ % change Change in budgeted expenditure from previous year -515,000 -26% -260,000 -18%
Change from actual spend to next year's budget 447,923 43% -355,380 -22%
39 Actual calculated expenditure for 2015 not yet available
19 February 2016 52
Evaluation of the Oak Processionary Moth Control Programme - Final Report
In 2015, the budget was set at just over £1.2 million, £350,000 (20%) less than the 2014 actual expenditure (Table 5.3). Total expenditure was £1.5 million (an over-spend of 19%). Communications and database management spend came in on budget, whilst the costs of surveying and research were significantly greater than expected. This is largely due to new outbreaks being identified which required substantial additional survey investment, and implementation of a much larger pheromone trapping programme. The cost of materials and supplies was substantially lower than expected (63% less), as was the cost of controls (18% less). This is largely due to funding for controls being withdrawn in the core zone. An attempt was made to resolve the tension between a reduction in the budget allocation and an expanding outbreak area in 2015 through a decision to stop funding surveys and control measures in the London core zone. Resources were focused solely on the buffer zone and beyond (previously some surveying and controls were being funded in the core, as well as the buffer). This measure did not fully protect budgets: an additional 2km survey of an outbreak site at Guildford had to be undertaken at an additional cost of £52,000. Feedback from stakeholders40 indicated that increased efficiencies had been achieved in the course of the OPMCP, with growing economies of scale and very good coordination between Defra and contractors41. The data suggest that more has been achieved each year with similar amounts in some areas, such as with the communications work stream. The survey cost per tree has remained stable over the three years (at about £10 per tree). The average cost of control increased from 2013 to 2014, control activities appear to have become more cost effective between 2014 and 2015 (see Section 5.4.2). One of the core challenges of the OPMCP from a budgetary perspective was the interaction between timing of Defra’s budgeting cycle and the programme’s annual planning cycle. For instance, budgets were often not confirmed until January / February, giving only a month or two of time before operations had to start with the emergence of larvae and the need to begin surveys and spraying. Operational decisions would ideally have been made in November of the previous year to allow for sufficient time to organise the various elements and activities for the following year. Uncertainty about funding levels made it difficult to decide on the funding to be made available for controls in a context where the period available for nest spraying was fixed and comparatively short. Some consultees also indicated that this lack of clarity and consistency acted as a barrier to contractors making longer-term investment decisions. Largely because of these challenges, and as already noted above in Section 5.2.3, several consultees reflected that at the outset of the pilot programme it would have been better if there had been a greater push for a multi-year funding award at the beginning of the pilot programme. This would have allowed for more flexibility, more stability and would have allowed for better planning. “There has always been a fundamental challenge about the timing of the budget approval process and then making plans for following year. It’s never as early as we would like it to be” “Contractors would have liked more notice in advance of controls being agreed once funding arrangements are put in place. … [But] because of the nature of the budgetary planning process, it made it quite difficult to clarify how much money would be available at different times.” “Defra’s budgetary programming timescales don’t fit with timescales of managing [the pest].”
40 OPM ‘Lessons Learned’ Meeting Minutes. May 1st, 2015. 41 OPMCP Projected and Forecast expenditure, 2015-2016.
19 February 2016 53
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5.2.5 EQ15. Did the programme provide capacity appropriate to the scale and nature of the challenge? Summary statement: 'Capacity' is here considered to include the sum of staff time, staff expertise and funding whilst the ‘challenge’ refers to the policy goals of containment of the south-west London outbreak, and eradication elsewhere. The decisions made in 2012/2013 on policy and on capacity allocation were made with less information than is now available, particularly on the scale of the outbreak and dispersal of OPM. It is now apparent that OPM is more widely distributed across the London area than was understood at the time. This has implications for the effort required for surveying and control. These issues are reflected in consultees’ responses to the evaluation question. The main conclusions are that: ■ The overall capacity allocated was appropriate to the challenge as it was defined and understood at that time; ■ Doing the same thing again, with the benefit of hindsight, programme managers would have allocated capacity differently across the programme activities; more effort would have been given to surveillance, communications, research and administrative support in the early stages of the programme. Providing ‘surge capacity’ to support the core programme team during the periods of peak workload that occur at certain points in the year would have relieved some of the constraints that were experienced. ■ The pilot programme has developed new capacity in areas such as information management and the targeting of controls. There have also been increases in the capacity of the programme’s supply chain to conduct surveillance, surveys and controls. There are few contractors with the capacity to apply controls in the manner required by the programme, although this is increasing; ■ Increasing pressures on the pilot programme’s budget have prompted changes to its (e.g. withdrawal of funding for controls in the core area) whose consequences for the core programme objectives are not yet fully understood. Analysis: The pilot programme was supported by 1.7 FTE staff, including one full-time member of staff to oversee, organise and implement the OPMCP. Staff members with a wider range of expertise were called upon to input into, and assist the programme. Aside from the programme manager and administrative support provided late in 2014, inputs from other staff were not directly funded through the OPMCP budget. This included a communications specialist, scientific expertise, expertise in tree health and pest management, administrative support (available only for the 2015 season), and policy expertise. Views amongst consultees on the adequacy of capacity varied. Some thought that, on the whole, the pilot programme’s capacity had been sufficient while others indicated that more capacity would have led to greater impact. A few consultees noted that capacity has become increasingly stretched as the known area of infestation has increased in size and new outbreaks have been identified. The available resources and the available staff and expertise (for coordination, surveying, surveillance and control) have had to cover a larger area. The withdrawal of funding for surveys and control work from the core zone, putting the sole focus on the buffer zone, was part of the response to this. A few consultees suggested that this decision has meant that it will be harder to contain OPM. “For the same level of control, we now need to employ more people, if we need to keep managing it at the same level we’re doing now.”
19 February 2016 54
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“It has become apparent as the pest has spread that funding is too thinly spread to enable the FC to be pro-active in the ever-increasing outer perimeter of the distribution area and the core outbreak zone. While understandable in the face of limited resources, pulling of funding in the core seemed short-sighted and is likely to have inevitable consequences as the numbers are likely to rise here, putting more and more pressure on management resource and the control zone” Some consultees noted that more capacity would have been beneficial in areas such as communications, research, and administrative support. More administrative capacity would have allowed for greater oversight of contractors and control activities, which might have provided better outcomes and information flows. There is also significant variation within the year in the programme workload – better matching of support capacity to demand would have helped. “On the communications side …there were a number of things we would have liked to do but we didn’t manage them because time and resource ran out” “[Capacity] hasn’t been sufficient. [The management role] has been too big, and involves too many things. That needed more supported, especially during the season to ensure there was a closer eye on contractors and to support them more” The issue of contractor capacity was also raised a number of times. It was felt that given the limited time window in which control measures could be carried out, having more capacity available in this area would have been beneficial. Only three contractors with the expertise, experience and equipment came forward and were awarded a framework contract for OPM control work. The contractors invested a significant amount of time and resources to enable them to meet the needs of the OPMCP. “There is a lack of capacity on the contractor end potentially … If they had had more staff, or if we’d had more contractors, we could have covered more sites.” “There were sites where nests were identified late on which hadn’t been cleared because there weren’t enough people to do the job” Almost all other stakeholders thought the level of expertise of those closely involved in the OPMCP was sufficient. One consultee however referenced reports of surveyors not having enough experience. The same issue came up in a response to the landowner survey; the respondent was not confident that the surveyors had been sufficiently experienced to conduct a proper assessment. “I have heard concerns about the use of casual labour for surveying and the consequent likely poor quality of some of the survey results” “Two young persons called and asked to check for OPM, access given. One thought they could see something with binoculars (very vague assessment). … We phoned to check if anyone fully qualified would recheck for the presence of OPM….Eventually a contractor for the FC revisited and confirmed no sign of OPM.”
5.3 Operational delivery
5.3.1 EQ16. Has the OPMCP day-to-day operational management function worked as expected? What has contributed to this result? Summary statement: Day to day operational management worked very well. The OPMCP benefitted greatly from having an effective programme manager. That the programme model was delivered and improved over time owed much to having a manager with a mix of technical, management and engagement skills who was already familiar with the institutional and policy context. The
19 February 2016 55
Evaluation of the Oak Processionary Moth Control Programme - Final Report
continuity associated with having the same person being in post for the duration helped with the accumulation of knowledge within the pilot programme. Consultees’ comments about potential improvements focused on structure and communications. Several of the programme components relied on single points of contact which risked the creation of bottlenecks. Feedback suggests some issues with communications with landowners, e.g. in relation to SPHNs and on response times. Analysis: The operational elements of the pilot programme are discussed in detail under the relevant EQs: ■ Control operations – Section 5.4; ■ Survey and surveillance efforts – Section 5.5; ■ Communications – Section 5.6; and, ■ Research – Section 5.7. Consultees thought that the pilot programme’s operations have worked well overall. Many noted in particular that the programme manager has been particularly effective in coordinating the different elements of the programme. “Day to day the programme has been very good, especially in terms of surveying and control.” “On the operational side, the programme seems to have been efficient” “I think it’s been great. …. The programme has been able to be really responsive” "It’s all gone pretty well. [The programme manager] has done a great job of pretty much delivering what the control plans said we would deliver.” The evidence discussed in the relevant sections elsewhere in this report on the individual operational elements of the OPMCP show that on the whole operations have worked well and as expected. For instance: ■ The organisation and implementation of control operations (see Section 5.4) worked well, despite several logistical challenges. Only a small proportion of the sites identified for spraying were not treated in each year. ■ Processes for data capture and management worked well overall (see Section 5.5). Large quantities of site and surveillance data were collected and used to inform programme operations. ■ Significant efforts were put into raising awareness and engaging with stakeholders. The evidence suggests that this has been largely effective and has led to collaborative relationships being developed with a number of stakeholder groups helped to support and promote the work of the OPMCP (see Section 5.6). ■ Processes for review and continued scientific input have resulted in improvements to the operational delivery of the programme (see Section 5.7). The overall picture on the programme’s operational delivery is therefore very positive. However, there were a few areas where the evaluation findings suggest improvements for the future. Several elements of the pilot programme had a single point of contact. Although rarely a problem in practice, this did create potential bottlenecks in key processes (e.g. programme management, communications, data management and research) that relied heavily on the expertise and availability of particular individuals.
19 February 2016 56
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“[The OPMCP] is supported by research and here it relies on the input from a small number of key individuals so there is an element of risk involved if research staff are not readily available” Responses to the landowner survey suggest more communication with, and quicker responses to the concerns of affected landowners would have been useful. Some landowners reported that the FC was slow to respond to their enquiries or did not respond. In two known cases this delay led landowners to carry out their own control measures. Data on response times were not available to the evaluation. “Having been notified by the FC the oak tree on our premises might have OPM…I phoned the FC to explain that as a pensioner I did not have the means to pay for future treatment. I was told he knew nothing personally about it and would contact me later. So far I have heard nothing and no inspection has been carried out” “We were advised that the Forestry Commission would come and survey, spray and/or remove nests. When we advised the Forestry Commission to the presence of OPM on our estate, we were advised the budget had been spent and the Forestry Commission could no longer carry out the work. The initial communication was misleading that led to it being incorrect, this was disappointing." “[I was told] to contact the Forestry Commission and report it, which I did immediately. I took six or seven days before I heard back from the Forestry Commission. In the meantime I had found out that my council (Kingston-Upon- Thames) does not help financially, so I had gone to Homebase, bought myself a flamethrower and full body suit and took care of the nest myself.” “We had hardly any input or information from the Forestry Commission and felt felling the tree was the only option and that we were under pressure to do so.” The evaluation found some limited evidence that communications with landowners weren’t always appropriate to their situation, and hadn’t taken into account what had previously happened on the property. One landowner noted that this gave the impression that the pilot programme was disjointed and could discourage landowners from supporting the programme. “The small fastigiated oak tree was removed with council approval well before your inspectors arrived. That however did not stop F.C. sending a computer generated letter threatening all sorts of sanctions if I did not deal with the OPM. Somehow that was a bit disjointed. Perhaps that might put certain people off from alerting the FC in future” The landowner survey suggested that some landowners would welcome more information about what had been done on their site, on OPM and the OPMCP more generally, and on what this means for them. Part of the problem may be that the FC does not have email addresses for all the landowners involved in the OPMCP. The type and frequency of communications that can be sent to landowners is restricted if postal addresses are the only contact data available. For instance, the OPM Tree Owner’s Manual and the regular updates on the pilot programme’s progress were only sent to landowners for whom an email address was available. “So far as I am aware, we did not receive any information about the OPM after the nests were found this year and the contractors came to remove them. We are concerned about the impact of OPM on our site.” “It would be useful to receive feedback on what impact recent preventative measures have had. … Updates via a website link or news update would be useful. Only then can we properly assess whether we should continue treatment or fund it” “[Landowners] need more detail on the spraying and nest removal programme”
19 February 2016 57
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“Findings about the number of nests removed [on the site] and records of spraying would have been useful.” “More information [is] required [for landowners] – [there has been a] lack of contractors to assist. [The] Forestry Commission [has] not [been] very helpful” A few consultees and respondents to the landowner survey also mentioned that day-to-day communication with landowners about what controls were being conducted, and when, could have been improved. Contractors noted that they could also have been given more, and better, information for contacting landowners and gaining access to sites (see Section 5.4.1 for more on this issue). “The initial communication of when [control] teams would show up was poor, leading to complaints for instance about spraying being done at 3 a.m.” "Possibly there should have been better communication as to when contractors were attending sites.” “It would have helped to have the correct information, like [landowners’] contact details.” The pilot programme developed a system for bulk issuing of SPHNs, increasing the efficiency of the notification process. The use of SPHNs prompted responses in the landowner survey and among consultees. Landowner survey results suggest that the use of SPHNs could create an unwillingness in landowners to support the work of the programme. One landowner suggested that such communications need to be less confrontational, especially where the SPHN is the first contact with a landowner. A few consultees also noted that the lack of enforcement means that SPHNs are largely ineffective. The efficacy issue was also reflected in one of the responses from the landowner survey. “The statutory side, the ability to force and enforce statutory notices has been an issue. That didn’t really happen and it could have and should have done” “We are aware of a large private property [with] heavily infested oak trees where no management of OPM has taken place. This property has been served with at least one Plant Health Notice, to our knowledge.” “The initial contact letter saying OPM may be present in a garden we were responsible for was way, way too threatening and unpleasant…The best way to get peoples' co-operation is not to come in all guns blazing with a lengthy letter of what will happen legally if they do not co-operate and with no indication either of what works and costs may be involved if there is not central funding. We found it worrying and upsetting” As mentioned in earlier sections, some consultees thought landowners should have been informed sooner about the decision to withdraw funding from the core zone, and that the message could have been clearer. "There was a lack of understanding of non-treatment in the core zone. It was not explained early enough about the change to core zone status. That could have affected the way people managed things. [Landowners] were saying, we’ll wait to see what the FC does. But by the time that message was too late”
5.3.2 EQ19. Did public financing of controls, both in terms of eradication and containment, improve the timeliness and efficacy of controls as intended? Summary statement: The availability of public funding for OPM controls allowed the FC to establish a centralised, coordinated approach which enabled much more control work to be carried out in the short period of time in which spraying is effective. Qualitative evidence suggests the timeliness
19 February 2016 58
Evaluation of the Oak Processionary Moth Control Programme - Final Report
and efficacy of controls increased very significantly. The evidence suggests that public funding, by addressing landowners’ fear of the financial implications of OPM, has made landowners more willing to give contractors access to their sites for survey and control purposes, and more willing to report sightings of OPM. It is highly likely that without OPMCP funding, fewer controls would have been carried out and that they would have been applied in a more ad hoc manner that would have reduced their overall effectiveness. Rates of re- infestation from sites where no controls were applied are likely to have been higher. As no data were collected on the proportion of landowners carrying out control work prior to 2013, no comparative quantitative evidence is available to determine the extent to which OPMCP funding has improved the number and effectiveness of controls being carried out. Analysis: Prior to the OPMCP, landowners were issued with SPHNs where OPM was thought to be present or where OPM had been identified. Landowners then had to organise and pay for control measures to be carried out on their site. Under the OPMCP, funding was made available on a case by case basis to pay for surveying and control measures42. Funding covered both pesticide spraying (two sprays, ten days apart) and manual nest removal. Approved contractors were used; the work was organised by the FC. A centralised, coordinated approach was established with the additional financial resources to plan and execute the surveying and control efforts. This included a database to collect information on where OPM had been found. This was used to plan and coordinate the control work. The qualitative evidence suggests that the availability of funding to establish this centralised approach to coordinating control measures significantly improved the timeliness and efficacy of control measures. It enabled control measures to be properly planned, and ensured that they were applied in the short time period in which spraying is effective. The public funding has meant that more control measures have been carried out than would have been the case if landowners were funding their own controls. Funding for surveys resulted in more OPM nests being identified, and so more OPM nests being sprayed and removed. There is some evidence to suggest that the public funding has made landowners more willing to report sightings of OPM as the financial implications to them of doing so are reduced. “The availability of public funding has allowed us to set up a centralised programme. Without that, we wouldn’t have had nearly the level of success that we’ve had. Undoubtedly, that’s the single most important component” “As a small charity the finding of OPM was worrying with financial implications….If a fund was well known for smaller and private persons to report without fear of costs that would be great.” There was a general consensus amongst consultees that public funding made a significant difference to the implementation of control measures. It was seen to be particularly important for smaller landowners who otherwise had little incentive to implement them. “Public financing allowed controls to be put in place. This has been a huge improvement on the previous situation…You won’t get compliance or any significant level of control if landowners have to foot the bill for the treatment.” "If government had not funded [controls], then a lot of the sites would not have been treated because people didn’t want to pay for it or didn’t want the hassle.” "For commercial landowners, there is an absolute commercial mandate to control [OPM]...It’s more intermediate and small landowners with back gardens who would
42 For the purposes of this report, this includes both chemical spraying and manual nest removal.
19 February 2016 59
Evaluation of the Oak Processionary Moth Control Programme - Final Report
take the minimum approach. There, funding of control has been very positive. Without it, cooperation is difficult to obtain due to the high cost [of controls].” Responses to the landowner survey suggest that without public funding some landowners would be less inclined to apply control measures. Other possible impacts include landowners organising their own controls, which could result in health implications if work is done by people lacking the proper training, expertise and equipment, and a greater number of landowners felling the affected oak trees to avoid ongoing control costs. "We have limited fund raising abilities and OPM treatment is likely to be way outside our resources” “I phoned the FC to explain that as a pensioner I did not have the means to pay for future treatment.” “We applied to have the tree removed altogether but this was turned down by the Council. I do not accept that landowners should have to pay for removal…. I would be happy to let [the FC] carry out such work on our property at their expense.” The willingness of landowners to carry out control measures seemed to be affected by whether they thought others were carrying out controls. Where this was not the case they thought their efforts were being undermined as re-infestation from neighbouring sites was more likely. In the absence of a centralised and coordinated control programme, it is likely that this issue would have been more pronounced and few landowners would feel that they had much reason or incentive to implement control measures on their site when there was little chance that other sites were doing the same. “We have two oak trees in our garden…I believe neighbouring trees were infested but not treated which undermines our willingness to treat” “Government help is needed for the control of OPM. If I pay for the control of OPM on my trees but my neighbour doesn't then my efforts are wasted.” “It is largely a waste of time for me to spray my trees if adjoining properties do not” “I understand that Kingston Council hasn't been addressing the problem so now I feel like there is no point in me as a private person addressing it when they aren't.” Some consultees noted that these issues were already becoming evident in the core zone after funding was withdrawn in 2015. “[As a result of] the withdrawal of funding from the core… we are aware of private owners of oak trees planning to have them felled rather than face the year on year cost of OPM management” “I feel that with the control of OPM in the core zone now needing to be funded by the landowner that there is an incoherent control of the pest - with possibly some landowners paying to control it whilst next door they are not doing anything” The majority of respondents to the landowner survey indicated that knowing the FC would pay for controls increased their willingness to provide contractors with access to their site for surveying and application of control measures (Figure 5.1). The majority of landowners (54%) responding to the single site survey received funding from the FC for controls. Responses to the prospect of the withdrawal of public funding were mixed (Figure 5.2). Landowners were slightly more open to continuing with manual nest removal than with pesticide spraying. A larger proportion of respondents to the multiple-site survey would stop controls if funding ceased than respondents to the single site survey. Just over a third of respondents responsible for single sites paid for their own control measures in 2015. Some provided data on their average annual spend on OPM controls. The average annual cost was just under £5,000. The average annual cost to private homeowners was less, at just over £1,000 (see Section 5.4.2). Some respondents indicated
19 February 2016 60
Evaluation of the Oak Processionary Moth Control Programme - Final Report
that having to pay for control measures themselves had reduced their willingness to provide contractors with access to their sites for surveying and carrying out control measures. Figure 5.1 Impact of the availability of public funding on landowners’ willingness to support the OPMCP “To what extent has the following affected your willingness to provide FC and/or contractors with access to site for surveying?” (N=100)
Knowing FC pays control 40 28 10 11 20
Knowing I/we pay control 12 22 18 8 7 33
Increased considerably Increased somewhat No impact Decreased somewhat Decreased considerably Don't know / Not applicable
“To what extent has the following affected your willingness to have control measures carried out on your site?”(N=100)
Knowing FC pays control 38 29 11 22
Knowing I/we pay control 17 24 18 8 9 24
Having to organise control on own 14 23 17 9 9 28
Increased considerably Increased somewhat No impact Decreased somewhat Decreased considerably Don't know / Not applicable
Figure 5.2 Likelihood of landowners continuing with controls in the absence of public funding “Please indicate how strongly you agree with the statement that you would do the following…” (N=55)
Continue spraying w/o public funding 3 13 21 10 5 3
Continue manual removal w/o public funding 5 18 20 3 5 3
Strongly agree Agree Undecided Disagree Strongly disagree Don't know / Not applicable
Of those landowners paying for control measures themselves in 2015 only 20% had previously received funding from the FC. Of these landowners, an equal proportion disagreed that the withdrawal of funding for controls had reduced the level of effort made to tackle OPM as felt that the withdrawal of funding had not had an impact on the effort put into managing OPM (Figure 5.3). Similar results were found for respondents to the multiple site survey. Sixteen percent of landowners responding to the survey had never received FC funding for control measures. Of these, almost all thought public funding would improve the effectiveness of OPM management and control on their site (Figure 5.4). All respondents representing multiple sites felt that public funding would have improved the effectiveness of OPM management and control on the site.
19 February 2016 61
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure 5.3 Impact of the withdrawal of public funding on landowners’ management of OPM “Please indicate how strongly you agree with the following statement: The change from having control measures publicly funded to having to pay for control measures myself/ourselves has reduced the level of effort put into the management of OPM on the site?” (N=21)
1 Strongly agree 2 5 Agree 6 Undecided
Disagree 3 4 Strongly disagree
Figure 5.4 Impact of the withdrawal of public funding on landowners’ management of OPM “Please indicate how strongly you agree with the following statement: Having public funding for control measures available would improve the effectiveness of OPM management and control on the site?” (N=16)
1 1 Strongly agree Agree Undecided 4 10 Disagree Strongly disagree Don't know / Not applicable
Overall, the qualitative evidence suggests that public funding for control measures has had a positive impact on the willingness and ability of landowners to carry out controls. It is highly likely that without OPMCP funding, considerably fewer controls would have been carried out, and that they would have been in a more ad hoc manner. This would have reduced the effectiveness of any control measures which were implemented as the rates of re-infestation from sites where no controls were applied would likely have been high. Data on the proportion of landowners carrying out control work prior to 2013 were not collected before 2013 so it is not possible to do a comparative analysis of the impact of public funding on control rates under the pilot programme with those in preceding years.
19 February 2016 62
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5.4 Implementation of control measures
5.4.1 EQ17. What worked well, and what did not work well, in the implementation of control measures? Has the implementation of control measures worked as expected? Summary statement: The organisation and application of control measures (including both spraying and manual nest removal) has worked well, especially when the many challenges are taken into account (e.g. the very short period in which spraying is effective, issues of access and weather conditions, uncertainty about funding). Over the programme period, more than 90,000 trees on almost 700 sites were sprayed and almost 21,000 OPM nests removed. The OPMCP, by providing a single centre of operations, enabled adoption of a much more consistent and methodical approach. Many consultees thought the implementation of controls had worked as well as could be expected. Notwithstanding the successes, some areas for improvements were identified. Communications between the contractors and the FC on progress, and communication with landowners on when and what controls were being carried out could both have been better. Greater oversight of contractors and their work would have provided additional assurance that the controls were being applied and to the appropriate standard. There are also some open questions about which pesticides and which methods of application are most appropriate and useful. Analysis: An overview of the implementation of controls is provided in Section 2.2.1. Despite the challenges facing the FC and control contractors, a large number of control measures were applied each year (Table 5.4 ). Over the programme period, more than 90,000 trees on almost 700 sites were sprayed and almost 21,000 OPM nests removed. “The efficacy of pesticide spraying is limited by the number of contractors with suitable equipment, the often inaccessible terrain in the park, the height of the trees, the limited time-frame and the weather. The necessity of closing off areas of the park while spraying takes place is problematic. Advance warning of temporary restricted access often backfires because spraying has to be cancelled at the last minute due to weather conditions. Attempts to avoid this and spray at night has brought complaints about noise, as above, and also curtails the effectiveness of the operation due to difficulties identifying the trees to be sprayed and the often challenging access, by contractor staff who are not familiar with the park or the trees.” For both London and Croydon / Bromley, the number of sites sprayed did not change significantly between 2014 and 2015, but the number of trees sprayed fell considerably. It appears that the numbers of trees sprayed per site fell in later years (Table 5.5). More sites with a smaller number of trees were being treated in 2015 compared to the previous two years (see Section 5.4.2). Funding for treatment in the London core zone was withdrawn in 2015 and so proportionately fewer sites and trees needed to be sprayed. The number of nests removed per site over time shows no clear trend. A detailed breakdown of the scale of the control activities carried out in each of the outbreak areas is provided in Annex 15. The organisation and implementation of control measures worked well, a success noted by several consultees. Despite the timing and logistical challenges, only a small number of sites identified for spraying were not treated (six in 2013; seven in 2014; four in 2015). “[The application of control measures] has become more nuanced. There is more expertise about where to do the spraying, and it’s now all linked to GIS. I think it’s getting better and better.”
19 February 2016 63
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“It’s all worked as well as resources have allowed. There isn’t anything we could have done better in terms of the technology or process, it’s a matter of scale.” Not all sites where pesticides were sprayed were subsequently visited for manual nest removal, and vice versa (Table 5.6). In 2014, considerably more sites were sprayed than had nests removed. Nest removal is a more labour intensive process than spraying, and it is possible that in some cases it was not seen as necessary, e.g. where pubic exposure to nests was unlikely to be an issue. However, some consultees suggested the lack of manual nest removal on some sites was sometimes due to a lack of contractor capacity. The issue of contractor capacity was raised by consultees on several occasions (see Section 5.2.5). In 2015 nests were removed from more sites than were sprayed, suggesting that new nests were found after the spraying season had passed. “There’s been a lack of nest removal capacity, especially when working on a short timeframe. … There are sites where nests were identified later on which hadn’t been cleared because there wasn’t enough people to clear them.” Control measures seem to have had a positive impact on OPM control. Data on the number of sites sprayed and levels of infestation in the following year show that a significant proportion of the sites surveyed the following year were no longer infested. In Croydon / Bromley controls appear to be have been particularly successful: 65% - 73% of sites were not infested the following year. In London, figures were lower: 38% - 42% of sites were not infested after spraying in the previous year. Other factors could have affected OPM mortality rates, including weather and natural population dynamics. Only a small number of sites were re-surveyed in London in 2015. This may reflect the withdrawal of funding from the core zone. Table 5.4 Scale of the control activities carried out across all areas of infestation (2013 - 2015) NB: includes London, Croydon / Bromley, Pangbourne43 and the Olympic site44
2013 2014 2015 Total # of sites sprayed 157 306 214 663 # of trees sprayed 39,283 45,924 5,547 90,754 # of sites where 158 180 235 573 nests were removed
# of nests were 5,078 13,036 2,724 20,838 removed Table 5.5 Change in the scale of activity per site over time (2013 - 2015)
2013 2014 2015 Number of trees per site requiring pesticide spraying London 118 131 27 Croydon / Bromley 308 166 28 Number of nests per site removed London 8 4 5 Croydon / Bromley 38 95 13
43 Spraying was carried out in Pangbourne in 2013 and 2014 as a precautionary measure. No nests were found and therefore no nests removed 4444 Only one nest was removed from the Olympic site in 2015
19 February 2016 64
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 5.6 Number of sites where spraying and nest removal were carried out (2013 - 2015)
Control measure 2013 2014 2015 Pesticide spraying (# of sites) 156 168 200 Manual nest removal (# of sites) 156 304 222 Table 5.7 Number of treated sites surveyed/re-infected45
# of sites # of same sites % of re- % of re-surveyed % of sites re- sprayed surveyed in surveyed sites sites no longer surveyed following year still infested infested London 2013 – 2014 108 66 62% 38% 61% 2014 – 2015 216 45 58% 42% 21% Croydon / Bromley 2013 – 2014 48 34 35% 65% 70% 2014 – 2015 76 48 27% 73% 63% The OPMCP developed and applied a centralised approach which meant that the application of control measures was better organised and coordinated than had previously been the case. The many different types of landowners and site conditions needed different approaches and equipment. The pilot programme coordinated efforts so that the most effective and efficient approach was adopted, although a few consultees highlighted that challenges remain. Communication and collaboration among contractors was essential, and seems to have worked well. “[The OPMCP] has meant that we could get sites done in a more logical way… Having that centralised coordination function makes the job so much more efficient and effective. The programme means there is a greater chance of the job being finished in the window you’ve got” “The general collaboration, and landscape scale approach that’s being employed makes a big difference.” “[Contractors] have collaborated between them and agreed where it is more appropriate for one to do one site or the other.” “Coordinating spray operations between many different landowners, (both large & small and public & private) has proven difficult and remains a major challenge” The OPMCP database was an important component of the system developed for rapid deployment of controls. It also enabled the programme to be much more responsive to changing information and circumstances, such as the discovery of new areas of infestation. “Getting things done in that small window has been quite good actually… [because of the database,] we’ve been able to generate job sheets instantaneously, that part has been really good”The pilot programme sometimes had problems accessing the information on site ownership that was needed to secure access (one respondent suggested that more use could have been made of the land registry). The updating of ownership and access data to inform control activities the following year was also an issue. “The biggest problem that we’ve had is knowing who owns certain bits of land. It’s difficult to contact landowners and get permission. There are a few sites that we’ve
45 The data are grouped by the year to which the infestation was relevant. For example, where nests found during the winter 2014 survey they are recorded as being infested in 2013. Where surveyors recorded that the trees were ‘previously infested’ in the winter survey they have been analysed as being infested in the previous summer.
19 February 2016 65
Evaluation of the Oak Processionary Moth Control Programme - Final Report
never managed to get on to. Nobody will take responsibility for the land, areas are locked and we’ve not been able to access them.” Despite the difficulties, consultees were satisfied with the contractors’ work, some noting that contractors had made extensive and commendable efforts to meet the FC’s needs despite the numerous challenges. “Given the short times available [for carrying out controls], they did a fantastic job.” "I’ve been very impressed by the contractors and how they’ve tackled the challenge. They’ve put a lot into it.” One issue emerging from consultations was that contractors did not keep the FC sufficiently informed of their progress, especially when sites scheduled for spraying had not been treated. This made it more difficult for OPMCP staff to ensure sites were sprayed in time. The issue was sometimes attributed to a lack of capacity on the part of the contractors, and the need for them to prioritise control activity rather than communication. “We could probably do more with more staff to supervise control contractors…We only found out toward the end of the season that [contractors] didn’t manage to do some sites. We needed to know that earlier.” “The information hasn’t been forthcoming. [Contactors] have been concentrating on making sure sites are sprayed, providing information is a lower priority. It’s potentially an issue of a lack of capacity on contractor end.” Some consultees suggested there should have been greater oversight of the contractors and their activities, not just to ensure that activities were properly reported but also to check that controls had been carried out appropriately. This lack of oversight seems to have created a degree of uncertainty amongst some consultees about the effectiveness and suitability of the work being carried out by contractors. “There’s a concern about whether spraying has been done once or twice. Different contractors do not seem to be spraying twice which is the recommended application rate. I think its logistics of timing and everything and it being a difficulty, rather than things not being done deliberately” “I think there is still a degree of doubt and uncertainty how well control measures are being applied…There’s always a slight doubt, for instance about how much care is being taken.” “I’ve heard there has been some variability between contractors in terms of efficacy of treatment. Some haven’t proved as effective as it should have been. Whether that’s a lazy day or conditions are not quite right I don’t know.” Although considerable changes and improvements have been made to the management and organisation of control activities over the three year programming period, there were few changes to the approaches used. This is despite there being a number of options available and some uncertainty about what technique is most effective. Contractors felt that having the flexibility to be able to use different techniques and approaches, rather than having them stipulated in the framework contract, had been helpful. “There are different techniques, and no conclusions at this stage about which is best.” “Spraying from the ground has had mixed successes with unexplained success and failures in some areas.” “The FC have allowed a range of spraying techniques [to be used] and that has worked well. You need different machines for different environments. It was good of them not to dictate what was acceptable to use or not to use.”
19 February 2016 66
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Early research into improving the effectiveness of control measures produced few conclusive results (Section 5.7.1). More recent (as yet unpublished research) provides some insights into the effectiveness of control measures, finding that ground spraying is much less effective at controlling OPM where nests are found in the canopy above 20m. The relative merits of aerial spraying and ground spraying were discussed by several respondents. Some noted that aerial spraying had advantages that included: ■ Being more cost effective; ■ Being more effective in terms of controlling OPM as more of the affected canopy could be reached; ■ Application could be much more targeted; ■ Being less detrimental to biodiversity given that initial research suggests that the pesticide drift is more limited than with ground spraying; and, ■ Poses fewer access issues than use of ground crew and equipment. The use of aerial spraying was cited by some respondents as a significant contributor to the relative success of the eradication efforts in Pangbourne. “Helicopter spraying looks to be very effective from what happened in Pangbourne…It’s far more accurate than ground spraying.” "The aerial spraying trial in Pangbourne worked really well... We don’t know how effective it was in controlling [OPM], there’s no way of assessing the full efficacy of the treatment as the numbers were so low. But the method as a whole worked well and could be applied elsewhere.” A few respondents were strongly opposed to aerial spraying. Others noted that, despite the benefits, the logistics of organising and carrying out aerial spraying in the face of public perception and opposition, especially in urban areas, meant the disadvantages outweighed the advantages. This is partly why aerial spraying was not used in any other areas of infestation. “In terms of aerial spraying, it would be potentially much more effective but it comes with the biggest logistical nightmare…Then there’s the biodiversity considerations, and having to engage with people, and getting their buy in to do it would be very difficult.” "There’s not enough evidence [about aerial spraying] to base decisions on but initial evidence suggest that it’s better, purely from an efficacy and control perspective. But a major challenge is public perception… It requires a huge amount of resource to manage it because of the public interest in it.” “There is a certain amount of public criticism about aerial spraying. Some NGOs think its excessive, unnecessary, ineffective.” “Aerial spraying is a very blunt tool, and I’m not sure it’s worth the effort.” The type of pesticide used is also relevant. Bt, the most targeted of the available pesticides, was used in almost all cases. Other chemicals (especially deltamethrin) affect a wider spectrum of species and therefore cause more collateral damage to biodiversity. Some respondents thought that use of these alternatives would be justified in certain cases to increase control efficacy, such as sites where the priority is public health and where the potential biodiversity impacts are less significant. “We’ve assumed the use of Bt is the right approach. But actually, have we dismissed more powerful insecticides without actually thinking there may be a place for them in certain situations? The control programme has been about large scale, low impact spraying. There is a debate to be had for a risk based approach where you could involve a harsher range of chemicals where appropriate, where biodiversity and things like that are a secondary concern. For instance, in very high
19 February 2016 67
Evaluation of the Oak Processionary Moth Control Programme - Final Report
profile recreational sites or schools where the most important thing is public health.”
5.4.2 EQ18. Has OPMCP improved the cost-effectiveness of control activities as expected? Summary statement: Overall, the available evidence suggests that the OPMCP improved the cost effectiveness of controls for the average tree, but by less than was expected. One contributory factor is that for all three years, most of the treated sites only included ten or fewer oak trees which were sprayed per site so economies of scale were harder to achieve (average treatment costs drop significantly as the number of trees treated per site rises) Detailed information on control costs was only collected from 2013 onwards. The few data available from the prior period suggest that costs prior to 2012 ranged from £370 to just over £1,000 per tree, but were lower in larger woodlands and parks (at £50-£100 per tree), due to economies of scale and the use of volunteer labour before the OPMCP started.46 The impact assessment for the OPMCP assumed treatment costs of £100 per tree. Data from the OPMCP suggest that the actual average costs per site have been higher than this estimate. Programme cost data needs to be treated with some caution. They indicate an average cost per tree for pesticide spraying of £120 - £180, based on calculating - for each year - the average cost per tree for each site, and then taking the mean of those estimates. The crude average (i.e. total expenditure on spraying divided by total number of trees sprayed) is much lower (£10-£33/tree) because of the economies of scale achieved at a comparatively small number of sites that had a large number of trees. The average cost of manual nest removal fell steadily over the three year period from of £138 to £95 per nest, calculated on the same basis.47 The crude average cost of nest removal was also lower (at £40-£87) but by a small margin than for spraying (indicating that – as would be expected – the economies of scale are smaller for nest removal than for spraying). The average combined cost per site for both spraying and manual nest removal under the OPMCP is less than what landowners reported paying for their own controls but this result must be treated with caution as the evidence base is small. Analysis: One of the expected benefits of the OPMCP was that a centralised, coordinated approach to organising and carrying out control measures would mean that the average cost of controls would fall due to economies of scale. An FC framework contract was used to commission control work. It enabled the FC to assign multiple sites in a given area to the same contractor, thereby creating efficiencies in planning, organisation and travel time. Many of the consultees thought this had increased the cost effectiveness of OPMCP control measures. “Because the control work is covered under one contract, it’s far more cost effective than each organisation or landowner having to arrange controls themselves. It’s more cost effective overall, and a lot more efficient” “Because of the framework contract, we’re able to plan routes rather than ping ponging around the place.” Data on the costs of control suggest that the OPMCP has improved the cost effectiveness of control activities for the average tree, but by less than was originally expected. The absence
46 Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment. 47 The analysis has not been adjusted for inflation; 2015 nominal prices are being compared with 2012 / 2013 / 2014 prices. Inflation over this period is not sufficient to explain the difference in the observed amounts.
19 February 2016 68
Evaluation of the Oak Processionary Moth Control Programme - Final Report
of detailed evidence on the cost of controls prior to 2013 makes the size of the gain difficult to judge. Caveats and limitations of the data are discussed in the box below. Landowners responding to the online survey were asked to provide data on the average annual cost of controls, where they had had to pay for controls themselves (see Table 5.8). The average reported annual cost for single sites was just under £5,000 (based on 31 responses). The average annual cost to private homeowners was much less, at just over £1,000. Public bodies, private companies and charities faced higher average costs as they were more likely to be responsible for a greater number of affected oak trees. However, half of those sites for which data were provided had fewer than five trees on the site. Only four of the sites had more than 500 trees.48
Caveats to the use of OPMCP data on the cost of controls There were three contractors on the FC framework contract used for the OPMCP. The FC is charged on daily fee rates rather a charge per unit (e.g. per tree). There is variation in the rates charged by the contractors. Hourly rates for spraying (on individual sites) ranged from £62.50 to £360 so the cost of treating a single site in a single year depends on who did the work. The total control costs vary depending on the relative size of the allocations of sites given to individual contractors. There are also some inconsistencies in the contractor data, such as where records indicate that trees have been removed from sites but no costs have been recorded. Changes in FC funding between years mean that some sites were funded by the FC in one year and not in other years. Control costs are influenced by the condition of the site. Those that are more difficult to access, or need pre-spraying clearance are typically more costly to treat. The cost data do not take this information into account. The number of trees being treated per site also affects the average cost per tree; this factor has been taken into account in the analysis. Costs were not adjusted for inflation.
OPMCP data are available on the average cost per site for pesticide spraying and manual nest removal. However, because the relative share of costs relating to manual nest removal and pesticide spraying is unclear, comparison of OPMCP costs and those reported by landowners funding their own controls is difficult. Data from landowners are likely to relate to both the costs of pesticide spraying and manual nest removal, as many landowners responding to the survey implemented one or both types of controls. Moreover, the costs per site vary considerably depending on how many trees are being treated and other site conditions (e.g. difficulties of access). The average combined cost per site for both spraying and manual nest removal under the OPMCP (Table 5.9) was less than the average cost reported by landowners paying for their own controls (e.g. £4,988 for landowner payments vs. £1,342 under the OPMCP in 2015). This result should be interpreted with caution given the small sample of landowner data and the various other caveats noted above. Table 5.8 Annual costs of control measures for landowners paying for their own controls
All Private Private Charity Public body respondents homeowner company Average cost £4,998 £1,011 £9,842 £3,167 £11,400 Minimum reported cost £100 £100 £200 £500 £200 Maximum reported cost £80,000 £4,800 £80,000 £10,000 £25,000 Total responses 31 13 8 6 3
48 Of the 31 landowners providing cost data on the average annual cost of controls, 15 had 0-5 oak trees on site, 7 had 6-50 oak trees on site, 5 had 51-500 oak trees on site and 4 had 500+ oak trees on site.
19 February 2016 69
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 5.9 Total average annual costs per site of spraying and manual nest removal under the OPMCP
2013 2014 2015 Average cost £3,198 £2,825 £1,342 Minimum cost £123 £151 £188 Maximum cost £128,761 £107,483 £23,765 The few data available on the costs of controls per tree prior to the OPMCP suggest that costs for landowners paying for their own controls could range from £370 to just over £1,000 per tree, although in larger woodlands and parks figures were lower (£50-£100 per tree), due to economies of scale and available volunteer time to conduct the work.49 In comparison, the OPMCP programme data show an average cost per tree for pesticide spraying of £120 - £180, with the average cost for manual nest removal of £138 - £95 per nest (Table 5.10). Although the data indicate that the OPMCP has improved the cost effectiveness of controls, the impact has therefore not been as large as was anticipated. This may be because ten or fewer oak trees were sprayed at the majority of sites (55% to 70% over the three years) and fewer than ten nests were removed at 70% to 83% of sites where manual nest removal was carried out (see Figure 5.7). Given that the average cost of treatment falls significantly as the number of trees per site rises (see above), this limited the achievable economies of scale. Table 5.10 Average cost of controls (2013 - 2015)
2013 2014 2015 Pesticide spraying Unweighted average of the 132 180 121 site-specific average cost per tree50 (£) Crude average cost per tree 10 13 33 (£)51 Minimum cost (£) 1 1 4 Maximum cost (£) 1,185 8,863 692 Total # of trees sprayed 39,283 45,927 5,547 Total cost (£) 381,988 611,330 184,518 Manual nest removal Unweighted average of the 138 122 96 site-specific average cost per nest (£) Crude average cost per nest 87 58 40 (£) Minimum cost (£) 12 4 7 Maximum cost (£) 923 1,031 781 Total # of nests removed 1,342 3,222 2,303 Total cost (£) 116,845 136,771 93,193
49 Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment. 50 The spraying cost per site is divided by the number of trees sprayed on that site. An unweighted average of each of these site values is then calculated. 51 Total expenditure on spraying divided by total number of trees sprayed.
19 February 2016 70
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The average costs for nest removal have fallen steadily every year since 2013. The average costs for pesticide spraying increased in 2014, but then fell below 2013 levels in 2015. Costs of manual nest removal decreased over time for almost all the categories of sites. Control activities become more cost effective between 2014 and 2015. Figure 5.5 Change over time in the average spraying cost per tree
£600.00
£500.00
£400.00
£300.00
£200.00 Averag Averag ecost per tree
£100.00
£0.00 1 2-10 11-50 51-100 101-500 501-1,000 1,001-5,000 5,001-10,000 Number of oak trees being sprayed
2013 2014 2015
Figure 5.6 Change over time in the average nest removal cost (2013 – 2015)
£300.00
£250.00
£200.00
£150.00
£100.00
£50.00
Average Average per cost nest removal of £0.00 1 2-10 11-50 51-100 101-500 Number of nests per site
2013 2014 2015
19 February 2016 71
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure 5.7 Number of trees sprayed per site over time (2013 – 2015)
80%
70%
60%
50%
40%
30%
Proportion of sites (%) 20%
10%
0% 1 2-10 11-50 51-100 101-500 501-1,000 1,001-5,000 5,001-10,000 Number of oak trees sprayed per site
2013 2014 2015
Figure 5.8 Changes in the average cost of nest removal over time and by number of nests per site (2013 – 2015)
£300.00
£250.00
£200.00
£150.00
£100.00
£50.00
Average Average per cost nest removal of £0.00 1 2-10 11-50 51-100 101-500 Number of nests per site
2013 2014 2015
Contractors indicated that the cost efficiency of controls could have been increased if they had been provided with more, and better, information on the sites and site ownership. They sometimes had to spend additional time trying to obtain the correct contact details and site access information. “We need to get the right information, information on access to the site and its restrictions. Sometimes we’re not given the right access information or contact details, which is pointless and time consuming.”
19 February 2016 72
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5.5 Data gathering and data management An overview of the processes and systems for data gathering and management under the OPMCP is provided in Section 2.2.1.
5.5.1 EQ20. Have the OPMCP processes for data gathering and management worked as intended? Summary statement: The pilot programme built processes and systems for data capture and management that improved over time and ultimately served operational needs as intended. Many stakeholders regarded this as having evolved into one of the most successful aspects of the programme. The arrangements now enable the programme management to react promptly to new information and to deploy contractors where they are most needed, thereby making the most of the brief period in which spraying is effective. There were year on year improvements in processes. Survey and surveillance plans were developed and implemented. Data from sightings, surveys, controls and surveillance were recorded and used to inform the pilot programme. A mobile app facilitated more timely and accurate capture of data. An outsourced management service for OPM site and surveillance data was generally able to serve programme management needs, though the database was not configured to facilitate direct access by programme and other FC staff. Consultations indicate scope for further improvement in data gathering, such as gathering information on observed tree health and OPM abundance. Analysis: Of interest in the context of this evaluation question are: the processes relating to the survey and surveillance of the extent of the OPM range and distribution; the storage of survey data in a central OPMCP database; and the use of those data to meet programme needs. A brief overview of the approach to data collection and management under the OPMCP is provided in Section 2.2.1. The data collection strategy and its fit to programme needs are discussed in the response to EQ21. The number of sites and trees surveyed by the OPMCP, in both summer and winter, is shown in Table 5.11 and Table 5.12. There was no winter survey in 2013. The number of trees surveyed increased considerably over the three years. No FC-contracted surveying was conducted in the core zone of the south-west London outbreak in 2015 so the number of sites surveyed decreased in that year. Approximately 20% of the sites surveyed in 2015 related to the Guildford outbreak. The number of pheromone traps deployed increased significantly over the programme period (Table 5.13), reflecting the greater effort being put into determining the distribution and spread of OPM and understanding whether new outbreaks were isolated incidents or linked to the original outbreak area. As this effort was scaled up the pilot programme also learned more about which traps were effective.
19 February 2016 73
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 5.11 Scale of the summer survey effort (2013 - 2015)52
Year # of sites surveyed # of trees Additional Additional by FC contractors surveyed by FC number of sites number of trees (% change on contractors (% surveyed by non- surveyed by non- previous year) change on FC contractors53 FC contractors54 previous year) 2013 2,158 30,792 251 5,730 2014 2,562 (19%) 53,947 (75%) 355 4,373 2015 1,609 (37%) 69,211 (28%) 72 3,694 Table 5.12 Scale of the winter survey effort (2014 - 2015)
Year # of sites surveyed # of trees by FC contractors surveyed by FC contractors 2014 703 6,199 2015 803 21,245 Table 5.13 Number of pheromone traps deployed (2013 - 2015)
Year # of pheromone traps deployed 2013 155 2014 302 2015 830 Many consultees noted that the data capture and management had been a particularly successful aspect of the pilot programme. This view was echoed by one of the landowners responding to the survey. “Database management and administration has been excellent” “I think we have a good process for the surveys now. Data management systems have worked extremely well. We are able to answer questions really quickly if we need to. Getting it all into a database has been really good” “Of particular use and benefit to this Borough was the work done by the FC surveyors / spotters. This was particularly helpful and enabled us to get on with the treatment works.” The systems for gathering, storing and using the data developed significantly over the three years. There were changes to the way that data were stored and manipulated. The type of data being collected also changed over time, with more detail and more information being requested. “It was cobbled together in the beginning, but it’s got an awful lot better now.” The way in which the data were being collected also changed. The survey workforce was issued with a mobile phone application that allowed site survey data to be submitted in real
52 The type of sites being surveyed over time affected the apparent number of trees being surveyed per site. The number of trees surveyed per site increased in large part because in 2013 every oak on Bromley / Croydon was surveyed, and these were generally back gardens with one or two oaks. Most of the survey in the buffer zone however, where the survey effort was focused in 2015, is on large estates / golf courses, etc. which have a lot more trees. 53 Including, for instance, local authorities, Transport for London, Network Rail reporting to the FC. 54 Including, for instance, local authorities, Transport for London, Network Rail reporting to the FC.
19 February 2016 74
Evaluation of the Oak Processionary Moth Control Programme - Final Report
time. This meant that management and operational decisions could be taken quickly in response to new information. The application also improved the quality of the data received as the standard pro forma reduced the number of transcription errors. “Using the app to collect data has been fantastic, we’ve been able to react as soon as we know there is a nest and to process large amounts of data very quickly.” In the first two years of the programme not everyone was using the mobile app and therefore information was still being submitted in different formats. It was only in 2015 that the framework contract for the survey work was changed to stipulate that the field application be used for surveys. Prior to 2015, transcription errors and variations in the kind of information being submitted were a much more significant issue. “The biggest source of error is transcription and removing that completely isn’t possible but [the mobile application] definitely helped. Post processing is also a lot quicker. The instantaneous reporting is brilliant as we can take management decisions really quickly.” “The big problem has been where the survey contractors did not use the app. That made it harder, as it required merging information from different datasets.” Although processes have improved, the fact that data requirements have not been consistent across the pilot programme and have been different each year has also caused some problems. Those submitting the information had to keep changing what they were doing. This is an inevitable part of the process of iteration and learning implicit within a pilot programme. “The requirements have changed over time. It’s deeply frustrating when you have a process and then you change it. Everyone’s been learning as you go along, in a sense we’ve been making things up as we go along, and that’s been frustrating – that’s possibly a reason why contractors have not been filling the information out properly.” Consultations indicate the further changes could add value to the data. An example is gathering data on observed tree health and the abundance of OPM. A small number of consultees noted issues with the expertise and experience of the surveyors used to collect the data. This was also raised as an issue by a respondent to the landowner survey. One consultee noted that given the number of surveyors involved, it is inevitable that the quality of the data being submitted would vary. “The surveys that were carried out by the Forestry Commission appeared to be carried out by inexperienced operatives and they only identified a part of the problem … External operatives identified more in minutes than Forestry staff in hours. Subsequent documentation sent from surveys was inaccurate. Vague and misinformed information given by surveying staff.” “There are issues over quality control given different people are submitting data. You have to be very careful when looking at data. But there are always limitations with extensive survey work using lots of people – that’s not specific to [the OPMCP].” There was no dedicated framework contract (equivalent to that used for control work) for survey work. One consultee noted that the general framework contract being used was not developed to suit the needs of the OPMCP and that those on the framework did not fully appreciate the nature or extent of the work that needed to be done. It was also felt that, as a consequence, the costs of the surveying were higher than necessary. The terms of the framework contract were revised and improved in 2014 to include more specific information and requirements related to OPM survey work.
19 February 2016 75
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“We’re using a set framework for surveying. But the magnitude is so different for OPM than other [pests], it’s also in a very defined geographic area. I don’t think people bidding have a full appreciation of how much needs to be done. We should really have had an OPM contract for surveying specifically. That could also have improved the cost effectiveness of surveying” Having the management of the database outsourced, although necessary due to a lack of in- house expertise and capacity to manage something of this type and scale, created some challenges. Most queries of the database went through the contractor as direct access to the database was limited, with a reliance on one person in particular for responding to enquiries. This presented a potential bottleneck to some processes. The outsourcing arrangement did, however, provide freedom to change and improve elements of the process relatively quickly. “The problem is we had to deal with just [one person] – if [that person] gets hit by a bus, I wouldn’t know what to do.” “The biggest problem with the system is that the database is sitting on the server at the University so it’s not as easily accessible to other people as it should have been.” “I suppose there is a broad argument about fundamentally relying on an external contractor to deliver a programme. But quite honestly if it was done in house, it wouldn’t have been as effective”
5.5.2 EQ21. How effective have data management systems been in supporting the programme delivery? Summary statement: Data management systems have been a crucial aid to programme delivery. Data supported operational decisions on communications, control measures, and on survey and surveillance efforts, and greatly facilitated liaison and engagement with stakeholders. The maps showing the results of the OPM survey - where OPM had been found and where it was absent - were especially useful. The systems developed have enabled the pilot programme to turn raw survey data into actionable intelligence, and the despatch of control contractors, far more quickly than was the case previously. This in turn facilitates a more comprehensive control programme and so a more effective programme overall. Consultees saw potential efficiency and effectiveness gains from the further development of existing systems, in particular providing live access to programme data, overlaid on digital maps, to field operatives. Better provision of data on matters such as site access and contact details should enable contractors to operate more efficiently. Greater use of ‘live’ data would also be useful, such as the provision of on-line maps in addition to the use and distribution of static PDF maps. Although the processes work well, the data strategy could have been improved. The strategy has been responsive - the pilot programme has increased its investment in surveillance, the results of which are raising new questions about the distribution of OPM. But the data collected are not well suited to monitoring, evaluation, research or longer-term strategic planning purposes. Had these questions been factored into the strategy at the start it would be possible to say more now about the programme’s impact in relation to its core objectives. Analysis: There was a general consensus amongst consultees that data collected through the summer and winter survey and the pheromone trapping surveillance programme had strongly supported timely and efficient operational delivery of the pilot programme. They had also been very useful for programme communications, enabling effective liaison and engagement with stakeholders. The programme produced maps showing data on the survey and control
19 February 2016 76
Evaluation of the Oak Processionary Moth Control Programme - Final Report
effort, OPM distribution and OPM spread that were particularly useful. Data were also used to support Project Board and Advisory Group meetings, and to inform wider stakeholders on progress being made under the OPMCP. “Think it’s supported the day to day management very well – that’s what it was designed for. It’s supported communications work and enabled regular updates.” “The data management systems have been essential to the planning and delivery of the OPMCP... Without it you are operating blind.” "The maps are the key to everything, showing where [OPM] is moving, how it’s spreading. It’s been really useful in terms of that information. The information comes regularly through the control period. We get it often, and respond to the information quite a lot. “ However, a significant issue with the data strategy was that it was focused on operational support rather than what data were needed to support the monitoring, research, evaluation and strategic planning of the pilot programme. “The control programme has been run logistically in terms of controlling pests and there hasn’t been an opportunity to generate a really good data set for other purposes. It’s really difficult to do when you’re being so flexible and responsive. In terms knowing the baseline situation and the spread of OPM – we would have had a better idea if we had a collected more and better info from the beginning.” Control contractors suggested that the data management systems could provide more effective support for their work. They noted that the current processes, despite the improvements that have been made, are still comparatively rudimentary and that other countries had much more sophisticated systems that the UK could have learnt from. They believed that providing field operatives with real time access to the database would have been more efficient and more effective. Other consultees also noted that greater use of ‘live’ data would have been useful, such as the provision of on-line maps in addition to the use and distribution of static PDF maps. “The FC has a huge amount of data to process...They need to be more efficient with what they’ve got. In the tree management world there is really robust software out there, but nothing comparable is being used for OPM. The FC is still relying on spreadsheets and PDFs where instead you could have the whole digital map right where you need it in the field….The survey function has improved, but everyone else who uses the data hasn’t had much support on that front. We can’t get access to that data when and where we need it. There’s a good level of data that we get asked to give but the data we get given is not always good.” “We don’t often get the right access information or contact details, which is pointless. Especially when we submitted the correct information the year before. Information just isn’t filtering back through the next year.” Some consultees thought the links between the data collected and research being conducted were weak, and that the data collected were not optimized for long-term research. One consultee thought this may be a result of a lack of capacity to do so, whilst others felt that this was largely a result of the data collection and management being outsourced. “The data has not really fed in to the research … it’s contracted out – so [FR] are not directly part of that. It means we’re a little bit out of touch with what’s going on. [We] can get the data, but it doesn’t build any long term capability or capacity.” “[The data] didn’t really inform the science work because it’s not designed to do that. But it should have been more accessible.” “I don’t know how the data is informing scientific advice – resources in FR may or may not be sufficient to maximise the use of this data.”
19 February 2016 77
Evaluation of the Oak Processionary Moth Control Programme - Final Report
5.6 Awareness and engagement The focus of this section is on general awareness raising amongst stakeholders and the public, the risks of OPM, their obligations and of the programme and Government policy. Communications with landowners and other stakeholders for operational purposes (e.g. for carrying out control measures) are discussed elsewhere in Section 5.3.1. Responses to sub- questions EQ24a and 24b, describing the impact that increased awareness has had on the willingness of landowners to provide access and implement controls, have been integrated into the text below. The impact of other factors, such as the availability of funding and operational issues, on the willingness of landowners to support the OPMCP is discussed in Section 5.3.
5.6.1 EQ22. Has the OPMCP raised stakeholders’ level of awareness of OPM, its risks, Government policy, their obligations and the action being taken as intended? Summary statement: The OPMCP’s communications efforts have expanded and improved significantly over the three years of the pilot programme. Evidence suggests that the programme raised awareness of OPM and related issues among landowners (especially those with sites infested OPM), arboriculturalists and tree health professionals, some local authorities, health professionals and other target groups. The extent to which the OPMCP has raised public awareness of OPM and the programme is less clear. Analysis: Awareness raising was a key element of the OPMCP programme model. It was hoped that as a result of stakeholders being better informed about OPM, its risks, their obligations and the action being taken under the OPMCP: ■ Landowners would be more willing to support control work, provide contractors with access to their site and act quickly to deal with OPM infestations; ■ Knowledge of where OPM is located would increase as landowners would be more willing to provide surveyors with access to their sites, and also would be more willing to report sightings of OPM; ■ There would be better reporting of health incidents and a reduction in health incidents as more people avoid OPM caterpillars and nests; These changes would in turn lead to more effective targeting of OPM controls, and a relative reduction in human and animal health impacts where OPM was present. The scale and type of communication activities changed considerably over the three-year programming period (see Annex 17 for data). Funding for communication became available too late in the first year for much activity to be organised. Significant efforts were made in 2014 using various media. Communication activity was extended further in 2015, both in terms of the scale of the effort and the methods used. Specifically: ■ More posters, leaflets and banners were produced and distributed; ■ There was a greater use of communications targeting specific stakeholder groups, such as the arboriculture sector and the animal and veterinary sector; ■ The use of social media was refined and increased; and, ■ During the operational season, email updates were sent - usually on a weekly basis, but sometimes at longer intervals - to all registered stakeholders on the status of OPM outbreaks and programme activity. This was a response to requests from participants at the 2014 feedback meeting for more information on what was being done under the OPMCP and how it would affect them.
19 February 2016 78
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Many consultees noted that the communications efforts have been a particularly positive and effective element of the OPMCP, with significant improvements being made over the programme period. “Communications have gotten so much better. We’re better at getting the message out, and we’ve simplified the message. We’re using better and more tools.” “The communications hasn’t got to everyone but it’s been very effective overall” “There’s been excellent [communications] activity to a wider range of stakeholders to lift their awareness and to provide them with the relevant and appropriate information regarding OPM.” The majority of respondents to the landowner survey (80%) reported being aware of the OPMCP (Figure 5.9). All of the respondents to the multi-site landowner survey had previously heard of the OPMCP; only one respondent indicated they had heard about it but that they did not know much about it. Figure 5.9 Awareness amongst landowners of the OPMCP Q: Have you heard of the OPM control programme before? (N=100)
20
Yes, know about it and its goals Yes, but don't know 63 17 much No, never
The majority of respondents to both the single site and multiple site landowner survey indicated that their awareness of both OPM and the OPMCP had increased as a result of the information received from the FC once OPM had been discovered on their site(s). In both cases, awareness levels of OPM had increased slightly more than that of the OPMCP (Figure 5.10). Some consultees thought that awareness raising had a positive impact on landowners’ willingness to provide access. This is consistent with results from the online surveys. The majority of landowners responding to both surveys reported that knowing more about OPM had increased its importance as an issue for them (Figure 5.11), and increased their willingness to support the OPMCP, either through allowing access to their sites for surveys or for application of control measures (Figure 5.12). In many cases landowners also indicated that funding had positively influenced their willingness to provide access (see Section 5.3.2). Some consultees suggested that the funding had a larger influence on landowner attitudes to access than the awareness-raising, a theory that is not verifiable with the available evidence. “I’d like to think that efforts to raise awareness will kind of push the door open a little bit, so that landowners let us come in and do what’s needed. Hopefully it makes things easier for contractors if they’ve already heard about [OPM].” “I’m sceptical. I think it’s most likely just the money affecting people’s willingness [to provide access]. Raising awareness has an impact, but money does talk.”
19 February 2016 79
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure 5.10 Change in awareness amongst landowners of OPM and the OPMCP as a result of the OPMCP. Q: Please indicate how strongly you agree or disagree with the following statements: As a result of the information I have received since OPM was discovered on the sites, my… (N=100)
OPM awareness increased 42 45 7 4 11
OPMCP awareness increased 22 54 10 9 2 3
Strongly agree Agree Neither agree or disagree Disagree Strongly disagree Don't know / Not applicable
Figure 5.11 Impact of increased awareness on the perception of the importance of OPM as an issue for landowners Q: Please indicate how strongly you agree or disagree with the statement “Knowing more about OPM has increased its importance to me” (N=100)
1 1 4 10 Strongly agree 30 Agree Neither agree or disagree Disagree Strongly disagree 54 Don't know / Not applicable
Figure 5.12 Impact of increased awareness on the willingness of landowners to support the OPMCP Q: To what extent has the following affected your willingness to provide FC and/or contractors with access to site for surveying? (N=100)
Knowing more about OPM and human, tree risks 48 35 14 3
Involvement in OPM discussions and management 26 43 23 8
Increased considerably Increased somewhat No impact Decreased somewhat Decreased considerably Don't know / Not applicable
19 February 2016 80
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Q: To what extent has the following affected your willingness to have control measures carried out on your site? (N=100)
Knowing more about OPM and human, tree risks 49 41 9 1
Involvement in OPM discussions and management 27 42 21 1 9
Increased considerably Increased somewhat No impact Decreased somewhat Decreased considerably Don't know / Not applicable
Some responses suggest that landowners’ general awareness of OPM was relatively low until OPM was identified on their property and they became actively engaged in the OPMCP. “The first I heard of OPM was a note through the door saying the tree has OPM and someone will be removing it.” “I suspect that I might have had OPM in the previous year(s) already as I have been suffering from symptomatic skin rashes every time I have been in the garden, but I had never heard about OPM before I saw a nest.” The withdrawal in 2015 of programme funding for controls in the core zone left landowners to respond to OPM as they felt was appropriate. To assist them the programme developed an OPM Tree Owners’ Manual providing advice for landowners with oak trees. The manual was distributed via email as a web link in April 2015 to all landowners registered on the OPMCP database who were located in the core area. It was also frequently referenced in the regular OPM updates on progress and programme activity, which was sent to all contacts in the OPMCP database that have an email address. A significant proportion of landowners on the database however only have postal addresses listed, and are therefore less likely to be aware of the manual. The results of the single-site landowner survey suggests that 34% of respondents were not aware of the manual or hadn’t used it, so they were not able to comment on its usefulness. Of the remaining 66% who were able to comment, most had found it useful. A larger proportion of respondents to the multiple-site landowner survey were aware of the manual and able to rate its usefulness; a larger proportion were unsure of its usefulness. The majority of respondents to both the single-site and multiple-site survey thought more guidance on OPM would be helpful. Figure 5.13 Usefulness of the OPM Tree Owners’ Manual and the need for more guidance
OPM manual was very useful 8 29 26 21 34
More guidance is needed 16 45 28 7 13
Strongly agree Agree Neither agree or disagree Disagree Strongly disagree Don't know / Not applicable
Consultees noted the potential for communication efforts to be interrupted by external events, such as formal or self-imposed restrictions on communicating about the pilot programme in pre-election periods. For instance, a few consultees noted that the elections in 2014 had had an impact on what and how information could be disseminated. Some noted that concerns about how information emerging from the programme would be interpreted by the press had had to be addressed. Consultations also yielded a few comments about the
19 February 2016 81
Evaluation of the Oak Processionary Moth Control Programme - Final Report
challenge of framing the messages delivered, such as striking an appropriate balance between the public health and other aspects of the OPM problem. “The problem of not being willing to engage with the press for political reasons was really unhelpful [just before the elections]. We could have raised awareness sooner if that hadn’t been the case” “We got over the restrictions on information….. There’s always this fear of things being turned into a negative story which results in you not doing the right things” Collaborative relationships have developed between the FC and Defra communications teams, as well as between the OPMCP operational team and the Advisory Group, with transparency increasing over time despite initial reservations about what information could be shared. “There was a problem last year and before; there was a resistance to share what could be deemed bad or negative news which meant they would hold back on information and that would mean people weren’t informed enough” "In terms of communications and public relations, prior to [2015], I don’t think they got it right … Things were kept rather hush hush. There was a reluctance to speak to the press about the issue. This year [2015] saw a big step forward, a lot more was done...Last year they got tied into elections which also made it difficult, things had to go through all sorts of processes before they were released” Little investment was made in awareness-raising in schools. Consultations suggest that this was in part due to a lack of ‘ownership’ of this stakeholder group – FC seeing communication to schools about health threats as a PHE responsibility, PHE seeing it as something for local authorities, and local authorities not being fully engaged in the OMPCP (see Section 5.6.3). As a result schools did not receive the targeted communication effort that might be expected. The impact of the pilot programme on OPM awareness levels among the general public is unclear. Views on this issue amongst consultees varied. Some felt that public awareness has improved, whilst others were less sure. Some suggested it would require more time. “We have this perception amongst ourselves that we’re not getting the message out there. [But] it seems a lot more people are aware of OPM than we think” “In terms of the public [awareness], it’s more of an unknown...more and more people are starting to know about it. I think we’re getting there, we just have to get a certain level of momentum going.” “I’m not convinced that the general population are aware of OPM... It’ll take a number of years.”
5.6.2 EQ23. Has the OPMCP improved the reporting of OPM sightings and health incidents as expected? Summary statement: Some consultees suggested the OPMCP actions had increased the reporting of OPM sightings. This is difficult to validate with the available evidence but there were examples in 2015 of new colonies being brought to the FC’s attention via independent reports. There is also now a means of health professionals coding health incidents as OPM-related in the UK's health data systems. Consultations suggest that a small number of such incidents have been recorded. It is not clear whether and to what extent the lack of evidence of large scale OPM-related health impacts is evidence of these impacts being uncommon, or whether other factors are at play (e.g. under-reporting).
19 February 2016 82
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Analysis: To be effective the OPMCP needed the best possible information on the location of OPM colonies. The programme theory assumed that increasing awareness of OPM would mean that more people were able to identify OPM and know to report sightings to the FC. It was also expected that public funding for control measures would make landowners more likely to report sightings of OPM and more willing to provide FC surveyors with access to their property. It was assumed that health incidents are often unrecognised and significantly under reported. Increased awareness amongst the public and health professionals was expected to lead to more reports of OPM-related health incidents and better diagnosis of OPM related cases. Feedback from consultees suggests that reporting of OPM sightings has increased. Some consultees thought the OPMCP was making a difference to awareness of OPM, such as among tree workers, and that this was contributing to more frequent reporting. Some thought that greater awareness of the funding available for control work had an impact on OPM reporting. The colonies confirmed in Hampstead Heath and the Guildford area in 2015 were identified as a result of reports to the FC made by people unconnected with the pilot programme. This suggests that awareness of OPM might be increasing to the point where OPM is more likely to be recognised and reported to the FC. One consultee suggested that concerns about the biodiversity impacts of OPM controls discouraged amateur conservationists from reporting OPM sightings but this theory could not be verified. "There’s definitely been an improvement. We seem to be getting a lot more reports coming in this year than the first and second year. I think also because [landowners] don’t have to pay for [controls]. The key bit of the communications message has been that there is funding available.” “I think we’ve improved OPM sightings a bit because more people are aware. But we don’t see much reporting of health incidents” “We work with amateur conservationists – some don’t want to report sightings for OPM because of known biodiversity impacts.” There are some data on who reported sightings but this information has not been collected consistently. Those data indicate fewer than five OPM sightings being reported by the public each year and around ten being reported by tree health professionals. There is insufficient evidence for trends or firm conclusions to be identified. The pilot programme tried to improve reporting of OPM-related health incidents. Public health impacts were an important part of the programme rationale, and evidence of impact is thus relevant. Reports of OPM-related health impacts are also a potential signpost to OPM colonies. PHE created a code specifically for OPM so that health professionals could record OPM-related health incidents. Whilst it is possible that OPMPCP actions have increased the number of OPM-related health reports, the numbers are too small to conclude that this is the case. A small number of health incidents have been registered to the new code55. The FC has independently kept records of the number of OPM-related health incidents reported to it since 2014. There were 17 reports in 2015 as compared to 11 in 2014. Some relate to multiple people. "The programme has definitely had an impact. [OPM] is now on the national [health] system for reporting outbreaks….Some incidents have been reported to
55 The data were not made available to the evaluation, despite requests being made.
19 February 2016 83
Evaluation of the Oak Processionary Moth Control Programme - Final Report
the national database. It’s only been a small number of reports but that’s better than nothing.”
5.6.3 EQ24. How effective has the OPMCP been in engaging and promoting collaboration with stakeholders? Summary statement: The pilot programme has been effective in engaging with landowners, arboriculturalists and tree health professionals, contractors, a small number of local authorities in London and NGO representatives. There was consistently positive feedback from stakeholders on the OPMCP’s efforts to develop a collaborative approach. The pilot programme has had less success in engaging the other local authorities despite a concerted effort to do so. It has also not achieved the level of collaboration with the health sector that was hoped for, an outcome attributed to the comparatively low priority given to OPM as a public health issue by the health sector. Analysis: The principal stakeholder groups for the OPMCP, beyond the public bodies engaged in its delivery, are private landowners (household and institutional), local authorities (both as landowners and as organisations with tree health and other duties), other public landowners, the health sector (especially PHE), the arboriculture sector, non-governmental organisations, schools, the science community and the general public. The pilot programme has engaged and collaborated with these stakeholders through mechanisms such as the Advisory Group, annual ‘lessons learned’ meetings, training and awareness-raising seminars and events. There has also been less formal, ad hoc day-to-day engagement in response to any queries or concerns, especially with the programme manager. Defra and FC recognised that OPM can only be effectively tackled with the involvement and cooperation of as many affected stakeholder groups as possible. This encouraged an approach based on partnership working and open dialogue that, the consultations suggest, is appreciated by many of the stakeholders involved. “We’ve had a great experience of working with other stakeholders in the control programme.…overall it’s been really positive and constructive.” “The control programme … has bought us a lot of collaborative expertise in tackling [OPM].” “Engagement has been very effective actually. [The FC] have worked really collaboratively with us … They’ve been good about engaging with us, very good.” One consultee noted that it would be useful to tailor information to different audiences to make it more useful and user friendly. “Tailoring [information] to audiences would be useful – for example, forest managers are being presented with raw data that could be better presented.” The majority (61%) of respondents to the single site landowner survey indicated that engagement with them had been effective (Figure 5.14). Similarly, most of the respondents to the multiple site landowner survey also thought that engagement had been effective. A majority of respondents to both the single site and multiple site survey indicated that the engagement had a positive impact on their willingness to provide access to their sites for surveys and the application of control measures (Figure 5.15).
19 February 2016 84
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure 5.14 Perceived effectiveness of FC engagement with landowners Q: Please indicate how strongly you agree or disagree with the statement “Engagement with landowners has been effective” (N=100)
2 2 11 15 Strongly agree Agree Neither agree or disagree
24 Disagree Strongly disagree Don't know / Not applicable 46
Figure 5.15 Impact of involving landowners in discussions about OPM and its management on their willingness to provide FC with access to sites Q: To what extent has the involvement in OPM discussions and management affected your willingness to provide FC and/or contractors with access to site for surveying? (N=100)
26 43 23 8
Increased considerably Increased somewhat No impact Decreased somewhat Decreased considerably Don't know / Not applicable
Q: To what extent has the involvement in OPM discussions and management affected your willingness to have control measures carried out on your site? (N=100)
27 42 21 1 9
Increased considerably Increased somewhat No impact Decreased somewhat Decreased considerably Don't know / Not applicable
One consultee thought that the greater role and level of commitment to the control of OPM provided for by Defra and the FC under the OPMCP had a detrimental impact on the level of commitment and involvement of some stakeholders – with them being less inclined to support the efforts as Defra and the FC are handling the problem. Another thought the extensive engagement with other stakeholders, and the emphasis placed on their input, meant that the role of expert judgement in informing decisions about the pilot programme’s implementation had been compromised. “I do think that since the programme had the money, some of the stakeholders stepped back a bit and just let us get on with it…they weren’t engaged in the same way anymore.” “There are a greater number of people involved …. This has meant that advice is not being taken sufficiently into account. …. This has had a detrimental impact by allowing resources to be used in ways that are inappropriate or unnecessary.”
19 February 2016 85
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The pilot programme has yet to achieve the level of engagement from local authorities that had been hoped for. This problem was mentioned by a significant number of consultees. In 2015 specific and targeted efforts were made to address this issue: articles were placed in local government news media and the director of FC England wrote a personal letter to all chief executives in affected local authorities encouraging them to get involved. These efforts seem to have had little impact. The lack of engagement is generally attributed to OPM not being the highest priority in a period when budgets are tight. “Engagement with certain boroughs and local councils has been very poor and very difficult, probably because problems of tree health is just one of many issues they’re dealing with, it’s not really on top of their agenda.” "It’s been very hard to engage local authorities, especially at the moment. They have no money and no staff.” Several consultees noted that the lack of engagement by local authorities had a negative impact on the programme. It meant there has not been a unified or consistent approach to OPM management which made it more difficult to limit the moth’s spread. This was also recognised as an issue by a local authority. "One of the main limiting factors to the success of the programme has been the failure of local authorities to coordinate their activities. What you end up having is some people treating some trees one year, whilst a neighbour doesn’t, and then the tree gets re-infested from the neighbour.” “[One of the] most notable factors in terms of limiting the success of the programme…is the failure of local authorities to work collaboratively on the issue” "As a London borough, the importance of working with the FC and other boroughs with a unified approach is paramount to the success or failure of the control of OPM.” Responses from the landowner survey suggest the lack of support by some local authorities could have an impact on the willingness of landowners to support the OPMCP and or lead landowners to take their own action without necessarily having the right experience or expertise for doing so. “I understand that Kingston Council hasn't been addressing the problem [of OPM] so now I feel like there is no point in me as a private person to be addressing it when they aren't.” “I had found out that my council (Kingston-Upon-Thames) does not help financially, so I had gone to Homebase, bought myself a flamethrower and full body suit and took care of the nest myself.” Some consultees were disappointed that PHE engagement did not extend much beyond attending Advisory Group meetings and thought not enough had been done to highlight the potential health impacts of OPM. There was recognition that since OPM does not rank high in the list of public health challenges facing the UK it might not be on PHE’s list of priorities, but also concern that long terms costs and reputational risks were not being taken into account. Consultation with PHE suggests the FC’s efforts to keep them engaged have resulted in higher levels of involvement than would otherwise have been the case. “PHE don’t see [OPM] as a biggie because its not a health biggie, but it’s a perception issue…. They’re not thinking through the consequences of it being a PR disaster. They say it’s not a problem, but that’s not what people’s perception is.” “I’m a bit disappointed with PHE – their view is that medically, clinically [OPM] is not a big deal, it’s not up there with Ebola or things that can kill people, so they don’t give it a high priority”
19 February 2016 86
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“Defra and FC have actively engaged with PHE. In light of available funds difficult decisions have to be made about how we use the resources we have available. We have remained engaged with the OPM work - although it is not a strategic priority for us or a significant public health issue - because the Advisory Group and FC have demonstrated to us that it is a priority issue for their stakeholders.”
5.7 Research and analysis Given the overlap between the issues raised by EQ26 and EQ27, responses to these two questions have been merged.
5.7.1 EQ26. Have investments in research (e.g. on OPM population distribution and dynamics, control measures and surveillance) contributed to the programme as envisaged?
EQ27. How effective has the OPMCP been in ensuring that programme delivery is informed by current scientific knowledge (including the results of research and analysis) and lessons learnt from within the programme? Summary statement: There has been a strong focus on continuous improvement. The OPMCP has changed and aspects of its delivery improved significantly over the three years. The OPMCP management approach has been based on an annual cycle of planning, implementation and taking stock of lessons learnt, for instance through annual lessons learned workshops with wider stakeholders. This has worked well and has resulted in continuous improvements to the pilot programme, for example with regard to communications. Scientific input and expert advice have played key roles in the OPMCP and its improvement over time. Links have been fostered between the programme and researchers; the programme managers have maintained access to scientific expertise, and used the advice to adjust the programme. Most of the input has been informal and the pilot programme has generally relied on Forest Research to find and filter relevant evidence. The primary research commissioned through the OPMCP is still on-going and so has not yet made a direct contribution to OPMCP actions. Other research which has already been completed has had a greater impact on the OPMCP’s delivery. There was an agreement amongst stakeholders that the research component of the programme was important. The lack of interim results that might inform programme delivery was a frustration for some. The measured pace of research review and publication processes did not always meet the evidence needs of the pilot programme. A more systematic approach to reviewing available scientific evidence, gaps and priorities would have helped to demonstrate that investments were focused on priority areas and that actions were feasible. Analysis: The pilot programme has been operating in a context of comparatively little scientific information on OPM, its impacts and its control. This has put a premium on in-programme learning and ensuring good access to new evidence as it emerges. The operational management arrangements provide OPMCP management with access to scientific expertise via Forest Research on an on-call basis. “There are frustrations regarding the gaps in the evidence.” “There’s not enough definitive information to inform the programme. That’s been challenging.” "A lack of evidence is a general thing [with the OPMCP], there’s little hard evidence to inform everything.”
19 February 2016 87
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The high level programme architecture of the OPMCP featured a ‘learning loop’ in which a review exercise at the end of each year was used to inform the planning and preparation of the following year’s activities. Many of these improvements and refinements (Table 2.1) were introduced based on prior experience, new scientific evidence and feedback from stakeholders. Aspects of the programme that benefitted from this approach to learning and improvement were data gathering and management (Section 5.5), communications (Sections 5.3 and 5.5.1) and the delivery of the control measures (Sections 5.4). Much was done to capture lessons from programme experience and to identify improvements. ‘Lessons learned’ workshops were held with wider programme stakeholders every autumn to capture feedback and identify improvement actions. Programme management and other stakeholders recognised these sessions as especially useful. Outputs from the workshops were logged, taken forward as action points and reviewed the following year – helping to ensure that the feedback was actually used. The workshops led, for instance, to significant improvements in the targeting and delivery of the communications programme. “I think that every single year has built on experiences of the previous year… Conversations now are more targeted and more strategic. It’s become much more coherent because we’ve really built on the learning of previous years” “We’ve learned every year through the monitoring and our operational reviews and the feedback form stakeholders. With the [lessons learned] workshops, we looked at what were the comments, which ones were substantive and what are we going to do about them.” A small number of stakeholders observed that there had been little change in the overall programme strategy which suggests that many consultees did not fully appreciate that the strategic objective was altered from strict containment to slowing the spread of the south- west London outbreak. This change was made once it became clear that limitations of detection and control efficacy meant that some annual spread was inevitable. Whilst some adjustments had been made at operational level, much of the original programme model is intact. “In terms of overall strategy, there was no change because approach was set from the beginning” “The basic approach has been the same each year. There’s been no need to make major changes” Many of the improvements and changes made to the operational delivery of the OPMCP were a result of scientific inputs. Some of the research was directly commissioned and paid for by the OPMCP, including research into the population distribution and dynamics of OPM, OPM natural predators and the efficacy of different control measures. For almost all of this research, results are not yet available and so its contribution to the OPMCP thus far has been minimal. Additional research which is not being directly funded by the OPMCP is also being undertaken, although very few of these are at the reporting stage. However, several of the consultees found that the research which has been commissioned and completed has been hugely valuable to improving the effectiveness of the OPMCP. A full list of relevant research work is provided in Annex 18. “There is a huge amount of research going on to inform our operations.” "[Science] has probably had quite a lot of impact, for instance in terms of the best treatment to use, the pheromone trapping research that has been ongoing… on the timing of controls. We’ve learned that sprays after a certain time are not effective. So we have to be sure get it there earlier than we would have thought.”
19 February 2016 88
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“[Research] has definitely been useful. For instance, we now know that 95% of female moths fly 500m from where they were born. That’s really important…So most of OPM happens where it was last year” Overall, the commitment to long term research under the OPMCP was welcomed by several consultees, with many consultees noting that the investment and weight given to scientific evidence was a particularly useful and important element of the OPMCP. “To me [the OPMCP] has highlighted the importance of scientific input, without which, management tools are a blunt instrument.” “The funding from the OPMCP for research has been one of the most positive and important aspects of the programme”. “The amount of funding dedicated by the OPMCP to research has seemed long- sighted and valuable.” One stakeholder was impressed by how readily the OPMCP team took on board new and emerging research findings, even where these were not what was expected or were not in line with current thinking. Instead of ignoring such evidence because it didn’t fit with the current ‘narrative’, the conversations and the approach taken to the OPMCP were changed. “There seemed to be in my opinion a really good discussion of the evidence. Evidence was taken on board despite people not wanting to hear it … [and] changed their tack entirely. I think that was very evidence based.” One stakeholder suggested that the influence of science on the pilot programme was being diluted as a wider range of interests are now involved in the programme decision-making. “It’s less good now than before. There are a greater number of people involved and the science is now just one voice amongst many. Before, [FR] were much more closely involved. This has meant that advice is not being taken sufficiently into account.” Expert inputs were mediated via informal processes and a collaborative relationship between the FC and FR. The scientific inputs were mostly provided via expert opinion rather than direct incorporation of findings from formal outputs (e.g. published scientific papers). There was a reliance on the FR identifying relevant research and providing the FC with that information on an ad hoc basis. "We’ve relied immensely on [FR and their] expertise. It’s a pretty informal relationship” “It’s a collaborative relationship. I don’t think that’s captured very well [in the evidence / documentation]” “[FR] provide advice and comment on most aspects of the control programme and all sorts of technical issues, like the effectiveness of treatment, rates of spread, what the OPM population is. They provide advice and support on whatever topic we ask them about” The lack of a more systematic approach to reviewing available scientific evidence, gaps and priorities at specific and regular points in time means that sometimes those involved in the OPMCP were made aware of a relevant piece of research by chance. This meant that useful information was not always identified or made available quickly enough. It was noted that FC’s in-house expertise on forest and pest research could be improved. “We’ve been talking about these things for three years, and you think you know the breadth of evidence, and then things come out of the woodwork and you just didn’t know about it”
19 February 2016 89
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“On occasions we may only know of important findings just at the point it is being published. This issue applies not only to external research agencies but FR as well – that we are surprised by research findings that we should have heard about sooner.” “FC is under-resourced on [the research] side of things. It’s a bit of a pinch point, really.” Some of the evidence suggests that more could have been done to cultivate links with equivalent OPM communities of interest in those countries in mainland Europe that have more extensive experience of the pest. Some effort was made before the OPMCP to collaborate with other countries but this does not seem to have continued to the same extent beyond the start of the pilot programme. This was also noted by some consultees. However, one consultee noted that very little research and investigation into OPM was actually on- going in Europe, and that much more research was in fact being conducted on the pest in the UK. There has therefore been little apparent need or benefit to more regular or systematic collaboration with counterparts in mainland Europe. This would suggest that there is a perception amongst other stakeholders that more work is being done on OPM elsewhere than is actually the case. “I suspect we could draw on more from abroad…We really need to draw on wider knowledge from the EU, as making better use of EU evidence base on pests and risks would be helpful.” “There needs to be more engagement with other parties on mainland Europe to gain and share knowledge.” Consultations illustrated the tension between the measured pace of scientific research and review process in general and the hunger of the programme for quick answers to pressing questions. The pilot programme had to wait for peer review and approval processes to be completed rather than getting the immediate access that would have enabled the information to be used in programme delivery. Research into the effectiveness of control methods, for instance, only became available to the OPMCP at the end of 2015. Some consultees thought it would therefore be useful to have access to interim results if these would provide insight that would help improve the control programme. The frustrations of some about the research timetables and lack of interim results were balanced by an appreciation by others that such research is a long term endeavour and that review processes are a necessary part of ensuring that the results are valid. “We’ve not had any information or updates on the research. The problem is that scientists don’t like to publish if it’s not peer reviewed. That can take months or years. We would really like to know as soon as they know that something is important. Scientists don’t appreciate that and the importance of the information” “The [OPMCP- funded research] hasn’t really informed the programme – [the results are] coming too late. It’s not feeding into the programme quickly enough as results come in.” “The speed of research findings is very slow. They’re loathe to share that until they’ve got it published or fully reviewed. That’s not helpful for us. If they know something, we need to know sooner than later really” “The research that has been funded through OPMCP is ongoing…it’s a long term process that doesn’t get fast results. Some people have unreasonable expectations on when the research can deliver information." “The research funded through [the OPMCP] will be valuable in the longer term but at the minute we’re only a couple of years in. It’s early days really.”
19 February 2016 90
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The consultations suggested that there may be scope to improve the research commissioning process and communication of the rationale for selection of research priorities.“One thing we looked at was detection but that wasn’t very successful. It was the right research to commission even though it didn’t tell us anything that could improve detection. Another one is which control options are most effective. That wasn’t that successful either because it was done when infestation was quite low” "I feel quite strongly that a lot of the research has been inappropriate and not useful at all. [Some of the research funded] was destined to fail from the beginning. …The commissioning bodies did not have the experience or competence to know what work was achievable or relevant. And also they were giving research projects to research groups who had never worked on insect pests and trees in woodland situations and were naïve in what they could achieve and what the costs could be. They couldn’t do what they thought they could do because of practical difficulties of working in this area. The wrong questions were being asked of the wrong people with wrong techniques being applied.” “I find it random what gets commissioned. … I find it a bit perplexing that we don’t have an understanding of the research landscape. I’m not an expert in any of it, but have we stepped back and reviewed what are the crucial research elements we need or the gaps that we need to work on for the future?” There was, overall, a general consensus amongst consultees that research was a critical element of the OPMCP. Some felt that more resource should have been put into research. “At the moment not enough is being done about R&D.” “There needs to be more research.” “Research is where resources should be going” Many of the stakeholders identified areas where research had not been commissioned, but where additional information and evidence would have been useful. These included: ■ the nature and biology of OPM, including its spread and how it would likely behave in the UK climatic context; “No work on [OPM] rates of spread has been done. Also nothing on climate issues, as in how far will it spread under different conditions. Understanding the biology and nature of pest would help with management and control – that’s not been looked at enough.” “More research into the biology and life cycles of OPM would be useful. It may be that the population goes down for a period as a result of cold winters and then recovers.” "There needs to be a greater understanding of the biology of the pest and the factors that influence its behaviour and spread. The actual edge of the outbreak needs to be determined and establish whether all ‘outbreaks’ are derived from a single introduction” ■ The biodiversity impacts of OPM and the OPMCP; “Clearly biodiversity research was a missed opportunity – both for UK research and applications in Europe. There are a lot of questions we don’t know all the answers to.” ■ Baseline information, for instance on OPM’s distribution and the outer extent of the original infestation; and,
19 February 2016 91
Evaluation of the Oak Processionary Moth Control Programme - Final Report
“I probably would have liked more work to be done on ways to monitor and detect OPM.” ■ What impact potential health effects of OPM might have on people’s attitudes to the use of public green spaces. “The next step could be for someone to do research into attitudes around OPM and its risks…. We don’t know what the impact of OPM would be on public recreation. Certainly there’s an opportunity for work to be done on that particular aspect before any decisions are made based on the assumption that people will stop using these spaces because of OPM. That’s not a given”
5.8 Capabilities
5.8.1 EQ29. Has the OPMCP improved the ability to collect and manage data on other plant pests / pathogens / diseases as expected? Summary statement: Much has been learned from the OPMCP on the collection and management of data related to OPM. Some of this experience and the lessons learned appear to be useful and transferable to other areas, particularly with regard to the need for a quick and responsive data management system which can be used to direct and inform operations, for instance when and where to treat a pest if there are reported sightings. The OPMCP also illustrates the benefits of using an electronic, web-based field application for collecting data. Experience with OPM data management systems has already been reviewed in scoping the potential for a data management system on Chalara. The outsourcing of OPM data management services provided freedom and flexibility to implement changes, and has provided insights that can inform future systems. However a few consultees felt that this approach limited the OPMCP’s contribution to the improvement of FC in-house capability in tree pest data management. Some consultees were sceptical about how much of the OPMCP experience could be transferred to other plant pests, pathogens and diseases. Some consultees thought the outsourcing arrangements had impeded access to the data, but the accessibility constraints owed more to the data being held for longer than expected (due to delays in delivery of the replacement FC system) on a software platform that was sub-optimal than to the contracting model used to procure it. Analysis: It was expected that the lessons learned and systems developed over the pilot programme period for data collection and management would help to develop capabilities which could be transferred to other plant pests, pathogens and diseases. There have been many developments and improvements to the collection and data management over the three year programme period (see Section 5.5). Some of the lessons learned over the three years, and the experience gained, are potentially useful for other plant pests, pathogens and diseases. One example is the use of an electronic, web-based field application for collecting data. The OPMCP has shown the importance of having a system which allows for a rapid turnaround of information and reports to inform operations, as well as ensuring that all the necessary information is collected in a consistent way over a programming period. The OPMCP has also shown the need for appropriate oversight of such systems where a programme relies on information being submitted accurately and quickly. "The field app side of things has been very useful. Surveying for most tree pests and diseases could be suitable for webapp development.” "[The OPMCP] has significantly improved the speed with which, and how we deal with information relating to OPM. That’s been really useful and is a model
19 February 2016 92
Evaluation of the Oak Processionary Moth Control Programme - Final Report
that is transferable across a range of other pests. That’s been the biggest advantage, the ability to produce reports quickly.” “Maps [developed from the database] are also loaded onto the apps, which can link to an interactive PDF. It’s all much tighter and more nuanced, it’s all stuff useful for other plant pests and diseases.” Lessons from the OPMCP have already been used to inform discussions and decisions about a data system to support control of Chalara, and a centralised FC database on plant pests which are both in progress. “Now we’ve got a better appreciation of what works and what doesn’t work. We’ve been able to inform the design of a new system for the Forestry Commission. Experiences are being sharing about what’s worked and what hasn’t. That’s also influenced the chalara database development. Lessons are already being shared.“ "The data management system is a very good model. There’s been talk of using this for Chalara, that was being investigated” The outsourcing of the data management system and lack of direct access to the data for FC and Defra users (the latter being a matter of system design rather than of the delivery model) were identified as factors which limited the ability of the pilot programme to improve capabilities and capacity for managing data on other pests, as little in-house capacity had been developed over the three years. The lack of direct access was in large part a consequence of the OPMCP having to rely on a MS Access database for longer than was originally anticipated due to delays in the delivery of the alternative, more sophisticated FC system for reasons beyond the OPMCP’s control. However, the approach did deliver other benefits, in that experience of what worked and what did not work with the OPMCP database has provided lessons that can be used to support the design of the other plant health databases. Moreover, the outsourced database was also able to be more flexible and responsive to change as internal processes and restrictions did not affect the database’s design and use as much they would have affected an in-house system. ““The Access database is not very robust. We need a database that is a lot stronger which can be looked at by many people at the same time, and that is accessible.” “There is no increase in our capacity [in collecting and managing data] through the OPMCP because everything is contracted out. We haven’t built up any capacity for the future.” "We’ve learned from how they’ve handled the data. All of it is transferable skills. The only thing that changes is the pest.”
5.8.2 EQ30. Has the OPMCP improved the capability to control OPM and similar plant pests / pathogens / diseases? Summary statement: Capability to control OPM has clearly improved over the three years. There is greatly enhanced capability to apply controls where they are needed, when they are needed. There is now a better understanding of what is required for delivering a large-scale spraying programme, including the necessary underlying systems and collaborative relationships. More is known about OPM and how to control it effectively, more stakeholders are aware of OPM and engaged in the process of managing it, and the systems for data collection and control work are more efficient and effective. Consultees also noted that the number of contractors with the necessary expertise is growing.
19 February 2016 93
Evaluation of the Oak Processionary Moth Control Programme - Final Report
The extent to which the OPMCP has improved capability to control other plant pests, pathogens and diseases is a more complex question. Elements of, and lessons from, the OPMCP on establishing an effective control programme are transferable, including the importance of communications and establishing collaborative relationships with stakeholders, governance arrangements, the learning and improvement systems, control contractor management, and data capture. The transferability of lessons learnt in application of control measures depends on whether other pests would respond to similar measures. Analysis: Overall, the OPMCP has enhanced experience, knowledge and systems to support and deliver a large scale spraying programme to control OPM, by for instance: ■ Understanding more about OPM as a pest, where it is and how it spreads through scientific research (see Section 5.7) and a large scale data gathering and management programme (see Section 5.5); ■ Understanding more about how to effectively control OPM, about the treatment options, and where and when is best to apply these (see Section 5.4); ■ Increasing the capability of the contractors that apply OPM control measures; ■ Being able to rapidly respond to new information and new areas of infestation (see Section 5.3); ■ Improving awareness amongst and engagement with stakeholders, and establishing effective partnerships and collaborative relationships with different stakeholders groups (see Section 5.6); and, ■ Having established effective governance and management structures to plan, coordinate and delivery a comprehensive control programme (see Section 0). These elements are discussed in detail in the Sections referenced above. There was a general consensus amongst consultees that the OPMCP has improved the FC’s capability to control OPM. “The programme has improved capabilities to control OPM. More people are now involved, more people are now aware. It’s certainly increased the numbers of people who know very well what to do and how to control [OPM]” “If something happened now [with OPM], we’d be well positioned to deal with it. From my point of view, it’s bought us time to learn what to do and to understand what the pest is doing” “The [OPMCP] has involved spraying things on a large scale…I don’t think it’s been done before. There is now a set of contractors who know how to do that.” Some noted that there is less certainty about the OPMCP’s impact on capability to tackle other plant pests, pathogens and diseases; the methods needed are likely to be different to those applied to OPM as the distinct features of each plant pest, pathogen and disease meant little of the OPMCP experience would be transferable. Some consultees thought that aspects of the OPMCP approach could nonetheless be useful for other plant pests. Elements and experience from the OPMCP provide lessons for the control of other plant pests and diseases, even if their biology is different. This includes, for instance, the importance of establishing good working relationships with the full range of stakeholders and the need for appropriate communication materials to raise awareness and support of the control work. "The challenge is that each tree threat is unique in what it’s about. You would hope there are things you’ve learned from and that are transferable but realistically it could be difficult."
19 February 2016 94
Evaluation of the Oak Processionary Moth Control Programme - Final Report
"If there is another example of pest with health impacts, then maybe things would be transferable. But it is unlikely. We would know what to do to launch programme if the pests were similar. We wouldn’t need to develop a new method of investigation from scratch for instance.” “In the future we’ll have experience of dealing with [OPM] so we would have more insight if something of a similar nature came up.” "I think OPM as a pest is quite unique and quite particular in the way it needs to be controlled. No pests similar to that are currently affecting the UK at the minute. The lessons learned could potentially be useful for future Lepidoptera pests should that happen, such as the pine processionary moth. So maybe it could be useful for that.” The OPMCP has demonstrated that affected stakeholders need and appreciate information on what activities are on-going, how this affects them and what they can do to contribute. The pilot programme issued regular OPM update newsletters to affected stakeholders during the ‘active’ season about what control work was being done, where, and why. After enquiries made by the public about contractors and their work, signs explaining what contractors were doing and why were developed to be put on their vehicles. The OPMCP has also provided the FC with an opportunity to learn how to work in a different (i.e. urban) context and with different stakeholders, which may prove useful in the future. One consultee noted that the OPMCP has also demonstrated how the efforts of volunteers can be mobilised to support a large scale control programme. “This pest is quite a specific one, ownership is very fragmented which the Forestry Commission isn’t used to dealing with. It’s also strange for us as an organisation to deal with an urban environment. This is all knowledge and systems banked for the future” "Think the main benefits have been that a couple of contractors now have the capacity to do similar treatments for similar organisms… we now have the capacity to deal with them where they are built up in the local area. We have more expertise and knowledge now, and we’ve built capacity.” “I think the programme’s been quite good with mobilising volunteers, and has helped with improving contingency planning and monitoring work. I think there are learning points there.”
5.8.3 EQ31. Has the OPMCP led to the development of new techniques to aid future OPM and other pest / disease control? Summary statement: The OPMCP has developed and improved techniques that are likely to aid future OPM control, and might contribute to the control of other pests and diseases. In some cases it is the combination of a set of existing technologies and processes in which they are used that is innovative in a UK tree health context rather than the technologies being especially innovative. It has: ■ Developed and demonstrated a successful approach using pheromone trapping (though the application of the results to aid OPM control remains work in progress, as does the transferability of the knowledge to other pests); ■ Led to improvements in ground spraying techniques and a better understanding of their limitations and effectiveness;
19 February 2016 95
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ Adopted and developed an in-field web-based application for data collection that has improved future control of OPM by enabling a rapid response to new infestations. This learning could also be applied to other pest and disease control; ■ Developed useful techniques for stakeholder engagement, and has established working relationships with stakeholder groups that are novel in a UK tree health context. The relationships and approaches should aid the control of future pests and diseases particularly if they also affect urban environments. Analysis: One of the main objectives of the OPMCP was to develop novel techniques in survey, control and stakeholder engagement to aid future OPM and other pest & disease control, especially in urban areas. It was therefore expected that experience and learning under the OPMCP would lead to some degree of innovation in these areas that could be applied both to future OPM control and to other pests and diseases. It has led to the application and better understanding of pheromone trapping, the success and usefulness of which has also improved over the three years. This identified various learning points, such as the need to test manufacturers’ claims that the pheromone would work for a given species on the specific population of interest before applying at scale. However, it is not yet clear how these surveillance results can be used to inform future surveying efforts or control work. The surveillance indicates that some moths are present in an area, but translation of the results into estimates of OPM nest location, population density and viability is not yet possible. As the pheromone traps and their application have been developed specifically for OPM, the transferability to other plant pests is not yet apparent. “I’m not sure, I don’t really know [if the OPMCP has made a difference]. Possibly with the pheromone trapping” The OPMCP has also led to a better understanding of, and improvements in the use of, ground spraying techniques. For instance, it appears that ground spraying of pesticides is much less effective when used on canopies above a height of 20 metres. This kind of knowledge will aid the control of OPM in the future, and might help the control of other plant pests and diseases for which ground spraying of pesticides is an appropriate treatment. Consultees’ views on the significance of the innovations and developments in technique fostered by the pilot programme were mixed. Some regarded them as important, others saw them as comparatively minor. “Yes [the OPMCP has led to the development of new techniques], such as aerial spraying and the use of ground based sprays. We know now how best to apply those and the limitations that those may have. There’s also the pheromone trapping, we didn’t know much about that before. That’s all been positive” “The programme hasn’t developed new techniques, it’s probably just helped us understand how to implement a spray programme at this level and the best way to do that, and understand which application methods are better.” “Not sure we’ve been driving [the development of new techniques] necessary... The programme’s impact on that has probably been quite limited” The OPMCP has developed approaches and relationships to stakeholder engagement and awareness raising that for the FC - which is unaccustomed to working in an urban environment - are novel. These could be used to aid in the management of future pests and diseases if they also affect urban environments.
19 February 2016 96
Evaluation of the Oak Processionary Moth Control Programme - Final Report
6 Learning evaluation findings
6.1 Introduction The OPMCP was a pilot programme that tested new strategies on a novel pest in a new operating context (the FC has historically not had to apply a large scale tree pest control programme in an urban environment). The programme theory involved a mix of complementary interventions - physical controls, economic incentives, stakeholder engagement, advice and management support – and new kinds of partnership and governance. This section considers the evaluation questions that address the learning outcomes of the pilot programme. The identification and dissemination of points of learning, and their application to future decisions and operations, is arguably as important as the results on the ground. The UK faces the prospect of more novel tree pests entering the country in the years ahead and needs the best possible toolkit of management approaches and control measures. Much of the potential economic return on the OPMCP investment is contained in those lessons and how they help to make future control strategies more effective and efficient. The learning questions considered are: ■ EQ32. Overall, has the OPMCP programme model worked as envisaged? What worked well and what not so well? ■ EQ33. What lessons can be drawn from the OPMCP for the design and delivery of similar programmes in the future? ■ EQ34. What lessons can be drawn from the OPMCP for the development of capability to tackle similar pest, pathogens and diseases? ■ EQ35. What lessons does the OPMCP provide for wider decision making processes in the future (i.e. to address similar plant pests / pathogens / diseases)? ■ EQ28. Has the control programme changed the collective understanding of the scale or timescale over which impacts can be realised? ■ Has enough been done to capture all the capability, system and knowledge impacts of the OPMCP and make them available for application to the control of similar pests in the future? If not, what needs to be done and by whom? An overview of the summary statements for each of the evaluation questions is given below. The analysis and supporting evidence is provided in the detailed responses in the sections that follow.
19 February 2016 97
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 6.1 Summary statements for the learning evaluation questions
Learning evaluation questions EQ28 Has the control The programme has confirmed if the opportunity to strike early and strike hard has been missed then containment and programme changed the eradication of OPM is difficult. Eradication outside London and strict containment (the original programme goal) of OPM in collective understanding London seem unlikely to be achieved in the near future. That these goals have not yet been achieved thus far by the pilot of the scale or timescale programme is not a consequence of deficiencies in the programme operational model and its implementation so much as some over which impacts can of the basic assumptions on which it was based now being open to question. For example: the surveillance data and be realised? identification of new outbreak areas suggest that OPM might have been more widely distributed in 2013 than was appreciated at the time; OPM being found by the surveillance programme in the smaller outbreak areas points to the challenges of achieving complete eradication even where robust controls are applied; and the experience gained with ground-based spraying points to the limitations of the technique, especially with larger trees and in a woodland environment. With the benefit of hindsight, it can be seen that the challenge set for the pilot programme was even greater than was appreciated at the time. If OPM is indeed more dispersed than was originally understood, questions are prompted about the future distribution of OPM control effort, how best to mitigate the public health and other risks posed by OPM, the role of public and private sectors in the application of controls, how best to avoid sub-optimal outcomes (such as landowners felling oaks to avoid OPM control costs), and the targeting and content of communications. The current trajectory raises some difficult issues. As the London outbreak area gets larger, the resources for control are likely to get more stretched. And if others do not step in to take its place as the programme exits from the core zone then OPM populations will be growing and dispersing into the buffer zone from the inner boundary even as the programme tries to contain its outer perimeter. With the buffer zone being continually repopulated and both detection and control imperfect, it seems unlikely that the current arrangements provide a long term solution to the OPM problem. Use of SPHNs also seems unlikely to provide a viable alternative given evidence on compliance, the cost burden on landowners and the potential for perverse outcomes (e.g. felling of oaks). The programme data also raise some questions about the scale of OPM impacts. Public health impacts within the outbreak areas are not visible at the scale that the literature suggests ought to be occurring, even allowing for the suppression of the OPM population and for mis-diagnosis and under-reporting of OPM-related health problems. The gap between assumed and observed health impacts needs to be better understood. The data collected by the programme addressed operational needs but were less well suited to supporting analysis of its impact. Some impacts, including those on public health, are expected to be influenced by OPM population density as well as the outbreak’s geographical extent. The pilot programme data do not support estimation of population density and how it is changing in the outbreak area, though the scale and efficacy of control suggests that the population in controlled areas is much less than would otherwise be the case. The programme design does not facilitate the comparison of outcomes in areas subject to the programme with outcomes in untreated areas. These issues could have been addressed had the measuring of impact been considered at the programme planning stage, and the collection of relevant data factored into the resource plan.
19 February 2016 98
Evaluation of the Oak Processionary Moth Control Programme - Final Report
EQ32 Overall, has the OPMCP The programme model has, overall, worked as envisaged. As explained in this report, there was evolution from year to year in programme model many aspects of programme delivery as informed by experience and new evidence, but the basic architecture has proven to be worked as envisaged? sound and the individual components worked as anticipated. A schedule of specific elements that worked well and those that What worked well and were less successful is offered in Table 6.1. what not so well? EQ33 What lessons can be Lessons for similar programmes are to: drawn from the OPMCP ■ Build challenge and external scrutiny into the development of the programme theory, goals and design. In the OPMCP for the design and context such challenge might have prompted the programme to think more about the relationship between OPM population delivery of similar density (as opposed to the movement of frontier of the outbreak) and the impacts on human / animal / tree health, and its programmes in the implications for the control strategy and for data collection. It might, for instance, have encouraged a strategy that took future? more of a risk-based approach to OPM control (e.g. targeting controls on locations in the core and buffer zones where public health risks are greatest and using public awareness-raising to reduce ambient exposure risks). ■ Consider evaluation and impact assessment data requirements at the design phase and build those into the programme data strategy. The OPMCP experience highlights the need for the programme design to consider and plan to meet the information needs of an impact evaluation as well as the programme’s own operational goals. Identifying the evaluation data requirements ex ante, at the design stage, and building the relevant data requirements into the detailed programme design and budgeting would support more robust monitoring of impact and more systematic capture of learning points. ■ Ensure that there is clarity on the relationship between the government policy objectives and the goals of pilot programmes, and that expectations of programme and stakeholders are aligned. ■ Look at the OPMCP approach to building formal ‘learning loops’ into project design to help to accelerate the rate of learning and improvement within the programme. This should include inputs from beyond the core programme delivery team. ■ When establishing new programme infrastructure provide enough time for inception phase planning, preparation and the acquisition of the capabilities to implement it, and calibrate expectations for what is possible in year 1 accordingly. ■ Secure multi-year funding or else try to ensure that the budgeting cycle will not trip over the programme timetable as shaped by the annual life cycle of the pest in question. Otherwise, funding may be confirmed too late for arrangements to be in place for controls to be applied at the optimum time. ■ Ensure the core programme management function is adequately resourced and that programme governance arrangements (e.g. programme board and high level stakeholder advisory bodies) include all significant stakeholders; ■ Ensure effective communication and stakeholder engagement at both strategic and operational levels when operating a control programme in urban areas and areas with public access, and restate and explain the programme goals on a regular basis. ■ For novel pests such as OPM, solutions need to be tested before it can be assumed that they will work. For example, it was found that two pheromones marketed for use in OPM surveillance did not work with the UK OPM population. A third source that did attract male moths to the traps was ultimately found. EQ34 What lessons can be OPMCP lessons for development of capability are: drawn from the OPMCP
19 February 2016 99
Evaluation of the Oak Processionary Moth Control Programme - Final Report
for the development of ■ Contracting mechanisms that engage suppliers for several years and which encourage collaboration and feedback can capability to tackle accelerate the development of control capabilities; similar pest, pathogens ■ Continuity of personnel in key roles helps programme delivery. The OPMCP benefits greatly from the lack of turnover of and diseases? staff in key roles, most especially the programme manager role. This helped retain knowledge and build capability within the programme; ■ Web-based technologies can enhance programme capabilities and cost-effectiveness (the evidence suggest the OPMCP made some progress in this area but did not achieve all available gains from such technologies); ■ Capabilities developed for one pest will not necessarily be applied to others without translational support – pilot programme plans should provide for activities that support the mainstreaming of new capabilities and of lessons learnt into institutional practice. EQ35 What lessons does the Lessons provided by the OPMCP (and the history of OPM control in the UK) for decision-making on similar plant pests and OPMCP provide for pathogens are: wider decision making ■ That effective control and eradication of a novel pest or pathogen is difficult once it has become established. This argues processes in the future for effective surveillance systems and a prompt and comprehensive response when novel pests are identified. Many of the (i.e. to address similar consultees for this evaluation had concluded that a key lesson of the OPM experience is that to be successful one has to plant pests / pathogens / act boldly and early. diseases)? ■ That a coordinated response by public agencies, backed by funding for controls, can be more effective in tackling outbreaks of novel pests than an approach based on prompting landowner action through statutory notices. The OPMCP deployed many more controls, much more quickly than had previously been possible. ■ That public commitments to absolute success – i.e. eradication – should be avoided if detection and control mechanisms are not 100% effective and the boundaries of the outbreak not known with full confidence. If pilot programme goals are conflated with overall policy goals it can then be difficult to manage expectations. ■ That identification of the key sources of uncertainty is helpful when setting goals and objectives, as is explanation of how the programme design proposes to address the uncertainty. In the OPMCP case, it now seems possible that assumptions about the ex ante distribution of OPM may have been incorrect. Had that risk been more explicit, extended surveillance might have been conducted earlier. ■ That early attention to identifying and engaging stakeholders is important to establish partnerships and identify concerns and constraints that may impact on the control efforts (e.g. acceptability of aerial spraying). An open, collaborative approach can make a programme stronger and can be made to work successfully even in complex multi-stakeholder operating environments, but it requires the building of trust, transparency on all sides, and leadership.
19 February 2016 100
Evaluation of the Oak Processionary Moth Control Programme - Final Report
6.2 Learning evaluation questions
6.2.1 EQ32. Overall, has the OPMCP programme model worked as envisaged? What worked well and what not so well? Summary statement: The programme model has, overall, worked as envisaged. As explained in this report, there was evolution from year to year in many aspects of the detailed programme as informed by experience and new evidence, but the basic architecture has proven to be sound and the individual components worked as anticipated. A schedule of specific elements that worked well and those that were less successful is offered in Table 6.2. Analysis: A schematic representation of the OPMCP programme model is provided in Annex 3 and the programme theory described in section 2.2. In the model a programme management system which sets strategic goals and budgets and handles high level stakeholder engagement sits above an operational delivery system that uses intelligence on OPM presence to direct control activity and manages communications, data services, surveillance, etc. The programme sits within host institutions that supply services to the programme, such as the scientific advice and communication support. It also uses third party contractors, such as for data services, surveillance and for the application of controls. The programme model has, overall, worked as envisaged. As explained in this report, there was evolution from year to year in many aspects of the detailed programme design but the basic architecture has proven to be sound: the separation of programme management and operational delivery has been effective; efforts to ensure in-year and year-to-year learning and improvement have been successful; stakeholders engagement has had positive results; landowners have been supported and mostly responded positively; control contractor efficacy and capacity has improved; information on OPM presence is collected quicker and acted upon very much faster than was the case hitherto; and the coordination and management of the fight against OPM is much stronger. And, importantly, there has been a great deal of knowledge gained on many aspects of the OPM control and the delivery of complex tree pest control programmes, especially programmes operating in an urban and suburban environment. Table 6.2 identifies aspects of the programme that worked well and what was less successful. It is structured around the five main components of the programme model.
6.2.2 EQ33. What lessons can be drawn from the OPMCP for the design and delivery of similar programmes in the future? Lessons for similar programmes that tackle established pests are: ■ Build challenge and external scrutiny into the development of the programme theory, goals and design. In the OPMCP context such challenge might have prompted the programme to think more about the relationship between OPM population density (as opposed to the movement of frontier of the outbreak) and the impacts on human / animal / tree health, and its implications for the control strategy and for data collection. It might, for instance, have encouraged a strategy that took more of a risk-based approach to OPM control (e.g. targeting controls on locations in the core and buffer zones where public health risks are greatest and using public awareness-raising to reduce ambient exposure risks). ■ Build an evaluation and learning cycle into the programme so that evaluation is embedded in its delivery. The OPMCP experience highlights the need for the
19 February 2016 101
Evaluation of the Oak Processionary Moth Control Programme - Final Report
programme design to consider, plan and budget to meet the information needs of an impact evaluation as well as the programme’s own operational goals in the design and inception stage. This would support more robust monitoring of impact and more systematic capture of learning points. ■ Ensure that there is clarity on the relationship between the government policy objectives and the goals of pilot programmes, and that expectations of programme and stakeholders are aligned. ■ Look at the OPMCP approach to building formal ‘learning loops’ into project design to help to accelerate the rate of learning and improvement within the programme. This should include inputs from beyond the core programme delivery team. ■ When a programme requires new delivery arrangements (as opposed to making use of systems set up under contingency planning arrangements) it is necessary to provide enough time for inception phase planning, preparation and the acquisition of the capabilities to implement it. This include staffing and contracting mechanisms (e.g. procurement processes for contractors to apply controls, data services). Expectations for what is possible in year 1 need to be calibrated accordingly. ■ Secure multi-year funding or else try to ensure that the budgeting cycle will not trip over the programme timetable as shaped by the annual life cycle of the pest in question. Otherwise, funding may be confirmed too late for arrangements to be in place for controls to be applied at the optimum time. ■ Ensure the core programme management function is adequately resourced and that programme governance arrangements (e.g. programme board and high level stakeholder advisory bodies) include all significant stakeholders; ■ Ensure effective communication and stakeholder engagement at both strategic and operating levels when operating a control programme in urban areas and areas with public access. Lessons include: – It is difficult to secure public buy-in to aerial spraying of pesticides, especially in amenity woodland; – There may be a need to restate and explain the programme goals on a regular basis56; ■ For novel pests such as OPM, solutions need to be tested before it can be assumed that they will solve a problem. For example, it was found that two pheromones marketed for use in OPM surveillance did not work with the UK OPM population. A third source that did attract male moths to the traps was ultimately found. The pilot programme might also have been designed to incorporate more experimentation and targeted research to tests assumptions embedded in the programme theory and identify what methods work best. There was, for instance, scope to test different communication strategies, or to research the public health impacts in specific areas. It is clearly preferable for new pests to be controlled and eradicated before they come established. This requires a system that provide early detection and a rapid, effective response. Consideration of current arrangements for this was beyond the scope of this evaluation.
56 Some consultations with the Advisory Group suggest that the shift from strict containment to slowing the spread of OPM in the London outbreak was not universally understood.
19 February 2016 102
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table 6.2 What worked well and less well in the OPMCP
Programme What has worked well What worked less well element
Oversight, governance and operational management
Governance The programme governance structures, notably the Project Board and It took time to balance the composition of the Advisory Group and provide the structures Advisory Group, worked well and were appreciated by internal and transparency that was necessary to build trust in the group, though these external stakeholders. They set the tone for the collaborative approach issues were satisfactorily resolved. Neither body was able to secure full that defined the OPMCP at programme management and operational engagement in the OPMCP from local authorities. levels. They provide useful forums for debate and communication of strategic options amongst the key actors. Programme The annual ‘plan, implement, review and change’ cycle that was adopted Insufficient time was allowed for mobilisation, planning and preparatory management helped to ensure that annual plans were informed by experience of the contracting such that objectives (and budgets) set for year 1 were not preceding year. achievable. Budgeting There was a structured budgeting process that was able to adjust to The annual cycle of planning and application of controls did not align reductions in the funds available each year. particularly well with the budget approval cycle. The need to secure ministerial approval for an annual budget each year and the timing of the decision in The programme quickly acquired knowledge about input prices and cost relation to the rhythm of the control programme created uncertainty about the drivers that helped with the budgeting process. availability and scale of funding. Expenditure was broadly consistent with budget, although there was some The announcement of the withdrawal of funding for controls in the core zone level of overspend on some operational elements, often due to unforeseen in 2015, as a consequence of budget constraints, came too late for some circumstances (e.g. new outbreaks being identified). The under-spend in landowner to budget for controls themselves and resulted in lower levels of year 1 was a consequence of there being insufficient available time to control in the core zone than would have been the case if the change had launch the project and appoint control contractors. been communicated earlier. There is evidence that the programme was able to deploy controls at a The contingent risk of discovery of new outbreaks was difficult to budget for. lower average cost than was achieved by landowners acting individually. Operational Having a dedicated programme manager for the duration of the The programme manager was not provided with dedicated administrative management programme was essential to the OPMCP’s operations. support until the third year of the programme, with some impacts on the tasking and oversight of controls.
19 February 2016 103
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Programme What has worked well What worked less well element
The continuity and efficacy of operational management had a positive impact. There was an active and effective programme manager who was present throughout the programme. Having dedicated resource was important to ensuring the programme was implemented as intended. Day-to-day communication and coordination between programme management and contractors was generally good. A shift to bulk production of SPHNs led to efficiency gains
Direct action through the implementation of control measures, including prophylactic spraying and manual nest removal
Controls The centralisation of control coordination and delivery in the OPMCP, Control costs were sometimes higher than necessary because contractors together with the supporting information systems, enabled the programme lacked access to data on site access, owners, etc. so incurred more time. to be far more responsive and apply far more controls in the critical time The programme did not always get timely information on when site controls periods than was feasible under a devolved model where responsibility lay had not been applied. with landowners: more than 90,000 trees on almost 700 sites were sprayed and almost 21,000 OPM nests removed. An opportunity was missed to do a structured test of the impact of withdrawal of public funding when money for controls in the core zone was withdrawn in The OPMCP approach of public funding of controls, communications, etc. 2015. appears to have encouraged reporting of OPM and landowner participation in the programme, particularly for small/private landowners. Coding of trees/sites did not always give the programme information about the height of trees, which meant that the control strategy could not be tailored to The use of framework contracts secured access to services at known conditions (ground-based spraying is thought to be less effective on trees over prices, from qualified contractors and provided flexibility in the about 20 m). commissioning of individual firms for particular tasks, and in the techniques used. There was good collaboration among contractors. The application of controls was sometimes limited by the capacity of contractors during the busiest periods. The evidence suggests the programme improved the cost-effectiveness of controls for the average tree, albeit by less than was expected. The notification of the withdrawal of funding for controls in the core zone in 2015 came too late for some major landowners to budget for the applications Programme demand has helped to strengthen the capability of the control of controls themselves. sector.
19 February 2016 104
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Programme What has worked well What worked less well element
The OPMCP contractors used Bt, which is the most targeted of the available pesticides. Previously some landowners’ contractors had used more damaging chemicals.
Surveying and surveillance efforts, including data storage and management;
Data capture and Data capture and management systems developed over the life of the The programme data (surveys, surveillance, control, etc.) specifications were management programme and came to be one of its stronger components. designed to meet OPMCP operational needs rather than to support analysis of systems the programme’s impact and progress towards its stated goals. Data were collected from large scale surveys and from a surveillance effort using pheromone traps that increased significantly in scale during the life Email/phone contact data for landowners were not systematically collected, of the programme. with negative impacts on follow-up research. Use of volunteers provided cost-effective survey capacity57 Delays in commissioning of a new Forestry Commission data management system meant that the OPMCP was reliant on a database held in MS Access A smartphone app increased the reliability of survey data and the for longer than anticipated. While copies of the database were held by some timeliness of its submission. FC staff, some consultees noted that there had been less access and more A programme database under expert management provided operational reliance on the contractor to mediate data queries than would be ideal. information to the programme manager in a timely manner. A general framework contract was used instead of one specific to the OPMCP. This did not specify that suppliers had to have survey capabilities relevant to OPM until later on in the programme period. Despite action to facilitate proper coding of OPM-related health incidents within the NHS, the programme is receiving little information on reported health impacts. It is not clear how far the small number of incidents reflected a continuation of the under-reporting issue or these incidents are just very uncommon. Data strategy Reporting systems were developed that met core operational needs. The data strategy did not service evaluation and impact assessment purposes well. Information relevant to the determination of the impact of the programme The system enabled survey data to be rapidly turned into actionable on spread and OPM population dynamics was not collected. intelligence.
57 Volunteers have been used by some landowners of large sites in the core zone (e.g. Richmond and Bushy Park), but were not used by the OPMCP.
19 February 2016 105
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Programme What has worked well What worked less well element Opportunities were missed to capture information about impacts (e.g. human health impacts) and the effect of programme interventions. For example: research was not conducted on public health impacts when the initial outbreak at Bethlem hospital was identified and a high density of OPM nests were present; the withdrawal of funding from the core zone in 2015 created an opportunity to see how this affected the level of control implemented by landowners who now had to fund the control measures themselves. Communications and awareness raising, including stakeholder engagement Communications Communication efforts expanded and improved over the programme Funding came too late in the first year of the programme for much period and appear to have been effective in reaching most of the key communication effort to be feasible and it was only in the third year that the stakeholder groups. effort was really scaled up. Evidence suggests that the communications effort has reached important The communications activity failed to reach schools effectively. stakeholder groups. Some landowners objected to the way SPHNs were deployed and the An OPM Tree Owners’ Manual received mostly positive feedback. language used, which some saw as unnecessarily confrontational. There are some independent reports of OPM sightings reaching the FC, Communication with landowners could have been improved, with more and suggesting awareness of OPM and the programme has improved. better information being provided to them about the OPMCP, and the control activities being carried out on their site. There were also delays in the time it took to respond to landowners which sometimes had unintended consequences (e.g. landowners implementing their own controls). The programme did not hold email addresses for many landowners and this restricted the reach of its electronic communications. Stakeholder Stakeholder engagement worked well and a collaborative working culture Local authorities proved to be difficult to engage in the programme. engagement was fostered. The Advisory Group was helpful in this regard. The programme did not have sufficient information on the potential negative impacts of different controls (e.g. on biodiversity) to inform engagement with stakeholders. There was limited engagement in the programme from public health professionals.
19 February 2016 106
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Programme What has worked well What worked less well element Scientific research and analysis Use of science The programme has been open to, and learned from, scientific advice and There was little engagement with other EU countries to see what lessons their research evidence, much of the information transfer to the programme experience, and capabilities, could provide for the UK. management being mediated via Forest Research. The programme’s access to and use of the scientific evidence base could have been more structured and systematic. The programme looked for results more rapidly than was consistent with standard research review and reporting processes, suggesting that alternative commissioning might be explored. Improvement and The efforts to foster improvement and learning worked well, and there learning were improvements in many aspects of the programme’s detailed design and delivery over the programme period.
19 February 2016 107
Evaluation of the Oak Processionary Moth Control Programme - Final Report
6.2.3 EQ34. What lessons can be drawn from the OPMCP for the development of capability to tackle similar pest, pathogens and diseases? OPMCP lessons for development of capability are: ■ Contracting mechanisms that engage suppliers for several years and which encourage collaboration and feedback can accelerate the development of control capabilities; ■ Continuity of personnel in key roles helps programme delivery. The OPMCP benefits greatly from the lack of turnover of staff in key roles, most especially the programme manager role. This helped retain knowledge and build capability within the programme. ■ Web-based technologies can enhance programme capabilities and cost-effectiveness (the evidence suggests the OPMCP made some progress in this area but did not achieve all available gains from such technologies). ■ Capabilities developed for one pest will not necessarily be applied to others without translational support – pilot programme plans should provide for activities that support the mainstreaming capabilities and lessons learnt into institutional practice. There has not yet been a structured process of identifying the knowledge, processes, tools and other potential capability enhancements that have been developed through the pilot programme and facilitating their integration into the UK’s plant health system. For Defra, FC and other parties to realise the full benefits of the OPMCP investment there is likely to be a need for actions that go beyond the dissemination of this report and of other ‘lessons learnt’ documents.
6.2.4 EQ35. What lessons does the OPMCP provide for wider decision making processes in the future (i.e. to address similar plant pests / pathogens / diseases)? Lessons provided by the OPMCP (and the history of OPM control in the UK) for decision- making on similar plant pests and pathogens are: ■ That effective control and eradication of a novel pest or pathogen is likely to be very difficult once it has become established. This argues for effective surveillance systems58 and a prompt and comprehensive response. Many of the consultees for this evaluation had concluded that a key lesson of the OPM experience is that to be successful one has to act boldly and early. ■ That public commitments to absolute success – i.e. eradication – should be avoided if detection and control mechanisms are not 100% effective and the boundaries of the outbreak are not known with full confidence. ■ That identification of the key sources of uncertainty is helpful when setting goals and objectives, as is explanation of how the programme design proposes to address the uncertainty. In the OPMCP case, it now seems possible that assumptions about the ex ante distribution of OPM may have been incorrect. Had that risk been more explicit, extended surveillance might have been conducted earlier. ■ That a coordinated response by public agencies, backed by funding for controls, can be more effective in tackling outbreaks of novel pests than an approach based on prompting landowner action through statutory notices. The OPMCP deployed many more controls, much more quickly than had previously been possible.
58 Early detection might be encouraged – e.g. by providing financial rewards to landowners / arboriculturalists / public for reporting sightings of novel and pathogen pests, subject to expert verification by FC.
19 February 2016 108
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ That early attention to identifying and engaging stakeholders is important to establish partnerships and identify concerns and constraints that may impact on the control efforts (e.g. acceptability of aerial spraying). ■ That, if a pest becomes more established and eradication is no longer possible, it is necessary to assess what risks the pest poses and what is an appropriate and proportionate response and/or approach to its management.
6.2.5 EQ28. Has the control programme changed the collective understanding of the scale or timescale over which impacts can be realised? The programme has if the opportunity to strike early and strike hard has been missed then containment and eradication of OPM is difficult. Eradication outside London and strict containment (the original programme goal) of OPM in London are policy goals that seem unlikely to be realised in the near future. That these goals have not yet been achieved thus far by the pilot programme is, on the basis of the evaluation research, not a consequence of deficiencies in the programme operational model and its implementation so much as some of the basic assumptions on which it was based now being open to question. For example: the surveillance data and identification of new outbreak areas suggest that OPM might have been more widely distributed in 2013 than was appreciated at the time; OPM being found by the surveillance programme in the smaller outbreak areas points to the challenges of achieving complete eradication even where robust controls are applied; and the experience gained with ground-based spraying points to the limitations of the technique, especially with larger trees and in a woodland environment. With the benefit of hindsight borne of three years of experience and the additional information that has been gathered it can be seen that the challenge set for the pilot programme was even greater than was appreciated at the time. If OPM is indeed more dispersed than was originally understood, questions are prompted about the future distribution of OPM control effort, how best to mitigate the public health and other risks posed by OPM, the role of public and private sectors in the application of controls, how best to avoid sub-optimal outcomes (such as landowners felling oaks to avoid OPM control costs), and the targeting and content of communications. The current trajectory raises some difficult issues. As the London outbreak area gets larger, the resources for control are likely to get more stretched. And if others do not step in to take its place as the programme exits from the core zone then OPM populations will be growing and dispersing into the buffer zone from its inner boundary even as the programme tries to contain its outer perimeter. With the buffer zone being continually repopulated and both detection and control imperfect, it seems unlikely that the current arrangements provide a long term solution to the OPM problem. Use of SPHNs also seems unlikely to provide a viable alternative given evidence on compliance, the cost burden on landowners and the potential for perverse outcomes (e.g. felling of oaks). The programme data also raise some questions about the scale of OPM impacts. Public health impacts within the outbreak areas are not visible at the scale that the literature suggests ought to be occurring, even allowing for the suppression of the OPM population and for mis-diagnosis and under-reporting of OPM-related health problems. The gap between assumed and observed health impacts needs to be better understood. There are only limited anecdotal reports of animal health impacts and evidence of tree health impacts is anecdotal rather than authoritative. The data collected by the programme addressed operational needs but were less well suited to supporting analysis of its impact. Some impacts, including those on public health, are expected to be influenced by OPM population density as well as the outbreak’s geographical extent. The pilot programme data do not support estimation of population density and how it is changing in the outbreak area, though the scale and efficacy of control suggests that the population in controlled areas is much less than would otherwise be the case. The
19 February 2016 109
Evaluation of the Oak Processionary Moth Control Programme - Final Report
programme design does not facilitate the comparison of outcomes in areas subject to the programme with outcomes in untreated areas. These issues could have been addressed had the measuring of impact been considered at the programme planning stage, and the collection of relevant data factored into the resource plan. As discussed at length in this evaluation, there are significant challenges in determination of some programme impacts due to gaps in data on relevant parameters. The evaluation has noted that some impacts, including those on public health, are expected to be a function of OPM population density as well as the outbreak’s geographical extent. The current programme data do not support estimation of population density and how it is changing in the outbreak area, though the scale and efficacy of control suggests that the population in controlled areas is much less than would otherwise be the case. The programme design does not facilitate the comparison of outcomes in areas subject to the programme and untreated areas. The programme has also raised questions about the scale of OPM impacts. Public health impacts within the outbreak areas are not visible at the scale that the literature suggests ought to be occurring, even allowing for the suppression of the OPM population. Whilst mis- diagnosis and under-reporting is likely to be factors, the gap between assumed and observed health impacts needs to be better understood. There are only limited anecdotal reports of animal health impacts and evidence of tree health impacts is anecdotal rather than authoritative. The programme has, however, changed the collective understanding of the timescales over which impacts can be realised. The programme has confirmed that tackling OPM once it is established is difficult. It has provided additional evidence that, if the opportunity to strike early and strike hard has been missed, the challenge of achieving containment and potential eradication is a great deal harder. The benefits associated with eradication outside London and strict containment (the original programme goal) for London seem unlikely to be realised in the near future. That these goals have not yet been achieved is, on the basis of the evaluation research, not a consequence of deficiencies in the programme operational model and its implementation so much as some of the basic assumptions on which it was based now being open to question. For example: ■ The surveillance data and identification of new outbreak areas suggest that OPM might have been more widely distributed in 2013 than was appreciated at the time; ■ OPM being found by the surveillance programme in the smaller outbreak areas points to the challenges of achieving complete eradication even where robust controls are applied; ■ The experience gained with ground-based spraying points to the limitations of the technique, especially on taller trees. With the benefit of hindsight born of three years of experience and additional information it can be seen that the challenge set was even greater than was envisaged at the time. If OPM was indeed more dispersed than was originally understood, questions are prompted about the distribution of OPM control effort, how best to mitigate public health risks, the role of public and private sectors in the application of controls, how best to avoid sub-optimal outcomes (such as landowners felling oaks to avoid OPM control costs), and the targeting and content of communications. The current programme trajectory raises some difficult issues. As the London outbreak area gets larger, the resources for control are likely to get more stretched. And if other sectors do not step in to take its place as the programme exits from the core zone then OPM populations will be growing and dispersing into the buffer zone from the rear even as the programme tries to contain its outer perimeter. With the buffer zone being continually repopulated and both detection and control imperfect, it seems unlikely that the current arrangements provide a long term solution. Reverting to the status quo ex ante, and its reliance on use of plant health legislation, also seems unlikely to provide a viable alternative
19 February 2016 110
Evaluation of the Oak Processionary Moth Control Programme - Final Report
given evidence on compliance, the cost burden on landowners and the potential for perverse outcomes (e.g. felling of oaks).
6.2.6 Has enough been done to capture all the capability, system and knowledge impacts of the OPMCP and make them available for application to the control of similar pests in the future? If not, what needs to be done and by whom? This evaluation describes various learning points to be taken from the pilot programme but there is more to do to ensure that the capabilities, systems and knowledge are captured. As noted elsewhere in this report, the OPMCP has had an influence on attitudes and approaches to tackling other novel pests. ICF’s research suggests that this knowledge and technology transfer has, thus far, happened informally and additional actions in this area during 2016 would help to maximise the return achieved on Defra’s OPMCP investment. These actions should be completed before the OPMCP formally closes and the programme team disbands. The challenge is in making sure that the lessons are embedded in institutional practice. Dissemination alone may not achieve this. It is suggested that the FC, with Defra, prepares and implements in 2016 an OPMCP benefits realisation plan. Components of this plan could include: ■ Preparation of a set of process notes describing in detail the key component processes within the programme, taking a programme manager’s perspective. This material could be prepared by a third party working with the programme manager, and should be designed to fit with FC knowledge management and training systems. The outputs should be designed to be used by FC staff involved in tree pest management. They could be compiled into a programme manager’s handbook. Topics that these notes could cover include: contracting, SPHNs, Advisory Group operation, surveillance planning & delivery, survey management, control management. The plan should extend across all programme components so that lessons are captured as used. For instance: – Communications: collection of stakeholder email addresses to better support dissemination of information and cost-effective collection of feedback; – Surveys: protocols that ensure the size of infestation at each site is measured (e.g. number of trees with OPM, and abundance of OPM), something that was not collected by OPMCP surveyors in a systematic way. ■ A one day event for relevant FC/FR staff on the OPMCP and its wider lessons for tree pest management. This could include content drawn from the evaluation supplemented by perspectives from selected FC/FR staff on the wider applicability of particular approaches and knowledge gathered. The control and data contractors could be involved. The papers/presentations could be posted online for wider access. ■ A workshop on lessons to be learnt in relation to goal-setting, the identification and management of uncertainty in those decisions and the implications for future OPM management. This would be for a Defra and FC audience. ■ Technology and technology transfer: the evaluators have not had sight of the contractual terms governing the OPMCP database or the survey app, and specifically the allocation of intellectual property rights. The database should be properly documented and, subject to the terms of contract, a copy of the database provided to FC in an appropriate form. Publishing the data available online under FC’s open data policy (anonymised where necessary) could help future researchers.
19 February 2016 111
Evaluation of the Oak Processionary Moth Control Programme - Final Report
7 Conclusions and recommendations
7.1 Conclusions This evaluation has examined the Oak Processionary Moth Control Programme in detail. It describes how a new pilot programme model was developed, deployed and improved over the 2013-2015 period. It has successfully demonstrated a collaborative approach to tackling a novel tree pest in heavily populated areas. It has taken a positive approach to learning - seeking feedback on what worked and what did not, and assimilating new scientific evidence – and improving year on year. It has developed the information and logistical capabilities required to apply targeted controls to a large number of sites in short periods of time. The evidence suggests that the programme has had an impact on the spread of the OPM outbreaks and the size of the human population at risk of exposure. That it has not yet achieved all its objectives is not indicative of lack of effort or skill on the part of those involved but instead a reminder of the difficulty of the challenge it was given and the many unknown factors it has faced. The table below provides a brief summary of achievements against programme objectives. A compilation of the answers to individual evaluation questions is provided at the start of this document.
Objective Conclusion Contain the pest to reduce the There is evidence that the OPMCP has slowed the rate of spread likelihood of spread to new areas of OPM in parts of the London outbreak area and contained the (thereby avoiding increased outbreak in Croydon. It is not yet clear whether the new OPM future control costs and buying locations identified in 2015 are extensions of the south-west time to enable new solutions to London outbreak or separate events. be developed) The programme has bought time for development and consideration of options. Eradicate the pest in areas Not achieved as of January 2016. The outbreak in Croydon is outside of the core infestation in contained but not eliminated. Though no new nests have been south-west London found in Pangbourne, the presence of a small number of males in pheromone traps suggests that there could be a residual population in the area, although the male moths may have arrived from elsewhere. Small numbers of nests have been found in the Olympic Park area. New outbreaks were identified, e.g. at Guildford. Reduce the impacts in the current The analysis suggests that without the OPMCP the OPM’s range infested area (e.g. human health) would have extended to cover an area where ~ 3.5 million more people would have been exposed to OPM than is currently the case59. The OPMCP has also reduced OPM population density within the outbreak areas, which should also have reduced public and other health risks. Develop novel techniques in The OPMCP has developed an integrated sighting-survey-control survey, control and stakeholder system that has improved the speed and efficacy of controls. It engagement to aid future OPM has developed approaches to stakeholder engagement and and other pest & disease control, communication that are novel for urban tree health protection. It especially in urban areas has run a large scale pheromone trapping surveillance project. In most instances it is the combination of measures taken, or the operating context, that is innovative, rather than there being fundamental shifts in technology.
59 See text for caveats and assumptions.
19 February 2016 112
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Objective Conclusion Retain core capacity and skills as The programme has helped to sustain capacity in FC and in its a contingency for quick future supplier base. The transferability of skills and systems to other redeployment contexts should be explored further before the programme closes.
7.2 Recommendations This section provides summary recommendations for 2016 and beyond.
7.2.1 Recommendations for 2016 It is recommend that 2016 is used to: ■ Ensure that all the benefits of the OPMCP investment are captured and embedded in the operational and decision-making practices of FC and other programme partners. A ‘benefits realisation’ plan should be prepared and implemented before the programme ends to ensure that the opportunities for learning (such as those identified in section 6 of this report) and transfer of capabilities to FC, FR and Defra are fully exploited. Components of this plan could include process notes documenting key aspects of programme operational practice (governance, surveys, communications, etc.) and events focused on (i) strategic, (ii) operational lessons for FC, FR and Defra; ■ Gather additional evidence that would help to support decisions on future OPM policy. For example: – Tackling the gaps identified in this report, such as the estimation of the outer limits of distribution, programme impacts on OPM abundance and spread, and OPM impacts on public and tree health. – If controls are not being applied systematically across the London outbreak area there is an opportunity to deploy experimental designs that compare impacts in areas where controls are applied and those where controls are not applied. – There is also an opportunity to test communications and interventions that would be needed to support a shift from a comprehensive to a selective, risk-based OPM management strategy. – Ensure that future UK strategies are informed by the management practice, technology and risk-mitigation measures used elsewhere in northern Europe ■ Review OPM policy and develop supporting measures for 2017 onwards. The evaluation confirms that a review of the OPM strategy and policy, based on the OPMCP experience, is warranted. This would need to begin early enough in 2016 for plans and resources to be in place for implementation early in 2017, but should also be flexible enough to accommodate evidence gathered in 2016. Given the OPMCP timetable, some of these items would require prompt action if the 2016 plan is to be modified. An enhanced monitoring scheme is required if Defra and FC are to be able to evaluate the impact of the programme on OPM spread. The OPM monitoring scheme should be subject to formal assessment prior to implementation and cost-benefit analysis should be used to
19 February 2016 113
Evaluation of the Oak Processionary Moth Control Programme - Final Report
decide how many sites should be monitored given the resource available. A well-designed monitoring scheme will be proactive60. Such a monitoring scheme should have: ■ Sites which are consistently surveyed. The monitoring scheme should have a ‘fixed component’ of a set of sites for repeated visits, subject to a good scientific design. One option would be to design corridors going from the centre to beyond the current edge of the OPM distribution (e.g. 2km beyond the current edge) within which all, or a subset of, sites are monitored regularly. If OPM spreads then the corridor of monitored sites should be extended (by adding to the total number of sites monitored or by stopping the monitoring some existing sites to give the capacity to add new sites). ■ Flexibility to survey additional sites when required. We recognise that a fixed design may not provide large enough sample sizes to estimate some parameters. Therefore any fixed scheme should be augmented by selecting additional sites during each monitoring period to ensure that minimum sample sizes (determined in advance) are met. For example: – Some survey effort could be allocated to sites where control did not take place in the previous season ensuring sample sizes are sufficiently high to estimate the survival of OPM in the absence of control (‘natural’ survival). – Some survey effort could be allocated to sites which have already been surveyed in that season in order to rigorously assess detection probability. ■ Surveying at ecologically relevant scales (i.e. surveying trees). Information on OPM at a tree-level is more valuable than information at a ‘site’ level, because sites are defined operationally, not ecologically. Information should be collected in such a way that it can be scaled up to the site level to ensure ‘backwards compatibility’ with the data already collected. ■ Surveying ecologically relevant parameters, especially abundance. Abundance is a crucial parameter allowing data to be interpreted better: it allows stronger inference of history. ■ No interpretation of the data without analysis in order to avoid misleading conclusions such as not taking detection probability, or background rates of natural survival into account when considering operationally important parameters such as control effectiveness. The costs of a properly designed and well executed monitoring strategy may be weighed against the benefits of being able to understand the impacts of the programme investment. This cost-benefit framework can be used to assess how much resource should be allocated. The principle applies to OPM but also to other novel pests entering the UK. Recommendations on further modelling work that could usefully be taken forward to expand on that conducted in this evaluation are provided in Annex A9.5.
7.2.2 Longer term and wider recommendations This evaluation provides lessons that reach beyond 2016 and have implications that extend beyond control of the oak processionary moth. It is recommended that: ■ Evaluation and learning is built into the initial design, planning and budgeting of future programmes. An evaluation plan should be in place before the programme is launched
60 as was the monitoring of horse-chestnut trees which pre-empted the arrival of bleeding canker and the horse- chestnut leaf-miner; Straw & Williams 2013 Agricultural and Forest Entomology, 15: 321–333.
19 February 2016 114
Evaluation of the Oak Processionary Moth Control Programme - Final Report
and the programme’s data strategy should be consistent with the expectations of that plan. It should make the programme theory explicit. ■ Programmes should have monitoring that specifically supports evaluation against core objectives. ■ The lessons of OPM are incorporated into future review of plant health strategy, notably the value of early detection of novel pests and a rapid, effective response.
19 February 2016 115
Evaluation of the Oak Processionary Moth Control Programme - Final Report
ANNEXES Annex 1 References
ADAS (2014) Improved methods for the early detection of oak processionary moth (TH0101) Boyd, L. et al. (2013), “The consequence of tree pests and diseases for ecosystem services”, Science, Vol.342. Crevecoeur, L. (2015), personal communication. DEFRA (2014), Improved Methods for the Early Detection of Oak Processionary Moth – Evidence Project Final Report. DEFRA, 2014. Tree Health Management Plan. Accessed on 23/12/2015 at https://www.gov.uk/government/publications/tree-health-management-plan Defra (2014) Plant Health (Great Britain) Order, as amended, implements the EC plant health regime. Further details are available at https://secure.fera.defra.gov.uk/phiw/riskRegister/plant-health/plant- health-legislation.cfm Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment. Ecosulis (2014), Herridge’s and Horns Copse, Pangborne: Monitoring of Impacts of Oak Processionary Moth Spraying on Birds, The Forestry Commission. EC (2015) Protected Zones. Available from: http://ec.europa.eu/food/plant/plant_health_biosecurity/protected_zones/index_en.htm Evans, H. (n.d.), Oak Processionary Moth Pest Risk, Forest Research, Forestry Commission. Fenk, L., B. Vogel, and H. Horvath (2007), “Dispersion of the bio-aerosol produced by the oak processionary moth”, Aerobiologia, Vol.23, pp.79-87. Forestry Commission (2011), Visit by FC England and Forest Research Staff to Germany to Inspect the Effects of Oak Processionary Moth (Thaumetopoea processionea). Forestry Commission (2013), “Impact assessment”, Control Options for London Outbreak of Oak Processionary Moth (OPM). Forestry Commission (2013) Minutes of: OPMCP Lessons Learned Meeting, Friends House, 17th September 2013. Forestry Commission (2013) Oak Processionary Moth: A review of 2010 challenges, progress since 2010 and further options to achieve greater control. Forestry Commission (2015), “Updates to the OPM Cost-Benefit Analysis”, Control Options for London Outbreak of Oak Processionary Moth (OPM), OPM Project Board. Forest Services Oak Processionary Moth Board Meeting minutes February 2013. Fraedrich, B. (n.d.), “Defoliation and its effect on tree vitality”, Bartlett Tree Research Laboratories Technical Report. Freed, T. and N. Reeve (2014), A Report of the Effects on Non-target Lepidoptera of Spraying with Bacillus thuringiensis var kurstaki for the control of Oak Processionary Moth in Richmond Park, The Royal Parks. Gottschling, S. and S. Meyer (2006), “An epidemic airborne disease caused by the oak processionary caterpillar”, Pediatric Dermatology, Vol.23/1, pp.64-66.
19 February 2016 116
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Gottschling, S. et al. (2007), “Outbreak report of airborne caterpillar dermatitis in a kindergarten”, Dermatology, Vol.215, pp.5-9. HM Treasury (2011), The Magenta Book: Guidance for evaluation, April, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/220542/magenta_book_ combined.pdf. JBA Consulting (2015) Moth population monitoring: Horns and Herridge’s Copse Klug, M. (2015), personal communication. Kuppen, H. (2015), personal communication. Letter from Plant Health Policy team (2014) Tree Health Policy: Update and Future management options on Oak Processionary Moth London Tree Officers Association (2010), Oak Processionary Moth – Thaumetopoea processionea (OPM) Guidance Note. Maier, H. et al. (2003), “The oak processionary caterpillar as the cause of an epidemic airborne disease: survey and analysis”, British Journal of Dermatology, Vol.149, pp.990-997. Maier, H. et al. (2004) “Caterpillar dermatitis in two siblings due to the larvae of Thaumetopoea processionea L., the Oak Processionary Caterpillar”, Dermatology, Vol.208, pp. 70-73. Manderino, R., T. Crist, and K. Haynes (2014), “Lepidoptera-specific insecticide used to suppress gypsy moth outbreaks may benefit non-target forest Lepidoptera”, Agricultural and Forest Entomology, Vol. 16, pp. 359-368. Maronna, A., H. Stache, and M. Sticherling (2008), “Lepidopterism – oak processionary caterpillar dermatitis: appearance after indirect out-of-season contact”, Journal der Deutschen Dermatologischen Gesellschaft, Vol.6, pp.747-750. Moran, D. et al. (2015), Comparing and valuing the impacts of Oak processionary moth in the Netherlands to England, SRUC. Morgan, M., G. Matthews, and R. Bateman (2013), Droplet deposition following helicopter-mounted ULV application of Bacillus thuringiensis in a mature oak forest. Oak Processionary Moth Advisory Group: Terms of Reference OPM 2013 Control Programme Review Report. OPMCP Projected and Forecast expenditure, 2013-2014. OPM ‘Lessons Learned’ Meeting Minutes. May 1st, 2015. OPMCP Projected and Forecast expenditure, 2015-2016. OPM Project Board Meeting 18 September 2015. OPM Project Board (2015) Updates to the OPM cost-benefit analysis Public Health England (2015a), Health Effects of Exposure to Setae of Oak Processionary Moth Larvae: Systematic Review. Public Health England (2015b), Health Effects of Exposure to Setae of Oak Processionary Moth Larvae: Systematic Review: Summary of Findings. Thomas, F. (2008), “Recent advances in cause-effect research on oak decline in Europe”, CAB Reviews, Vol. 3/037, pp.1-12. Townsend, M. (2009), Report on Survey and Control of Oak Processionary Moth Thaumetopoea processionea (Linnaeus) (Lepidoptera: Thaumetopoeidae) (OPM) in London in 2008, Forestry Commission.
19 February 2016 117
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Townsend, M. (2014), Report on Monitoring the Impact on Non-target Lepidoptera of Aerial Spraying to Control Oak Processionary Moth Thaumetopoea processionea (Linnaeus) (Lepidoptera: Thaumetopoeidae) (OPM) in Pangbourne, Berkshire in 2013-14 (year 1). Twery, M. (1990), “Effects of defoliation by gypsy moth”, USDA Gypsy Moth Research Review. Wargo, P. (1979), “Insects have defoliated my tree – now what’s going to happen?”, Ornamentals Northwest Archives, Vol. 3/2, pp. 13-18.
.
19 February 2016 118
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 2 Life cycle of OPM and available control options
A2.1 Survey and intervention in relation to different phases of the oak processionary moth life cycle Figure A2.1 shows the life cycle of the oak processionary moth. The timings of the various stages are approximate, reflecting the relative lack of precise information on larval and pupal development under British conditions, and also seasonal and local variation. In some years, L1 may appear in mid-April and L4 by the first week of May. The stages are described in more detail below and in the Forest Research leaflet on OPM61. Figure A2.1 OPM life cycle and treatment options
Source: Forest Research
61 Available from: http://www.forestresearch.gov.uk/pdf/fr_advice_note_oak_processionary_moth.pdf/$FILE/fr_advice_note_oak_pr ocessionary_moth.pdf
19 February 2016 119
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A2.2 Control options during the OPM life cycle
Egg stage: This is the most difficult stage to survey for as the eggs are laid in batches on a twig, but in the absence of detailed information on typical egg hatch periods in Great Britain, a search of branches for egg masses during the winter months is recommended. Any egg batches found can then be marked for regular inspection from late March onwards and used as indicators of egg hatch and appearance of the first stage larvae, which are the primary targets for insecticide applications. Larval stage: The larvae pass through six stages (instars) during their development and get progressively bigger from one stage to the next. Two different strategies are recommended in relation to larval size, as size influences the susceptibility to insecticides, the risk of tree damage and irritation from the larval hairs. Stages L1-L3: The most effective method of controlling OPM is to apply insecticides against the early stage (L1-L3) larvae. The first three larval stages are the most susceptible to the insecticides approved for use against OPM, especially the biological and growth regulating insecticides (Bacillus thuringiensis and Dimilin) that have a lower overall environmental impact. Fourth stage and older larvae also remain mainly within their silken nests during the day where they are protected from pesticide sprays OPM larvae are very small when they hatch (around 2mm long) and are still less than 1 cm long by the time they reach the third stage. As a rule of thumb therefore, insecticide applications will be most effective when applied to larvae smaller than 1 cm. The majority of larvae will have reached this size by the end of May (by mid May in warmer years). Stages L4-L6 and pupae: The main method of control for larger larvae and pupae is to manually remove and destroy the larvae and their nests, either by using professional vacuum equipment or by removing the nests by hand. Larvae spin bigger silken nests and spend more time within these nests during the day as they grow larger. Eventually, the larvae moult to the pupal stage, again within the nest. By this stage the nest tends to be tougher and usually brown in colour (whitish when first formed), containing cast skins and shed hairs. Removing nests immediately after they are discovered will reduce further damage to trees and minimises risks from dislodging the irritating hairs. However, delaying nest removal until the larvae have completed feeding and have moulted to the pupal stage increases the chances of destroying all of the larvae/pupae within the nests. Nests tend to become visible at stage four onwards when the caterpillars start to form the nests, however at this stage the two pesticides use by the FC (Bt and Dimilin) to control OPM are not particularly effective.
19 February 2016 120
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Large, old nests need to be removed with considerable care, as indicated in the operational safety advice, to reduce the exposure of operatives to the hairs, which are inevitably shed, especially from the cast skins adhering to the nests. During this phase of the life cycle, larvae may also be seen massing on the trunks and branches of trees and moving in the characteristic nose to tail processions that give the moth its common name. Removing larvae and nests manually, by vacuum equipment or by hand, can be very effective in reducing OPM populations, but this method alone is unlikely to lead to eradication, because it may not be possible to find and locate every last larvae and pupae. Eradication of OPM is most likely to be achieved by a combination of methods which includes correctly timed applications of insecticide that treat the whole of the tree canopy. Adult stage: Adults emerge and fly from around the middle of July to early September. Males are strong fliers, the females less so. Deployment of pheromone traps, baited with the female sex attractant pheromone of OPM, will provide an indication of population size and distribution. However, the traps only capture males and, since they are strong fliers, it is uncertain whether the distribution of captures in the traps is an accurate reflection of the local distribution of the breeding population of the moth. Consequently, captures soon after initial adult emergence will tend to provide the most accurate measure of the distribution of OPM in the local area.
19 February 2016 121
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 3 OPMCP Schematics
19 February 2016 122
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 123
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 124
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 125
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 126
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 127
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 4 Full list of evaluation questions
Evaluation questions [unconstrained]62 Impact evaluation questions
Policy 1. Has the OPMCP supported the delivery of the policy goal of containment63 of the oak processionary moth within the main outbreak area and eradication elsewhere? 2. What impacts has the OPMCP had on preparedness and contingency planning for similar plant pests, and for plant pathogens / diseases more generally?
Social 3. What impact has the OPMCP had on public, occupational and animal health (i.e. through the diagnosis and treatment of OPM related conditions)? 4. What impact has the OPMCP had on public recreation? 5. What impact has the OPMCP had on the level of public confidence that OPM is being adequately addressed?
Economic 6. What impact has the OPMCP had on the landscape value of woodlands? 7. What impact has the OPMCP had on the economic value of the timber sector?64 8. What have been the other economic impacts of the OPMCP (e.g. on tourism, house prices, etc.)?
Environmental 9. What impact has the OPMCP had on oak tree health? 10. What impact has the OPMCP had on biodiversity?
Process evaluation questions
Governance and management 11. How effective has the OPM Project Board been in steering the OPMCP? 12. How effective have the OPM Advisory Group been in supporting the work of the OPMCP? 13. How effective have programme level management arrangements been? 14. Have OPMCP budgeting arrangements worked as intended? 15. Did the programme provide capacity appropriate to the scale and nature of the challenge?
Operational management
62 The evaluation was not able to answer all of these questions due, for instance, to a lack of information. 63 Now defined as slowing the rate of spread to <1km per year. 64 i.e. cost savings from OPM being contained and therefore not affecting potential commercial timber sites
19 February 2016 128
Evaluation of the Oak Processionary Moth Control Programme - Final Report
16. Has the OPMCP day-to-day operational management function worked as expected? What has contributed to this result?
Control activities 17. What worked well, and what not so well in the implementation of control measures? Have the implementation of control measures worked as expected? 18. Has the OPMCP improved the cost effectiveness of control activities as expected? 19. Did public financing of controls, both in terms of eradication and containment, improve the timeliness and efficacy of controls as intended?
Data gathering and data management 20. Have the OPMCP processes for data gathering and management worked as intended? 21. How effective have data management systems been in supporting the programme delivery?
Communication & engagement 22. Has the OPMCP raised stakeholders’ level of awareness of OPM, its risks, Government policy, their obligations and the action being taken as intended? 23. Has the OPMCP improved the reporting of OPM sightings and health incidents as expected? 24. How effective has the OPMCP been in engaging and promoting collaboration with stakeholders? a. Has the OPMCP increased the willingness of landowners to implement control activities and to act quickly to address OPM infestations as expected? b. Has the OPMCP improved the ability of the FC and contractors to gain access to sites for survey and control activities as expected? 25. Has the OPMCP improved stakeholders’ perceptions of the public authorities involved (FC, Defra) as intended?
Research, learning and knowledge management 26. Have investments in research (e.g. on OPM population distribution and dynamics, control measures and surveillance) contributed to the programme as envisaged? 27. How effective has the OPMCP been in ensuring that programme delivery is informed by current scientific knowledge (including the results of research and analysis) and lessons learnt from within the programme? 28. Has the control programme changed the collective understanding of the scale or timescale over which impacts can be realised?
Capabilities 29. Has the OPMCP improved the ability to collect and manage data on other plant pests / pathogens / diseases as expected? 30. Has the OPMCP improved the capability to control OPM and similar plant pests / pathogens / diseases as intended? 31. Has the OPMCP led to the development of new techniques to aid future OPM and other pest / disease control as intended?
19 February 2016 129
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Learning evaluation questions 32. Overall, has the OPMCP programme model worked as envisaged? What worked well and what not so well? 33. What lessons can be drawn from the OPMCP for the design and delivery of similar programmes in the future? 34. What lessons can be drawn from the OPMCP for the development of capability to tackle similar pest, pathogens and diseases? 35. What lessons does the OPMCP provide for wider decision making processes in the future (i.e. to address similar plant pests / pathogens / diseases)?
A4.1 Evaluation questions excluded from the scope of the evaluation The evaluation was not able to cover all of the evaluation questions listed above, either because the data and information were unavailable or because other questions had to take priority in the design of the research tools and collection of information. Results for questions 5-8 and question 25 are therefore not presented in the sections below. Table A4.1 Evaluation questions not within the scope of the evaluation
EQ Evaluation question Reason for exclusion 5 What impact has the OPMCP had on the level of public Other questions took priority in the design of confidence that OPM is being adequately addressed? the stakeholder interviews and landowner survey. The scope of the evaluation could 25 Has the OPMCP improved stakeholders’ perceptions of the not extend to cover this question. public authorities involved (FC, Defra) as intended? 6 What impact has the OPMCP had on the landscape and carbon The desk research was unable to identify sequestration value of woodlands? any data or evidence on this that could be used to answer the evaluation question. 7 What impact has the OPMCP had on the economic value of the timber sector? 8 What have been the other economic impacts of the OPMCP (e.g. on tourism, house prices, etc.)?
19 February 2016 130
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 5 Further details on the different methodologies used for the evaluation
A5.1 Stakeholder consultation – interviews with core stakeholders Semi-structured interviews were conducted with a set of priority consultees that were regarded as essential for the evaluation based on expectations of their ability to provide information that would inform the evaluation questions. To allow for all of these consultees to be interviewed with the given evaluation resources, more than one person was consulted in a single interview where it was considered that their experience and/or expertise was similar enough to make this viable. Ideally, stakeholders would be interviewed individually. It was not, however, possible to speak to each priority consultee individually with the available evaluation resources. Interviewing multiple people at the same time can risk individuals being influenced by each other, which can affect the results. One interviewee can dominate the discussion and either limit or modify the input others would have given if they had been interviewed individually. ICF tried to limit such effects by encouraging even and fair representation from all those involved. A list of the stakeholders that were interviewed is provided in Annex 6. Twenty people were consulted via 15 interviews. Interviews lasted between one and three hours. Single person interviews were conducted by telephone, two person interviews were conducted face-to-face or through a mix of face- to-face and telephone participation. A list of interview questions covering all the evaluation questions was developed (see Annex 7). A topic guide was prepared for each individual interview using these questions, the guide being tailored to the interviewee’s experience with the programme and area of expertise. The list of specific questions put to each interviewee was determined on a case by case basis. Where the same questions were asked of multiple individuals, the wording of the question remained the same to ensure that the responses were comparable. All interviewees were made aware that their responses would be treated in confidence and would only be available to members of the research team. The results of the interviews were transferred into a coding framework. Responses each interviewees were coded against the relevant evaluation question or additional topic of interest. This provided a framework by which the results could be comprehensively analysed to identify common issues and areas where interviewees differed in their views. A5.2 Stakeholder consultation – written responses gathered from additional stakeholders The design phase identified a number of additional stakeholders whose input would add value and help increase confidence in the findings, but who were not considered essential to the evaluation. The evaluation resources also did not allow for these stakeholders to be interviewed. They were asked to submit written responses to a core set of priority questions which largely mapped onto the key questions developed for the semi-structured interviews (Annex 8). Stakeholders were given three weeks to submit their input, and sent two reminders over that time period. Sixteen people were contacted and nine responses received (a list is provided in A6.3). A5.3 Online survey of landowners A survey was constructed that was intended to inform answers to the following: ■ Impact evaluation questions: – “Did the OPMCP increase the level of public confidence that OPM is being adequately addressed?” [in the absence of a full survey of the public, landowners could act a proxy] (EQ5) – “What impact has the OPMCP had on oak tree health?” (EQ9) ■ Process evaluation questions:
19 February 2016 131
Evaluation of the Oak Processionary Moth Control Programme - Final Report
– Questions related to the implementation of control measures (EQ17 – EQ19) – All questions related to communications and stakeholder engagement (EQ22 – 25). Separate online surveys were developed for landowners representing single sites, and those responsible for multiple sites. Landowners responsible for multiple sites were able to select multiple responses where situations across sites differed (e.g. control measures might have been paid for by the Forestry Commission on one site, but not another). The landowner survey had 18 quantitative questions, several of which were ‘routed’ or tailored to landowners who had selected specific responses to previous questions. Not all landowners were therefore asked the same questions. Respondents were also given the opportunity to provide comments in a free text box at the end of the survey. The sample for the survey was constructed using the landowner contact details held in the OPMCP database. These landowners would therefore have interacted with the OPMCP. Details on the type of landowners responding to the survey, and their experience with the OPM and the OPMCP are provided in 0. The Forestry Commission led the administration of the survey as it was felt that the response rates would be better if landowners were contacted directly by the Forestry Commission. In total, 447 landowners were contacted. Of these, 30 represented more than one site. An email was sent to 252 landowners for whom an email address was available. A postal letter was sent to a further 195 landowners. The letter included a URL for respondents to manually enter into their web browser. The emails and the letters were modified for landowners representing single and multiple sites, i.e. landowners received a single “link” that was appropriate to their circumstance. Two reminders were sent out to those landowners for whom email addresses were available. Landowners had two weeks to submit their response. One hundred responses to the single site survey were received and 21 responses to the multi-site survey. Taking the entire sample of landowners contacted by post and by email, these 121 responses provide a response rate of 27%. The response rate to the email survey is thought to have been much higher than that of the postal survey. The response rate to the multiple-site survey was very high, at 70%. The response rate to the single-site survey is estimated at 24%. A response rate of 20% had been anticipated, based on an assumption that all landowners could be contacted by email. An aggregate response rate of 27% is very encouraging given the number of landowners contacted by postal letter (response rates of 5% or less can generally be expected from postal surveys). Table A5.1 Online survey of landowners – number invitations sent and responses received
Single site survey Multiple site survey Total Email invites sent 226 26 252 Postal invites sent 191 4 195 Total sent 417 30 447 Responses received 100 21 121 Response rate 24% 70% 27%
A5.4 Purpose of the desk research and data analysis The data were expected to inform the following EQs: ■ EQ1 – Has the OPMCP supported the delivery of the policy goal of containment of the oak processionary moth within the main outbreak area and eradication elsewhere? (with specific reference to the impact of Programme activities in Pangbourne);
19 February 2016 132
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ EQ3 - What impact has the OPMCP had on public, occupational and animal health (i.e. through the diagnosis and treatment of OPM related conditions)? (involving further enquiries on human/animal health data; review of PHE report); ■ EQ9 - What impact has the OPMCP had on oak tree health? (rapid scan of reports from selected EU countries); ■ EQ11-14 (Through a review of programme documentation (e.g. Advisory Board minutes)); ■ EQ18 - Has the OPMCP improved the cost effectiveness of control activities as expected? A5.5 Modelling the spread of OPM A model was developed in order to predict the spread of OPM. This model was constructed with parameter values which were estimated from the data available in the OPMCP database. The data in the database are detections of OPM associated with a ‘site’ (defined primarily by land ownership). Therefore the data could be treated as presence/non-detection records at the site level. The design of the model was based upon the life cycle of OPM and it used the data available from the summer and winter surveys to parameterise the model and describe the observed spread of the OPM (Fig. 3.1). The model was a simulation model, so the parameter values could be altered and the results would be a prediction of how the change affects OPM spread. The model was stochastic, which means that each time the model was run the results were slightly different. This is because it incorporated variation (for example: it incorporated the uncertainty associated with some of the parameter estimates, and when applying control, it did so randomly). The results of running the model multiple times could therefore be considered as both the average effect and its variation. In order to construct a simulation model, to predict where OPM would spread, it was necessary to have four pieces of information: ■ The current distribution of OPM (i.e. infested trees, which act as sources of dispersers); ■ The distribution of hosts of OPM (i.e. the distribution of oak trees, beyond the current distribution); ■ The probability that OPM persisted (and was detected) in a site. Persistence could vary by season and would be affected by the application of control. ■ The distant-dependent rate of colonisation of OPM (expressed as a ‘dispersal kernel’ to give the probability x of dispersing and establishing a new outbreak on a host at distance y). This was based on the presence / absence data with exponential distance decay. The impact of control is a key aspect of the OPMCP. To assess this it was also necessary to have information on: ■ The occurrence of control measures per tree/site (i.e. where spraying and nest removal was carried out); ■ The effectiveness of control measures per site/tree (i.e. whether the control was successful in eradicating OPM); and ■ The effectiveness of surveys in detecting extant infestations (i.e. whether infestations occurred but were not detected).
19 February 2016 133
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A5.1 OPM life cycle and control regime – a simplified schematic showing the data that were available and the parameters that were calculated.
Calculating parameter values The model was parameterised with key variables from data collected under the OPMCP during 2013, 2014 and 2015 (Fig. 3.1). By carefully selecting subsets of sites which fitted pre-defined criteria, it was possible to calculate all these relevant parameters one-by-one. (It was not possible to use all sites to estimate each parameter because of missing data or confounding effects; see Annex 7.) However, often the subsets were small and spatially biased (e.g. being from specific regions or from the edge of the range) thus limiting the confidence in the accuracy and generality of the calculated values (Annex 9). This was because the data were collected for the operational requirement to inform the application of control. (More advanced methods, such as maximum likelihood or MCMC Bayesian approaches would have allowed the uncertainty associated with these parameters to be estimated and more of the data to be used, but this was beyond the scope of the current project.) Specifically, the utility of the data was limited because: ■ The data are site-based and sites are defined operationally (e.g. on land ownership) rather than ecologically and vary in size; ■ The spatial extent of sites (i.e. their boundary) is not recorded; ■ There is no consistent information on abundance of OPM, e.g. number of trees infested and number of nests per tree. (This information is sometimes recorded, but it is not collected or recorded in a systematic way); ■ Surveillance is reactive, i.e. surveys occur in places where there is immediate need;
19 February 2016 134
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ Surveys tend to be focussed on the edge of the range and in regions where spread has recently been found to occur; and, ■ Lack of repeated visits within a season (i.e. poor assessment of detection probability) Surveys are focussed on establishing where to apply control (i.e. especially focussing on the edge of the distribution) and recording the success of control (i.e. revisiting controlled sites), but there is not a focus on establishing the result of no control (i.e. natural population dynamics and rates of survival) even though resource limitations means that there are sites with known presence where control was not carried out. The data from the OPMCP were sufficient for all the parameters to be calculated and we gained an approximate estimate of their uncertainty (via a method known as bootstrapping). One of the challenges is that imperfect detection of OPM affects the calculation of the parameters. However, because we could calculate an estimate of detection probability for the winter and summer surveys, we could use this to adjust the estimate of survival in each season, with or without control. The estimates rates of detection were 74% and 95% in the winter and summer surveys, respectively (i.e. the data available shows that OPM was detected in almost every site it was present during the summer, but OPM was missed in about 1 in 4 sites it was presented and which were surveyed during the winter). For some parameters, taking imperfect detection probability into account had a substantial influence on the parameter estimate. Therefore, the OPMCP data should not be interpreted without considering the importance of detection probability. The calculated values of the parameters should be treated as provisional because they are entirely subject to the data available, and this data were collected for operational purposes (to inform control measures) rather than as part of monitoring. The data were collected at a ‘site’ level and this is not comparable elsewhere in Europe, so it is not possible to undertake independent verification of the calculated parameter values. Simulation model and its sensitivity analysis The parameter values were used to construct a simple simulation model. This model was run multiple times (because the model was stochastic each run of the model gave a slightly different result). It was used to predict the spread of OPM in 2013 based on the records of infested sites in 2012. We could not predict the spread of OPM after 2013 because we did not have good enough information on the sites with oak trees into which OPM could spread. On average, the model predicted the spread of OPM reasonably well, but the results from individual runs of the model were very variable, especially in the north of the west London outbreak. However, other sources of evidence showed that there were fundamental limitations of the model, specifically we knew that some of the simplifications in our modelling were unrealistic, and parameter uncertainty was not included. Therefore we had low confidence in using the simulation model to accurately predict a counterfactual to the OPMCP. In particular, when we conducted a sensitivity analysis by systematically vary each of the parameters and recording the impact of this on the predicted spread of OPM, we found that the model was unexpectedly insensitive to these alterations. We even found little impact when changing the effectiveness of or application rate of control or dramatically changing the rate of control of application. This suggests that the model was not adequately parameterised and provided low confidence in using the model to construct a counterfactual to the OPMCP. In general, varying parameters had minimal effect on the predicted rate of spread of OPM (see Annex 9), which was contrary to prior expectations. The parameter which the model was most sensitive to was distance-dependent colonisation. However, we noted that the pattern of spread of OPM appeared to be confounded with the density of sites, because the rate of spread was very sensitive to stochastic effects and changes in the colonisation parameter where there was a low density of sites (e.g. in the north of the West London outbreak) and very resistant to this where there was a high density of sites (e.g. in Croydon/Bromley). Site density is likely to partly artefactual (depending on site size, the
19 February 2016 135
Evaluation of the Oak Processionary Moth Control Programme - Final Report
patterns of land ownership that contribute to ‘site’ definition, and patterns of surveying) and its influence on modelling distribution spread needs to be considered carefully in future work.
Caveats to the modelling work ■ The data on OPM infestation were collected for operational reasons to support the OPMCP, rather than for scientific monitoring of distribution and prevalence. These data were subject to potential bias (towards well-surveyed regions) and small sample size. ■ Parameters were based on ‘sites’ not ‘trees’, but sites are defined operationally (e.g. based on land ownership) rather than ecologically. Some large sites (e.g. Richmond Park) are probably better represented as multiple sites in the model. The local density of ‘sites’ appears to have a large influence on the modelled results and varied across the region of infestation (e.g. the Croydon region contains one- third of the sites in only 2% of the total area of infestation). ■ The parameters were estimated from subsets of the available data. Investing in a combined model to simultaneously estimate the parameters could lead to more rigorous estimates of these parameters, especially effective dispersal (distance-dependent colonisation). ■ The distribution of oak trees is poorly known. This limits the capacity to predict OPM spread beyond the currently surveyed area. ■ Control is currently applied randomly in the model, whereas systematic application of control to a site could be more effective than random control. ■ There is no effect of site size in the model because resource limitations did not allow this to be considered. ■ The model does not account for OPM abundance in a site because the data were not suitable for estimating abundance and it was beyond the remit of the project. All our conclusions are based on spread (site colonisation and extinction) even though reducing abundance of OPM at a site may be a valuable outcome.
Observed distribution and spread of OPM The distribution of OPM was defined for each year by using alpha hulls (where alpha = 2.5km) and from these estimated the observed rate of spread. This approach is standardised and repeatable. The linear rate of spread was calculated as the median distance of points on the perimeter of the distribution to the nearest part of the distribution in the previous year. A5.6 GIS mapping of the population in the area colonised by the oak processionary moth The number of people living within the area currently known to be colonised by OPM was estimated by laying the distribution of OPM (taken from the alpha hull of the OPM distribution, see above) over the human population density (from the 2011 census). The predicted distribution of OPM in the absence of the OPMCP was then calculated. This was based on expert judgement that OPM would have spread about 5km per year in the absence of a coordinated control programme, where treatment was left to the ad hoc and spontaneous action of landowners.65 Data on human population density was then taken from the 2011 census and applied overlaid onto this predicted area of distribution in order to calculate the number of people who would have potentially been living in an area affected by OPM in the absence of the OPMCP.
65 Defra (2013) Control Options for London Outbreak of Oak Processionary Moth (OPM). Impact Assessment.
19 February 2016 136
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 6 Stakeholders consulted
A6.1 Individual interviews
Stakeholder interviewed Organisation Role Date Andrew Hoppit Forestry Commission OPMCP operations 16/10/2015 Charlton Clark Forestry Commission OPMCP communications 16/10/2015 Nigel Straw Forest Research OPM research 20/10/2015 Julia Branson University of Southampton OPMCP data 03/11/2015 management Simon Levy Coombe Forestry OPMCP data collection 30/10/2015 Emer O’Connell Public Health England OPM human health, 16/11/2015 OPMCP project board member Mark Parsons Butterfly Conservation OPMCP Advisory Board 27/10/2015 member Christine Reid Woodland Trust Defra tree health policy 23/10/2015 group member Andrew Smith* Forestry Commission OPMCP Project Board 22/10/2015 previous chair Craig Harrison* Forestry Commission OPMCP Project Board 06/11/2015 previous chair * Two scheduled participants for group interviews, Andy Hall and Alison Field, were unavailable. A6.2 Joint interviews with multiple stakeholders
Stakeholders interviewed Organisation Role Date Amanda Mitchell Defra Tree Health Policy 27/10/2015 Jemilah Vanderpump Defra Mark Townsend Gristwood and Toms OPM control activities 13/10/2015 Dominic Blake Advanced Tree Services Richard Tippet Bartlett UK Nicola Spence Defra Chief Plant Health Officer 16/10/2015 Richard McIntosh Defra Assistant CPHO Colin Buttery Royal Parks OPMCP Advisory Board 23/10/2015 Member Sue Ireland City of London OPMCP Advisory Board Member
A6.3 Written responses
Stakeholder contacted Organisation Role Response received Martin Townsend Independent contractor OPMCP data collection Yes Ralph Park Independent contractor OPMCP data collection No John Morgan Forestry Commission Head of Plant Health GB - Yes OPMCP Project Board Richard Edwards LTOA OPMCP LTOA contact Yes Bill Malin Tree Company OPM control No
19 February 2016 137
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Jon Stokes Tree Council Defra tree health policy Yes group member Mike Robinson FERA OPMCP Advisory Board Yes Member Adam Wallace Natural England OPMCP Advisory Board Yes Member Andrew Jones Greater London Authority OPMCP Advisory Board No Member David Allister Richmond Borough OPMCP Advisory Board No Council Member Craig Ruddick Richmond Borough OPMCP Advisory Board Yes Council Member Dougal Driver SE Regional Advisory OPMCP Advisory Board No Committee Member Gillian Jonusas Royal Parks OPMCP Advisory Board Yes Member Neil Strong Network Rail OPMCP Advisory Board Yes Member David Lighthouse LTOA OPMCP Advisory Board No Member
19 February 2016 138
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 7 Interview questions
A7.1 Impacts 1. How successful has the control programme been in containing OPM within the main outbreak area and eradicating it elsewhere? a. What has contributed to its success? What has limited its success? 2. The government’s impact assessment estimated that, in the absence of the control programme, OPM would have spread between 5 and 10km per year. This means that in 40 years’ time most of England would be infested with OPM. How accurate do you think is this prediction of spread? 3. How serious do you think are the a. human health impacts of OPM? [occupational vs. public] b. animal health impacts of OPM? c. the impacts of OPM on oak tree health? d. The impacts of OPM on public recreation and public access? 4. What impact has the control programme had on mitigating these impacts? a. Human health impacts (occupational and public) b. Animal health impacts c. Oak tree health d. Public recreation / public access 5. What impact has the control programme had on biodiversity? Do you think the impact is net positive or net negative? 6. What impact has the control programme had on the preparedness and contingency planning for similar plant pests and other pathogens? A7.2 Programme level & operational delivery 7. How effective has the OPM Project Board been in steering the OPMCP? What has worked well and what not so well? 8. How effective have the OPM Advisory Group been in supporting the work of the OPMCP? What has worked well and what not so well? 9. How effective have management arrangements been? What has worked well and what not so well? a. Programme level b. Day-to-day operational level 10. Has the capacity and resources provided for under the OPMCP been sufficient? Why or why not? a. Funding b. Staff time c. Staff expertise 11. How well have the OPMCP budgeting arrangements worked? What has worked well and what not so well?
19 February 2016 139
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A7.3 Control measures 12. What worked well, and what not so well in the implementation of control measures? Why? 13. What impact do you think the availability of public funding for control measures has had? a. More effective? b. More timely? c. More willingness / cooperation from landowners? d. More cost effective? e. Any negative / unintended consequences? 14. Has the OPMCP led to improved skills and capabilities to control OPM and other similar plant pests / pathogens / diseases? A7.4 Data gathering and data management 15. How well have the processes for data gathering and management worked? What has worked well and what not so well? 16. How effectively have the data management systems supported the programme delivery? What has worked well and what not so well? 17. Has the OPMCP led to improved skills and capabilities for collecting and manage data for other plant pests / pathogens / diseases? A7.5 Communications and engagement 18. How successful has the OPMCP been in raising stakeholder awareness of OPM and the OPMCP? 19. How effective has the OPMCP been in engaging and promoting collaboration with stakeholders? What has worked well and what not so well? 20. Do you think the stakeholder communication and engagement activities under the OPMCP have had any impacts on stakeholders? Why or why not? a. increased willingness of landowners to implement control activities? b. improved ability of the FC and contractors to gain access to sites? c. improved the reporting of OPM sightings and health incidents? A7.6 Research, learning and analysis 21. What impacts has the research related to OPM had on the programme delivery? (e.g. on OPM population distribution and dynamics, control measures and surveillance) a. Do you think the right research been commissioned? 22. To what extent is programme delivery informed by current scientific knowledge (including the results of research and analysis) and lessons learnt from within the programme? 23. What impact has the OPMCP had on the development of new techniques to aid future OPM and other pest / disease control? A7.7 Research, learning and analysis 24. Overall, do you think the OPMCP programme model has worked well? Why or why not? 25. What lessons can be learned from the OPMCP? a. For the design and delivery of similar programmes in the future?
19 February 2016 140
Evaluation of the Oak Processionary Moth Control Programme - Final Report
b. For the development of capability to tackle similar pest, pathogens and diseases? c. For wider decision making processes in the future (i.e. to address similar plant pests / pathogens / diseases)?
19 February 2016 141
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 8 Questions asked in the request for written responses
Defra and the Forestry Commission are inviting you to share your views The Department for the Environment Food and Rural Affairs (Defra), in partnership with the Forestry Commission (FC), has recently commissioned ICF International to carry out a research study to evaluate the Oak Processionary Moth Control Programme (OPMCP). The purpose of the evaluation is to provide an in-depth assessment of the OPMCP’s achievements, its effectiveness and impacts. The findings of the evaluation will inform ongoing policy, operational design and decision-making relating to the control of oak processionary moth and other similar tree pests. As part of this study we are asking some key stakeholders to submit written responses to questions in order to gather their views and experiences. Your input will help to ensure that future policy and operational decisions regarding OPM control are based on a thorough understanding of the current control programme’s strengths and weaknesses. Below are 15 questions relating to various aspects of the OPMCP. Please provide as much information as possible. You do not have to respond to any questions you don’t want, or do not feel able, to answer. Please email your response to [email protected] by 4 November at the very latest. Please put ‘OPM evaluation written response’ in the subject line of your email. Thank you in advance for your support. If you have any questions with regard to the questions or the study, please do not hesitate to contact Gillian Brown at [email protected] .
19 February 2016 142
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Respondent details – please complete
Name Organisation Role in / experience of the OPMCP * *e.g. member of the Advisory Board, independent contractor to FC for OPM control
Impacts of Oak Processionary Moth (OPM) and the Oak Processionary Moth Control Programme (OPMCP) 1. How serious are the potential impacts of OPM? On human health:
On animal health:
On oak tree health:
On public recreation and the use of greenspace:
Other impacts of OPM (please specify):
2. What impact has the OPMCP had on mitigating these impacts?
3. What impact has the OPMCP had on biodiversity? Do you think the impact is net positive or net negative? Why?
4. How successful has the OPMCP been in containing OPM within West London and eradicating it elsewhere? Why do you think the control programme has been successful / unsuccessful?
19 February 2016 143
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Programme level & operational delivery 5. How effective have the OPM Advisory Group and/or OPM Project Board been in steering and supporting the OPMCP? In what ways have they been effective? [please be clear in your response whether you are talking about the OPM Advisory Group or the OPM Project Board]
6. How effective have the management arrangements been (on both a programme level and day-to-day operational level)? What has worked well / not so well?
7. Have the capacity and resources dedicated to the OPMCP been sufficient? Please explain your answer with reference to staffing resource, staff experience and knowledge, the funding made available for control activities / surveying and surveillance / programme data management / communications / research.
Control measures 8. What has worked well / not so well in the use and implementation of control measures? Has the implementation of control measures worked as expected?
9. What impact has the availability of public financing for control measures had on landowners and the implementation of control measures?
Data gathering and data management 10. How well have the processes and systems for data gathering and management worked? What has worked well / not so well?
11. In what ways, and how effectively, have the data management systems supported the planning and delivery of the OPMCP?
Communications and engagement 12. How effective has the OPMCP been in raising stakeholder awareness of OPM and the OPMCP? What has worked well / not so well?
19 February 2016 144
Evaluation of the Oak Processionary Moth Control Programme - Final Report
13. How effective has the OPMCP been in engaging and promoting collaboration with stakeholders? What has worked well / not so well?
14. What have been the positive (and/or negative) impacts of the stakeholder communication and engagement activities under the OPMCP?
Capabilities 15. Has the OPMCP improved capacity (e.g. human resources) and capability (e.g. knowledge, experience, and skills) to manage and deal with OPM? If so, how and why? If not, why not?
16. Does the improved capacity / capability delivered by the OPMCP mean we are better placed to manage and deal with other tree pests /diseases? Please explain your answer.
Learning 17. Overall, has the approach taken by the OPMCP worked? Why or why not? What elements have been most critical to its successes and/or failures?
18. What lessons can be learned from the OPMCP?
Is there anything else you would like to add?
19 February 2016 145
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 9 Full method and results of the modelling exercise
A9.1 Collation of the data and calculating parameter values
A9.1.1 Methods All analysis was carried out on the presence/non-detection data at the level of a ‘site’. With the simple modelling framework in this project, we calculated the value of parameters (detection in surveys, survival between surveys with and without the application of control, and distance-dependent colonisation). The limitations of the data and the relatively modest resources available for modelling in this evaluation project meant that we did not model spread at the level of a tree, or to include information on levels of infestation (i.e. abundance), even though these would be valuable aspects to include in future, more advanced, modelling. Firstly, we summarised the available survey data (provided by Julia Branson, Geodata, University of Southampton) using ‘site histories’ to efficiently summarise the information for each site and using ‘site codes’ to uniquely identify each site. Therefore, each site is defined as an ‘operational unit’ with the trees within a site owned by a single landowner (the same landowner can own adjacent sites). Although the level of infestation (number of nests) for trees is recorded, the way the surveys were carried out means that we did not have confidence that this information was collected in a consistent way (i.e. we did not know whether number of nests recorded was based on a thorough survey of the tree, and we know that the trees recorded as having OPM present were probably not recorded in a consistent way across sites.). In the database, each site was represented by one or more records of trees without OPM detected (including attributes such as the number of oak trees in the site). Each tree on which OPM was detected was also recorded as an entry in the database. We aggregated all this information at the site level and recorded the status of the site across the attributes each year (Table A9.2) and recorded it as a ‘site history’ across all years. Therefore, a site history “ . 1 -1 0 . ”, means that the site: 1. was presumed not to have been sprayed in spring 2013 (.); 2. was surveyed in summer 2013 and OPM was found (1); 3. nest removal was carried out in summer 2013 (-1); 4. was surveyed in winter 2013/4 and OPM was not found (0); 5. and old nests were not reported during summer 2014 (.). In order to calculate the parameter values, we selected a subset of sites which fitted the criteria necessary to allow calculation of the parameter (Table A9.2). A combined approach to estimate all parameters simultaneously would be more rigorous, but this was not within the remit of the current proposal. We estimated the 95% confidence limits for each estimate by obtaining the 95 percentiles via bootstrapping – this provides an estimate of the uncertainty of the estimate. Selection of the subsets of sites were conditional on OPM being recorded as present at the start, but whether OPM was recorded at the end is also dependent upon the detection probability. Therefore, in order to incorporate the effect of detection probability, the number of site histories in the numerator and denominator of each parameter were scaled by the calculated detection probability. For example, when considering the natural mortality in the summer, we considered a range of sites surveyed at the appropriate times (see Table A9.2 for details) and divided the number of site histories in which OPM was not detected by the total (where OPM was and where it was not detected). However, because of imperfect detection, some of the sites in which OPM was not detected would, on average, have had OPM present. Therefore the number of these sites was reduced accordingly with the estimate of detection rate. Similarly, the number of sites in which OPM was recorded was, on average, too low because OPM would have been present at more, but was not detected. Therefore the number of these sites was increased accordingly with the estimate of detection rate. Therefore, there were changes to both the numerator (the reduced number of sites without detections) and the denominator (the
19 February 2016 146
Evaluation of the Oak Processionary Moth Control Programme - Final Report
reduced number of sites without detections plus the increased number of sites with detections), which changed the overall value of the parameter.
A9.1.2 Results Overall, we were able to calculate all the parameters required Figure A9.1, Table A9.3), but several were obtained from small sample sizes. Distributions of the sites from which the data were obtained also showed that the distribution of sites often appeared to be spatially biased especially during the winter surveys (Figure A9.1). We did not formally quantify this bias, partly because the distribution of hosts is not known, however, the bias is clear in winter 2013/14 when surveys were focussed in three specific regions and from summer 2014 onwards when there is an obvious ring of surveyed sites around the West London outbreak. The consequence of surveys being focussed in particular regions is the risk of bias in the parameters. In particular, survey effort seems to be focussed recently around the edge of the distribution of OPM (at least, in West London). If these are recently colonised locations, then they are likely to have lower abundances of OPM. Also we suspect that these sites will tend to over-represent small sites (especially because there are several large parks in the centre of the West London outbreak). Therefore these parameters may be biased towards values for recently-colonised, low OPM abundance and small sites. This potential bias, and its impacts, could be tested in future work but were outside of the remit of this preliminary modelling approach. The calculated values of the parameters varied between years (Table A1.3), but it was not clear whether this was due to: ■ Small sample sizes causing stochastic variation between years ■ Differences in the type or degree of bias between years (for the reasons discussed above) due to the sites that were sampled. ■ Functional variation due to biological variation, e.g. in the light of the spread of OPM or due to weather. Any, or all, of these reasons could be important. However, for the preliminary modelling presented here we used calculations from the two years combined. Properly understanding variation between years would require further data and a more rigorous method for calculating the parameter values. A9.1.2.1 Detection error (summer) To estimate detection error (summer), we considered the sites where surveys were carried out in the summer, nest removal was not carried out and where OPM was recorded the following winter. We consider what proportion of these sites had OPM detected during the first summer. If OPM was found in the winter then we know it must have been present at that site in the summer, whether it was recorded or not. It is possible that OPM larvae might process between trees, but it is unlikely that these larvae (caterpillars) will move between sites. As far as we know there is no evidence that this has occurred, although it is theoretical possible where sites are very close and the caterpillars move between trees in the two sites, or where caterpillars are successfully transported by human transportation. We expected that detection error in the summer might be relatively high because of the difficulty of detecting the larvae in the tops of trees, but this appears not be have been the case. The estimate suggests that only 5% of sites with OPM were missed by surveyors in the summer. Sample sizes are large and so this suggests that the confidence in this parameter is high. Estimates were conditioned on OPM being detected in the winter, so the imperfect detection of OPM in the winter does not bias this estimate. Possible biases: ■ This was dependent upon sites that were surveyed in the winter and had OPM detected in the winter. We found that winter detection rates were relatively low (see Results), so if winter detection was dependent upon local abundance of OPM (as seems likely) then we could only calculate summer detection for a subset of sites with high abundance, and if summer detection is dependent
19 February 2016 147
Evaluation of the Oak Processionary Moth Control Programme - Final Report
upon abundance (as also seems likely) then our calculated value is likely to be an overestimate of detection probability. ■ The time of the summer survey (early or late season) could have influenced detection, but the estimate is so low that we are not concerned this would have an effect. – If detection in the winter is higher when the infestation by OPM nests (in a tree or a site) is higher, then this estimate represents the proportion of relatively highly-infested sites that were missed during the summer. It provides less clear information on the proportion of lightly- infested sites that were missed in the summer (because they would have been missed in the winter as well). However, we do not have information on the level of infestation per site. (The only relevant information is the number of trees recorded to be infested per survey per site, but we know that the surveys were undertaken for operational reasons and so once OPM was found, the surveyors did not undertake a thorough survey of the remaining trees.) – It includes the effect of spraying events which were unreported to the Control Programme (but these are probably negligible). A9.1.2.2 Detection error (winter) In order to calculate detection error (winter) we considered sites which had been surveyed in the winter, and also surveyed the following summer during which ‘old nests’ were recorded. We considered what in what proportion of these sites was OPM recorded in the winter. If a site was recorded as ‘previously infested’ in the summer, we know that (1) it must have infested at some time in the past (and we presume it was the previous season) and (2) it was not currently infested (because otherwise it would have been recorded as ‘infested’. Therefore if OPM was not recorded in the winter surveys at these sites then it must have been missed. We expected that detection error in the winter might be relatively low because the nests would be more conspicuous when the trees had dropped their leaves. Our estimate suggests, however, that OPM was missed in winter surveys in a reasonably high proportion of sites. However, sample sizes are small, and will be influenced by the pattern of surveying in the winter, which we know was focussed on specific parts of the recently-colonised edge of the OPM range (where probably OPM density per tree/site would be lower). We presume that detection error in sites where OPM was well- established would have been much lower (if winter surveys had been carried out in these sites), and tending towards zero, but this cannot be confirmed from the data. Possible biases: ■ We assume that ‘previously infested’ means that the old nests were from the previous year, and years prior to that. ■ In the dataset available, each record had only one status: ‘not infested’, ‘infested’, ‘previously infested’ or ‘not visited’ so when a site comprised one infested tree it could not be recorded as ‘previously infested’. This would have reduced the potential sample size. Therefore the subset of data used to estimate detection error in the winter was restricted to sites not currently infested with OPM, and so is biased towards sites with low density of OPM in the winter survey (because OPM had died out by or was at such low densities it could not be detected by the following summer). ■ Winter surveys were a focussed survey tool, and so focussed on the edge of the range, where OPM had only just colonised, or around pheromone traps, so for estimating global parameters, the sites represent a biased subset of sites. A9.1.2.3 Apparent extinction (summer) To estimate the ‘apparent extinction (summer) we considered sites where OPM was recorded in the summer, no nest removal was carried out and which were surveyed in the winter. Extinction was defined as the proportion of sites where OPM had not been recorded in the winter despite it being present in summer. ‘Apparent extinction (summer)’ provided an estimate of the probability that OPM will ‘naturally’ disappear from a site during the summer/autumn when control is not applied, e.g. due to predation or exposure to weather.
19 February 2016 148
Evaluation of the Oak Processionary Moth Control Programme - Final Report
We expected that apparent extinction in the summer would be low, since OPM is unlikely to completely fail at a site between the summer survey and the production of pupal nests. We found that the rate was low (i.e. of sites found with OPM in the summer, 80% of the have pupal nests recorded in the winter). Possible biases: ■ It includes instances of nest removal control which were unreported to the Control Programme (but these are probably negligible). ■ The imperfect detection of OPM in the winter could explain why apparent extinction was not even lower. If some sites actually had OPM but it was not detected (as suggested by the estimate of winter detection error) then the true rate of apparent natural mortality would have been even lower. ■ The number of sites under consideration was relatively small, thus causing relatively high uncertainty. ■ This will be affected by the types of sites which are surveyed in both summer and winter. These tend to be at the edges of the distribution, and so are likely to be sites in which OPM is less well established (i.e. lower prevalence). We could sub-divide sites into established (=’core’) and periphery, but the core was not surveyed in the winter, so clearly sample sizes will be limiting in producing the estimates. A9.1.2.4 Apparent extinction (winter) To estimate the apparent extinction (winter), we considered sites where OPM was known to have produced nests in the previous year (based on ‘infested’ in the winter survey or ‘previously infested’ the following summer), where spraying was not carried out and which were surveyed in the summer, what proportion had OPM recorded in the summer? ‘Apparent extinction – winter’ was defined as the proportion of sites where OPM will ‘naturally’ be lost from a site during the winter/spring when control is not applied. The persistence of OPM at a site is due to a combination of (1) adults from that site laying eggs at that site and which survive to produce caterpillars the following year; and (2) all adults emigrating and the site being colonised by adults from elsewhere. We expected that apparent extinction in the winter would be higher than the summer, because this represents so many more life stages (adult, egg and early instar caterpillar) and this is what we found. Estimates were conditioned on OPM being detected in the winter, so the imperfect detection of OPM in the winter does not bias this estimate. Possible biases: ■ It includes the effect of spraying events which were unreported to the Control Programme (but these are probably negligible). ■ This will be affected by the types of sites which are surveyed in both summer and winter. These tend to be at the edges of the distribution, and so are likely to be sites in which OPM is less well established. We could sub-divide sites into established (=’core’) and periphery, but the core was not surveyed in the winter, so clearly sample sizes will limit the confidence we have in the estimates. A9.1.2.5 Apparent extinction with nest removal (Effectiveness of control – nest removal) To estimate the apparent extinction of sites when nest removal was carried out, we considered sites where OPM was found in the summer, nest removal was carried out and when a survey was carried out the following winter. We considered the proportion of these sites where OPM was not detected in the winter survey. If a site was infested in the summer and still infested in the winter even after nest removal, then it suggests either (1) nest removal was not effective at eradication at that site, or (2) nest removal was effective and it had become re-infested. The latter is implicitly incorporated in the estimate of apparent extinction in the summer, so the difference between apparent natural extinction and effectiveness of nest removal gives the specific benefit of nest removal at eliminating OPM at the site level. We did not include information on ‘previously infested sites’ even though this was available
19 February 2016 149
Evaluation of the Oak Processionary Moth Control Programme - Final Report
because these data are presence only (we can be confident that ‘previously infested’ means that, but we have no information about ‘previously infested’ status of sites which were not reported as such). We expected that apparent extinction in sites with nest removal would be higher than in sites without nest removal and this was indeed the case: without nest removal 19% of sites appeared to become uninfested, whereas with nest removal 54% of sites become uninfested. There was a three-fold increase in the probability of apparent extinction of sites after nest removal. Possible biases: ■ The sites with information on winter surveys were a biased subset of all sites, which could influence the applicability of the estimates. ■ If some sites actually had OPM but it was not detected (as suggested by the estimate of winter detection error) then this would have inflated the estimate of apparent mortality with nest removal. These values should therefore be taken as maximum values. ■ We assume that ‘previously infested’ means that the old nests were from the previous year, and years prior to that. ■ In the dataset available, each site had only one status: ‘not infested’, ‘infested’, ‘previously infested’ or ‘not visited’. Therefore the subset of data used to estimate detection error in the winter was restricted to sites not currently infested with OPM, and so is biased towards sites with low density of OPM in the winter survey (because OPM had died out by or was at such low densities it could not be detected by the following summer). ■ Winter surveys were a focussed survey tool, and so focussed on the edge of the range, where OPM had only just colonised, or around pheromone traps, so for estimating global parameters, the sites represent a biased subset of sites. A9.1.2.6 Apparent extinction with spraying (Effectiveness of control – spraying) To estimate the apparent extinction of OPM at sites where spraying was carried out, we considered the sites where OPM had been surveyed for and found in the winter, where spraying was carried out and where a survey had taken place the following summer. We considered what proportion of these sites had OPM not detected in the following winter. If a site was infested in the winter and still infested in the summer even after spraying, then it suggests that (1) spraying was not effective at eradication at that site. The overall estimate implicitly includes the apparent extinction in the winter, so the difference between apparent natural extinction and effectiveness of spraying gives the specific benefit of spraying at eliminating OPM at the site level. We expected that apparent extinction in sites with spraying would be higher than in sites without spraying and this was indeed the case: without spraying 44% of sites appeared to become uninfested, whereas with spraying 50% of sites become uninfested. Spraying increased the probability of sites apparently becoming extinction by 1.12-fold. Of course, spraying may have had other benefits such as reducing the number of OPM nests, which are not considered here. Possible biases: ■ Estimates were conditioned on OPM being detected in the winter, and (as discussed above) detection in the winter could have been affected by the abundance of OPM. Therefore the assessment of spraying effectiveness would be biased towards sites with high OPM abundance, i.e. where spraying was less likely to be effective. In general, this reveals how important it is to be aware of underlying (and often not obvious) biases in the underlying data. ■ The sites with information on surveys not randomly selected, so it is likely that they were biased in some way (e.g. the focus on survey effort at the edge of the West London outbreak from summer 2014 onwards). This could influence the estimates. ■ Winter surveys were a focussed survey tool, and so focussed on the edge of the range, where OPM had only just colonised, or around pheromone traps, so for estimating global parameters, the sites represent a biased subset of sites.
19 February 2016 150
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A9.1.3 Limitations of the parameter estimation ■ Fundamentally, we are limited in being able to calculate parameters in a statistically rigorous way because the data were collected for operational requirements (to direct the application of control measures) rather than being part of a designed monitoring programme. ■ In all cases the parameters are estimated per site because the data were collected at the site level. We note that a ‘site’ was defined operationally (partly based on site ownership) rather than ecologically. ■ In all cases the parameters are estimated based on presence/absence records, not abundance of level of infestation. With the available resources and data, we are not able to incorporate effects of site size or abundance of OPM: i.e. control may have the benefit of reducing the abundance of OPM even if it does not eliminate it at a site level. ■ These estimates are limited by the availability of information. Parameters were calculated based on selected subsets of sites fitting the required criteria. We are therefore making best use of the data to estimate parameters for modelling. We note that the sites surveyed were often from a potentially biased subset of sets (especially towards the edge of the range, or being conditioned on a previous condition, such as OPM being detected, which might bias towards sites with high OPM abundance) which could reduce the generality of the estimates to all sites. Overall, these biases will interact in complicated ways to affect parameter estimates, making it difficult to determine exactly how they will be biased. ■ Parameters could be influenced by the details of the site: characteristics such as regions (west London and Croydon/Bromley), site context (surrounded by housing or parkland) or site size (number of oak trees). One of the site attributes which is likely to influence these parameters is site size (in terms of the number of oak trees) and although initial observation of the results (not shown) indicated this may be important, e.g. apparent natural extinction in winter and effectiveness of control appears to be higher in small sites. However the data were too few to be able to assess this further. ■ We have assumed that there is no site ‘memory’, i.e. the success of nest removal is unaffected by whether spraying had previously occurred. The occurrence of previous control measures could be associated with the success of current control. This could be biological causality (consecutive bouts of control weaken the population at the site) or it could be association (consecutive bouts of control are more likely at sites with a larger, or smaller, population size – directionality depending on various circumstances). ■ We assume that we have perfect knowledge of the control. Andrew Hoppitt and Julia Branson (pers. comm.) suggest that since the Control Programme started in 2013 there has been very little control carried out by individual landowners outside the Control Programme, except by Royal Parks, Network Rail and Transport for London (and these data are incorporated in the Control Programme dataset), although it is possible that control occurred at some sites even though this was not recorded.
19 February 2016 151
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table A9.1 Description of parameters and their attributes. Attribute Description Options Description Spraying Spraying of early instar caterpillars in April -1 Information that the site was before they are detected through surveys. sprayed . No information, so we presume the site was not sprayed Summer Summer surveys (June to July) of aggregations 1 Site was surveyed and of late instar caterpillars and pupal nests. These OPM was detected surveys were operationally designed to inform 0 Site was surveyed and control via nest removal. Sometimes old nests OPM was not detected were reported (see ‘Previous’ below). . No information, so presume site was not surveyed Nest Physical removal of the larval/pupal nests (July- -1 Information that nest removal August) before adults emerge removal was carried out . No information, so we presume nest removal was not carried out Winter Winter surveys (November-January) of nests 1 Site was surveyed and from which adults had emerged. These surveys OPM was detected* were operationally designed to inform control 0 Site was surveyed and via spraying. They were assigned to the year in OPM was not detected which they were begun. . No information, so presume site was not surveyed Previous During the summer survey, old nests were 1 Site was surveyed during sometimes observed and reported. Their the summer and old nests presence confirms the infestation by OPM in the were reported ** previous year, but the lack of a report does not . No information, so presume allow us to assume the absence of old nests. site was not surveyed, or site was surveyed and old nests were not reported.
* ‘Previously infested’ appeared to be used inconsistently during the winter survey (potentially for fresh nests from which adults have just emerged, or for old nests from the previous season). We have assumed that in all cases the record refers to recent infestations (i.e. emergence of adults in the previous 6 months). ** Although we have information on which sites were surveyed, we have no knowledge of the detection bias for old nests in the oak canopy and so cannot confidently attribute ‘0=not present’ to any of these sites. The site data were an aggregation of all records from that site, e.g. all infested trees and further site-level data (including ‘uninfested’ and ‘previously infested’), therefore sites with one infested tree could not also be recorded as previously infested.
19 February 2016 152
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table A9.2 Site histories for the selection of subsets of data for which to calculate the parameter values.
SPRAYING_YR1 SUMMER_YR1 NEST_RM_YR1 WINTER_YR1 PREVIOUS_YR1 SPRAYING_YR2 SUMMER_YR2 NEST_RM_YR2 WINTER_YR2 PREVIOUS_YR2 PARAMETER POSITION
? 1 NA 0 ? ? ? ? ? ? Natural Mortality - Summer Numerator ? 1 NA 0 ? ? ? ? ? ? Natural Mortality - Summer Denominator ? 1 NA 1 ? ? ? ? ? ? Natural Mortality - Summer Denominator
? ? ? 1 ? NA 0 ? ? ? Natural Mortality - Winter Numerator ? ? ? ? 1 NA 0 ? ? ? Natural Mortality - Winter Numerator ? ? ? 1 ? NA 0 ? ? ? Natural Mortality - Winter Denominator ? ? ? ? 1 NA 0 ? ? ? Natural Mortality - Winter Denominator ? ? ? 1 ? NA 1 ? ? ? Natural Mortality - Winter Denominator ? ? ? ? 1 NA 1 ? ? ? Natural Mortality - Winter Denominator
? 1 -1 0 ? ? ? ? ? ? Effectivness of control - Nest Numerator ? 1 -1 0 NA ? ? ? ? ? Effectivness of control - Nest Denominator ? 1 -1 1 ? ? ? ? ? ? Effectivness of control - Nest Denominator
? ? ? 1 ? -1 0 ? ? ? Effectivness of control - Spray Numerator ? ? ? ? 1 -1 0 ? ? ? Effectivness of control - Spray Numerator ? ? ? 1 ? -1 0 ? ? ? Effectivness of control - Spray Denominator ? ? ? ? 1 -1 0 ? ? ? Effectivness of control - Spray Denominator ? ? ? 1 ? -1 1 ? ? ? Effectivness of control - Spray Denominator ? ? ? ? 1 -1 1 ? ? ? Effectivness of control - Spray Denominator
? 0 NA 1 ? ? ? ? ? ? Detection error - Summer Numerator ? 0 NA 1 ? ? ? ? ? ? Detection error - Summer Denominator ? 0 NA 0 ? ? ? ? ? ? Detection error - Summer Denominator
? ? ? 0 1 ? ? ? ? ? Detection error - Winter Numerator ? ? ? 0 1 ? ? ? ? ? Detection error - Winter Denominator ? ? ? 1 1 ? ? ? ? ? Detection error - Winter Denominator This table provides a template detailing the selection of sites used to calculate each parameter. This is a general template, so “Yr1” represented 2013 or 2014 depending on the year of interest. The number of sites fitting each site history was calculated and then these totals were summed in order to calculate the numerator and denominator for each parameter. ? = any status SUMMER and WINTER could take the values 0; 1; NA, which represented sites surveyed and recorded as uninfested; sites surveyed and records as infested; and sites that were not surveyed, respectively. SPRAYING and NEST_RM could take the values: NA; -1 which represented no control recorded to be applied; and control applied
19 February 2016 153
Evaluation of the Oak Processionary Moth Control Programme - Final Report
PREVIOUS could only take the value 1 or NA, which indicated the site was surveyed in the summer and ‘old nests’ (i.e. at least one year old) were recorded, or the site was surveyed and the presence of ‘old nests’ was not recorded (i.e. they could have been present or not, because recording old nests was not a priority for surveyors).
19 February 2016 154
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table A9.3 The estimation of the parameters (summarised in Figs 3 and 4 in the main text). 95 percentiles were obtained by bootstrapping. Parameter Year Number of Value* 95 percentiles Value when sites (bootstrap) taking detection (numerator/ probability into denominator) account** Apparent extinction - 2013 0/5 0 0 0 summer Apparent extinction – 2014 7/32 0.22 0.09, 0.38 0.13 summer Apparent extinction - All 7/37 0.19 0.08, 0.32 0.11 summer Apparent extinction - winter 2013 16/31 0.51 0.35, 0.68 0.49 Apparent extinction – 2014 11/30 0.37 0.20, 0.53 0.34 winter Apparent extinction - winter All 27/61 0.44 0.33, 0.57 0.42 Detection error - summer 2013 2/210 0.01 0, 0.02 - Detection error – summer 2014 16/177 0.09 0.05, 0.14 - Detection error - summer All 18/387 0.05 0.03, 0.07 - Detection error - winter 2013 6/13 0.46 0.23, 0.77 - Detection error – winter 2014 4/25 0.16 0.04, 0.32 - Detection error - winter All 10/38 0.26 0.13, 0.40 - Apparent extinction - 2013 2/4 0.40 0.0, 1.0 0.26 summer with nest removal Apparent extinction - 2014 43/79 0.54 0.43, 0.66 0.39 summer with nest removal Apparent extinction - All 45/83 0.54 0.43, 0.65 0.38 summer with nest removal Apparent extinction – 2013 83/143 0.58 0.50, 0.66 0.55 winter with spraying Apparent extinction – 2014 33/87 0.38 0.28, 0.48 0.36 winter with spraying Apparent extinction – All 116/230 0.50 0.44, 0.57 0.48 winter with spraying * The variation in parameter values between years is discussed in the text (Section A1.1.2). ** The way in which the numerator and denominator for the parameters is adjusted is explained in the text (Section A1.1.1).
19 February 2016 155
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A9.1 The distribution of sites which were surveyed, OPM was recorded and which were controlled in each year. The data for summer 2015 are the data available up to the end of August 2015. These are presented diagrammatically for simplicity and to show the variation between winter and summer surveys in each year. The West London outbreak is left of centre and the Croydon outbreak is bottom right, for further spatial detail see Annex 11.
19 February 2016 156
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 157
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 158
Evaluation of the Oak Processionary Moth Control Programme - Final Report
19 February 2016 159
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A9.1.4 Distance-dependent rate of colonisation: Methods and results Colonisation was estimated in a different way to the other parameters above. We needed to estimate the probability of an uninfested site becoming infested due to dispersal from a source (infested) site. This pairwise site-site dispersal parameter will be influenced by distance and the final probability of becoming colonised based on this parameter when aggregated across all source sites. We restricted estimation of the dispersal parameter to data from Croydon/Bromley only because this site had the best and most comprehensive coverage of survey effort. We used a method known as inverse modelling to estimate the dispersal curve [NOTE: source to be added for final edition of the report]. The model assumes that all source sites are perfectly known (an assumption which we contend is reasonably well met in Croydon/Bromley due to the intensity of survey effort. ■ Source sites: We take the data from the winter surveys as the known source sites (i.e. adults were produced from these sites, whereas sites infested in the summer could have been subject to successful nest removal control measures). ■ Target sites: We take all sites surveyed the following summer which were previously uninfested (therefore unsprayed, so we excluded all sites within 100m of an infested site) as the target sites. These are the subset of sites which could become infested due to dispersal. ■ Dispersal kernel: we use an exponential distance decay curve to model effective dispersal. In the inverse modelling exercise, we found optimal values for a and b in the following dispersal function which assumes that the only factor affecting spread is the distance between sites:
s P_inftot = 1 - [ exp(a + b.dists)/(1+ exp(a + b.dists)) ] where s is the number of sites. We used an optimisation approach called simulated annealing to estimate a and b. This method provided an assessment of the variance in the estimates of a and b, although it did not provide an estimate of the way they co-varied. The estimated values for a and b differed between the two years (Table A9.4; Figure A9.2) and appeared to show relatively high uncertainty, which was due to the relatively small number of sites that could be included in the modelling. Although there is variation at short distances (<1km) between the two years, it can be seen that longer distance effective dispersal (i.e. colonisation of a focal site from a single donor site, which will be influenced by female dispersal, not male dispersal) is extremely unlikely. This supports what is known about the biology of OPM. The resulting curve for colonisation showed that site-to-site colonisation rate was relatively low (4-8% for adjacent pairs of sites; Figure A9.2a), however when aggregated to many sites then colonisation becomes much more likely (i.e. the probability of an uninfested site becoming infested is >85% if 100 infested sites lie 1km away; Figure A9.2b). In simulation models, colonisation was the summed probability of infestation across all infested sites at all distances.
A9.1.5 Limitations of the estimate of colonisation ■ This modelled was undertaken without considering other parameters than distance. As noted below (Annex A9.2) a unified approach to simultaneously estimate all parameters would be likely to reduce the uncertainty in colonisation and the other parameters. ■ We had to assume that the source sites are perfectly known, but in general, the Croydon/Bromley region was well-surveyed in the summer and infested sites were re-visited in the winter. ■ The modelling is based on the number (and location) of sites which become infested, but there were only 14 and 17 sites which become infested in 2013-14 and 2014-15, respectively, thus limiting the potential precision of the estimates. ■ We assumed that all infested sites have an equal contribution to causing infestation (once distance is taken into account), whereas in reality sites with a greater abundance of OPM would have a greater contribution. There are other reasons that sites could have varied in their contribution to colonisation probability, e.g. local management, the intervening habitat, and so on.
19 February 2016 160
Evaluation of the Oak Processionary Moth Control Programme - Final Report
■ The values of a and b are likely to covary, but the method did not provide an estimate of their covariance, only their point estimates and variance. We believe that further use of the approach of inverse modelling will only have a limited impact on improving the estimates of colonisation because it depends on data availability, and is limited by needing to know the distribution of all sources. An integrated approach to parameter estimation by estimating all parameters simultaneously from all the data could provide better estimates of colonisation. However, field-based studies of colonisation (e.g. genotyping populations across years) will also be valuable in gaining improved estimates of colonisation. Table A9.4 Estimated parameters of distant-dependent colonisation
Years a (SE) 95% b (SE) 95% Number of Number of Number of confidence confidence sites sites new intervals of a intervals of b original originally infestations infested uninfested 2013-14 3.89 3.42, 4.46 0.60 0.33, 0.95 6 362 14 (0.74) (0.42) 2014-15 2.41 1.96, 2.94 1.39 1.01, 1.90 12 128 17 (0.65) (0.59)
Figure A9.2 Outputs from the model showing the probability of becoming infested from (a) a single tree at each distance and (b) 100 trees at each distance. Both cases are used simply to exemplify the outputs of the model – simulation models used the actual distrbution of source sites. They do not include variance of the parameter estimates (Table A9.3) because we did not obtain estimates of their co-variance, which would be neccesary for plotting confidence limits on the curves.
19 February 2016 161
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A9.2 Running the simulation model
A9.2.1 Methods The model we developed was a simple, stochastic simulation model. The observed records from 2012 were used as the starting position of the model. The model was then run in a stepwise fashion, stochastically implementing survival and colonisation at each time step, e.g. since the calculated survival rate of sites with nest removal was 0.38, for an occupied site where nest removal occurred we randomly chose for it to persist (with probability 38%) or go extinct (with probability 62%). The model was run with the parameter values as calculated for the overall dataset (i.e. 2013 and 2014 combined) except for distance-dependent colonisation, for which we used the estimates for 2014 alone (because it is not trivial to aggregate the curves for the years separately). An additional parameter was the probability of a site having control being applied. We obtained these values from the data for 2014 (i.e. summer 2014 and winter 2014/15). We applied this value across years. West London and Croydon were treated separately (Table A9.5), so the recorded rate of control for each region was applied to that region. When applying control, the simple options were to apply control to a fixed number or a fixed proportion of sites. Each approach has advantages and disadvantages. We chose to apply control to a fixed proportion of sites because this seemed closer to the action of the control programme. In order to select sites to have control applied we counted the set of potentially controlled sites, multiplied this by the fixed proportion to give the target number of sites to be controlled. We randomly selected this number of sites from the potential sites. The potential sites were all infested sites in the model (for nest removal), or all the infested sites plus all sites within 100m of an infested site (for spraying). In applying control in this way, we did not take account of site history (length of time OPM had been present or previous application of control) or site context (relative to other infested sites). Also we did not take account of the actual constraints on the OPM control programme, i.e. budget and time for a limited number of surveyors to undertake the control. The model was run from the recorded distribution in 2012 for 2 years, so the spread was recorded from 2012 to 2013 and from 2013 to 2014. The model was run 20 times. The rate of spread was calculated in the same way as for the observed rate of spread with the alpha hull approach (see Annex A12.2 for full details). In brief, we applied an alpha hull to the point distribution (with an alpha of 2.5km) which provides an outline of the distribution. We then randomly located 1000 points on the perimeter of the alpha hull and calculated the distance of these points to the nearest point of the alpha hull around the previous year’s distribution. These distances were averaged to give the median rate of spread. The results were averaged for the whole distribution, and for separate regions within the distribution (Figure A12.1 and Figure A12.2). Where OPM was modelled to retreat, then spread was negative.
A9.2.2 Results The results reveal that this simple simulation model produced results that were fairly consistent across simulations, at least across the whole distribution (Figure A9.3), with a median rate of spread of about 1.3km in 2012-13 and 0.5km in 2013-14. However, there is marked difference in the rate of spread in different parts of the modelled distribution of OPM. Firstly, considering the difference between West London and Croydon: the model produced results in West London which were close to the observed spread in 2012-13, but predicted lower than observed spread for in 2013-14. This appeared to be because the rate of spread in 2013-14 was limited by the known distribution of potential host sites: i.e. the modelled spread of OPM was constrained by knowledge of host distribution by 2013. In contrast the model vastly over-estimated the spread of OPM in Croydon in 2012-13 but, as in West London, it had practically reached the edge of the distribution of known potential sites by 2013 which is why the predicted rate of spread in 2013-14 was much lower than that predicted in 2012-13. Further distinction can be seen when considering the northern and southern parts of the distribution in West London: the southern part spreads at a higher rate than the northern part. The northern part even shows an overall retraction in the distribution in some of the simulations.
19 February 2016 162
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Although the overall summary of the simulation model, for West London at least, is a reasonable fit to the observed distribution, the pattern of that spread (spread in the south of West London, but little spread or even retraction in the north of West London, and initially rapid spread in Croydon) does not match the observed pattern of spread (Figure A9.1). Therefore, we concluded that the simulation model does not adequately describe, the observed spread of OPM. We suggest three reasons for this lack of fit. Firstly, that the parameter estimates were incorrect (either the estimates are wrong or there were important covariates that should have been included). In particular the sensitivity analysis (Annex A9.3) indicates that distance-dependent colonisation was a very influential parameter, but it was difficult to estimate with the limited data available. Secondly, the model was limited (modelling site- level presence of OPM, rather than abundance at the tree-level) because more informative data to parameterise and test the model were not available. Thirdly, the information on potential sites was limited to the area surveyed and the definition of the sites. We noted anecdotally that in the models OPM presence tended to cluster in the regions of high site density, and so these regions (e.g. in Croydon and the south east of the West London outbreak) were especially influential on the persistence and spread of OPM. We believe that this is an artefact of the combination of the site distribution data and the model parameters, particular colonisation. We also noted anecdotally that for many parts of the distribution and for many simulations the model predicted that OPM would quickly reach the edge of the known distribution of potential host patches, so further information on the distribution of oak trees in the region is required to realistically model the spread of OPM. Although we apply control in the simulation model in a way that aimed to reflect the control programme, this could be modelled more thoroughly, e.g. by considering how site history and resources available influence the application of control across sites. Overall, although this model did not adequately match the observed distribution spread, we were able to conduct a sensitivity analyses in order to inform what additional data would best help the model perform better. This helps to provide clear recommendations on how to focus future research effort. Ideally future modelling would be undertaken at a scale that is more ecologically meaningful (abundance per tree rather than presence per site), with parameters that are estimated with greater rigour and with a more comprehensive distribution of oak trees.
A9.2.3 Known limitations of the model In addition to the limitations of the parameter estimates, as described in the preceding sections, there were additional limitations of the simulation model. ■ At each time step we applied control randomly by choosing sites which were infested. We did not consider site history or location. We know that there are particular criteria which are applied by staff involved in the control programme as to where to apply control and some of these criteria could be implemented in the simulation model. ■ The model does not consider site size (i.e. number of oak trees, because data were too sparse to model this adequately with the current approach) or level of infestation by OPM (because this information is not collected due to the operation reasons for data collection). ■ In simulation models the probability of colonisation was obtained from the summed probability of infestation across all infested sites at all distances. This assumes potentially unlimited dispersal distances for an individual and potentially an unlimited number of dispersers from an occupied site. Neither of these assumptions are true and a threshold for dispersal distance could be applied to limit dispersal while individual-based models may be necessary to avoid the assumption of an unlimited number of dispersers. We used the overall estimates of parameters, apart from colonisation, where we used the estimate for 2014-15 as based on the data in the Croydon region. Therefore, we have assumed that site-to- site colonisation was the same in Croydon as elsewhere in the OPM distribution although there could have been substantial differences between regions, e.g. based on the wider landscape characteristics
19 February 2016 163
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table A9.5 The proportion of sites for which control was known to be applied during the OPMCP, separated by region. The values for 2014 were used in the simulation model. * The proportion of sites with control was calculated based on either the sites known to be infested (for nest removal) or the sites known to be infested plus the sites within 100m of an infested site.
Proportion of sites with control* Year Type of control West London Croydon 2013 Spraying 0.89 0.67 Nest removal 0.74 0.97 2014 Spraying 0.64 0.64 Nest removal 0.56 0.93
Figure A9.3 The results of the simulation model, showing the results for 20 simulations as compared to the observed results (displayed as circled crosses) shown for West London and Croydon
Shown are the median rate of spread of OPM in 2012-13 (white bars) and 2013-14 (grey bars), the interquartile range for the simulations (error bars) and the individual simulation results (open circles). The results are shown for the whole range (i.e. equivalent to the Figs. A4.1-3), for West London and Croydon, and for the northern and southern parts of the West London distribution (see Fig. A1.1). A9.3 Sensitivity analysis Despite some lack of confidence in the model, we used the model to undertake a sensitivity analysis of the model to variation in the parameters. We assessed this individually for each parameter. For demographic parameters (summer survival, summer survival with nest removal and winter survival, winter survival with spraying) we used the calculated parameter values 0.1, which in many cases spanned the 95% confidence intervals of the estimates (obtained by bootstrapping). For colonisation
19 February 2016 164
Evaluation of the Oak Processionary Moth Control Programme - Final Report
we used the values of a and b and their 95% confidence intervals, obtained by simulated annealing in the inverse modelling. For the proportion of infested sites being controlled we independently varied the proportion of sites with nest removal and with spraying, and so ran every combination of each form of control taking one of the following values: 0.1, 0.3, 0.5, 0.7, 0.9. We estimated the sensitivity of the results to varying the parameter by expressing it as the percentage change in spread compared to the median across all simulations while varying each parameter. We ran 20 simulations for each parameter value, except for dispersal where we ran 10 simulations (because the increased number of sites increased runtime of the model) and calculated the median and the inter-quartile range of the distribution of individual simulations for each choice of parameter value. As with the initial results (Annnex A9.2.2), we presented the results as the overall spread of OPM. We calculated this with an alpha hull approach. We fitted an alpha hull (alpha = 2.5km) around the distribution of infested sites from the model. Then we randomly placed points on the perimeter of the alpha hull and calculated their distance to the nearest point of the previous year’s alpha hull. These distances were averaged (by calculating their median) to provide an assessment of the overall rate of spread (see Annex A9.1). Each simulation was compared to the median across all simulations where a parameter was being varied. This was expressed as the percentage difference from the overall median spread (about 1.1km), so a relative spread of < -100%A9.1 represents a range contraction.
A9.3.1 Results The results of individual iterations of the simulation model tended to show quite high variation (change in the rate of spread was typically 20-30 percentage points in each combination of parameters, represented by the spread of white points compared to the grey bars in Figs A4.1-3). However the overall variation across the different parameter combinations was relatively small (the median values, represented by the grey bars, varied by 5-10% in each of the plots in Figs A4.2-3). The exception to this was distance-dependent colonisation in which varying the values of a and b across their 95% confidence estimates (Appendix A2) caused variation the change in spread to be 70% compared to the overall median. In these simulations we varied a and b independently. In reality they co-vary but because an estimate of their covariance was not obtained (Section A1.1.5) we could not take that into account. Overall the variation in the rate of spread in Croydon was lower and less variable than in West London (but results not shown, due to lack of confidence in the fit of the model; Appendix A3). This sensitivity analysis could also serve as an indication of the effects of varying the application of control. However, even with very high rates of control, no simulation showed range retraction (i.e. change in spread compared to the overall median was always > -50%, where < -100% is a retraction). However, we know that the model does not adequately represent the observed distribution and patterns of spread (Appendix A3), therefore this result should not be interpreted as evidence that it is impossible to cause range contraction in OPM. Instead, it demonstrates how a better model could be used to support future decision-making.
19 February 2016 165
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A9.4 The sensitivity of the simulation model to variation of the parameters for survival in summer (i.e. between the summer survey and the following winter survey) and the winter (i.e. between the winter survey and the following summer survey).
Zero indicates that the rate of spread is that same as the median overall all simulations in the plot. A negative and a positive figure indicates that the rate of spread is, respectively, less than and more than the median. For instance, reducing winter survival from 0.58 to 0.48 causes an average reduction in the rate of spread (the grey bar is a negative value), but that the results from the individual runs of the simulation (open circles) are very variable, with rates of spread between the rate of spread being 20% greater to 40% less than the overall average. The range of values used are the calculated proportion 0.1. The grey bars show the median, error bars show the inter-quartile range and open circles show the results of the 20 individual runs of the simulation model.
19 February 2016 166
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A9.5 The sensitivity of the simulation model to variation of the parameters for the proportion of infested sites that were controlled.
The grey bars show the median, error bars show the inter-quartile range and open circles show the results of the 20 individual runs of the simulation model.
19 February 2016 167
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A9.6 The sensitivity of the simulation model to variation in the two parameters for distance- dependent colonisation showing the deviation from the overall mean.
The grey bars show the median, error bars show the inter-quartile range and open circles show the results of the 10 individual runs of the simulation model. The parameters values were the mean and 95% confidence intervals of the estimates from the results of the inverse modelling of colonisation probability. A9.4 Available information on distribution of oak trees. Beyond the sites surveyed in the OPM control programme, there is a very limited range of sources of information for oak tree distribution. Tree preservation orders are one potential source of information on the distribution of oak trees although they will only be relevant in built-up regions. We contacted many of the local councils at the edge of the current OPM distribution, but few had information available in an electronic format (Table A9.6). The Natural England GIS layers of woodland and the Forestry Commission woodland surveys may provide other sources of information on the distribution of wooded land parcels, although information on the presence of oak trees may be limited. Table A9.6 Availability of information on the distribution of oak trees with tree preservation orders.
Region Data on tree preservation orders in electronic format and potentially available to be shared? Brent Borough Council No Harrow Borough Council No Hillingdon Borough Council No Hounslow Borough Council Yes Spelthorne District Council Yes Richmond upon Thames Borough Council No Elmbridge District Council Yes Kingston upon Thames Borough Council No Wandsworth Borough Council No Merton Borough Council No Croydon District Council No Bromley District Council Yes
19 February 2016 168
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A9.5 Recommendations on extensions to the modelling Priority areas for any further investment in OPM modelling are: ■ Integrated estimation of model parameters. The size of the best subsets to individually estimate each parameter in each year was limited. These estimates cannot be improved upon using the current approach with the current data. A more integrated and thorough approach would be to simultaneously estimate these parameters from all the data available, e.g. with Bayesian approaches. ■ Including covariates in the estimation of parameters. Covariates that should be considered because they are already available in the database are (i) spatial position (relative to other infestations) and (ii) size of site (number of oak trees). The size of infestation (number of trees with OPM, and abundance of OPM) is an especially important covariate but this information is not currently collected by surveyors in a systematic way. ■ Constructing simulation models which include the spatial patterns of current control measures, such as site history and site location. ■ Investigate the potential to map host trees within and beyond the current distribution of OPM, because knowledge of the distribution of potential hosts of OPM is a severe limitation to using modelling to predict future spread of OPM. Although crowd-sourcing this task could work well in urban areas (via citizen science reporting oak trees), remote sensing might be more tractable for larger regions. ■ Investigate the potential of a focussed study of dispersal. The models are highly sensitive to the estimate of dispersal. A genetic approach is likely to be the best approach to achieve this. It will be much better at quantifying short to medium range dispersal rates (e.g. < 2km) and rare long-distance dispersal events, but with nest removals samples should be easy to obtain.
19 February 2016 169
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 10 Landowner survey screening questions
A10.1 Q1: What type of site/property are you responsible for/what types of sites/properties are you responsible for? Almost half of single-site responses represented residential gardens or private allotments. For the multi-site survey, respondents were predominantly responsible for highways, railways and roads (15/21 responses), followed by parks and playing fields (10/21 responses) and woodland/nature reserves (10/21 responses). Figure A10.1 Q1: What type of site/property are you responsible for? (N=100, single site survey)
Residential garden / private allotment 44
Woodland / nature reserve 5
Park / playing field / public garden 14
Golf course / business park 7
Stables / farmland 2
School / nursery 12
Cemetery 1
Hospitals / health facilities 7
Other 8
A10.2 Q2: Who owns the site/property you’re responsible for/who owns the sites/properties that you’re responsible for? Almost half of single-site respondents indicated that the site concerned was owned by a private homeowner/landlord (42/100 respondents) followed by a public body (34/100 respondents). For the multi-site survey, over half (12/21) respondents indicated the sites they were responsible for were owned by a public body. Figure A10.2 Q2. Who owns the site/property you’re responsible for? (N=100, single site survey)
12 Private homeowner / landlord 12 42 Public body
Charity / NGO
Private or corporate entity
34
19 February 2016 170
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A10.3 Q3: How many oak trees are there on site/How many sites are you responsible for where there are oak trees present? Almost half (49/50) of single-site respondents indicated that the number of oak trees on site was low (0-5). A further 22 respondents indicated that 6-50 trees were located on site. For the multi-site survey, respondents were asked a slightly different question – how many sites they were responsible for where there were oak trees present. In this case, the greatest number of respondents (8/20) pointed to being responsible for 51+ sites. Figure A10.3 Q3: How many oak trees are there on site? (N=100, single site survey)
4 13
0-5 6-50 12 49 51-500 500+ Don't know / Not applicable
22
A10.4 Q4: What is the history of OPM on the site/what is the history of OPM on the sites that you’re responsible for? For the single-site survey, a substantial number of respondents (44/100) indicated that OPM had been found on the site for the first time in 2015. A further 34 indicated that OPM had been found in the past, including in 2015. For the multi-site survey, a substantial number of respondents (16/21) indicated that OPM had been found in the past, including in 2015. Figure A10.4 Q4: What is the history of OPM on the site? (N=100, single site survey)
1 2 Never found 17
Found in past, not 2015
44 Found in past, including 2015 Found 2015, 1st time
34 None / not applicable
Q5: What OPM-related surveys and control measures have been applied on the site/sites you’re responsible for? For the single-site survey, 76/100 respondents indicated that surveys had been undertaken, closely followed by manual nest removal (68/100) and pesticide spraying (49/100). A similar pattern was seen in the multi-site survey, with (18/21) respondents having had survey measures undertaken, a similar
19 February 2016 171
Evaluation of the Oak Processionary Moth Control Programme - Final Report
number reporting manual nest removal as being undertaken (17/21) and a further (13/21) reporting pesticide spraying as being undertaken on the sites. Figure A10.5 What OPM-related surveys and control measures have been applied on the site? (N=100, single site)
1 Surveys
17 18 Chemical spraying
Manual nest removal Tree felling
13
A10.5 Q6: Who has paid for these control measures to be carried out? Over half (54/100) of single-site respondents indicated that the Forestry Commission had always paid for control measures to be carried out, with a further 20 indicating that they were now meeting these costs themselves. For the multi-site survey, 6/21 respondents indicated that they were meeting costs previously met by the FC, with 5/21 indicating that costs had always been borne by the FC. Figure A10.6 Who has paid for these control measures to be carried out? (N=100, single site survey)
4 Always FC 16 Previously me / landowner, now FC Previously FC, now me / landowner 54 Always me / landowner 20
None / not applicable
1
A10.6 Q7: Have you heard of the OPM control programme before? Single-site survey respondents pointed to a good level of awareness of the programme and its goals (63/100). Some 17/100 knew of the programme but didn’t know much about it, whilst 20/100 had reportedly never heard of the control programme. All multi-site respondents had heard of the programme and an overwhelming number (19/21) claimed to have knowledge of its objectives.
19 February 2016 172
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A10.7 Have you heard of the OPMCP before? (N=100)
Total
20
Yes, know about it and its goals Yes, but don't know 17 much 63 No, never
19 February 2016 173
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 11 Annual observed extent of OPM
19 February 2016 174
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A11.1 Change in annual extent of OPM (2006 – 2015)
19 February 2016 175
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A11.2 2013 survey results
19 February 2016 176
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A11.3 2014 survey results
19 February 2016 177
Evaluation of the Oak Processionary Moth Control Programme - Final Report
A11.4 2015 survey results
19 February 2016 178
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 12 Calculating the OPM rate of spread (2012 – 2015)
Three different methods were used to estimate the rate of OPM spread under the OPMCP. A12.1 The convex hull method The convex hull method for calculating the spread of OPM is based upon the number of 1km squares in which OPM was recorded. For each year, the assumption is that all 1km squares that were previously infested, remain infested. Then any new squares are added to this and the total area is converted to a radius and the increased length of the radius compared to previous years is taken as the rate of spread. The convex hull around points is the smallest shape including all the points where all of the angles are convex (i.e. not less than 90). By analogy, a rubber band stretched around all the points forms a convex hull. When producing a convex hull around two sets of points (e.g. south-west London and Croydon) the user needs to decide which points are associated with which hull.
A12.1.1 Increase in geographical extent using the convex hull method The increase in annual geographic extent was calculated based on the estimated total area (km2) where OPM was known to be present (see below). The percentage increase for each year was then calculated compared to the previous year’s known extent. Table A12.1 Calculated rates of OPM spread
Year of infestation Known area of OPM infestation (total km2) 2012 187.37 2013 219.19 2014 328.33 2015 469.46
A12.2 The alpha hull method We defined the distribution of OPM each year by using alpha hulls (where alpha = 2.5km) and from these estimated the observed rate of spread. The results of the alpha hull method are presented diagrammatically in Figure A12.3 below. The alpha hull method provides a more accurate description of complex shapes than the convex hull. The alpha hull (strictly: the negative alpha hull; Edelsbrunner et al. 1983. IEEE Trans. Inform. Theory, IT-29, 551–559) is the shape remaining when the circles with radius of alpha than do not include points are removed. By analogy, using a ‘cookie cutter’ to cut excess surface from the point distribution will produce an alpha hull. The alpha hull therefore tends to have a scalloped edge including concave regions, and it can produce holes inside polygons and islands separate from each other. It will produce a more accurate description of the point distribution than the convex hull and does not rely on the user to specify separate regions in the distribution. However, there is no ‘correct’ value of alpha; larger values of alpha approximate to the convex hull, whereas smaller values are produce a more sinuous perimeter. For this analysis we used an alpha of 2.5km, and any holes in distribution were filled. We used the available data up to 31st Dec 2015. The linear rate of spread was calculated as the median distance of points on the perimeter of the distribution to the nearest part of the distribution in the previous year. In order to estimate the linear rate of spread of OPM, we placed 1000 points randomly on the perimeter of the alpha hull and calculated the distance to the nearest point of the perimeter of the previous year’s alpha hull. Any points falling inside the previous year’s alpha hull were assigned a negative distance (representing a range retreat). The range of spread was calculated as the median of these values.
19 February 2016 179
Evaluation of the Oak Processionary Moth Control Programme - Final Report
For 2015 there are relatively few records for OPM in the north west of its distribution and from the centre of the south-west London infestation. When applying the alpha hull approach, this resulted in an artefact of a ‘cavity’ in the OPM distribution. Therefore we linked the two closest edges of the distribution at the mouth of the cavity in order to manually fill this distribution.
A12.2.1 Results This is a repeatable and standardised method for describing the extent of a point distribution and quantifying the rate of spread. The alpha hull method allows for fragments and holes in the distribution (with a suitable value of alpha) making it suitable for any similar distribution. Only two user inputs were required for this process: the choice of the value of alpha and the manual filling of the artefactual cavity in the distribution in 2015. Although varying the value of alpha would have impacted upon the shape and area of the distribution (with smaller values resulting in a more convoluted perimeter and a more fragmented range, and larger values tending towards a minimum convex hull), we are confident that any reasonable value of alpha (e.g. 1-3km) would have resulted in a similar rate of spread. More thorough surveying in 2015 would have reduced the likelihood of there being artefacts in the distribution, but as the distribution of OPM increases and survey effort is likely to become less comprehensive this may become an increasing problem. In the main text we report upon the distribution of OPM if it had spread at a rate of 5 or 10km yr-1 since 2012, but other values were also considered showing how even small increases in the linear rate of spread (e.g. from 1 to 1.5km yr-1) result in large differences in the number of people living within the area of infestation (an increase of 0.5 million people). However, this is a simple prediction because it assumes that OPM spreads at an equal rate in all directions and does not take jumps in spread and new outbreaks into consideration. From 2012 to 2015 the overall area of infestation (including Olympic Park and Guildford as well as south-west London and Croydon/Bromley) has risen from 136 to 412 km2. The linear rate of spread of OPM (the increase in the spatial spread of infestations) from 2012 to 2015 has been increasing from 920 to 2350m per year in South-west London and has varied between 270 and 710m per year in Croydon/Bromley. Table A12.2 Summary of OPM distribution, number of people living within the area of infestation and its annual rate of spread. These results have been obtained using a standardised, repeatable method of using alpha hulls (with alpha = 2.5km) to describe the distribution. Area covered, (km2) Human population within area Rate of spread, (thousands) (km y-1) West Croydon/ All West Croydon/ All West Croydon/ London Bromley London Bromley London Bromley only only only 2006 0.4 0.4 1 1 2007 5 5 8 8 1.0 2008 17 17 47 47 0.8 2009 18 18 58 58 0.3 2010 47 46 137 137 2.1 2011 100 101 317 317 1.4 2012 133 3 136 518 3 522 0.9 2013 177 5 182 650 9 658 0.9 0.3 2014 272 11 282 1024 25 1049 1.6 0.7 2015 388 14 412 1777 37 1815 2.4 0.3
19 February 2016 180
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Table A12.3 The area of infestation and number of people living within this area for different scenarios of OPM spreading at a constant rate since 2012 (see maps below).
Buffer distance (km) Area covered (km2) Human population Start (observed in 2012) 0 136 522,000 Observed in 2015 n/a 348 1,432,000 1km y-1 3 390 1,519,000 1.5km y-1 4.5 553 2,108,000 2.5km y-1 7.5 947 3,402,000 5km y-1 15 2054 5,267,000 10km y-1 30 5272 7,199,000 Figure A12.1 The alpha hull (where alpha = 2.5 km) of the distribution of sites where OPM was found to be present in 2015.
The hatched region shows the artefactual ‘cavity’ that was filled manually in order to undertake analysis. Also shown is the categorisation of the distribution into regions, as referred to elsewhere in the report. The separation of West London from Croydon was at the easting 534500 and the separation of West London into a northern and southern half was at the northing 174000. The grey dots at the background shows every site that had been surveyed during the winter or summer surveys up to the end of August 2015.
19 February 2016 181
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A12.2 Alpha hulls (where alpha = 2.5km) of the distribution of records of OPM from 2006 to 2015. Shown are the distributions for summer and winter surveys begun in each year; records for 2015 show records up to the end of 2015. The grey background points indicate the location of all sites which had been surveyed up to the end of August 2015. The hatched area for 2015 is an artefactual cavity in the alpha hull which was manually prior to analysis.
19 February 2016 182
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Figure A12.3 Calculated extent of OPM using the convex hull method
19 February 2016 183
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 13 Number of infested trees
The below sections provide details of the number of infested sites, trees and nests found between 2012 and 2015 for the different sites of infestation, including London, Croydon/Bromley, the Olympic site and Pangbourne. It should be noted that: ■ The extent of the survey effort changed from year-to-year, therefore some of the apparent change in infestation is likely to be related to the fact that a larger area was surveyed. ■ The core area was not systematically surveyed in 2015 and therefore the infestation figures for 2015 are likely to be underestimated. ■ Reports on the number of trees / sites surveyed does not include sites that were visited but which the surveyor could not obtain access to ■ The number of nests reported by surveyors are likely to be under-estimates. Typically, once a surveyor has found a nest they know the tree is infested so will move onto the next tree without spending a lot of time counting nests, whilst the contractor will count the number of nests that they found when they had climbed the tree. A13.1 Infestation by area Table A13.1 Infested sites*
Area 2011 2012 2013 2014 2015 Bromley / 32 35 90 72 Croydon London 2 127 174 285 262 Olympic Park 1 5 Pangbourne 1 1 Guildford 40 Bromley / 32 35 90 72 Croydon Total sites 3 160 209 376 379
* Includes 3 Lewisham sites in 2014 and 2 in 2015 Table A13.2 Infested trees*
Area 2011 2012 2013 2014 2015 Bromley / 240 667 349 289 Croydon London 2 4197 1991 4093 3376 Olympic Park 1 8 Pangbourne 59 8 Guildford 110 Total sites 61 4445 2658 4443 3783 * Includes 3 Lewisham sites in 2014 and 2 in 2015 Table A13.3 Infested nests*
Area 2011 2012 2013 2014 2015 Bromley / 236 431 451 Croydon
19 February 2016 184
Evaluation of the Oak Processionary Moth Control Programme - Final Report
London 2 47 4847 13185 13889 Olympic Park 2 8 Pangbourne 65 9 Guildford 124 Total sites 67 56 5083 13618 14472
* Includes 3 Lewisham sites in 2014 and 2 in 2015 A13.2 Infestation by core* Table A13.4 Infested sites
Area 2011 2012 2013 2014 2015 Buffer 1 53 75 198 354 2015 core 2 108 136 179 28 Total 3 161 211 377 382 * 6 sites cross the core boundary and are therefore double counted Table A13.5 Infested trees*
Area 2011 2012 2013 2014 2015 Buffer 59 441 760 1001 1825 2015 core 2 4004 1898 3442 1958 Total 61 4445 2658 4443 3783
Richmond Park 1026 1759 1522 Bushey Park 69 263 389 Total 1095 2022 1911 Grant total ex. 1563 2421 1872 Richmond Park and Bushey Park * Richmond Park and Bushey Park are in the core Table A13.6 Nests*
Area 2011 2012 2013 2014 2015 Buffer 65 9 364 1516 2883 2015 core 2 47 4719 12102 11589 Total 67 56 5083 13618 14472
Richmond Park 3534 7898 10237 Bushey Park 202 728 1233 Total 3736 8626 11470 Grant total ex. 1347 4992 3002 Richmond Park and Bushey Park * Richmond Park and Bushey Park are in the core
19 February 2016 185
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 14 Additional evidence on OPM health impacts
The first known occurrence of health impacts resulting from OPM in the UK was recorded in 2006 and involved residents of an apartment building near an OPM-infested tree (Public Health England, 2015a). Other incidents in the UK have involved tree workers treating OPM infestations. In some cases these were severe enough to cause cessation of work (DEFRA, 2014). Eleven incidents were reported to the Forestry Commission from June to July 2014, and 17 from March to August 2015 (though one of the latter was a pre-emptive warning rather than a reaction). Several of these incidents affected multiple people. In 2014, for example, all the residents in a block of eight flats appeared to have suffered health issues ranging from rashes to an increase in asthma attacks, and multiple children in at least four sites were hospitalized or affected by rashes. In 2015, incidents affected multiple children on at least two occasions. Most of the remaining incidents in both 2014 and 2015 appeared to involve reactions that included rashes and chest infections, some of them severe. Others occurred multiple times and/or had been persistent over several weeks. They affected people who had been in close proximity to trees, such children playing in parks, and arboricultural workers. In one case, a horse was also affected. A number of incidents in both years occurred in Richmond-upon-Thames. There are records of problems in Hounslow in 2014 but not 2015. In 2015 incidents were distributed also evenly between core and control areas, with a few incidents in each of Lambeth, Westminster, Bromley, and Wandsworth. In areas of Europe colonised by the OPM, thousands of people have been affected (Townsend, 2009). OPM is considered to be a public health threat in some European countries, examples being the Netherlands and Germany (London Tree Officers Association, 2010). In the Netherlands, approximately 4.3% of the population in areas with higher oak abundance experience health problems (Moran et al., 2015). In Kleve, Germany, out of 740 patients with OPM-related symptoms, 86% suffered from skin rashes and 4.8% from respiratory problems; 0.8% were hospitalised, and 0.2% went into allergic shock (Klug, 2015). Boyd et al. (2013) suggest that the human health implications of the OPM are of greater concern than the ecological implications. Several severe cases of OPM-related health problems have occurred in Germany. Even six years after health impacts were first reported, workers were still suffering reactions, in one case at least enough to require hospitalization. At another site, health impacts forced all the staff of a wildlife park to seek treatment and almost caused the park’s closure (Forestry Commission, 2011). OPM-infested trees resulted in health impacts in 10 of 24 children attending a neighbouring kindergarten (Gottschling et al., 2007). In another case, 42 out of 90 people at a rest area, including several children who had been playing with OPM caterpillars, required treatment (Gottschling and Meyer, 2006). Lastly, an outbreak of health impacts from OPM affected 150 soldiers in a military community in Heidelberg in 1995 (Fenk, Vogel, and Horvath, 2007). Other cases have involved roadside workers (Townsend, 2009); forestry workers in Belgium (Crevecoeur, 2015). Maier et al. (2003) report an incident in the suburbs of Vienna in which 57 of 1025 people surveyed (5.6%) reported one or more symptoms. Public Health England (2015a) provides some evidence on the proportion of people in an area where OPM is presence who can be expected to be affected by OPM. A survey taken at the first recorded incident in England found 47 of 69 survey respondents (68%) experiencing some symptoms, representing 63% of households in the area. The Vienna suburb study cited above targeted all households and institutions within 500 metres of an infected tree, and was one of two studies to investigate incidence rates from general (rather than acute) exposure. The second study tracked an epidemic in southern Holland and Belgium in which GPs were treating dozens of patients a day at its peak, mostly associated with woodland recreational activities. 6% and 7.5% of respondents reported OPM-related complaints in 1997 and 1998 respectively, although 7% of these may have been inaccurately attributed. Both surveys were conducted after a public information campaign.
19 February 2016 186
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 15 List of Advisory Group Members
Name Organisation Sue Ireland Head of Open Spaces, City of London Andrew Hoppit FC OPM Manager Craig Harrison FC London Manager Charlton Clark FC Communications Officer Stewart Snape FC GB Plant Health Colin Buttery Royal Parks Dave Lofthouse LB Merton / London Tree Officers Association Mike Robinson FERA Mark Parsons Butterfly Conservation (also representing RSPB and Buglife) Adam Wallace Natural England Andrew Jones Greater London Authority Craig Ruddick Richmond Borough Council Barry Walsh Public Health England Emer O’Connell Public Health England Mark Townsend Gristwood & Toms John Deakin SE Regional Advisory Committee Richard Trippett Bartlett Tree Experts Amanda Mitchell Defra Plant Health Policy team
19 February 2016 187
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 16 Scale of control activities carried out by outbreak area as has been reported to the OPMCP
Table A16.1 Scale of the control activities carried out in London (2013 - 2015)
2013 2014 2015 Total # of sites sprayed 108 216 104 428 #of trees sprayed 33,305 35,861 2,94766 72,113 # of sites where nests were 125 136 183 444 removed
# of trees were nests were 4,809 12,881 2,448 20,138 removed Table A16.2 Scale of the control activities carried out in Croydon / Bromley (2013 - 2015)
2013 2014 2015 Total # of sites sprayed 48 76 96 220 #of trees sprayed 5,661 9,958 2,600 18,219 # of sites where nests were 33 44 51 128 removed
# of trees were nests were 269 155 275 699 removed Table A16.3 Scale of the control activities carried out in Pangbourne (2013 - 2015)67
2013 2014 2015 Total # of sites sprayed 168 14 - 15 #of trees sprayed 317 105 - 422
66 Not including data from Richmond and Bushy Park where a considerably number of controls were conducted which were not funded by the OPMCP 67 Spraying was carried out in Pangbourne in 2013 and 2014 as a precautionary measure. No nests were found and therefore no nests removed 68 In 2013 the trees in Pangbourne were not distinguished between each other, therefore the whole area was considered as one site.
19 February 2016 188
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 17 Communications activity
Table A17.1 Indicators of OPMCP communications activity (2013 - 2015)
Activity 2013 2014 2015 Public information posters produced and distributed 2400 3000 Public information leaflets produced and distributed 80000 90,000 Addresses to which leaflets and posters were distributed 1,863 2,300 OPM signs for contractor vehicles (2 per vehicle) 30 301 Press releases at key milestone points in OPM season 4 3 1 Portable twin-set banners produced and provided to affected local 36 402 authorities, and to TfL NGO on-line forums engaged 2 0 FC site signage produced and deployed (Sulham Woods) 1 0 Tweets sent from FC accounts 5 83 ‘Internal’ partnership ‘OPM Update’ newsletters issued 0 15 Bespoke articles placed in trade or specialist media by FC 2 1 Bespoke articles placed in trade or specialist media by others 1 0 1 Specialist leaflets produced (to local authorities) 14 Guidance produced and published on website5 1 1 Notes: 1) No new ones produced - reused the 30 from 2014; 2) 4 new ones produced for new partners, previous year’s 36 re-used; 3) These are original tweets – retweets from FC accounts not included; 4) No record remaining of number of copies produced and distributed; 5) For tree surgeons 2014; for tree owners 2015 .
19 February 2016 189
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Annex 18 Overview of OPM research and monitoring activity
Type of project Funding body Subject / objectives Lead scientists / Duration Reports to date / link to supervisors summary or abstract Forestry Commission led 3 yr PhD studentship FC England Population dynamics of Dr Nigel Straw (FR) 2013-March 2016 (Richard Sands) OPM in woodland Dr Robbie Girling (ORC, environments, with a Newbury) specific focus on the Prof Guy Poppy impact of natural enemies (Southampton University) (parasitoids & predators) Dr Judith Lock (Southampton University) 2-year post-doctoral FC England Development of molecular Dr Darren Evans (Hull 2014-2016 research project techniques to identify OPM University) parasitoids and assess Dr Dave Lunt (Hull rates of parasitism University) Dr Nigel Straw (FR) FR FC GB Improved efficiency and Dr David Williams (FR) 2012-2016 FC Practice Note 020 calibration of pheromone Dr Nigel Straw (FR) published July 2013. traps Specific 5yr site monitoring FC GB Moth, bird and bat surveys Stewart Snape (FC) 2013-2017 Helicopter report Dec 13 at Pangbourne following Bat & Bird report Jan 14 the helicopter spraying of Interim moth report Jan 14 Herridge Copse with Bt in Moth report Feb 2015 2013 Specific 5yr site monitoring FC England Assessment of impact of Bt Jay Doyle 2013-2014 Report Autumn 2013 spraying on Purple 2014 Report hairstreak populations on Wimbledon Common Analysis of OPM control in Stewart Snape Mar-July 2015 Report finalised Nov 15 Netherlands vs GB Defra led Defra research call Defra Improved methods for early Dr Robert Weaver (Fera) 2012-Mar 2015 Mar 2015 – draft report detection of OPM (Defra project TH0101) including
19 February 2016 190
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Type of project Funding body Subject / objectives Lead scientists / Duration Reports to date / link to supervisors summary or abstract remote sensing, modelling, social media, public health reporting Defra research call Defra Improved control methods Dr Jude Bennison (ADAS) 2012-2015 Jan 2014 – draft report for OPM (Defra project TH0102) - Defra research call Defra Analysis of the Kew & Royal Parks also 2012-2013 June 2013 – draft report management of OPM involved. Includes outbreaks in the UK (Defra Syngenta work on stem project TH0103) injection Defra research call Defra Identification of the trail Dr Robert Weaver (Fera) 2013-2014 Not aware of any results pheromone of OPM larvae Dr David Hall (NRI) published/shared (Defra project TH0109) Effectively part of TH0102
Defra Defra OPM Control Plan Evaluation framework - 2015-Mar 2016 Evaluation framework Evaluation including Jake Morris (Defra) development assessment of future range Economic impact analysis Economic impact analysis of options / objectives, (Jonathan Bonas & Peter first done 2012/13 future outbreak trajectories Greene) Defra THABPI Defra Public perception of tree Prof Clive Potter (Imperial) 2014-TBC health risks (including OPM) PHE led Literature Search & Site Public Health England Literature review of human Barry Walsh, Emer (PHE) 2013 Presentation to OPM Study health impact of OPM; site Advisory Group Autumn study to establish risk of 2013 hairs to public and occupational health Medical Journal article Public Health England Review of human health Barry Walsh (PHE) 2013 Reports published June impact of OPM in 2015 Richmond 2006 Literature Search & Site Public Health England Review of 2014 public 2014 British Medical Journal Study health impact at Richmond/East Sheen?
19 February 2016 191
Evaluation of the Oak Processionary Moth Control Programme - Final Report
Type of project Funding body Subject / objectives Lead scientists / Duration Reports to date / link to supervisors summary or abstract Led by others Non target impact Royal Parks Impact of OPM control Gillian Jonusas, Nigel 2013 Report drafted Jan 2014. monitoring treatments (spraying with Reeve (Richmond Park) Bt) on non-target Lepidoptera Specific site monitoring Kew Gardens Kew biodiversity impact Jude Bennison using some monitoring? of their data for TH0102 Control efficacy Kew Gardens Their work has been mentioned but no data / report provided Literature review Butterfly conservation Review of evidence for Mark Parsons (Butterfly 2014 impact of spraying on non- Conservation) target species Commercially led research and monitoring work Stem injection research Part of TH0102 but extra Stem Injection to control Bartletts, Syngenta, Dealga Articles in sector being done commercially? OPM (and other P&Ds) O’Callaghan publications re potential and some heavy promotion; seeking CRD approval
19 February 2016 192