This report has been prepared on behalf of, and for exclusive use of the New Zealand Ministry of Foreign Affairs (MFAT) and the Australian Ministry of Foreign Affairs and Trade (DFAT), as required for ongoing planning related to TCCRP and TCRP. Its preparation follows consultation between stakeholders and the New Zealand and Australian Aid Programs. The report is subject to, and issued in connection with, the provisions of the agreement between Kurrajong Hill Pty Ltd and MFAT. Kurrajong Hill Pty Ltd accepts no responsibility whatsoever for, or in respect of, any use of or reliance upon this report by any third party or any use outside the purpose for which it was prepared. The evaluation team comprised Dr Fiona Kotvojs (Team Leader), Ms Anne Lockley (Evaluation Specialist), Ms Pisaina Leilua-Lei Sam (Samoa Economic Evaluation Specialist) and Ms Mardi Trompf (Procurement Specialist and Evaluation Mentee). We recognise the extensive contribution of so many people to this evaluation. The contribution of the Evaluation Steering Committee (MFAT, DFAT and GoS through MoF) was enormous. In particular, we would like to thank Tuanui Felauai (Lagi) of MFAT for her professionalism in managing and supporting this assignment. Each of those we interviewed provided valuable information, our thanks to the many representatives of donors, the Central Bank and financial institutions; all the Chief Executive Officers/Heads of the Government Ministries and Public Bodies we interviewed including those from Agriculture, Foreign Affairs and Trade, Health, Works and Infrastructure, Tourism, Women, Community and Social Development, Commerce, Industry and Labour, Environment and Fire Emergency Services; the The views expressed in this report are those of the author(s) and do not necessarily reflect the position of the New Zealand Government, the New Zealand Ministry of Foreign Affairs and Trade or any other party. Nor do these entities accept any liability for claims arising from the report’s content or reliance on it. DRAFT TCRRP and TCRP Evaluation Report Tourism Sector, Education Steering Committees, and Civil Society organisations. To each of those from the private sector who gave of their time for interviews and workshops; thank you. This includes tourism operators, builders and suppliers, and the business and community organisations. The input provided those who attended the sense making workshop and who provided comment on the aide-memoire and drafts of each appendix or sections of the report, thank you for your honesty and openness. The final result was only possible because of you. Our team would like to thank GoS, MFAT and DFAT for the opportunity to undertake this assignment. It was enjoyable and interesting. We trust that the final report will assist future support to recovery after natural disasters. Our team comprises Dr Fiona Kotvojs, Ms Anne Lockley, Ms Pisaina Leilua-Lei Sam and Ms Mardi Trompf. Dr Fiona Kotvojs is an evaluator who specialises in international development assistance programmes. She has over 25 years’ experience in designing and implementing quantitative and qualitative evaluations. Fiona is a principal at Kurrajong Hill. Ms Anne Lockley specialises in gender, social inclusion and evaluation, with over 20 years’ experience in working in these areas across the Pacific and South-East Asia. She has particular expertise in humanitarian responses and recovery. Ms Pisaina Leilua-Lei Sam is an expert in the public economic policy and practice, development, corporate governance, monitoring and evaluation and practice in the Pacific and Samoa with more than 25 years’ experience in these areas. Ms Mardi Trompf is a procurement and costing specialist. She brings this expertise to her evaluation practice. 3 DRAFT TCRRP and TCRP Evaluation Report Table of Contents Abstract 6 Executive Summary 7 Background 7 Overarching findings 7 Background 11 THE ACTIVITY 11 EVALUATION PURPOSE AND DESIGN 12 Purpose 12 Scope 12 Design 12 Overarching Findings 14 Objective 1: The effectiveness of TCRRP and the TCRP 14 Objective 2: Programme impact 17 Objective 3: Programme efficiency 19 Incorporation of recovery principle: value for money 19 Management 20 Private sector involvement 25 Objective 4: Sustainability 26 Sustainability of improved capacity acquired from the programmes 26 Incorporation of recovery principle: Build back better 26 Maintenance 28 Objective 5: Relevance of the TCRRP and TCRP 29 Alignment with Government of Samoa, NZ MFAT and DFAT development policy 29 Objective 6: Lessons learned and recommendations 30 Application of previous lessons 36 Application of lessons to Cyclone Gita in 2018 36 4 DRAFT TCRRP and TCRP Evaluation Report Evaluation Conclusions 38 Appendices 40 APPENDIX ONE: TERMS OF REFERENCE 41 APPENDIX TWO: PROGRESS, ACHIEVEMENTS, AND RESULTS FRAMEWORK 52 (1) General recovery and rehabilitation 52 (2) The Tourism Cyclone Recovery Programme 59 (3) TCRP Results Framework 62 APPENDIX THREE: SECTORAL STUDY TOURISM 63 BACKGROUND 63 METHODOLOGY 65 FINDINGS 66 APPENDIX FOUR: SECTORAL STUDY EDUCATION 85 BACKGROUND 85 METHODOLOGY 86 FINDINGS 86 APPENDIX FIVE: SECTORAL STUDY HEALTH 100 BACKGROUND 100 METHODOLOGY 101 FINDINGS 102 APPENDIX SIX: PROCUREMENT SYSTEMS ANALYSIS 116 BACKGROUND 116 METHODOLOGY 116 FINDINGS 117 Conclusions 124 RECOMMENDATIONS 125 APPENDIX SEVEN: COST UTILITY ANALYSIS 130 REFERENCES 133 Abbreviations and acronyms 136 5 DRAFT TCRRP and TCRP Evaluation Report 1 Abstract Tropical Cyclone Evan passed over Samoa on 13-14 December 2012 and caused widespread damage. New Zealand and Australia provided support to both the response and recovery phase. Samoa’s recovery priorities were defined in their recovery framework. This formed the base for assistance in the recovery phase provided through the Tropical Cyclone Evan Recovery and Rehabilitation Programme (TCRRP), which was implemented as budget support, and the Tourism Cyclone Recovery Programme (TCRP) which was implemented as a program. This evaluation identifies the relevance and effectiveness of TCRP and TCRRP and whether they built awareness and capacity in relation to disasters. Applying a utilisation focused approach, three sector studies were completed (education, health and tourism) to inform the findings. For each of these sector studies, data was collected from a breadth of sources and analysed using content analysis. A cost-utility analysis was also undertaken for TCRP. The evaluation found TCRRP and TCRP to be highly relevant to the post-disaster needs assessment, donor strategies, and in general, to sectoral plans. In both education and health, the infrastructure would not have occurred without external assistance. Most expected TCRRP and TCRP outcomes were achieved. However, ‘soft’ outcomes specified in Samoa’s recovery framework were not well resourced and often not achieved. Gender and disability inclusion were also poorly addressed. Through TRCP and TCRRP, the quality of buildings have improved. However, the longer term impact of support is mixed because (i) some facilities are no longer used or not used as intended and (ii) there is a lack of proactive and curative maintenance. 6 DRAFT TCRRP and TCRP Evaluation Report 2 Executive Summary Background Tropical Cyclone Evan passed over Samoa on 13-14 December 2012. It is considered the worst tropical cyclone to hit Samoa since 1991, and caused widespread damage to public services, buildings, roads, agriculture, and communications infrastructure. The Government of Samoa (GoS) implemented an immediate response, declared a state of emergency, and called for international assistance. A multi-agency damage and loss assessment coordinated by the Ministry of Finance (MoF) was finalised in March 2013. This provided the basis for the prioritisation of activities under the Tropical Cyclone Evan Recovery and Rehabilitation Programme (TCRRP). TCRRP’s goal was that Samoa recovers from Tropical Cyclone Evan, reduces vulnerability and enhances resilience to withstand future shocks. A number of bi- and multilateral donors contributed to this programme including the New Zealand Ministry of Foreign Affairs and Trade (MFAT) and the Australian Department of Foreign Affairs and Trade (DFAT). DFAT support was primarily to the health and education sectors, and MFAT’s to the tourism sector under the Tourism Cyclone Recovery Programme (TCRP). This scope of this evaluation is limited to the recovery and rehabilitation period (May 2013 to end 2015). This summary focuses on the key points and priority recommendations. Overarching findings Relevance: The evaluation found the TCRRP and TCRP to be highly relevant to the post- disaster needs assessment, donor strategies, and in general, to sectoral plans. In both education and health, the infrastructure would not have been built or rebuilt without external assistance. Effectiveness: At both a sectoral and programme level, most expected TCRRP and TCRP outcomes specified in the monitoring and evaluation framework during the implementation period were achieved. However, there is a common pattern that ‘soft’ outcomes such as investments in public health and primary health care, disaster resilience (including building national awareness and capacity on areas of disaster risk reduction, management, preparedness and responsiveness), and psycho-social support, were not as well-resourced or achieved as those related to infrastructure. Recommendation: During recovery and rehabilitation, GoS and donors ensure appropriate attention to ‘soft’ priorities for which existing in country capacity