IN-DEPTH EVALUATION OF THE GLOBAL MARITIME CRIME PROGRAMME MID-TERM CLUSTER INDEPENDENT IN-DEPTH EVALUATION Global Maritime Crime Programme GLOX99, GLOZ06, GLOW63, GLOZ81, GLOZ80, XEAX94 October 2020 This independent evaluation report was prepared by an evaluation team consisting of Mr. Peter Allan (Lead Evaluator), Ms. Camille Massey (Evaluator) and Mr. Mark Blaine (Evaluator). The Independent Evaluation Section (IES) of the United Nations Office on Drugs and Crime (UNODC) provides normative tools, guidelines and templates to be used in the evaluation process of projects. Please find the respective tools on the IES web site: http://www.unodc.org/unodc/en/evaluation/evaluation.html The Independent Evaluation Section of the United Nations Office on Drugs and Crime can be contacted at: United Nations Office on Drugs and Crime Vienna International Centre P.O. Box 500 1400 Vienna, Austria Telephone: (+43-1) 26060-0 Email: [email protected] Website: www.unodc.org Disclaimer Independent Project Evaluations are scheduled and managed by the project managers and conducted by external independent evaluators. The role of the Independent Evaluation Section (IES) in relation to independent project evaluations is one of quality assurance and support throughout the evaluation process, but IES does not directly participate in or undertake independent project evaluations. It is, however, the responsibility of IES to respond to the commitment of the United Nations Evaluation Group (UNEG) in professionalizing the evaluation function and promoting a culture of evaluation within UNODC for the purposes of accountability and continuous learning and improvement. The views expressed in this independent evaluation report are those of the evaluation team. They do not represent those of UNODC or of any of the institutions or Member States referred to in the report. All errors and omissions remain the responsibility of the evaluation team. © United Nations, October 2020. All rights reserved worldwide. The designations employed and the presentation of material in this publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area, or of its authorities, or concerning the delimitation of its frontiers or boundaries. This publication has not been formally edited. ii CONTENTS CONTENTS ......................................................................................................................................................... iii ABBREVIATIONS AND ACRONYMS ............................................................................................................. iv MANAGEMENT RESPONSE ........................................................................................................................... vi EXECUTIVE SUMMARY .................................................................................................................................... x I. INTRODUCTION ......................................................................................................................................... 1 BACKGROUND AND CONTEXT ................................................................................................................ 1 EVALUATION METHODOLOGY ................................................................................................................. 3 LIMITATIONS TO THE EVALUATION ........................................................................................................ 5 II. EVALUATION FINDINGS ...................................................................................................................... 7 DESIGN AND RELEVANCE ......................................................................................................................... 7 EFFICIENCY ................................................................................................................................................. 16 EFFECTIVENESS ........................................................................................................................................ 19 IMPACT .......................................................................................................................................................... 23 SUSTAINABILITY ......................................................................................................................................... 29 PARTNERSHIPS AND COOPERATION .................................................................................................. 31 III. CONCLUSIONS .......................................................................................................................................... 34 IV. RECOMMENDATIONS ............................................................................................................................. 36 V. LESSONS LEARNED AND BEST PRACTICES .................................................................................... 40 ANNEX I: TERMS OF REFERENCE ............................................................................................................ 43 BACKGROUND AND CONTEXT ............................................................................................................................. 43 DISBURSEMENT HISTORY .................................................................................................................................... 53 PURPOSE OF THE EVALUATION ........................................................................................................................... 53 SCOPE OF THE EVALUATION ................................................................................................................................ 54 KEY EVALUATION QUESTIONS ............................................................................................................................. 54 EVALUATION METHODOLOGY ............................................................................................................................. 56 TIMEFRAME AND DELIVERABLES ......................................................................................................................... 58 EVALUATION TEAM COMPOSITION ..................................................................................................................... 60 MANAGEMENT OF THE EVALUATION PROCESS .................................................................................................. 60 ANNEX II: EVALUATION TOOLS: QUESTIONNAIRES AND INTERVIEW GUIDES ........................... 63 ANNEX III: DESK REVIEW LIST ................................................................................................................... 69 ANNEX IV: STAKEHOLDERS CONTACTED DURING THE EVALUATION ......................................... 73 iii ABBREVIATIONS AND ACRONYMS Abbreviation Abbreviation Full name Full name or Acronym or Acronym AO Atlantic Ocean HoA Horn of Africa ASEAN Association of Southeast Asian HQ Headquarters Nations CARRICOM Caribbean Community HRGE Human Rights and Gender Equality CCP Container Control Programme HRMS Human Resources Management Services CCTV Closed Circuit Television HTMSS Human Trafficking and Migrant Smuggling Section CPP Counter Piracy Programme IES Independent Evaluation Section CSO Civil Society Organisation IO (East) Indian Ocean (East) ECCAS Economic Community of Central IO (West) Indian Ocean (West) African States ECOWAS Economic Community of West IPB Integrated Programme and African States Oversight Branch ER Evaluation Report IR Inception Report EU European Union KPI Key Performance Indicator FRMS Financial Resources Management LAC Latin America and Caribbean Services GARS Gulf of Aden and Red Sea MASE Maritime Security GMCP Global Maritime Crime MLE Maritime Law Enforcement Programme GP Global Programme MS Member State iv IN-DEPTH EVALUATION OF THE GLOBAL MARITIME CRIME PROGRAMME Abbreviation Abbreviation Full name Full name or Acronym or Acronym MSC Most Significant Change UN United Nations MST Management Support Team UNDP United Nations Development Programme NGO Non -Governmental Organisation UNEG United Nations Evaluation Group OHCHR Office of the United Nations High UNCLOS United Nations Convention on the Commissioner for Human Rights Law of the Sea OSCE Organisation for Security and UNICEF United Nation’s Children Fund Cooperation in Europe PO Pacific Ocean UNODC United Nations Office on Drugs and Crime PSC Programme Support Cost UNON United Nations Office in Nairobi SDG Sustainable Development Goal UNTOC United Nations Convention against Transnational Organised Crime TL Team Leader UNWOMEN United Nations Entity for Gender Equality and the Empowerment of Women ToR Terms of Reference VIC Vienna International Centre TPB Terrorism Prevention Branch VBSS Visit, Board, Search, and Seizure UK United Kingdom of Great Britain WLFC Wildlife and Forest Crime and Northern Ireland ABBREVIATIONS AND ACRONYMS v MANAGEMENT RESPONSE Management Overview anticipated Response actions Recommendation (accepted/partially accepted/rejected) 1: MONITORING and EVALUATION Partially accepted GMCP will work with Through a maturing Results Based Management The evaluation UNODC HQ, Strategic (RBM) system UNODC Global Maritime Crime acknowledged that Planning and Inter-agency Programme (GMCP) Headquarters (HQ)Team in Sri GMCP does well in Affairs (SPIA) to identify Lanka and the Strategic Planning and Interagency developing and options to better reflect