Final Evaluation Report
Total Page:16
File Type:pdf, Size:1020Kb
UNITED NATIONS OFFICE ON DRUGS AND CRIME Vienna Independent formative In-depth evaluation of the Western Balkans Counter-Serious Crime Initiative (WBCSCi) in the context of the Integrative Internal Security Governance (IISG) mechanism including the European Union action: “Instrument for Pre-Accession Assistance (IPA II) Multi-Country Action Programme 2017 - Support to the Western Balkan Integrative Internal Security Governance” Western Balkans This independent evaluation is funded by the European Union February 2019 UNITED NATIONS New York, 2019 This independent evaluation report was prepared by an evaluation team consisting of Punit Arora, Jim Newkirk, Peter Allan, Eleni Tsingou, Katherine Aston, and Emanuel Lohninger, under the supervision of Ms. Katharina Kayser. The Independent Evaluation Unit (IEU) of the United Nations Office on Drugs and Crime (UNODC) provides normative tools, guidelines and templates to be used in the evaluation process of projects. Please find the respective tools on the IEU web site: http://www.unodc.org/unodc/en/evaluation/evaluation.html The Independent Evaluation Unit of the United Nations Office on Drugs and Crime can be contacted at: United Nations Office on Drugs and Crime Vienna International Centre P.O. Box 500 1400 Vienna, Austria Telephone: (+43-1) 26060-0 Email: [email protected] Website: www.unodc.org Disclaimer The views expressed in this independent evaluation report are those of the evaluation team. They do not represent those of UNODC or of any of the institutions or Member States referred to in the report. All errors and omissions remain the responsibility of the evaluation team. Considering that this evaluation is funded by the European Union, all references to Kosovo* in the present document will exceptionally follow the EU-referencing: “* This designation is without prejudice to the position on status and is in line with UNSCR 1244/99 and the ICJ opinion on the Kosovo declaration of independence”. All references to Macedonia in the present draft document should be understood in the context of the UN Security Council resolution 817 par.2. © United Nations, February 2019. All rights reserved worldwide. The designations employed and the presentation of material in this publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area, or of its authorities, or concerning the delimitation of its frontiers or boundaries. This publication has not been formally edited. ii CONTENTS Page Management response: IISG Support Group ...................................................... vi Executive summary .............................................................................................. x Summary matrix of findings, evidence and recommendations ........................... xiv I. Introduction ......................................................................................................... 01 Background and context ...................................................................................... 01 Evaluation methodology ...................................................................................... 05 Limitations ........................................................................................................... 07 II. Evaluation findings .............................................................................................. 08 Design .................................................................................................................. 08 Relevance ............................................................................................................. 16 Effectiveness ........................................................................................................ 19 Efficiency .............................................................................................................. 26 Partnerships and cooperation .............................................................................. 30 Impact and sustainability .................................................................................... 33 III. Conclusions .......................................................................................................... 37 IV. Recommendations ............................................................................................... 39 V. Lessons learned and best practices ...................................................................... 41 Annexes I. Terms of reference of the evaluation ................................................................... 42 II. Evaluation tools: questionnaires and interview guides ....................................... 66 III. Desk review list .................................................................................................... 68 iii IV. List of persons contacted during the evaluation .................................................. 71 V. Revised evaluation questions and data collection strategy ................................. 72 PUBLICATION TITLE HERE TITLE PUBLICATION VI. Events observed ................................................................................................... 76 VII. Operating costs for the IISG mechanism ............................................................. 77 VIII. Terms of Reference for the IISG ......................................................................... 78 iv ABBREVIATIONS AND ACRONYMS Abbreviation Full name Abbreviation Full name CEPOL The EU Agency for Law iPA Integrative Plan of Action Enforcement Training CSO Civil Society Organisation DCAF Democratic Control of Armed JHA Justice and Home Affairs Forces DG NEAR European Commission– OCG Organised Crime Groups Directorate-General for Neighbourhood and Enlargement Negotiations EMG Evaluation Management PCC SEE Police Cooperation Convention for Group Southeast Europe EU European Union SOC Serious Organised Crime Eurojust The European Union’s ToR Terms of Reference Judicial Cooperation Unit Europol The European Union’s Law UNODC United Nations Office on Drugs Enforcement Agency and Crime HRGE Human Rights and Gender WB Western Balkans Equality IEU Independent Evaluation Unit of UNODC GIZ Deutsche Gesellschaft für WBBSi Western Balkan Border Security Internationale Initiative Zusammenarbeit – German Development Agency IISG Integrative Internal Security WBCSCi Western Balkan Counter Serious Governance Crime Initiative IPA Instruments of Pre-Accession WBCTi Western Balkan Counter- Terrorism initiative v MANAGEMENT RESPONSE: IISG SUPPORT GROUP The Management Team of the Integrative Internal Security Governance (IISG) as the mechanism and policy process representing the overarching concept of the Western Balkan Counter Serious Crime Initiative (WBCSCi), would like to express its gratitude to the evaluation team, Mr Punit Aurora, Mr Jim Newkirk, Ms. Eleni Tsingou, Mr. Peter Allan, Mr Emanuel Lohninger1 and Ms Katherine Aston2, under the guidance and supervision of Ms. Katharina Kayser3, for their great efforts on evaluating the WBCSCi in the context of IISG mechanism, incl. the EU Action IPAII2017 (Support to the Western Balkan Integrative Internal Security Governance, hereinafter: “the Action”). This evaluation has been innovative in terms of conducting it at this strategic level, setting thereby a best practice for similar initiatives. It also constitutes a baseline against which IISG as well as other future evaluations can and should be conducted, allowing also for comparison and measuring of change. Furthermore, the political ownership of the evaluation as well as its results (the M&E framework had been approved by the IISG Board and is being familiarised with the results) constitute an excellent practice. The Interim Evaluation Report was requested by the IISG leadership at the time of drafting the Action – it was intended to contribute to improvements of this unique process, and we are convinced that the findings and follow-up will enable us to achieve our mission and future impact. The Report is now being presented to the members of the 3rd IISG Board. The evaluation process itself has offered space for honest, open dialogue and exchanges, supporting learning and building trust. This and the insightful contributions from the Core Learning Partners led to actionable recommendations. We are hereby also extending a special thanks to the Core Learning Partners and all other stakeholders for contributing their valuable time to the evaluation exercise during 2018. The Management Team would also like to express its preparedness to address, in the best way possible and in close consultation with Beneficiaries and all Partners, the set of recommendations in the next year, and look forward to the next phase of evaluation. The Report highlights the role of the IISG as it has been realised so far, throughout the beginning phase of the IISG following the official launch by the IISG Board on 8 September 2017. The Report provides findings as to both the IISG as the overarching mechanism on the one hand, and the WBCSCi and the Action (IPAII2017) on the other hand. It is our estimate that the evaluation team fully comprehended the complex structure of the Action, which supports, through individual strands of GIZ, CILC, MoI Italy, UNODC, CEPOL and Europol (our respected partners with extensive experience in the region and unparalleled track record as to their contributions to policy reform and cooperation in the Western Balkans), nothing