Policy Influence in the Russian Federation: Evaluation of Advocacy
Total Page:16
File Type:pdf, Size:1020Kb
EVALUATION: MAY 2014 PUBLICATION: NOVEMBER 2014 POLICY INFLUENCE IN THE RUSSIAN FEDERATION Evaluation of advocacy and campaigning on climate change Effectiveness Review Series 2013/14 Photo: Report on development and launch of Clicr.ru climate campaign (July-September 2009) SVETLANA KOZLOVA & MIKHAIL PALTSYN OXFAM GB www.oxfam.org.uk/effectiveness ACKNOWLEDGEMENTS The evaluation team thanks Claire Hutchings and Christine DeBlase, Oxfam GB for making this evaluation possible. The evaluation team especially acknowledges the contribution of Yulia Yevtushok, Climate Change and Economic Justice programme coordinator (2008–2013) for providing necessary data and participating in the analysis of the evaluation findings. The evaluation consultants are very grateful to Andrey Rakhmanov, cur- rent Climate Change and Economic Justice programme coordinator for providing the necessary documents, organising the evaluation field mission in Moscow, and participating in the verification of the evaluation find- ings. The Evaluators also acknowledge the assistance of Johnathan Puddifoot, Richard English and Fionna Smyth in the analysis of planning, monitoring, evaluation, and reporting frameworks of Oxfam GB. The evaluation team thanks all stakeholders who took part in the evaluation and provided their valuable comments on the project outputs and outcomes, especially, Yuriy Kotenev (Director of ‗Black Stork‘ NGO) and Natalia Chibrikova (President of Ecological Club Inter-Regional NGO). ABBREVIATIONS AKKOR Association of Farmers and Agriculture Cooperatives of Russia CAN Climate Action Network COP Conference of the Parties CSO Civil Society Organisation DAC Development Assistance Committee ER Effectiveness Review GPF Global Performance Framework logframe Logical Framework M&E Monitoring and Evaluation MECIS Middle East and Commonwealth of Independent States MEL Monitoring, Evaluation, and Learning NGO Non-governmental organisation OECD Organisation for Economic Co-operation and Development OPAL Oxfam Programme, Accountability and Learning system Oxfam GB Oxfam Great Britain OI Oxfam International RBM Result-Based Management RGU Russian Grain Union RIAC Russian International Affairs Council SMART Specific, Measurable, Achievable, Realistic and Time-bound TOC Theory of Change TOR Terms of Reference UNEP/GEF The United Nations Environment Program Division of Global Environment Facility Coordination UNFCCC The United Nations Framework Convention on Climate Change WWF World Wide Fund for Nature CONTENTS Acknowledgements ...................................................................................................................... 2 Abbreviations ................................................................................................................................ 3 CONTENTS .................................................................................................................................... 4 1 Executive summary ................................................................................................................ 6 1.1 Brief Overview of the project ............................................................................................... 6 1.2 Identification of key outcomes ............................................................................................ 6 1.3 Evaluation design and methodology .................................................................................. 7 1.4 Findings ................................................................................................................................ 7 1.5 Programme learning considerations .................................................................................. 9 1.5.1 Key lessons ...................................................................................................................... 9 1.5.2 Key recommendations ................................................................................................... 11 2 Introduction ........................................................................................................................... 14 3 Evaluation design ................................................................................................................. 15 3.1 Methodology ....................................................................................................................... 15 3.2 Development of logic model.............................................................................................. 16 3.3 Data collection strategy ..................................................................................................... 26 3.4 Limitations .......................................................................................................................... 28 4 Project description ................................................................................................................ 30 5 Findings ................................................................................................................................. 32 5.1 Processes affecting attainment of project results ........................................................... 32 5.1.1 Project preparation and readiness ................................................................................ 32 5.1.2 Project design ................................................................................................................ 33 5.1.3 Implementation approach, management and project finance ..................................... 35 5.1.4 Oxfam supervision ......................................................................................................... 37 5.1.5 Reporting, monitoring and evaluation .......................................................................... 38 5.2 Outcome 1 .......................................................................................................................... 43 5.2.1 Salient causal stories ........................................................................................................ 46 5.2.2 Findings ........................................................................................................................... 46 5.2.3 Outcome rating................................................................................................................. 50 5.3 Outcome 2 .......................................................................................................................... 51 5.3.1 Salient causal stories....................................................................................................... 52 5.3.2 Findings ............................................................................................................................ 52 5.3.3 Outcome rating................................................................................................................. 59 5.4 Outcome 3 ......................................................................................................................... 59 5.4.1 Salient causal stories....................................................................................................... 66 5.4.2 Findings ............................................................................................................................ 66 5.4.3 Outcome rating ................................................................................................................. 75 6 Programme learning considerations ................................................................................... 77 6.1 Key lessons ........................................................................................................................ 77 6.2 Key recommendations ....................................................................................................... 83 6.2.1 Recommendations from Oxfam experts and partners ................................................. 83 6.2.2 Recommendations by Evaluators .................................................................................. 85 7 Conclusion ............................................................................................................................ 91 Appendix 1: Terms of Reference ............................................................................................... 96 Appendix 2: Evaluation questions, data analysis ..................................................................... 99 Appendix 3: Glossary of terms ................................................................................................ 101 Appendix 4: Documentation reviewed .................................................................................... 102 Notes .......................................................................................................................................... 107 1 EXECUTIVE SUMMARY 1.1 BRIEF OVERVIEW OF THE PROJECT As per Oxfam Great Britain‘s (OGB) Global Performance Framework (GPF), samples of mature projects are being randomly selected each year and their effectiveness rigorously assessed. Project ‗Advocacy and campaigning on Climate Change in the Russian Federation‘ (hereafter the project/RUSA 34/Campaign) was selected in this way under the ‗Policy Influence/Citizen Voice‘ thematic area according to OGB GPF standards. Two external consultant Evaluators, Svetlana Kozlova and Mikhail Paltsyn, were hired by Oxfam GB to perform the Effectiveness Review of the project between March and July 2014. The project was a part of