Sri Lanka Post-Closure Evaluation 2015
Total Page:16
File Type:pdf, Size:1020Kb
VSO Sri Lanka post-closure evaluation 2015 Sri Lanka post-closure evaluation Report Karen Iles September 2015 0 VSO Sri Lanka post-closure evaluation 2015 This report This report presents the findings of the post-closure evaluation of the VSO program in Sri Lanka, which took place between March and June 2015. Acknowledgments We would like to thank all those who participated in and supported the post-closure evaluation. Our thanks are extended to: VSO’s Partners in Sri Lanka for so generously sharing their time, experiences and rich insights: Dr. Mendis and all the NIMH staff, and the occupational therapists; Mr Thayaparan and the team at PCA, Mr. Thatparan and the team at Shantiham, and Mr. Sukirtharaj and the team at JSAC; all the former VSO volunteers for so readily and openly shared their experiences and thoughts; the former VSO country program team in Sri Lanka for taking time out of their busy schedules to share their reflections; and the staff at VSO for sharing their thinking on theory of change and post closure evaluations. A very special thanks to Mrs. Ruvanthi Sivapragasam and Mrs. Manchula Selvaratnam for their absolutely invaluable role in liaising with the Partners to organise the evaluation and logistics, which enabled the activities to run smoothly. Many thanks are also extended to our two translators, Mr. Kennedy and Mr. Tilak Karunaratne, who were valuable members of the evaluation team, contributing their own insights. The feedback on the evaluation findings and draft report from the Steering Group was very valuable and much appreciated. Finally, many thanks to Janet Clark and Patrick Proctor of VSO for their support throughout the process. Prepared by: Karen Iles Learning and Change Initiatives [email protected] 1 VSO Sri Lanka post-closure evaluation 2015 Table of Contents ACKNOWLEDGMENTS ............................................................................................................................. 1 FIGURES ................................................................................................................................................... 5 TERMS ..................................................................................................................................................... 6 EXECUTIVE SUMMARY ............................................................................................................................ 7 1. INTRODUCTION ............................................................................................................................ 13 1.1 Overview ............................................................................................................................... 13 1.2 VSO program in Sri Lanka ...................................................................................................... 13 1.2.1 Sri Lanka context ........................................................................................................... 13 1.1.2 VSO country program.................................................................................................... 14 1.3 The post closure evaluation .................................................................................................. 16 1.3.1 Aim and key questions .................................................................................................. 16 1.3.2 Focus and scope of the post-closure evaluation .......................................................... 17 1.3.3 Participating Partners ................................................................................................... 17 2. METHODOLOGY ............................................................................................................................ 19 2.1 Evaluation strategy ............................................................................................................... 19 2.1.1 Conceptual framework ................................................................................................. 19 2.1.2 Approach ....................................................................................................................... 20 2.2 Data collection, analysis and interpretation ......................................................................... 22 2.2.1 Participants and sampling ............................................................................................. 22 2.2.2 Assessing change - retrospective views and baselines ................................................. 22 2.2.3 Methods of data collection ........................................................................................... 23 2.2.4 Analysis and interpretation of findings ......................................................................... 23 2.2.5 Limitations of the methodology .................................................................................... 24 3. FINDINGS ...................................................................................................................................... 25 3.1 Meaning of ‘Partner Capacity’ .............................................................................................. 25 3.1.1 Focus of this section ...................................................................................................... 25 3.1.2 Partners’ perceptions of capacity ................................................................................. 25 3.1.3 Description of the main capacity elements .................................................................. 26 3.1.4 Most important capacity elements ............................................................................... 28 3.2 VSO’s contributions to Partners’ capacity development ...................................................... 29 3.2.1 Focus of this section ...................................................................................................... 29 3.2.2 Specific contributions of VSO volunteers to capacity development ............................. 29 3.2.3 Specific contributions by the VSO country program ..................................................... 31 3.2.4 Change in capacity: patterns of interaction and emergent outcomes ......................... 31 3.3 Relative contributions of VSO to Partners’ capacity development ...................................... 35 2 VSO Sri Lanka post-closure evaluation 2015 3.3.1 Focus of this section ...................................................................................................... 35 3.3.2 Alternative explanations for the capacity developments of Partners .......................... 35 3.4 How change in Partners’ capacity arises............................................................................... 38 3.4.1 Focus of this section ...................................................................................................... 38 3.4.2 VSO volunteers .................................................................................................................. 38 3.4.3 Partners’ internal factors .............................................................................................. 42 3.4.4 VSO’s programmatic approach ..................................................................................... 43 3.4.5 Contextual factors and donor partners ........................................................................ 44 3.4.6 Capacity development as patterns of interaction and emergent outcomes ................ 44 3.4.7 Capacity developing activities that were less successful .............................................. 45 3.4.8 How change happens: congruence with VSO’s global Theory of Change .................... 46 3.5 Sustainability of capacity gains ............................................................................................. 48 3.5.1 Focus of this section ...................................................................................................... 48 3.5.2 Extent to which VSO volunteers’ contributions to Partner’s capacity development have been sustained from 2013/2014 to 2015. ........................................................................... 48 3.5.3 Factors enabling and hindering the sustainability of capacity gains created with ....... 50 VSO’s support ................................................................................................................................ 50 3.5.5 Type of Partner, length of partnership, number of international volunteer placements 55 3.6 Unanticipated consequences of VSO’s capacity development work .................................... 56 3.6.1 Focus of this section ...................................................................................................... 56 3.6.2 The unexpected............................................................................................................. 56 3.7 Unique effectiveness of VSO’s approach .............................................................................. 57 3.7.1 Focus of this section ...................................................................................................... 57 3.7.2 VSO’s approach compared to other organisations ....................................................... 57 3.7.3 Approach to capacity development: congruence with VSO’s theory of change .......... 59 3.8 Assessing change in capacity: VSO’s M&E tools and Partner perceptions ........................... 60 3.8.1 Focus of this section .....................................................................................................