Summative Evaluation of UNICEF India’s Cotton Corridors Project: ‘Preventing Exploitation and Protecting Children’s rights in Andhra Pradesh and Karnataka.’ Final evaluation report Oxford Policy Management and Glocal Research Services 13th July 2016 UNICEF Cotton Corridors Project Final Evaluation report © Oxford Policy Management i UNICEF Cotton Corridors Project Final Evaluation report Acknowledgements United Nations Children’s Fund (UNICEF) contracted Oxford Policy Management (OPM) Limited for the summative evaluation of UNICEF India’s Cotton Corridors Project: Preventing Exploitation and Protecting Children’s rights in Andhra Pradesh (AP) and Karnataka, for the duration of October 2015 to June 2016. Both rounds of data collection have been conducted by OPM and Glocal Research Services (Glocal, the fieldwork and research partner) in Kurnool and Raichur, the two project districts in AP and Karnataka. Writing of the evaluation report has been undertaken by OPM. The roles and responsibilities of the team members were as follows: Roles and responsibilities Name Roles and responsibilities Research guidance and coordination, overall Ian MacAuslan management and quality assurance Development of theory of change, evaluation matrix and survey tools. Mehjabeen Jagmag, Shruti Viswanathan, Udit Conducted first phase of data collection and part of Ranjan and Vanika Grover the data collection in the second phase Coding and analysis Davuluri Venkateswarlu Overall field management, research guidance G. Kranthi Data collection management in Andhra Pradesh T.N. Reddy Data collection management in Karnataka Neelam Singh Technical guidance (child protection issues) Aprajita Mukherjee Technical guidance (gender issues) Research guidance on methodology and quality Saltanat Rasulova assurance Tom Newton-Lewis Peer reviewer Glocal staff and surveyors Data collection and providing field insights The team acknowledges the contributions of the Evaluation Reference Group members, B.M. Naidu and Prof Vijay Kumar and all the members of the Steering Committee (Trang Ho Morton, Deepak Kumar Dey, M. Murali Krishna and Vandhana Kandhari) for providing quality assurance for the entire survey process and report content. The team also acknowledges and thanks UNICEF’s project partners for sharing information and supporting the fieldwork that supported the development of the theory of change (TOC) and evaluation questions. We also thank our peer reviewers for their helpful comments. All errors remain the authors’. All photographs are by the authors. © Oxford Policy Management ii UNICEF Cotton Corridors Project Final Evaluation report Executive summary Purpose and methodology This report presents findings from an independent summative evaluation conducted in 2015– 2016 by OPM and Glocal of the “Cotton Corridors Project: Preventing Exploitation and Protecting Child Rights in AP and Karnataka” (henceforth Cotton Corridors project). The Cotton Corridors project ran between June 2008 and December 2014, with a budget allocated by IKEA Foundation of EUR 4,794,135, and was implemented by UNICEF India and the Governments of AP and Karnataka. The objective of the Cotton Corridors project was to create improved systems and structures for preventing exploitation and protecting children in Kurnool and Raichur districts, building on the work of an earlier phase of project activity that started in 2006. These districts were selected because they had a particularly large estimated number of children working in cottonseed and cotton production and were out of school. The project aimed to improve child protection outcomes in these districts, specifically to: return 18,750 long-term and short-term drop outs aged 6-14 to school; improve the quality of child friendly infrastructure in 350 schools; train 1,500 adolescent girls in relevant skills; form 591 collectives for young people to discuss and have a dialogue with policy-makers and equip 4,500 women as leaders on child protection issues. The Cotton Corridors project took an innovative approach to achieving these objectives. The key characteristics of this approach were: working closely with state, district and block-level government officials and community structures to create an enabling environment for improved child protection through engagement, technical assistance, and information, education and communication (IEC); forming, training and supporting three types of community groups in each village (a Child Protection Committee [CPC], a Child Club [CC], and a Girls’ Collective) to work with community volunteers (CVs) to identify and address child protection issues in their villages; collecting, monitoring and agreeing data on children who were not in school; and conducting with government and communities a set of activities designed to improve child protection outcomes, including: o conducting enforcement drives on farms or businesses employing children, rescuing children and returning them to formal or bridge education, and fining employers; o conducting enforcement drives on under-age marriages and fining parents and priests; o funding children at risk of dropping out to attend open school from 10th class and sit for exams; o organising and funding Non-Residential Special Training Centres (NRSTCs) for children who have recently dropped out of school and need to catch up before re-entering; o organising and funding Residential STCs (RSTCs) for long-term dropouts; o organising and funding Child Migration Prevention Centres (CMPC) to allow children to remain in school while their parents migrate for work; and o organising and funding skills training for adolescents. © Oxford Policy Management iii UNICEF Cotton Corridors Project Final Evaluation report The purpose of the evaluation is to provide an independent assessment of whether the project’s strategies and approaches contributed to the achievement of objectives, and to identify lessons that can be learned for future attempts to improve child protection. The evaluation answers five key questions organised around five evaluation criteria: relevance: To what extent were the outputs of Cotton Corridors project consistent with recipients’ requirements, country needs, global priorities and UNICEF’s policies? effectiveness: To what extent were outcomes of the various strands of the Cotton Corridors project successfully achieved? efficiency: How economically were resources/inputs converted into results? sustainability and replicability: To what extent have benefits from Cotton Corridors project continued after UNICEF support has been completed and why, and to what extent are the successes of the project replicable elsewhere? equity and gender: To what extent have activities affected marginalised communities, addressed differences in gender, and empowered both girls and boys? The key intended audiences are: i) departments in the Governments of AP, Karnataka and India relevant to child labour (Labour, Education, Women and Child Development [WCD], and Planning), and ii) UNICEF India staff, specifically, the Hyderabad Chief of Office, India Chief of Child Protection, and the Policy, Planning and Evaluation (PPE) and Child Protection networks. In order to ensure that the evaluation delivers on its objectives to its key audiences, UNICEF constituted an Evaluation Steering Committee of the Cotton Corridors project and evaluation staff, and an Evaluation Reference Group also including government officials and independent experts. The methodology used for the evaluation was limited by its timing and scope: the evaluation was commissioned nine months after the project ended and there was no possibility of a counterfactual. This meant that the evaluation relied on (non-representative) primary data collected from 64 focus group discussions (FGDs) with group members and parents, 144 scale questionnaires with group chairs, 192 in-depth interviews (IDIs) with key stakeholders in 16 villages, and 19 IDIs at district and state level, all conducted a year after project completion, and on project and secondary documents. This introduces limitations to the rigour of the assessment, in particular further down the ‘results chain’ to child protection outcomes, compelling the evaluation team to rely more heavily on triangulation, induction and judgement. However, despite these limitations, the timeline of the evaluation provided an opportunity to assess the sustainability of the project. Key findings Relevance The Cotton Corridors project is highly relevant, both to global, national and state priorities, and to the needs of children in Raichur and Kurnool districts. The project design closely aligns with current (and past) international and national priorities to protect the rights of children. The project engaged the government to work together to address child labour and child marriage in the project areas. In addition, the project drew on the institutional strengths of each district it worked in and engaged key stakeholders in child protection. © Oxford Policy Management iv UNICEF Cotton Corridors Project Final Evaluation report Effectiveness We assessed effectiveness by developing a detailed TOC for how project activities and strategies should lead to improvements in child protection, and then seeking evidence and the perceptions of key stakeholders on whether this theory held in practice by relying on contribution analysis methods. Overall, the evidence on changes in child protection outcomes during the project is positive. The balance of evidence from different sources suggests