EVALUATION OF THE CARD AND UNICEF CASH TRANSFER PILOT
PROJECT FOR PREGNANT WOMEN AND CHILDREN IN CAMBODIA
Annexes – Volume II September 2017 – March 2018
Cambodia EVALUATION REPORT EVALUATION T
EVALUATION REPORT
APRIL 2018 Photo Credit © UNICEF Cambodia/2012/Andy Brown
EVALUATION REPORT EVALUATION OF THE CARD
AND UNICEF CASH TRANSFER
PILOT PROJECT FOR PREGNANT WOMEN AND CHILDREN IN CAMBODIA
Annexes – Volume II September 2017 – March 2018 Cambodia
Authors: Ashish Mukherjee (Team Leader), Kriti Gupta on behalf of IPE Global Limited, New Delhi, India and Dr. Chey Tech on behalf of Dynamic Alliance Consulting
(DAC) Group Co., Ltd, Cambodia
Submitted to the Council for Agricultural and Rural Development and UNICEF Cambodia Country Office on 31 March 2018
APRIL 2018 iii
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
EVALUATION OF THE CARD AND UNICEF CASH TRANSFER PILOT PROJECT FOR PREGNANT WOMEN AND CHILDREN IN CAMBODIA: Annexes (Volume II)
© United Nations Children’s Fund, Phnom Penh, 2018 United Nations Children’s Fund P.O. Box 176 Phnom Penh, Cambodia 12201 [email protected]
April 2018
UNICEF Cambodia produces and publishes evaluation reports to fulfill a corporate commitment to transparency. The reports are designed to stimulate the free exchange of ideas among those interested in the study topic and to assure those supporting UNICEF work that it rigorously examines its strategies, results and overall effectiveness.
The evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia was prepared by Ashish Mukherjee and Kriti Gupta on behalf of IPE Global Limited with contribution from Dr. Chey Tech, Dynamic Alliance Consulting (DAC) Group Co., Ltd, Cambodia. The evaluation was jointly commissioned by the Council for Agricultural and Rural Development and UNICEF Cambodia and managed by the evaluation management team comprising Erica Mattellone, Evaluation Specialist (UNICEF Cambodia); Phaloeuk Kong, M&E Officer (UNICEF Cambodia); Kimsong Chea, Social Policy Specialist (UNICEF Cambodia) and Sambo Pheakdey, Chief of Pension Department (Ministry of Economy and Finance), assisted by Cody Minnich, Evaluation Intern (UNICEF Cambodia) and Elizabeth Fisher, Evaluation Intern (UNICEF Cambodia).
It was supported by Reference Group members H.E. Sann Vathana, Deputy Secretary General (Council for Agricultural and Rural Development); Maki Kato, Chief of Social Inclusion and Governance (UNICEF Cambodia); Sophannha Chhour, Director of Social Welfare Department (Ministry of Social Affairs, Veterans, and Youth Rehabilitation); Betina Ramirez Lopez, Social Protection Technical Officer (International Labor Organization (ILO) Cambodia); Jillian Popkins, Chief of Social Policy (UNICEF China); Rim Nour, Consultant (UNICEF Regional Office for East Asia and the Pacific (EAPRO)) and Som Sophorn, Chief of Zone Office (UNICEF Siem Reap Zone Office). Further, the Regional Evaluation Adviser, Riccardo Polastro, (UNICEF EAPRO), and Evaluation Officer, Hiroaki Yagami (UNICEF EAPRO) provided guidance and oversight throughout.
The purpose of this report is to facilitate exchange of knowledge among UNICEF personnel and its partners. The contents do not necessarily reflect the policies or views of UNICEF. The text has not been edited to official publication standards and UNICEF accepts no responsibility for error. The designations in this publication do not imply an opinion on the legal status of any country or territory, or of its authorities, or the delimitation of frontiers.
The copyright for this report is held by the United Nations Children’s Fund. Permission is required to reprint, reproduce, photocopy or in any other way cite or quote from this report in written form. UNICEF has a formal permission policy that requires a written request to be submitted. For non-commercial uses, permission will normally be granted free of charge. Please write to UNICEF Cambodia to initiate a permission request.
ii
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Title: EVALUATION REPORT FOR THE EVALUATION OF THE CARD AND UNICEF CASH TRANSFER PILOT PROJECT FOR PREGNANT WOMEN AND CHILDREN IN CAMBODIA Geographic Region of the Pilot: Prasat Bakong District, Siem Reap Province, Cambodia Timeline of the Evaluation: September 2017 – March 2018 Date of the Report: 31 March 2018 Country: Cambodia Evaluators: Ashish Mukherjee and Kriti Gupta for IPE Global Limited, India; and Dr. Chey Tech for Dynamic Alliance Consulting (DAC) Group Co., Ltd, Cambodia Name of the Organization The Council for Agricultural and Rural Development Commissioning the Evaluation: (CARD) and United Nations Children’s Fund (UNICEF) in Cambodia
iii
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
ABBREVIATIONS AND ACRONYMS
4Ps Pantawid Pamilyang Philipino Project ADB Asian Development Bank ADRA Adventist Development and Relief Agency ANC ante-natal check-up AMK AMK Microfinance Institution Plc BCC behaviour change communication BCG Bacillus Calmette–Guérin BDT Bangladeshi taka BFP Bolsa Familia Programme BLT Bantuan Langsung Tunai BPS Badan Pusat Statistik CARD Council for Agricultural and Rural Development CBT community-based targeting CC commune council CCT conditional cash transfer CCWC Commune Committee for Women and Children CDHS Cambodia Demographic and Health Survey CEDAC Community Economic Development Assistance Corporation CSG Child Support Grant CSO Civil Society Organization CT cash transfer CWD Child Welfare Department DCWC District Committee for Women and Children DDC District Development Committee DFID Department for International Development EAPRO Regional Office for East Asia and the Pacific EMT evaluation management team FGD focus group discussion GEROS Global Evaluation Reports Oversight System GOI Government of Indonesia HC Health Centre HEF health equity fund HH household IDPoor identification of poor households programme ILO International Labour Organization IP3 3-year Implementation Plan iv
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
ISO International Organization for Standardization IT information technology KAP knowledge, attitudes and practices KAPE Kampuchean Action for Primary Education KII key informant interview LBM Línea de Bienestar Mínimo (minimum well-being line) MEF Ministry of Economy and Finance M&E monitoring and evaluation MFI microfinance institution MIS management information system MoEYS Ministry of Education, Youth and Sport MoH Ministry of Health MoP Ministry of Planning MoSVY Ministry of Social Affairs, Veteran and Youth Rehabilitation MMR Measles, Mumps, and Rubella NSCB National Statistical Coordination Board NCDD-S National Committee for Sub-National Democratic Development Secretariat NGO non-governmental organization NSPPF National Social Protection Policy Framework OECD/ DAC Organisation for Economic Co-operation and Development/ Development Assistance Committee PhP Philippine Peso PMT Proxy Means Test POS point of sale PKH Program Keluarga Harapan QMS quality management system RGC Royal Government of Cambodia Rp Indonesian rupia SAE Small Area Estimates SAM Severe Acute Malnutrition SEDESOL Secretaria de Desarollo Social SNC Safety Net Beneficiary Cell SNDD Sub-National Democratic Development SP social protection SPA Safety Net Program Assistant ToC theory of change ToR terms of reference
v
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
UNDP United Nations Development Programme UNEG United Nations Evaluations Group UNICEF United Nations Children’s Fund US$ United States Dollar VHSG village health support group WBG World Bank Group ZAR South African Rand
vi
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
TABLE OF CONTENTS ACRONYMS AND ABBREVIATIONS ...... iii Annex 1. Terms of Reference ...... 1 Annex 2. Poverty Estimates for Cambodia ...... 13 Annex 3. Nutrition and Health indicators ...... 16 Annex 4. Analysis of IDPoor Data for Select Provinces ...... 18 Annex 5. Evidence of Selected Cash Transfer Projects ...... 20 Annex 6. Commune-wise IDPoor Data ...... 30 Annex 7. Transfer Amount, Co-responsibilities for Bonus Transfer ...... 31 Annex 8. Cash Transfer Pilot Implementation Steps ...... 33 Annex 9. Stakeholder Analysis ...... 34 Annex 10. Evaluation Matrix, Indicative Questions to Guide Development of Data Collection Tools and Analytical Framework for the Evaluation ...... 38 Annex 11. List of Documents for Review ...... 44 Annex 12. List of Activities and People Met during Scoping Visit ...... 46 Annex 13. Key Stakeholder List, Data Collection Methods and Data Collection Tools ...... 47 Annex 14. Critical Cost-effectiveness Drivers ...... 102 Annex 15. Ethics and United Nations Evaluation Guidelines ...... 103 Annex 16. Health Centre Data ...... 104 Annex 17. Comparison of Survey Findings to Baseline ...... 109 Annex 18. Trainings Conducted during the Pilot Project ...... 110 Annex 19. Break up of Costs of the CARD – UNICEF Cash Transfer Pilot ...... 111 Annex 20. Indicative Parameters for Consideration for Phased Scaling-up ...... 112 Annex 21. Indicative Areas of Monitoring ...... 113 Annex 22. Grievance Redress Mechanism of Bangladesh’s Income Support Programme ...... 114 Annex 23. Internal Quality Review Process ...... 115 Annex 24. Team Composition ...... 116
vii
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 1. Terms of Reference
IMPROVING CASH-BASED INTERVENTIONS: EVALUATION OF THE CARD AND UNICEF CASH TRANSFER PILOT PROJECT FOR PREGNANT WOMEN AND CHILDREN IN CAMBODIA
Terms of Reference
UNICEF CAMBODIA COUNTRY OFFICE 08 JUNE 2017
1
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
1. INTRODUCTION
1. This Terms of Reference (ToR) document outlines the purpose and scope of a formative, learning-oriented, Evaluation of the Council for Agricultural and Rural Development (CARD) and the United Nations Children’s Fund (UNICEF) Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia. The evaluation seeks to examine the cash delivery from May 2016 until its current implementation, but it will also look at the inception phase thus covering the project design and targeting from 2013 to 2016. To this end, the ToR presents methodological options and operational modalities for an institutional contract of a team of two evaluation consultants (one international and one national). This independent evaluation is commissioned by UNICEF Cambodia, on behalf of the Ministry of Economy and Finance (MEF) and CARD. UNICEF is hence looking for institutions with deep commitment to, and strong background in, the evaluation of social cash-based interventions. Findings and recommendations from this evaluation will inform the design of the national cash transfer project for nationwide roll-out lead by the Royal Government of Cambodia (RGC).
2. BACKGROUND AND RATIONALE
2.1 COUNTRY CONTEXT
2. Following a return to political stability in mid-2014 after a year-long political deadlock, Cambodia is continuing to pursue its transitional approach to economic and social development: gradually promoting greater decentralisation, moving the focus of planning from rehabilitation to inclusive growth, shifting from establishing systems and developing capacity to more of a focus on the efficient performance of systems and use of capacity. With an annual average Gross Domestic Production (GDP) growth rate of more than 7 per cent since 2011, Cambodia is now a low- middle-income country and has fully achieved economic integration into the Association of South East Asian Nations (ASEAN). This brings a related challenge of reduced inflow of external financial resources and the need to raise more domestic resources and forge stronger partnerships within the region and with other developing nations. Cambodia has a large, very young population of children and adolescents; 45 per cent of the population is aged 19 years or younger. More than 11 per cent of the total population is under 5 years of age. 3. Economic growth has contributed to a steep decline in poverty, from 47.2 per cent in 2007 to 18.6 per cent in 2012, with around 3 million Cambodians living in poverty. Of these, 90 per cent live in rural areas. This recent economic growth has not benefited all, and significant geographic disparities exist, with poverty rates ranging from around 15 per cent in Phnom Penh to up to 37 per cent in the mostly rural north-east provinces. Of the estimated total population of 15.3 million, around 40 per cent live just above the poverty line and are highly vulnerable to small economic changes, natural disasters and other shocks. Similarly, Cambodia’s human development indicators remain low, with a Human Development Index (HDI) ranking of 0.52 versus an average of 0.67, for the rest of the East Asia and the Pacific region.1 4. Furthermore, nutrition poses a particular challenge for current and future human development in Cambodia with one in every three children under-five being stunted (low height for age). This negatively affects cognitive and physical development and it is likely to later affect reaching full potential to be a productive adult. Malnutrition is particularly acute among the poor, with children from the poorest quintile being more than twice as likely to be stunted compared to children in the richest quintile. This is due to the inability of many poor households to make use of existing health and nutrition services, inadequate food consumption, inappropriate hygiene practices and unsuitable feeding practices which cause frequent preventable diseases, especially among young children. To this end, the RGC had introduced several measures aimed at improving the coverage and quality of health and nutrition services through the public health system. However, with the exception of Health Equity Fund, supply-side interventions seem unable to address
1 http://www.worldbank.org/en/country/cambodia/overview
2
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
barriers to access faced by poor households and utilisation of those services among the poor is deemed to be low.
2.2 THE CAMBODIA SOCIAL CASH TRANSFER PILOT PROJECT
5. CARD, supported by UNICEF, designed a pilot cash transfer project in 2013 targeting pregnant women and children under-five living in poverty. The pilot project has been implemented within the existing government structure – without creating any external-funded posts – to test whether this type of interventions could be implemented by the Government. The pilot targets poor women and young children under five, in order to improve maternal health, nutrition and human capital development. The first cash delivery to the women started in May 2016. 6. The objectives of the pilot are as follows: • Enable women with cash transfer or income support to utilize basic social services and to improve their diet intake for reduction of chronic malnutrition of children under-five and pregnant mothers living in poverty; and • Test the implementation of the cash transfer only using the existing government structure to prepare for future roll-out by the RGC, including (a) at the institutional level, to test the capacity of central and local authorities for implementation and coordination of social protection projects and to oversee community-level supply-side services, in line with the legislative framework for Sub-National Democratic Development (SNDD) and the current three year Implementation Plan (IP3); and (b) at the operational level, to develop the overall design of the operations cycle and test the effectiveness of the proposed mechanisms for women enrolment, case management and community participation, benefit payments, monitoring and reporting, and to test the linkages with complementary supply-side activities such as education/communication sessions, services provided by Health Centres and possibly future related social protection services. 7. The cash transfer project is piloted in Prasat Bakong district in Siem Reap province, under implementation of the District Administrator in close coordination with the CARD. Beneficiaries were identified as all poor children and women registered as IDPoor I and II in the Prasat Bakong district. According to CARD, this list includes 1,298 people composed of 69 pregnant women and 1,179 mothers with a child under five years old. 8. AMK Microfinance is currently in charge for delivering the cash to designated household receivers (usually the mother or the female guardian). Designated cash receivers of each household opened an individual account with AMK Microfinance to which a transfer is made every two months. The transfer amount is USD$ 5 per month per individual woman and child, with bonus upon completion of co-responsibilities (i.e., prenatal check-up, institutional delivery and postnatal care, growth monitoring and vaccination, attendance of education or communication sessions on early childhood development). The total maximum benefit per child amounts to USD$ 90 per year. 9. In practice, cash transfer recipients receive payment from the AMK Microfinance representative at designated pay points by showing the national identification card and ATM card. Distribution points were identified within maximum five kilometres from recipients (on average two distribution points per commune). Moreover, implementation on the ground is done by the Commune Councils. Below are the list of the roles and functions in relation to cash transfer implementation by the Commune Councils: • Raise awareness about the cash transfer pilot among members of the community; • Identify and enrol women, compile/receive complaints and ensure primary data collection of lists of compliance from Health Centres; • Promote birth registration; • Support community-based education through co-facilitation of education sessions with Heath Centre personnel; • Monitor the attendance of women at community-based education sessions;
3
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
• Verify co-responsibilities to inform bonus payment; and • Through village chiefs, inform the cash transfer recipients on upcoming payment schedule. 10. This evaluation is expected to be formative (learning-oriented) in nature and to produce credible, reliable and useful evidence from the cash transfer pilot project – what is working, what is not working, how and why – to inform the design of the national cash transfer project, as indicated in the recently approved National Social Protection Policy Framework (NSPPF) 2016-2025.2
3. PURPOSE, OBJECTIVES AND SCOPE OF WORK
3.1 PURPOSE
11. The primary purpose of this independent (formative) evaluation is to foster learning and improvement within the cash transfer pilot project. With a view of RGC commitment to implement a cash transfer for pregnant women and children as described in the NSPPF, the evaluation will therefore assess both project design and implementation mechanism, (i.e., to what extent the pilot project has been implemented as intended), assess its programmatic effectiveness, women’s preferences and satisfaction, and what adjustments are required moving forward. The evaluation will also compare the CARD-UNICEF cash transfer pilot with other cash transfer interventions. 12. The primary users of the evaluation include MEF (General Department of Financial Industry and General Department of Budget), CARD, Ministry of Social Affairs, Veterans and Youth Rehabilitation (MoSVY) and UNICEF. Secondary users include other agencies involved in cash transfer programming in Cambodia, civil society organizations, development partners, and UNICEF’s Regional Office for East Asia and the Pacific (EAPRO) and others.
3.2 OBJECTIVES
13. The objectives of the evaluation include the following: • Analyse the extent to which the cash transfer project has been appropriately designed, efficiently and effectively implemented (including targeting and coverage, inclusion and exclusion errors, cash distribution mechanism, financial management, reporting compliance, data management, etc.) and its cost-effectiveness (e.g., analysis of administrative costs, etc.); • Understand how women (and families) have used the money provided, their satisfaction, adequacy of the transfer level, and the extent to which the spending of the money translated (or not) into benefits for children; • Assess the institutional capacity at national and sub-national level for management and implementation of the CARD-UNICEF cash transfer project, identifying key gaps and bottlenecks in relation to the cash transfer pilot project life-cycle; and • Assess the strengths and weaknesses of the CARD-UNICEF cash transfer pilot project versus other cash transfer interventions in Cambodia (including its cost-effectiveness).
14. Evaluation evidence will be judged using modified Organisation for Economic Co-operation and Development (OECD)/Development Assistance Committee (DAC) criteria of relevance, efficiency, effectiveness and sustainability, as well as equity, gender equality and human rights considerations. Key evaluation questions include the following: Relevance of the project design and approach, considering: • To what extent did the selection of targeted pregnant women and children under five complement the targeting of other social projects to reach to the worst-off and most
2 https://www.UNICEF.org/cambodia/National_Social_Protection_Strategy_for_the_Poor_and_Vulnerable_Eng.pdf
4
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
vulnerable women? Were there any gaps in relation to targeting and coverage of the pilot project? • To what extent was the choice to use conditional cash transfer rather than unconditional justified with regards to the needs among pregnant women and children, availability and quality of services, capacity of local government and service providers, and government preferences? • To what extent was the size and regularity of the cash transfer adequate? Were the different needs of pregnant women and under five children met within the objectives of the pilot project? Efficiency of the delivery mechanism, considering: • How well was the delivery process managed, considering the time and resources at each stage of implementation and coordination between UNICEF, CARD, sub-national administrations, and AMK Microfinance? • How cost-effective was the cash transfer pilot compared to other modalities and mechanisms and what potential is there for efficiency savings at all stages? Effectiveness of the project, including better consumption patterns, nutrition and care of newborn children and children under-five: • To what extent and how was the cash transfer used for better consumption of under-five and other children in the household, comparatively with adults (considering food quality, quantity and diversity)? How has the cash transfer supported nutrition and care of new-born children, and children under-five? Were there any unintended results? • How effective were the complementary community-based education sessions from both implementers and women’s perspectives? • How well did the financial management system establish including reporting compliance? • How well did the monitoring of co-responsibilities and other reporting mechanisms function, including the role of Health Centres, Commune Councils and village chiefs? Sustainability in terms of the cash transfer pilot, considering: • How can the cash transfer pilot be replicated at the national level given the current capacities at the national and sub-national levels? To what extent can the major capacity gaps and bottlenecks at national and sub-national levels be overcome during the life-cycle of this project? 15. One of the key tasks to be initiated at the proposal stage will be to interrogate these questions and criteria and determine if all key issues have been given due prominence. Bidders are required to propose appropriate evaluation criteria (e.g., OECD/DAC criteria for evaluating development projects, including sub-criteria such as equity, gender equality, human rights). Improvements and/or refinements to the draft questions may be offered at the proposal stage. However, the expectation is that the inception process will yield the final set of questions.3
3 The actual final decisions on the detailed questions will be taken in the inception phase, based on the following principles: 1. Importance and priority: the information should be of a high level of importance for the various intended audiences of the evaluation; 2. Usefulness and timeliness: the answer to the questions should not be already well known or obvious, additional evidence is needed for decisions; 3. Answerability and realism: all the questions can be answered using available resources (budget, personnel) and within the appropriate timeframe; data and key informants are available and accessible, and performance standards or benchmarks exist to answer the questions; and 4. Actionability: the questions will provide information which can lead to recommendations that be acted upon to make improvements.
5
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
3.3 SCOPE OF WORK
16. The evaluation will not be an impact evaluation, but it will cover the inception and the implementation phase of the cash transfer pilot, which began implementation in May 2016 and it is expected to run until the end of 2017 in the Prasat Bakong district in Siem Reap province. The evaluation should include pregnant women and mothers with a child under five years old, and put an emphasis on children who benefited from the intervention.
4. EVALUATION APPROACH AND METHODOLOGY4
17. Based on the objectives of the evaluation, this section indicates possible approaches, methods, and processes for the evaluation. Methodological rigor will be given significant consideration in the assessment of proposals. Hence bidders are invited to interrogate the approach and methodology proffered in the ToR and improve on it, or propose an approach they deem more appropriate. Bidders are encouraged to also demonstrate methodological expertise in evaluating social cash transfer projects. 18. It is expected that the evaluation will employ a mixed methods approach drawing on key project documents and the monitoring framework for guidance. The evaluation should also be situated within current debate about the use of unconditional cash transfer interventions and social protections projects to improve the welfare of women and children5, and throughout it should consider issues of equity, gender equality and human rights. 19. The timing of the evaluation is such that it will take mainly a formative approach, identifying and assessing its project effectiveness and women’s preferences and satisfaction with the project to date to inform design of a national cash transfer project as envisaged in the NSPPF. To this end, the evaluation will provide continuous and rapid feedback to primary users in the course of the evaluation process. 20. At minimum, the evaluation will draw on the following methods: • Desk review of project documents and other relevant data; • Review and analysis of secondary quantitative data; • Key Informant Interviews (KIIs); • Focus Group Discussions (FGDs); • Case studies of women participating in the project; • Cost-effectiveness analysis; and • Surveys. The data collected should be disaggregated by sex, age, etc. where relevant and focus on both the implementers (incl. district and commune focal points, village chiefs, AMK Microfinance, and other implementing partners) as well as pregnant women and mothers of children under-five. 21. Sampling of KIIs and FGDs should be done in consultation with MEF, CARD and UNICEF. A purposively selected sample of villages in the Prasat Bakong district should be taken, considering a balance of criteria such as socioeconomic indicators, remoteness, ethnicities, etc. 22. Baseline data will be provided based on the survey conducted for the selected beneficiaries. Additionally, secondary data sources such as IDPoor and DHS can be used. Other secondary data specific to the project will be available from the CARD, such as project monitoring reports and AMK Microfinance reports.
4 The proposed methodology is just indicative, and based on internal experience in conducting similar evaluations. The will be a need to develop a detailed design, analytical methods and tools during the inception phase based on additional literature review and in consultation with CARD and UNICEF. 5 UNICEF Social Protection Strategic Framework: https://www.UNICEF.org/socialprotection/framework/files/UNICEF_SPSFramework_whole_doc(1).pdf
6
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
23. Likewise, conventional ethical guidelines are to be followed during the evaluation. Specific reference is made to the revised UNEG Norms and Standards for Evaluation in the UN System, as well as to the UNICEF’s revised Evaluation Policy, and the UNICEF Procedure for Ethical Standards in Research, Evaluation and Data Collection and Analysis and UNICEF’s Evaluation Reporting Standards.6 Good practices not covered therein are also to be followed. Any sensitive issues or concerns should be raised with the evaluation management team as soon as they are identified.
5. MANAGEMENT AND CONDUCT OF THE EVALUATION 5.1 EVALUATION MANAGEMENT STRUCTURE
24. The evaluation will be conducted by an external evaluation team to be recruited by UNICEF Cambodia, on behalf of MEF and CARD. The evaluation team will operate under the supervision of an evaluation management team comprised of an Evaluation Specialist, an M&E Officer and a Social Policy Specialist at UNICEF Cambodia. The evaluation management team will be responsible for the day-to-day oversight and management of the evaluation and for the management of the evaluation budget as well as to assure the quality and independence of the evaluation and to guarantee its alignment with UNEG Norms and Standards and Ethical Guidelines, to provide quality assurance checking that the evaluation findings and conclusions are relevant and recommendations are implementable, and to contribute to the dissemination of the evaluation findings and follow-up on the management response. The final report will also be approved by the Country Representative at UNICEF Cambodia. 25. A reference group will be established, bringing together the Chief of Social Inclusion and Governance at UNICEF Cambodia, and representatives from MEF (General Department of Financial Industry, General Depart of Budget, General Department of Macroeconomic and Public Finance Policy), CARD (Deputy Secretary General), and Ministry of Social Affairs, Veterans and Youth Rehabilitation (Child Welfare Department), CSO representative and eventually other subject-matter experts. The reference group will have the following role: contribute to the preparation and design of the evaluation, including providing feedback and comments on the inception report and on the technical quality of the work of the consultants; provide comments and substantive feedback to ensure the quality – from a technical point of view – of the draft and final evaluation reports; assist in identifying internal and external stakeholders to be consulted during the evaluation process; participate in review meetings organized by the evaluation management team and with the evaluation team as required; play a key role in learning and knowledge sharing from the evaluation results, contribute to disseminating the findings of the evaluation and follow-up on the implementation of the management response. 5.2 EVALUATION TEAM PROFILE
26. The evaluation will be conducted by engaging an institution. The proposed team consists of one (1) international senior-level consultant (Team Leader) to conduct the evaluation that will be supported by at least one (1) national consultant (Team Member/Technical Expert). 27. The Team Leader should bring the following competences: • Having extensive evaluation experience (at least 15 years) with an excellent understanding of evaluation principles and methodologies, including capacity in an array of qualitative and quantitative evaluation methods, and UNEG Norms and Standards. • Having extensive experience on social cash transfer interventions – planning, implementing, managing or monitoring and evaluation. • Holding an advanced university degree (Masters or higher) in international development, public policy or similar, including sound knowledge of policy and systemic aspects; familiarity
6 Please refer to: http://www.UNICEF.org/evaluation
7
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
with social protection projects. • Bringing a strong commitment to delivering timely and high-quality results, i.e., credible evaluations that are used for improving strategic decisions. • Having in-depth knowledge of the UN’s human rights, gender equality and equity agendas. • Having a strong team leadership and management track record, as well as excellent interpersonal and communication skills to help ensure that the evaluation is understood and used. • Specific evaluation experience of cash programming is strongly desired, but is secondary to a strong mixed-methods evaluation background, so long as the cash transfer expertise of the other team member (see below) is harnessed to ensure the team’s collective understanding of issues relating to cash programming from a UN or NGO perspective. • Previous experience of working in an East Asian context is desirable, together with an understanding of the Cambodian context and cultural dynamics. • The Team Leader must be committed and willing to work independently, with limited regular supervision; s/he must demonstrate adaptability and flexibility, client orientation, proven ethical practice, initiative, concern for accuracy and quality. • S/he must have the ability to concisely and clearly express ideas and concepts in written and oral form as well as the ability to communicate with various stakeholders in English. 28. The Team Leader will be responsible for undertaking the evaluation from start to finish, for managing the evaluation, for the bulk of data collection, analysis and consultations, as well as for report drafting in English and communication of the evaluation results. 29. One (1) national Team Member/Technical Expert: • Holding advanced university degrees (Masters-level) in international development, public policy or similar. • Hands-on experience in collecting and analysing quantitative and qualitative data, but this is secondary to solid expertise in cash transfer interventions related to social protection. • Strong expertise in equity, gender equality and human rights based approaches to evaluation and expertise in data presentation and visualisation. • Be committed and willing to work in a complex environment and able to produce quality work under limited guidance and supervision. • Having good communication, advocacy and people skills and the ability to communicate with various stakeholders and to express concisely and clearly ideas and concepts in written and oral form. • Excellent Khmer and English communication and report writing skills.
30. The Team Member will play a major role in data collection, analysis and presentation, and preparation of the debriefings and will make significant contributions to the writing of the main evaluation report. 31. The evaluation team is expected to be balanced with respect to gender to ensure accessibility of both male and female informants during the data collection process. Back-office support assisting the team with logistics and other administrative matters is also expected. It is vital that the same individuals that develop the methodology for the RFPS will be involved in conducting the evaluation. In the review of the RFPS, while adequate consideration will be given to the technical methodology, significant weighting will be given to the quality, experience (CV’s and written samples of previous evaluations) and relevance of individuals who will be involved in the evaluation. 5.3 EVALUATION DELIVERABLES
32. Evaluation products expected for this exercise are: a) An inception report (in English), including a summary note in preparation for data collection (in both English and Khmer);
8
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
b) A report of the initial evaluation findings from primary data collection (in English), including a desk review analysis and a PowerPoint presentation to facilitate a stakeholder consultation exercise; c) A draft and final report (in English) that will be revised until approved (incl. a complete first draft to be reviewed by the evaluation management team and UNICEF; a second draft to be reviewed by the reference group and Regional Evaluation Adviser within UNICEF EAPRO, and a penultimate draft); d) A PowerPoint presentation (in both English and Khmer) to be used to share findings with the reference group and for use in subsequent dissemination events; and e) A four-page executive summary (in both English and Khmer) that is distinct from the executive summary in the evaluation report and it is intended for a broader, non-technical and non-UNICEF audience. 33. Other interim products are: a) Minutes of key meetings with the evaluation management team and the reference group; and b) Presentation materials for the meetings with the evaluation management team and the reference group. These may include PowerPoint summaries of work progress and conclusions to that point.
34. Outlines and descriptions of each evaluation products are meant to be indicatives, and include: • Inception report: The inception report will be key in confirming a common understanding of what is to be evaluated, including additional insights into executing the evaluation. At this stage evaluators will refine and confirm evaluation questions, confirm the scope of the evaluation, further improve on the methodology proposed in the ToR and their own evaluation proposal to improve its rigor, as well as develop and validate evaluation instruments. The report will include, among other elements: i) evaluation purpose and scope, confirmation of objectives and the main themes of the evaluation; ii) evaluation criteria and questions, final set of evaluation questions, and evaluation criteria for assessing performance; iii) evaluation methodology (i.e., sampling criteria), a description of data collection methods and data sources (including a rationale for their selection), draft data collection instruments, for example questionnaires, with a data collection toolkit as an annex, an evaluation matrix that identifies descriptive and normative questions and criteria for evaluating evidence, a data analysis plan, a discussion on how to enhance the reliability and validity of evaluation conclusions, the field visit approach, a description of the quality review process7 and a discussion on the limitations of the methodology; iv) proposed structure of the final report; v) evaluation work plan and timeline, including a revised work and travel plan; vi) resources requirements (i.e., detailed budget allocations, tied to evaluation activities, work plan) deliverables; v) annexes (i.e., organizing matrix for evaluation questions, data collection toolkit, data analysis framework); and vi) an evaluation briefing note for external communication purposes. The inception report will be 15-20 pages in length (excluding annexes), or approximately 15,000 words, and will be presented at a formal meeting of the reference group. • Initial evaluation findings: This report will present the initial evaluation findings from primary data collection, comprising the desk-based document review and analysis of the cash transfer pilot. The report developed prior to the first drafts of the final report should be 10 pages, or about 8,000 words in length (excluding annexes, if any), and should be accompanied by a PowerPoint presentation that can be used for validation with key stakeholders. • Final evaluation report: The report will not exceed 40 pages, or 25,000 words, excluding the executive summary and annexes.
7 UNICEF has instituted the Global Evaluation Report Oversight System (GEROS), a system where final evaluation reports are quality assessed by an external company against UNICEF/UNEG Norms and Standards for evaluation reports. The evaluation team is expected to reflect on and conform to these standards as they write their report. The team may choose to share a self-assessment based on the GEROS with the evaluation manager.
9
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
• PowerPoint presentation (both in English and Khmer): Initially prepared and used by the evaluation team in their presentation to the reference group, a standalone PowerPoint will be submitted to the evaluation management team as part of the evaluation deliverables. • An evaluation briefing note, data and a four-page executive summary for external users will be submitted to the evaluation management team as part of the evaluation deliverables. • Reports will be prepared according to the UNICEF Style Guide and UNICEF Brand Toolkit (to be shared with the winning bidder) and UNICEF standards for evaluation reports as per GEROS guidelines (referenced before). • The first draft of the final report will be received by the evaluation management team and UNICEF who will work with the team leader on necessary revisions. The second draft will be sent to the reference group for comments. The evaluation management team will consolidate all comments on a response matrix, and request the evaluation team to indicate actions taken against each comment in the production of the penultimate draft. 35. Bidders are invited to reflect on each outline and effect the necessary modification to enhance their coverage and clarity. Having said so, products are expected to conform to the stipulated number of pages where that applies. 36. An estimated budget has been allocated for this evaluation. As reflected in Table 1, the evaluation has a timeline of four months from August to November 2017. Adequate effort should be allocated to the evaluation to ensure timely submission of all deliverables, approximately 10 weeks on the part of the evaluation team. Table 1: Proposed evaluation timeline8
ACTIVITY DELIVABLE TIME ESTIMATE RESPONSIBLE PARTY 1. INCEPTION, DOCUMENT REVIEW AND 4 weeks, concurrent ANALYSIS (Aug, 2017) 1. Inception meeting by Skype with evaluation Meeting minutes Week 1 Evaluation team, management team evaluation management team 2. Inception visit (incl. initial data collection and Draft inception Week 2 Evaluation team desk review; development of evaluation report matrix, methodology and work plan, data collection material, drafting of the inception report) 3. Present draft inception report to the PowerPoint Week 3 Evaluation team, reference group presentation evaluation management team, reference group 4. Receive inception report and feedback to Evaluation Week 3 Evaluation evaluation team commenting management team, matrix reference group 5. Present inception report, confirm planning for Final inception Week 4 Evaluation team, field visit report evaluation management team, reference group 2. DATA COLLECTION 3 weeks, consecutive (Sept, 2017) 1. Pilot data collection tools and conduct field- - Weeks 5-7 Evaluation team based data collection
8 Please note that the timing of the data collection may change depending on the possibility of carrying out KIIs and FGDs and other contextual factors.
10
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
ACTIVITY DELIVABLE TIME ESTIMATE RESPONSIBLE PARTY 2. Prepare initial evaluation findings report and Initial evaluation Week 7 Evaluation team, prepare presentation for validation workshop findings report evaluation to validate data collection results (incl. desk management team, review), reference group PowerPoint presentation, meeting minutes 3. REPORTING AND COMMUNICATION OF 6 weeks, consecutive RESULTS (Oct – Nov, 2017) 1. Prepare and submit first draft of evaluation Draft report Week 8 Evaluation team report 2. Receive first draft and feedback to evaluation Evaluation Week 9 Evaluation team commenting management team matrix 3. Prepare and submit second draft of Draft report Week 10 Evaluation team evaluation report 4. Receive second draft and feedback to Evaluation Weeks 11-12 Evaluation evaluation team commenting management team, matrix reference group 5. Prepare and submit penultimate draft of Draft report Week 13 Evaluation team evaluation report 6. Submit and present final report to reference Final report, Week 14 Evaluation team, group and prepare presentation and other executive evaluation materials summary, management team, PowerPoint reference group presentation, meeting minutes
6. CONTENT OF THE PROPOSERS’ TECHNICAL PROPOSAL
37. The written technical proposal will be submitted in hard copy and electronic (PDF) format and include the following elements, as a minimum requirement: a) Request for proposals for services form (provided above). b) Presentation of the bidding institution or institutions if a consortium (maximum two institutions will be accepted as part of the consortium), including: • Name of the institution; • Date and country of registration/incorporation; • Summary of corporate structure and business areas; • Corporate directions and experience; • Location of offices or agents relevant to this proposal; • Number and type of employees; • In case of a consortium of institutions, the above listed elements shall be provided for each consortium members in addition to the signed consortium agreement; and • In case of a consortium, one only must be identified as the organization lead in dealing with UNICEF. c) Narrative description of the bidding institution’s experience and capacity in the following areas: • Evaluation of cash transfer interventions; • Process evaluation of social protection interventions, ideally implemented by government institutions; • Previous assignments in developing countries in general, and related to social protection projects, preferably in East Asia; and • Previous and current assignments using UNEG Norms and Standards for evaluation.
11
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
d) Relevant references of the proposer (past and on-going assignments) in the past five years. UNICEF may contact references persons for feedback on services provided by the proposers. e) Samples or links to samples of previous relevant work listed as reference of the proposer (at least three), on which the proposed key personnel directly and actively contributed or authored. f) Methodology. It should minimize repeating what is stated in the ToR. There is no minimum or maximum length. If in doubt, ensure sufficient detail. g) Work plan, which will include as a minimum requirement the following: • General work plan based on the one proposed in the ToR, with comments and proposed adjustments, if any; and • Detailed timetable by activity (it must be consistent with the general work plan and the financial proposal). h) Evaluation team: • Summary presentation of proposed experts; • Description of support staff (number and profile of research and administrative assistants etc.); • Level of effort of proposed experts by activity (it must be consistent with the financial proposal); and • CV of each expert proposed to carry out the evaluation. 38. Please note that the duration of the assignment will be from August to November 2017, and it is foreseen that the Team Leader and the Team Member will devote roughly half of their time to the evaluation. The presence of a conflict of interest of any kind (e.g., having worked for or partnered with CARD or UNICEF on the cash transfer pilot project in the design or implementation phase will automatically disqualify prospective candidates from consideration).
7. CONTENT OF THE FINANCIAL PROPOSAL
39. The financial proposal must be fully separated from the technical proposal. The financial proposal will be submitted in hard copy. Costs will be formulated in USD$ and free of all taxes. It will include the following elements as a minimum requirement: a) Overall price proposal; and b) Budget by phase and by cost category (incl. personnel costs, transportation, DSA, translation services, report editing, and overheads).
8. PAYMENT SCHEDULE
40. Unless the proposers propose an alternative payment schedule, payments will be as follows: a) Approved inception report: 25% of the contractual amount; b) Approved initial evaluation findings report: 30% of the contractual amount; c) Approved final report: 30%; and d) Approved final presentation and other materials: 15%.
12
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 2. Poverty Estimates for Cambodia9
POVERTY LINE New poverty line: Calculations of Poverty lines are calculated for 3 regions: Phnom Penh, other urban areas, and rural areas from CSES 2009. These are presented in Table 1. Table 2 shows a comparison of the new poverty line with the earlier poverty line (drawn in 1997, based on CSES 1993-1994 data).
Table 1. New poverty lines (expenditure per person per month and day, 2009 prices) based on CSES 2009 1. Phnom Penh a. Food poverty line: 94,945 Riels per month, at 2,200 K-calories b. Non-food allowance: 98,106 Riels per month c. Water: nil Total (a + b + c): 193,052 Riels monthly per capita, or 6,347 Riels/day per capita 2. Other Urban Areas a. Food poverty line: 79,293 Riels per month, at 2,200 K-calories b. Non-food allowance: 53,032 Riels per month c. Water: 61 Riels per month Total (a + b + c): 132,386 Riels monthly per capita, or 4,352 Riels/day per capita 3. Rural Areas a. Food poverty line: 69,963 Riels per month, at 2,200 K-calories b. Non-food allowance: 35,350 Riels per month c. Water: 1,247 Riels per month Total (a + b + c): 106,560 Riels monthly per capita, or 3,503 Riels/day per capita
Table 2. Comparing poverty lines of 1997 (based on CSES 1993-1994 data) and new poverty lines (based on CSES 2009 data), Riels/day at 2009 prices (daily expenses) Old poverty lines (1997 method) New poverty lines Phnom Penh 4,185 6,347 Other urban areas 3,438 4,352 Rural areas 3,213 3,503 Cambodia 3,332 3,871
The sum of the food bundle (2,200 calories for the Reference Food Basket in the bottom 5th-30th percentile group), non-food items (14 items in all, in the bottom 20 th-30th percentile group) and the gap between the cost of purified water paid for in Phnom Penh and elsewhere, constitutes the poverty line.
9 Ministry of Planning, Royal Government of Cambodia, ‘Poverty in Cambodia – A new approach, Redefining the poverty line,’ April 2013.
13
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
POVERTY RATES Estimates for 2009: Poverty rates, estimated for the 3 regions and the whole country, are given below in Table 3: Table 3. Percentages of persons below the poverty line according to the New Poverty Line based on CSES 2009 data
Region Food poverty rates (%) Poverty rates (%) Phnom Penh 0.3 12.8 Other urban areas 2.0 19.3 Rural areas 5.1 24.6 Cambodia (weighted average) 4.23 22.89
Intra Rural Inequality10 Table 4. Quintile-wise per capita disposable income of other rural population in Cambodia from 2009 to 2015 based on annual CSES data Disposable income per capita (Value in US$) Quintile Group 2009 2010 2011* 2012* 2013* 2014* 2015* 1 (lowest 20% income quintile) 4 6 7 8 12 4 14 2 (20%-40% income quintile) 11 15 17 20 26 31 39 3 (40%-60% income quintile) 19 24 29 33 41 51 59 4 (60%-80% income quintile) 30 37 44 50 60 75 84 5 (highest 20% income quintile) 88 108 106 113 132 169 187 * Preliminary results ** Other than Phnom Penh
Table 5. Quintile-wise year on year growth rate of per capita disposable income of other rural population in Cambodia from 2009 to 2015 based on annual CSES data
Year-on-year growth rate of disposable income per capita
Quintile Group 2009-10 2010-11* 2011-12* 2012-13* 2013-14* 2014-15*
1 (lowest 20% income quintile) 46% 11% 19% 44% -64% 236% 2 (20%-40% income quintile) 41% 11% 19% 28% 20% 25% 3 (40%-60% income quintile) 31% 18% 16% 24% 23% 17% 4 (60%-80% income quintile) 23% 19% 13% 20% 25% 12% 5 (highest 20% income 23% -2% 7% 16% 29% 10% quintile) * Preliminary results ** Other than Phnom Penh
10 Source: Cambodia Social and Economic Survey (CSES)
14
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Graph 1. Year on year growth rate of per capita disposable income of other rural population in Cambodia from 2009 to 2015 based on annual CSES data
300% Year-on-year disposible income growth rate 250%
200%
150%
100%
50%
0% 2009-2010 2010-2011* 2011*-2012 2012*-2013 2013*-2014* 2014*-2015* -50%
-100% Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5
15
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 3. Nutrition and Health Indicators
UNICEF, Save
the Children & Save the Children World Bank World Bank
Cambodia Siem Reap Pursat Battambang Banteay Meanchey Nutrition Children under 5 who 32 36 39 25 29 are stunted (%) Children under 5 who 10 10 12 8 8 are wasted (%) Children age 6-59 months with any 56 52 65 49 40 anaemia (%) Women age 15-49 yrs 45 41 47 43 26 with any anaemia (%) Maternal Health Antenatal Care from a 95 96 95 97 99 skilled provider (%) Births delivered in 83 92 78 90 88 health facility (%) Births assisted by 89 93 86 94 96 skilled provider (%) Child Health Children age 12-23 months who received all 73 79 80 89 91 basic vaccinations (%) Childhood Mortality (deaths per 1000 live births) Infant mortality 28 40 31 28 29 Under five mortality 35 56 36 37 32 Fertility Total fertility rate (number of children per 2.7 2.7 3.1 2.9 2.8 woman) Source: Cambodia Demographic Health Survey (CDHS), 2014
16
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Indicator Data (from CDHS, 2014) The median age • 22.4 (for all women in Cambodia currently aged 25-49) at first birth • 21.6 (for all women in Siem Reap Province currently aged 25-49) Attended birth • 83 per cent births in the five years before the survey were delivered in a health facility, and 17 per cent were delivered at home • 89 per cent of births are delivered with the assistance of a trained health professional (i.e., a doctor, nurse, or midwife), an increase from 71 per cent in 2010 Infant mortality • Infant mortality in Cambodia - 28 deaths per 1,000 live births • Under-5 mortality in Cambodia - 35 deaths per 1,000 live births Exclusive • 63 per cent of children are breastfed within one hour of birth, and 87 per cent are breastfeeding breastfed within one day of birth rates • 73 per cent of Cambodian children aged 0-3 months are exclusively breastfed, and only 65 per cent of children aged 0-5 months are exclusively breastfed. • Among children less than 6 months old, 11 per cent consume breast milk and plain water and 7 per cent consume other milk in addition to breast milk
Source: Cambodia Demographic Health Survey (CDHS), 2014
17
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 4. Analysis of IDPoor Data for Select Provinces
Brief ID Poor Data for Select Provinces UNICEF, Save the Children Save the Children World Bank & World Bank Battambang Banteay Siem Reap Pursat (Round 7, Meanchey (Round 9, 2015) (Round 7, 2013) 2013) (Round 9, 2015) Total Number of Households in 190,106 95,614 240,201 161,006 Coverage Area (Poor & Non-Poor) HH in Poor Level 1 12,561 10,376 32,460 9,960 Average % of Total HHs per Village in 6.6% 10.9% 13.5% 6.2% Poor Level 1 Female-Headed HH 4,539 4,740 10,488 3,877 in Poor Level 1 % of Female-Headed 36.1% 45.7% 32.3% 38.90% HH in Poor Level 1 HH in Poor Level 2 21,070 12,928 44,601 18,241 Average % of Total HHs per Village in 11.10% 13.5% 18.6% 11.3% Poor Level 2 Female-Headed HH 6,568 5,040 12,219 5,832 in Poor Level 2 % of Female-Headed 31.2% 39.0% 27.4% 32.0% HH in Poor Level 2 Source: ID Poor Website - http://www.idpoor.gov.kh
18
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Nutritional status of children disaggregated by age and gender
Table 1. Nutritional status of children (by age)
Age in Stunting Wasting months (Height for age percentage below -2 SD) (Weight for height percentage below -2 SD) <6 16.1 12.8 6-8 13.1 6.5 9-11 16.6 14.2 12-17 28.1 10.6 18-23 33.8 10.9 24-35 38.5 8.0 36-47 39.8 9.3 48-59 36.0 8.7
Table 2. Nutritional status of children (by gender)
Stunting Wasting Gender (Height for age percentage below -2 SD) (Weight for height percentage below -2 SD) Male 32.9 9.9 Female 31.9 9.3
19
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 5. Evidence of Selected Cash Transfer Projects Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location World Bank Conditional Pregnant women and children To help increase the • Readiness of delivery mechanisms of the Cambodia Transfer consisted of two under 5 registered as IDPoor 1 utilization of essential social protection system tested in this pilot are Cash types - basic transfer and and 2 in Srey Snam district health services by satisfactory and can support implementation of Transfer Pilot bonus transfer. Basic transfer (Siem Reap) and Phnom Srok pregnant women and scaled-up versions. Programme, was US$ 5 per month per district (Banteay Meanchey). children (0 to 5 years of • Rationalisation of conditions is needed to Cambodia individual woman & child, 2,330 beneficiaries – 381 age), and enhance the reduce exclusion of vulnerable populations. with bonus upon completion women and 1,949 children. readiness of delivery • The design needs to take into account supply- of co-responsibilities mechanisms of the side services needed for growth monitoring to (attendance in health and social protection be an effective tool in preventing and education sessions, prenatal system. combating malnutrition, importantly during the check-up, institutional first 1,000 days, but also beyond that given delivery, post-natal check-up high stunting levels among older children. for mother and growth • A full programme communication strategy monitoring and vaccination should be designed, not just to strengthen the for child). Maximum benefit linkage between payments and co- per child amounted to US$ 90 responsibilities but to support the strategy on per year. behavioural change. Source: Cambodia Cash Transfer Pilot Programme Process Evaluation Nourish Conditional Pregnant women and children To reduce preventable • Digitization is required to reduce time lag Cambodia Eligible women can receive under two years old registered maternal and new-born between recording conditional cash transfer up to six payments to reach a as IDPoor 1 and 2 in Pursat, deaths, apply key (CCT) conditions and the payments made. maximum of US$ 65 over the Battambang, and Siem Reap government policies and Mobilization of CCT beneficiaries to attend course of their participation in provinces. improve the nutritional community-level CCT activities with the help of the project, after certain 13,378 beneficiaries – 3,392 status and wellbeing of mobile technology should also be done. conditions are met. pregnant women and 9,986 pregnant women and • Share good practices through CCT refresher Conditions include at least caregivers of children under 2 children less than two trainings. four antenatal care visits, (as of June 2017). years. childbirth at health care facility, two postnatal care visits, use of hand washing device and monthly growth
20
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location monitoring for children under two. Source: NOURISH First 1,000 Days CCT Process Review Pantawid Conditional Residents of the poorest The 4Ps has dual • There is an estimated poverty reduction impact Pamilyang It provides grants conditioned municipalities, based on 2003 objectives as the of 1.4 percentage points per year. Philipino on health-related behaviours Small Area Estimates (SAE) of flagship poverty • Household heads, other adults are more Project for children aged 0–5 years the National Statistical alleviation programme encouraged to work and set up their own (Pantawid), and schooling for those aged Coordination Board (NSCB). of the Aquino businesses. Phillipines 10–14 years. administration: • 87% of 4Ps parents are now more optimistic • Households whose economic 1. Social assistance, Two types of grants – health about their situation and their children’s futures. condition is equal to or below giving monetary and education were provided: the provincial poverty support to extremely Health: threshold; Health: Poor households with poor families to • Reduction in severe stunting among children 0–14 years old • Households that have respond to their beneficiary children. and/or pregnant women children 0-18 years old immediate needs; • Lower maternal mortality in the past five years received up to PhP 500 (US$ and/or have a pregnant and because more mothers deliver babies in health 11) per household per month. woman at the time of 2. Social development, facilities (7/10 live births). assessment; and The conditions included breaking the • Drastic decrease in alcoholism in 4Ps. monitoring of children (0-14 • Households that agree to intergenerational households (spending on vices was lowered by years) and pregnant women. meet conditions specified in poverty cycle by 39%). investing in the health Education: The education the programme. and education of poor • 4Ps beneficiaries consume more rice and grant of up to PhP 300 (US$ It operates in all the 17 regions children through cereals than non-beneficiaries. 6.50) per child per month. in the Philippines, covering 79 programmes such as: Education: Beneficiary households provinces, 143 cities, and received the education 1,484 municipalities. a. Health check-ups • Near-universal school enrolment of elementary transfer for each child as long for pregnant age children for 4Ps households (98%). The Government expanded the as the child was enrolled in women and • 6% higher gross enrolment rate for beneficiary programme in December 2016 primary or secondary school children aged 0 to high school students. and attended 85 per cent of to reach a total of 20 million 5. • Higher spending on education among 4Ps the school days every month. people belonging to 4.4 million b. Deworming of households (PhP 206 more per school-aged households. schoolchildren child per month vs non-4Ps). aged 6 to 14. c. Enrolment of • Decrease in child labour days (7 days less a children in day- month for 4Ps households).
21
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location care, elementary, • 333,673 graduated from high-school in 2015, and secondary 13,400 of whom received honours. schools. Local Economies: d. Family development • Households invest more in working assets sessions. (livestock, machineries) than non-beneficiaries. • 4Ps households spent more on basic needs such as food, education and medicine that stimulate the growth of the local economy. Source:
22
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location years old attending health urban areas, respectively, in children 6–59 workshops. months; • The greatest changes in health and education were observed in women and indigenous populations; • The longer the participation by beneficiaries in the project, the better the health and education outcomes; • The project still faces significant challenges in reaching its objective of breaking the intergenerational cycle of poverty; and • Nutrition, health, education, and job indicators for the target population still lag significantly behind. Source:
23
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location Shombhob Conditional Following a Public Information • The objective of the • Combination of nutrition information, growth Pilot Project, Campaign, Proxy Means Test Shombhob project monitoring and cash are able to motivate Households with children Bangladesh (PMT) scores were used to was threefold: mothers towards nutrition-enhancing feeding aged 0-36 months determine household eligibility practices. • Test the delivery of Nutrition allowance of BDT to create the initial beneficiary conditional cash • A significant impact on the incidence of stunted 400 per month roster. Shombhob set up an transfers to the and underweight children was not found, which open registration process Conditions poorest households is not surprising since these outcomes are where interested households through local generally more stubborn to affect and require a Monthly attendance at growth with at least one child age 0-36 governments to longer term intervention. monitoring of children aged 0 months and/or at least one reduce their • This modality of conducting the nutrition and – 36 months, and nutrition primary-school-aged child were household poverty growth monitoring sessions allowed non- invited to come and apply to be session for mother/caregiver levels; beneficiary mothers to have access to nutrition- selected by the project. each Households with children related knowledge. household had to fill in a • Increase school going to primary school (6- questionnaire which gathered attendance of 15 years) all the information and beneficiary children Education allowance of BDT variables needed to construct a going to primary 400 per month PMT score based on the school; and formula and weights and • Improve the Conditions eligible beneficiaries were nutritional status of identified. Regular school attendance beneficiary children (at least 80 percent every 7,004 children receiving aged 0 to 36 months month) for enrolled children in nutrition-related benefits and old. primary school 15,774 children receiving education-related benefits
Source: Ferré, Céline and Sharif, Iffath, World Bank, ‘Can Conditional Cash Transfers Improve Education and Nutrition Outcomes for Poor Children in Bangladesh?’, 2014. Programme Conditional • The targeting for PKH was • The Government of As per the World Bank Group (WBG) Evaluation Keluarga Payments are made conducted by the Indonesia Indonesia (GOI) Study conducted in 2011, following were some Harapan quarterly. Conditions include: Statistics Agency (Badan P introduced the PKH impact indicators: (PKH), covering use of specific usat Statistik — BPS). programme to • The average monthly expenditure of Indonesia health care services and address inequalities beneficiaries increased by Rp 19,000 per children being enrolled in • Households were eligible in health and person, equal to a 10 per cent increase in school and having at least 85 based on their level of education service comparison to pre-project levels. Households
24
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location per cent attendance. poverty and fulfilment of and to provide a used this additional income to increase their Compliance is monitored demographic direct cash transfer spending on food (especially high-protein using an online system. Non- characteristics. for very poor foods) and health costs. There is no evidence compliance results in a households. that beneficiaries misspent the additional funds warning and then, if not • Conditional requirements for on non-productive goods such as tobacco or receiving PKH benefits • The cash transfer rectified, a 10 per cent cut to alcohol. transfer size and, finally, include expectant mothers contributes to receiving prenatal check- immediate poverty • The likelihood of mothers from beneficiary exclusion from the project. By households completing four pre-natal check- design, recertification of ups, new-borns and alleviation while the toddlers receiving post-natal continuing ups increased by more than 13 per cent above eligibility is conducted every pre-project levels, and completing the three years. care and health check-ups, commitments to and children aged 6 to 18 preventative health recommended two post-natal visits increased attending nine-year care practices and by nearly 21 per cent. compulsory education. education contribute • The likelihood of children (ages zero to 5 years to breaking inter- old) being taken to local health facilities to be Initial roll-out was to 432,000 generational poverty weighed increased by 30 per cent above households, gradually by increasing baseline levels. At the same time, beneficiary expanding to several million. productive households increased the likelihood of As of 2010, the project had investments in completing their children’s vaccinations by expanded to reach 810,000 children so that they approximately 11 per cent. households across 13 have better • Increased usage of health services also provinces. According to the opportunities for the contributed to an increase in the share of latest world bank documents, future. The GOI households that treated their children for PKH coverage has increased intends for PKH to diarrhoea by 13 per cent. to 6,000,000 in 2016) and is produce changes in targeting up to 15,000,000 • Beneficiary households, however, did not indicators such as poor families by 2019, covering demonstrate increased usage of recommended child malnutrition, all 34 provinces, 426 districts, vitamins (iron tablets for pregnant women or expenditure on high- and 98 cities, as well as by vitamin A for children), which is partly due to protein foods, expanding and integrating insufficient stocks. There is no evidence yet of education, and child other social accountability changes in long-term health outcomes, such as labour. interventions. child malnutrition and mortality rates, which are not expected to be observed over the short timeframe of the three-year survey. • The benefits of the project also extended to neighbouring households that did not receive cash transfers but nevertheless changed their
25
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location behaviours (i.e., a “spill-over effect”). Their pre- natal visits increased by over six per cent compared to pre-project levels and child weighing increased by 10 per cent compared to pre-project levels. Changes among these neighbouring households may have been encouraged by the positive example of beneficiaries as well as PKH facilitators who played an important role in helping to spread information about healthy behaviours. • The pilot project had little impact on changing education behaviours. For children from beneficiary households who were already in school, the project helped to increase the time they spent in school. Junior secondary students spent approximately 40 minutes more in school per week, while primary school students spent 20 more minutes per week. During the initial pilot phase, however, PKH had no impact on drawing more children into the education system and keeping them in school. Enrolment rates, drop-out rates, and the incidence of wage labour remained unchanged after three years. Source: World Bank. "Indonesia Social Assistance Reform Program." Documents & Reports.
26
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location protection to poor households subsidy regime and most at Presidential Decree No. • BLT households made regular and safe in a manner that was more risk from the negative impacts 3 of 2008 is: consumption choices. progressive than the on consumption from price subsidies it was replacing. increases. (a) To help the poor in • BLT allowed poor households to plan for and Coverage: maintaining the adjust to increases in fuel prices. The program ran for 12 months fulfilment of their basic • Where average household spending was most in 2005-2006 and nine months necessities; anaemic, BLT households increased their in 2008-2009. Approximately (b) To prevent the expenditure most; BLT had positive effects on 50 percent of all households in decline of welfare level community-wide expenditure. the poorest quintile (according among the poor to expenditure) received BLT. • Rates of child labour fell faster in poor BLT resulting from economic The poorest 40 percent of households, though these same households hardships; and households received nearly had slightly higher rates of child labour before two-thirds of total BLT benefits (c) To promote social BLT. available. Though designed responsibility and social • BLT did not create handout-dependent and deployed in less than 5 awareness. households and in fact, households receiving months, BLT reached over 19 BLT cash benefits found new jobs at increased million households. More than rates. a third of all households in Indonesia received BLT in 2005. The post office distributed benefits in every one of Indonesia’s provinces. In 2008, there were approximately 600,000 fewer beneficiaries but every province continued to be served. Source: World Bank. "BLT Unconditional Cash Transfer: Social Assistance Program and Public Expenditure Review 2." Documents & Reports. 2012.
27
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location South Africa Unconditional Eligibility: The primary CSG aimed to reduce • The CSG generates positive developmental Child Support Cash transfers to poor caregiver must be a South poverty and promote impact that multiplies its benefits in terms of Grant (CSG) households with young African citizen, permanent investments in the directly reducing poverty and vulnerability. children to provide social resident or refugee; both the physical, social and protection, reduce poverty, applicant and the child must human capital of poor • Early enrolment in the CSG programme and decrease inequality. reside in South Africa; the children. substantially strengthens impacts. Promoting applicant must be the primary continuous access to the CSG for eligible caregiver of the child/children children through adolescence would help to concerned; the child must be maximise the potential benefits of the grant. under the age of 18; the • Receipt of the grant by adolescents generates caregiver cannot apply for a range of positive impacts, not least of which more than six non-biological is the reduction in risky behaviours, which in children; the child cannot be the context of high HIV prevalence, generates cared for in a state institution; a particularly protective impact. and the caregiver is subject to a means-test threshold of ZAR 3,300 per month (or annual income of ZAR 39,600) in 2015; if the caregiver is married then the combined threshold is double (ZAR 6,600 a month and/or ZAR 79,200 per annum). From 10 per cent of poor children covered when it was introduced in 1998, it reached 85 per cent in 2015 (11.7 million)
Source: ‘Child Support Grant (CSG).’
28
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Name of the Conditional/ Targeting and Coverage Objectives Findings/Impact programme Unconditional and Location United Nations Children’s Fund. ‘The South African Child Support Grant Impact Assessment Evidence from a survey of children, adolescents and their households.’ May 2012.
29
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 6. Commune-wise IDPoor Data Commune Poor Poor Poor Poor Total Poor Total HHs in Level 1 Level 2 Level 1 HHs Level 2 HHs Level 1&2 HHs Commune HHs HHs % % %
Bakong 32 147 1.8 % 8.4 % 10.3 % 1,744
Ballangk 65 116 4.7 % 8.4 % 13.1 % 1,381
Kampong 22 59 3 % 8.1 % 11.1 % 731 Phluk
Kantreang 47 84 2.3 % 4.2 % 6.5 % 2,011
Kandaek 72 218 2.5 % 7.6 % 10.1 % 2,875
Mean Chey 59 158 4.8 % 13 % 17.8 % 1,220
Roluos 152 141 9.3 % 8.7 % 18 % 1,630
Trapeang 145 130 7.7 % 6.9 % 14.6 % 1,883 Thum
30
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 7. Transfer Amount, Co-responsibilities for Bonus Transfer
Beneficiary Amount Frequency Total Co-responsibilities (US$) (US$) Basic Transfer (Unconditional transfer upon enrolment)
Pregnant 5 x 9 5 Meet eligibility and enrolment criteria women 9 times* times = (month) 45 (*until delivery of child) Children 0-5 12 times 5 5 x 12 years (every Meet eligibility and enrolment criteria (month) times = 60 month) Bonus Transfer (Conditional transfer based on compliance of co-responsibilities)
Pregnant Have at least 4 prenatal visits as per MoH Bonus for women (and safe motherhood standards prenatal mothers) 1 time 10 x 1 -1st visit: 1st trimester (before 16 weeks) check-ups 10 bonus time = 10 -2nd visit: 2nd trimester (24-28 weeks) during -3rd visit: 3rd trimester (30-32 weeks) pregnancy -4th visit: at term (36-38 weeks)
Have institutional delivery and postpartum Bonus for care package in a public Health Centre institutional (HC) as per MoH safe motherhood delivery & 1 time 14 x 1 14 standards post-natal bonus time = 14 -1st check-up upon delivery care -2nd check-up during the first week package -3rd check-up before 6 weeks Children Attend 3 growth monitoring sessions and below 1 obtain 3 recommended vaccinations below, as per MoH safe motherhood standards: Bonus for 1. BCG and Hep B at birth first 3 2. OPV-1 and DPT-HepB_Hib1 at 6 growth 1 time 10 x 1 weeks monitoring 10 bonus time = 10 3. OPV-2 and DPT-HepB_Hib2 at 10 sessions weeks and first 3 4. OPV-3 and DPT-HepB_Hib3 at 14 vaccinations weeks 5. Measles at 9 months 6. Japanese Encephalitis one month after measles vaccination Bonus for 3 additional Attend an additional 3 growth monitoring growth sessions and obtain an additional 3 monitoring 1 time 10 x 1 10 recommended vaccinations above, as per sessions bonus time = 10 MoH safe motherhood standards: and 3
additional vaccinations Children between 1 Bonus for 1 time $10 x 1 Attend 3 growth monitoring sessions $10 and 5 first 3 bonus time = $10 (children 1 to 5 years old) growth
31
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Beneficiary Amount Frequency Total Co-responsibilities (US$) (US$) monitoring sessions Bonus for 3 additional 1-time 10 x 1 Attend additional 3 growth monitoring growth 10 bonus time = 10 sessions (children 1 to 5 years old) monitoring sessions Mothers Bonus for attending 3 Participate in 3 community-based community- 10 x 3 education sessions for each bonus 10 3 times* based times = 30 * there is a total of 9 modules – once education every two months, over 18 months sessions
32
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 8. Cash Transfer Pilot Implementation Steps
# 1. Planning
Preparation of: a. Instruments and materials b. Operational plans and venue c. Enrolment teams
#7. Processing Payroll and # 2. Training of Enrolment Payments Teams a. General principles a. Methodology applied b. Generation and transfer of b. Cascaded training payroll c. Training content c. Payments to beneficiaries d. Training coverage d. Reconciliation of accounts e. Timing and venue
#8. Case Management
a. Updates of family information b. Complaints (e.g., exclusions, compliance, # 6. Monitoring Compliance payment) and # 3. Information Campaign of Co-responsibilities responses/resolutions a. Principles b. Posting of information
a. Updating beneficiary lists c. Dissemination to villages b. Compliance verification d. Updating IDPoor list e. Processing of completed forms
# 5. Enrolment # 4. Targeting
a. Validating and registering eligible family members a. Assessment by IDPoor (pregnant women and b. Identifying eligible families children) (IDPoor 1 and 2) b. Orientation of eligible families at registr ation c. Final registration check list d. Closure process
33
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 9. Stakeholder Analysis
Stakeholder Roles/responsibilities in the Assessment of potential impact Potential project of project on stakeholder and strategies for stakeholder on project obtaining support or reducing obstacles Rights Holders Mothers and Rights holders who benefit Recommendations and opinions Scheduling must not Pregnant directly from cash transfer funds made upon the project will lead to conflict with work Women an improvement in the quality and schedules frequency of the services they receive Children The rights holders who will Recommendations and opinions Scheduling must not benefit from cash transfer funds made upon the project by their conflict with work mothers will lead to an schedules of improvement in the quality and mothers frequency of the services they receive Husbands/ Secondary actors involved in Recommendations and opinions Scheduling must not Heads of determining usage of cash made upon the project will lead to conflict with work Household transfer money and key an improvement in the quality and schedules influencers in the household frequency of services to household members
National Level (Duty Bearers) CARD Function as the implementation Inputs from CARD will provide Identify focal point agency, provide national level insights for the design and and schedule prior leadership and management, implementation mechanism of the appointment strengthen structures within cash transfer projects; this will sub-national administration, affect design and implementation provide capacity building of future projects support, undertake monitoring and evaluation
MEF Financier, responsible for Opinion and recommendations Identify focal point funding the project and will affect future implementation and schedule prior distributing funds to AMK strategies/approaches for cash appointment Microfinance transfer deliveries, inform replication of the project, cost- effectiveness will be assessed and lessons learnt and good practices will be provided MoP MoP being the implementing Data on IDPoor and opinions will Identify focal point agency for IDPoor provided affect targeting mechanisms for and schedule prior IDPoor data for targeting and future cash transfer projects appointment enrolment purpose. MoP also provided all relevant details related to IDPoor as required for different implementation activities
34
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Stakeholder Roles/responsibilities in the Assessment of potential impact Potential project of project on stakeholder and strategies for stakeholder on project obtaining support or reducing obstacles MoH Agency ensuring proper Recommendations will inform the Identify focal point implementation of health supply side limitations of health and schedule prior services and support in delivery service provision and provide appointment of community based health and lessons learnt and good practices nutrition sessions for community-based health and nutrition sessions
Sub-national Level (Duty Bearers) District Administrative unit responsible Insights and opinions will affect Identify focal point Administration for data collection and future implementation strategies and schedule prior Office/Deputy reporting, case management appointment Governor and coordination with social protection agencies Commune Administrative unit responsible Insights and opinions will affect Identify focal point Council/ for raising awareness about the future implementation strategies and schedule prior Commune project, enrolling beneficiaries, appointment; Focal Person/ supporting community based reassure them that Commune health and nutrition sessions the evaluation will Chief/ CCWC and undertaking monitoring only help in improving project quality and progress towards achieving outcomes Village Chiefs/ Administrative unit helping in Insights and opinions will affect Identify focal point VHSG implementation of community future implementation strategies and schedule prior based health and nutrition appointment; sessions, leading reassure them that communication functions, the evaluation will supporting CCWC in only help in identification and enrolment of improving project beneficiaries quality and progress towards achieving outcomes Health Centre Primary provider of health Recommendations will inform the Identify focal point services, support supply-side limitations of health and schedule prior service provision and provide appointment lessons learnt and good practices for community based health and nutrition sessions Implementing Partners AMK Payment agency undertaking Data and recommendations will Reassure them that Microfinance cash delivery to beneficiaries at affect future implementation the evaluation will local level strategies/approaches for cash only help in transfer deliveries improving project quality and progress towards achieving outcomes
35
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Stakeholder Roles/responsibilities in the Assessment of potential impact Potential project of project on stakeholder and strategies for stakeholder on project obtaining support or reducing obstacles UNICEF Designed and provided Recommendations and insights Commissioning the Cambodia technical support to the project will affect future implementation evaluation, whose role/contribution to and design of projects/strategies cooperation should outcomes will be assessed not be a problem
36
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Cash transfer pilot implementation structure
UNICEF CARD
UNICEF CARD Finance
UNICEF CARDMonitor ingFinance & evaluation District/Commune
(focal persons) UNICEF CARDMonitorData ingmanagement & evaluation Finance
District/Commune Data management (focal persons) MonitoringFinance & evaluation
AMK Data management District/Commune Monitoring & evaluation (focal persons) AMK Beneficiaries Beneficiaries DataBeneficiaries management
District/Commune AMK (focal persons)Beneficiaries Beneficiaries Beneficiaries
AMK Beneficiaries Beneficiaries Beneficiaries
Beneficiaries Beneficiaries Beneficiaries
37
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children Annex 10. Evaluation Matrix, Indicative Questions to Guide Development of Data Collection Tools and Analytical Framework for the Evaluation
S.N. Criteria Evaluation Questions Indicators Expected Sources 1. Relevance of ▪ To what extent did the selection of beneficiaries ▪ Number of beneficiaries ▪ Project documents, the project (pregnant women and children under five) complement ▪ Average transfer amount per including theory of design and the targeting of other social projects to reach the beneficiary change / log-frame approach worst-off and most vulnerable women? Any gaps in ▪ Frequency of cash transfers ▪ Other social project relation to targeting and coverage of the pilot project? ▪ Average number of education sessions documents ▪ To what extent was the choice to use conditional cash attended by beneficiaries ▪ Situation transfer rather than unconditional justified with regards ▪ Percentage of women indicating that analysis/needs to the needs of beneficiaries, availability and quality of cash transfer is the preferred assessment services, capacity of local government and service mechanism to improve health and ▪ Review of secondary providers, and government preferences? nutrition service utilization would be quantitative data ▪ To what extent was the size and regularity of the cash preferred ▪ Documents on cash transfer adequate? Were the different needs of ▪ Number of children identified as transfer methodologies beneficiaries met within the objectives of the pilot malnourished/ Severe Acute ▪ Assessment of local project? Malnutrition (SAM) capacity ▪ Was the choice to use cash rather than in-kind ▪ Data on assistance justified in terms of the needs (among budgets/financial different social/gender groups), availability of markets disbursements and beneficiary and government preferences? ▪ FGDs ▪ Was the usage of the cash by beneficiaries in line with ▪ Information gathered the project objectives? through KIIs ▪ Does the inclusion of beneficiaries happen on a ▪ Survey findings regular basis and what could be a possible mechanism for regular inclusion and exit? 2. Efficiency of ▪ How well was the delivery process managed, ▪ Number of beneficiaries who were not ▪ Information gathered the delivery considering the time and resources at each stage of a part of the original targeting under through KIIs mechanism implementation and coordination between UNICEF, IDPoor ▪ Roles/responsibilities CARD, sub-national administrations, and AMK ▪ Amount of time taken to open a bank documents Microfinance? account ▪ Assessment of cash ▪ Is there potential for improving efficiency and savings ▪ Amount of time spent by beneficiaries transfer methodology at various stages of implementation? in receiving cash (wait time) ▪ Document review ▪ How timely was the project in relation to the needs of ▪ Attendance at orientation sessions ▪ Secondary data different social groups and in comparison with other analysis cash transfer projects? How could timeliness have ▪ Assessment of local been improved? capacity
38
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children S.N. Criteria Evaluation Questions Indicators Expected Sources ▪ How efficient was access to the project in terms of ▪ Assessment of potential private/opportunity costs from the beneficiary stakeholder perspective, and considering different social groups? engagement and ▪ Is there an efficient mechanism for dissemination of capacity lessons learnt and best practices? ▪ Is the project congruent to other social protection projects related to nutrition of pregnant women and children under five? 3. Effectivenes s ▪ To what extent and how was the cash transfer used for ▪ Average opinion of participants ▪ Survey findings of the project better food consumption of under-five and other regarding the cash transfer (simplicity ▪ Information gathered children in the household, compared with adults? How and clarity) through KIIs and FGDs has the cash transfer supported nutrition and care of ▪ Change in household expenditure ▪ Secondary data new-borns and children under-five? Were there any ▪ Percentage of women who consumed sources unintended results? iron pills during their pregnancy ▪ Project documents ▪ How effective were the complementary community- ▪ Percentage of women who had ▪ Documents related to based education sessions from both implementers and institutional deliveries financial management women’s perspectives? ▪ Percentage of women who use iodised ▪ Review of programme ▪ Has there been utilization of health services attached salt methodology to the conditions and/or other health services? ▪ Percentage of women who give their ▪ Data related to costs ▪ How well was the financial management system child/children sprinkles incurred established, including reporting compliance? ▪ Percentage of women whose children ▪ Cost effectiveness ▪ How well has the existing targeting mechanisms have been immunized assessment helped achieve the desired objectives of the pilot? ▪ Percentage of women indicating that ▪ Identification and ▪ How well did the monitoring of co-responsibilities and food consumption of children has comparison of certain other reporting mechanisms function, including the role increased after the cash transfer indicators on of HCs, Commune Councils and Village Chiefs? ▪ Average number of times each child’s knowledge and ▪ How effective was the process of information growth has been monitored behaviours from the dissemination in terms of awareness regarding the ▪ Percentage of beneficiaries who baseline project? borrowed money to cover the cost of ▪ Has there been a change in the knowledge, attitudes delivery of their child since receiving and practices of women because of the cash transfer? the cash transfer ▪ Are there any grievance redress mechanisms ▪ Percentage of beneficiaries who available and if so, are they effective? borrowed money to cover the cost of ▪ Were there any significant gaps in inclusion for healthcare since receiving the transfer particular social groups? ▪ Average bonus payment as a result of ▪ Was the pilot has cost effective, provided value for fulfilling co-responsibilities money? ▪ Attendance at nutrition and health education sessions ▪ Time taken for complaint resolution
39
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children S.N. Criteria Evaluation Questions Indicators Expected Sources 4. Sustainabil ity ▪ To what extent can the major capacity gaps and ▪ Financial sustainability ▪ Information gathered of the cash bottlenecks, if any, at national and sub-national levels ▪ Capacity building needs through KIIs transfer be overcome? ▪ Coordination project ▪ What are the implications in re-targeting? ▪ Synergy with other projects at the local ▪ Is the project sustainable without creating externally- level funded institutions? What can be the role of national, provincial and sub-national government in implementation of the cash transfer project? ▪ Where does the existing implementation capacity of the Government stand given that the cash transfer is to be implemented as a regular project and not as a limited duration pilot? ▪ What are some good practices witnessed in the project that are replicable at the national level? 5. Equity and ▪ Was the project design and delivery equitable to ▪ Number of beneficiaries reached across ▪ Information gathered gender different social groups and gender? various communes under the pilot through KIIs ▪ Did the project achieve the same level of success in ▪ Survey findings different places and with different social groups? ▪ Information gathered ▪ Were there any negative effects felt by any social through FGDs groups?
6. Com parison ▪ Carry out a comparison to other cash transfer projects ▪ Type (conditional / unconditional) ▪ Project documents with other (both conditional and unconditional) in the region and ▪ Design and implementation features ▪ Evaluation reports cash transfer other successful ones, such as Nourish and World ▪ Size of transfer ▪ Studies projects Bank Cash Transfer pilot in Cambodia; Pantawid ▪ Project governance ▪ KIIs (in case of (4Ps) Pamilyang Pilipino Programme in Philippines; Cambodian cash Program Keluarga Harapan (PKH) in Indonesia; and transfer projects) Prospera (PROGRESA / Oportunidades) in Mexico, etc.
40
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Indicative questions to guide development of data collection tools Category Respondent Category Indicative Questions to Guide Tools Governance and Pregnant women and mothers • Was sufficient effort made to inform institutional issues of children under the age of the community about the cash five, field level government transfer project? officials including commune • Is there any systemic leakage by members, focal points, village local elites or authorities? chiefs, district advisors • Is there any discrimination in targeting or distribution? • Are all targeted beneficiaries receiving the cash transfer? • Is the enrolment and payment mechanism discriminatory in any way? • Were there any constraints in completing co-responsibilities? Roles and Field level government officials • What are the methods of raising responsibilities of the including commune members, awareness about cash transfers sub-administrative units focal points, village chiefs, and to what extent are they district advisors successful? • Has there been an increase in birth registrations? • What is the process of enrolment and what are the challenges faced? • What does community based education entail and what is the level of participation? • Are the conditions being followed to receive bonus payments? • Who is responsible for reporting/monitoring the conditions that need to be followed to receive benefit? Understanding Pregnant women and mothers • What are the documents needed to response to payments of children under the age of receive the payment? done by AMK five, AMK Microfinance officials • Is cash delivered on time and in a Microfinance secure manner? representatives at pay • Are any recipients disadvantaged points by the payment system?
Health Centre facilities Health Centre medical • What are the supply-side problems and services professionals, service delivery at Health Centres? staff • What are the demand side constraints in utilization of services? • What is the response of beneficiaries to co-responsibilities, which are a part of the conditional cash transfer project? • What training was provided to Health Centre staff? • What is the level of record-keeping at Health Centres?
41
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Category Respondent Category Indicative Questions to Guide Tools Usage, adequacy and Pregnant women and mothers • What are recipients’ views on the satisfaction with the of children under the age of five use of cash? Is it being used in the cash received expected areas? • If both cash and in-kind assistance were available, which option would recipients prefer? • Is the cash amount sufficient to achieve its objectives? • What is the opinion on conditionality of receiving cash payments (prenatal check-up, institutional delivery and postnatal care, growth monitoring and vaccination, attendance of education or communication sessions on early childhood development)? • What are the main challenges being faced? • Have there been any changes in Knowledge, Attitude and Practices (KAP) as a result of the education sessions?
42
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Analytical framework for the evaluation Level of Analysis Criteria being Respondent Category Broad Areas of Analysed Analysis Policy level analysis • Relevance • Representatives from • Governance • Equity and UNICEF Cambodia, CARD and Gender and MEF institutional • Policy makers and officials issues from government • Social department (national level) protection • Other development policy partners (national level) • Comparison with other similar projects
Design level analysis • Relevance • Officials from government • Design of the • Efficiency department cash transfer • Sustainability • Project officers and their system and Scalability affiliates (provincial and • Cost • Equity and district level) effectiveness Gender
Implementation level • Efficiency • Representatives from • Roles and analysis • Effectiveness UNICEF Cambodia, CARD responsibilities • Sustainability and MEF of the District and Scalability • Implementing partners Office, DCWC, (field level government Commune officials including commune Council, members, focal points, CCWC, VHSG village chiefs, district • Payment advisors, officials from AMK process Microfinance) undertaken by AMK Microfinance • Monitoring mechanisms • Grievance redressal
Demand-side • Relevance • Implementing partners • Understanding analysis • Effectiveness (field level government response to officials including commune payments done members, focal points, by AMK village chiefs, members of Microfinance CCWC and VHSG, among representatives others) at pay points • Beneficiaries – pregnant • Usage, women adequacy and • Beneficiaries – mothers satisfaction with children under the age with the cash of 5 received • Heads of household and • Demand and husbands of beneficiaries utilization of • Non-beneficiaries health services (excluded due to economic status) • Health Centre staff
43
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 11. List of Documents for Review
▪ CARD-UNICEF Cambodia Cash Transfer Pilot Operations Manual, 2016. ▪ UNDP Cambodia Country Profile
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
▪ World Bank, 2014, ‘International Development Association project appraisal document on a proposed credit in the amount of SDR 202.4 million (US$300 million equivalent) to the People’s Republic of Bangladesh for an Income Support Program for the Poorest Project.’ ▪ Intergovernmental Panel on Climate Change. ‘Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change’ [Stocker, T.F., D. Qin, G.-K. Plattner, M. Tignor, S.K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex and P.M. Midgley (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA. ▪ United Nations Children’s Fund, ‘UNICEF’s approach to scaling up nutrition for Mothers and their Children,’ 2015. ▪ Brinkman, Henk-Jan, et. al., ‘High food prices and the global financial crisis have reduced access to nutritious food and worsened nutritional status and health.’ The Journal of Nutrition vol. 140 no. 1: 153S-161S, 2010. ▪ Molyneux, Maxine and Thomson, Marilyn, ‘Cash transfers, gender equity and women's empowerment in Peru, Ecuador and Bolivia,’ Gender & Development, vol. 19 no. 2, pp. 195-212, 2011. ▪ Asian Development Bank, ‘Conditional Cash Transfers – are they helping promote gender equity?’ 2013. ▪ United Nations Children’s Fund, ‘Cash Transfer as a Social Protection Intervention: Evidence from UNICEF Evaluations, 2010-2014.’ June 2015 ▪ Department for International Development, ‘Cash Transfers Evidence Paper,’ 2011. Evaluation guidance documents:
▪ United Nations Children’s Fund, ‘Guidelines on Piloting and Scaling Up of Innovations and Good Practices.’ ▪ United Nations Children’s Fund, ‘Brand Book and Brand Manual,’ 2017. ▪ Global Evaluation Reports Oversight System Handbook, 2017. ▪ United Nations Children’s Fund, ‘How to design and manage Equity-Focused Evaluations.’ ▪ United Nations Evaluation Group Code and Conduct, Ethical Guidelines, Handbook, Norms & Standards for Evaluation. ▪ United Nations Children’s Fund-Adapted UNEG Evaluation Reports Standards, 2017. ▪ United Nations Children’s Fund, Evaluation Policy, 2013. ▪ UN Women, ‘How to Manage Gender-Responsive Evaluation.’ ▪ United Nations Children’s Fund Procedure for Ethical Standards in Research, Evaluation, Data Collection and Analysis, 2015.
45
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 12. List of Activities and People Met during Scoping Visit
Description of Activities Thursday, 7 September 2017 Briefing meeting on inception mission itinerary, Dropbox and logistics with EMT of UNICEF Presentation on the cash transfer pilot to understand the design and other details of the pilot at UNICEF Office Meeting to understand UNICEF key priorities for Cambodia and role in social protection projects
Meeting to understand the nutrition issues in Cambodia and potential of cash transfers Friday, 8 September 2017 Meeting with Save the Children to understand the design and implementation of the Nourish project Meeting with the CARD team to understand the process of design and implementation of the CT pilot
Meeting with MEF to understand their role and expectations from cash transfers
Meeting with Health Department officials to understand their role and delivery of services
Meeting with MoP officials to understand the identification process Monday, 11 September 2017 Meeting with district officials and commune focal persons to understand the implementation of the pilot in District Bakong Interview with Prasat Bakong District Focal Person and IT Assistant to understand monitoring and reporting under the CT pilot Meeting with AMK based in Siem Reap and Focal Person involved in cash distribution Tuesday, 12 September 2017 Interview with Focal Person in Rolus Commune and interview with five beneficiaries, including pregnant women and mothers Visit to Health Centre to understand role of service providers in the pilot, utilization of health and nutrition services, etc. Thursday, 14 September 2017 Development of draft tools and preparation of presentation for reference group meeting Friday, 15 September 2017 Discussions with Evaluation Management Team to debrief about the inception mission visits Meeting with Reference Group Members Meeting with C4D, UNICEF to understand the process development of IEC/BCC material and trainings conducted under the CT pilot
46
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 13. Key Stakeholder List, Data Collection Methods and Data Collection Tools Level Objectives of Data Collection Respondents Data Collection Number of Broad Category of Questions Tool Interviews • Analysing the extent to which the • UNICEF’s targets on health and nutrition National UNICEF KII Key project has been appropriately • Efficacy of using cash transfer Cambodia personnel designed and effectively • Design of the project from 10 implemented • Project funding institutions • Understanding parameters to • Provision of TA and support determine cost-effectiveness of • Capacity development the project • National policy on social protection, CARD KII • Assessing the institutional health and nutrition targets, cash capacity at the national level. transfers • Identifying key gaps in relation to • Roles and responsibilities the programme life-cycle • Design of the project • Assessing the strengths and • Budget plan weaknesses of the project • Implementation mechanisms including • Identifying the learning for project payments, communication scale-up • Tracking, M&E mechanisms • Providing comparison with similar • Institutional capacity development and cash transfer projects trainings • National policy on social protection, MEF KII health and nutrition targets, cash transfers • Efficacy of using cash transfer • Design of the project • Implementation of IDPoor MoP KII • Targeting, enrolment, grievance redressal for the project • Health service provision MoH KII • Utilization of health services as a result of co-responsibilities • Community education sessions • Community nutrition projects • Institutional capacities
47
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Level Objectives of Data Collection Respondents Data Collection Number of Broad Category of Questions Tool Interviews • World Bank Cash Transfer project- NCDD KII design, implementation and monitoring mechanism • Budget and project costs • World Bank Cash Transfer project- World Bank KII design, implementation and monitoring mechanism • Budget and project costs • Nourish Project- design, implementation Save the KII and monitoring mechanism Children • Budget and project costs • Roles and responsibilities AMK KII • Payment process (to beneficiaries) Microfinance • Institutional capacity • Grievances • Flow of funds • Verification mechanisms • Data collection and management information system generation • Roles and responsibilities District Office – Semi Structured • Data collection and reporting Planning and Interview • Monitoring Commune • Case management • Coordination with other departments • Institutional capabilities/requirements • Analysing the extent to which the District Office – Semi Structured • Roles and responsibilities Province/District 4-6 project has been effectively Sangat Support Interview • Data collection and reporting implemented, with a particular Office • Monitoring focus on MIS management, cash • Case management distribution mechanism, • Coordination with other departments monitoring and case • Institutional capabilities/requirements management • Roles and responsibilities • Assessing the institutional District Council KII • Payment process capacity at the sub-national level • Institutional capacity
48
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Level Objectives of Data Collection Respondents Data Collection Number of Broad Category of Questions Tool Interviews • Analysing the level of AMK KII • Grievances coordination between sectoral • Flow of funds structures involved in social • Verification mechanisms protection • Data collection and MIS generation • Identifying key gaps in monitoring, Commune • Roles and responsibilities FGD institutional capacity Council • Awareness about the project • Assessing the strengths and Members • Enrolment procedures weaknesses of the project in including • Grievance redressal terms of data collection and flow, Commune • Community-based education sessions coordination arrangements, Committee for • Monitoring including HC services transparency, grievance redressal Women and • Institutional capabilities/requirements Children (CCWC) • Understanding the extent to • Roles and responsibilities Commune Commune focal Semi-structured 8 which the cash transfer has been • Awareness about project persons questionnaire successfully implemented in • Enrolment procedures terms of targeting, enrolment, • Grievance redressal inclusion and exclusion errors, • Community based education sessions promoting birth registration • Monitoring including HC services • Assessing the institutional • Institutional capabilities/requirements capacity at the sub-national level • Community based education sessions • Assessing the effectiveness and Village chief KII 8 • Communication activities utilization of behaviour change • Identification and enrolment of communication (BCC) activities beneficiaries • Assessing the strengths and • Beneficiary satisfaction/grievances weaknesses of the programme in terms of targeting, enrolment, quality and reach of BCC activities • Analysing the extent to which the • Socio-economic status Village Household head Survey 24 project has been effectively • Payment mechanism including and husbands of implemented in reaching out to documents required, regularity and target groups beneficiaries adequacy of cash transfer • Assessing the effectiveness and • Understanding the usage of cash by utilization of BCC activities households
49
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Level Objectives of Data Collection Respondents Data Collection Number of Broad Category of Questions Tool Interviews • Understanding the usage of • Overall view, opinion and need of the
money being provided along with cash transfer programme beneficiary satisfaction and • View and effectiveness of the nutrition adequacy of the transfer level and health education sessions • Understanding the grievances of • Knowledge, Attitudes and Practices beneficiaries in utilization of (KAP) towards immunization, maternal services nutrition, child nutrition, breastfeeding etc. • Grievances and redress mechanisms • Success stories Beneficiary Structured 240 • Socio-economic status and need for questionnaire cash transfers • Payment mechanism including documents required, regularity and adequacy of cash transfer • Understanding the usage of cash by households • Overall view, opinion and need of the Beneficiary Semi-structured 24 cash transfer programme husbands questionnaire • View and effectiveness of the nutrition and health education sessions • Knowledge, Attitudes and Practices (KAP) towards immunization, maternal nutrition, child nutrition, breastfeeding Non- 24 etc. beneficiaries • Grievances and redressal mechanisms (excluded due to • Success stories economic • Reasons for non-inclusion in the status) programme • Overall view and opinion of the cash transfer project • Roles and responsibilities Facility in- KII 24 • Availability and utilization of health charge services • Institutional capacity
50
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Level Objectives of Data Collection Respondents Data Collection Number of Broad Category of Questions Tool Interviews • Assessing the availability and • Roles and responsibilities Health Centre Service provider KII 4 utilization of health services • Availability and utilization of health (Nurse/Doctor) • Assessing the institutional services capacity at Health centres • Institutional capacity
51
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
List of attendees at the Validation of Fieldwork and Desk-Study Findings Workshop
1. Deputy Secretary General of CARD 2. Deputy Director, Planning and Health Information, MoH 3. Deputy Director, DDC, MoI 4. Director of Social Welfare, MoSVY 5. Child Welfare Department, MoSVY 6. Chief Social Inclusion and Governance, UNICEF Cambodia 7. Chief Siem Reap Zone Office, UNICEF Zone Office 8. Deputy Representative, UNCIEF Cambodia 9. Junior Assistant, CARD 10. Chief of Party, Save the Children 11. Deputy Governor, Prasat Bakong 12. District Focal Point, Prasat Bakong 13. District Advisor, Prasat Bakong 14. Commune Focal Point, Trapeng Thom 15. Commune Focal Point, Balang 16. Commune Focal Point, Kantreang 17. Commune Focal Point, Rolours 18. Evaluation Specialist, UNICEF Cambodia 19. M&E Officer, UNICEF Cambodia 20. Team Leader, IPE Global 21. Assistant Team Leader, IPE Global 22. National Consultant, Dynamic Alliance Consulting 23. Community Development Specialist, UNICEF Cambodia 24. C4D Officer, UNICEF Cambodia 25. Evaluation Intern, UNICEF Cambodia 26. Deputy of CWD, MoSVY
52
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
List of people interviewed
1. Director of Social Protection Department, CARD 2. Junior Assistant, CARD 3. Project Assistant, CARD 4. Chief Pension Unit, Ministry of Economy and Finance 5. Deputy Director of Planning and Health Information Department, Ministry of Health 6. Director of Welfare for Persons with Disability Department, MoSAVY 7. Director of IP Poor Department, Ministry of Planning 8. Deputy District Governor, Prasat Bakong district 9. District Focal Point, Prasat Bakong district 10. IT Person, Prasat Bakong district 11. Distict Advisor, Prasat Bakong district 12. Country Representative, UNICEF Cambodia 13. Deputy Country Representative, UNICEF Cambodia 14. Chief Social Inclusion and Governance, UNICEF Cambodia 15. Social Policy Specialist, UNICEF Cambodia 16. Chief Siem Reap Zone Office, UNICEF Cambodia 17. C4D Officer, UNICEF Cambodia 18. Health Specialist, UNICEF Cambodia 19. Child Survival Development, UNICEF Cambodia 20. Chief of Party, NURISH Project, Save the Children 21. Former Project Manager, CT Pilot project of World Bank 22. Senior Mobile Banking Support, AMK 23. Focus group interviews with commune council members in Bakong commune, Rolous commune, Kantreang commune, Ballank commune, Kampong Phlok commune, Trapaing Thum commune, Kandaek commune, and Meanchey commune. 24. Key formant interview with commune focal points from 8 communes, village chief, and village members from 23 target villages. 25. Key informant interview with 4 health centers (HCs): Bakong HC, Rolous HC, Meanchey HC, and Trapaing Thum HC.
53
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Data collection tools Survey Questionnaire for Households Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia (District Prasat Bakong)
Interviewer’s Name: ______Date of Interview: ____/_____/2017
Part 1 – Identification (Fill out before interview)
Commune Name Commune Code
Village Name Village Code
Name of Respondent Age (years)
Household ID ID from HH Roster
House No. (in family book)
House No. (on road)
Contact Number
Household Type IDPoor 1 IDPoor 2 Other
No. of Children under the
age of five
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an independent evaluation on behalf of CARD and UNICEF regarding the Cash Transfer Programme. We would very much appreciate your participation in this survey. We are very interested to hear your valuable opinion on the cash transfer programme and appreciate your participation in this interview. The information will help the Government to understand the cash transfer services, which were provided. The survey will take approximately 1.5 hours to complete. The information you provide will be kept confidential and will not be shown to other persons. It is not mandatory to participate in this survey and you can opt out at any point in the course of the survey. If I ask a question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this survey, since your views are important. Do you want to ask me anything about the survey? May I begin the interview now?
RESPONDENT AGREES TO BE INTERVIEWED Yes No
54
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Part 3 – Survey [1] Yes 1. Are you literate [2] No
2. How many years have you been at school? 3. Who is the Head of the Household? [1] Myself [2] Partner [3] Parent/Parent in law [8] Other (specify) 4. What kind of card do you have? (prompt - [1] ID Poor Card please mark all applicable answers – [2] Other (specify) multiple answers possible) 5. Can I please see your ID Poor card? (prompt [1] Have – please see the document and click [2] Don't Have pictures) 6. Do you have a maternal health book? (prompt [1] Yes – please see the document and click [2] No pictures) [3] Lost it [4] Don’t know 7. How many children do you have? Please No. of Children ______specify the age of each child. (prompt: please Age (Child 1)______write age of all children) Age (Child 2)______Age (Child 3)______8. Does your child have a birth certificate? Child 1 Child 2 Child 3 (prompt – if the respondent has more than [1] Yes [1] Yes [1] Yes one child, please ask individually for each [2] No [2] No [2] No child and respond accordingly) (prompt – please see the document and click pictures) 9. Did the child have a birth certificate before the Child 1 Child 2 Child 3 programme? [1] Yes [1] Yes [1] Yes [2] No [2] No [2] No
10. Do you have a yellow immunization/growth Child 1 Child 2 Child 3 monitoring card for your baby? [1] Yes [1] Yes [1] Yes [2] No [2] No [2] No [3] Lost it [3] Lost it [3] Lost it [4] Don’t know [4] Don’t know [4] Don’t know 11. Does your household have any means of [1] Bicycle transport? How many? (prompt - please mark [2] Horse/Oxen Cart all applicable answers – multiple answers [3] Kou Yon possible) [4] Small Rowboat Or Canoe (No Motor) [5] Motorbike [6] Tuk Tuk [7] Car/Van/Truck [8] Boat With Motor [9] Tractor [10] Others 12. Counting all sources together, how much is the [1] Less than 50 USD (<200,000 R) monthly income of your family?
55
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
[2] 50 to 100 USD (200,001-400,000R) [3] 101 to 150 USD (400,001-600,000R) [4] 151 to 200 USD (600,001-800,000R) [5] More than 200 USD (800,000R) 13. Can you please state the month in which you started receiving the Cash Transfer Benefit Amount? 14. How many installments of cash transfer have you received? 15. How much of the cash transfer money have [1] None you withdrawn from your bank account? [2] Entire Amount [3] Others, please specify …………………. 16. Do you know how much money you have in your Cash Transfer Pilot bank account? (Prompt: Skip if answer to 16 is [2] – Entire Amount) 17. In the previous month, how much money did [1] Food …………………… you spend on the following items? [2] Cloth …………………… [3] Health care …………………. [4] Education-related costs …………. [5] Paying off debt …………………… [6] Other ……………. 18. Have the household expenditures changed as [1] Yes, decreased a result of the cash transfer programme? [2] Yes, increased [3] No, remained the same [4] Don't know 19. If expenditures of household decreased/ [1] Food______increased, on which items and how much on [2] Debt Repayment ______each? [3] Clothing/Shoes ______(Prompt: Skip if answer to 18 is [3] No, remained the same) [4] Saved ______(Prompt: please write all appropriate [5] Business investment ______responses) [6] Transport ______[7] Rent/Shelter ______[8] Water ______[9] School Fees ______[10] Gift/Share ______[11] Livestock ______[12] Household Items ______[13] Medical ______[14] Agricultural inputs ______[15] Firewood ______[16] Other ______TOTAL ______20. If your household expenditure was decreased/ [1] Quantity consumed by all HH members increased on food, what was decreased/ [2] Quantity consumed by children increased? [3] Quantity consumed by adults (Prompt: please tick all appropriate responses) [4] Quality of food consumed [5] Others (specify)______
56
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
21. If your household expenditure on food was [1]Quantity of meat purchased/consumed increased, please state on which type of food. [2]Quantity of fish purchased/consumed (Prompt: please tick all appropriate [3]Quantity of fruits purchased/consumed responses) [4]Quantity of vegetables purchased/consumed [5]Quantity of milk purchased/consumed [6]Others (specify)______
22. If you spent the Cash Transfer Programme [1] Cereals (rice, corn, wheat etc.) money on food, how much did you spend on ______the following: [2] Vitamin A rich tubers and vegetables (pumpkin, carrots, sweet potatoes etc.) ______[3] White tubers and vegetables (white potatoes, yams, cassava root) ______[4] Vitamin A rich fruits (ripe mangoes, papaya, cantaloupe etc.) ______[5] Flesh Meats (beef, pork, chicken, birds, wild game) ______[6] Eggs ______[7] Fish ______[8] Oils and Fats ______[9] Micronutrient (tablets or syrup) ______[10] Commercial Products (Ovaltine, formula etc.) ______
Part 4 – Cash Transfer 1. How did you come to know about Cash [1] Heard from Commune Council Transfer Programme? [2] Heard from HC (Prompt: please allow the respondent to [3] CT staff answer and tick all appropriate responses) [4] Heard from VHSG [5] Heard from village chief [6] Neighbour [7] Relatives [8] Other (specify) 2. Are you aware of the selection criteria used by [1] Yes the govt. to choose your household as a [2] No, why not? beneficiary? 3. If yes, what are the selection criteria? [1] IDPoor (Prompt: Skip this question if answer to 2 was [2] Pregnant Woman [2] No) [3] Mother of Child Under 5 (Prompt: please allow the respondent to answer [4] One essential prenatal check-up and tick all appropriate responses) [5] Availability of Child Health yellow card or Birth Certificate of Child [6] Others (specify) 4. How were you enrolled? [1] Mass Enrolment in Communes [2] Admission on Demand
57
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
5. If you were enrolled under Admission on [1] Name not in IDPoor list Demand, Why was enrolment not done during [2] Child did not have birth certificate mass admission at the commune? [3] Child did not have yellow card (Prompt: Skip this question if answer to 4 was [4] Did not have HC documentation regarding [1] Mass Enrolment in Communes) pregnancy (Prompt: please allow the respondent to answer [5] Not present during registration day and tick all appropriate responses) [6] Others (specify) 6. What were the documents required for [1] ID Poor Card enrolment? [2] Birth Certificate or Yellow Card (prompt: please allow the respondent to answer [3] Mothers Card [4] Others (specify) and tick all appropriate responses) 7. Did you participate in the Enrolment Orientation [1] Yes Session? [2] No 8. Do you know that there are two types of [1] Yes transfer – basic and bonus? [2] No 9. Do you know about the co-responsibilities for Pregnant Women and Mothers bonus payment? [1] 4 Prenatal check-ups during pregnancy [2] Institutional Delivery & 3 post-natal care package (prompt: please tick all the appropriate and [3] Participate in 3 community based education correct responses) sessions Children below 1 [4] Attend 3 growth monitoring sessions and 3 recommended vaccinations [5] Attend 3 additional growth monitoring sessions and 3 additional vaccinations Children between 1 and 5 [6] Attend 3 growth monitoring sessions [7] Attend 3 additional growth monitoring sessions 10. Which co-responsibilities did you complete in Pregnant Women and Mothers order to receive bonus payment? [1] 4 Prenatal check-ups during pregnancy (prompt: please tick all the appropriate and [2] Institutional Delivery & 3 post-natal care package correct responses) [3] Participate in 3 community based education sessions Children below 1 [4] Attend 3 growth monitoring sessions and 3 recommended vaccinations [5] Attend 3 additional growth monitoring sessions and 3 additional vaccinations Children between 1 and 5 [6] Attend 3 growth monitoring sessions [7] Attend 3 additional growth monitoring sessions 11. Were you given full information about what you [1] Yes are entitled to in an open manner? [2] No (Prompt: Ask: amount, where, frequency…) 12. When were you given this information? [1] Orientation Session (Prompt: please allow the respondent to answer [2] Education Sessions and tick the appropriate and correct responses [3] At Health Centre only) [4] At Payment Points [5] Others 13. Did you have to open a Bank Account for the [1] Yes Cash Transfer Programme? [2] No
58
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
14. How much time did it take for the Bank Account [1] Less than 1 month to open? [2] 1 - 2 months [3] 2 - 3 months [4] 3 - 4 months [5] Over 5 months 15. Was the process to open the bank account [1] Strongly disagree clear and simple? Kindly rate on a 5 point scale [2] Disagree [3] Neither agree nor disagree [4] Agree [5] Strongly agree 16. How long ago did you receive your last cash [1] Less than 1 month transfer? [2] 1 - 2 months [3] 2 - 3 months [4] 3 - 4 months [5] Over 5 months
17. After what interval do you receive the cash [1] More frequently than once a month transfer? [2] Monthly [3] Every 2 months [4] Every 3 months [5] Less frequently than 3 months 18. How long did you take to reach the pay point? [1] Less than 30 minutes [2] 0.5 - 1 hour [3] 1 - 2 hours [4] 2 - 3 hours [5] over 3 hours 19. How much money did you spend on travelling to the pay point? 20. What was the waiting time before receiving [1] Less than 30 minutes cash? [2] 0.5 - 1 hour (prompt: please state that the waiting time [3] 1 - 2 hours starts after completion of the Health and Nutrition Education Session) [4] 2 - 3 hours [5] over 3 hours 21. Who is the household member designated to [1] Pregnant lady/mother be the receiver of the cash? [2] Female Guardian [3] Father [4] Male Guardian [5] Other (specify) 22. In your opinion is the process for receiving [1] Strongly disagree money as a part of the Cash Transfer [2] Disagree Programme clear/ simple? [3] Neither agree nor disagree [4] Agree [5] Strongly agree 23. Was the amount received sufficient to cover [1] Yes the health and nutrition needs of the [2] No household? 24. Who mainly makes the decision about how the money is used?
59
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
25. Do you consider that the money received is [1] Yes properly used in the household? [2] No 26. Has the cash transfer project addressed your [1] Yes immediate needs [2] No
27. If yes, what has it addressed? [1] Food [2] Medical [3] Clothing [4] Shelter [5] Other, please specify …………………….. [6] Not Applicable 28. If no, what has not been addressed? [1] Food [2] Medical [3] Clothing [4] Shelter [5] Other, please specify …………………….. [6] Not Applicable 29. How much money did you spend on food in the past week? 30. How much money did you spend on health care (including transportation) in the past one year? 31. Have you encountered any problem while [1] Yes processing/accessing the cash? [2] No 32. If yes, what was the nature of the problem?
33. Have you ever raised a complaint regarding the [1] Yes project? [2] No 34. If yes, what was the nature of complain? If No, Are you aware of any mechanisms to report any complaint on this project?
35. Was it resolved? [1] Yes (Prompt: Skip this question if answer to 33 [2] No was [2] No) 36. If yes, how was it resolved? If no, why not? (Prompt: Skip this question if answer to 33 was [2] No)
37. In how much time was the complaint resolved? [1] less than 2 week (Prompt: Skip this question if answer to 33 [2] 2-4 weeks was [2] No) [3] 4-6 weeks [4] 6-8 weeks [5] 8-10 weeks [6] more than 10 weeks 38. In your opinion, what should be done to improve the cash transfer process? (Simpler enrolment processes, more cash, more pay points, more frequent transfers etc.) 39. Given options, which alternative to Cash [1] Food voucher Transfer would you prefer?
60
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
[2] Free Food [3] Free Medical Care [4] Food for Work [5] Business grants [6] Other: ______[7] None (Cash Transfer is better) 40. What tangible benefit/success story/testimony of how the project has impacted you or your household
(prompt: to be used in case studies)
Part 5 – Healthcare Service Utilization 1. Have you ever had to borrow to cover the cost [1] Yes of delivery of your children since getting the [2] No Cash Transfer? (please tick Not Applicable if [3] Not Applicable the woman has not given birth after enrolment in the Cash Transfer Programme) 2. Have you ever had to borrow to cover the cost [1] Yes of health care for you or your children since [2] No getting the Cash Transfer? [3] Not used health care services after enrolment in Cash Transfer Programme 3. How much have you borrowed to cover the cost of health care in total since getting the Cash Transfer? 4. Does the closest public Health Centre in your [1] Yes area provide free services for you with your [2] No IDPoor card? 5. Have you ever used your IDPoor card to [1] Yes receive health services from a public Health [2] No facility? 6. If not, why not? [1] Not seriously ill Prompt: Please skip if answer to 5 is [1] [2] Not sick at all Yes) [3] Prefer private doctor [4] No money for transport [5] Doctor takes money [6] Just got the card [7] Too sick to travel [8] Doctors discriminate HEF members [9] Lost card [10] Distant facility/no money for transport [11] Other(specify)
Part 6 – Knowledge, Attitude and Practices 1. From where do you get information [1] Spouse regarding maternal and child [2] Parents/In-laws nutrition? [3] Healthcare Professionals [4] Traditional healers
61
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
[5] Others 2. From whom did you seek advice or [1] Hospital treatment for yourself or your baby? [2] Health Centre [3] Traditional birth attendant [4] Private clinic/doctor [5] Village health volunteer [6] Relatives & friends [7] Pharmacy/drug seller [8] Traditional healer [9] Other (specify) 3. How many Nutrition and Health [1] Less than 3 Education (NHED) Sessions did you [2] 3 attend? [3] 3-6 [4] 6-9 [5] None 4. Were the NHED sessions useful in [1] Strongly disagree gaining knowledge about health and [2] Disagree nutrition? [3] Neither agree nor disagree [4] Agree [5] Strongly agree 5. Do you have any recommendations/ changes (ផ្លាស់叒ត ូរ ) to help improve the sessions?
Breastfeeding 6. What is the first food a new-born
baby should receive? 7. When should a mother start adding [1] Start adding earlier than 4 months of age foods to breastfeeding? [2] Start adding between 4-6 months of age (prompt: please allow the respondent [3] At 6 months to answer and tick the appropriate [3] Start adding later than 6 months of age and correct responses only) [4] Don't know 8. Have you ever breast-fed your [1] Yes baby? [2] No, why not? 9. After the delivery, when did you [1] During the first hour after delivery breast-feed your baby for the first [2] From 1 to 8 hours after delivery time? [3] More than 8 hours after delivery [4] Do not remember 10. How long did/will you exclusively breast feed without liquid supplements? 11. When did/will you introduce solids? 12. What solids did/will you start with? [1] Rice cereal [2] Fruits [3] Vegetables [4] Meat/chicken/fish [5] Others (specify)
62
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Anaemia 13. Have you ever heard about [1] Yes anaemia? [2] No 14. Can you name some symptoms or [1] Pallor health problems caused by [2] Weakness Anaemia? [3] Tiredness (prompt: please allow the respondent to answer and tick the appropriate [4] Cravings from non-food items and correct responses only) [5] Shortness of breath [6] Poor growth and development [7] Other causes [8] Don't know 15. Can you tell some measures to [1] Access to more information prevent anaemia? [2] Good diet (prompt: please allow the respondent [3] Iron and folic acid supplements to answer and tick the appropriate [4] Medical care and correct responses only) [5] Other [6] Don't know Supplements 16. In your pregnancies, did you take [1] Yes any iron and folic acid pills? [2] No [3] Don't know 17. For how many days do you need to take iron tablets, when pregnant? (correct answer should be 42 days, if incorrect answer is provided, please write it down and inform the beneficiary of the correct response) 18. Have you ever given your child iron [1] Yes supplements? [2] No [3] Don't know 19. Where did you obtain the iron [1] Health Centre / distributed supplements? [2] Pharmacy / sold / reimbursed [3] Other / specify [4] Don't know 20. Can you name some symptoms or [1] Pallor health problems caused by iron/folic [2] Weakness acid deficiency? [3] Tiredness (prompt: please allow the respondent [4] Cravings from non-food items to answer and tick the appropriate [5] Shortness of breath and correct responses only) [6] Poor growth and development [7] Other causes [8] Don't know 21. Do you use iodized salts? [1] Yes [2] No [3] Don't know 22. Do you know the benefits of using [1] Yes iodized salt?
63
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
[2] No
23. Please tell us about the benefits of [1] Healthy pregnancies using iodized salts? [2] Improved brain function (prompt: please allow the respondent [3] Improved thyroid function to answer and tick the appropriate [4] Fights depression and correct responses only) [5] Weight control [6] Fights cancer [7] Healthy heart [8] Removes toxin [9] Irritable bowel syndrome [10] Improved appearance [11] Other (Specify) 24. Which vitamin helps you prevent [1] Vitamin A "night blindness"? [2] Don't know or other (prompt: please allow the respondent to answer and tick the appropriate and correct responses only) 25. Which foods contain vitamin A? [1] Green leafy vegetables (prompt: please allow the respondent [2] Orange or yellow fruits to answer and tick the appropriate [3] Meat/fish and correct responses only) [4] Breast milk [5] Egg yolks [6] Others [7] Don’t know 26. Did/ Do you give your child [1] Yes sprinkles? [2] No 27. What is the use of giving sprinkles? [1] Promoting child’s growth (prompt: please allow the respondent [2] Improving immunity to answer and tick the appropriate [3] Preventing Diseases and correct responses only) [4] Improving Child’s Appetite [5] Preventing Anaemia [6] Others, Specify ……………. Dietary Diversity 28. Do you usually consume foods such [1] Organ meat Yes/No as organ meat, meat, fish, eggs, [2] Fish Yes/No legumes, green leafy vegetables? [3] Eggs Yes/No (prompt: one week recall to be asked) [4] Legumes Yes/No [5] Green leafy vegetables Yes/No [6] Others Yes/No [7] Don't know Immunization and Healthcare 29. Has your baby ever received any [1] Yes immunizations? [2] No 30. If yes, where did he/she get his/her [1] Provincial Hospital vaccination? [2] Referral Hospital [3] Health Centre
64
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
[4] NGO Clinic [5] Private Facility [6] Others 31. If no, why not? 32. Look at the growth monitoring Child 1 Child 2 Child 3 Child 4 card of the child, and record the Age: Age: Age: Age: number of times the child has been weighed and measured. Also check if the child is malnourished and tick Yes/No accordingly. Malnourished- Malnourished- Malnourished- Malnourished- [1] Yes [1] Yes [1] Yes [1] Yes [2] No [2] No [2] No [2] No 33. If the child was diagnosed as [1] Referral to Hospital/ Treatment Centre malnourished in the growth [2] Given Supplements monitoring chart, ask the [3] Provided advice/counselling on better nutrition following question: What action was taken by the HC [4] Others, specify staff or you? 34. When you were pregnant with your [1] Yes baby did you visit any health site [2] No (hospital, HC) for prenatal care? [3] Don’t know 35. Look at the maternal health book [1] One and record whether the mother [2] Two ever made any antenatal visit? [3] Three or more [4] None [5] Don’t have the card 36. Where do you go for antenatal [1] Provincial Hospital check-ups? [2] Referral Hospital [3] Health Centre [4] Private Facility [5] NGO Clinic, please specify ______[6] Others 37. Did/ Will you give birth at a [1] Yes health/medical facility? [2] No [3] Don’t know 38. If yes, where? [1] Provincial Hospital [2] Referral Hospital [3] Health Centre [4] Private Facility [5] NGO Clinic, please specify [6] Others 39. If no, why not? 40. Where do/will you take your child [1] Provincial Hospital when he/she is ill? [2] Referral Hospital [3] Health Centre [4] Private Facility [5] NGO Clinic, please specify ______[6] Others
65
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Others 41. What is the minimum year [1] Less Than 1 gap/space that should be there [2] 1-2 between two pregnancies? [3] 2 [4] 2-3 [5] More than 3 42. Do you always wash your hands [1] Yes before meals? [2] No If No, why not ______43. Does/Do your child/children always [1] Yes wash their hands before meals? [2] No If No, why not
Thank you so much for sharing your thoughts and opinions with us.
Part 7 – Questions for Head of the Household/ Husband of Beneficiary
1. Name of respondent and ID from HH roster
2. Age
3. Gender
4. What is your relation to the beneficiary?
5. How many years have you been at school? 6. What are the typical occupations and activities
that women conduct in your household?
7. What are the typical occupations and activities
you conduct in your household?
8. If the Government wants to start programmes for
improving women’s status, what do you think these programmes should focus on?
66
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
9. From where did you learn about the Cash
Transfer Programme?
10. In what areas of the cash transfer programmes For Enrolment: did you participate? Please elaborate on activities undertaken
For Payment:
For Co-responsibilities:
For Education Sessions:
11. Did you accompany the beneficiary to the Health
Centre for check-ups, vaccinations etc.?
12. How many check-ups was the beneficiary [1] One supposed to get during her pregnancy? [2] Two [3] Three [4] Four [5] More than four [6] None [7] Don’t know 13. Have you heard about anemia? [1] Yes [2] No 14. Can you tell some measures to prevent anemia? [1] Access to more information [2] Good diet [3] Iron and folic acid supplements [4] Medical care [5] Other [6] Don't know
67
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
15. Do you know the benefits of using iodized salt? [1] Yes [2] No 16. Please tell us about the benefits of using iodized [1] Healthy pregnancies salts? [2] Improved brain function [3] Improved thyroid function [4] Fights depression [5] Weight control [6] Fights cancer [7] Healthy heart [8] Removes toxin [9] Irritable bowel syndrome [10] Improved appearance [11] Other (Specify) [12] Don’t know 17. Please list the type of vaccines that need to be [1] BCG given to babies. [2] Hep B (prompt: please allow the respondent to [3] Vitamin A answer and tick the appropriate and correct responses only) [4] ABZ/MBZ [5] DPT 1st [6] OPV 1st [7] DPT 2nd [8] OPV] 2nd [9] DPT 3rd. [10] OPV 3rd [11] Measles [12] Don't know 18. What was the Cash Transfer money used on?
(Food, Healthcare, Clothing, debt repayment)
19. Did you help in deciding how the money should
be spent? If not, who made these decisions?
20. Are you aware of any problems/ challenges in
availing benefits under the Cash Transfer Programme (access to money, fulfilling co- responsibilities, documentation)
68
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
21. Do you see any changes in the quality of life of
the household as a result of the cash transfer? (including as a result of the education sessions and the money received)
22. What are your recommendations to improve the
Cash Transfer Programme?
Thank you so much for sharing your thoughts and opinions with us
69
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
KII Tool for Non-Beneficiaries Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia (District Prasat Bakong)
Interviewer’s Name: ______Date of Interview: ____/_____/2017
Part 1 – Identification (Fill out before interview)
Commune Name Commune Code
Village Name Village Code
Name of Respondent Age
Designation of Respondent
Contact Information
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an independent evaluation on behalf of CARD and UNICEF regarding the Cash Transfer Programme. We would very much appreciate your participation in this survey. We are very interested to hear your valuable opinion on the cash transfer programme and appreciate your participation in this interview. The information will help the Government to understand the cash transfer services, which were provided. The interview will take approximately 45 minutes to complete. The information you provide will be kept confidential and will not be shown to other persons. It is not mandatory to participate in this survey and you can opt out at any point in the course of the survey. If I ask a question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this survey, since your views are important. Do you want to ask me anything about the survey? May I begin the interview now?
RESPONDENT AGREES TO BE INTERVIEWED Yes No
70
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Part 3 – Questions 1. A) Do you know about the Cash
Transfer Programme?
B) What do you know about the programme?
C) From whom did you learn about these details?
2. Why were you not a part of the [1] Not in ID Poor 1 or 2 Cash Transfer Programme? [2] Did not have the necessary documents [3] Did not enrol in the given time period [4] Did not enrol voluntarily, please explain ……………………………………………………….………………… …………………………………………………………………………. [5] Others, please explain …...... ………………………………………………………………………….
3. A) Do you think that including
people in the programme through IDPoor Categorisation is a good method for enrolling women for the programme?
B) In your opinion, what criteria should be used for including women in the programme?
71
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
4. Do you wish to be a part of the
Cash Transfer Programme? Please elaborate on the reasons including specific needs, which you may have
5. A) Do you know anybody who is
a part of the Cash Transfer Programme? What is your relationship with them?
B) Do you think they have benefitted from the programme? Please elaborate on your answer with examples.
6. What is your overall opinion on
the Cash Transfer Program? Do you think it is a good way to improve the status of health and nutrition for pregnant women and children?
7. Do you have any
recommendations/suggestions regarding the cash transfer programme?
Thank you so much for sharing your thoughts and opinions with us.
72
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
KII Tool for Village Chief/ Commune Focal Person Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia (District Prasat Bakong)
Interviewer’s Name: ______Date of Interview: ____/_____/2017
Part 1 – Identification (Fill out before interview)
Commune Name Commune Code
Village Name Village Code
Name of Respondent Gender
Designation of Respondent
Contact Information
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an independent evaluation on behalf of CARD and UNICEF regarding the Cash Transfer Programme. We would very much appreciate your participation in this survey. We are very interested to hear your valuable opinion on the cash transfer programme and appreciate your participation in this interview. The information will help the Government to understand the cash transfer services, which were provided. The interview will take approximately 1 hour to complete. The information you provide will be kept confidential and will not be shown to other persons. It is not mandatory to participate in this survey and you can opt out at any point in the course of the survey. If I ask a question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this survey, since your views are important. Do you want to ask me anything about the survey? May I begin the interview now?
RESPONDENT AGREES TO BE INTERVIEWED Yes No
73
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Part 3 – Questions 1. What are your overall roles
and responsibilities as Village Chief/ Commune Focal Person?
Please ask the following question to only the Commune Council Focal Person
What are your responsibilities as a part of CCWC (Commune Committee for Women and Children)?
2. A) When and how did you
first come to know about the Cash Transfer Programme?
B) Did you receive any training or orientation with regard to the Cash Transfer Programme? If yes, please elaborate?
C) What materials/ documents were provided to help undertake the Cash Transfer Programme? How useful were they? (probe: ask about operational guidelines and whether they understand it or not)
74
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Can you provide any recommendations to help improve the quality of the material received?
3. What activities do you
have to undertake as a part of the Cash Transfer Programme?
4. Can you please elaborate
on your role in raising awareness about the Cash Transfer programme? What were the methods used? 5. A) Did you have a role in
conducting the community- based education sessions? Can you explain your role?
B) Do you feel these
sessions are useful and have benefitted the cash transfer recipients?
C) Do you have any recommendations for improving these sessions?
6. Are you compensated in
any way for the services that you render for the cash transfer? 7. A) How much time do you
devote per month to activities related to the cash transfer?
B) Do you have adequate time to complete all your other tasks? 8. Please state the key challenges in supporting the implementation of the Cash Transfer Programme including capacity gaps
75
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
9. Kindly elaborate on the details required in the table below:
Involvement of Appropriateness/ Method Transparency Community Effectiveness Members Targeting Probe: Was the targeting mechanism clearly described and documented to you as well as to the beneficiaries?
Enrolment Probe: Were the steps for enrolment and list of beneficiaries to be enrolled shared with you? Were beneficiaries aware of the above?
Payment Probe: Was the process of payment delivery and generation of payroll clearly told to you and to the beneficiaries?
76
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Reporting Probe: Is the report after which is generated after Payment payment shared with all relevant people and is it available for everyone to view?
Grievance Probe: Is the grievance Redress mechanism known to everyone and are complaints resolved in an open and transparent way?
10. According to you, what are the
challenges faced by the beneficiaries in accessing cash? How can these be resolved?
11. According to you, can there be
an alternate method for cash distribution? If yes, please elaborate.
12. Have you received complaints
from the beneficiaries regarding the Cash Transfer Programme? Please elaborate on the type of complaints.
77
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
13. What is the method of resolution
of beneficiary complaints/grievances?
14. What improvements should be
put in place by the administration and leaders to ensure the beneficiaries get better services in the Cash Transfer Programme? 15. Do you think that the Cash Availing more health services: Transfer Programme has led to an increase in beneficiaries availing more health services, eating more nutritious food and having better knowledge? Eating more nutritious food:
Having better knowledge:
16. Can you provide examples of the Cash Transfer leading to an increase in use of health and nutrition services?
17. In your capacity as an
administrative member, what according to you are some of the main social problems in your commune/village which need to be addressed? (Food security, livelihood support, better health services, water and sanitation services, infrastructure, education)
18. How has the cash transfer
programme helped to address these issues? Please provide examples
78
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
19. In your opinion, was the
programme design and delivery equitable/fair/non- discriminatory/unbiased to different social groups such as migrant families, children living with grandparents, other vulnerable groups? If not, please provide examples. 20. In the context of your
village/commune, do you think the cash transfer programme has been successful in improving the care and nutrition status of pregnant & lactating mothers and children of the age 0 to 5 years?
21. Do you have any
recommendations/ suggestions to improve the Cash Transfer Programme? (probe – is the current mechanism for targeting, enrolment, payment effective? What are the capacity gaps?)
Thank you so much for sharing your thoughts and opinions with us.
79
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
KII Tool for Health Centre Staff1 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia (District Prasat Bakong)
Interviewer’s Name: ______Date of Interview: ____/_____/2017
Part 1 – Identification (Fill out before interview)
Commune Name Commune Code
Village Name Village Code
Health Centre Name Health Centre ID
Name of Respondent
Designation of Respondent
Contact Number
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an independent evaluation on behalf of CARD AND UNICEF regarding the Cash Transfer Programme. We would very much appreciate your participation in this survey. We are very interested to hear your valuable opinion on the cash transfer programme and appreciate your participation in this interview. The information will help the Government to understand the cash transfer services, which were provided. The interview will take approximately one hour to complete. The information you provide will be kept confidential and will not be shown to other persons. It is not mandatory to participate in this survey and you can opt out at any point in the course of the survey. If I ask a question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this survey, since your views are important. Do you want to ask me anything about the survey? May I begin the interview now?
RESPONDENT AGREES TO BE INTERVIEWED Yes No
.
1 This is to be administered to 2 functionaries – 1) Functionary who has been involved in delivering nutrition & health education sessions under the cash transfer programme and 2) Functionary who has been involved in providing services at the health facility
80
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Part 3 – Questions What activities have you undertaken as a part of the Cash Transfer programme? a. Conducted Nutrition and Health Education sessions (NHED) as part of the cash transfer programme b. Only provided health services to beneficiaries who came to the health centre c. Both of the above d. Any other activity (please specify) (In case response is (b) or (d) then skip Section 3.1)
Section 3.1 – Nutrition & Health Education Sessions (NHED) 1. With regard to the NHED, Were you provided any training to conduct these sessions? Was it Please answer the following questions: adequate?
Do you feel any further training is required?
Can you recall how many sessions you have conducted? (Probe: please ask no. of instalments for which sessions conducted, no. of days sessions conducted and no. of sessions per day)
What was the content of these sessions?
81
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
What was the average duration of each session? Was it adequate? In your opinion, what should be the appropriate duration?
How has the response of the beneficiaries been to the sessions? Are they able to understand and comprehend the message being conveyed?
Were they able to correctly recall what had been conveyed to them in the previous sessions?
In your opinion, are these sessions beneficial to the women and the households?
In your opinion what percentage of beneficiaries has adopted appropriate practices with regard to healthcare and nutrition during pregnancy and nutrition of the child?
82
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
What additional content or methods are needed to improve the effectiveness of these education sessions?
2. A) How and who informs you
of the date, time and location of the NHED sessions which are to be conducted?
B) Did you come to know of the scheduled date sufficiently in advance?
C) Do you have any suggestions with regard to scheduling of the sessions and coordination for conducting the sessions?
3. Did you receive any
compensation for delivering the education sessions? If yes, what are they?
83
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
4. How much time does it take
you to prepare for a session?
How much time does one session take?
5. In your opinion, has the cash
transfer programme led to an increase in utilization of health services?
6. Can you please describe
some key challenges, if any, of delivering education sessions?
7. Apart from the education
sessions conducted as a part of the UNICEF CARD cash transfer pilot project, are any other education sessions also conducted?
Who conducts these sessions and are any agencies/NGO’s involved?
84
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
What is the content of these sessions and does it complement with the education sessions conducted as a part of the Cash Transfer Pilot?
8. Do you have any
recommendations/suggestions to improve the overall delivery of the Cash Transfer Programme?
Section 3.2 – Health Centre services 1. Are there any supply side
constraints/ problems at your Health Centre (infrastructure, skilled staff, supplies, etc.)?
2. Were any additional
resources (E.g.: vaccines, weighing scales etc.) provided as a part of the Cash Transfer programme?
85
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
3. What are the resources that
are required to further improve the Cash Transfer programme?
4. Do majority of the women in For Institutional Delivery: your catchment area visit the HC for institutional deliveries and health check-ups for themselves and their children? For Health Check-ups for themselves:
For Health Check-ups for their children:
5. If women don’t come to the HC, what are the reasons? (Costs, distance, lack of knowledge, skewed beliefs)?
6. In your opinion, has the cash transfer programme led to an increase in utilization of health services?
86
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
7. What is the method of keeping records at the HC? (manual – hard copy, computerized)
8. A) Do you keep records to track health and nutritional indicators of women? If yes, how and on which MIS/reporting system/document is this record-keeping done?
B) Do you keep records to track health and nutritional indicators of children? If yes, how and on which MIS/reporting system/document is this record-keeping done?
C) Kindly show the documents/MIS. (prompt: kindly click pictures. If not available, ask and note the reason). 9. Do additional records need to be kept as a result of the Cash transfer programme?
10. A) Did you identify any
malnourished or SAM children who were a part of the Cash Transfer Programme? If yes, how many?
87
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
B) If yes to above, then what actions/ steps were taken to ensure their treatment? (Referred the children to treatment centres, provided nutritional supplements, asked mothers to give better food etc.)
11. Overall, what is your opinion
of the cash transfer programme?
12. What are some of the main
constraints/ problems in the programme?
13. What are your
recommendations to improve the programme?
Thank you so much for sharing your thoughts and opinions with us.
88
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
FGD Guide for Commune Council Members Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia (District Prasat Bakong)
Names of the Facilitators: ______
______
Date of FGD: ____/_____/2017
Part 1 – Identification (Fill out before the FGD)
Commune Name Commune Code
Village Name Village Code
Sl. Name of the Respondent/Participant Role/Position Gender
1.
2.
3.
4.
5.
6.
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an Independent Evaluation on the Cash Transfer Pilot Programme on behalf of CARD and UNICEF. We are very interested to hear your valuable opinion on the cash transfer programme and appreciate your participation in this discussion. The information will help the Government to understand the cash transfer services, which were provided. The discussion is expected to take 1 to 2 hours to complete. We understand how important it is that this information is kept private and confidential. We request all participants to respect each other’s confidentiality. The information you provide will be kept confidential and will not be shared outside the group. It is not mandatory to participate in this survey and you can opt out at any point in the course of the discussion. If I ask a question you don't want to answer, just let me know and we will not discuss the same. Do you want to ask me anything about this discussion? Can we start the discussion now? Let's start by going around the circle and having each person introduce themselves. (Members of the research team should also introduce themselves and describe each of their roles.)
89
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Part 3 – Discussion Points 1. What are the overall roles and responsibilities of the Commune Council and the CCWC in the cash transfer programme? (Probe - Number of members, relationship with village chiefs, fiscal management)
2. What were the specific roles performed by each Member (category of member) in the cash transfer programme?
3. Did you receive any orientation / training regarding the cash transfer programme? Were these sessions useful? What did you learn? Do you have any suggestions to improve these sessions? (Probe – give specific examples of learnings, did the CC further provide any training to, for example, the village chiefs?)
4. Was any material given to you during the orientation or training sessions? Please provide details and also state whether the material is easy to understand. Was the operation manual given to you? Do they understand the operation manual clearly?
5. What is your opinion regarding the cash transfer project in the community? (Probe: effect on beneficiaries, change in attitudes of people towards health and nutrition, ease of implementation of project, targeting of beneficiaries vs. universal cash transfer given budget constraints, use of cash by beneficiaries)
6. Do you think there is an alternative way that people of the community would prefer to address the issues related to care during pregnancy and nutrition of mother and child instead of receiving a cash transfer? (Probe – free Food, free healthcare, other in-kind assistance etc.)
7. Do you think the conditions required for payment are relevant? Do you have any suggestions regarding these? (Probe – opinion on conditional vs. unconditional cash transfers)
8. In your opinion was the amount paid during the cash transfer sufficient enough to address the nutrition needs of pregnant & lactating mothers and their children?
9. Did you receive or are aware of any complaints/grievances from the beneficiary households? How were they resolved?
10. In your opinion what are some of the challenges/shortfalls faced by cash transfer project? Probe: What is not working well and how can it be addressed?
11. Do you think that the commune council has adequate capacity to undertake the Cash Transfer Programme? If not, what are the key challenges and capacity gaps which need to be addressed? Are the roles at each level clear and different (at district, commune, village level) or is there an overlap of responsibilities?
12. What are your recommendations and suggestions to improve the cash transfer programme?
Thank you so much for coming and sharing your thoughts and opinions with us.
90
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Format for Case Studies Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia (District Prasat Bakong)
Interviewer’s Name: ______Date of Interview: ____/_____/2017
Part 1 – Identification (Fill out before interview)
Commune Name Commune Code
Village Name Village Code
Name of Respondent
Is the respondent a recipient of the cash transfer or related to the beneficiary?
If related to the beneficiary, then state the relationship.
Contact Number
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an Independent Evaluation about the Cash Transfer Programme on behalf of CARD and UNICEF. We are very interested to hear your experience from the cash transfer programme and how it has affected your life. We would very much appreciate your participation in this study. The information will help the Government to understand the cash transfer services, which were provided. The discussion will take approximately 1 hour to complete
We are obligated to protect your privacy and not disclose your personal information (information about you and that identifies you as an individual e.g. name, date of birth, etc.). In the event that this story is published or presented, your identity will not be disclosed.
It is not mandatory to participate in this discussion and you can opt out at any point during the course of this discussion. If I ask a question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will Yes No participate in this discussion, since your views are important. Do you want to ask me anything about the survey? May I begin the interview now? RESPONDENT AGREES TO BE INTERVIEWED?
91
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Part 3 – Questions
Area of Information Collection Responses to be recorded from the Beneficiary Beneficiary Profile Age Occupation (including homemaker) Educational Qualification (attended school till which class, whether drop out/ never been to school/ studying, want to continue studies) Reasons behind educational status (Supportive or non- supportive family/ poverty/ lack of facilities/ early marriage and pregnancy) Husband’s Name and Occupation Husband’s Age Current Marital Status (collect full details from the beneficiary like age at marriage (her and husband’s); current status- happily married, widow, divorced, remarried /separated etc.) Number of children (Note ages and gender for each child separately) 1 2 3 4 5 6 Total # of Death / Details Live Abortion Age Age Age Age Age Age Kids (if any) Boy Girl TOTAL
When did the beneficiary enrol for the cash transfer (dd/mm/yy)? (see AMK opening acknowledgement, if available) Case Study Questions What were the health services that the beneficiary availed after registering for the cash transfer: a) No. of pre-natal check-ups: ……………………. b) Was the child delivered at a health institution? (Yes/No) ………………………… c) No. of post-natal check-ups ………………………………………… d) Has the child received full immunization? (Yes/No) (See Mother & Child Card)………………… e) No. of Nutrition and Health Education (NHED) sessions attended? ………………………….
What is the status of weight and height according to his/her age (observation based, see the Mother & Child card)?
92
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Any other information that you think necessary to mention.
Note: LISTEN TO RESPONDENTS STORY: (ask her to tell you her story from childhood, sit with her and just listen carefully, don’t take out paper and pen, and don’t take notes during the narration) Understand what she has gone through in her life. What were her struggles in life? What has she faced? Who all helped her (collect details if the Cash Transfer Programme has helped her, what help she got, did it help her in managing her life and the problems she faced)? Who did not help her at all or created problems for her? What is her life story, what is the most touching and emotional part in her narrations? FOCUS ON THAT PART AND BUILD YOUR STORY AROUND IT NOW START WRITING AND ASK YOUR QUERIES: 1. Before participating in the Nutrition and Health Education Sessions, did you have access to essential information on Maternal and Child Nutrition & Health?
2. Before the cash transfer programme, did you have enough money for HC visits and to buy nutritious food?
3. What are some of the important things that the cash transfer programme has done in your life and the lives of the members of your household? Please provide examples.
4. Has the cash transfer programme improved the nutrition and health status of your child/children? Please elaborate on your answer.
5. In your opinion what are some of the challenges/shortfalls faced by cash transfer project?
Take written consent from the beneficiary (in the format provided)
Take photo/s of Beneficiary, family (whoever is available and willing to give photo with a prior written consent in the format provided), get photos of children as well.
Take her or husband’s or family members’ quote in their own words and write it verbatim.
Thank you so much for sharing your thoughts and opinions with us.
93
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
FGD Tool for Households Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children in Cambodia
Names of the Facilitators: ______
______
Date of FGD: ____/_____/2017
Part 1 – Identification (Fill out before the FGD)
Commune Name Commune Code
Village Name Village Code
Sl. Name of the Participant Category2 Gender
1.
2.
3.
4.
5.
6.
7.
8.
Part 2 – Introduction and Consent Hello! My name is ______and I am here on behalf of IPE Global Limited. We are conducting an Independent Evaluation on the Cash Transfer Pilot Programme on behalf of CARD and UNICEF. We are very interested to hear your valuable opinion on the cash transfer programme and appreciate your participation in this discussion. The information will help the Government to understand the cash transfer services, which were provided. The discussion is expected to take 1 to 2 hours to complete. We understand how important it is that this information is kept private and confidential. We request all participants to respect each other’s confidentiality. The information you provide will be kept confidential and will not be shared outside the group.
2 Pregnant mother, Mother with child 0-5 years, care-giver, husband, head of household etc.
94
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
It is not mandatory to participate in this survey and you can opt out at any point in the course of the discussion. If I ask a question you don't want to answer, just let me know and we will not discuss the same. Do you want to ask me anything about this discussion? Can we start the discussion now?
Part 3 – Discussion Points 1. Could you briefly describe to us how you were selected for participating in this cash transfer project? Probe: transparency – was information provided in an open and complete manner, community involvement—were community members consulted regarding the design and implementation of the programme, feelings of some people not included who feel they deserve to be included, information on entitlements—amount of unconditional transfer they are supposed to receive, frequency of transfer, conditionality/co-responsibilities and bonus payments.
2. What is your opinion regarding the cash transfer project in your community? Probe: Did the cash transfer programme help you in your daily needs; on what do you use the cash transfer money; Did you learn something new in the education sessions which you did not know earlier? What? Do you think cash transfers are the best way to improve nutrition and health of children under 5? How much time and money do you have to spend to get the cash transfer? Are the cash transfer processes easy to understand? ; What is your opinion of having conditional cash transfer? Do you know why conditional cash transfers are used? Do you think they are helpful?
3. Could you briefly describe to us the process of accessing the cash from the project? Probe: what documents did you have to provide for enrolment? What is the waiting time at service points – to get cash? Are implementing partners (AMK, Health Centre, and Commune Council) helpful? Do they answer your questions? Can you suggest better alternatives approaches of getting the cash?
4. Who has control over resources (cash, food, mobile phones) within households? Who makes decisions about spending? (Spending decisions may vary by type of resource. For example, women may make decisions about the household food budget, while men may make decisions about purchasing household assets)
5. Has the cash transfer affected your life or the life of your family in any manner? How or what is the change? (testimonies)
6. In your opinion what are some of the challenges/shortfalls faced by cash transfer project? Probe: What is not working well and how can it be addressed?
7. What alternatives (if any) to cash transfer do you think people in this community would prefer to address their needs related to food (nutrition during pregnancy / lactation for children) and health care?
Thank you so much for sharing your thoughts and opinions with us.
95
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Indicative guide for key informant interviews at national and district level
Criteria Questions Key Respondents Project ▪ Please explain the constitution of the committee which designed this project. What were the specific roles and ▪ UNICEF Design responsibilities of each? Cambodia ▪ Why was Prasat Bakong in Siem Reap chosen to undertake the pilot project? ▪ CARD ▪ What was the external technical advice provided for the design and implementation of this project? Will this be ▪ Ministry of available for the scaling-up phase? Did this result in capacity building within the existing institutional structures? What Economy and guidance/support is being provided to support the implementation of the cash transfer project? Finance (MEF) ▪ Were any challenges faced during the implementation of the project – in legal or policy terms?
▪ Was there any Theory of Change or causal pathways developed during the design of the project? ▪ Several cash transfer projects (example Bolsa Familia project (BFP) Project in Brazil) include aspects of education, sanitation and hygiene in their nutrition project to have a unifying force in social policy, integrating social policy across sectors. Was this a consideration while defining the objectives and conditionality of this project? Are the impact indicators likely to change during the scale-up? ▪ Were civil society organizations involved in the design, implementation and monitoring of the project(s)? ▪ As per the project documents, there were consultations with the villagers. Can you provide any details of participatory processes which were followed during the consultations? ▪ Please explain the end-to-end monitoring mechanism for the project. ▪ Can the draft as well as actual timeline of the project roll out and project cycle be shared? Which processes required more time than initially considered? ▪ Please share the budget for the project. Relevance ▪ Understanding the project approach. ▪ UNICEF o What do you see as key features of CARD and UNICEF’s approach to cash transfer? How is this cash transfer Cambodia project distinct from others? ▪ CARD o How does CARD or UNICEF prioritize its cash transfer project? What support is provided? To what extent are ▪ Ministry of activities not directly related to the cash transfer being leveraged to strengthen the project? Planning (MoP) o What do CARD and UNICEF do to ensure sustainability of the project? ▪ Ministry of o How do considerations of scalability figure in decisions about what interventions to support? Health (MoH) o What is CARD’s role in the project? What is UNICEF’s role, vis-a-vis that of other partners? How does CARD- UNICEF project fit with the work of other partners? ▪ To what extent did the selection of targeted pregnant women and children under five complement the targeting of other social projects to reach the worst-off and most vulnerable women? Were there any gaps in relation to targeting and coverage of the pilot project?
96 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Criteria Questions Key Respondents o The consideration for targeting only the IDPoor was possibly because of poor nutrition status amongst the lower quintiles, availability of targeting criteria and limited financial resources. However, were there any other considerations for not having a universal cash transfer? o Does IDPoor data update and processing take place in a timely manner? What are the challenges in utilization of IDPoor Data for targeting, if any? o Apart from IDPoor lists, is there any other mechanism to seek out beneficiaries from the village population (for example through surveys undertaken at the village level)? o In your opinion, how successful is the cash transfer project in reaching targeted beneficiaries? Were there some specific target groups which got excluded for some reason? o What are some of the most common reasons for beneficiaries getting excluded as seen in Form 06? o What are the nutrition and health targets which the Government is looking to achieve? ▪ To what extent was the choice to use conditional cash transfer rather than unconditional justified with regards to the needs among pregnant women and children, availability and quality of services, capacity of local government and service providers, and government preferences? o What were the considerations while designing this project as a mixed-conditionality project? What are the arguments for conditional vs. unconditional cash transfer projects? o Was the Ministry of Health (MoH) involved in the design phase of the project? o What were the activities, organizational and infrastructural, which were undertaken by the MoH for proper implementation of health services expected under the cash transfer project? (Infrastructure would include, for example, provision of equipment such as weighing machines) o Were any additional Health Centre records required to be created as a result of the project or were existing records only used? o Is the healthcare infrastructure in Cambodia adequate for fulfilment of the conditionality of the Cash Transfer? Are Health Centres available in the vicinity of beneficiaries and are they adequately staffed? Is there a possibility of private providers being included in the setup? o It is mentioned in the Operational Manual that there is inadequate contact of women with the health system. What are the main demand-side reasons for this and how does the project address these issues? ▪ To what extent was the size and regularity of the cash transfer adequate? Were the different needs of pregnant women and under five children met within the objectives of the pilot project? o On what basis was the amount of the monthly transfer as well as bonus payments decided? ▪ Was the choice to use cash rather than in-kind assistance justified in terms of needs (among different social/gender groups), availability of markets and beneficiary and government preferences? ▪ Was the usage of the cash by beneficiaries in line with the project objectives? o Please explain the ’Community Nutrition Projects’ which are supported by the MoH.
97 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Criteria Questions Key Respondents o Please provide details of other measures introduced by RGC to improve coverage of health and nutrition services. o What are some of the most common reasons for early exit from the project? o How has the cash transfer contributed to / improved the health and nutrition status of the households or community? Please provide some practical examples. Efficiency ▪ How well was the delivery process managed, considering the time and resources at each stage of implementation ▪ UNICEF and coordination between UNICEF, CARD, sub-national administrations, and AMK Microfinance? ▪ CARD o Was there any challenge in coordination between different agencies and implementation partners? ▪ Ministry of o Is there adequate capacity within each administrative unit (CARD, MoP, MoH, District Administration, Commune Planning (MoP) Council and Health Centres for implementation of the project? What is CARD/UNICEF staff capacity / expertise to ▪ Ministry of deliver on the cash transfer project? Health (MoH) o Do the implementers have adequate time to undertake the activities for the project? Are any incentives given for ▪ AMK additional job responsibilities? Microfinance ▪ How cost-effective was the Cash Transfer pilot compared to other modalities and mechanisms and what potential is ▪ District and there for efficiency savings at all stages? Commune Level o What is the process for prioritization and allocation of resources within CARD/UNICEF, and to what extent are the Officials human/financial resources available sufficient for implementing the cash transfer project? o Was the pilot project implemented within the pre-decided budget plan? If not, what were some areas where unexpected costs were incurred? o What were the main implementation costs? o Was there a possibility to combine this project with some other to reduce start-up costs? o Considering the available options in Cambodia (and Prasat Bakong, specifically), was the most cost effective method applied in ensuring the cash is transferred to the beneficiaries at minimal cost? What alternatives would you propose? ▪ How timely was the project in relation to needs of different social groups, and comparatively with other cash transfer projects? How could timeliness have been improved? o Since a condition for enrolment is to have at least one prenatal check-up before the 14th week of gestation, when is the first payment to the pregnant woman made? ▪ How efficient was access to the project in terms of potential private/opportunity costs from the beneficiary perspective, and considering different social groups? o Given the occupational structure of the target population (largely agrarian), what is the opportunity cost to beneficiaries for participation in the project? What are the transportation costs to the Health Centre and what is the poverty gap?
98 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Criteria Questions Key Respondents o Is it mandatory for beneficiaries to visit the cash points for withdrawal? What percentage of beneficiaries come alone? o Were exclusive bank accounts created for this project or were existing bank accounts also used? Can the bank accounts which were opened be used for other purposes? ▪ What are the strengths and weaknesses of the CARD-UNICEF approach to monitoring the cash transfer project – both its activities and outputs and its contribution to broader country outcomes? ▪ Is there an efficient mechanism for dissemination of lessons-learnt and best practices? To what extent is CARD- UNICEF engaged in generating knowledge/data on cash transfer? ▪ Is the project congruent to other social protection projects related to nutrition of pregnant women and children under the age of five? How is the cash transfer project integrated or coordinated with other projects in health/nutrition? Effectiveness ▪ To what extent and how was the cash transfer used for better food consumption for under-fives and other children in ▪ UNICEF the household, comparatively with adults (considering food quality, quantity and diversity)? How has the cash transfer ▪ CARD supported nutrition and care of new-born children, and children under-five? Were there any unintended results? ▪ AMK o What is the pattern of withdrawal of cash by beneficiaries? Microfinance ▪ How effective were the complementary community-based education sessions from both implementers and women’s ▪ Ministry of perspectives? Planning (MoP) o What was the various communication material developed for the project as well as the communication sessions? ▪ Ministry of o What is the content of information packs distributed to households? Does it target only the beneficiary or the Health (MoH) entire household? ▪ District, o For Village Meeting One, what is process of developing the presentation and discussion material? Commune and o What material composes the modules which are a part of the community based education sessions? Are the Village Level modules correlated? What is the rationale behind giving bonuses for attending 3 out of 9 modules? Officials o What is the content and delivery process of the orientation session which takes place during enrolment? Is this the same across communes? ▪ How well did the financial management system establish including reporting compliance? o What was the process/criteria for selection of the microfinance institution (MFI)? Were any other alternatives considered? o What was the process followed by CARD to ensure timely payment to beneficiaries by AMK Microfinance? o For payments to beneficiaries, were additional distribution points created? o What was the preference amongst beneficiaries between using Mobile Saving Card and Fixed Banking Services? What was the cost for each? o What are the main bottlenecks as cash moves from the Government to MFI to beneficiaries?
99 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Criteria Questions Key Respondents ▪ How well did the monitoring of co-responsibilities and other reporting mechanisms function, including the role of Health Centres, Commune Councils and village chiefs? o What is the MIS system being used by the district office? What information is captured? How often is it updated and what are the validation checks? Kindly explain the flow of Health & Nutrition indicators from the Health Centres to the National Level. o What are the various tracking, monitoring and evaluation activities undertaken by CARD? o What support is provided by MoH in the verification of health co-responsibilities at the Health Centre? o What activities are undertaken by the CC to monitor availability of preventive health services at HCs? o What is the system of monitoring attendance at education sessions by the CC? o Is there a formal system by which the VHSG provides feedback to the CCs on beneficiary satisfaction? o What was the mechanism to document and disseminate lessons learnt from the project planning and implementation? o What is the mandate for spot-checks, which are to be made by District-level officials? o Have the monitoring and validation systems been working effectively? o What are the strengths and weaknesses of the CARD and UNICEF approach to monitoring the cash transfer project – both its activities and outputs and its contribution to broader country outcomes? o What activities are undertaken as a part of Growth Monitoring? ▪ How effective was the process of information dissemination in terms of awareness regarding the project? o What are the methods employed by the CCs to raise awareness about the CT pilot among community members? ▪ Are there any grievance redressal mechanisms available and if so, are they effective? o What activities are undertaken by MoP for grievance redressal? What is the mechanism? o What is the turn-around time for resolution of grievances? Is it in accordance with the operation manual guidelines? o What are some of the most common grievances? o Is there a system to utilize these grievances to ‘learn lessons’ and accord changes in processes? o Do you have any specific recommendations for improvements which should be put in place by the Government or implementers to ensure the beneficiaries get better services? ▪ Were there any significant gaps in inclusion for particular social groups? Sustainability ▪ How can the cash transfer pilot be replicated at the national level given the current capacities at the national and sub- ▪ UNICEF and national levels? ▪ CARD Scalability o What capacity building and strengthening activities / efforts were undertaken to strengthen structures within sub- national administrations?
100 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Criteria Questions Key Respondents o Was any training provided at the National Level (CARD), including to the individuals designing the project and ▪ District and those providing training to the district? Commune Level o What training sessions were undertaken to strengthen Health Centre Human Resource capabilities, particularly Officials for imparting BCC? Kindly provide the training plan, if any. o Which NGOs were involved in supporting training of trainer sessions? What was the role envisaged for NGOs? What was the selection process for NGOs? What were the actual tasks carried out? ▪ To what extent can the major capacity gaps and bottlenecks at national and sub-national levels be overcome during the life-cycle of this project? ▪ Is the project sustainable without creating any externally-funded institutions? o What were the specific challenges faced in undertaking the pilot without creating any externally-funded posts? o World Bank Cash Transfer Project structurally seems very similar to the UNICEF CARD Cash transfer project, with the main difference being only the use of government structures in the UNICEF project. Were there any other main differing components? Was this intentional and what are the expectations for convergence while scaling-up? ▪ For what time period is the National Level Project expected? Have funds been raised accordingly, taking into consideration factors like inflation? ▪ What is the level of technology and equipment across Cambodia, in terms of phone and internet connections, smartphone usage, internet and phone banking, etc.? ▪ What alternative options / complementary interventions should Cambodia develop to ensure smooth transition from over-dependence on cash transfers to being self-reliant? ▪ Why were the Save the Children and World Bank projects chosen for comparison? What will be their role in the National Level scaling-up? ▪ What are some best practices, which were witnesses in this project that are replicable at the national level? Equity and ▪ CARD Gender ▪ How is equity and gender integrated into CARD and UNICEF’s cash transfer pilot project? What approaches are ▪ District, CARD and UNICEF deploying, or does it plan to deploy, to ensure that the project targets the most vulnerable Commune and women? Probe for specific examples. Village Level Officials
101 Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 14. Critical Cost-effectiveness Drivers
Driver Parameters
Targeting efficiency • Targeting approach and its cost as percentage of total cost • Percentage of recipients not in target group • Percentage of target group not receiving transfers • Percentage of total transfers reaching target group(s) • Frequency of retargeting Transfer levels • Nominal level(s) of transfer per direct recipient per month at scheme inception • Arrangements for periodic review of levels Cost of conditionality • Public costs of monitoring conditions and private costs of compliance • Recipients’ additional use of services specified in conditions, and cost of supplying additional services Implementation systems • Costs of registration, enrolment, recipient identification and payments • Regularity of payments to recipients • Grievance / appeals procedures, actual frequency of use, including by those excluded, and outcomes • Integrity of financial management systems and control over fiduciary risk
102
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 15. Ethics and United Nations Evaluation Guidelines
Box 1: Obligations to Participants
RESPECT FOR DIGNITY AND DIVERSITY Respect differences in culture, local customs, religious beliefs and practices, personal interaction, gender roles, age and ethnicity. Consult locally when planning the research and developing protocols. Optimize demands on the respondents’ time, and respect people’s right to privacy. Ensure that opinions and answers provided by the adolescents and children are respected.
RIGHTS The participants will be treated as autonomous agents and will be given the time and information to decide whether or not they wish to participate. They reserve the right to making independent decisions without any pressure or fear of penalty for not participating. Care will be taken to ensure that relatively powerless, ‘hidden’, or otherwise excluded groups are represented.
CONFIDENTIALITY Respect people’s right to provide information in confidence and making the participants aware of the scope and limits of confidentiality. Ensure utmost care and sensitivity towards children’s right to privacy and to ensure that their information remains confidential. Securely store, protect and dispose of information / data that has been collected.
AVOIDANCE OF HARM Evaluators shall seek to minimize risks to, and burden on, those participating in the evaluation. Employ strategies to minimize distress for all participants, especially adolescents and children participating in the research Take measures to ensure that harm is not caused to adolescents, children, families or communities.
103
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 16. Health Centre Data
Pre-natal check-up utilization Figure A. Pre-natal checkup utilization in Prasat Bakong Health Centres (Source: Health Centre records)
Pre-natal check-up utilization in Prasat Bakong 900 842 812 809 795 781 800 762 773
ups 745 746
- 733 737 737 724 688 704 6… 667 686 697 698 700 657
600 537
natal Check - 500
400
NumberofPre 300
Linear (Prenatal Checkup uptake in Prasat Bakong )
Figure B. Pre-natal check-up utilization in Prasat Bakong Health Centres – IDPoor only (Source: Health Centre records)
Pre-natal check-up utilization in Prasat Bakong - IDPoor
40 3… ups - 35 29 30 30 25 24 24 23 24 23 24
25 22 21 21 20 natal Check - 20 18 18 15 13 12 10 10
5 NumberofPre 0
Linear (Prenatal Checkup uptake in Prasat Bakong - IDPoor)
104
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Figure C. Pre-natal check-up utilization in Prasat Bakong of number of pre-natal check-ups over time commune-wise (Source: Health Centre records)
Trapaing Thum HC Rolous HC 150 150 100 100 50 0 50 0
Trapaing Thum HC Linear (Trapaing Thum HC) Rolous HC Linear (Rolous HC)
Bakong HC Kantreang HC 150 100 100 50 50 0 0
Bakong HC Linear (Bakong HC) Kantreang HC Linear (Kantreang HC)
Institutional delivery and post-natal care package utilization
Figure D. Institutional delivery data for Health Centres in Prasat Bakong (Source: Health Centre records)
Institutional deliveries in Prasat Bakong 20 18
16 14 12
10 8 6 4
Figure2 E. Post-natal check-up utilization in Prasat Bakong health centres (Source – health centre records) No.of Institutional Deliveries 0
Trapaing Thum HC Rolous HC Kantreang HC Bakong HC
105
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Figure E. Post-natal check-up utilization in Prasat Bakong Health Centres (Source: Health Centre records)
Post-natal check-up utilization in Prasat Bakong 140 127 118 120 112 ups 108 97 - 105 103 97 97 102 100 100 74 81 73 78 77 80 66 65 70 63
natal Check 60 - 60 47 40
20 No.of Post 0
Linear (Total)
Figure F. Post-natal check-up utilization in Prasat Bakong Health Centres – IDPoor only (Source: Health Centre records)
Post-natal check-up utilization in Prasat Bakong - IDPoor 25 20
ups 20 - 17 17 17 16 16 16 14 14 14 14 15 13 13 12 11 10 11 10 natal Check 9 - 10 9 8 6
5 No.of Post 0
Linear (Total IDPoor)
106
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Figure G. Post-natal check-up utilization in Prasat Bakong Health Centres commune-wise (Source: Health Centre records)
Commune-wise post-natal check-up data
- 100% 90% 80%
70% natal Check - 60% 50% 40% 30% 20% 10%
ups in Prasat in ups HC's Bakong 0% Percentage ofMonthly Post
Trapaing Thum HC Rolous HC Kantreang HC Bakong HC
Figure H. Growth monitoring services utilization in Prasat Bakong Health Centres (Source: Health Centre records)
Growth monitoring utilization in Prasat Bakong 600 554
500 438 412 408 3… 388 380 387 400 375 357 355 365 346 339 330 345 330 342 307 326 313 300 291
200
100 No.of Growth Monitoring
0
Linear (Total)
107
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Figure I. Growth monitoring services utilization in Prasat Bakong Health Centres – IDPoor beneficiaries (Source: Health Centre records)
Growth monitoring utilization in Prasat Bakong - IDPoor
120 109
100 80 79 73 74 80 70 68 69 68 71 63 64 64 62 58 60 54 51 53 54 48 46 41 40
No.Growth of Monitoring 20
0
Linear (Total IDPoor)
108
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 17. Comparison of Survey Findings to Baseline Some key results from CCT Maternal Health Baseline survey, conducted by Indochina Research Ltd. are given in the box below. As mentioned earlier, the studies have a different purpose and sampling methodology and results are not directly comparable. The survey conducted as a part of the evaluation was not an end-line survey.
Key results from the Indochina Research Ltd. baseline assessment:
• 93% pregnant women in the past took any iron pills, vitamin tablets or other supplements (n=268); • 18% responded that their child had received iron supplements (n=255); • 62% households were using iodized salts (n=268); • 88% had given birth at a Health Centre, district hospital or provincial hospital (n=244).
• 93% pregnant women in the past took any iron pills, vitamin tablets or other supplements. (n=268) • 18% responded that their child had received iron supplements. (n=255) • 62% households were using iodized salts. (n=268) • 88% had given birth at a health centre, district hospital or provincial hospital. (n=244)
• 93% pregnant women in the past took any iron pills, vitamin tablets or other supplements. (n=268) • 18% responded that their child had received iron supplements. (n=255) • 62% households were using iodized salts. (n=268) • 88% had given birth at a health centre, district hospital or provincial hospital. (n=244)
• 93% pregnant women in the past took any iron pills, vitamin tablets or other supplements. (n=268) • 18% responded that their child had received iron supplements. (n=255) • 62% households were using iodized salts. (n=268) • 88% had given birth at a health centre, district hospital or provincial hospital. (n=244)
109
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 18. Trainings Conducted during the Pilot Project
Training Date Training Type Participants 18-20 May 2016 Training on CT project implementation and its District focal persons, commune operation guideline, Siem Reap town hotel in focal persons, and Health Centre Siem Reap province staff 03 August 2016 Health Education Training on Maternal District focal persons, commune Healthcare and new-born baby care at Prasat focal persons, and Health Centre Bakong district hall staff 05 October 2016 Health Education Training on Child Nutrition District focal persons, commune at Prasat Bakong district hall focal persons, and Health Centre staff 07 Dec. 2016 Training on Child Vaccination and Growth District focal persons, commune Monitoring for Children at Prasat Bakong focal persons, and Health Centre staff 07 February 2017 Health Education Training on how to take District focal persons, commune care of sick children at Prasat Bakong district focal persons, and Health Centre hall staff 04 April 2017 Health Education Training on Child District focal persons, commune Development at Prasat Bakong district hall focal persons, and Health Centre staff 12 June 2017 Health Education Training on Hand Washing District focal persons, commune and Sanitation (Toilet using) at Prasat Bakong focal persons, and Health Centre district hall staff 07 August 2017 Health Education Training on Child District focal persons, commune Development at Prasat Bakong district hall focal persons, and Health Centre staff 11 October 2017 Health Education Training on Maternal District focal persons, commune Healthcare and new-born baby care at Prasat focal persons, and Health Centre Bakong district hall staff (The training was cancelled. Only commune focal persons and Health Centre staff provided training to beneficiaries during payment day.)
110
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 19. Break-up of Costs of the CARD – UNICEF Cash Transfer Pilot
Cost Type Components Breakup of Costs Total Cost Set-up costs • Design cost • Database: US$ 8,000 US$ 8,000 • Planning cost (US$ 6,000 for first round • Training cost and US$ 2,000 for second • System / infrastructure round of adjustment) cost Roll-out costs • Targeting cost • Training and refresher US$ 28,000 • Costs for enrolment training to district and commune: US$ 1,000*3+ US$ 500*6 = US$ 6,000 • Field visit and support for refresher training and training: US$ 1,000*6 times = US$ 6,000 • Communication materials: US$ 5,000 • Distribution and registration: US$ 1,000*11 times = US$ 11,000 Operational costs • MFI payment delivery • Internet connection: US$ US$ 21,460 fees 660 • Cost of verification of co- • Junior assistants: US$ responsibility 600*18 mths = US$10,800 • Cost of education • Senior advisor: US$ sessions 10,000 Monitoring and • On-going monitoring • Monitoring and support: US$16,500 evaluation costs cost US$ 1,500*11 times= US$ • Periodic costs of 16,500 external evaluation Service fee to AMK US$ 6,285 Microfinance Total US$ 80,245
111
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 20. Indicative Parameters for Consideration for Phased Scaling-up
Area Indicator Rationale
Poverty level Per capita income Indicator of poverty levels in the Number of families under IDPoor geographic area
Level of under- Low birth weight Indicator of child nutrition status nutrition and health Anthropometric measures service utilization Severely thin mothers (measured as Indicator of maternal nutrition status BMI) % of pregnant women (measured as crude birth rate) To estimate the target group for the project No. of IDPoor households % of women receiving ante-natal and post-natal check-ups % of women having institutional To understand existing utilization of deliveries health services and potential for gains
% of children getting immunization
To understand banking infrastructure Infrastructure and Access to banks (measured as and appropriateness and operational human resource population served per bank) capability cost of payment mechanisms Number of sub-administrative staff and To understand gap in human number of vacancies resources Number of Health Centres To understand health infrastructure for Checklist of services available at the fulfilment of co-responsibilities Health Centre
112
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 21. Indicative Areas of Monitoring
Area Example of possible indicators Possible sources
• % of beneficiaries in the target group Commune council members Targeting • % of beneficiaries out of target group (for example those enrolled with no IDPoor) • % of children enrolled who were malnourished / SAM • % of beneficiaries from floating villages • % of children enrolled who were orphans • % of pregnant women enrolled were single
• % of beneficiaries who received payment during AMK Microfinance Delivery payment day mechanism • % of beneficiaries who received cash at the bank / ATM • % of beneficiaries who did not withdraw the cash transfer amount in that cycle
• Monthly number of IDPoor households who Health Centre, commune focal Conditionalities visited the health centre point • Average number of health and nutrition education sessions visited • Number of malnourished children identified in the age group of 0 to 5 years coming from IDPoor households • Utilization of health services – institutional delivery, ante-natal check-up, post-natal check- up, immunization etc.
• Number of beneficiaries Commune council members Project • Cash transferred and district administration implementation • Coverage rate • Average value of cash transfer
• Reduction in number of malnourished children Impact assessment, pay-point Results and assessment, nutritional impacts • Improvement in stunting, wasting and underweight indicators assessment etc. • Dietary diversity and consumption of better quality and more quantity of food • Usage of cash
113
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 22. Grievance Redress Mechanism of Bangladesh’s Income Support Programme
Grievance redress mechanism of Bangladesh’s income support programme for the poorest
▪ In Bangladesh’s conditional cash transfer project ‘Income support program for the poorest project’, the proposed management information system (MIS) includes a grievance redress mechanism which will address beneficiary complaints concerning targeting, payments, information updates, and complaints on quality of service. ▪ The mechanism will be managed by Grievance Redress Officers, who are appointed at various levels to address grievances according to the Operations Manual and keep a record of the details of cases lodged, cases resolved, pending cases and actions taken.
▪ There will be a feedback loop which includes informing the complainant about the action taken and feeding this information into the implementation of the programme. ▪ Complaint forms will be made available at the Safety Net Beneficiary Cells (SNCs). Completed forms must be submitted to the Safety Net Programme Assistant (SPA), who will issue applicants with a stamped receipt confirming that the form has been received. ▪ SNC staff will make a fortnightly inventory of the complaints received and enter the information into the MIS. Source: World Bank, 2014, International Development Association project appraisal document on a proposed credit▪ In in the the case amount of valid of SDR complaints 202.4 million about (US$300payments, million the office equivalent) will make to thethe People’scorrection Republic and inform of Bangladesh the for anbanks Income to releaseSupport the Program corrected for theamount Poorest with Project. the next payment. ▪ If beneficiaries fail to receive a solution they can appeal.
Grievance Redress Mechanism of Bangladesh’s Income support program for the poorest ▪ In Bangladesh’s conditional cash transfer project - ‘Income support program for the poorest project’, the proposed Management Information System (MIS) includes a grievance redress mechanism which will address beneficiary complaints concerning targeting, payments, information updates, and complaints on quality of service. ▪ The mechanism will be managed by Grievance Redress Officers, who are appointed at various levels to address grievances according to the Operations Manual; and keep a record of the details of cases lodged, cases resolved, pending cases and actions taken. ▪ There will be a feedback loop which includes informing the complainant about the action taken, and feeding this information into the implementation of the programme. ▪ Complaint forms will be made available at the Safety Net Beneficiary Cells (SNCs). Completed forms must be submitted to the Safety Net Program Assistant (SPA), who will issue applicants with a stamped receipt confirming that the form has been received. ▪ SNC staff will make a fortnightly inventory of the complaints received, and enter the information into the MIS. ▪ In the case of valid complaints about payments, the office will make the correction and inform the banks to release the corrected amount with the next payment. ▪ If beneficiaries fail to receive a solution they can appeal.
Grievance Redress Mechanism of Bangladesh’s Income support program for the poorest ▪ In Bangladesh’s conditional cash transfer project - ‘Income support program for the poorest project’, the proposed Management Information System (MIS) includes a grievance redress mechanism which will address beneficiary complaints concerning targeting, payments, information updates, and complaints on quality of service. 114 ▪ The mechanism will be managed by Grievance Redress Officers, who are appointed at various levels
to address grievances according to the Operations Manual; and keep a record of the details of cases lodged, cases resolved, pending cases and actions taken. ▪ There will be a feedback loop which includes informing the complainant about the action taken, and feeding this information into the implementation of the programme. Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 23. Internal Quality Review Process
IPE Global is an ISO 9001:2015 certified company (accreditation agency: Bureau Veritas UKAS Quality Management). Quality procedures have been refined as per the ISO certified quality and procedure manual to ensure better services to employees and clients and enhance project outputs. There is a well-entrenched system comprising of periodic departmental and peer reviews, interactive feedback mechanisms, complaint redressal and quality checks. Quality Policy Statement: “We are committed to exceed client expectation by delivering value-for- money services that lead to the twin objectives of economic growth and social equity through continual improvement of our quality management system.” Policies, Procedures and Practices for Quality Outputs: We have policies and practices to promote quality in (i) the workplace, (ii) interaction with clients, and (iii) outputs produced by us. The human resource department is responsible for quality at the work place; we have regular reviews to check the quality of work at different stages by the certified internal auditors in each department. We place strong emphasis on effective management to ensure that projects are successfully completed on time and within the stipulated budget. We have developed an interactive approach to promote liaison between the IPE Global team and client personnel. Features of this approach, including undertaking peer review by the Directorial staff of IPE Global has proved very successful. Quality Management System: The quality planning is done for undertaking each project so as to meet the requirements of each client and QMS requirements set under ISO 9001:2008. All activities are monitored at appropriate points / stages through collection of appropriate data in set formats. Evaluation and analysis of this data is carried out to identify problems and take appropriate corrective actions leading to continual improvement. Deliverables to be submitted are frequently assessed by the respective team through peer review and by senior experts committed to the project. Client interaction at regular intervals helps us to produce the output to their satisfaction. The effectiveness of the QMS in the overall functioning is monitored by the Management Representative and reviewed in the Management Review Meeting by the Managing Director. Staff for Quality Assurance: Quality of IPE Global’s performance over the life of an assignment is ensured by the staff responsible for quality assurance, including Team Leader, Head of the Department which is undertaking the assignment, Project Manager concerned, Peer Reviewers, and the certified internal auditor. In addition, IPE frequently takes inputs of senior personnel and experts on outputs developed for any consulting assignment. The administration and finance wing is solely responsible for ensuring quality control in areas of administration, invoicing and other related areas. Value for Money: Our experience in providing consulting services in a competitive environment allows us to set realistic fees which are cost-effective for clients and allow the deployment of appropriate and qualified resources to meet the particular needs of each project undertaken. In addition, we are continually developing our systems to provide improvements to our services and to make our processes more cost efficient. In this regard, the implementation of our QMS supported by a commitment to Total Quality Management ensures we remain in the forefront of our industry in terms of value and service provided. Internal Controls: IPE Global has set up an internal complaint redress system which works through a certified internal auditor in each department and the Human Resource Department. Any complaint during the process of work and even after completion of the project is immediately brought to the attention of the Head of the Department undertaking the assignment, and is addressed at the earliest by the Head of Department, Peer Reviewer, and/or the assignment Team Leader.
115
Evaluation of the CARD and UNICEF Cash Transfer Pilot Project for Pregnant Women and Children
Annex 24. Team Composition
Ashish Mukherjee, Team Leader has over 18 years’ experience of leading and managing various long-term evaluations and technical assignments in the areas of health, nutrition and social protection. He has vast understanding of various evaluation techniques, data analysis and evidence based research methodologies. He brings on-board extensive experience in conducting qualitative and quantitative evaluations. His skills lie in M&E, surveys & research studies, project planning & leading multi-disciplinary implementation teams. He has broad experience of working as a Team Leader with government and non-government bodies and multilateral and bilateral donor agencies like UNICEF, DFID, ADB, World Bank, etc. across South and East Asian countries. Ashish has recently completed implementation of a Cash Transfer assignment in the education sector funded by DFID - Delivery of a Conditional Secondary School Incentive Project for Disadvantaged Girls in Odisha, India. In this project, he led the monitoring and evaluation component, developed the log frame for M&E and undertook analysis of key performance indicators of the project. Another relevant assignment that he undertook in East Asia was Independent Monitoring and Evaluation of the Nutrition Improvement through Community Empowerment (NICE) Project, Indonesia. In this project, he designed the M&E system for all phases of implementation and undertook robust analysis of performance indicators such as nutrition status of children under-five years of age & pregnant and lactating women.
Dr. Chey Tech, National Technical Expert has over 16 years of work experience in Cambodia with expertise in monitoring and evaluation of projects in the domains of health, governance, gender, education and agriculture. He has successfully provided consultancy services in terms of project planning, monitoring & evaluation design, project mid-term and final evaluation and conducting surveys using mixed method evaluation techniques for several assignments including cash transfer, democratic development, and social protection projects. He has undertaken implementation, organizational management, assessment and other studies for various organizations including local and International NGOs, UN agencies, multilateral funding agencies and government institutions such as UNICEF, UNDP, ADB, CARD, CEDAC, Banteay Srei, ADRA, KAPE etc. Chey has developed evaluation frameworks, developed and implemented strategies for project implementation, conducted mixed method evaluations and undertaken quantitative and qualitative data collection for projects such as Qualitative Governance Survey for NCDD-S, Outcome Evaluation of UNDP Cambodia Country Program Action Plan 2011-2015 for Democratic Governance Outcomes, Evaluation of the Beacon School Initiative project, among others.
Kriti Gupta, Research Assistant: Kriti has completed her degree in Economics from Delhi University, India and her Masters in Economics from Jawaharlal Nehru University, India. Kriti is an Assistant Manager at IPE Global and brings in 3 years of experience in quantitative and qualitative data analysis, designing surveys and research. She is trained in undertaking quantitative data analysis using STATA and SAS software. Kriti has experience in Finance and International Development sectors, with specific expertise in Socio-Economic and Health Assessments, Economic Profiling, Primary Data Collection, Risk Management, Credit Rating & Reporting and Financial Analysis.
116