Of the SHOUHARDO III Program

Mid-Term Evaluation (MTE) of the SHOUHARDO III Program

A USAID-funded Title II Development Food Security Activity implemented by CARE Bangladesh Award Number: AID-FFP-A-15-00009

Methodology Report (MTE Report, Volume II)

17 July 2018 (Final)

SHOUHARDO III MTE Methodology Report 17 July 2018

Acknowledgement

The members of the MTE team would like to gratefully acknowledge the support provided by the staff of the SHOUHARDO III Program. Special thanks go to Khaledul Ahsan from CARE for assisting the team with data collection for the period when Rukhsana was away due to other commitments and to Walter Mwasaa, SHOUHARDO III Chief of Party, and Raquibul Hasan, the Senior M&E Coordinator for SHOUHARDO III, for their diligent support throughout the MTE process. We also want to thank all of the field staff from CARE and the implementing partners who went out of their way to provide logistical support to enable the team to implement an intensive schedule over a relatively short period. We also greatly appreciated their participating in various events during the evaluation and for providing frank insights while answering our questions. Innumerable program participants and beneficiaries have also provided useful responses and participated in stimulating discussions. Our gratitude goes out to them as well. Hopefully we've listened well, our observations are grounded in reality, our assessment is accurate, and our recommendations will be useful.

Sincerely,

Mike DeVries, Independent Consultant (Team Leader) Rukhsana Haider, Independent Consultant Mark Langworthy, TANGO Jeanne Downen, TANGO Golam Kabir, Independent Consultant

Translators: Apurba Kumar Roy Al Masud Litu Mohammed Razu Ahmed

Cover pictures taken by Jeanne Downen (left) and Hedyatul Islam (right)

This report was made public by support from the U.S. Agency for International Development (USAID). The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the Government of the United States of America.

SHOUHARDO III MTE Methodology Report 17 July 2018

TABLE OF CONTENTS Page I. EXECUTIVE SUMMARY 1 II. MTE SCOPE OF WORK 2 III. MTE PROTOCOL 35 IV. PERSONS INTERVIEWED AND SITES VISITED 131

iii

SHOUHARDO III MTE Methodology Report 17 July 2018

ACRONYMS AND ABBREVIATIONS ADS Automated Directive System Ag & L Agricultural & Livelihoods AOR Agreement Officer’s Representative APM Assistant Project Manager ARR Annual Results Report BBSS Beneficiary-Based Survey System BCC Behavioral Change Communication C&F Cargo & Freight C<5 Child Under Five Years of Age CAP Community Action Plan CAV Community Agriculture Volunteer CBO Community-Based Organization CC Community Clinic CCA Climate Change Adaptation CEV Community Empowerment Volunteer CFW Cash for Work CG Community Groups CHD Comprehensive Homestead Development CHV Community Health Volunteer C-IMCI Community-based Integrated Management of Childhood Illness CLTS Community-led Total Sanitation COG Core Occupational Group CoP Chief of Party CSC Community Score Card CSG Community Support Group CSV Comma Separated Value CVCA Climate Vulnerability & Capacity Assessment DAE Department of Agricultural Extension DAM Dhaka Ahsania Mission DCRM Disaster and Climate Risk Management DEC Development Experience Clearinghouse DFAP Development Food Assistance Project DFSA Development Food Security Activity DIP Detailed Implementation Plan DPHE Department of Public Health Engineering DRR Disaster Risk Reduction DYD Department of Youth Development EBEK Ektee Bari Ektee Khamar EHV Extremely Highly Vulnerable EKATA Empowerment Knowledge and Transformative Action EMMP Environmental Monitoring & Mitigation Plan EoI Expression of Interest EPI Extended Program of Immunization ESDO Eco-Social Development Organization EVAW Ending Violence Against Women

SHOUHARDO III MTE Methodology Report 17 July 2018

FF Field Facilitator FFBS Farmer Field and Business School FFP Food for Peace FFPMIS Food for Peace Management Information System FGD Focus Group Discussion FNS Food & Nutrition Security FS Field Supervisor FY Fiscal Year GBV Gender-Based Violence GMP Growth Monitoring and Promotion GoB Government of Bangladesh HCA Host Country Agreement HHN Health, Hygiene and Nutrition HQ Headquarters HV Highly Vulnerable I&EO Infrastructure & Environment Officer IDI In-Depth Interview IEE Initial Environmental Examination IGA Income-Generating Activity IMCI Integrated Management of Childhood Illness INGO International Non-Governmental Organization IPTT Indicator Performance Tracking Table ITSH Inland Transport Storage and Handling IYCF Infant & Young Child Feeding KII Key Informant Interview KMAL Knowledge Management, Advocacy and Learning KML Knowledge Management & Learning LEB Locally Elected Body LF Logical Framework LGD Local Government Division LGED Local Government Engineering Department LGRD&C Ministry of Local Government, Rural Development & Cooperatives LoA Life of Activity or Life of Award M&E Monitoring and Evaluation MCHN Maternal and Child Health and Nutrition MEAL Monitoring & Evaluation Accountability & Learning MFI Micro-Finance Institution MJSKS Mahideb Juno Somaj Kallayan Somity MoH&FW Ministry of Health & Family Welfare MT Metric Ton MTE Mid-Term Evaluation MUAC Middle Upper Arm Circumference NBD Nation Building Department NDP National Development Programme NGO Non-Governmental Organization NNPC Nari Nirjaton Protirodh Committee

SHOUHARDO III MTE Methodology Report 17 July 2018

NRM Natural Resource Management P&P Pen & Paper PACC Program Advisory and Coordinating Committees PEA Program Environmental Assessment PEP Poor and Extreme Poor PIIF Program Integration Implementation Framework PLW Pregnant and Lactating Women PM&ES Planning, Monitoring and Evaluation System PM Project Manager PMP Performance Monitoring Plan PNGO Partner Non-Governmental Organization PoC Point of Contact POPI People’s-Oriented Programme Implementation PRA Participatory Rural Appraisal PREP Pipeline and Resource Estimate Proposal PVO Private Voluntary Organization QMR Quarterly Monitoring Report QR Quarterly Report RF Results Framework SAM Severe Acute Malnutrition SAPQ Standardized Performance Annual Questionnaire SBCC Social and Behavioral Change Communication SBTB School-Based Teen Brigade SHOIII or SHO3 SHOUHARDO III SHOUHARDO Strengthening Household Ability to Respond to Development Opportunities SKS SKS Foundation SoW Scope of Work SO-TSR Senior Officer Technical Support & Representation SP Sub-Purpose SPSS Statistical Package for the Social Sciences STATA Statistics and Data STC Senior Technical Coordinator SUAP Safer Uses Action Plan TANGO Technical Assistance to NGOs TM Technical Manager TO Technical Officer ToC Theory of Change ToR Terms of Reference ToT Training of Trainers UDCC Union Disaster Coordination Committee UDMC Union Disaster Management Committee UDV Union Disaster Volunteers UP Union Parishad UHC Upazila Health Complex USAID United States Agency for International Development

SHOUHARDO III MTE Methodology Report 17 July 2018

USD United States Dollar UzDMC Upazila Disaster Management Committee VDC Village Development Committee VFA Veterinary Field Assistant VSLA Village Savings and Lending Association WASH Water, Sanitation and Hygiene WBA Well-Being Analysis WE Women’s Empowerment

vii

SHOUHARDO III MTE Methodology Report 17 July 2018

SHOUHARDO III Mid-Term Evaluation METHODOLOGY REPORT

I. EXECUTIVE SUMMARY This Volume II of the report for the Mid-Term Evaluation (MTE) of the SHOUHARDO III Program implemented by CARE and local partners in Bangladesh summarizes the methodology that was used to conduct the MTE over the period from late October 2017 through mid-June 2018. This resource document is being provided as a separate document from the MTE Summary Report (Volume I) which summarizes the priority recommendations that emerged from the MTE Process. The section which immediately follows provides the Scope of Work for the MTE which initiated the planning for the exercise. The second section provides the MTE Protocol that was used to operationalize the exercise. The MTE protocol contains the data collection tools which were used by various team members over the course of the exercise. The various schedules that were used to implement the MTE are also included in the MTE Protocol. The third section lists the actual interviews and focus group discussions held by the MTE team members while conducting primary data collection in Bangladesh from February 13 through March 6, 2018.

II. INITIAL SCOPE OF WORK FOR THE SHOUHARDO III MTE

SHOUHARDO III MTE Methodology Report 17 July 2018

MID TERM EVALUATION SCOPE OF WORK

CARE Bangladesh SHOUHARDO III

USAID/FFP Title II Program

Bangladesh, 2015 – 2020

Final Submission 22nd August 2017

SHOUHARDO III MTE Methodology Report 17 July 2018

Table of Contents

1 INTRODUCTION ...... 7 1.1 Program Overview ...... 7 1.2 Activity Background ...... 11 2 MIDTERM EVALUATION (MTE) ...... 15 2.1 MTE Objectives ...... 15 2.2 Key Evaluation Questions ...... 16 3 MTE METHOD ...... 18 4 MTE CONTRACTOR’S RESPONSIBILITIES ...... 20 4.1 Deliverables from the MTE Contractor ...... 20 4.1.1 MTE Final Report ...... 22 4.2 Pertinent Permissions, Approvals, Insurance, and Other Required Permits ...... 24 4.3 Time Frame and Interim Submission ...... 24 4.5 Logistics ...... 26 5 MTE TEAM COMPOSITION, QUALIFICATIONS, AND ROLES ...... 26 5.1 General Roles & Responsibilities ...... 26 5.2 MTE Team Composition ...... 27 5.3 MTE Team Members’ Specific Roles ...... 28 5.3.1 Team Leader’s Role ...... 28 5.3.2 Technical Specialists ...... 29 6 PROGRAM RESPONSIBILITIES ...... 29 7 INTELLECTUAL PROPERTY ...... 31 8 ETHICAL GUIDELINES ...... 31 9 BUDGET ...... 32 10 EVALUATION CRITERIA ...... 32 11 MANAGING UNEXPECTED ...... 32 12 LIST OF ANNEXES ...... 33 13 RESOURCES (To be provided upon signature of agreement) ...... 33

SHOUHARDO III MTE Methodology Report 17 July 2018

GLOSSARY AND LIST OF ACRONYMS

ARR : Annual Results Report AOR : Agreement Officer’s Representative ADS : Automated Directive System BCC : Behavior Change Communication C-IMCI : Community-based Integrated Management of Childhood Illness C-MAM : Community-based Management of Acute Malnutrition CBO : Community based Organization CSV : Coma Separated Value CoG + : Core occupational Group Plus DFAP : Development Food Assistance Plan DEC : Development Experience Clearinghouse DCRM : Disaster and Climate risk management DIP : Detailed Implementation Plan EoI : Expression of Interest EKATA : Empowerment, Knowledge and Transformative Action EHV : Extremely Highly Vulnerable EVAW : Ending Violence against Women EMMP : Environmental Monitoring & Mitigation Plan FFBS : Field & Farmer Business School FNS : Food and Nutrition Security FFP : Food for Peace FFPMIS : Food for Peace Management Information System FY : Fiscal Year FGD : Focus Group Discussion GoB : Government of Bangladesh GBV : Gender-based Violence GMP : Growth Monitoring & Promotion HV : Highly Vulnerable HHN : Health Hygiene & Nutrition HQ : Head quarter IPTT : Indicator Performance Tracking Table IEE : Initial Environment Examination IDI : In-depth Interview IGA : Income Generating Activity INGO : International Non-governmental Organization KML : Knowledge Management & Learnings KII : Key Informant Interview LGD : Local Government Division LGRD&C : Ministry of Local Government, Rural Development and Cooperatives LF : Log Frame LEB : Local Elected body LOA : Life of Award

SHOUHARDO III MTE Methodology Report 17 July 2018

MTE : Midterm Evaluation M&E : Monitoring & Evaluation MoH&FW : Ministry of Health and Family Welfare MEAL : Monitoring & Evaluation Accountability & Learning NBD : Nation Building department NGO : Non-governmental Organization NRM : Natural Resource Management PVO : Private Volunteer Organization PREP : Pipeline Resource Estimate & Proposal PNGO : Partner Non-Governmental Organization PEP : Poor and Extremely Poor PACC : Program Advisory and Coordinating Committee PRA : Participatory Rural Appraisal POC : Point of Contact P&P : Pen and Paper QR : Quarterly Report RF : Results Framework SBCC : Social and Behavioral Change Communications STATA : Statistics and Data SPSS : Statistical Package for the Social Sciences SOW : Scope of Work SHOUHARDO III: Strengthening Household Ability to Respond to Development Opportunities III SP : Sub purpose SAPQ : Standardized Performance Annual Questionnaire TOC : Theory of Change USAID : The United States Agency for International Development VDC : Village Development Committee VSLA : Village Saving and Loan Association WBA : Well Being Analysis WASH : Water, Sanitation and Hygiene

SHOUHARDO III MTE Methodology Report 17 July 2018

INTRODUCTION

Program Overview

Strengthening Household Ability to Respond to Development Opportunities III (SHOUHARDO III) program is a Development Food Security Activity (DFSA) funded by United States Agency for International Development (USAID). It is a five-year program, with a goal to achieve improved gender equitable food and nutrition security and resilience of vulnerable people living in the Char1 and Haor2 regions of Bangladesh by 2020. The program is designed to reach Poor and Extreme Poor (PEP) people living in the program areas covering eight districts in Northern Bangladesh. SHOUHARDO III is implemented in partnership with six national Partner Non- Government Organizations (PNGOs) who deliver all activities with technical support from CARE. The integrated, multi-sectoral program includes activities that together increase agricultural productivity and household incomes; enhance maternal and child nutrition and health status; build household and communal resilience to disasters; achieve women’s and youth empowerment and improve public service delivery to rural communities. Specifically, the program runs from 29th September 2015 to 28th September 2020. The overall program value is USD 80 million from the United States Government with a complementary funding of USD 7,707,490 million from the Government of Bangladesh (GoB). It aims to achieve lasting change for 674,856 PEP women, adolescent girls, and children below five, men, and the youth – being made possible through the integrated development approach in its interventions.

To achieve its goal, five purposes have been designed namely:

i. Increased equitable access to income for both women and men, and nutritious food for men, women, boys and girls; ii. Improved nutritional status of children under five years of age, pregnant and lactating women and adolescent girls; iii. Strengthened gender equitable ability of people, households, communities and systems to mitigate, adapt to and recover from man-made and natural shocks; iv. Increased women’s empowerment and gender equity at family and community level and v. Provision and utilization of public services (i.e. Local Elected Bodies & Nation Building Departments) for communities especially for PEP increased.

SHOUHARDO III program has been touted as a model for reducing child malnutrition while contributing to greater livelihood security, resilience and women's empowerment. The program works across multiple sectors to address the underlying causes of poverty among the PEP people residing in Northern Bangladesh. The cross-sectoral integration is further strengthened by the inclusion of two cross-cutting approaches: women empowerment and push of improved governance.

1 Char a tract of land surrounded by the waters of an ocean, sea, lake, or stream; it usually means, any accretion in a river course or estuary. (http://en.banglapedia.org/index.php?title=Char) 2 A Haor is a wetland ecosystem in the north eastern part of Bangladesh which physically is a bowl or saucer shaped shallow depression, also known as a back swamp. (https://en.wikipedia.org/wiki/Haor) 7

SHOUHARDO III MTE Methodology Report 17 July 2018

CARE SHOUHARDO III program’s Logical Framework

The Logical Framework (LF) of SHOUHARDO III Program was primarily designed based on the program’s Theory of Change (TOC) and vulnerability assessment. The LF matrix captures program details including narrative summary, purposes, sub-purposes, intermediate outcomes, outputs, indicators and assumptions.

Table 1: SHOUHARDO III LF: Goal, Purposes, and Sub-purpose (This is a summary of the Log Frame. A complete version of LF is available in Annex A)

Goal: Improved gender equitable food and nutrition security and resilience of the vulnerable people living in the Char and Haor in Bangladesh by 2020 Purpose 1: Increased equitable access to income for both women and men, and nutritious food for men, women, boys and girls Sub-Purpose 1.1: Increased Sub-Purpose 1.2: Increased Sub-Purpose 1.3: Sub-Purpose 1.4: agricultural production of access to agricultural Increased off farm Increased utilization PEP markets for PEP income for PEP of financial services by PEP Purpose 2: Improved nutritional status of children under five years of age, pregnant and lactating women and adolescent girls Sub-Purpose 2.1: Increased Sub-Purpose 2.2: Improved Sub-Purpose 2.3: utilization of nutritious food access to health and Reduced for PLW, C<5, and nutrition services prevalence of adolescent girls water-borne diseases Purpose 3: Strengthened gender equitable ability of people, households, communities and systems to mitigate, adapt to and recover from man-made and natural shocks Sub-Purpose 3.1: Increased Sub-Purpose 3.2: Local preparedness of PEP Government Institutions’ households and (Union Parishad) Capacity communities to mitigate and and implementation of respond from shocks DCRM activities increased Purpose 4: Increased women’s empowerment and gender equity at family and community level Sub-Purpose 4.1: Sub-Purpose 4.2: Improved Strengthened agency of environment for women’s women empowerment Purpose 5: Provision and utilization of public services (i.e. Local Elected Bodies & Nation Building Departments) for communities especially for Poor and Extreme Poor (PEP) increased

Sub-Purpose 5.1: Sub-Purpose 5.2: Communities (especially Accountability of LEBs & PEP) are better able to NBDs to the needs of demand and negotiate communities especially to increased public (UPs & PEP increased NBDs) services

SHOUHARDO III MTE Methodology Report 17 July 2018

Theory of Change (ToC)

A detailed diagram of the TOC can be found in Annex B, showing the interconnections between the various components of the program, as well as how the sub-purposes (SP) and outcomes contribute to the purposes and the overall goal, with risks and assumptions identified.

Figure 1. Summary of the Theory of Change (See Annex B for complete version of the TOC)

Program Coverage and Context

SHOUHARDO III serves the most vulnerable and marginalized people. In the context of Bangladesh, these are the "extreme poor"—those do not have land to grow food. According to the final baseline study report of DFSA for Bangladesh, among the PEP, the typical household has adequate food only for 6.8 months per year and the per capita cash income is about $96 per year. Over 35% of children from these households aged under two are underweight and over 58% have stunted weight to height ratio. Only 11% children of 6-23 months receive an acceptable diet other than breast milk. In the Char and Haor regions where the program operates, an estimated 55% of the population are classified as PEP with high malnutrition. The situation is exacerbated by the geographical setup of the two regions which are frequented by floods due to monsoon season and upstream water flows:

Char Region: Kurigram, Gaibandha, Sirajganj and Jamalpur Districts

Haor Region: Kishoreganj, Netrokona, Habiganj and Sunamganj Districts

SHOUHARDO III MTE Methodology Report 17 July 2018

Figure 2: A map of SHOUHARDO III Geographical Coverage

SHOUHARDO III MTE Methodology Report 17 July 2018

Table II: SHOUHARDO III geographic area and population coverage

Activity Background

SHOUHARDO III is implemented through six local Partner Non-Governmental Organizations (PNGOs). The PNGOs implement all program activities at community level, monitor results and share program learning, among others. Each PNGO brings to SHOUHARDO III specific technical expertise as well as long and successful experience in improving food security and resilience in the program’s targeted districts. In addition, CARE has facilitated partnerships with World Fish, an international organization specializing in fisheries and private sector in the finance and agricultural marketing and production.

At the village level, the program works with Village Development Committees (VDC) from the communities representing a cross section of its demographics and interest with 50/50 male and female membership. The VDCs perform a dual role for supporting SHOUHARDO III implementation and ensuring sustainability for continued delivery of community-led activities. They also oversee the work of community volunteers.

To enhance ownership and facilitate implementation, as well as achieve national and local level program integration and alignment, the program works closely at different levels with the GoB. SHOUHARDO III formed the Program Advisory and Coordinating Committee (PACC) - a formalized structure at national, division, district and Upazila levels that bring together representatives from 14 ministries and relevant nation building departments. The committee plays a significant role to advice on strategic directions and policy guidelines of the program.

For further details, please see attached guidelines titled as Geo Priority and Vulnerable People Targeting Guides Steps Tools_SHOUHARDO III in Annex C.

SHOUHARDO III MTE Methodology Report 17 July 2018

Key Activities3

SHOUHARDO III’s key activities include: distribution of food rations to pregnant women and lactating mothers having children under two years of age; cash for work for program participants in areas where relevant activities are identified during lean period; infrastructure development and maintenance; increase in opportunities for women and push towards gender equity in communal socio-economic affairs; farmers training and agricultural inputs support; non-farm income generation skills training and building up of productive assets; savings promotion and access to loan; nutritional supplement promotion; linkages to health care service providers; training on nutrition, health and sanitation; mobilization of local resources for economic gains and hazard preparedness; inclusive local planning, early storm and flood warning; preparation of community and regional disaster response plans; and creating both demand and supply of improved service provision to communities.

To achieve the goal, SHOUHARDO III focuses on three principals and two cross-cutting purposes all of which are fully integrated and interdependent. The below section briefly states what each purpose covers. More details are found in the program’s Theory of Change and implementation guidelines.

(Further details are available in DFAP proposal, work plans and program guidelines/manuals.)

Purpose 1: Increased equitable access to income for both women and men, and nutritious food for men, women, boys and girls.

In rural Bangladesh, most poor people grow food to survive, yet most have access to limited or no land apart from their homesteads. The program provides training to the target communities to practice homestead gardening, with poultry and livestock rearing. Other participants are engaged in field crop production who are provided training on improved farming methods when they have access to land.

Non-farm income generating opportunities is also a key program area along with improved access to markets, savings mechanisms and access to credit. In addition to improve access to markets and income opportunities from non-farm sources, the program is also working at developing the life skills of its participants and employability among the youth.

The program has taken initiatives to build capacity of off-farm income-generating activities (IGA) participants based on the opportunity and interest of participants and considering market demand involving market actors. There are less off-farm income options to involve more number of participants in the remote villages. This has been a challenge for the program as participants are often not willing to move out of their villages.

3 Further details on program interventions, strategy, operational modalities, are available in the DIP and Operation Manuals/ Guidance developed for each purposes as listed in the resource section at the of the SOW.

SHOUHARDO III MTE Methodology Report 17 July 2018

Purpose 2: Improved nutritional status of children under five years of age, pregnant and lactating women and adolescent girls.

SHOUHARDO III is working to improve the nutritional and health outcome for children, women and other household members. In a context where ante and pre-natal care is weak, with stunting, wasting and underweight children are common, SHOUHARDO III is conducting courtyard sessions for pregnant and lactating mothers, sessions on growth monitoring practices (GMP), sustained lobbying and engagement with both government and private sector service providers to improve service delivery both in quality and penetration to remote locations.

The program aims to achieve this purpose through increased utilization of nutritious food for pregnant and lactating women, children under five, and adolescent girls, improved access to health and nutrition services, and reduced prevalence of water-borne diseases. In line with the SHOUHARDO III design, the program employs push-and-pull strategies to achieve the purpose. Push strategies include activities geared at stabilizing participants’ nutritious food consumption through promoting homestead gardens and nutrition-enhancing IGAs, access to micro-nutrient supplements especially from Ministry of Health and Family Welfare (MoH&FW) sources, provision of supplementary food rations to targeted pregnant women, lactating mothers and family members, targeted Social and Behavioral Change Communications (SBCC), i.e. Exclusive Breastfeeding, Separation of Children from Animal Feces, Intra-household food equity and Hand washing by mothers at critical times. These efforts will ultimately help in improved nutritional status of children under five years of age, pregnant and lactating women and adolescent girls.

Purpose 3: Strengthened gender equitable ability of people, households, communities and systems to mitigate, adapt to and recover from man-made and natural shocks.

Bangladesh is particularly vulnerable to the effects of weather related disasters and emergencies mainly experienced in the form of floods, shifting rivers and cyclones. The program is increasing awareness of potential disasters, defining and implementing preparation and mitigation measure at community and institutional (Union Parishad) level. These are coupled with higher level action that includes national level resilience building initiatives especially around early warning and disaster response by GoB and other development actors to provide infrastructure and early warning dissemination that helps to reduce impact and effect of shocks on communities through preparedness, mitigation and response from the community and union level disaster management plan. Further activities towards diversifying income sources, increasing access to formal and informal safety nets, savings among other adaptation mechanisms are assisting residents better adapt to the shocks.

Purpose 4: Increased women’s empowerment and gender equity at family and community level.

Ensuring that women can play their role is critical to achieve economic and social development. The program is working to increase the agency of women and adolescent girls through Empowerment, Knowledge and Transformative Action (EKATA) model. EKATA uses a participatory process to enable groups to identify root causes of gender-based issues and find solutions to these problems. They prepare and implement action plans using their own

SHOUHARDO III MTE Methodology Report 17 July 2018

resources. Women and girls use training in leadership, group dynamics and legal rights to complete gender analyses in their communities and work on VDCs to contribute to their own and local development.

The program is also using boys and men engagement approach to address social barriers to women empowerment. This is coupled with specific work with local influencers such as religious leaders and other people in the society to initiate and sustain a discussion on the challenges.

Purpose 5: Provision and utilization of public services (i.e. Local Elected Bodies & Nation Building Departments) for communities especially for poor and extreme poor increased.

SHOUHARDO III acknowledges that the primary mandate and role of service provision lies with the GoB. As the country progresses on a steady path of economic advancement, there are still gaps. It is in this context that the program engages local planning and government offices to ensure that services are reaching the farthest communities. This is done by ensuring enhanced supply of the services and at the same time building community level capacity to demand for the services. Many communities do not have local mobilization mechanisms, hence the program is continuing its work with VDCs as the key vehicle to ensure inclusive planning, prioritization and demand for services from lower level government responsible agencies e.g. Union Parishad and government service delivery departments. SHOUHARDO III is using a three-pronged strategy to work with youth to transform gender and power relationships in their homes and communities, ultimately leading to sustainable positive changes in the nutritional and health status of women and children.

SHOUHARDO III Management Structure

SHOUHARDO III Program Structure Review: SHOUHARDO I and II achieved global recognition for results in women’s empowerment and nutrition. Since its inception in 2004, SHOUHARDO has built up a cadre of qualified staff and supportive government, NGO and community relations. To further enhance this impact and to align the program to the changing context and focus areas, a program structure review took place. The purpose of the structure review was to review SHOUHARDO III staffing functions and structure, to identify gaps and areas to be aligned, and to make recommendations for functional and structural alignment to enable SHOUHARDO III to achieve its goal.

Drawing the findings, three interconnected recommendations came out from the structure review. The first recommendation is to revise current functions & structure in order to: (i) remove overlap and inefficiencies and to increase ownership and accountability; (ii) achieve the coherence demanded of the design (i.e. strategically layered, sequenced, and integrated); and (iii) promote an equitable and enabling workplace. These changes have resulted in some downsizing which will lead to efficiencies and free up resources to support quality assurance and learning both within CARE and the partners. The second recommendation is to establish a more intentional learning agenda that also promotes collaboration across functions. And finally, the third recommendation is to move beyond ‘stand-alone training’ to a deeper understanding of capacity development linked to performance metrics. Followed through a consultative process at various level, the staffing structure, functions and organogram has finalized and

SHOUHARDO III MTE Methodology Report 17 July 2018

approved by USAID and CARE SMT which already began the implementation from March 1, 2017. The revised functional organogram for Dhaka, CARE field offices and Partner NGOs staffing are featured in an annex to this – Annex E SHOUHARDO III Program Management, Coordination and Staffing Plan

2. MIDTERM EVALUATION (MTE)

Primary Purpose

The MTE will be conducted as a process evaluation centered on the implementation methods, interventions management and operational modalities. The MTE will primarily test how well the program activities are following implementation plans and provide a clearer picture on whether the specific interventions are relevant to addressing the underlying causes of food and livelihood insecurity in SHOUHARDO III areas.

Additionally, this evaluation is intended to verify evidence of early changes in the target communities and population: positive and negative, intended and unintended, and compare them to the changes anticipated in the TOC and LF. Moreover, it will also seek to identify the factors in the activity’s implementation or context that appear to promote or impede the changes. It also aims to recommend adjustments to improve the effectiveness and appropriateness of interventions in the targeted communities.

Moreover, the MTE should examine the internal staff and management structure and the impact of recent structural revision over the activity implementation. The MTE should review how resources are utilized, sub grant managed, internal and external communication and coordination done, compliance USAID policy and regulation, partnerships and relationship with host government and other stakeholders

MTE Objectives

Being a process evaluation, the MTE will focus on implementation processes – including project management, monitoring, the application of results/findings of program studies, and intervention protocols including outputs and inputs.

OBJECTIVES: The specific objectives of the MTE are as follows:

I.To evaluate the quality and effectiveness of program delivery – this will cover both CARE and its implementing partners – by looking into the strengths and weaknesses of activity implementation and program management; quality of outputs in terms of adherence to terms, agreed with FFP and their appropriateness and perceived value to target communities; and identifying factors that appear to enhance or detract from the quality. II.To present yearly evidence of changes (intended and unintended, positive and negative) associated with activity interventions and outputs; assess how well the observed changes reflect the TOC and LF; and identify factors in the implementation or context that impede or promote the observed and intended changes.

SHOUHARDO III MTE Methodology Report 17 July 2018

III.To review systems for capturing and documenting lessons learned and assess the extent to which they are used in project implementation and refining project design, including feedback from the perspective of stakeholders and participants. This would cover assessing processes to use evidence including baseline results and monitoring data for adjusting project strategies. Also it will assess how well the project is seeking out, testing and adapting new ideas and approaches to enhance projects’ effectiveness or efficiency. IV.To determine the extent to which outcomes, systems and services are designed and being implemented to continue after the project ends and assess progress made on implementing sustainability strategies. V.To determine the appropriateness and effectiveness of support for gender equity in terms of access to, participation in, and benefit from project interventions. Assess the extent to which project interventions target youth, support greater capacities for local governance and address sources of environmental risk. VI.To recommend adjustments to the TOC and LF, activity design, resource allocation, activity management, M&E Plan, or implementation that could improve the likelihood of achieving sustainable results by the program’s end, based on the yearly evidences collected and conclusions drawn for the evaluation objectives above. Key Evaluation Questions

For Objective 1: 1. How well have the interventions met planned schedules, targets, and outputs? What factors promoted or inhibited adherence to schedules? How were problems and challenges managed? 2. What are the strengths and challenges to the overall implementation process, management, communication and collaboration so far? What factors appear to promote or challenge the activity operations or effective collaboration and cooperation among the various stakeholders? Specifically: The strengths and challenges should also be streamlined by each technical component, in terms of acceptance by local communities, producing quality outputs. Evaluation of the linkages established by the project with complementary projects in Char and Haor areas and beyond as well as relevant government departments. 3. In each technical sector, what are the strengths of and challenges to the efficiency of interventions’ implementation and their acceptance in the target communities? How well do implementation processes adhere to underlying principles and activity protocols? What factors in the implementation and context are associated with greater or lesser efficiency in producing Outputs of higher or lower quality? Which interventions and implementation processes are more or less acceptable to members of the target communities and why?  To what extent are the project interventions meeting the needs of the target beneficiaries, specifically needs of women and youth? Are they aligned with the Bangladesh’s country strategies and policies and/or with USAID development goals, objectives, and strategies?

SHOUHARDO III MTE Methodology Report 17 July 2018

For Objective 2: 4. What early evidences of changes—expected and unexpected, positive and negative— do community members and other stakeholders associate with the program interventions? What factors appear to promote and deter these changes? Which interventions appear to be more or less potential to influence knowledge or behaviors? How do the changes correspond to those hypothesized by the TOC? . Specifically: to focus on the TOC and assess any changes, identify gaps, external outcomes and stakeholders. . Are objectives of the project in keeping with locally defined needs and priorities? Should the direction of the project be changed to better reflect those needs and priority? . To what extent have interventions aligned with planned targets of participant, timelines and outputs? Any extenuating elements that impeded or accelerated plans? How were challenges mitigated or addressed? For Objective3: 5. How have SHOUHARDO III management and technical specialists used data to inform programmatic decisions, referral and follow up? What processes have been instituted to improve data collection and data quality? How has the project improved effectiveness or efficiency as a result of new ideas or approaches brought into the activities? How is information generated by the projects used to inform decision- making? How can this be made more effective? For Objective 4: 6. Has the project developed and implemented sustainability strategies? What organizations, services or relationships are required to sustain the early evidences of changes observed by the review team? Has the project identified the changes to be sustained, and the necessary services required to sustain these changes? Have the projects identified the potential service providers? What are the motivations of the service providers to continue service provisioning after the projects end? What has been done so far to increase the motivation? What would be the motivation of the beneficiaries to receive these services? Have the projects identified the resources and capacity strengthening supports for the service providers? What has the project done to ensure that this motivation does not diminish after the project ends? To what extent are government officials, formal and informal local leaders (whose support and understanding will be critical for continuing program initiatives once the project has ended) involved in project activities and included in ongoing program discussions and decision-making processes? Are there proper identification of roles on who should be doing what for a sustainable exit for the program – particularly on the improvement of services from the GoB? For Objective 5: 7. How effective are program design and implementation mechanisms in addressing the cross-cutting issues of gender, governance, the environment and targeting of youth? What (if any) challenges have projects encountered in these areas that may not have been anticipated in the project design, and how have the projects responded? To what extent do project interventions and implementation mechanisms reflect integration of these cross-cutting priorities? What steps have the activities taken to ensure that staff has adequate capacity for addressing these cross-cutting issues? Are

SHOUHARDO III MTE Methodology Report 17 July 2018

there any yearly evidences of changing roles, relationships, communication and decision-making dynamics among women and men, young and old, in relationship to food security at the household and community levels? How were the findings and recommendations of the Year 1 gender analysis considered, and what specific changes to the program strategy and project activities, were made? Is the project activities drawing on the potential of women, men, boys and girls as much as possible? For Objective 6: 8. Based on the findings from Questions 1–5, how could the activity be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the TOC and LF be refined or modified?  An assessment of efficiency should consider whether the same results could have been achieved with fewer resources or whether alternative approaches could have been adopted to achieve the same results.

3. MTE METHOD

The MTE will take a mixed methods approach. The contractor will be responsible for defining and carrying out the overall evaluation approach. This will include specification of the techniques for data collection (such as individual interviews and/or focus group discussions, observations in the field etc.) and analysis, structured field visits and interactions with beneficiaries and evaluation team. Evaluation tools, methodology and findings should be reviewed and validated with various stakeholders and approved by the donor.

While largely a qualitative process evaluation, the contractor will: a) review and assessment of the quantitative data available on program performance from the baseline survey results; b) review of the existing M&E data (Census survey data, Longitudinal Study, Beneficiary Based Sample Survey4, Routine Monitoring data) stored at the central and regional levels and c) consider small scale beneficiary based quantitative survey or other measures if deemed necessary by the contractor and approved by the program and donor. To meet the data requirement for objective 2, the contractor may need some quantitative data. The primary sources of the quantitative data would be baseline survey results, existing M&E data (census survey, annual beneficiary based sample survey results, routine monitoring data, panel data/longitudinal study) available. However, small scale beneficiary based quantitative survey or other measures, if deemed necessary by the contractor, can be considered with approval from the donor. In such circumstances, the contractor should explain the purpose, sampling methods, and potential biases and representativeness for such quantitative survey. For example, the small scale beneficiary based survey can be used to provide additional information on the changes in behavior, practice, attitudes and knowledge that individuals have made themselves as well as for making conclusions from the extensive qualitative assessments. Contractors should consider the existing quantitative data and potential qualitative data in considering whether or not to conduct a small-scale beneficiary based survey on the following

4 The values collected through Beneficiary Based Sample Survey (BBSS) are not designed to be compared, instead it provides only progress/performance value of specific Indicator Performance Tracking Table (IPTT) indicator against set target and eventually to report to FFP annually through ARR 18

SHOUHARDO III MTE Methodology Report 17 July 2018

key areas: Agriculture and livestock, maternal child health and nutrition, hygiene practices, access to government service and participation and women economic empowerment. Results from these surveys should not be extrapolated beyond the local context. The data supplements the MTE qualitative data. This data will be used to provide additional information for making the conclusions from the extensive qualitative work.

It is critical for the MTE team, through discussion with the program and attending formal briefing, to be well oriented for identifying key stakeholder groups, e.g., various target groups and beneficiaries, implementing partners, government partners, and other external collaborators, who should be included among MTE informants. The MTE informants should include people representing the different population subgroups benefiting as from only one program component/activities, as well as from the multiple program interventions. The evaluation must also include individuals who live in targeted communities but who have not directly benefited from any program intervention, as well as individuals who were excluded from the program interventions (looking for evidences of both inclusion and exclusion errors) and to get outsiders’ impressions of the interventions’ implementation and relevance. Furthermore, learning about non-beneficiaries’ practices and changes during the activity implementation period will help evaluators distinguish which reported and observed changes might be associated with activity interventions and which are related to general shifts in the context.

Secondary Data

The process will include: Desk review of existing literature, including project proposal, reports, baseline report, formative research, Longitudinal Survey report, Beneficiary Based Sample Survey Report, M&E data, MIS system and other relevant quantitative and qualitative secondary data to establish and assess the theory of change of the project, implementation strategies, allocations of resources, and relevant results. Information will be provided to the MTE team as per the MTE schedule at section 4.3 Time Frame.

Primary Data

The collection of primary data will involve mostly qualitative methods, especially unstructured or semi-structured interviews and observation. To answer the Key Evaluation Questions the direct observations and inquiries at a purposively selected sample of intervention sites and outputs will be required, as well as discussions with members of a variety of stakeholder groups, including direct, indirect and non-beneficiaries in targeted communities. Some of the key stakeholders groups that must be assessed through the primary data collection include:  CARE Bangladesh Staff  USAID/ Bangladesh Mission Staff  USAID/FFP/WDC staff, especially AOR and M&E contacts  FFP and FTF awardees in Bangladesh  GOB Officials/ Local Government Division (LGD) – National to Union/ Village level  Project staff from all levels – Head Quarter (HQ) to frontlines  Partner NGO staff and executives  INGOs, local NGOs, PVOs, CBOs 19

SHOUHARDO III MTE Methodology Report 17 July 2018

 Community Volunteers  Village Saving and Loan Association (VSLA), Village Development Committee (VDC), Youth Group and Women groups  Technical partners

The process will include: Qualitative data collection conducted with beneficiaries and stakeholders. Within each community, several subgroups will be selected for focus group discussion (FGD) to better understand the effectiveness of key interventions in the project. FGDs may include: Community Health, Agriculture and Empowerment Volunteer, Village Development Committee, Mothers Group, community leaders, farmers group members (FFBS), Village Savings Group members, EKATA Group members, Lead Farmers, Participants, Union Disaster Committees and other project related stakeholder groups. Respondents for the focus group discussions will be randomly selected to minimize selection bias. Key informant interviews (KII)s will be conducted with the donor, implementing partners, government officials, and Union Parishad members and Program advisory coordination committee (PACC) at Divisional, District and Upazila level. In-depth interviews (IDIs) will be conducted with direct, indirect and non- beneficiaries in targeted communities.

If deemed necessary and approved by the donor, small scale beneficiary based quantitative survey by the contractor will be conducted. In such circumstances, the contractor should explain the purpose, sampling methods, and potential biases and representativeness for such quantitative survey. For example, the small scale beneficiary based survey can be used to provide additional information on the changes in behavior, practice, attitudes and knowledge that individuals have made themselves as well as for making conclusions from the extensive qualitative assessments. Results from these surveys should not be extrapolated beyond the local context.

4. MTE CONTRACTOR’S RESPONSIBILITIES

Deliverables from the MTE Contractor

The Mid Term Evaluation (MTE) will be in form of an external review and will be conducted by an independent external contractor. The MTE team is expected to comprise expatriate and local experts with expertise in relevant program sectors (e.g., Agriculture & Livelihood, MCHN, food security, WASH, DRR, gender, empowerment and Good Governance) in Bangladesh.

This section itemizes the key deliverables (neither inclusive nor organized by order of priority) required from the MTE contractor within the agreed timeline:  Inception Report – will include the proposed evaluation approach, Key Evaluation Questions matrix, sampling approach, timeline and roles, responsibilities, anticipated outputs and associated levels of effort of each of the evaluation team members. The MTE contractor will prepare a Key Evaluation Questions matrix based on the objective, key questions and methods to consider as part of the Inception Report. The Key Evaluation Questions matrix should identify more-detailed areas of focus for each

SHOUHARDO III MTE Methodology Report 17 July 2018

question, aspects to consider within each focus area, and methods for investigation. See USAID’S Office of Food for Peace Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities, December 2016 Annex 3 for illustrative examples.  MTE implementation plan - will include the final overall mid-term evaluation process, timeline with detailed calendar of key activities and milestones, supervisor and enumerator training manual/guidance (English and Bengali 5 ), sampling approach and methods, data collection tools, validation (data quality assurance) and data analysis plan showing how each question will be analyzed from the data collected.  Lists of sites visited with types and numbers of informants at each, list of stakeholders interviewed and consulted  Digital transcripts of interviews including discussions, such as FGDs, KIIs and IDI.

Key Evaluation Questions Matrix Areas of focus Aspects to Methods of investigation/evaluation consider How well have the activity’s interventions met planned schedules, participant numbers, and Outputs? What Adherence to Start dates and rates Use secondary data from routine monitoring, planned of expansion of ARRs, and other reports to compare planned and schedules coverage for each actual start dates, numbers of Outputs, and other intervention targets, noting differences in achievements according to location, implementing partner, or sector.

 The MTE contractor must adhere all the requirements including USAID/ FFP M&E Policy and Guidance, relevant ADS Chapters, and DEC. They must maintain completeness, integrity, timeliness and overall quality of the survey.  Furthermore, the MTE Contractor must follow the USAID’s evaluation policy requires that the report describe the strengths and limitations of the evaluation methods and how and to what degree these factors influenced the process and findings of the evaluation. The report must clearly separate in different sections the evidence (i.e., raw data) collected by the evaluation team, the conclusions and recommendations that are based on the presented evidence. It should also state that sources of all evidences must be identified; conclusions must be based only on evidences presented in the report; and recommendations must directly correspond to the conclusions.

5 The primary product should be in English, translated into Bengali and back translated into English to ensure correct and quality translation. 21

SHOUHARDO III MTE Methodology Report 17 July 2018

 After the MTE field work is completed, the contractor/firm will submit a draft report (in Standard English) to CARE SHOUHARDO III program for comments and feedback from CARE and donor.  Draft key findings made available to CARE by the MTE contractor for use in validation workshop. SHOUHARDO III team will facilitate the event.  After responding to the comments, the contractor will submit the final report within 10 days for further review and approval by the program and FFP. The final report (see detailed requirements below) must adhere to the requirements of USAID’s Evaluation Policy (https://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf)  The deliverables include, but not limited to, the presentations for diverse audiences (including USAID, Government and Partners)

5. MTE Final Report The MTE Contractor/Team Leader is accountable to maintain the requirements for the content, format, or length of the final report, overall quality and approved timelines. They will produce a comprehensive report that assesses the achievements, relevance, coherence, coverage, effectiveness, efficiency, outputs and early outcomes of SHOUHARDO’s interventions so far, and provide prioritized recommendations to maximize results during the remainder of the project. The report must include executive summary and clearly separate evidence collected by the MTE contractor(s) from the conclusions and recommendations. Sources of all evidence must be identified, conclusions must be based only on evidence presented in the report, and recommendations must directly correspond to the conclusions. Recommendations must be prioritized and categorized, with justification for the prioritization and classification. They should also mark the recommendations, which are sequential, have huge resource implication and external support is required.

The MTE contractor should deliver all files, including, quantitative data sets (raw and refined products, in proprietary and non- proprietary format) if applicable, transcripts of qualitative data and others in an easy to read format; organize them professionally; and maintain naming conventions and labelling for the use of SHOUHARDO III program and key stakeholders and ready for posting to DEC and DDL. All documents should be 508 compliant before submission to DEC. The final submission will include the following key elements with appropriate analysis, interpretation and evidence:

a) Outline of the MTE Report The recommended outline for the MTE report is the following:

 Format and Size - The report must not exceed 50 pages (excluding all annexes) in MS Word and use the standard page set-up, margin, fonts, and line spacing.  Cover Page, Table of Contents, List of Acronyms;  Executive Summary - must be a clear, specific and concise stand-alone document that states the most salient findings, conclusions, and recommendations of the evaluation and gives the

SHOUHARDO III MTE Methodology Report 17 July 2018

readers the essential contents of the MTE in two to four pages. The Executive Summary should help the readers to build a mental framework for organizing and understanding the detailed information within the report;  Introduction - must include MTE Objectives, purpose, audience, and synopsis of task and MTE Questions  Methodology and Study Design - must describe the methodology and design of the MTE, constraints and limitations to the study process and rigor, and issues in carrying out the study;  Tabular summary and graphical presentation of quantitative and qualitative evaluation results - must present findings of the MTE in table and graphical forms as suitable;  Findings and Discussion - must present results from the MTE and associated evidences. Results must be analyzed and discussed, using findings from the qualitative and quantitative investigations in a complementary fashion. In addition, the findings from the community questionnaires must be analyzed and presented. The source of each finding must be clearly identified;  Conclusions and Recommendations - must provide high-level conclusions about the food security situation, vulnerabilities, and capacities of the population and sub-groups, and contextual, cultural, and individual factors that influence the current situation. All conclusions must be based solidly on the presented findings. If information from other sources is used to reach these conclusions, valid references must be provided, and reference documents or internet links to these included. Recommendations must be relevant to project and context and include concrete and realistic steps for implementing or applying the recommendation;  Programmatic Limitations and Challenges - must provide a list of key technical and/or administrative limitations, if any, that SHOUHARDO3 should consider; and  Annexes - must document all in complete manner, be succinct, pertinent, and readable. The MTE contractor should include a matrix in the Annex6 that identifies more-detailed areas of focus for each question, aspects to consider within each focus area, and illustrative methods for investigation. This matrix should be tailored to the details and concerns of each objective. The contractor must submit both electronic and printed versions of the report (3 copies of final report – multi color print) within the timeline and comply CARE IT policy and USAID ADS 579, ADS 597 and others as relevant. https://www.usaid.gov/who-we-are/agency-policy.

b) FGD/KII/IDI materials –transcripts and discussions conducted as part of the MTE c) Presentation files d) Time sheets e) Interview schedule f) Respondents list

6 See illustrative matrix in Annex 3 of the USAID/ FFP M&E Plan, in USAID’S Office of Food for Peace Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities, December 2016. 23

SHOUHARDO III MTE Methodology Report 17 July 2018

Pertinent Permissions, Approvals, Insurance, and Other Required Permits

The MTE team will be responsible for obtaining all necessary permissions, approvals, insurance, and other required permits and for adhering to national and local formalities. These include required permits related to data collection from human subjects, including necessary internal review board approvals and health and accident insurance for MTE team members.

Time Frame and Interim Submission

The MTE contractor should identify the data collection period for the MTE, that suffices the MTE schedule agreed by the program and FFP (see Section 4.2.1 of Annex F USAID’S Office of Food for Peace Policy And Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities, December 2016, and provide an illustrative time allocation for the various stages of the evaluation. This eventually will help USAID, CARE SHOUHARDO program, and MTE team estimate the financial resources needed for the evaluation. The time allocation should consider the number of team members and the accessibility of intervention sites. The MTE Contractor should also specifically state that the time frame provided is illustrative and subject to adjustment until the MTE work plan is finalized and approved by the program and USAID. Time line for the MTE:

4.4 An illustrative timeline for the MTE

The bulk of the primary data collection must be completed by March 2018. There is flexibility in the timing of milestones between the receipt of the secondary data on (date) 2018 and the submission of draft report on (date), which may be affected by team size and logistics.

Intervals of 2-4 weeks are included where review, feedback, and approval by CARE and USAID are required for plans and reports.

6. MTE Timetable

Task / Deliverable Start Date – End Date SHOUHARDO III will finalize the draft SOW and submit to FFP 4th June 2017 Review of the final Statement of Work of MTE by the responsible USAID/ FFP M&E Specialist 27th August 2017 Approval of the final Statement of Work of MTE by the FFP AOR of the program Advertisement, Recruitment and Procurement of Evaluators 25th September 2017 The MTE Contractor receive Secondary Data from 30th September 2017 SHOUHARDO III and other sources Review and analyze secondary data and submit inception report, draft evaluation work plan and timeline to CARE SHOUHARDO 01 Oct 2017 – 15 Oct 2017 III for initial review and feedback by CARE partners and USAID Present revised evaluation plan, including draft logistical plan, data collection tools, and timeline to program and USAID staff 16 Oct 2017 – 31 Oct 2017 for feedback

SHOUHARDO III MTE Methodology Report 17 July 2018

Submit final evaluation plan, data collection tools, timeline and 01 Nov 2017 – 30 Nov logistical plan for approval. 2017 Recruit translators and finalize logistics for evaluation 01 Dec 2017 – 31 Dec 2017 Identify indicators and variables for MTE and establish survey 01 Jan 2018 – 30 Jan 2018 location, sampling design and other protocol for the MTE Field works for qualitative study. Collect primary data, conduct 01 Feb 2018 – 15 March preliminary analyses, and validate findings with community 2018 members and other stakeholder Finish preliminary analyses and present key findings to 16th March 2018 – 15th April CARE and USAID/Mission staff 2018 Validation workshop – with participation of CARE, PNGOs and Between 16-20 April 2018 USAID MTE complete the data analysis, interpretation and prepare a draft evaluation report and submit it to CARE (to include 10 May 2018 highlighting findings that may suggest changes in the TOC and LF) De-briefing with CARE, USAID and other key stakeholders on 13th – 15th May 2018 preliminary observation and findings CARE submit the draft MTE reports to the AOR, who will review and provide feedback for the MTE team before the 25thMay – 25th June2018 report is finalized MTE Contractor submit final report revised based on feedback 10th July 2018 from CARE and USAID for approval USAID/FFP AOR approves Report 5th August 2018 MTE Contractor submit digital copies of all evaluation 15 August 2018 documents and data sets to CARE Upon approval of the final report, the CARE submits a plan of 20th August 2018 action in response to the MTE results and recommendations. CARE submit the final MTE report to the DEC within 30 days of Within 30 Days of FFP FFP approval and upload it to FFPMIS Approval Within 30 days of FFP approval, the CARE will provide the AOR with digital transcripts of interviews and discussions conducted Within 30 Days of FFP as part of the MTE and data sets created or used by the Approval evaluators, including copies of the monitoring database tables or other activity data, as analyzed by the evaluators FFP will review and submit the quantitative and qualitative MTE files to the Development Data Library, as appropriate, in accordance with USAID ADS 579: USAID Development Data and USAID will submit to DDL the award’s standard provisions. Section 4.3.1 describes the requirements for submissions. Within 45 days of FFP’s approval of the final report, the Within 45 Days of FFP CARE and FFP will develop and agree to a plan of action in Approval response to the MTE results and recommendations.

SHOUHARDO III MTE Methodology Report 17 July 2018

There is no need for the MTE data collection to match the seasonal timing of the baseline study or final evaluation. However, the MTE contractor should have consideration on practicality, such as seasonality, travel time, type of transport, local context and weekend and holidays.

1.5 Logistics

The MTE team will be responsible for the evaluation logistics including accommodation, transportation and meals. To avoid compromising activity implementation during the MTE and to maintain a separation between the MTE team and the implementers, activity vehicles and other vehicles branded to identify them with SHOUHARDO III or any of the implementing partners may not be used by the MTE team while they are in the activity area.

7. MTE TEAM COMPOSITION, QUALIFICATIONS, AND ROLES

General Roles & Responsibilities

 The MTE Team Leader (TL) will be the primary investigator of the MTE. S/he will provide oversight to the evaluation throughout the entire evaluation cycle.  The MTE TL will work in close collaboration with the program’s Point of Contact and USAID at all times throughout the MTE process, ensuring good communication, receiving and acting on all inputs and comments, and calling meetings as necessary to resolve any problems that may arise during the whole process.  To ensure the security of the expatriates and all other MTE team members, all movements within the country will be made following the country security protocol. The individual team members will manage their security by themselves, CARE will have no liability to assure it.  If the security situation permits and MTE requests, CARE may assist the MTE contractor to rent unmarked vehicles for the team's use. The MTE team should present themselves as separate from CARE as much as possible, but more importantly, the program activities must not be compromised by a lack of transportation. The MTE team needs to see how the project normally works, not how it works with insufficient resources.  To ensure the MTE is as unbiased as possible, and to avoid the disruption of project implementation that could affect the results of the evaluation, the MTE contractor will not use CARE’s marked vehicle and project staff as translators, primary data collectors, or supervisors.  The MTE contractor / team members will be responsible for adhering to national and local formalities and all security protocol while in-country.  The MTE contractor will be responsible for obtaining, and providing CARE with visa(s), written proof of health and accident insurance for all foreign personnel working with the MTE contractor in Bangladesh.

SHOUHARDO III MTE Methodology Report 17 July 2018

MTE Team Composition

The MTE team will typically consist of a team leader who is an evaluation specialist/ expatriate, plus three to five technical specialists who brings expertise and practical experience in one or more of the activity’s technical sectors and interventions in addition to strong qualitative research skills and experience. The team may include a data analyst experienced in analyzing and relating data across various technical sectors. The MTE should be a mix of international and national members with gender consideration. The following areas are counted as minimum requirement for MTE team members’ qualifications: • Every team member’s resume must show substantial application of qualitative and quantitative research skills in developing countries – preference to south Asia.

• The team leader must have significant formal education in a field relevant to evaluation (e.g., program evaluation, statistics, economics, agricultural economics, anthropology, applied research, organizational development, sociology, or organizational change) at a post-graduate or an evaluation professional continuing-education level. S/he must have extensive practical experience and expertise in evaluation using mixed methods of investigation (qualitative and quantitative) in developing countries – south Asia is preferred, and demonstrated strengths in communicating clearly and concisely – both for written and oral. Knowledge and experience of the conceptual framework of food security and experience evaluating food security programming, Log Frame and TOC is highly desirable.

 Each technical specialist should have a post-graduate degree in a field related to at least one of the technical sectors of the activity, plus extensive practical experience in developing countries with interventions similar to those implemented by the activity.  At least one member of the team must have substantial demonstrated experience in gender integration.  The MTE team should comprise technical expertise from all activity sectors and activity management.  No member of the MTE team in particular should have had any prior input or substantial engagement to the program’s activity design or implementation.  Having conflict sensitivity expertise on the MTE team is also desirable, particularly when an MTE will be undertaken in a context that is prone to conflict.

All personnel working on the MTE must show substantial successful application of qualitative research skills (combination of qualitative, quantitative research skill and moderator / facilitator of PRA sessions, FGD, KII and IDI is required) in developing countries – south Asia preferred – and have excellent communication skills in English and Bengali with understanding on local dialects for program areas. The team must include expertise from all of the technical sectors in the program, which are:

 Logical Framework and Theory of Change 27

SHOUHARDO III MTE Methodology Report 17 July 2018

 M&E system including automation  Knowledge management systems  Staff management  Partnerships  Food Security  SMART Agriculture/Natural Resources Management (NRM)  Value Chain  Village Savings and Loan Association (VSLA)  Health, Hygiene and Nutrition/Public Health  Disaster Risk Reduction, Climate Change and Resilience  Gender  Governance  WASH and Behavior Change Communication (BCC)  Commodity and Financial Management

The application / Expression of Interest (EoI) must include resume for each MTE team member that details relevant education, experience and other required qualifications. The application must clearly describe how the person is qualified for his/her specific role(s) on the team and adds value in the MTE.

MTE Team Members’ Specific Roles

Each team member’s role in the evaluation is as follows:

Team Leader’s Role  Organizes and leads the overall evaluation. Provides oversight to the evaluation throughout the entire evaluation cycle. Be the point person for all liaison with the CARE and USAID.  Ensures a thorough review and analysis of activity monitoring data and other available secondary data by the appropriate team members  Leads the selection of a purposively selected sample of implementation sites and outputs for primary data collection  Ensures an MTE plan that includes adequate triangulation and validation of evidence collected in all sectors  Leads the collection and analyses of primary and secondary data to evaluate the activity’s M&E processes and the integration of activity sectors and interventions  Ensures that final report presentation is logical, follows the required format and adequately supported by primary and/or secondary data.  Leads the team and deals with the human and financial resources  Serves as a technical specialist for specified sector (optional) as long as it does not affect her/his performance in his/her primary team leadership role and analyses of M&E processes and inter-sector integration.

SHOUHARDO III MTE Methodology Report 17 July 2018

Technical Specialists  Lead the collection and analyses of primary and secondary technical data related to his/her field(s) of expertise, document findings, and draw conclusions and form recommendations for the sector(s)  In close collaboration with the Team Leader, purposively-select a sample of implementation sites and outputs for primary data collection for his/her technical sector(s)  Evaluate the general aspects of the implementation of all interventions related to his/her sector(s). While the team leader will likely be tasked as the primary investigator for the overall activity management, a technical specialist must consider management aspects of the implementation of interventions in his/her technical sector and the interaction between his/her technical sector and other activity sectors by examining:

- Staff and material resources - Communication, both internal and external - Community involvement - Participant targeting (especially overlap/consistency with other sectors) - Management of food and non-food commodities - Transfers of entitlements (food, non-food, cash) - Branding and marking - Partnerships and linkages - Consortium management - Routine monitoring and data quality assurance for all interventions - Exit/sustainability strategies - Gender integration - Environmental protection - Draft the report sections assigned by the team leader in the specified format

8. PROGRAM RESPONSIBILITIES During data collection and analysis, the primary roles of the program staff and any implementing partner with direct stake in the project, are as informants and reviewers. They may review and provide comments on data collection tools and instruments before they are finalized. They must not collect primary data, or participate in translation, analysis, or interpretation of the data.

The following will be the key roles and responsibilities for the Mid Term Evaluation Participants: Person/Unit/Organization Activity Chief of Party (CARE  Give orientations for mid-term review team, and advice Bangladesh/SHOUHARDO III) the team  Provide approvals for any expenses related to MTE  Provide the review team with the entire project documents for review.

SHOUHARDO III MTE Methodology Report 17 July 2018

 Coordinate the mid-term review exercise both to office and field work, in terms of follow up the project Management  Oversee results dissemination with key partners  Advice for dissemination results of MTE  Provide support to the MTE contractor to carefully review the TOC and LF and adjust the program activities accordingly based on the evaluation findings

Deputy Director Food  Give the final feedback to the MTE tools Security and Senior Technical  Review the ToR for MTE Advisor-M&E (CARE HQ)  Give inputs for external contractor or MTE team  Participate on MTE reporting process  Review the MTE report  Oversee quality assurance of the mid-term evaluation  Comment on the deliverables of the mid-term evaluation.  Oversee results dissemination with key stakeholders.  Advice for dissemination results of MTE Deputy Chief of Party, Senior  Ensure that the review team has all logistical support Team Leader-KMAL and necessary for the conduct the process properly Senior M&E Coordinator  Review the tools for data collection (CARE  Provide selection of enumerators and supervisors for Bangladesh/SHOUHARDO III) MTE process  Monitor the data collection  Monitor the conduct of the project review by the evaluation team.  Oversee quality assurance of the mid-term evaluation.  Ensure that all deliverables of the project review meet the expectations of SHOUHARDO III Program  Review the 1st draft report of MTE and submit to SHOUHARDO III Program Management Team (PMT)  Comment on the deliverables of the review  Oversee results dissemination with key stakeholders. Mid-term evaluation field  Participate in training, as MTE supervisors team (Regional Coordinators,  Supervise all field data collection Regional Senior Technical  Give necessary support for MTE logistics, as well during Manager, Regional ME&L the field data collection Manager, Technical  Provide any additional support for the contractor when Managers)- (CARE needed Bangladesh/SHOUHARDO  Participate in dissemination process of MTE results III) Partners, Govt. and other  Provide feedback to MTE tools Project staff  Participate in all meetings with Mid-term review team

SHOUHARDO III MTE Methodology Report 17 July 2018

 Provide feedback and information on the project as requested for the Mid-term review  Provide feedback on the deliverables of the mid-term review Additional stakeholders  Participate in meetings with mid-term review team, (beneficiaries, on the ground discussions, and agreements about the process, etc. partners as UP chairs)  Provide feedback and information as requested for the mid-term review  Participate in evaluation debrief and stakeholders meeting USAID  Review MTE SoW and give recommendations to operationalize the process  Provide relevant information about the project procedures, approaches and operations through MTE questions addressed to the main key informant, prior to the field data collection.  USAID officers may provide comments on, and will give final approval for, the evaluation plan, data collection tools, timeline and logistical plan before the MTE contractor begins field work.  Participate in discussions regarding the evaluation findings and recommendations (evaluation debrief and stakeholders meeting)

9. INTELLECTUAL PROPERTY The ownership of the report for publication rests with CARE Bangladesh. All the data and the reports including the findings and recommendations will remain the property of CARE Bangladesh and must not be published or shared with a third party by the consulting agency. The completed data set and report will be the sole property of CARE and USAID. The contractor should not have any right to use the data for its own research purposes, nor license the data to be used by others, without the written consent of USAID & CARE Unless otherwise provided in the award’s provisions, CARE may retain the rights, title, and interest to data that are first acquired or produced under the award. In addition, “USAID reserves a royalty-free, worldwide, nonexclusive, and irrevocable right to use, disclose, reproduce, prepare derivative works, distribute copies to the public, and perform and display publicly in any manner and for any purpose and to have or permit others to do so.”

10. ETHICAL GUIDELINES Every member of the evaluation team must adhere to ethical guidelines as outlined in the American Evaluation Association’s Guiding Principles for Evaluators. A summary of these guidelines is provided below.

 Systematic inquiry: Evaluators conduct systematic, data-based inquiries.  Competence: The evaluation team possesses the education, abilities, skills, and experience appropriate to undertake the tasks proposed in the evaluation. Evaluators

SHOUHARDO III MTE Methodology Report 17 July 2018

practice within the limits of their professional training and competence, and decline to conduct evaluations that fall substantially outside those limits. The evaluation team collectively demonstrates cultural competence.  Integrity/honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process.  Respect for people: Evaluators respect the security, dignity, and self-worth of respondents, activity participants, clients, and other evaluation stakeholders. Evaluators regard informed consent for participation in evaluation and inform participants and clients about the scope and limits of confidentiality.  Responsibilities for general and public welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation.

Further details are available at:http://www.eval.org/p/cm/ld/fid=51.

11. BUDGET The MTE proposal should include a reasonable detailed budget to cover all costs associated with the MTE. This should be submitted by major activities and line items for CARE SHOUHARDO III program review and decision. This would break-down the cost of contracted MTE team members, international and local travel, and in-country lodging and per diem. Salary for technical specialists of CARE HQ may be charged for the days that they are directly involved with the evaluation. Other related costs that might be in the budget include expenditures for hiring local personnel (drivers, translators, enumerators and other local technical experts), translating reports, and renting meeting rooms for presentations.

12. EVALUATION CRITERIA The Proposal Evaluation Committee (PEC) will evaluate the Technical proposals of potential MTE contractors. Based on the submitted technical proposal, the contractor may be asked for further information (if deemed necessary). The short listing criteria for the evaluation are as follows: Proposed Methodology for Carrying Out The Task 45 - Demonstrates clear understanding of the context and specific task as outlined in this SoW. - Methodology is in line with FFP/USAID MTE guidelines - Responsive to all key objectives of the MTE Proposed Key Staff (Team leader and other technical specialists) Directly 30 Assigned to the task Experience of conducting similar sort of USAID/FFP funded large Food and 25 Nutrition Security (FNS) Title 2 project evaluation TOTAL POINTS 100

13. MANAGING UNEXPECTED The contractor must keep this in their prime consideration how to manage if any unexpected situation like strike and political uprising, natural disaster that that may affect the MTE. They

SHOUHARDO III MTE Methodology Report 17 July 2018

should keep options for contingency plan and alternatives without compromising the overall quality, purpose and timeline.

LIST OF ANNEXES Annex A. SHOUHARDO III Program Logical Framework (LF) Annex B. SHOUHARDO III Program Theory of Change (ToC) Annex C. SHOUHARDO III Geo Priority and Vulnerable People Targeting Guides Annex D. SHOUHARDO III Life of Award (LOA) Detailed Implementation Plan (DIP) Annex E. SHOUHARDO III Program Management, Coordination and Staffing Plan Annex F. of the USAID/ FFP M&E Plan, in USAID’S Office of Food for Peace Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities, December 2016,

RESOURCES (TO BE PROVIDED UPON SIGNATURE OF AGREEMENT) General  Integration Strategy for SHOUHARDO III  All Annual Results Reports  All Quarterly Progress Reports Monitoring and Evaluation (M&E)  Baseline Survey Report  Two Participant Based Sample Survey Report  SHOUHARDO III Program, M&E Plan, IPTT with Targets and updated achievement, Standardized Annual Performance Questionnaire,  Automated Directive System (ADS) Chapters Knowledge Management • Knowledge Management and Advocacy strategy Partnerships, Operations and Resources  Partnership (implementing partners) Agreement and Activity Plan  List of PNGOs with key contacts  Technical Partnership Agreement Formative Studies and Program Strategies  Labor Market Assessment Study and Strategy report  Inclusive Value Chain Study and Strategy report  Social Behavior and Change Communication (SBCC) Study and Strategy report  Gender Analysis Study and Strategy Report Purpose 1: Agriculture & Livelihoods  FFBS manual  VSLA manual  IGA training

SHOUHARDO III MTE Methodology Report 17 July 2018

Purpose 2: Health, Hygiene & Nutrition  HHN training module Purpose 3: Resilience  UDMC, UDV training module  UDV training module  Community Risk Assessment Tool and Guideline  SBTB formation guideline Purpose 4: Women Empowerment  EKATA facilitation module  Module on couple’s dialogue  Men Engagement Guideline and Tools  Tools and guidelines for addressing Gender Based Violence Purpose 5: Governance  VDC & CAP Guidelines  Training module for VDC/ UP and Youth Group members Life Skill  Life Skill training modules Infrastructure  Infrastructure, CFW and LCS Guideline  Large Structure Planning and Implementation Guideline to be implemented by LGED Commodity  Commodity management and accounting manual Environment  Initial Environmental Examinations (IEE)Environmental Management and Monitoring plan (EMMP)  Safer Use Action Plan (SUAP) – PMP and Safer Use Action Plan (SUAP) – FMP

SHOUHARDO III MTE Methodology Report 17 July 2018

III. MTE PROTOCOL

SHOUHARDO III MTE Methodology Report 17 July 2018

Mid-Term Evaluation (MTE) of the SHOUHARDO III Program

USAID-funded Title II Development Food Security Activity in Bangladesh led by CARE Award Number: AID-FFP-A-15-00009

MTE Protocol

Submitted on 22nd January 2018 Revised 6 February 2018 FINALIZED 15 FEBRUARY 2018

SHOUHARDO III MTE Methodology Report 17 July 2018

TABLE OF CONTENTS Page I. INTRODUCTION 1 II. MTE PURPOSE & OBJECTIVES 1 A. Purpose 1 B. Specific Objectives 1 III. PROGRAM DESCRIPTION 2 A. SHOUHARDO III Overview 2 1. Purpose 1: Income & Food Production 4 2. Purpose 2: MCHN 5 3. Purpose 3: Mitigating, Adapting to and Recovering from Shocks 7 4. Purpose 4: Women's Empowerment & Gender Equity 9 5. Purpose 5: Public Services for PEP 10 6. Cross-Cutting Themes 11 7. Partner Roles and Responsibilities 11 IV. EVALUATION TEAM 12 A. Team Members 12 B. Assignment of Responsibilities 13 V. INFORMATION TO BE OBTAINED 14 A. Objective 1: Quality of Program Delivery 14 B. Objective 2: Evidence of Change 16 C. Objective 3: Collaborative Learning and Action (including M&E) 17 D. Objective 4: Sustainability 18 E. Objective 5: Cross-Cutting Themes 19 F. Objective 6: Implementation Systems 21 VI. EVALUATION PROCESS 22 A. Phase 1: MTE Preparation (October 23 through February 12) 22 1. MTE Protocol 22 2. Background Document Review 23 3. Draft Data Collection Tools 24 4. Site Selection 24 5. Key Meetings 24 B. Phase 2: Data Collection & Preliminary Analysis 25 1. SHOUHARDO III Program Orientation Meeting, February 13 25 2. Meetings with SHOUHARDO III Stakeholders in Dhaka, February 14-15 25 3. Field Data Collection, February 16 through March 3 26 4. Information Processing, Preparation for the Verification Workshop, March 4-6 26 5. Verification Workshop, March 7-8 26 6. Post-Workshop Processing, March 9-10 26 7. Stakeholder Debriefing, March 11 27 8. USAID/FFP Bangladesh Debriefing, March 12 27 C. Phase 3: Evaluation Recommendations Finalization, March 13 to June 15 27 1. Draft Report, April 16 27 2. Debriefing at FFP HQ, Second Half of May 28 3. Final Report, June 15 28

SHOUHARDO III MTE Methodology Report 17 July 2018

4. MTE Recommendation Processing & Planning Workshop 28 VII. QUALITATIVE DATA COLLECTION 28 A. Composition of Key Informant Interviews and Focus Group Discussions 28 B. Topical Outlines or Checklists 28 C. Observation of Program Activities 30 D. District-Level Data Collection 30 E. Other Data Collection 31 F. Sampling and Participant Selection 31 G. Data Analysis 34 VIII. IMPLEMENTATION SCHEDULES 35 IX. CARE RESPONSIBILITIES 35 X. MTE SUMMARY REPORT 35 ANNEXES Annex A: MTE Schedules (Overall and Field Work) 36 Annex B: Data Collection Tools 41 Annex C: Illustrative Verification Workshop Plan 86 Annex D: Illustrative MTE Summary Report Format 89 Annex E: List of Sites Selected for Data Collection 91

TABLES AND FIGURES Table 1: Goal, Purposes and Sub-Purposes for the SHOUHARDO III Program ...... 43 Table 2: Participants and Sites in the SHOUHARDO III Program under Purpose 1: Income & Food Production ...... 45 Table 3. Participants and Sites in the SHOUHARDO III Program under Purpose 2: MCHN .... 47 Table 4: Participants and Sites in the SHOUHARDO III Program Targeted under Purpose 3: DRR/CCA ...... 48 Table 5: Participants and Sites in the SHOUHARDO III Program Targeted under Purpose 4: Women's Empowerment and Gender Equity ...... 49 Table 6: Participants and Sites in the SHOUHARDO III Program Targeted under Purpose 5: Public Services ...... 51 Table 7: PNGOs Implementing the SHOUHARDO III Program ...... 51 Table 8: MTE Team Member Responsibilities ...... 53 Table 9:Evaluation Questions, Methods, and Data Sources for MTE Objective 1 ...... 54 Table 10: Evaluation Questions, Methods, and Data Sources for MTE Objective 2 ...... 56 Table 11: Evaluation Questions, Methods, and Data Sources for MTE Objective 3 ...... 58 Table 12: Table 12: Evaluation Questions, Methods, and Data Sources for MTE Objective 4 .... 59 Table 13: Evaluation Questions, Methods, and Data Sources for MTE Objective 5 ...... 60 Table 14: Evaluation Questions, Methods, and Data Sources for MTE Objective 6 ...... 62 Table 15: Number of Villages in SHOUHARDO III Who Also Participated in SHOUHARDO I72 Table 16: Allocation of Sample Sites by District for the SHOUHARDO III MTE ...... 73

SHOUHARDO III MTE Methodology Report 17 July 2018

ACRONYMS AND ABBREVIATIONS ARR Annual Results Report C<5 Children Under Five Years of Age CAP Community Action Plan CAV Community Agriculture Volunteer CCA Climate Change Adaptation CG Community Groups CHD Comprehensive Homestead Development CHV Community Health Volunteer CLTS Community-led Total Sanitation COG Core Occupational Group CoP Chief of Party CVCA Climate Vulnerability and Capacity Analysis CSC Community Score Cards CSG Community Support Group CFW Cash for Work DAE Department of Agricultural Extension DAM Dhaka Ahsania Mission DCRM Disaster and Climate Risk Management DFSA Development Food Security Activity DIP Detailed Implementation Plan DPHE Department of Public Health Engineering DRR Disaster Risk Reduction DYD Department of Youth Development EKATA Empowerment Knowledge and Transformative Action ESDO Eco-Social Development Organization EVAW Ending Violence against Women EMMP Environmental Mitigation and Monitoring Plan FF Field Facilitator FFBS Farmers' Field and Business School FFP Food for Peace FGD Focus Group Discussion GBV Gender-Based Violence GMP Growth Monitoring and Promotion GoB Government of Bangladesh IGA Income-Generating Activity IMCI Integrated Management of Childhood Illness IPTT Indicator Performance Tracking Table ITSH Inland Transport Storage and Handling IYCF Infant and Young Child Feeding KII Key Informant Interview KMAL Knowledge Management, Advocacy and Learning LEB Locally Elected Body LGED Local Government Engineering Department LGRD Local Government and Rural Development

SHOUHARDO III MTE Methodology Report 17 July 2018

LoA Life of Activity or Life of Award M&E Monitoring and Evaluation MCHN Maternal and Child Health and Nutrition MFI Micro-Finance Institution MJSKS Mahideb Juno Somaj Kallayam Somity MoHFW Ministry of Health and Family Welfare MT Metric Ton MTE Mid-Term Evaluation NBD Nation Building Department NDP National Development Program NGO Non-Governmental Organization PACC Program Advisory and Coordinating Committees PEA Programmatic Environmental Assessment PEP Poor and Extreme Poor PIIF Program Integration Implementation Framework PLW Pregnant and Lactating Women PM&ES Planning, Monitoring and Evaluation System PMP Performance Monitoring Plan POPI People's Oriented Program Implementation PREP Pipeline and Resource Estimate Proposal QMR Quarterly Monitoring Report SAM Severe Acute Malnutrition SBCC Social and Behavioral Change Communications SHOUHARDO Strengthening Household Ability to Respond to Development Opportunities SKS SKS Foundation SUAP Safer Uses Action Plan TANGO Technical Assistance to NGOs ToC Theory of Change ToR Terms of Reference ToT Training of Trainers UDCC Union Disaster Coordination Committee UDV Union Disaster Volunteers UHC Upazilla Health Complex UISC Union Information and Service Center UP Union Parishad USD United States Dollar UzDMC Upazilla Disaster Management Committee VDC Village Development Committee VFA Veterinary Field Assistant VSLA Village Savings and Lending Association WASH Water, Sanitation and Hygiene WE Women's Empowerment

SHOUHARDO III MTE Methodology Report 17 July 2018

Protocol for the Mid-Term Evaluation (MTE) of the SHOUHARDO III Development Food Security Activity (DFSA) CARE Bangladesh I. INTRODUCTION The Strengthening Household Ability to Respond to Development Opportunities III (SHOUHARDO III) program is a five-year Development Food Security Activity (DFSA) funded by the Office of Food for Peace (FFP) in the United States Agency for International Development (USAID). The program began implementation on 29 September 2015 and is expected to be completed by 28 September 2020. Since the program is nearing the mid-point in its life, a Mid-Term Evaluation (MTE) is being scheduled in the second quarter of FY 2018. This protocol describes the plans and procedures that will be used to implement the MTE. II. MTE PURPOSE & OBJECTIVES A. Purpose The MTE is a formative process evaluation intended to review the progress of the program in producing planned outputs, to assess the intended and unintended effects that are appearing as a result of these outputs, and to examine the quality of various processes being used to implement the program to formulate recommendations to be implemented in the remaining life of the program. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) piloting new interventions relevant for targeted impact groups, (e) improving the effectiveness of implementation systems, or (f) improving efficiency in use of resources. The evaluation process will ensure that the recommendations are implementable within the time frame remaining and with the resources available to the program. The ultimate purpose of the MTE is to enhance the effectiveness of the program in addressing the food/nutrition security and resilience of poor and extreme poor people living in the northern/middle Char and Haor areas of Bangladesh while making the most efficient use of resources available to the program. B. Specific Objectives The specific objectives of the MTE are to: 1. Evaluate the quality of program delivery. This investigation will cover the activities of both CARE and its implementing partners by looking into the strengths and weaknesses of activity implementation; the quality of outputs in terms of the effects they are producing, their adherence to terms agreed upon with FFP, and their appropriateness and perceived value to target communities; and identifying factors that appear to enhance or detract from the quality. 2. Identify and analyze evidence of changes7, including positive and negative changes as well as intended and unintended changes, being induced by the activities and outputs of the program. Using qualitative information obtained during the MTE as well as available quantitative data, the investigations will assess how well the observed changes coincide with the Theory of Change (TOC)and logical framework guiding the program and identify factors

7 These changes can occur at the individual, household, community and higher levels, including systemic changes.

SHOUHARDO III MTE Methodology Report 17 July 2018

in the implementation or context that impede or promote the observed changes. This will entail reviewing the strategies that ensure that targeted impact groups are reached by the program, reviewing the theory of change, and assessing the hypotheses, risks, and assumptions made during the design of the program. 3. Review Collaborative Learning and Action systems (including monitoring an evaluation). The investigations will examine the effectiveness and efficiency of program systems for monitoring and evaluation and for capturing and documenting lessons learned and will assess the extent to which they are used in modifying implementation and refining program design, including feedback from the perspective of stakeholders and participants. The MTE will also assess how well the program is seeking out, testing and adapting new ideas and approaches to enhance the program's effectiveness or efficiency. 4. Assess the quality of and progress made on implementing sustainability strategies. The investigations will determine the extent to which outcomes, systems and services are designed and being implemented to continue after the program ends. 5. Determine the appropriateness and effectiveness of activities focused on the cross-cutting themes for the program. The investigations will examine the support for promoting gender equity, the targeting of youth, support for increasing capacities for local governance, and activities addressing sources of environmental risk. 6. Review implementation systems, including management systems, partner relations, coordination and collaboration with external organizations, commodity management systems, financial management systems, human resource management systems (including staffing structure) and materials management systems to identify recommendations for improving the effectiveness of implementation and/or the efficiency in use of program resources. III. PROGRAM DESCRIPTION A brief description of the SHOUHARDO III Program is provided below, focusing on the information that is relevant for planning the MTE, including the types of outputs being produced, the types of participants (beneficiaries and intermediaries8) that will need to be interviewed, and the key partners (implementation and technical) that will also need to be interviewed. Prior to undertaking the MTE, team members will review key background documents to understand the theory of change for the program, the approaches that are being used in producing outputs, and the outcomes and impact that are expected from the achievement of outputs. A. SHOUHARDO III Overview The goal of the SHOUHARDO III Program is to achieve improved gender equitable food and nutrition security and resilience for the vulnerable people living in the flood-prone Char and Haor regions of Bangladesh by 2020. The program is specifically targeting people defined by their communities as poor or extreme poor (PEP), expecting to have lasting impact by the end

8 Beneficiaries are program participants from the targeted impact groups which are, for SHOUHARDO III, Poor or Extreme Poor (PEP) households. Intermediaries are program participants who are not from the targeted impact groups, but the program works with these participants, e.g., community leaders, private sector business owners, government extension agents, in order to achieve impact on PEP households. 42

SHOUHARDO III MTE Methodology Report 17 July 2018 of its life on around 675,000 persons9. The overall program value10 is USD 88 million, including USD 80 million in resources from the United States Government and USD 8 million from the Government of Bangladesh. A total of 110,910 MTs of commodities are planned for monetization over the life of the program, and 13,190 MT of commodities are planned for distribution under the maternal and child health and nutrition component (Purpose 2) of the program. Under the leadership and technical guidance of CARE, the SHOUHARDO III Program is being implemented by six national Partner Non-Governmental Organizations (PNGOs) in 947 villages in 115 unions in 23 Upazillas in eight districts11 in northern Bangladesh. Table 1 shows the current Goal, Purposes and Sub-Purposes for the program. The approaches used and expected outcomes in the SHOUHARDO III Program are described in the final technical proposal as well as a number of specific sectoral program strategy documents that have been developed, and the performance indicators at each level are described in the program's Indicator Performance Tracking Table (IPTT). The sections which follow list the major types of outputs for each of the five purposes along with the major types of participants. The last section provides an overview of the partners in the SHOUHARDO III Program. Table 1: Goal, Purposes and Sub-Purposes for the SHOUHARDO III Program GOAL: Improved gender equitable food and nutrition security and resilience for the vulnerable people living in the Char and Haor in Bangladesh by 2020 Purpose 1: Increased equitable access to income for both women and men, and nutritious food for men, women, boys and girls Sub-Purpose 1.1: Sub-Purpose 1.2: Increased Increased access to Sub-Purpose 1.3: Sub-Purpose 1.4: agricultural agricultural markets for Increased off-farm Increased utilization of production of PEP PEP income for PEP financial services by PEP Purpose 2: Improved nutritional status of children under five years of age (C<5), pregnant and lactating women (PLW) and adolescent girls Sub-Purpose 2.1: Increased Sub-Purpose 2.2: Sub-Purpose 2.3: Reduced utilization of nutritious food for Improved access to health prevalence of water-borne PLW, C<5 and adolescent girls and nutrition services diseases Purpose 3: Strengthened gender equitable ability of people, households, communities, and systems to mitigate, adapt to and recover from man-made and natural shocks Sub-Purpose 3.2: Local government Sub-Purpose 3.1: Increased preparedness of institutions' (Union Parishad) capacity and PEP households and communities to mitigate and implementation of Disaster and Climate Risk respond to shocks Management (DCRM) activities increased Purpose 4: Increased women's empowerment and gender equity at family and community level Sub-Purpose 4.1: Improved environment for Sub-Purpose 4.2: Strengthened agency of women's empowerment women Purpose 5: Provision and utilization of public services (Local Elected Bodies and Nation Building Departments) for communities especially for poor and extreme poor increased

9 This number includes a target of 168,521 households having 384,000 direct recipients (2.28 persons/household). 10 Includes cash resources as well as the value of monetized and distributed food from FFP 11 Kurigram and Gaibandha in the Northern Chars Region, Jamalpur and Sirajganj in the Middle Chars Region and Netrakona, Kishoreganj, Sunamganj and Habiganj in the Haor Region. 43

SHOUHARDO III MTE Methodology Report 17 July 2018

Sub-Purpose 5.1: Communities (especially PEP) are better able to demand and negotiate Sub-Purpose 5.2: Accountability of LEBs and increased public services (UPs and NBDs) NBDs to the needs of communities especially to services PEP increased 1. Purpose 1: Income & Food Production. Under this component, the SHOUHARDO III Program plans to reach approximately 168,500 women, men and youth smallholder and tenant farmers with interventions intended to increase crop diversity and homestead production of nutritious food, increase access to agricultural markets, increase opportunities for generating off-farm income, increase savings and access to capital, and improve the enabling environment for on-farm and off-farm income generation and asset protection. Major Outputs. Following are the major outputs in the SHOUHARDO III Program under Purpose 1 as defined following the revised Theory of Change finalized in September 2016.  Advocacy conducted on improved implementation of khas land policy (also under Purpose 5)  Seed supply through micro-seed dealers and seed sellers  Capacity building of farmers on improved production and technology (improved modern poultry and livestock production, crop production, culture and capture fisheries) and climate change adaptation approaches (floating gardens, compost preparation, and crop rotation)  Capacity building of PEP participants on entrepreneurship and basic business skills (for farm production and off-farm IGAs)  Development of an integrated system of local and regional entrepreneurs/buyers/collectors  Development of linkages to affordable transportation services (After farm production levels increased)  Research conducted on business barriers for PEP participants and opportunities for input suppliers to reach PEP participants  Expanded availability of market information for IGA participants  Capacity building of youth through mentorships, apprenticeships or internships  Development of employment and placement support services for youth  Capacity building on basic life skills for PEP participants, including business-related decision-making, business-related communications, negotiation skills, financial management and numeracy skills  Capacity building of PEP households on savings, lending and financial management skills  Formation of savings associations (VSLAs) of men, women and youth  PEP savings groups linked to MFIs and banks  Advocacy action for pro-poor financial support  Support for MFIs and banks to conduct feasibility studies on PEP financial products  Construction of raised homesteads in conjunction with promotion of Comprehensive Homestead Production (CHP) systems

SHOUHARDO III MTE Methodology Report 17 July 2018

 Cultivation of business relations through district-level agricultural fairs, buyer visits, and shared-value business models between PEP farm households and private sector partners  Provision of micro-enterprise matching grants for agriculture and off-farm opportunities The following outputs were described in the program proposal, but due to resource limitations, they were removed from the program strategy when the TOC was revised.  Capacity building of public and private extension services  Facilitation of linkages for basic education capacity building for youth  Promotion of use of mobile financial services especially for women Types of Participants. Table 2 outlines the different types of participants engaged and sites developed under Purpose 1. The MTE will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 2: Participants and Sites in the SHOUHARDO III Program under Purpose 1: Income & Food Production

Current Membership/Sex LOA Type of Participant/Site Total Female Male Target Micro-seed sellers (Small shops) 67 2 65 170 Micro-seed dealers 9 5 4 30 Community Agriculture Volunteers 956 382 574 956 Field Crops - Maize 13,072 2963 10,109 26,103 Farmer Field and Business Field Crops – Chili 4555 1051 3504 3262 Schools (farmers trained on Field Crops – Pumpkin 2807 1177 1630 3264 improved farm production, Goats 33,974 28,557 5417 100,358 improved technology, Ducks 2410 1634 776 11,151 entrepreneurship and basic business skills) Homestead (CHD) 25,947 23,698 2249 32,214 On-Farm IGA 42,222 34,264 7958 79,295 Persons trained on off-Farm IGAs 9793 4708 5085 14,877 Persons Trained in Fisheries (Capture or Culture) 3630 148 3482 9505 Participants in Cash for Work Activities (Number of 63,754 26,754 37,000 150,000 participants unavailable, only person-days of work) Youth participants in vocational and life skills training 860 177 683 10,000 Village Savings and Loan Associations (VSLAs) 944 26,447 670 947

2. Purpose 2: MCHN. Under this component, the SHOUHARDO III Program intends to reach around 57,000 children under two years of age, 76,000 pregnant and lactating women, and 45,000 other family members with interventions designed to increase access to nutritious food, improve household health, hygiene and nutrition behavior, improve access to health and nutrition services and improve WASH infrastructure and services. Major Outputs. Following are the major outputs in the SHOUHARDO III Program under Purpose 2 as defined following the revised Theory of Change finalized in September 2016.  Capacity building of community health volunteers (CHVs) on basic health, hygiene and nutrition issues.  Capacity building of community groups (CGs) and community support groups (CSGs) on their roles and responsibilities in improving community health systems

SHOUHARDO III MTE Methodology Report 17 July 2018

 Training for households on dietary diversity and importance of nutritious food  Implementation of social and behavioral change communications on early marriage, early child birth and nutrition for adolescent girls  Implementation of growth monitoring and promotion sessions  Distribution of supplemental food rations to PLW, C<2 and their families  Community-level capacity building on the importance of equitable intra-household food allocation  Implementation of community-based cooking and feeding demonstrations  Training households on good WASH practices  Implementation of demonstrations on improved low-cost household latrines  Facilitation of field visits for MoH&FW staff for monitoring and supervision (also part of P5)  Advocacy for a functioning SAM/IMCI corner in Upazilla Health Complex (UHCs)  Implementation of arsenic and coliform testing in existing wells  Training for Private Health Service Providers (PHSP) on reduction of harmful practices, IYCF, maternal nutrition, and sales/business for common nutrition and hygiene items  Facilitating the participation of community members in events conducted by the health and family planning department  Mobilization of communities for Community-Led Total Sanitation (CLTS)  Provision of support for communities to lobby for increased water and sanitation services The following outputs were described in the program proposal, but due to resource limitations, they were removed from the program strategy when the TOC was revised.  Capacity building of Community Health Care providers for health and nutrition referrals, including piloting ICT-based health referral applications  Development of a micro-franchise model for making micro-nutrient supplements available  Piloting of the Private MCHN Provider model  Rehabilitation of potable water access points using cash for work and Local Governance and Community Development Grants  Formation and training of community-based water management committees  Formation of a system of local maintenance service providers to be able to support Water Management Committees on maintaining water points  Advocacy with local government for accountability and responsiveness on community WASH from VDC through District development planning Types of Participants. Table 3 lists the different types of participants engaged and sites developed in the SHOUHARDO III Program under Purpose 2. The MTE will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites.

SHOUHARDO III MTE Methodology Report 17 July 2018

Table 3. Participants and Sites in the SHOUHARDO III Program under Purpose 2: MCHN

Current Membership/Sex LOA Type of Participant/Site Total Female Male Target Community Health Volunteers (CHV) 956 956 0 956 Members of Community Groups (CG) 762 228 534 2280 Participants in Courtyard Sessions (for dietary diversity, importance of nutritious food, good 40,780 40,780 0 52,681 WASH practices, improved household latrines) Participants in cooking and feeding demonstrations 50,141 50,141 0 40,780 Members of Community Support Groups for 834 293 541 3420 health and nutrition (CSG) Food ration recipients 40,780 40,780 0 52,681 Private Health Service Providers 0 ------120 Village leaders including the VDC and others (awareness related to intra-household food 9647 5560 4087 9647 distribution, improved household latrine demonstrations, and CLTS) Upazilla Heath Complex (SAM & IMCI corners) 23 Not Available 23 VDCs implementing health advocacy campaigns 947 Not Available 947

3. Purpose 3 Mitigating, Adapting to and Recovering from Shocks. The SHOUHARDO III Program will target 135,000 PEP community members to benefit directly from disaster preparedness, risk mitigation and risk reduction activities through interventions to improve climate-resilient livelihoods for PEP households, improve disaster risk reduction (DRR) strategies for households, communities and local government, increase DRR and Climate change Adaptation (CCA) capacities of civil society and government institutions, and increase advocacy and mobilization capacities of PEP households and communities for DRR/CCA. Major Outputs. Following are the major outputs in the SHOUHARDO III Program under Purpose 3 as defined following the revised Theory of Change finalized in September 2016.  Advocacy and capacity building for Union Digital Centers (UISC) to access early warning and weather forecasts  Capacity building of Disaster Volunteers on Disaster and Climate Risk Mitigation (DCRM) and dissemination of early warning messages  Community-level capacity building for improved disaster risk reduction strategies through implementation of Climate Risk Analysis (CRA), training on emergency contingency planning, training on disaster response, and use of disaster simulations  Support development of emergency contingency plans using a Community Risk Assessment process at the households, community, UP, Upazilla and District levels  Promotion and construction of hazard-resilient disaster mitigation infrastructure (schools as flood shelters, CRCs, protection walls and drainage structures) through Labor Contracting Societies and Joint Contracting Systems  Promote use of energy-efficient stoves  Support for VDCs to advocate for increased funding for DCRM 47

SHOUHARDO III MTE Methodology Report 17 July 2018

 Community capacity building to lobby for DCRM at the union level  Capacity building of Union Disaster Management Committees (UDMCs) and development of Union Disaster Management Plans (UDMPs) using key DCRM concepts  Facilitate the UDMC to engage Union Disaster Volunteers and pilot School-Based Teenage Brigades  Union capacity building to lobby at the upazilla and district levels for increased DCRM funding The following outputs were described in the program proposal, but due to resource limitations, they were removed from the program strategy when the TOC was revised.  Capacity building of extension services on disaster risk reduction and climate change adaptation, including promotion of flood or drought resistant seed and training on conservation agriculture  Support for community mobilization of resources to construct disaster management infrastructure  Capacity building for addressing nutrition and WASH-related challenges during emergencies  Improve environmental and eco-system management approaches through formative research on climate change impact, piloting Environmental-Based Adaptation approaches, linking integrated soil fertility management and integrated water resource management to DRR/CCA, and promoting Green Building Technology  Facilitate use of formative research and advocacy to improve DRR/CCA policies at the national level, including equitable access to public safety nets Types of Participants. Table 4 lists the different types of groups, participants and sites in the SHOUHARDO III Program under Purpose 3. The MTE will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 4: Participants and Sites in the SHOUHARDO III Program Targeted under Purpose 3: DRR/CCA

Current Membership/Sex LOA Type of Participant/Site Total Female Male Target Entrepreneurs in Union Digital Centers (UDCs) 230 115 115 230 Union Disaster Volunteers 4140 2070 2070 4140 Village Development Committees (for DCRM capacity building, community action plans, 947 5560 4087 947 resource mobilization, and lobbying) School-Based Teen Brigades 46 920 920 Pilot Union Disaster Management Committees (for 115 Not Available 115 DCRM capacity building and lobbying) Upazlilla Disaster Management Committees 23 Not Available 23

4. Purpose 4: Women's Empowerment & Gender Equity. Using the Empowerment, Knowledge and Transformative Action (EKATA) model, the SHOUHARDO III Program will

SHOUHARDO III MTE Methodology Report 17 July 2018 target 33,000 women and adolescent girls with interventions designed to increase the agency of women and adolescent girls and strengthen the enabling environment for women's empowerment. Major Outputs. Following are the major outputs in the SHOUHARDO III Program under Purpose 4 as defined following the revised Theory of Change finalized in September 2016.  Community mobilization on Gender-Based Violence and Women's Empowerment  Formation of a system for gender-based violence prevention  Design and implementation of community-level actions on violence against women  Facilitation of women to attain leadership positions in community groups and committees  Capacity building of women on influence and negotiation skills  Formation of women community support groups  Capacity building of women on life skills  Formation of School-Based Teenage Brigades with capacity-building to promote changes in gender and social norms  Formation of a Men's Forum at the community level and capacity building to enable the forum to influence other men in the community  Capacity building of community-level actors on women's empowerment  Community awareness raising on the need for women's empowerment The following outputs were described in the program proposal, but due to resource limitations, they were removed from the program strategy when the TOC was revised.  Build capacities of men from PEP households using a Social Analysis and Action process to address inequitable household workload issues and gender-based violence  Build capacities of religious and other leaders to address social barriers to women's empowerment  Build capacities of EKATA networks to be able to advocate for a common agenda on the needs of women and girls in UP planning and budgeting Types of Participants. Table 5 lists the different types of groups, participants and sites in the SHOUHARDO III Program under Purpose 4. The MTE will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites. Table 5: Participants and Sites in the SHOUHARDO III Program Targeted under Purpose 4: Women's Empowerment and Gender Equity

Current Membership/Sex LOA Type of Participant/Site Total Female Male Target EKATA Groups 33,145 0 Men from households represented in EKATA Not Not 0 No target Groups Available Available VDCs (for GBV and women's empowerment 947 5560 4087 947 action plans) Representatives from Union Parishads (LEBs) 440 101 339 440

SHOUHARDO III MTE Methodology Report 17 July 2018

Representatives from Nation Building 345 Not Available 345 Departments (NBDs) Union Parishad Nari Nirjaton Protirodh Committees 115 Not Available 115

5. Purpose 5: Public Services for PEP. The SHOUHARDO III Program expects that all residents in the 947 villages targeted by the program will benefit directly or indirectly from interventions to improve service provision by local elected bodies and nation building departments. Major Outputs. Following are the major outputs in the SHOUHARDO III Program under Purpose 5 as defined following the revised Theory of Change finalized in September 2016.  Mobilize youth groups to enhance community awareness on public services.  Train community leaders on rights, entitlements, and governance, leadership and advocacy issues.  Train locally elected bodies (LEBs) and nation-building departments (NBDs) on governance and development challenges  Establish a coordination mechanism among government departments  Facilitate the organization of remote-level service campaigns by LEBs and NBDs  Facilitate the organization of ward shavas bi-annually (shown at the output level in the current ToC, but actually may be a result/effect)  Establish a community-level Union Parishad service delivery monitoring system The following outputs were described in the program proposal, but due to resource limitations, they were removed from the program strategy when the TOC was revised.  Provide support through Local Governance Grants and Cash for Work for programs that support Community Action Plans  Support the Local Government Division to enable Union Parishad staff to achieve responsibilities outlined in the UP Act of 2009  Strengthen specific UP committees, including the Development Coordination Committee. Disaster Management Committee, Nari Nirjaton Protirodh Committee (GBV),WATSAN Committee, and Health Committee  Promote the use of the Right to Information Act, UP Information Boards, UP Information Centers, and development of citizen charters  Build capacities of Union Parishads on advocacy, good governance, women's empowerment, environmental stewardship, and disaster preparedness  Establish Program Advisory Coordination Committees to promote accountability and responsiveness in service providers Types of Participants. Table 6 lists the different types of groups, participants and sites in the SHOUHARDO III Program under Purpose 5. The MTE will conduct interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites.

SHOUHARDO III MTE Methodology Report 17 July 2018

Table 6: Participants and Sites in the SHOUHARDO III Program Targeted under Purpose 5: Public Services

Current Membership/Sex LOA Type of Participant/Site Total Female Male Target Village Development Committees 947 5560 4087 947 Youth Groups 947 6145 5521 947 Union Parishads (LEBs) 115 101 339 115 Representatives of Nation Building Departments 345 Not Available 345 (NBDs) Program Advisory Coordination Committees 37 Not Available 37

6. Cross-cutting Themes. The SHOUHARDO III Program has four major cross-cutting themes, including promoting gender equity and gender equality across all activities and sectors, targeting of youth, support for increasing the practice of good governance and addressing sources of environmental risk. Investigation on each of these will be assigned to specific MTE team members. 7. Partner Roles and Responsibilities. The SHOUHARDO III Program has the following partners. Each of these will be consulted during the MTE through interviews or remotely depending on time and location. Implementing PNGOs. For implementation, CARE is responsible for overall programmatic and financial oversight. CARE also manages (1) the commodity pipeline, (2) program-level knowledge management systems, (3) and technical support for the six implementing PNGOs as listed in Table 7. Table 7: PNGOs Implementing the SHOUHARDO III Program

Targeted PNGO District Upazila Union Village PEP HH Char Region SKS Foundation (SKS) Gaibandha 3 19 186 39,506 Eco-Social Development Organization Jamalpur 2 9 70 18,326 (ESDO) Mahideb Jubo Somaj Kallayam Somity Kurigram 4 18 188 34,540 (MJSKS) National Development Programme Sirajganj 3 16 140 33,258 (NDP) Haor Region Habiganj 2 12 63 9,554 Dhaka Ahsania Mission (DAM) Sunamganj 2 9 71 10,187 People's Oriented Program Kishoreganj 4 18 118 15,079 Implementation (POPI) Netrakona 3 14 111 14,134 TOTAL 23 115 947 174,584

Other Implementation or Management Partners. Two other types of bodies are critical for implementation of the SHOUHARDO III Program. At the community level, Village Development Committees (VDCs) perform a dual role in facilitating program implementation and ensuring sustainability for continued delivery of community-led activities. SHOUHARDO 51

SHOUHARDO III MTE Methodology Report 17 July 2018

III has also formed Program Advisory and Coordinating Committees (PACCs) at the national, division, district and Upazilla levels to provide advice on strategic directions and policy guidelines for the program. Both of these structures will be interviewed during the MTE under Purpose 5. Technical Partners. The SHOUHARDO III Program currently has signed technical cooperation MOUs with WorldFish, the GoB's Local Government Engineering Department (LGED), and the GoB's Department of Public Health Engineering (DPHE). Sub-awards have been made to WorldFish and the LGED. The program also proposed collaboration with a number of other technical partners, including Advanced Chemical Industries (ACI), Lal Teer Seed, BRAC Seed, Banglalink, bKash, Krishibid, PetroChem, Bangladesh Institute for ICT in Development (BIID), PRAN Foods, and Bangladesh Center for Communication Programs, Bangladesh Water Development Board Flood Forecasting Warning Center (FFWC), Bangladesh Meteorological Department (BMD), and AgroMet. The MTE will plan to interview organizations that have been engaged as technical partners. IV. EVALUATION TEAM The MTE will be implemented by a team of development professionals from TANGO International as described below. A. Team Members Mike DeVries (MTE Team Leader). A Design, Monitoring and Evaluation Specialist, Mike will be responsible for coordinating implementation of the MTE, including facilitation of meetings/workshops/debriefings, working with CARE to develop implementation schedules, facilitating the sharing of information between team members, and completion of the final report. In addition, he will have the lead for investigations on farm production, off-farm income, and financial services under Purpose 1 (Income & Food), program management and partnerships, financial management systems and the cross-cutting theme of addressing sources of environmental risk. Rukhsana Haider. Rukhsana is a physician, public health nutritionist, and International Board Certified Lactation Consultant with extensive experience providing technical support for formulating policies, program design and implementation. She will lead investigations on Purpose 2 (Health & Nutrition) as well as be the point person for investigations on coordination and collaboration. Mark Langworthy. Mark, a vice-president for TANGO, is an expert in the microeconomic analysis of rural households, specializing in a wide range of resilience, household food and livelihood security issues that include nutrition and agriculture. Mark will lead investigations on marketing under Purpose 1 (Income & Food), all investigations for Purpose 3(Disaster Preparedness & Response), and the use of technology for collaborative learning and action. He will also be the point person for investigations on the targeting of beneficiaries a well as the cross-cutting theme of targeting youth. Jeanne Downen. Jeanne is also a vice-president for TANGO with extensive international development experience with specialization in gender, program design, livelihoods analysis, disaster risk reduction, post-conflict transition, democracy and governance, HIV and AIDS, urban programming, and NGO management. She will lead the investigations for Purpose 4 (Women's Empowerment & Gender Equity) for the program's Theory of Change and for the 52

SHOUHARDO III MTE Methodology Report 17 July 2018 processes used by the program for collaborative learning and action. She will also be the point person for investigations on the cross-cutting themes of gender equity and human resource management systems. Golam Kabir. Golam has nearly 38 years of experience in design, management, evaluation and research related to food security and disaster management programs in and outside of Bangladesh, including 29 years in different positions within USAID Bangladesh. His areas of specialization include livelihoods development, gender equity and empowerment, coalition building, food aid commodity monetization and management, environmental compliance, and disaster risk reduction. Golam will lead the investigations for Purpose 5 (Public Services) and will be the point person for investigations on the cross-cutting theme of governance, program integration strategies, commodity management systems and materials and equipment management systems. B. Assignment of Responsibilities Table 8 indicates team member responsibilities in the MTE. The person indicated will be responsible for finalizing data collection tools, leading the analysis of information, and coordinating the writing of the content for the final report on each assigned topic. Table 8: MTE Team Member Responsibilities

INVESTIGATION TOPIC TEAM MEMBER Overall Program Design Theory of Change, including risks and assumption Jeanne Targeting of Beneficiaries Mark Collective Impact at the Goal Level Mike Collective Effect at the Purpose level - P1 Income & Food Mike Collective Effect at the Purpose level - P2 Health & Nutrition Rukhsana Collective Effect at the Purpose level - P3 Disaster Preparedness & Response Mark Collective Effect at the Purpose level - P4 Women's Empowerment & Gender Equity Jeanne Collective Effect at the Purpose level - P5 Public Services Golam Inputs, implementation, Outputs, Outcomes and Sustainability SP 1.1 Agricultural Production Mike SP 1.2 Access to Markets Mark SP 1.3 Off-Farm Income Mike SP 1.4 Financial Services Mike SP 2.1 Behavioral Change Rukhsana SP 2.1 Access to Health & Nutrition Services Rukhsana SP 2.3 WASH Rukhsana SP 3.1 Household & Community Preparedness Mark SP 3.2 Union Parishad Capacities Mark SP 4.1 Agency of Women Jeanne SP 4.2. Women's Empowerment Enabling Environment Jeanne SP 5.1 Community Capacities to Negotiate for Pubic Services Golam SP 5.2 Accountability of LEBs and NBDs Golam Cross-Cutting Themes Gender Equity Jeanne Targeting of Youth Mark Governance Golam Addressing Sources of Environmental Risk Mike Implementation Systems and Resource Management

SHOUHARDO III MTE Methodology Report 17 July 2018

Management & Partnerships Mike Collaborative Learning and Action Mark (M&E) & Jeanne (KMAL) Program Integration Golam Coordination and Collaboration Rukhsana Financial Management Systems Mike Commodity Management Systems Golam Human Resource Management Systems Jeanne Materials & Equipment Management Systems Golam Table 8 does not specify sustainability as a specific area of investigation for any one MTE team member. All team members assigned responsibilities for analyzing the effects and impact of the program will be responsible for analyzing the sustainability of the effects and impact being achieved by SHOUHARDO III. Annex B contains a topical outline that each team member will use to guide the data collection for analyzing sustainability. V. INFORMATION TO BE OBTAINED The following sections outline the information that will be obtained to achieve each of the six objectives of the MTE while meeting the purpose of the MTE to generate recommendations for the remainder of the life of the SHOUHARDO III Program. A. Objective 1: Quality of Program Delivery The MTE will examine the outputs produced under each Sub-Purpose to identify what the SHOUHARDO III Program has accomplished on the ground. The inputs and processes used to produce outputs will be investigated to identify what is working well, what can be improved, and what is not working well in terms of producing outputs against the targets that have been set for the life of the program. The MTE team will also examine the history of the program, particularly how it has evolved since inception and the critical features of the operating environment that have affected, positively or negatively, program implementation and the impact that has been achieved. In assessing the quality of the outputs, the MTE team will identify the changes in behavior or systems that are being induced by the outputs. Both qualitative and available quantitative information will be used to assess the processes and quality of the outputs being produced by the program. Table 9 provides some of the key evaluation questions and sources of information for the MTE team to be able to address each question.

Table 9: Evaluation Questions, Methods, and Data Sources for MTE Objective 1 Evaluation questions Methods Data sources Objective 1: To evaluate the quality and effectiveness of program delivery – this will cover both CARE and its implementing partners – by looking into the strengths and weaknesses of activity implementation and program management; quality of outputs in terms of adherence to terms, agreed with FFP and their appropriateness and perceived value to target communities; and identifying factors that appear to enhance or detract from the quality. 1. How well have the interventions met planned Secondary DIP, MIS data, schedules, targets, and outputs? What factors promoted data and Annual monitoring or inhibited adherence to schedules? How were documents data, IPTT, problems and challenges managed? review, Interview of KII, FGD program staff at different level

SHOUHARDO III MTE Methodology Report 17 July 2018

Table 9: Evaluation Questions, Methods, and Data Sources for MTE Objective 1 Evaluation questions Methods Data sources 2. What are the strengths and challenges to the overall KII, FGD Program staff, Govt. implementation process, management, communication staff, Implementing and collaboration so far? What factors appear to partners, other promote or challenge the activity operations or effective collaborative collaboration and cooperation among the various partners/institutions stakeholders? Specifically: The strengths and challenges should also be streamlined by each technical component, in terms of acceptance by local communities, producing quality outputs. Evaluation of the linkages established by the program with complementary programs in Char and Haor areas and beyond as well as relevant government departments. 3. In each technical sector, what are the strengths of and KII, FGD Program staff, Govt. challenges to the efficiency of interventions’ staff, Implementing implementation and their acceptance in the target partners, other communities? How well do implementation processes collaborative adhere to underlying principles and activity protocols? partners/institutions What factors in the implementation and context are associated with greater or lesser efficiency in producing Outputs of higher or lower quality? Which interventions and implementation processes are more or less acceptable to members of the target communities and why?>> To what extent are the program interventions meeting the needs of target beneficiaries, specifically women and youth? Are interventions aligned with Bangladesh’s country strategies and policies and/or with USAID development goals, objectives, and strategies? Information on processes, outputs and effects that are emerging will be obtained through reviews of program monitoring reports, including quarterly and annual reports, reviews of key secondary data, interviews/focus group discussions with program participants, and interviews with program implementation staff. All team members will have responsibilities for achieving this objective relative to their assigned sub-purpose investigations. MTE team members will analyze performance monitoring data collected by the program, as well as review the performance management plans and IPTTs. MTE team members will review technical guidance, including implementation manuals, on key processes and approaches used by the program and observe learning/training sessions that are planned when the MTE team is in the area.

SHOUHARDO III MTE Methodology Report 17 July 2018

B. Objective 2: Evidence of Change To accomplish Objective 2, the MTE will start by reviewing and understanding the theory of change and logical framework for the program. This has been developed around a vision and pathways of change for particular types of targeted impact groups, so the MTE will review the targeting systems for the program to identify strengths and weaknesses in how participants are identified to assess how well the program is reaching intended beneficiaries as reflected in the Theory of Change. Using qualitative information obtained during the MTE as well as available quantitative information, the MTE team will identify changes that have occurred as a result of program interventions. These changes will occur at different levels, including effects at the individual level, the household level, the community level, the mezzo-level (union, Upazilla and District) as well as at the national level. They can include positive or negative changes as well as changes predicted by the program's Theory of Change or unexpected changes that might result in a recommendation for a revision to the program's ToC. At the individual and household levels, MTE team members will look for evidence on how PEP participants have changed their ideas, attitudes, and practices as a result of program activities

Priority Topic for Investigation Priority Topic for Investigation WASH Interventions Value Chain Development WASH is emerging as a critical issue in One of the priority topics of interest to the the Haor and Char areas. The program SHOUHARDO III stakeholders is to examine the value had not laid much emphasis on WASH chain and market development strategies of the program. Concerns have been raised that SHOUHARDO I and II at proposal time but has during were not effective enough in achieving significant growth implementation noted clear gaps in in targeted value chains. A fundamental challenge has safe-water provision and sanitation in been that, by targeting PEP households, the program is the Haor. MTE should give attention as not working with households who have sufficient assets to how to more effectively address to be able to expand engagement and promote significant these. expansion of value chains. and will identify potential reasons for why some beneficiaries have started applying program promoted practices while others have not. At the community and mezzo-levels, the MTE team will look for changes in behavior by intermediaries and changes in systems that have been induced by the program. At the national level, the MTE will look for changes that have been induced in behavior, systems and national-level policies. Table 10 provides some of the key evaluation questions and sources of information for the MTE team to be able to address each question. Table 10: Evaluation Questions, Methods, and Data Sources for MTE Objective 2 Evaluation questions Methods Data sources Objective II: To present yearly evidence of changes (intended and unintended, positive and negative) associated with activity interventions and outputs; assess how well the observed changes reflect the TOC and LF; and identify factors in the implementation or context that impede or promote the observed and intended changes.

SHOUHARDO III MTE Methodology Report 17 July 2018

Evaluation questions Methods Data sources 4. What early evidences of changes—expected and Review Community based unexpected, positive and negative—do community program Govt. and Non-Govt. members and other stakeholders associate with the documents, organizations, program program interventions? What factors appear to KII/FGD, participants, program promote and deter these changes? Which direct staff interventions appear to be more or less potential observation to influence knowledge or behaviors? How do the of program changes correspond to those hypothesized by the activities TOC? Specifically: to focus on the TOC and assess any changes, identify gaps, external outcomes, and stakeholders. Are objectives of the program in keeping with Review Community based locally defined needs and priorities? Should the program Govt. and Non-Govt. direction of the program be changed to better documents, organizations/stakehold reflect those needs and priority? KII/FGD, ers, program direct participants, program observation staff of program activities To what extent have interventions aligned with Secondary DIP, MIS data, Annual planned targets of participant, timelines, and data and monitoring data, IPTT, outputs? Any extenuating elements that impeded or documents Interview of program accelerated plans? How were challenges mitigated review, KII, staff at different level, or addressed? FGD program participants Information for understanding the changes induced by the program will come primarily through key informant interviews and focus group discussions with targeted beneficiaries and intermediaries, as well as through interviews with program implementation staff and partner leadership representatives. C. Objective 3: Collaborative Learning and Action (Including Monitoring and Evaluation) Collaborative Learning and Action refers to (a) how programs bring in and test new ideas and approaches, (b) how monitoring and evaluation systems provide information for decision- making and reporting, and (c) how best practices and lessons learned emerging from program implementation are documented and shared outside the program. The MTE will obtain information to understand the systems being used for capturing, documenting and disseminating lessons learned and best practices. SHOUHARDO III's Planning, Monitoring and Evaluation System (PM&ES) will be examined to understand how information generated from the system is being used to enhance the effectiveness or efficiency of the program in achieving impact. Emphasis will be placed on the collection, analysis and management of data to enable iterative learning and evidence-based improvements to program design and implementation. The MTE will identify changes that have already occurred in the SHOUHARDO III strategy or implementation as a result of new ideas or approaches brought into the program from outside sources and will formulate recommendations for how to seek out appropriate new ideas and 57

SHOUHARDO III MTE Methodology Report 17 July 2018 approaches in the remaining life of the program. The research agenda for the program will be reviewed for relevance and timeliness in providing information for decision-makers. Table 11 provides some of the key evaluation questions and sources of information for the MTE team to be able to address each question. Table 11: Evaluation Questions, Methods, and Data Sources for MTE Objective 3 Evaluation questions Methods Data sources Objective III: To review systems for capturing and documenting lessons learned and assess the extent to which they are used in program implementation and refining program design, including feedback from the perspective of stakeholders and participants. This would cover assessing processes to use evidence including baseline results and monitoring data for adjusting program strategies. Also it will assess how well the program is seeking out, testing and adapting new ideas and approaches to enhance programs’ effectiveness or efficiency. 5. How have SHOUHARDO III management and M&E report M&E reports, M&E technical specialists used data to inform review, KII, team, Technical programmatic decisions, referral and follow up? FGD specialists, program What processes have been instituted to improve implementing staff data collection and data quality? How has the program improved effectiveness or Direct Field activities, efficiency as a result of new ideas or approaches observation, program staff, brought into the activities? How is information FGD, KII, program participant, generated by the programs used to inform document M&E documents decision-making? How can this be made more review effective? Information for collaborative learning and action investigations will be obtained from a review the program's PMPs, monitoring reports and other reports, interviews with program management and representatives of program steering and advisory committees, interviews with program implementation and technical staff, interviews with the leadership of partner organizations, and interviews with knowledge management staff. D. Objective 4: Sustainability The MTE will review the SHOUHARDO III sustainability strategy, interview staff and management, interview program participants, and interview private and public stakeholders to determine how likely program-generated outcomes and impact are to be sustained after the program ends. The investigations will identify the organizations, services and relationships that need to be maintained after the program ends in order to sustain each of the outcomes planned by the programs and will analyze the threats to these that could affect likely sustainability. The primary threats that will be considered are the resources available, the technical and managerial skills, and the motivation required to sustain the organizations, services or relationships necessary to sustain the impact achieved. Specific attention will be paid to the appropriateness, efficiency and efficacy of capacity building activities targeted to participants and local partners to enable them to sustain program impact. Table 12 provides some of the key evaluation questions and sources of information for the MTE team to be able to address each question.

SHOUHARDO III MTE Methodology Report 17 July 2018

Table 12: Table 12: Evaluation Questions, Methods, and Data Sources for MTE Objective 4 Evaluation questions Methods Data sources Objective IV: To determine the extent to which outcomes, systems and services are designed and being implemented to continue after the program ends and assess progress made on implementing sustainability strategies. 6. Has the program developed and implemented Direct Field activities, sustainability strategies? What organizations, services observation, program staff, or relationships are required to sustain the early FGD, KII program participant, evidences of changes observed by the review team? community Has the program identified the changes to be organizations, sustained, and the necessary services required to stakeholders sustain these changes? Have the programs identified the potential service Direct Field activities, providers? What are the motivations of the service observation, program staff, providers to continue service provisioning after the FGD, KII program participant, programs end? What has been done so far to increase community the motivation? What would be the motivation of the organizations, beneficiaries to receive these services? Have the stakeholders programs identified the resources and capacity strengthening supports for the service providers? What has the program done to ensure that this Direct Field activities, motivation does not diminish after the program ends? observation, program staff, To what extent are government officials, formal and FGD, KII program participant, informal local leaders (whose support and community understanding will be critical for continuing program organizations, Govt. initiatives once the program has ended) involved in staff, implementing program activities and included in ongoing program partners and other discussions and decision-making processes? stakeholders Are there proper identification of roles on who FGD, KII Program staff, should be doing what for a sustainable exit for the program participant, program – particularly on the improvement of services community from the GoB? organizations, Govt. staff, implementing partners and other stakeholders

E. Objective 5: Cross-Cutting Themes For each of the four-fundamental cross-cutting themes for the program, including gender, governance, targeting of youth, and environment, the MTE team will assess how well the program is operationalizing the cross-cutting theme and achieving the proposed impact. Information on cross-cutting themes will be obtained through interviews with program management, program implementation staff, leadership of partners and stakeholders, and program participants. Program monitoring reports will be reviewed, and specific strategy

SHOUHARDO III MTE Methodology Report 17 July 2018 documents related to each cross-cutting theme, including assessment reports and barrier analyses, will be reviewed.

Priority Topic for Investigation Advocacy for Good Governance SHOUHARDO III is in a good position to influence governance and GoB policies/strategies since the program works closely with government, has also made a commitment to implementing advocacy activities, and CARE in Bangladesh and elsewhere has good experience with effective advocacy. The MTE should look at the question of how a program like SHOUHARDO III can best implement advocacy/governance strategies in the current Bangladesh context in which NGOs have limited influence with government and are reluctant to become too visible or confrontational with advocacy activities.

Priority Topic for Investigation Gender & Women's Empowerment The Bangladesh context is one in which women's empowerment is critical to accelerating development and reducing food insecurity, and the SHOUHARDO programs have elevated women's empowerment to the purpose level in the program strategy. The MTE should review in-depth the gender activities of the program to ensure that they are as effective as possible, including activities not only under Purpose 4 but also gender-related activities under all other purposes.

Table 13 provides some of the key evaluation questions and sources of information for the MTE team to be able to address each question. Table 13: Evaluation Questions, Methods, and Data Sources for MTE Objective 5 Evaluation questions Methods Data sources Objective V: To determine the appropriateness and effectiveness of support for gender equity in terms of access to, participation in, and benefit from program interventions. Assess the extent to which program interventions target youth, support greater capacities for local governance and address sources of environmental risk. 7. How effective are program design and Document Program documents, field implementation mechanisms in addressing the review, activities, program staff, cross-cutting issues of gender, governance, the Direct program participant, environment and targeting of youth? What (if observation, community organizations, any) challenges have programs encountered in FGD, KII Govt. staff, implementing these areas that may not have been anticipated partners and other in the program design, and how have the stakeholders programs responded? To what extent do program interventions and Document Program documents, field implementation mechanisms reflect integration review, activities, program staff, of these cross-cutting priorities? Direct program participant, observation, community organizations, FGD, KII Govt. staff, implementing partners and other stakeholders What steps have the activities taken to ensure Document Program documents, field that staff has adequate capacity for addressing review, activities, program staff, these cross-cutting issues? Direct program participant, observation, community organizations, FGD, KII Govt. staff, implementing

SHOUHARDO III MTE Methodology Report 17 July 2018

Evaluation questions Methods Data sources partners and other stakeholders

Are there any yearly evidences of changing Document Program documents, field roles, relationships, communication and review, activities, program staff, decision making dynamics among women and Direct program participant, men, young and old, in relationship to food observation, community organizations, security at the household and community FGD, KII Govt. staff, implementing levels? partners and other stakeholders How were the findings and recommendations Document Program documents, field of the Year 1 gender analysis considered, and review, activities, program staff, what specific changes to the program strategy Direct program participant, and program activities, were made? Is the observation, community organizations, program activities drawing on the potential of FGD, KII Govt. staff, implementing women, men, boys and girls as much as partners and other possible? stakeholders

F. Objective 6: Implementation Systems Program implementation systems include management systems, partner relations, program integration across purposes, coordination and collaboration with other programs or organizations, knowledge management systems (covered under Objective 3), financial management systems, commodity management systems, human resource management systems and materials and equipment procurement and management systems. The MTE will review these systems to identify recommendations for improving the effectiveness of implementation and/or the efficiency in use of resources. Information on implementation systems will be obtained from directly reviewing the Priority Topic for Investigation systems being used, program monitoring Food Distribution, Vouchers or Cash reports including quarterly reports, annual SHOUHARDO III is the only one of the three current results reports, financial reports, and DFSAs in Bangladesh that is still distributing food. The commodity management reports. HKI Program uses vouchers to make food available, and WVI provides cash transfers for food purchase by Interviews will be held with program participating households. SHOUHARDO III has looked management staff, implementation staff, and at the possibility of using vouchers to provide access leadership of CARE, PNGOs and technical to lentils, and the MTE should examine this, as well as other options, more closely to identify the best partner organizations, and relevant approach for making supplemental food available technical staff from each of the partner more effectively and efficiently i n the Char/Haor context. . organizations. Table 14 provides some of the key evaluation questions and sources of information for the MTE team to be able to address each question.

SHOUHARDO III MTE Methodology Report 17 July 2018

Table 14: Evaluation Questions, Methods, and Data Sources for MTE Objective 6 Evaluation questions Methods Data sources Objective VI: To recommend adjustments to the TOC and LF, activity design, resource allocation, activity management, M&E Plan, or implementation that could improve the likelihood of achieving sustainable results by the program’s end, based on the yearly evidences collected and conclusions drawn for the evaluation objectives above. 8. Based on the findings from Questions 1–5, Document Program documents, field how could the activity be modified to improve review, activities, program staff, its acceptability to targeted communities or the Direct program participant, efficiency and effectiveness of its observation, community organizations, implementation? How should the TOC and LF FGD, KII Govt. staff, implementing be refined or modified? partners and other stakeholders An assessment of efficiency should consider Document Program documents, field whether the same results could have been review, activities, program staff, achieved with fewer resources or whether Direct program participant, alternative approaches could have been observation, community organizations, adopted to achieve the same results. FGD, KII Govt. staff, implementing partners and other stakeholders

VI. EVALUATION PROCESS The MTE will be undertaken over a period of approximately six months from mid-December through mid-June of 2018 and will be implemented in three phases:  Phase 1: Evaluation Preparation (October 23- February 12)  Phase 2 (in Bangladesh): Data Collection & Preliminary Analysis (February 12-March 12)  Phase 3: Evaluation Recommendations Finalization (March 13-June 15) The current schedule for the MTE is provided in Annex A. The sections which follow describe the major activities planned in each phase. A. Phase 1: MTE Preparation (October 23 through February 12) During the MTE preparation period, a draft protocol including a basic operational plan (this document) will be drafted, the MTE team will review the background documents listed below, the data collection tools will be drafted, and the sites for field visits will be selected. The members of the MTE team will assemble in Dhaka by February 11. On Monday, February 12, the team will hold an initial meeting to meet each other and to discuss the MTE plan. Also on that day, the MTE team leader will meet with CARE to select sites for field visits. As soon as possible, either Monday afternoon (12th) or Tuesday (13th), the MTE team should also have an introductory meeting with USAID/FFP Bangladesh. The following sections describe these MTE processes and meetings in more detail. 1. MTE Protocol. This document is the protocol that guides implementation of the MTE. The Zero Draft of the protocol will be circulated on December 15 to the MTE team members and 62

SHOUHARDO III MTE Methodology Report 17 July 2018

the SHOUHARDO III management team. This draft will be incomplete and both the MTE team and the SHOUHARDO III management team will work on clarifying and completing the protocol. The first Full Draft of the protocol will be released for wider review by MTE organizers and FFP on January 16. Comments, questions or suggestions on the draft should be sent to Mike DeVries ([email protected]). Feedback will be incorporated in a working draft that will be widely circulated by January 30, as the planning guide going into the implementation phase of the MTE. Further changes will be may be made once the MTE assembles in-country and has a chance to discuss the program in more depth with CARE SHOUHARDO III staff. The final version of the protocol will be developed before the MTE team departs to begin field data collection. 2. Background Document Review. There are many documents that could be reviewed to prepare for the MTE. However, given time limitations for team members, the available documents have been divided into two sets. One set represents required reading for all team members in order to be able to understand the full strategy for the program as well as the range of activities being undertaken. The second set of documents should be reviewed by individual team members when possible. Required Reading. The following documents are required reading for all team members and must have been reviewed before the team assembles in-country on February 12.  Approved Program Technical Narrative  Theory of Change and Logical Framework (from the Performance Monitoring Plans)  Indicator Performance Tracking Table (from the Performance Monitoring Plans unless there has been a subsequent revision)  Annual Results Reports for FY 2016 and FY 2017  All Quarterly Monitoring Reports  The Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Programs Paper by Rogers & Coats  The USAID Evaluation Policy document from 2011 Other Reference Documents. The following documents are also available and should be reviewed by team members when possible.  FFP Minimum Standards for Farmer Field Schools, Village Savings and Loan Associations, and Community-Led Total Sanitation  FFP Checklists for Asset Reviews, Irrigation, Road Assets and Elevated Structures  SHOUHARDO III Baseline Study Report  FY 2016 & FY 2017 PREPs  Full Performance Monitoring Plan  Knowledge Management & Learning Background Documents  Program Design Background Documents  SHOUHARDO III Program Integration Strategy and the Program Integration Implementation Framework (PIIF)  Value Chain Analysis Reports

SHOUHARDO III MTE Methodology Report 17 July 2018

 Labor Market Assessment Report  Gender Analysis Reports  SBCC Barrier Analysis Reports  Program Implementation Manuals (or training curricula and training materials in lieu of a PIM)  Participant Registration Data  Environmental Mitigation and Monitoring Plan  Data Quality Assessments  Human Resource Background Documents  The SHOUHARDO II Final Evaluation report Obtaining Documents. All background documents will be made available on the DropBox (CARE Bangladesh SHOUHARDO MTE). The key point of contact for this repository is Mike DeVries ([email protected]). 3. Data Collection Tools. Section VII below provides guidance on the data collection tools, primarily topical outlines which will be needed for the MTE. MTE Team Members will be responsible for developing the initial drafts of these data collection tools with the deadline for completing the drafts on January 10. These drafts will be incorporated in Annex B in the draft of the protocol that will be reviewed by CARE and FFP. In the program orientation meeting to be held on February 13 (described below), there will be an opportunity for SHOUHARDO III staff to provide input into the topical outlines, particularly to discuss topics that appear on the outline but are irrelevant or topics that need to be added to the topical outlines. The orientation will be also be used to obtain clarity on the types of participants in the program for which the topical outlines need to be tailored. Final working versions of the topical outlines will be included in the final version of the protocol that will be developed before the MTE team departs of field data collection. 4. Site Selection. Section VII provides guidance on how sites will be selected for field visits for the MTE. During this preparation phase, CARE should identify within the program's participant database the clusters of sites that have been requested in Section VII. The MTE Team Leader will meet with the SHOUHARDO III management team shortly after arriving to finalize the preliminary selection of sites by February 13. 5. Key Meetings. Two meetings are planned for February 12 and 13 after the MTE team has assembled in Dhaka as described below. Initial MTE Team Meeting, February 12. The initial meeting of the MTE Team will occur immediately after team members have assembled in Dhaka. In this meeting, team members will be introduced to each other, the MTE team leader will provide an overview of the evaluation process, and answer any questions that team members may have. The additional required work required to finalize the protocol before the team leaves to begin field data collection will also be discussed. Orientation Meeting with USAID/FFP Bangladesh, February 12 or 13. Following the initial team meeting, an orientation meeting will be held with representatives of USAID/FFP Bangladesh at which the MTE Team will be introduced, an overview of the review process will be provided by the team leader and the MTE team will answer any questions that USAID/FFP may have about 64

SHOUHARDO III MTE Methodology Report 17 July 2018

the evaluation. In addition, the MTE team will seek to understand the expectations of USAID/FFP and any specific interests that they would like to see covered by the review.12 B. Phase 2: Data Collection & Preliminary Analysis From Tuesday, February 13, through Thursday, February 15, the MTE Team will be in Dhaka to continue preparations for the MTE. On Friday, February 16, the team will travel to the field to gather information at the field level. Field level data collection will be undertaken for 16 days over the period from February 16 through March 3. On March 3, the team will travel to Dhaka where the Verification Workshop will be held. Dhaka is the most convenient site for SHOUHARDO III field staff from the two regions to participate. From Sunday, March 4 through Tuesday, March 6, the team will process the information obtained and prepare for the Verification Workshop which will be held for two days, on Wednesday and Thursday, March 7 and 8. Following the Verification Workshop, the MTE team will have two days to continue gathering information to fill "gray" areas identified in the Verification Workshop and to prepare for the two debriefings scheduled in Dhaka. These are a Stakeholder Debriefing which will be held in the morning on Sunday, March 11, and a debriefing with USAID/FFP to be held in the afternoon of March 12. The following sections provide more detail on the key events in this phase of the MTE. 1. SHOUHARDO III Program Orientation Meeting, February 13 (Tuesday). In this half-day meeting, the SHOUHARDO III implementation team will present an orientation to the program for the MTE Team. The key content should include an overview of the strategy of the program, an overview of the resources (money, commodities and staff) available for the program, a map showing geographic locations, a description of the outputs and activities under each component, a description of the roles and responsibilities of partners, and a description of the key challenges affecting the program. The purpose of this meeting is to obtain clarity on the types of outputs produced by the program, the stakeholders that need to be interviewed to understand the impact of these outputs, and additional data sources for information to support the MTE. Discussions will also be held around (a) the draft topical outlines that have been developed and are provided in Annex B, (b) the site selection for the qualitative interviews to determine what the sites represent in terms of outputs and quality, and (c) the meeting schedule that has been compiled for the MTE for the next two days in Dhaka. Attendees to this meeting include the SHOUHARDO III management and technical team, selected representatives from implementing partners, and anyone else from the CARE likely to be involved in implementing or supporting the MTE. 2. Meetings with SHOUHARDO III Stakeholders in Dhaka, February 14 & 15 (Wednesday & Thursday). On these days, MTE Team members will conduct interviews with key stakeholders that are based in Dhaka. These could include interviews with representatives of implementing partners, technical partners, government stakeholders, and private sector stakeholders. Given the limited time, the MTE will need to be strategic on the use of this time and will rely on the SHOUHARDO III management team to provide guidance on prioritizing the meetings that need to held.

12 In the actual implementation of the protocol, because of scheduling conflicts, this meeting with USAID had to be scheduled a few days later than planned. 65

SHOUHARDO III MTE Methodology Report 17 July 2018

3. Field Data Collection, February 16 through March 3. Over this period, the MTE team will conduct key informant interviews, hold focus group discussions and observe program activities to obtain qualitative data. The team will plan to spend up to three days in each of six districts covered by the program. A preliminary schedule has been developed (and is provided in Annex A). With this schedule, the MTE expects to conduct interviews, focus group discussions and site visits to 28 sites (see section VII. F. below on sample selection for additional detail on the types of sites that have been selected). The specific data collection process in each district is described in section VII.C below. Annex A contains a preliminary schedule for this period. A detailed itinerary for the field work will be developed in collaboration with SHOUHARDO III management team and implementing partners. 4. Information Processing & Preparation for the Verification Workshop, March 4 - 6 (Sunday through Tuesday). During the field work, MTE team members will begin processing data as it is obtained, and in this three-day period the team will continue analyzing information to identify major findings and recommendations that will be presented for discussion at the Verification Workshop. Each MTE team member will use this time to prepare the PowerPoint presentation(s) that she/he plans to present in the Verification Workshop. In the afternoon of March 5, the MTE team will sit together and share ideas on the recommendations that are emerging from the analysis. 5. Verification Workshop, March 7 & 8 (Wednesday and Thursday). In this two-day workshop, the major observations and the recommendations resulting from the MTE investigations will be shared with program implementation staff. These will be discussed to ensure that they reflect reality and are described appropriately. Two major outputs are targeted for the workshop. These are (1) agreement on the validity of key observations assembled so far from the review and (2) preliminary agreement on major recommendations for the remaining life of the program. Participants in the workshop will be SHOUHARDO III implementation staff and the MTE team. This is an in-house event, only for those participants who are fully engaged in the program, so that the discussions held can be candid and frank. The workshop venue will be at a location in Dhaka, and an illustrative Workshop Plan is provided in Annex C. 6. Post-Workshop Processing, March 9 & 10 (Friday and Saturday). During this period, MTE team members will revise their PowerPoint presentations to capture key points from the discussions in the Verification Workshop. Each team member will also prepare a condensed version of her/his presentation that can be consolidated with other team member presentations for the stakeholder and USAID/FFP debriefings. The MTE team leader will collate these into a single presentation for the Stakeholder Debriefing. In addition, during this post-workshop period, the MTE team leader will meet with the SHOUHARDO III management team to discuss the recommendations that were presented in the workshop. The purpose of this meeting is to obtain feedback on those recommendations that are perceived to be most useful and those that are seen to be less useful either because they are already being undertaken or because the time and resources available to implement a recommendation are insufficient. The MTE team will be prioritizing recommendations after the

SHOUHARDO III MTE Methodology Report 17 July 2018

workshop in preparation for the stakeholder and USAID debriefing, and the perspectives of the SHOUHARDO III management team will be a key input into this prioritization. 7. Stakeholder Debriefing, March 11 (Sunday). A draft summary of the MTE product with prioritized observations and recommendations will be presented for discussion to the leadership of CARE, leadership of SHOUHARDO III's implementing partners, representatives from other program stakeholders, such as government agencies or technical partners, and the FFP representatives who have direct responsibilities for the SHOUHARDO III Program13. This is an opportunity for implementation stakeholders to provide input into the evaluation as another step in the process of producing final recommendations. The debriefing will be held at a set location in Dhaka to which representatives will be invited and participants are expected to be the SHOUHARDO III CoP, CARE Bangladesh leadership, leadership of PNGOs implementing SHOUHARDO III, representatives from key technical partners, representatives from other agencies having active roles in the program and the FFP AOR. Given the tight schedule, the Stakeholder Debriefing should be scheduled for the morning on March 11, so that the MTE team will have sufficient time to prepare for the USAID debriefing scheduled for the next day. 8. USAID/FFP Bangladesh Debriefing, March 12. Following the Stakeholder Debriefing, changes will be made to the MTE based on the discussions, and a summary of the product to this point will be presented to USAID/FFP Bangladesh. Participants in this meeting will be the leadership of CARE, the SHOUHARDO III CoP and representatives from USAID. The MTE team leader will facilitate the presentation, and all MTE team members will attend. To maximize the preparation time for this debriefing, the debriefing should be scheduled in the afternoon of March 12. Customarily, two hours of time is allotted for presentation and questions. C. Phase 3: Evaluation Recommendations Finalization, March 13 to June 15 After the USAID debriefing on March 12, the MTE team will disperse and work remotely to assemble a draft report by April 16. Sometime in the latter half of May, a debriefing by the MTE team leader will be provided to HQ-based stakeholders from FFP and CARE. Then, by June 15, a final MTE report will be completed incorporating feedback received on the draft and from FFP/CARE stakeholders in headquarters. Recently, the MTE evaluation process has been utilizing an additional step in which the MTE team works with the program staff to operationalize the recommendations as part of the preparations for submitting the next Pipeline and Resource Estimate Proposal (PREP). Discussions around doing this for the SHOUHARDO III MTE are on-going and have not been concluded. However, a description of this step in the process is also provided below. 1. Draft Report, April 16. MTE team members will continue reviewing and analyzing background documentation and information collected toward assembling content for the draft

13 In the past, donor representatives have not attended the Stakeholder Debriefing. The USAID/FFP debriefing, which is the first event in the process at which FFP representatives have usually attended, however, tends to be relatively short with insufficient time to go into much depth on observations and recommendations. The Stakeholder Debriefing provides a better opportunity for those FFP representatives who are directly responsible for the SHOUHARDO III Program to see more detail on observations and recommendations as well as to provide key input. 67

SHOUHARDO III MTE Methodology Report 17 July 2018

report. The MTE team leader will consolidate the writing into a draft report to be finalized and circulated by April 16. 2. Debriefing at FFP HQ, Latter Half of May. Major content from the draft report, including the final list of prioritized recommendations, will be presented to HQ-based stakeholders sometime in the latter half of May for discussion and feedback. These stakeholders include FFP representatives, other USAID representatives and representatives from CARE headquarters. 3. Final Report, June 15. Following the headquarters-based debriefing, final changes will be made by the MTE team leader based on the feedback received in the meeting as well as other feedback received on the draft report to produce a final report on the MTE by June 15. 4. MTE Recommendation Processing and Planning Workshop. The draft report prepared by April 16, while not being final, will fully document recommendations and supporting observations. This document is used as the key document for a MTE Recommendation Processing and Planning Workshop. This workshop is usually two to three days in length, and the objective of the exercise is to review the final recommendations with program staff and discuss how best to operationalize the recommendations. Since SHOUHARDO III management and technical staff will have seen most, if not all, of the recommendations in the Verification Workshop, they will have already begun strategizing on how to respond to the recommendations. The MTE team leader and other members of the MTE team who can be available for the workshop will be present to provide clarification on the recommendations and input on how best to operationalize them in the next PREP. VII. QUALITATIVE DATA COLLECTION Qualitative data will be collected primarily through key informant interviews and focus group discussions with representatives of SHOUHARDO III implementation staff, program participants (both beneficiaries and intermediaries), and program partners including PNGOs, technical partners and GoB partners. Topical outlines or checklists will be used to guide interviews and focus group discussions. Annex B contains the version of the tools that have been refined with input from SHOUHARDO II staff. These represent the working tools with which the MTE team will begin data collection. Ideally, the MTE will gather information from all of the partners and participants listed in Section III above describing the program. MTE Team members will provide program management staff with a list of the meetings, interviews and FGDs that they would like to have arranged, and the MTE team will work with program management staff to develop schedules for meetings. A. Composition of Key Informant Interviews and Focus Group Discussions Key informant interviews are normally held with between one to four persons, ideally no more than two, and these interviews typically last no more than one hour. Focus Group Discussions (FGDs) are organized with from five to no more than ten persons. They are called focus groups because all of the members of the group have a common feature, so FGD facilitators must ensure that participants meet the desired common criterion, i.e., members of a VSLA group, farmers who received seed from the program, lead mothers, lead fathers, etc. FGDs should normally not extend beyond two hours. In both cases, interviews and discussions should be held in secluded locations so that bystanders or passersby cannot influence the discussions. B. Topical Outlines or Checklists 68

SHOUHARDO III MTE Methodology Report 17 July 2018

MTE Team members will develop topical outlines or checklists which include key thematic questions to be used in key informant interviews or focus group discussions. These tools were drafted by the individual MTE Team Member assigned to specific investigations and tailored to the types of participants and approaches being used by the program. The questions in a topical outline are fairly general and used to stimulate discussion. Reviewers will keep in mind at all times that the purpose of the information gathering is to understand what the program has done, what changes have occurred as a result, what has helped or hindered achievement of these changes, and how likely are the changes to be sustained after the program ends. The discussion facilitator will be free to explore in more depth any relevant topics that may come up during each discussion, related to these objectives, and is also free to skip particular questions on a topical outline if it is apparent that the respondent being interviewed has no relevant information for those questions. A suggested sequencing of questions in a topical outline, drawn from a Program Constraints Assessment (PCA) approach, would be as follows. 1. Are you familiar with the SHOUHARDO III Program? 2. How would you describe what this program seeks to accomplish? 3. How have you or other members of your household participated in this program? For how long have you/they been involved? 4. Please describe how you or members of your family have benefitted from the program. 5. Please describe how you or members of your family have been negatively affected by the program. 6. Who, in your opinion, has benefitted most from the program? 7. Are there other people who should be benefitting from the program but are not? Please describe them for us. 8. What constraints do you believe inhibit the program from fully accomplishing its purposes? 9. What suggestions do you have for addressing these constraints or otherwise enabling the program to have greater impact? For each question, the interviewer/facilitator should have an idea about what kind of response to expect, based on a review of the background documents, but should avoid leading the respondent to make these responses. After a respondent has completed answering a question and an expected topic has not come up, the interviewer/facilitator can then ask…what about this?…. noting that the respondent did not spontaneously report on the topic. Before beginning an interview or discussion, an introduction and explanation of the purpose of the evaluation should be provided with stress put on the importance of obtaining useful information that reflects reality. The evaluation team will seek consent from the participants, and no names will be recorded in interviews and group discussions. If portable recording devices are used, the device should be shown to respondents and not activated until after respondents have introduced themselves. Annex B contains the final data collection tools. After the MTE team assembled in-country, each team member gathered information from SHOUHARDO III Program staff to further refine the tools, including tailoring a tool for specific types of respondents and adding more detailed information on the types of interventions that have been implemented and would be expected to come up in interviews or focus group discussions. The SHOUHARDO III Program 69

SHOUHARDO III MTE Methodology Report 17 July 2018

orientation scheduled for February 13 was critical to this process. After the orientation, individual MTE team members began interviewing SHOUHARDO III staff to further refine the topical outlines. The name of the MTE team responsible for a particular tool, corresponding to the team member assignments indicated in Table 8, is shown on each tool. C. Observation of Program Activities In addition to conducting interviews and focus group discussions, MTE team members will also observe implementation of program activities and physical sites where program investments have been made. These should be regularly scheduled program activities, not activities organized only for the MTE. Section III, describing the program, lists the different types of sites/activities that the members of the MTE team should visit. D. District-Level Data Collection A three-day process is envisioned for data collection in each of six districts in which the SHOUHARDO III Program is being implemented. The MTE team will conduct interviews and discussions an at least 31 communities or sites. In any one community, the team will not spend more than a half day, and the team will break up into smaller teams as necessary to be able to visit as many sites as possible to view the full range of major program interventions and successful pilots. Site selection, following the parameters described in Section F below, and development of a field work schedule will be done in collaboration with SHOUHARDO III management staff by February 13. In addition to conducting interviews and focus group discussions, MTE team members will also observe, where possible, implementation of program activities and physical sites where program investments have been made, as already mentioned. During visits to communities and district stakeholders, it is expected that SHOUHARDO III implementing staff will accompany the evaluation team to facilitate the introduction of the team to the community or stakeholders. These staff will not participate in interviews and FGDs after the introductions have been made, but they should be available to answer questions from the MTE team members after the interviews/FGDs to discuss the information that has been obtained. The following sections describe the sequencing of activities in each district14. Day 1 First Half: Program Orientation and Interviews with Implementation Staff. A different program partner is implementing activities in each district. In order for the MTE to begin to understand similarities and differences in implementation between different partners, the first activity in each district will be a meeting between the MTE team and the district-based program staff. Items on the agenda for this meeting are (1) introductions and explanation of the purpose and objectives of the MTE, (2) review of the schedule for activities in the district including the meetings scheduled with district-based stakeholders, (3) separate meetings between individual MTE team members and SHOUHARDO III implementation staff by sector, and (4) a reconvening of the full group to discuss the unique features of and challenges for SHOUHARDO III in the district. The district Program Manager should prepare a short

14 The sequence will vary depending on the day of the week, since some stakeholders may not be available on a Friday or Saturday. In situations like these, the stakeholder interviews will be moved forward or backward in the process in order to fall on a regular business day. 70

SHOUHARDO III MTE Methodology Report 17 July 2018

presentation on the unique features and challenges for the program as a basis for the discussion and provide a handout on the quantitative achievements in the district. Day 1 Second Half: Field Visits to Communities and Sites. In the second half of the first day in each district, the MTE team will conduct interviews and focus group discussions with program participants, including both targeted beneficiaries and intermediaries, in sample sites. Day 2 First Half: Meetings with District-Based Stakeholders. In the first half of Day 2, MTE team members will conduct interviews with District-based stakeholders for SHOUHARDO III. These could include relevant government departments, private sector representatives based in the district or technical partners based in the district. The community visits held on the previous day will provide information to inform these interviews, i.e., MTE team members will be able to query stakeholders on observations made in the community visits. Day 2 Second Half & Day 3: Field Visits to Communities and Sites. In the remaining time in each district, the MTE team will continue conducting interviews and focus group discussions with representatives of program participants, including both targeted beneficiaries and intermediaries, in sample communities. MTE team members are expected to share and discuss ideas and observations whenever opportunities arise, such as when traveling together or during off-time at the beginning or end of each day. In addition, the MTE team will meet formally at the end of the second day of data collection in the district to share information that is relevant for investigations being conducted by other team members, specifically to be able to identify synergies and interaction between different interventions in the program. External Bangla translators will be recruited by TANGO to accompany members of the MTE team who do not speak Bangla. Team member notes will be taken by individual team members during the data collection process. Team members will also use photography as a data collection tool. E. Other Data Collection Because of time constraints and the locations of key informants, especially for some technical partners for the program who do not have a presence in Dhaka or the program areas, interviews will need to be organized remotely either by telephone or skype. F. Sampling and Participant Selection Since this is a Mid-Term Review, the sample of participants and sites for field visits will not be chosen randomly. These will be selected strategically, so that the MTE team can observe what is working and what is not working, as well as any particularly innovative approaches. The recommendations to be generated by the MTE will propose scaling up interventions that have good impact, modifying interventions to have more impact, suspending interventions that do not have enough impact relative to investment, piloting new interventions relevant for targeted impact groups, improving the effectiveness of implementation systems, or improving efficiency in use of resources. The MTE team needs to see the problems and challenges in order to be able to propose recommendations to address them. The MTE will collect data from six of the eight districts being covered by SHOUHARDO III, one district for each implementing partner. The program has in place a database of

SHOUHARDO III MTE Methodology Report 17 July 2018 participating households by village, union, and Upazilla in each district. To facilitate the selection of sites for field visits, the SHOUHARDO III management team classified unions by accessibility and then the villages in each union were classified according to performance. The villages that had participated in the SHOUHARDO I Program were also noted in order to ensure a representative sample of these villages were included in the MTE. Accessibility. Accessibility of each of the 115 unions participating in SHOUHARDO III was assessed in terms of the time and type of transport (vehicle, motorcycle, boat, donkey cart, or by foot) required to reach the union when accessibility is most difficult. Unions were classified as “remote access”, “moderate access” or “easy access”. Across the program as whole, 50% of the participating unions were classified as easily accessible, 20% as moderately accessible and 30% as being difficult to access. The logistical challenges are substantial in the Haor and Char Regions. For most remote and moderately accessible sites, multiple modes of transportation are required to reach the village from the district headquarters where the MTE team will be based, including traveling by car, boat and motorcycle. The difference between being classified as moderate or remote is mainly the amount of time required, e.g., 2 to 2 ½ hours for moderate versus 3+ hours one-way for remote sites. Even some of the easily accessible sites require two forms of transport, but the time required is an hour or less. Performance. It is difficult to assess performance relative to achieving impact in this early stage of a program’s life. Nevertheless, it is critical for the MTE to see sites in which the program is performing well as well as sites where the program may be having difficulty implementing program activities. The SHOUHARDO III Program Management Team worked with implementing partners to classify all villages as to how they were performing in terms of their participation in project activities. Villages classified as performing well had functioning program groups across almost all interventions. Villages classified as performing poorly were less active in implementing program interventions. The classification was done specifically for the MTE sampling and should in no way be perceived as a measure of the performance of the program. The exercise was done specifically for sampling for the MTE. Across the program as whole, this sampling frame showed 25% of villages classified as performing well and 25% classified as performing poorly at this stage. Participation in SHOUHARDO 1. Table 15 indicates that nearly 20% of the current SHOUHARDO III villages participated in the SHOUHARDO I Program. Table 15: Number of Villages in SHOUHARDO III Who Also Participated in SHOUHARDO I

SHOUHARDO I Village PNGO No Yes Total Char Region ESDO 49 21 70 MJSKS 161 27 188 NDP 117 23 140 SKS 162 24 186 Haor Region DAM 104 30 134 POPI 165 64 229

SHOUHARDO III MTE Methodology Report 17 July 2018

TOTAL 758 189 947

Selecting Sites for a Mid‐Term Evaluation Organizers of mid‐term evaluations are often reluctant to rely too much on program managers to select the sites for field visits in an evaluation. The perception is that program implementers will want the reviewers to produce the best view possible about the program. This, however, is not in the best interests of development, the program, or organizations implementing programs. Just as all programs are achieving at least some good impact, all programs also have implementation challenges and problems. Mid‐term evaluations are opportunities to fix these problems and address the challenges. Moreover, if the problems do not get addressed by the time of the final evaluation, final evaluators will find them and criticize the program for not addressing them. The program that has used the Mid‐Term Evaluation to improve its effectiveness and efficiency will ultimately have greater impact and be viewed more favorably in the final assessment.

Based on the sample frames developed, the MTE team purposively selected sites in collaboration with the SHOUHARDO III Program Management Team to ensure clarity on classification, representative balance on types of sites, a wide representation on interventions, and logistical considerations. Based on the preliminary field schedule that has been developed and included in Annex A, the MTE expects to conduct data collection in at least 28 sites allocated by district as shown in Table 16. The MTE team will plan to spend a half day in each of the selected sites, with each MTE team member conducting two to three data Table 16: Allocation of Sample Sites by District for the SHOUHARDO III MTE

Minimum Percent Targeted Percent PNGO District Sample of PEP HH of Total Sites Sample Char Region SKS Foundation (SKS) Gaibandha 39,506 22.6% 5 17.9% Eco-Social Development Jamalpur 18,326 10.5% 4 14.3% Organization (ESDO) Mahideb Jubo Somaj Kurigram 34,540 19.8% 5 17.9% Kallayam Somity (MJSKS) National Development Sirajganj 33,258 19.0% 5 17.9% Programme (NDP) TOTAL Char Region 125,630 72.0% 19 65.5% Haor Region Dhaka Ahsania Mission Habiganj 9,554 --- 11.3% 14.3% (DAM) Sunamganj 10,187 4 People's Oriented Kishoreganj 15,079 5 Program Implementation 16.8% 17.9% Netrakona 14,134 --- (POPI) TOTAL Haor Region 48,954 28.0% 9 34.5% TOTAL 174,584 100% 28 100% collection events (a data collection event is a key informant interview, a focus group discussion or a session to observe a program activity) in a specific site. Included in Annex E is the list of the 28 sites selected for MTE field data collection. The sample covers all 19 Upazillas in the six

SHOUHARDO III MTE Methodology Report 17 July 2018

districts, 28 Unions, and one village in each Union. For the villages, a little over half are classified as having easy access and the remaining half are divided nearly equally between moderate and remote accessibility. It should be noted that going to a single remote location will require an entire day for the team going there. In terms of performance, eight sites were classified as performing at a low level and nine sites were classified as performing well. The remaining eleven were considered as displaying average performance. Six sites were selected because they were locations where the SHOUHARDO I program had been implemented, these included sites from across all of the accessibility and performance clusters. MTE Team members will provide program management staff with a list of the meetings, interviews and FGDs that they would like to have arranged in each site, and the MTE team will work with program management staff to develop schedules for the field work. For interviews and FGDs in each site to be visited, program staff, volunteers and village leaders will identify individuals for interviews and FGDs to ensure that households selected represent the economic and geographic diversity of a village. In each district, the MTE team will hold at least one FGD with persons at the village-level who are not participating in the program in order to obtain information to verify program targeting and information to compare to the responses received from program participants on changes induced by the program. G. Data Analysis The information gathered by the MTE team will be analyzed at multiple points during the evaluation process. As the MTE team members are holding interviews and focus group discussions, they will probe and explore topics in more depth with respondents to ensure clear understanding. This represents the first level of analysis. A second level of analysis occurs when an evaluator cross-references responses from interviews and discussions with existing data bases and other secondary sources of information. In addition, the evaluator will also query program implementation staff for clarifications on information after field visits. The result of this level of analysis is that each evaluator will formulate his/her own key observations relative to the assigned topics. A third level of analyses occurs when information is shared and discussed between MTE team members. This occurs (a) at the MTE team meeting scheduled for the second day in each district, (b) through the sharing of observations between team members over the course of data collection, and (c) in discussions immediately before the Verification Workshop when the evaluation team is processing information individually and sharing information with other team members. The advantage of qualitative methods is to have the ability to conduct real time analysis of the information. As individual team members encounter information from other team members that they may not have had a chance to discuss with respondents, they will have the opportunity to do so when they go again to conduct interviews/discussions. A fourth level of analysis occurs in the Verification Workshop in which preliminary observations and findings are presented to program implementation staff. If program implementation staff seriously question a particular finding that has been presented by the evaluation team, the opportunity exists in the workshop and afterwards to further discuss and analyze the information to reach the truth.

SHOUHARDO III MTE Methodology Report 17 July 2018

Finally, a fifth level of analysis occurs after the MTE team has dispersed and begun assembling the content that will go into the draft report. Team members will have the opportunity to analyze the information that has been gathered in more depth as they are writing.

VIII. IMPLEMENTATION SCHEDULES The complete schedule for implementing the MTE is provided in Annex A. A preliminary schedule for site visits is also included in Annex A. A more detailed schedule for field visits and stakeholder meetings will be added to the protocol in February after the MTE Team Leader has worked with the SHOUHARDO III management team to develop the logistical plans. IX. CARE RESPONSIBILITIES CARE will ensure that the necessary information has been passed to the GoB to allow the team to implement the evaluation in the communities that will be selected. SHOUHARDO III program staff will serve as informants to the MTE and support the evaluation process by supplying lists of program sites, sharing program documents, advising about local protocols, making orientation presentations to the MTE team, arranging meetings with the stakeholders, and making logistical arrangements. In addition, CARE will ensure that program partners are informed of the process and that implementing partners will need to make staff available as possible for interviews and other consultations with the the MTE team. CARE will also facilitate in-country travel and logistical arrangements for the MTE team members including vehicles, printing/copying, access to data bases, work space for the MTE team, and venues for meetings and workshops. X. MTE SUMMARY REPORT The MTE Summary Report, not to exceed 50 pages excluding annexes, will:  Describe the evaluation methodology in detail with an explanation of any limitations encountered in the evaluation methodology and all tools used in conducting the evaluation, such as questionnaires, checklists and analysis summaries, included in an Annex,  Document MTE observations supported by qualitative evidence and available quantitative evidence,  Clearly articulate detailed recommendations supported by evidence with examples of how they can be operationalized. An illustrative format for the report is provided in Annex D

SHOUHARDO III MTE Methodology Report 17 July 2018

Annex A: MTE Schedules (Overall and Field Work) ACTIVITIES, RESPONSIBILITIES AND TIMELINE - December 2016 through June 2017

Responsible # Step Date(s) Comments Person (s) EVALUATION PREPARATION PHASE Initial Conference Call between CARE and Call organized by October 26 Initial call to introduce the Team Leader and CARE and begin 1 TANGO for planning TANGO (Thursday) discussions on organizing the MTE MTE Provision of initial CARE CoP & List of requested background documents sent to CoP by the background November 23 2 MTE Team MTE Team Leader and the drop box fully populated by the documentation required (Thursday) leader CoP to develop the protocol Development of zero This incomplete draft is for internal review by the TANGO MTE Team December 15 3 draft of Evaluation evaluation team and the management team for Leader (Friday) Protocol SHOUHARDO III. It will not be ready for wider circulation. Review background documentation and MTE Team January 10 Draft tools sent to the MTE team leader to be reviewed and 4 development of draft data Members (Wednesday) incorporated in the Evaluation Protocol collection tools The draft protocol will include the description of Completion of first draft MTE Team January 15 5 methodology, draft tools and preliminary operational plan for of Evaluation Protocol Leader (Monday the evaluation. Discussion of feedback on MTE Team January 19 Discussion on the draft protocol toward revising the 6 draft Evaluation Protocol - Members, CARE (Friday) document into a working protocol/operational plan Conference Call & FFP Completion of Working MTE Team January 30 Draft protocol revised, incorporating feedback received and 7 Draft Evaluation Protocol Leader (Tuesday) updated with more complete operational plan Categorization of February 9 8 participating communities CARE CoP Community and union lists sent to the MTE team members (Friday) and unions by criteria

SHOUHARDO III MTE Methodology Report 17 July 2018

specified in the Evaluation Protocol Evaluation Team MTE Team February 12 Team members coming from outside Bangladesh will arrive 9 Assembles at CARE Members (Monday) by February 11 Bangladesh Team members meet face-to-face for the first time to get MTE Team February 12 10 MTE Team Meeting acquainted and the MTE team leader answers any questions Leader (Monday, am) members have on the evaluation process MTE Team Preliminary selection of February 12 Field work sites will be selected jointly by MTE Team Leader 11 Leader with field work sites (Monday, pm) and CARE as per the Evaluation Protocol CARE CoP February 12 Orientation meeting with To clarify evaluation process and begin obtaining information. 12 CARE or 13 (Monday USAID/FFP Meeting attended by MTE team and CARE CoP. or Tuesday) DATA COLLECTION & ANALYSIS PHASE This is an orientation provided by SHOUHARDO III for the evaluation team to further clarify program activities, SHOUHARDO III February 13 stakeholders, best practices, and challenges. The orientation 13 CoP & Key Staff Orientation (Tuesday) will be followed by interviews with CARE SHOUHARDO III implementation staff conducted by individual MTE team members. February 14- Individual meetings by MTE Team members with Meetings with MTE Team 15 Implementing Partners, Technical Partners, Government 14 SHOUHARDO III Members (Wednesday & Stakeholders, Private Sector Stakeholders and others based Stakeholders in Dhaka Thursday) in Dhaka as specified in Evaluation Protocol Field visits to four Implementing Partners in four Districts (we should think about trying to do six). Two to three days February 16- Field Visits to districts MTE Team per district, with one day devoted to program orientation 15 March 3 selected for the MTE Members and District-based stakeholder interviews and one to two (16 days total) days for community visits per district. A more detailed schedule for the field work will be developed.

SHOUHARDO III MTE Methodology Report 17 July 2018

March 4 - 6 Information processing, and preparation of PPTs for Information processing MTE Team (Sunday Verification Workshop (Note that a draft workshop plan is 16 and preparation for Members through included in Annex C). Verification Workshop Tuesday) MTE Team with Presentation of preliminary findings and recommendations to March 7 & 8 SHOUHARDO SHOUHARDO III implementers for verification and 17 Verification Workshop (Wednesday & III Implementing refinement, identification of "Gray Areas" for further Thursday) Staff investigation. March 9 & 10 Finalization of key points to be put in the evaluation report Post-Workshop MTE Team 18 (Friday & and preparation of debriefings for SHOUHARDO III Processing Members Saturday) Stakeholders and USAID Bangladesh Half day workshop (must be scheduled for morning to be Stakeholder Debriefing in able to have time to prepare for USAID debriefing on the MTE Team March 11 19 Dhaka (Location in following day) with leadership of CARE implementing Members (Sunday) Dhaka?) partners, and technical partners to discuss main findings and recommendations A brief presentation (never more than two hours) of the De-Briefing with MTE Team March 12 20 MTE product provided to representatives of USAID/FFP USAID/FFP in Dhaka Leader (Monday) Bangladesh EVALUATION RECOMMENDATIONS FINALIZATION PHASE MTE team members continue analysis and development of MTE Team April 16 21 Draft Report Completed content for the Evaluation Report and MTE team leader Members (Friday) consolidates sections into final draft report CARE HQ & Sometime De-briefing to DC-based Evaluation final recommendations and findings presented to 22 MTE Team second half of Stakeholders FFP staff in Washington, DC Leader May MTE Team June 15 23 Final Report Submitted Draft incorporates feedback received in DC Leader (Friday)

SHOUHARDO III MTE Methodology Report 17 July 2018

DRAFT Schedule for Field Work - SHOUHARDO III MTE

EVENING DATE MORNING ACTIVITIES AFTERNOON ACTIVITIES ACTIVITIES (MTE Team) Friday, Intro Meeting and Brief orientation to Char Travel to Sirajganj Feb 16 Program by CARE regional Staff Travel to Belkuchi, then District-level orientation Field visits to two sites in Sirajganj – One Saturday, with NDP (Intro, schedule review, staff interviews, group will go to one site and the other group Feb 17 plenary presentation by NDP) will go to another site Sunday, Field visits to three sites - One group will go to two sites and the other group will take the whole MTE team meeting Feb 18 day to go to one remote site. (Need a venue) Team A travel to Gaibandha Meetings with Sirajganj district-based stakeholders District-level orientation with SKS (Intro, Monday, from GoB, technical partners and private sector schedule review, staff interviews, plenary Feb 19 (schedule to be developed by NDP & CARE) presentation by SKS) Team B travel to Jamalpur Team A: Field visit to one site in Gaibandha Team A: Field visit to one site in Gaibandha Tuesday, Team B: District-level orientation with ESDO Feb 20 (Intro, schedule review, staff interviews, plenary Team B: Field visit to one site in Jamalpur presentation by ESDO) Team A: MTE team Wednesday, Team A: Field visit to one site in Gaibandha Team A: Field visit to one site in Gaibandha meeting Feb 21 Team B: MTE team (HOLIDAY) Team B: Field visit to one site in Jamalpur Team B: Field visit to one site in Jamalpur meeting Team A: Meetings with Gaibandha district-based Team A: Field visit to one site in Gaibandha stakeholders from GoB, technical partners and private sector (schedule to be developed by SKS & Thursday, CARE) Feb 22 Team B: Meetings with Jamalpur district-based stakeholders from GoB, technical partners and Team B: Field visit to one site in Jamalpur private sector (schedule to be developed by ESDO & CARE)

SHOUHARDO III MTE Methodology Report 17 July 2018

District-level orientation with MJSKS (Intro, Friday, Both teams travel to Kurigram schedule review, staff interviews, plenary Feb 23 presentation by MJSKS) Saturday, MTE team meeting Field visits to two sites in Kurigram Field visits to two sites in Kurigram Feb 24 (Need a venue) Meetings with Kurigram district-based stakeholders Sunday, from GoB, technical partners and private sector Field visits to two sites in Kurigram Feb 25 (schedule to be developed by MJSKS & CARE) Monday, Intro Meeting and Brief orientation to Char Fly to Dhaka and travel to Kishoreganj by road Feb 26 Program by CARE regional Staff District-level orientation with POPI (Introductions, Tuesday, schedule review, staff interviews, plenary Field visits to two sites in Kishoreganj Feb 27 presentation by POPI) Meetings with Kishoreganj district-based Wednesday, stakeholders from the GoB, technical partners and MTE team meeting Field visits to two sites in Kishoreganj Feb 28 private sector (schedule to be developed by POPI & (need a venue) CARE) Travel to Sunamganj Thursday, District-level orientation with DAM Field visits to two sites in Kishoreganj March 1 (Introductions, schedule review, staff interviews, plenary presentation by DAM) Friday, MTE team meeting Field visits to two sites in Sunamganj Field visits to two sites in Sunamganj March 2 (need a venue) Meetings with Sunamganj district-based Saturday, stakeholders from GoB, technical partners and MTE travels to Sylhet and then travels by air March 3 private sector (schedule to be developed by DAM & from Sylhet to Dhaka CARE)

Site Summary: Char ‐ 19 sites total with 5 sites in each of Sirajganj, Kurigram, and Gaibandha and 4 sites in Jamalpur Haor ‐ 19 sites total with 5 sites in Kishoreganj and 4 sites in Sunamganj 28 sites in Total

SHOUHARDO III MTE Methodology Report 17 July 2018

ANNEX B: DATA COLLECTION TOOLS

SHOUHARDO III Mid-term Evaluation Sustainability (All Team Members) TOPICAL OUTLINE

The sustainability of the effects and impact being achieved by the SHOUHARDO III Program is an area of investigation that will be covered by each team member who is investigating the effects and impact being achieved by the program for specific purposes and sub-purpose. The following questions are intended to guide interviews and discussions around analyzing sustainability when the discussions/interviews reach that point.

1. For the changes that you have described as having been facilitated by the program, how permanent are the changes? 2. What resources are required to sustain the changes? Where do those resources come from now? Where will they come from after the SHOUHARDO III Program has ended? 3. What relationships, such as for technical support, inputs, marketing, social capital or political capital, are required to sustain the changes? What role has the SHOUHARDO III Program had in developing or facilitating these relationships? After the program ends, how do you think these relationships will change? 4. How happy are beneficiaries and intermediaries with the changes? How motivated will they be to continue to maintain or support the changes? 5. What are the biggest threats to sustaining the changes induced by the SHOUHARDO III Program? How can these threats be addressed?

======TOPICAL OUTLINE PURPOSE 1: INCOME & FOOD Sub-Purposes 1.1: Farm Production, 1.3: Off-Farm Income & 1.4: Financial Services (Mike)

Purpose 1: Increased equitable access to income for both women and men, and nutritious food for men, women, boys and girls Sub-Purpose 1.1: Increased agricultural production for poor or extreme poor households Sub-Purpose 1.3: Increased off-farm income for poor or extreme poor households Sub-Purpose 1.4: Increased utilization of financial services by poor or extreme poor households The purpose of the investigations using the following questions is to obtain information to be able to formulate recommendations for the remaining life of the program. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) or piloting new interventions relevant for targeted impact groups. The questions are designed to be able to: (1) Assess the quality of the outputs produced under this purpose

SHOUHARDO III MTE Methodology Report 17 July 2018

(2) Identify the outcomes and impact being generated by these outputs (3) Assess the quality of the approaches used by the program (4) Judge the potential sustainability of the impact that has been achieved

Information will be gathered through key informant interviews, focus group discussions or large group discussions with program participants, interviews with program implementation staff and program managers, and observation of program activities and outputs.

TOPICS TO BE COVERED WITH COMMUNITY-LEVEL PARTICIPANTS The following persons or groups at the community-level will be engaged in interviews or focus group discussions:

Farmer Field and Business Schools Community Agriculture Volunteers Youth Participants in Life Skills and Employability Trainings Village Savings and Loan Associations Participating Micro-Finance Institutions or Banks

Following are the major activities being undertaken under Sub-Purposes 1.1, 1.3 & 1.4. These will be discussed with participants as relevant for the respondent.

 Capacity building of farmers on improved production and technology (improved modern poultry and livestock production, crop production, culture and capture fisheries) and climate change adaptation approaches (floating gardens, compost preparation, and crop rotation)  Capacity building of PEP participants on entrepreneurship and basic business skills (for farm production and off-farm IGAs)  Capacity building of youth through mentorships, apprenticeships or internships  Development of employment and placement support services for youth  Capacity building on life skills for PEP participants  Capacity building of PEP households on savings, lending and financial management skills  Formation of savings associations (VSLAs) of men, women and youth  PEP savings groups linked to MFIs and banks  Advocacy action for pro-poor financial support  Support for MFIs and banks to conduct feasibility studies on PEP financial products  Construction of raised homesteads in conjunction with promotion of Comprehensive Homestead Production (CHP) systems  Provision of micro-enterprise matching grants for agriculture and off-farm opportunities The general sequencing of questions is as follows: 1. Are you familiar with the SHOUHARDO III Program? 2. How would you describe what this program seeks to accomplish? 3. How have you or other members of your household participated in this program? For how long have you/they been involved?

SHOUHARDO III MTE Methodology Report 17 July 2018

4. Please describe how you or members of your family have benefitted from the program. How sustainable are these benefits? 5. Please describe how you or members of your family have been negatively affected by the program. 6. Who, in your opinion, has benefitted most from the program? 7. Are there other people who should be benefitting from the program but are not? Please describe them for us. 8. What constraints do you believe inhibit the program from fully accomplishing its purposes? 9. What suggestions do you have for addressing these constraints or otherwise enabling the program to have greater impact? TOPICS TO BE COVERED WITH PROGRAM MANAGERS, IMPLEMENTATION STAFF AND GOB STAKEHOLDERS

The following managers, staff or GoB stakeholders associated with Purpose 1 will be interviewed:

CARE Senior Technical Coordinator (STC) for Agriculture & Livelihoods (Dhaka) CARE Technical Manager (TM) for Agriculture & Livelihoods (Kishoreganj & Sirajganj) PNGO Technical Officers for Agriculture & Livelihoods (1 in each District) PNGO Field Supervisors (I in each Union included in MTE Sample) Ministry of Agriculture Representatives at the district-level Ministry of Fisheries & Livestock Representatives at the district-level Ministry of Youth & Sports Representatives at the district-level

The general sequencing of questions is as follows:

1. Explain your understanding of the strategy for the SHOUHARDO III Program, i.e., how is the program expected to achieve Purpose 1? 2. What are the main activities being implemented (check against the list provided above)? 3. What is working well and why? 4. What is not working well and why? How are the challenges being addressed? 5. Who, in your opinion, has benefitted most from the program? 6. Are there other people who should be benefitting from the program but are not? Please describe them for us. 7. What have been the major changes that you have observed with poor or extreme poor households as a result of the program? What is the likelihood that these changes remain after the program ends? 8. Where changes are slow or not evident, what are the obstacles to change? 9. What changes in the way the program is being implemented would make it more effective in achieving its outputs and outcomes? ======

SHOUHARDO III MTE Methodology Report 17 July 2018

Sub-Purpose 1.2: Marketing (Mark)

Purpose 1: Increased equitable access to income for both women and men, and nutritious food for men, women, boys and girls Sub-Purpose 1.2: Increased access to agricultural markets for poor and extreme poor households

The purpose of the investigations using the following questions is to obtain information to be able to formulate recommendations for the remaining life of the program. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) or piloting new interventions relevant for targeted impact groups. The questions are designed to be able to: (1) Assess the quality of the outputs produced under this purpose (2) Identify the outcomes and impact being generated by these outputs (3) Assess the quality of the approaches used by the program (4) Judge the potential sustainability of the impact that has been achieved

TOPICS TO BE COVERED WITH COMMUNITY-LEVEL PARTICIPANTS The following persons or groups at the community-level will be engaged in interviews or focus group discussions:

Farmer Field and Business Schools Community Agriculture Volunteers Micro-seed dealers Micro-seed sellers Local and regional entrepreneurs/buyers/collectors Participating suppliers of market information for IGAs

Following are the major activities being undertaken under Sub-Purposes 1.1, 1.3 & 1.4. These will be discussed with participants as relevant for the respondent.

 Seed supply through micro-seed dealers and seed sellers  Development of an integrated system of local and regional entrepreneurs/buyers/collectors  Development of linkages to affordable transportation services (After farm production levels increased)  Research conducted on business barriers for PEP participants and opportunities for input suppliers to reach PEP participants  Expanded availability of market information for IGA participants

SHOUHARDO III MTE Methodology Report 17 July 2018

 Cultivation of business relations through district-level agricultural fairs, buyer visits, and shared-value business models between PEP farm households and private sector partners The general sequencing of questions is as follows: A. Members of FFBS, CAV 1. Are you familiar with the SHOUHARDO III Program? 2. How would you describe what this program seeks to accomplish? 3. Please describe your activities under the project. 4. What have been the challenges you have faced in carrying out your activities for the project? 5. Who, in your opinion, has benefitted most from the program? 6. Are there other people who should be benefitting from the program but are not? Please describe them for us. 7. To what extent do women/adolescents participate in your activities? 8. Do you follow any strategy to target your activities toward women/adolescents? If so, what are these strategies 9. What constraints do you believe inhibit the program from fully accomplishing its purposes? 10. What suggestions do you have for addressing these constraints or otherwise enabling the program to have greater impact? 11. Do have any suggestions about how the activities that you carry out can by maintained in the future, after the end of the project?

B. Seed suppliers (company distributers, local micro-suppliers) and commodity buyers (local maize traders, feed producers, local vegetable traders) 1. Please briefly describe your current activities. 2. How has your business changed in the past 5 years? 3. Have you had any interaction with SHOUHARDO project? If so, please describe this interaction, and how it has affected your business activities. 4. Who are your major clients? Do you sell to very small farmers? What challenges do you face to sell to very small farmers? 5. Do you sell to women? If not, why not? 6. What do you see as major challenges and opportunities to expand your business in the future?

TOPICS TO BE COVERED WITH PROGRAM MANAGERS AND IMPLEMENTATION STAFF The following managers or staff associated with Purpose 1 will be interviewed: CARE Senior Technical Coordinator (STC) for Private Sector Engagement (Dhaka) CARE Technical Manager (TM) for Private Sector Engagement (Kishoreganj & Sirajganj) PNGO Technical Officers for Agriculture & Livelihoods (1 in each District) PNGO Field Supervisors (I in each Union included in MTE Sample) Ministry of Agriculture Representatives at the central and district-levels Ministry of Fisheries & Livestock Representatives at the central and district-levels The general sequencing of questions is as follows:

SHOUHARDO III MTE Methodology Report 17 July 2018

======

PURPOSE 2: MCHN SHOUHARDO III MTE Purpose 2: Improved Health and Nutrition of Pregnant and Lactating Women and Children under Five (Rukhsana) TOPICAL OUTLINE The purpose of the mid-term evaluation using the following questions is to obtain information to ascertain how the program is progressing so as to be able to formulate recommendations for the remaining program period. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) or piloting new interventions relevant for targeted impact groups. The questions are designed to be able to: (1) Assess the quality of the outputs produced under this purpose (2) Identify the outcomes and impact being generated by these outputs (3) Assess the quality of the approaches used by the program (4) Judge the potential sustainability of the impact that has been achieved

Information will be gathered through key informant interviews, focus group discussions or large group discussions with program participants, program implementation staff, program managers and observation of program activities and outputs.

For Purpose 2, the following persons or groups will be engaged in interviews or focus group discussions with representatives from each of these different types of participants and visit a sample of sites.  Community health volunteers (CHVs)  Community support groups for health and nutrition  Participants who attended courtyard sessions (for breastfeeding, dietary diversity, good WASH practices, improved household latrines) 86

SHOUHARDO III MTE Methodology Report 17 July 2018

 Lactating mothers for breastfeeding, cooking and feeding demonstrations  Food ration recipients  Village leaders (awareness regarding intra-household food distribution, improved household latrine demonstrations, modified/hybrid)  Community-based water management committees  Relevant MOH &FW staff – CHCP, HA, FWA, FWV, CS, DDFP, UFHPO  VDCs and Health Committees implementing health advocacy campaigns  Site Visits/Observations: 1) GMP/ BCC session 2); MCHN Ration Post-Distribution activity (HH visit) (2): Community Health Center /Community clinic visits (2); Latrine, Hand Washing Facilities and Water Point visits  Implementing Partner NGOs – Upazilla Managers of SKS Foundation, Eco-Social Development Organization (ESDO), Mahideb Jubo Somaj Kalyan Somity (MJKSS), National Development Programme (NDP), Dhaka Ahsania Mission (DAM), People’s Oriented Program Implementation (POPI)  Program implementing staff: Purpose 2 Coordinator, Sector Technicians (WASH and MCHN)  Program Technical staff: MCHN Advisor and Capacity Strengthening Advisor

PROGRAM BENEFICIARIES FROM TARGETED IMPACT GROUPS

Pregnant and Lactating mothers Community Support Group participants Grandmothers group (?) Adolescent girls group Fathers/ Mens’ group Growth Monitoring Session (GMP) participants

1. Are you familiar with CARE’s SHOUHARDO III Program? (Describe the program, if the name is not familiar) 2. Since you have participated in this program, please tell me what the program seeks to accomplish? 3. How have you participated in this program? How were you selected? For how long have you been involved? 4. Please describe how you or your family have benefitted from the program. 5. Please describe how you or your family have been negatively affected by the program. 6. Who, in your opinion, has benefitted most from the program? 7. Are there other people who should be benefitting from the program but are not? Please describe them for us. 8. What constraints do you believe inhibit the program from fully accomplishing its purposes? 9. What suggestions do you have for addressing these constraints or otherwise enabling the program to have greater impact? 10. What else would you like the reviewers to know about the program? PROGRAM INTERMEDIARIES (Local MoH officials – Civil Surgeon, DDFP staff, Community Health Care Provider) 87

SHOUHARDO III MTE Methodology Report 17 July 2018

Activities Community Group and Community Support Group Promoters and Lead Parents SBCC caregivers – grandmothers, fathers Village Development Committee, UDC

1. Are you familiar with the SHOUHARDO III Program? (Describe the program, if the name is not familiar) 2. How would you describe what this program seeks to accomplish? 3. How have you or your organization participated in this program? For how long have you been involved? 4. What has changed as a result of program activities? 5. How have people benefitted with respect to health and nutrition because of these changes? Which kind of people have benefitted most from the program? 6. To what extent have you perceived changes in the people in your area as a result of the program activities? 7. Are there other people who should be benefitting from the program but are not? Please describe them for us. 8. In terms of what the program has done with you, what has been most useful? What has been least useful? 9. What constraints do you believe inhibit the program from fully accomplishing its purposes? 10. What suggestions do you have for addressing these constraints or otherwise enabling the program to have greater impact? 11. What else would you like the reviewers to know about the program? PROGRAM TECHNICAL AND IMPLEMENTATION STAFF CARE: STC-HHN, Advisor Environment, technical Managers HHN, Technical Managers SBCC, Regional Senior Technical Manager, Infrastructure & Environment Officer (I&EO)

Partner NGOs: Technical Officer, HHN, APM – Technical Field Supervisor, Field Facilitator 1. How would you describe what this program seeks to accomplish? How does what you do appear in the program’s Theory of Change? 2. Please list for us the different types of activities that you have implemented as part of your job in the program? 3. Which of these activities has been most effective in achieving change on health and nutrition for targeted impact groups? 4. Which of these activities has been least effective in achieving change on health and nutrition for targeted impact groups? 5. Who, in your opinion, has benefitted most from the program under Purpose 2? 6. Are there other people who should be benefitting from the program but are not? Please describe them 7. In terms of improving health and nutrition, what other activities could the program undertake? How important are these relative to what the program is already doing, i.e.,

SHOUHARDO III MTE Methodology Report 17 July 2018

the program budget is limited, so if we want to do something new we have to stop doing something that we are already doing in order to release resources? 8. What constraints do you face in doing your job? What suggestions do you have for addressing these constraints? 9. What else would you like the reviewers to know about the program? SITE VISITS: Community Clinic (any other?)

Community Group/ Community Support Group BCC activity MCHN Ration Post-Distribution activity Homestead Gardening Activity GMP session Training/ Refresher training Farmer Field Business School (FFBS) – under P1, but includes 6 Topics for Nutrition & WASH Referral system – (fieldCCsecondary/tertiary level health facility Latrine, Hand Washing Facilities and Water Points Village Development Committee

1. Are you in charge of the clinic/ FFBS? 2. How many years have you been qualified? 3. How long have you been at this clinic/facility? 4. How long have you been in your current position? 5. Are any other organizations providing support to this facility / district? a) Who are they and what support are they providing? 6. Are you familiar with the SHOUHARDO III program? (Describe the program, if not familiar) 7. How would you describe what the program seeks to accomplish? 8. How have you or your organization participated in this program? For how long have you been involved? 9. Did you participate in any of the following activities with SHOUHARDO III? a) (Baseline) assessment b) Gender assessment c) Development of action plans d) Training of Community volunteers/ Field and Farmer Business School e) Self-assessment f) Supportive Supervision g) Development of tools or activities 10. Have the below subject areas been supported (at any time) by SHOUHARDO III in your area of responsibility? a) Maternal, child and neonatal health human resources b) Health education c) Antenatal care d) Normal labor & delivery and essential newborn care e) Postnatal care

SHOUHARDO III MTE Methodology Report 17 July 2018

f) Emergency obstetric care g) Emergency neonatal care h) Kangaroo mother care i) Maternal or neonate death audit j) Infection prevention k) Immunization l) IMNCI for 0 – 2 months m) IMNCI for 2 months – 5 years n) Maternal and child nutrition o) Youth Friendly Health Services p) Referrals and community follow up 11. How have people benefitted with respect to health and nutrition because of these changes? Which kind of people have benefitted most from the program? 12. To what extent have you perceived client level changes as a result of their participation in SHOUHARDO III activities? ======Village Development Committee (in P5) Focus Group Discussion 1. What do you know about SHOUHARDO III as a program in your area? How do you work with the program/organization implementing the program? 2. What are the main issues relating to water and sanitation that concern you in your area? 3. Where do you present your water and sanitation concerns to when you have them? 4. How responsive to your water needs are the ones you take your concerns to? 5. Are there any measures that you have put in place to ensure that water sources are protected from pollution in your area? 6. Are you involved in discussions concerning water supply in your area/ District as a whole? 7. How are you involved? 8. What specific support do you get from local government for your water points in your area? 9. What improvement would you want to see made in the way the District Council and other actors working in water supply service delivery provide support to you? 10. Which other organizations are working in your area in water supply service delivery? 11. How do you ensure that water points in your area remain functional? How often are they checked? 12. On the existing boreholes and other improved community water sources in your area, how were you involved (a) in selecting the site (b) in choosing the type of technology being used? ======Key Informant Interviews Director/Environment Adviser (covers WASH) 1. What do you know about SHOUHARDO III as a program in your area? How do you work with the program/organization implementing the program? 2. How many organizations in the district are providing WASH services?

SHOUHARDO III MTE Methodology Report 17 July 2018

3. Duplication of efforts are bound to happen where there are no coordination mechanisms. How do you coordinate all the agencies and government departments implementing WASH interventions in your district? 4. What coordination challenges do you face in your district 5. What compliance measures have you put in place to ensure that WASH sector actors comply with government policies, regulations and guidelines? 6. How does water feature in your socio-economic profile in terms of priority rating? 7. How does water financing compare with other sectors in the District? 8. Does the District have a WASH planning and review forum? 9. If yes, how are community members involved? 10. Who else participates in these forums? 11. How is the implementation of WASH monitored in this district? 12. Considering that there are many government agencies and NGO partners, who leads monitoring of WASH in the district? 13. How could programs like SHOUHARDO III help improve WASH service delivery at different levels in your district, especially in the Chars and Haors? ======Key Informant Interviews Department of Public Health Engineering 1. What do you know about SHOUHARDO III as a program in your district? How do you work with the program/organization implementing the program? 2. What are the WASH challenges that you face in your district? 3. Is your district running any Community-led Total Sanitation (CLTS) programs? 4. If yes, where are these being implemented? Who is implementing them? 5. What success has CLTS programs registered in the District? (Number of villages triggered for CLTS, Verified and Declared ODF). 6. How are the programs implemented to ensure sustainability of CLTS gains? 7. What are the general challenges that CLTS programs are facing in the districts? 8. What are CLTS management challenges that the District faces? 9. What can programs such as SHOUHARDO III that are not directly implementing CLTS do to improve performance and sustainability of CLTS in your district? 10. Do you have a WASH database? How do you update it? 11. Who manages it and how are sectors involved in data generation? 12. Does the District Council have a water quality monitoring team? Who are members? 13. What water quality monitoring programs does the District Council run in the district? 14. What are the regimes and testing kits does the district use? 15. Can you give an example of how the District Council ensures that water quality testing is done on all water points before they are handed over to communities? 16. What are some of the challenges that you face in this regard? 17. Do you take water quality results back to communities for their information? 18. In places where the quality of water is beyond human consumption, how 19. When did the district last carry out the last round of water quality testing? 20. What can programs such as SHOUHARDO III that are not directly implementing CLTS do to improve performance and sustainability of CLTS in your district? ======91

SHOUHARDO III MTE Methodology Report 17 July 2018

PURPOSE 3: DISASTER PREPAREDNESS AND RESPONSE (Mark) TOPICAL OUTLINE Purpose 3: Strengthened gender equitable ability of people, households, communities, and systems to mitigate, adapt to and recover from man-made and natural shocks. Sub-Purpose 3.1: Increased preparedness of poor and extreme poor households and communities to mitigate and respond to shocks Sub-Purpose 3.2: Local government institutions' (Union Parishad) capacity and implementation of Disaster and Climate Risk Management (DCRM) activities increased

The purpose of the investigation using the following questions is to obtain information to be able to formulate recommendations for the remaining life of the program. These recommendations will be oriented around (a) scaling up effective interventions, (b) modifying interventions to improve effectiveness, (c) suspending interventions that are not effective enough relative to investment, (d) or piloting new interventions relevant for targeted impact groups. The questions are designed to be able to: (1) Assess the quality of the outputs produced by the program under this purpose (2) Identify the outcomes and impact being generated by these outputs (3) Assess the quality of the approaches used by the program (4) Judge the potential sustainability of the impact that has been achieved

The following intermediaries related to Purpose 3 will be interviewed directly or through a focus group discussion:

Union Information and Service Centers (UISCs) Community-based Disaster Volunteers Village Development Committees (for DCRM capacity building, community action plans, resource mobilization, and lobbying) CBO actors Union Disaster Management Committees (for DCRM capacity building and lobbying)

The major activities undertaken by the program under Purpose 3 include:

 Advocacy and capacity building for Union Information and Service Centers (UISC) to access early warning and weather forecasts  Capacity building of Disaster Volunteers on Disaster and Climate Risk Mitigation (DCRM) and dissemination of early warning messages  Community-level capacity building for improved disaster risk reduction strategies through implementation of Climate Risk Analysis (CRA), training on emergency contingency planning, training on disaster response, and use of disaster simulations  Support development of emergency contingency plans using a Community Risk Assessment process at the households, community, UP, Upazilla and District levels

SHOUHARDO III MTE Methodology Report 17 July 2018

 Promotion and construction of hazard-resilient disaster mitigation infrastructure (schools as flood shelters, CRCs, protection walls and drainage structures) through Labor Contracting Societies and Joint Contracting Systems  Promote use of energy-efficient stoves  Support for VDCs to advocate for increased funding for DCRM  Community capacity building to lobby for DCRM at the union level  Capacity building of Union Disaster Management Committees (UDMCs) and development of Union Disaster Management Plans (UDMPs) using key DCRM concepts  Facilitate the UDMC to engage Union Disaster Volunteers and pilot School-Based Teenage Brigades  Union capacity building to lobby at the upazilla and district levels for increased DCRM funding TOPICS TO BE COVERED WITH PROGRAM PARTICIPANTS General background on shocks and impacts 1. Please describe any concerns you may have about changes in environmental hygiene or natural resource degradation) over the past 5 years 2. What changes have you noticed in the weather/climate that affects your activities (water wells running dry? Floods more intense/frequent?) over the past 5 years 3. Who in your community is affected by these changes/shocks and how are they affected? Are different groups affected differently? 4. Have there been any changes in how your community has prepared for or responded to these changes/shocks over the last 5 years? What changes? SHOUHARDO 3 activities 5. Are you aware of any activities by the SHOUHARDO III program to help your community prepare for or respond to these shocks/changes? 6. How have you participated in the program? For how long have you been involved? 7. Please describe what has changed as a result of the SHOUHARDO III Program, e.g., how have people benefitted, what changes have occurred in policies or procedures for DRR or CCA, and so on. How lasting are these changes, i.e., are they temporary or permanent? 8. Please describe any negative changes that have resulted because of SHOUHARDO III. How would you suggest these negative changes be addressed? 9. What support have you received from the program related to adapting to environmental impact? Coverage and targeting of impacts 10. Have project activities provided effective support to all groups in the community (PEP, women, adolescents/youth)? 11. Are there other people who should be benefitting from the program but are not? Please describe them for us. Challenges of implementation/sustainability 12. What suggestions do you have for changing the program activities to have greater impact? 13. What do you expect will happen to support for disaster preparedness and response after the project ends? 14. What else would you like the reviewers to know about the program?

SHOUHARDO III MTE Methodology Report 17 July 2018

TOPICS TO BE COVERED WITH PROGRAM TECHNICAL AND IMPLEMENTATION STAFF General background 1. How would you describe what this program seeks to accomplish? How do your activities within the project contribute to the overall goals of the project? 2. Please list for us all of the different types of activities that you have implemented as part of your job in the program? 1. Which of these activities has been most effective in achieving change and resilience to shocks and stresses for targeted groups in the community (the most vulnerable)? 2. What do you know about the program's EMMP? 3. What are your responsibilities relative to the EMMP? Coverage and targeting 4. Do the activities under Purpose 3 provide effective service and coverage to all community members? 5. Who, in your opinion, has benefitted most from the program under Purpose 3? 6. Have the activities under Purpose 3 effectively addressed the needs of women, how they are affected by different kinds of shocks, and limitations they may face in mitigation or recovery? Any improvements in implementation strategy that could more effectively address the needs of women? 7. How are women engaging in disaster preparedness and responses planning and preparation? What is the project doing to engage women in this process? Do you have any suggestions for how to engage women more effectively? 8. How are adolescents and youth engaging in disaster preparedness and responses planning and preparation? What is the project doing to engage adolescents and youth in this process? Do you have any suggestions for how to engage adolescents and youth more effectively? 9. YOUTH TARGETING: which activities have specifically targeted youth under Purpose 3? 10. Are there other people or groups in the community who should be benefitting from the program but are not? Please describe them 11. What do you consider the greatest challenges to enhancing disaster preparedness and response to support all community members? 12. What constraints do you face in doing your job? What suggestions do you have for addressing these constraints? 13. How have youth been engaged in these activities? Sustainability 14. What challenges to you see in maintaining support to disaster preparedness and response activities after the end of the project? 15. Do you have any suggestions on how best to sustain this support after the project ends? 16. What else would you like the reviewers to know about the program?

======

SHOUHARDO III Mid-term Evaluation 94

SHOUHARDO III MTE Methodology Report 17 July 2018

PURPOSE 4: Women's Empowerment and Gender Equity TOPICAL OUTLINE

Persons/groups to interviews for Purpose 4: 1. EKATA Groups 2. Men from households represented in EKATA Groups 3. Male/female participants of Couples Workshop 4. Community Empowerment Volunteers 5. VDCs (for GBV and women's empowerment action plans) 6. Representatives from Local Elected Bodies (LEB) and Nation Building Departments (NBDs – Agriculture Dept., Dept of Livestock, Dept of Health) for knowledge and action on GBV and women's empowerment 7. Union Parishad Nari Nirjaton Protirodh Committees (NNPC – Ending Violence Against Women) 8. District Women’s Affairs Officers 9. FFBS groups for CHD, possibly agriculture (goats)

Introduction  Who we are; why we are here; how long the process will take; What will be done with the results of our work  Participation is voluntary/ can leave at any time/ don’t have to answer anything they don’t want to. Gain consent to interview. FGD: Beneficiaries

TOPICAL OUTLINE Topical Outline for FGDs with EKATA Group Participants, Community Women’s Empowerment Volunteers (CWEV), and with male/female participants of Couples Workshop

Introduction/consent:  Who we are; why we are here; how long the process will take; what will be done with the results of our work.  Participation is voluntary; can leave at any time; identifying information will not be connected to responses; don’t have to answer anything they don’t want to. Gain consent to interview. Project participation: 1) What activities from the SHOUHARDO3 project have you participated in? (Probe to understand which inventions women and girls may have received – especially FFBS and value chains)

2) What activities have other family members participated in?

 How long have you been participating?  Do you or any of your family members belong to an EKATA group?

SHOUHARDO III MTE Methodology Report 17 July 2018

Group function and capacity: (for ekata and empowerment volunteer groups) 3) How long has the group been functioning? # of members? Are certain types of women and girls more likely to join than others? Explain. Trends in group number. Reasons for attrition.

4) Describe any benefits of belonging to the (EKATA or empowerment volunteer) group? Any challenges to belonging? Has this changed over the last year/when the group started?

5) What influenced you to be an EKATA member/or empowerment volunteer? What issues do women face? What issues do men face?

 What messages do you try to deliver? How?  What have been your greatest successes so far? Greatest challenges?

6) What training have you received from the project to support your work with the EKATA/Volunteers? On what topics? How have you applied the topics of the training (give examples)?

Household decision-making 7) Have there been any changes to how you make decisions in your household since participating in the project? Note the following probes:  Who makes decisions in your household about what… Probe specifically for which decisions are considered important regarding: o Nutrition: food purchases, meal choices, if boys and girls receive equal portions at meals; o Healthcare: taking a sick child to the health clinic, seeking family planning services; o Income: how to spend the income earned by women, or saved through the VSLA? o Has it always been this way?  Any differences in the types of important decisions that women or men have been making since the SHOUHARDO III Program began? Probe for differences in “important” decisions for men and for women.  In your opinion who in a household should contribute to making decisions about crops and livestock? Other income-earning activities? Who actually makes these decisions in the households? Has it always been this way? Other income-earning activities? Probe for reasons/details especially differences between women and men.  What changes have occurred during the past year? Probe for details.  Are there certain types of households where women have a very strong influence in decision-making? Types of households where women have very little influence? Probe for details.

SHOUHARDO III MTE Methodology Report 17 July 2018

Gendered division of labor 8) Since you or your family began participating in the project, have your household responsibilities changed (for you personally)? (Childcare (in particular), domestic work, farming, income generation, and other activities). How? Why?

 If so, how has it affected your relationship with your spouse (positive or negative)? With your mother-in-law or other members in the household?)  Has it affected your relationship to other community members who have observed you sharing household responsibilities? (Note: This is a particularly important probe for the male champions.) Control over productive assets and resources 9) Since you or your family began participating in the project, has there been any changes in women’s income? Explain. 10) If yes, what is that income used for? Who makes the final decision over how that income is used? 11) Are you a member of a VSLA? What were the reasons for joining? 12) What are the benefits? Any problems with VLSA membership (either internal or with own HH) 13) Any changes in women’s ownership of high-value productive assets? [See more cross-cutting questions related to livelihoods/income at end of this document.]

Control over one’s body & violence and restorative justice 14) How common is marriage of girls under 18 years of age in this community? Do community members speak up publically about early marriage of girls? About girls’ education? Who speaks up? Why/why not?

15) What actions can/have people taken to prevent early marriage? Any changes on issues of early marriage/girls education in the last year?

16) How do you define violence against women (Probe also for psychological, economic violence)? How common is violence against women in this community? 17) Are there times when you believe it is appropriate to use violence in the household? Under what circumstances? Is it acceptable for men to hit their wives? Daughters? For women to hit husbands? Daughters? Other HH members to hit females? 18) Has there been any change on GBV since the beginning of the project? How? Why? 19) Do men speak up publically about gender-based violence? Why/ why not? What type of men? Changes in the last year? Do women? Why/ why not? Changes in last year?

SHOUHARDO III MTE Methodology Report 17 July 2018

 Do you know of any services in your community to support a women facing gender- based violence? Who/What do they offer? Access to public spaces and services (including mobility) 20) How common is it in this community for men to accompany their wives to a hospital or clinic?

21) To public places such as a religious place, meetings, social celebrations, the market?? Why/ why not? Other places? Has this changed in the past year? How? Why?

22) What places are women able to go alone? Has this changed in the past year? How? Why? Meaningful public participation and governance processes 23) In what types of settings (i.e., contexts) are women/men comfortable speaking up? What subjects do people speak out about relating to community decisions/challenges? Differences between men and women speaking out?

24) Are there women in the community who have an easier time speaking up about public issues? What factors help them to speak up in public? What subjects do they speak out about relating to community decisions/challenges? Has this changed in the past year? How? Why? 25) Has women holding leadership positions within the groups and community changed over the last year? How? Are they involved with decision-making for the group? (E.g., holding leadership positions in the VSLAs or VDCs)? Probe for meaningful participation in leadership positions

Aspirations for oneself 26) What is your definition of “women’s empowerment” – describe a woman whom you would consider to be “empowered” in this community. Are women more empowered now than a year ago? How and in what form? (E.g., leadership, confidence, assertiveness, etc.) Probe to understand why some women are not empowered.

sustainability 27) For the changes that you have described as having been facilitated by the program, how permanent are the changes?

28) What resources are required to sustain the changes? Where do those resources come from now? Where will they come from after the SHOUHARDO III Program has ended?

29) What relationships, such as for technical support, inputs, marketing, social capital or political capital, are required to sustain the changes? What role has the SHOUHARDO III Program had in developing or facilitating these relationships? After the program ends, how do you think these relationships will change?

SHOUHARDO III MTE Methodology Report 17 July 2018

30) How happy are beneficiaries and intermediaries with the changes? How motivated will they be to continue to maintain or support the changes?

31) What are the biggest threats to sustaining the changes induced by the SHOUHARDO III Program? How can these threats be addressed?

Wrap-up  We have come to the last part of the discussion. Before we end, we have a few short questions to ask you to help us be able to improve the project that supports this community.  How would you recommend improving the project for the remaining years? Probe.  Any questions for us?  Repeat main points of the discussion for validation; any important information we are missing?  Remind interviewees of objective of study and what will be done with shared information.  Thanks to all for their time and active/honest participation.

KII: PROJECT PARTNERS/STAFF

TOPICAL OUTLINE Topical Outline for KIIs with Project Staff, Implementing Partners, and Technical Team Members who focus on Purpose 4 implementation (e.g., including team for strengthening men/boys’ engagement). Note: The questions are open-ended to encourage discussion.

Introduction: 1) What is your current position? What is your association with the SHOUHARDO3 project? How long have you worked with the project? What activities are you engaged in on gender? Project design and implementation: 2) What documents or plans currently guide your work on gender? To what extent is the Gender Action Plan guiding the day-to-day activities of the project?

3) Have you received any training related to gender to support you to do your work? If yes, have you applied any of what was covered? What / how? (Probe for applying gender equity capacities in their own personal lives, as well.)

4) In your opinion, were the right gender activities identified? Why or why not?

5) Have the activities fit the needs of the communities? Are the activities appropriate to the local context?

SHOUHARDO III MTE Methodology Report 17 July 2018

6) Has the project had to drop or alter any of the original planned activities for Purpose 4? If yes, why? Key evolutions since project start up and implementation.

7) How has the project engaged gender champions/role models (women and men, religious leaders, elders/etc.) on gender issues since the start of the project? Has that strategy been effective? Why or why not?

8) Has the project integrated gender monitoring tools into the M&E system? If yes, what has that looked like? Has this been effective for capturing how gender interventions are impacting other related purpose outcomes? Why or why not?

9) How has the project ensured involvement of project participants for different genders, people with disability and other often-marginalized people throughout the project cycle?

10) What factors contribute to some villages being more successful at adopting changes in gender attitudes? Less successful?

Collaboration with Government Offices, Community Leaders, Service Providers, and Partners 11) Tell me a bit about your working relationship with community partners? With upazila/union leaders? What other counterparts do you work with on gender-related priorities?

12) Has SHOUHARDO3 successfully integrated gender activities within the local development plans? What is the collaboration with other programs or projects? Factors affecting local collaboration. Success/ challenges?

13) What does the network of GBV resources and services look like in this community/upazila/union? What have been some of the greatest successes to date? The greatest challenges? (Probe: quality and reliability of services, involvement of local authorities and religious leaders, etc.)

14) In your opinion, what is the level of awareness of households in this community/upazila/union of the GBV support services and laws/policies that do exist? Any evidence of the changes in awareness over the past year? (Probe also for any barriers to the services being utilized, etc.)

15) What have been the key lessons learned from the implementation of the project so far? What have been key learnings from partnering with different stakeholders to achieve gender equality?

Overall impressions 16) What changes, positive or negative, have you noticed in the communities in which you work since the project began?

100

SHOUHARDO III MTE Methodology Report 17 July 2018

 Do you feel the well-being of the project participants (women, men, girls, boys) has changed over the last year? How? Why?

17) In your opinion, which gender activities have been the most successful? For whom? What project activities should be included/ or changed to promote gender and equity for the remaining period of the project? Why?

Wrap-up  Any questions for us?  Repeat main points of the discussion for validation; any important information we are missing?  Remind key informants of objective of study and what will be done with shared information.  Thanks to all for their time and participation.

101

SHOUHARDO III MTE Methodology Report 17 July 2018

SHOUHARDO III Mid-term Evaluation PURPOSE 4: Women's Empowerment and Gender Equity SHOUHARDO III Staff Topical Outline

These questions will be asked of CARE technical staff and partner managers and staff and Local Government officials who know the program at the national and local levels. The questions are open-ended to encourage discussion.

I. General Background  What is your current position? What is your association with the SHOUHARDO III Program? How long have you worked with program?

 What activities are you engaged in on gender?

II. Activities

A. Program Design and Implementation

1. What documents or plans currently guide your work on gender? To what extent is the original gender strategy guiding the day-to-day activities of the program?

2. Has the program had to drop any of the original planned activities? If yes, why?

3. In your opinion, were the right gender activities identified? Have the activities fit the needs of the communities? Are the activities appropriate to the local context? Why or why not?

4. Have you received any training related to gender to support you to do your work? If yes, have you applied any of what was covered? What / how?

B. Collaboration with Government Extension Offices & Partners

5. Tell me a bit about your working relationship with the SHOUHARDO III Program. What other counterparts do you work with on gender-related priorities?

6. Has SHOUHARDO III successfully integrated gender program activities within the local development plans? What is the collaboration with other programs or programs? Factors affecting local collaboration. Success/ challenges?

7. Please describe the training you have received from the SHUHARDO III Program. What types of gender training? How were participants selected to participate? What were the most successful training activities? What were less successful training activities? Why? What has been happening after the trainings?

102

SHOUHARDO III MTE Methodology Report 17 July 2018

8. What have been some of the greatest successes to date?

The greatest challenges?

9. Who in a household makes important household decisions? Any changes for the past 2 years (Probe for which decisions are considered “important”; e.g., deciding medical treatment, education, taking out loans, bearing more children, etc.?) Probe for reasons/details for all questions.

C. Overall impression 1. What is your understanding of empowerment? Do you think the program has been able to empower women and men in the program area? How? probe

2. What changes, positive or negative, have you noticed in the communities in which you work since the program began? Do you feel the well-being of the program participants have changed over the last two years? Probe for how it’s better/worse and why

3. In your opinion, which activities have been the most successful? For whom? What program activities should be included to promote gender and equity for the remaining period of the program? Why

4. What has been the key lessons learned from the implementation of the program as well as partnering with different stakeholders to achieve gender equality?

 Any questions for us; important information we are missing?  Repeat main objective of study and what will be done with shared information. Thanks to all for their time and active/honest participation

======SHOUHARDO III Mid-Term Evaluation TOPICAL OUTLINES Public Services for PEP (Golam) (Purpose 5 and Sub-Purposes 5.1 and 5.2)

A. SHOUHARDO III Technical Staff (Primarily Staff Responsible for Purpose 5) & B. SHOUHARDO III Partner NGO Staff 1. How would you describe what SHOUHARDO III seeks to accomplish with regards to good governance to increase the responsiveness and accountability of government to the needs of PEP households, particularly of women? 2. What are the governance interventions implemented/planned by the SHOUHARDO III program? 3. What changes have you seen as a result of the governance interventions implemented by the program?

103

SHOUHARDO III MTE Methodology Report 17 July 2018

4. How well are LEBs and NBDs performing? Why do you think they are performing well or not so well? 5. How effective have LEBs and NBDs been in delivering basic public services? 6. How well are PACCs at district and upazila levels performing to improve government services for PEP? Can you describe any recent actions of the PACCs taken to resolve problems that arise from field implementation around public services for PEP? 7. How well are VDCs and Youth Groups performing? Why do you think they are performing well or not so well? 8. Does the LEBs have an Act / NBDs have action plan (citizen charter) to improve delivery of basic public services? 9. Can you describe three recent actions the LEBs and NBDs have taken to implement these Acts/plans? What are the differences you have seen in how well the LEBs/NBDs delivers these basic public services? 10. Could you share with us three examples of how LEBs allow women, men and youth to participate in local governance – for example, budgeting, planning, service delivery monitoring, communal asset development or restoration, early warning systems, etc.? 11. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities? 12. Are there particular groups of women, men or youth who should be benefitting from these basic public services but are not?

13. How well is the program performing in facilitating the organization of remote-level service campaigns by LEBs and NBDs? How effective is this intervention?

14. How useful is the bi-annual ward shavas (meetings) with respect to the governance and public service toward PEP and community? (This is a result area. Some UPs are yet to adopt it. So may not apply to all UPs)

15. What constraints do you believe inhibit LEBs and NBDs from delivering basic public services in ways that benefit all women, men and youth?

16. What suggestions do you have for addressing these constraints or otherwise for enabling LEBs and NBDs to deliver basic public services more effectively and fairly? What specifically would you be willing to take responsibility for to address these constraints?

17. What suggestions do you have to help local government increase its accountability to the communities and citizens it serves?

18. What else would you like the reviewers to know about the public services for PEP?

104

SHOUHARDO III MTE Methodology Report 17 July 2018

C. Village Development Committees (VDCs)

1. How would you describe the performance of VDCs formed by the program to support local communities in developing and implementing community-driven action plans to liaise between PEP households and LEBs/NBDs?

2. How well are LEBs and NBDs performing their core functions? Why do you think they are performing well or not so well?

3. What are the services you have received from LEBs and NBDs? What else you are supposed to receive from them?

4. How much have VDC members and PEP benefited from delivery of these basic services? Could you describe some specific examples? Please describe how VDC members / PEP could benefit more from these basic services?

5. Are you aware of LEBs constitution / NBDs action plan to improve delivery of basic public services for the community?

6. Have you participated in any bi-annual ward shavas in your community? How useful you find these shavas?

7. Are you aware of PACC at your upazila? Have you or any PEP invited to the PACC held at your upazila? If so, how useful you find the PACC as a platform for advocacy for good governance and public service delivery for PEP?

8. Can you describe three recent actions the LEBs and NBDs have taken to implement these constitution /plan? What are the differences you have seen in how well the LEBs/NBDs delivers these basic public services?

9. Could you share with us three examples of how LEBs allow women, men and youth to participate in local governance – for example, budgeting, planning, service delivery monitoring, communal asset development or restoration, early warning systems, etc.?

10. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities?

11. Are there particular groups of women, men or youth who should be benefitting from these basic public services but are not?

12. What else would you like the reviewers to know about the services of LEBs and NBDs for PEP?

105

SHOUHARDO III MTE Methodology Report 17 July 2018

D. Youth Groups

1. How was the youth group organized? What are the responsibilities of the group?

2. Who and how do you interact with? What is experience interacting with them?

3. What types of training and orientation you received from the program? How useful you find these trainings/orientation?

4. What are the governance interventions implemented by the program in your community?

5. How have you been involved in the governance interventions implemented by the program in your community?

6. How do you see your role as catalyst for greater engagement between communities, UP and public service providers? How does your community, LEBs and NBDs value your works?

7. What changes have you seen in PEP households and your community as a result of these governance interventions?

8. How well are LEBs and NBDs performing? Why do you think they are performing well or not so well?

9. How do you reach out to the community on information about government services? Do you face any problem to reach out to the community?

10. What else would you like the reviewers to know about the services of LEBs and NBDs for PEP?

E. LEBs - Union Parishads (UP)

1. Overall, how well is your UP performing? Why do you think the UP is performing well or not so well?

2. How does the UP interact with the community and community-based organizations in delivering public services? How effective has UP been in delivering basic public services?

3. What types of support UP receives from NBDs to improve delivery of basic public services for the community?

4. What are your plans and programs about basic services to the community? PEP? Are you aware of LEBs Act / NBDs action plan (citizen charter) to improve delivery of basic public services for the community?

106

SHOUHARDO III MTE Methodology Report 17 July 2018

5. Could you describe with us the results you have seen from implementing this action plan? In other words, what difference has it made in how well you deliver these basic public services?

6. How effective have VDCs, EKATA and Youth Groups been in playing their role in facilitating the basic public services to PEP? How can they improve their performance?

7. How effective is UDCC meetings to discuss community concerns and demands for services from the UP and NBDs for PEP?

8. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities?

9. What constraints do you believe inhibit your institution from delivering basic public services in ways that benefit all women, men and youth?

10. What suggestions do you have for addressing these constraints or otherwise for enabling your institution to deliver basic services more effectively and fairly?

11. Besides elections, what kinds of accountability mechanisms are being used to increase your institution’s accountability to the communities where they deliver these basic public services?

12. What suggestions do you have to help your institution increase its accountability to the communities and women, men and youth you serve?

F. Representatives of NBDs

1. Overall, how well are LEBs/NBDs performing? Why do you think they are performing well or not so well? 2. How effective has your particular institution (LEBs/NBDs) been in delivering basic public services? 3. What role do VDCs, EKATA, Youth Groups and other civil society and community- based organizations play in helping you to deliver these basic public services? 4. How effective have these been in playing that role? How can they improve their performance? 5. Have you participated in the SHOUHARDO III program good governance training? If so, how useful was these trainings? What are the topics you find most appropriate for your work?

107

SHOUHARDO III MTE Methodology Report 17 July 2018

6. Could you share with us three examples of how women, men and youth are participating in helping to implement this action plan – for example, budgeting, planning, service delivery monitoring, communal asset development or restoration, early warning systems, etc.? 7. Are there particular groups of women, men and youth who participate more actively in activities within the local development plans? What is their role? Describe types of leadership positions held by women, men and youth in these activities? 8. Could you describe with us the results you have seen from implementing these local development plans? In other words, what difference has it made in how well you deliver the basic public services? 9. Are there particular groups of women, men and youth who should be benefitting from these basic public services but are not? 10. What constraints do you believe inhibit your institution from delivering basic public services in ways that benefit all women, men and youth? 11. What suggestions do you have for addressing these constraints or otherwise for enabling your institution to deliver basic services more effectively and fairly? 12. What kinds of accountability mechanisms are being used to increase your institution’s accountability to the communities where they deliver these basic public services? 13. What suggestions do you have to help your institution increase its accountability to the communities and women, men and youth you serve? 14. What else would you like the reviewers to know about the services of NBDs for PEP?

108

SHOUHARDO III MTE Methodology Report 17 July 2018

G. Program Advisory Coordination Committees (PACC)

1. How well PACC is performing in meeting its objectives of (a) effective coordination of SHOUHARDO III program activities, (b) support from both national and operational level GOB ministries and departments for program implementation and (c) sharing on field evidence on policy implementation issues?

2. What kind of support is PACC at upazila level providing with regards to public service and good governance to the community? PEP? How effective are these services of PACC?

3. What kind of support are PACC at National and District levels providing with regards to public service and good governance to the community? PEP? How effective are these services of PACC?

4. How do you describe the role do LEBs/NBDs play in delivering basic public services to the community? PEP? How effective is their role? How can LEBs/NBDs improve their performance?

5. What is your assessment of the PEP representation and voice in the Upazila level PACC? What are their contribution to improve public service delivery activities? Why do you think they are performing well or not so well?

6. Could you describe with us the results you have seen from implementing the governance interventions by SHOUHARDO III for PEP? Target community?

7. What constraints do you believe inhibit LEBs/NBDs from delivering basic public services in ways that benefit all women, men and youth?

8. What suggestions do you have for addressing these constraints or otherwise for enabling these institutions to deliver basic services more effectively and fairly?

9. What suggestions do you have to help these institutions increase their accountability to the communities and women, men and youth you serve?

10. What else would you like the reviewers to know about the services of PACC with respect to good governance and public service to community? PEP?

======

109

SHOUHARDO III MTE Methodology Report 17 July 2018

SHOUHARDO III Mid-Term Evaluation Gender Equity (Cross-Cutting Theme) (Jeanne) TOPICAL OUTLINES (See Topical Outline for Purpose 4) ======SHOUHARDO III Mid-Term Evaluation Governance (Cross Cutting Theme) (Golam) TOPICAL OUTLINE

1. How well does the SHOUHARDO III program implement advocacy/governance strategies in the current Bangladesh context in which NGOs have limited influence with government and are reluctant to become too visible or confrontational with advocacy activities? 2. How does the SHOUHARDO III program PACC coordinate with relevant government ministries at different levels to mainstream improved nutrition governance? What works well? What has not worked well? 3. Does the SHOUHARDO III program use any national advocacy platforms or forums to play a knowledge brokering role for improved governance? What works well? What has not worked well? 4. How well does the SHOUHARDO III program in its partnership with government and its engagement throughout to ensure greater ownership of results at the national level and local level, leading to greater success of advocacy efforts around the evidence base? 5. How well does SHOUHARDO III program address gender issues with respect to governance? ======

SHOUHARDO III Mid-Term Evaluation Environmental Compliance and Climate Risk Management (Mike) TOPICAL OUTLINE

Investigations in this area cover two major topics, how have capacities of beneficiaries and intermediaries been expanded to be able to manage climate-related risk and how well has the program complied with environmental compliance demands for FFP programs. Questions for the first area of investigation are included mainly in the data collection for Purpose 1, Purpose 3, Purpose 5 and the context are of investigation (see page 86. The following questions are focused on data collection for the environmental compliance analysis. In general, we want to inquire on what potential environmental impacts have been identified and how they are being dealt with. Did the program plan well in terms of staffing, budgeting, and monitoring? How do they develop the capacities of program beneficiaries and intermediaries to deal with environmental and climate mitigation (training, consulting, monitoring of the activities to verify implementation at all levels).

KEY QUESTIONS

110

SHOUHARDO III MTE Methodology Report 17 July 2018

1. How has the IEE been used since it was first developed? Have any environmental compliance conditions been set by FFP/USAID based on the IEE? How has the program dealt with these? 2. Who is responsible for environmental compliance in the SHOUHARDO III Program? 3. Has the program developed an EMMP? What environmental issues have been identified and how is the program addressing these? 4. What other work has been done by SHOUHARDO III and/or the local mission relative to identifying environmental concerns of the the program? What issues have been identified? How is the program dealing with these? 5. Is the program working in or near any protected areas? How does the program ensure minimal negative environmental effects? 6. Is the program working with any agricultural chemicals either directly or through intermediaries? Has a PERSUAP been completed? How has the program addressed any environmental concerns associated with these agricultural chemicals?

======SHOUHARDO III Mid-Term Evaluation Cross-Cutting Theme: TARGETING OF YOUTH (Mark) TOPICAL OUTLINE

This topical outline provides key questions related to investigating the effectiveness of the SHOUHARDO III Program in targeting youth aged 18-35. The questions will be asked during focus group discussions held with different types of participant groups.

KEY QUESTIONS FOR GROUPS

1. Please tell me what you know about the SHOUHARDO III Program? 2. How have you been engaged with or otherwise participated in the program? When did this participation begin? 3. Roughly what percentage of your members are youth between the ages of 18 and 35? 4. What positive or negative changes have occurred as a result of program activities (changes in livelihoods activities, changes in knowledge on messages promoted by the program, changes in group or individual activities, changes in social capita or political capital), especially for youth aged 18-35? 5. Which program activities have had the greatest impact in achieving this change? Why? 6. Which program activities are having the least amount of impact? Why? 7. Are there other youth in the community who should be participating in the program, but are not? Please describe them. Why do they not participate? 8. What suggestions do you have for enabling the SHOUHARDO III Program to have greater sustained impact on youth between the ages of 18 and 35? ======

111

SHOUHARDO III MTE Methodology Report 17 July 2018

IMPLEMENTATION SYSTEMS SHOUHARDO III Mid-Term Evaluation Context & Operating Environment (Mike) TOPICAL OUTLINE (All Team Members) The following topics should be covered in interviews with representatives of program partners, implementation staff, and participants.

1. Since the SHOUHARDO III Program began implementation, what changes have occurred in the operating context (e.g., major events like disasters, slow onset events like climate change, government policy changes, etc.) that have affected program implementation, either positively or negatively?

2. How did these specifically affect implementation? What did the program do to adapt to these changes?

3. How have these changes affected the food insecurity or resilience of targeted impact groups? What other groups are becoming more food insecure or are having reduced resilience as a result of these contextual changes?

4. Given these changes in the operating environment, which program activities seem most irrelevant now in terms of having impact on the lives of targeted impact groups?

5. What suggestions do you have for adapting the program's strategy or implementation systems in order to be better able to respond to changes in the operating context? ======SHOUHARDO III Mid-term Evaluation Targeting (Mark) TOPICAL OUTLINE (All Team Members) Topics for Program Managers and Implementation Staff 1. Please explain your understanding of the SHOUHARDO III Program's strategy for targeting communities and for identifying poor or extreme poor participants from these communities. o Are the targeting criteria well understood by communities/participants? o To what extend did communities/participants participate in drawing the criteria? o What is the level of community/participants engagement in the targeting process? o Has there been any influence from Government on targeting and selection of participants? o Any political interference with targeting criteria and selection processes? o Any alignment of the targeting framework with the government of Bangladesh’s overall targeting framework for vulnerable households?

112

SHOUHARDO III MTE Methodology Report 17 July 2018

2. How effective has this targeting strategy been, i.e., have the right people/communities been reached by the program or are their other communities/people that should have been targeted but were not? Who is not being reached well enough by the program? Why? 3. Are there any mechanisms put in place to gather community feedback on targeting including inclusion and exclusion errors? 4. In what ways did the program respond to problems with beneficiary selection that were observed? Were these responses effective, or could more have been done to resolve problems? 5. How does the program monitor and manage targeting issues such as changes in the list of target communities, movement of program beneficiaries out of the target communities, non-participation of selected beneficiaries in program activities? 6. Targeting of Purpose 1 activities – Value chain activities focused on field crops, but not relevant for landless households. What are options for landless households, and are the prospects for uptake of these options by landless households? 7. What has the project done to target youth? What is the appropriate definition of ‘youth’? Should different definitions be used in different contexts or issues? What challenges has the project faced in targeting youth, identifying their specific concerns, and identifying appropriate strategies to interact and support youth? Topics for Beneficiaries and Intermediaries 8. What are the characteristics of the individuals and households are benefitting from the activities of SHOUHARDO III in your community? (after asking generally, probe about PEP, women, youth) 9. Do you think that the activities of SHOUHARDO III are effectively helping the most vulnerable individuals and households in your community? Why or why not? 10. Have any individuals or households benefitted from the SHOUHARDO III program and how have they benefitted? If yes, what are the characteristics of these individuals and households? ======SHOUHARDO III Mid-Term Evaluation Management (Mike) TOPICAL OUTLINE 1. What structures are used to manage implementation of the SHOUHARDO III Program across all the partners, including staffing of management teams and implementing partners, steering or advisory committees, meetings of managers, and so on? 2. What is the history of these management structures, including turnover of personnel in key positions, restructuring or other changes? 3. Who is responsible for the overall vision for the program? How well has the vision been articulated? How effectively has this vision been imparted to staff within the different partners?

113

SHOUHARDO III MTE Methodology Report 17 July 2018

4. Who is responsible for the vision within each implementing partner organization? Are the visions of the individual partners generally coherent with each other and with the vision at the program level level? If not, how are the differences managed? 5. If there are differences, what impact has this had on program delivery? 6. How are operational plans developed for the program? What has worked well in this process? What has not worked well? 7. How are problems with implementation identified, analyzed and solved? How is information generated by the program's M&E systems used for decision-making? What has worked well in terms of problem-solving? What has not worked well? 8. Please describe the working relationships between the SHOUHADO III CoP, program technical coordinators and senior leadership in your organization’s Headquarters. How much management responsibility is devolved and how much is exercised from HQ? 9. Please describe the working relationships between the CoP and regional and district-based managerial staff in the field. How much management responsibility is devolved to the field and how much is exercised by the CoP? 10. What kinds of information are communicated (i) within CARE at different levels, (ii) among implementing partners at different levels, (iii) with USAID/FFP? What has worked well? What has not worked well? 11. How well has the program communicated with external stakeholders, e.g., ministry officials (national and local), other donors and UN agencies, other NGOs with similar programs and programs in Bangladesh? What has worked well? What has not worked well? 12. What have been the biggest challenges relative to administrative support for the program in the areas of financial management, commodity management, human resource management, procurement, transport or anything else related to program management? 13. What solutions have been devised to address these challenges? What has worked well? What has not worked well? 14. What changes would you propose to improve program management in the remaining life of the program? ======SHOUHARDO III Mid-Term Evaluation Partnership (Mike) TOPICAL OUTLINE These questions should be asked of representatives of the major partner organizations, as shown in the table. Due to time limitations, the MTE may not be able to reach out to all other partners face-to-face and may send the questions requesting responses by email.

TYPE OF PARTNERS PARTNER Program Management CARE & Technical Lead

114

SHOUHARDO III MTE Methodology Report 17 July 2018

SKS Foundation (Gaibandha), ESDO (Jamalpir), MJSKS (Kurigram), Implementing Partners NDP (Sirajganj), DAM (Habiganj & Sunamganj), POPI (Kishoreganj & Netrakona) Major Technical WorldFish Center, GoB LGED and GoB DPHE Partners Advanced Chemical Industries (ACI) , Lal Teer Seed, BRAC Seed, Banglalink, bKash, Krishibid, PetroChem, Bangladesh Institute for ICT in Development (BIID) , PRAN Foods, Bangladesh Center for Other Partners Communication Programs, Bangladesh Water Development Board Flood Forecasting Warning Center, Bangladesh Meteorological Department, and AgroMet

 In terms of working with CARE as the lead agency, what did you like about the relationship with them? What didn't you like and how would you like to see that changed in the remaining life of the SHOUHARDO III Program?  Relative to relationships with implementing partners, what has worked well in terms of working together to achieve impact or making more effective use of resources? What has not worked well? What would you like to see done differently in the remaining life of the program with regard to partnership relationships with implementing partners?  What other partnership relationships have been important for SHOUHARDO III? What has worked well in these relationships? What has not worked well? What would you like to see done differently in the remaining life of the program with regard to partnership relationships with these other organizations?  How do you feel about how major strategic or problem-solving decisions were made in the program? How much influence did you have on the decisions? How informed were you on the reasoning behind the decisions that were made?  How would you describe your overall level of satisfaction with the SHOUHARDO III Program? What would you suggest be done differently in future programs like SHOUHARDO III?

SHOUHARDO III Mid-Term Evaluation Collaborative Learning and Action (Mark & Jeanne) TOPICAL OUTLINE

The purpose of the investigation is to;  Assess the adequacy and quality of the collaborative learning and action processes used in the SHOUHARDO III Program  Review systems for capturing and documenting lessons learned and assess the extent to which they are used in program implementation and refining program design  Assess the use of monitoring and evaluation information in managing for results, adjusting program strategies and overall program decision-making  Assess how well the program is seeking out, testing and adapting new ideas and approaches to enhance program implementation efficiency and effectiveness  Assess opportunities for improving efficiency and effectiveness of M&E systems 115

SHOUHARDO III MTE Methodology Report 17 July 2018

Structure and Capacity of the M&E System -Mark  How does the M&E system work? Who is responsible for what? Who collects data? How often? How does the data/information flow from Implementing Partners to CARE?  Is the M&E Unit adequately staffed? What are the challenges with regards to human resources? Workload of the staff?  What has been done so far to develop M&E capacity? Did CARE provide any M&E training to partner NGOs? What is the role of implementing partners in M&E? What is working well? What is not working? Where are the gaps?

Utilization of M&E Products/Information (or how have SHOUHARDO III management and technical specialists used data generated by the program to inform programmatic decisions, referral and follow-up?) - Mark  What is the contribution of M&E to the program?  How does the program monitor progress against the implementation plans? What has worked well? What has not worked so well? Why?  Do you have any mechanisms to know whether the outputs being achieved are turning to outcomes? Do you have evidence to show that this is happening?  How is M&E informing the programmatic decisions that have been made? Do you have any evidence of using M&E to make any programmatic adjustment?  How has the program strategy or approach changed as a result of information generated by the program's M&E systems?  How is feedback provided back down the reporting chain on program monitoring reports? What has worked well? What has not worked so well? Why?  PMTS and QLSM, description, purpose, use by management and field staff, challenges in implementation

Beneficiary Accountability -Mark  How were the program staff oriented on the basic principles and practices of beneficiary accountability?  How did program staff consult with male and female beneficiaries in one or more communities to define indicators for program success following startup? How soon after startup?  What kind of feedback and response channel(s) did program staff establish with members of targeted communities? How soon was it established? How well does the channel reflect the preferences of the targeted communities?  How do program staff document and respond to community feedback (including constituent voice) received? Through what kind of media? In what language? And how frequently?

Stakeholder Reporting -Jeanne  Do the Program's systems meet USAID requirements?  How do program staff communicate performance monitoring findings to key stakeholders? How frequently?

116

SHOUHARDO III MTE Methodology Report 17 July 2018

Knowledge In - Jeanne  Who is responsible for bringing new ideas and approaches into the program from outside sources? How is this responsibility formalized and monitored?  What new ideas and approaches have been brought into the program from outside the program? Who can be credited for bringing these into the program? Which of these have been most useful? Which have not been very useful? Why?

Knowledge Out - Jeanne  Who is responsible for identifying, documenting and disseminating knowledge out of the program?  How does the program identify best practices and lessons learned? Please provide some examples.  How do these get documented and disseminated?  What is working well as far as identifying, documenting and disseminating lessons learned and best practices from the SHOUHARDO III experiences?  How can the program do a better job of capturing and disseminating lessons learned and best practices being generated by the program?  Strategies for applying evidence from project in effective advocacy strategy.

M&E System Improvements - Mark  What lessons learned relative to M&E have emerged from the experience of SHOUHARDO III up to now?  What is your recommendation to improve the system at various levels? At partner level? At CARE level?  Recommendations for improving M&E system efficiency?  Recommendations for improving M&E system effectiveness?

Baseline Survey - Mark How has SHOUHARDO III used the baseline results? Any feedback on the baseline survey? What worked? What did not work well? What can be improved upon?

======

117

SHOUHARDO III MTE Methodology Report 17 July 2018

SHOUHARDO III Mid-Term Evaluation Program Integration (Golam) TOPICAL OUTLINE

Across Purposes within the SHOUHARDO III Program • The SHOUHARDO III program is designed to be implemented as an integrated set of activities for different beneficiary categories within each target community. Activities, outputs and their related lower level outcomes under each higher level outcome are designed to be mutually re-enforcing to other outcomes, within and across the five purposes. To what extent do you think the program succeeded in this?

• What potential synergies within the program were not sufficiently capitalized upon in terms of complementary activities that could produce benefits for all households within a target community, not only those of the beneficiaries of a specific program purpose? Why?

• How would you describe the synergies/complementarities within the outputs under different program purposes?

• Do you suggest any changes in program design and implementation methods that might make it possible to realize a greater degree of synergy across different program purposes at community level in the rest of life of the program?

• What is the status of implementation strategies as identified in the SHOUHARDO III Implementation Strategy (Coordination within SHOUHARDO III, Integrated Platforms, Prioritization of Convergences, Monitoring and Evaluation for Integration)?

======SHOUHARDO III Mid-Term Evaluation Coordination and Collaboration (Rukhsana) TOPICAL OUTLINE

For SHOUHARDO III Senior Management and Technical Staff – Across Purposes: • The SHOUHARDO III program is designed to be implemented as an integrated set of activities for different beneficiary categories within each target community. Activities, outputs and their related lower level outcomes under each higher level outcome are designed to be mutually re-enforcing to other outcomes, within and across the five purposes. To what extent do you think the program succeeded in this? • What potential activities within the program were not sufficiently utilized that could produce benefits for all households within a target community, not only those of the beneficiaries of a specific program purpose? Why? • How would you describe the achievements as per programed outputs under different program purposes? • Do you, as Senior Coordinator/ Staff of Purpose Area (1/2/3/4/5) think you coordinate and collaborate with senior staff of other Purpose Areas? How frequently do you meet?

118

SHOUHARDO III MTE Methodology Report 17 July 2018

• Is that enough? Or could this coordination be improved? How? • Which areas do you think are currently weak and would need more input? • How well does SHOUHARDO III program align with other NGOs strategies for poverty reduction, agricultural development, food and nutrition security, health and nutrition and disaster risk reduction and mitigation? • How does SHOUHARDO coordinate and collaborate with the two other FFP programs in Bangladesh being implemented by World Vision and HKI, respectively? • What has worked well with them? Was there anything that did not work as well? • What were the reasons? • Are there other organizations in Bangladesh that are implementing activities similar to those in SHOUHARDO III? Which organizations? • How does SHOUHARDO III collaborate with these organizations? • Which activities are in common with these organizations? • Do you think SHOUHARDO III can undertake some other activities that were not initially planned? And why? • Do you suggest any changes in program design and implementation methods that might make it possible to achieve greater synergy across different program purposes at community level for the remaining duration of the program? • With which technical networks in Bangladesh does SHOUHARDO III staff interact? • What has been the benefit of these interactions?

For Field Implementers and Other Union-level Stakeholders: • Do you know of other NGOs that are working in the same unions/villages as SHOUHARDO III? • How does the program coordinate with these organizations or programs? • Do you see any benefits/ results of this coordination? • What are the challenges that the program has had to face for this type of collaboration? ======SHOUHARDO III Mid-Term Evaluation Financial Management (Mike) TOPICAL OUTLINE The following topics should be discussed with finance managers of CARE and implementing partners in the partner head offices in Dhaka as well as in the district field offices.

Budget Questions ∞ What is the current LOA approved budget? How has this been revised since program start-up? ∞ What are expenditures through the most recent reporting period in FY '18? ∞ What percentage of the approved budget has been spent through this period? ∞ How are budgets prepared and compiled? What works well? What does not work well? ∞ What suggestions do you have for improving the program's budgeting processes? ∞ The following table should be completed (the table line items can be modified as needed).

119

SHOUHARDO III MTE Methodology Report 17 July 2018

Cash Expenditure Summary Programed Through 1st Quarter FY 2018 (US$) Monetization GoB Cost Cost Center 202e ITSH Total Proceeds Contribution Share CARE Implementing Partners (listed individually) Total Direct Costs NICRA for CARE NICRA for any Implementing

Partners (listed individually) TOTAL Expenses through

December 2017 TOTAL LOA Budget at Time of

Approval Current Amended LOA Budget Percent of Current LOA Budget

Spent by December 2017

Cash Flow ∞ How is cash for program implementation being provided to partners? ∞ How effective has cash flow been managed in the program? Have there been any significant delays in cash flow either from the donor to CARE or from CARE to the implementing partners? What was the cause of the delays? What changes were made in managing cash flow? ∞ What suggestions do you have for improving the cash flow systems in the program? Reporting ∞ How do financial reports for the program get prepared? ∞ What problems have occurred with financial reporting and how have these been resolved? ∞ What suggestions do you have for improving the financial reporting systems in the program? GoB Commitment ∞ What is the GoB financial commitment to the SHOUHARDO III Program and how is this being met? ∞ What percentage of the GoB commitment will be achieved by December 2017? Cost Share ∞ What is the cost-share commitment and how is this being met? ∞ What percentage of the cost-share commitment will be achieved by December 2017? ∞ What other forms of cost-share have arisen since the program was initiated? How are these being reported? Audits

120

SHOUHARDO III MTE Methodology Report 17 July 2018

∞ What audits have been completed on program funding since the program was initiated? ∞ What have been the audit findings? ∞ How have these been addressed? ======SHOUHARDO III Mid-Term Evaluation TOPICAL OUTLINES Commodity Management (Golam)

The following topics will be covered in interviews with commodity management staff.

Pipeline ∞ How does the food pipeline work? Who is responsible for call forwards, port off-loading and transport to the port warehouse, inland transport, and distribution to end users? ∞ What problems or changes have occurred with call forwards and how were these managed? ∞ What problems have occurred with port commodity management and how were these managed? ∞ What problems have occurred with inland transport and how were these managed? ∞ What problems have occurred with warehousing/final distribution and how were these managed? ∞ What lessons have you learned with your commodity management? ∞ What losses have occurred and how did these occur? What was the impact of the losses on the program as well as on subsequent call forwards? What changes were made in commodity management systems in response to the losses? (The following tables should be completed.) LOA Commodity Summary (MT)

Direct Monet Total Distribution ization FY 15 FY 16 FY 17 FY 18 (Through March 2018) Cumulative disbursements through March 2018 Original Proposed LOA Quantities Revised Programed LOA Quantities Percentage of LOA Program Achieved Through March 2018

121

SHOUHARDO III MTE Methodology Report 17 July 2018

Commodity Loss Summary (MT) Amount Amount Ocean Inland Percent Fiscal Year Called Received Losses Losses Lost Forward in Country Monetized Commodities FY 15 FY 16 FY 17 FY 18 Thru Qtr 2 TOTAL Distributed Commodities FY 15 FY 16 FY 17 FY 18 Thru Qtr 2 TOTAL

Monetization ∞ What has been the history of monetization in the program? ∞ How is monetization currently being done? ∞ What problems have occurred with monetization and how were these managed? ∞ What has been the cost recovery rate on monetization over the life of the program? What is the cost recovery rate on the most recent monetization? ∞ How is monetization being used to support market development?

(The following table should be completed.) Monetization Cost Recovery

SALE FISCAL QUANTITY C&F SALE PRICE COST COMMODITY No. YEAR ( MT) ($/MT) ($/MT) RECOVERY

TOTAL FY 2015 thru FY 2018 QTR 2 * * *Weighted averages by volume

Reporting ∞ How are commodity reports prepared? ∞ What problems have occurred in report preparation and how have these been managed? ∞ What has been the response from the donor on commodity reports?

Rations 122

SHOUHARDO III MTE Methodology Report 17 July 2018

∞ How are ration composition and quantities calculated? ∞ How have they changed over the life of the program? ∞ How do recipients accept the ration commodity? ∞ How have rations been used by recipients? ∞ What changes should be considered relative to ration composition or quantities? Why should these changes be considered? ======SHOUHARDO Mid-Term Evaluation Human Resource Management (Jeanne) TOPICAL OUTLINE

The following topics should be covered in interviews with those responsible at different levels i n the SHOUHARDO III Program for human resource management.

Staff Recruitment and Retention ∞ What significant challenges has the SHOUHARDO III Program faced relative to human resources in the program? How did the program deal with these challenges? ∞ How many positions total are there in the SHOUHARDO III Program? How many vacancies are there at this point in time? ∞ What percentage of the total staff positions in the SHOUHARDO III Program are held by women? What percentage of the management positions in the program are held by women? What strategies has the program used to be able to recruit and retain female staff? What works well? What more can be done? ∞ How would you describe staff turnover in the program? How has this affected program implementation? ∞ In general, what has worked well and what has not worked well relative to hiring and retaining staff in the SHOUHARDO III Program? Staff Capacity Building and Performance Management ∞ What types of activities has the SHOUHARDO III Program undertaken to build staff capacities to be able to work effectively? ∞ How would you describe the systems in the SHOUHARDO III Program for motivating staff and facilitating high levels of performance? What has worked well? What has not worked well?

======SHOUHARDO III Mid-Term Evaluation TOPICAL OUTLINES Materials and Equipment Management (Golam)

The following topics will be covered, as relevant, in interviews with staff from all partners who are directly responsible for managing administrative support for the SHOUHARDO III Program, particularly related to materials and equipment procurement, management and logistics.

123

SHOUHARDO III MTE Methodology Report 17 July 2018

1. What systems and policies are in place for ensuring effective procurement, utilization and maintenance of materials and equipment authorized for SHOUHARDO III within your organization and with other technical and PNGO partners? 2. What is working well? 3. What problems have occurred relative to procurement, inventory, management, maintenance and logistics? 4. How have these problems affected the program? 5. What changes would you recommend be made to these systems to make them more effective?

124

SHOUHARDO III MTE Methodology Report 17 July 2018

ANNEX C: ILLUSTRATIVE VERIFICATION WORKSHOP PLAN SHOUHARDO III Mid-Term Evaluation (MTE) Wednesday, March 7 & Tuesday, March 8

WORKSHOP PURPOSE Based on information obtained from field work and other interviews undertaken thus far, the MTE Team has started formulating observations and recommendations for the remaining life of the SHOUHARDO III Program around what is working well and what can be improved in the program. These will be shared in the Verification Workshop with SHOUHARDO III implementation staff and discussed further to ensure that the observations reflect reality and the recommendations are appropriate and feasible. WORKSHOP OUTPUTS Two major outputs are targeted for the workshop. These are (1) agreement on the key observations assembled so far from the MTE and (2) refinement of recommendations being considered for the remaining life of the program. WORKSHOP FLOW DAY 1, Wednesday SESSION 1-1: Introductions, Workshop Purpose, Objectives and Plan, Logistics (1/2 hour, Mike) Participants will introduce themselves, and the workshop purpose, objectives, and agenda will be presented, along with details on logistics for the workshop. Participants will be asked to write a short answer to the following question on a small slip of paper: If you had the power to change anything in the SHOUHARDO III Program to enable it to have greater impact or to use program resources more efficiently, what one thing would you change? Participants will not present their answers. Workshop facilitators will collect the answers for review by the MTE Team later. SESSION 1-2: History & Operating Context (1/2 hour, Mike) Key events in the history of the program will be presented and discussed for clarity and completeness, ensuring that major events in the life of the program that have affected implementation have been captured. Observations on contextual factors that have influenced the program will also be presented, along with observations on how the program adapted to these. Finally, emerging contextual issues will be discussed, along with possible actions that the program could take to be positioned to adapt effectively to these. In preparation for the next session, a brief overview of the program strategy will also be presented. SESSION 1-3: Purpose 1 - Income & Food Production (2 hours, Mike & Mark) The MTE Team members responsible for investigation the Income and Food Production Component of the program will present an overview of their key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic.

125

SHOUHARDO III MTE Methodology Report 17 July 2018

SESSION 1-4: Purpose 2 - MCHN (2 hours, Rukhsana & Nicole) The MTE Team member responsible for investigation the Maternal and Child Health and Nutrition Component of the program will present an overview of key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 1-5: Purpose 3 - Disaster Preparedness & Response (1 1/2 hours, Mark) The MTE Team member responsible for investigation the Disaster Preparedness and Response Component of the program will present an overview of key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. DAY 2, Thursday SESSION 2-1: Recap of the Previous Day's Discussions and Stage Setting for Day 2 (¼ hour, Mike). A brief summary of the previous day's discussions will be provided along with some introductory information to set the stage for day two of the workshop. SESSION 2-2: Purpose 4 - Women's Empowerment and Gender Equity (1 1/2 hours, Jeanne) The MTE Team member responsible for investigating the Women's Empowerment Component of the program as well as the cross-cutting theme of gender equity will present an overview of key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-3: Purpose 5 - Public Services and Governance (1 1/2 hours, Golam) The MTE Team member responsible for investigating the Public Services Component under Purpose 5 of the program as well as the cross-cutting theme of governance will present an overview of key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-4: Cross-Cutting Themes – Youth Targeting & Environment (1 hour, Mark & Mike) The MTE Team members responsible for investigating the two cross-cutting themes of youth targeting and environment in the program will present an overview of their key observations and preliminary recommendations. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-5: Overall Program Design - Theory of Change, Targeting, & Program Impact (1 hour, Jeanne & Mike)

126

SHOUHARDO III MTE Methodology Report 17 July 2018

In the previous sessions, an overall picture for the program has been presented in terms of key observations on outputs and resulting outcomes as well as activities associated with the cross- cutting themes. In this session, the evaluation team will present (1) a summary of observations on targeting in terms of who is benefitting from the program, (2) an overall assessment on the program-level impact collectively across the Purposes toward achieving the program's Goal, and (3) the implications for the program's Theory of Change. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-6: Implementation Systems (1 1/2 hours, All Team Members) The MTE team investigated the program's implementation systems, including the management systems, partner relations, knowledge management systems (including program monitoring and evaluation), program integration, coordination and collaboration, financial resource management, human resource management, and material resource management. The MTE Team members responsible for these investigations will present an overview of the key observations and recommendations being considered for addressing challenges in these systems that are affecting implementation or for capitalizing on strengths that have been observed. At the end of the session, participants will be asked to jot down down on anonymous note cards any additional thoughts or observations that they did not have a chance to express in the session relative to the topic. SESSION 2-7: Next Steps (1/4 hour, Mike). Over the course of the workshop, some topics may have emerged from the discussions that represent areas that need to be further investigated by the MTE team. In this final session of the day, participants will discuss these topics and agree on how best to obtain information to resolve the questions that have arisen. In addition, participants will be asked to write on a small slip of paper short answers to the following questions and will be asked to leave their answers with a MTE team member before they leave the workshop: In your opinion, what recommendations are most important? Why? What recommendations are least important? Why? What additional recommendations would you suggest be considered by the MTE team?

127

SHOUHARDO III MTE Methodology Report 17 July 2018

ANNEX D: ILLUSTRATIVE MTE SUMMARY REPORT FORMAT SHOUHARDO III Mid-Term Evaluation (MTE) (The length of this report should not exceed 50 pages, excluding Annexes. Preliminary target page limits for each section are shown in parentheses.)

I. EXECUTIVE SUMMARY (2 Pages)

II. BACKGROUND INFORMATION (4 Pages)

A. Overview of SHOUHARDO III (including program history, targeted impact groups, theory of change, logical framework, geographic coverage, and resources) B. Contextual Factors Affecting Program Implementation or Impact C. MTE Methodology III. OUTPUTS PRODUCED UNDER EACH SUB-PURPOSE (including an overview of the implementation process including quantity of production, quality of production, description of participants and beneficiaries, key observations, lessons learned and specific recommendations for each sub-purpose) A. SP 1.1: Agricultural Production B. SP 1.2: Access to Markets C. SP 1.3: Off-Farm Income D. SP 1.4: Financial Services E. SP 2.1: MCHN Behavioral Change F. SP 2.2: Access to MCHN Services G. SP 2.3: WASH H. SP 3.1: Household and Community DRR/CCA I. SP 3.2: Union Parishad Capacities for DRR/CCA J. SP 4.1: Agency of Women and Adolescent Girls K. SP 4.2: Enabling Environment for Women's Empowerment L. SP 5.1: Community Capacities to Negotiate for Public Services M. SP 5.2: Accountability of Locally Elected Bodies and Nation-Building Departments IV. Emerging Outcomes and Impact (including an assessment of the immediate life-of- program impact, likelihood of this impact being sustained after the program ends, lessons learned and recommendations for enhancing sustained impact.) V. Cross-Cutting Themes (Including a summary of specific actions taken by the program to operationalize the cross-cutting themes, an assessment of how these have contributed to achieving program impact, lessons learned, and recommendations for the remaining life of the program.) VII. Program Implementation Systems (Sections on program management, partnership, knowledge management, program integration and complementarity, financial resource management, commodity management, human resource management and materials/equipment management system will provide an overview of the specific system, major observations from the review, lessons learned and recommendations for the remaining life of the program.)

128

SHOUHARDO III MTE Methodology Report 17 July 2018

VI. Overall Program Design and Major Themes for the Future (Including an analysis of the current relevance of the targeting systems, theory of change and logical framework with recommendations for the remaining life of the program) VIII. CONCLUDING REMARKS (1 Page)

ANNEXES Annex A: MTE Terms of Reference Annex B: MTE Protocol Annex C: Persons Interviewed and Sites Visited Annex C: Full List of Prioritized Recommendations Organized by Theme Other Annexes as Necessary

129

SHOUHARDO III MTE Methodology Report 17 July 2018

Initial Site Selection for the SHOUHARDO III MTE

District Upazilla Union Village Accessibility Performance SHOUHARDO I (PNGO) Char Region Kaijuri Char Gudhai Bari Easy (Car – 2 ¼ hour) Average Yes Shahjadpur Rapabati Khamar Shanila Moderate (Car, Boat, MC – 2 ¼ hr) Average Sirajganj Rajapur Chandra Para Easy (Car – ¾ hr) Low (NDP) Belkuchi Belkuchi Dalua Madhya Easy (Car – ¾ hr) Low Chauhali Sthal Sthal Naohata Remote (Car & Boat – 2 ¾ Hr) Good Bamandanga Maliani Easy (Car – 1 hr) Average Nageshwari Bhitarband Shantipur Wabda Easy (Car ‐ ¾) Good Kurigram Phulbari Shimulbari Abason Para Easy (Car – 1 ½ hrs) Low (MJSKS) Char Rajibpur Mohanganj Kirtaniatari Remote (Car & Boat – 4 ½ hrs) Good Yes Rajarhat Chhinai Chhinaihat Easy (Car – ½ hr) Average Saghata Saghata Hasilkandi Easy (Car – 1 hr) Average Sundarganj Kapasia Paschim Wapda Bandh Easy (Car – 1 hr) Average Gaibandha Erendabari Uttar Sanyasi Paschim Remote (Car, Boat, MC – 2 ¼ hrs) Low Yes (SKS) Fulchhari Uria Uttar Uria Moderate (Car & MC – 1 hr) Good Sundarganj Kapasia Ton Gram Easy (Car – 1 hr) Good Bagar Char Zula Para Easy (Car – 1 ½ hrs) Average Bakshiganj Jamalpur Sadhur Para Char Kamalerbati Easy (Car – 1 ¼ hrs) Low (ESDO) Char Goalini Char Gobindi Easy (Car – ¾ hr) Average Islampur Palbandha Char Charia Easy (Car ‐ 1 Hr) Good Yes Haor Region Itna Badla Simla Moderate (Car & Boat – 1 ½ Hrs) Average Yes Karpasha Saharmul Remote (Car, Boat, MC – 3 hrs) Average Kishoreganj Nikli Dampara Shaheb Hati Moderate (Car & MC – 2 hrs) Good (POPI) Mithamain Dhaki Borokanda Remote (Car, Boat, MC – 2 ½ hrs) Low Austagram Austagram Rangpur Hati Moderate (Car, Boat, MC – 2 ¼ hrs Good Sunamganj Bougla Bazar Bag Mara Moderate (Car, Boat, MC – 2 ¾ hrs) Average Dowarabazar (DAM) Pandergaon Notun Krishno Nagor Easy (Car – 1 ¾ hr) Good 130

SHOUHARDO III MTE Methodology Report 17 July 2018

Uttar Sreepur Cherar goan Remote (Car & MC – 2 ½ hrs) Low Tahirpur Dakshin Baradal Chaturvuj Moderate (Car & MC, 2 ¼ hrs) Low Yes TOTALS Easy – 2 Easy – 15 Low – 8 Moderate – 2 Remote ‐ 2 19 Upazillas 26 Unions 26 Villages Moderate – 5 Average – 11 Low – 2 Remote – 6 Good – 7 Average – 2 Good ‐ 2

131

SHOUHARDO III MTE Methodology Report 17 July 2018

IV. PERSONS INTERVIEWED & SITES VISITED

132

SHOUHARDO III MTE Methodology Report 17 July 2018

Persons Interviewed/Sites Visited in the SHOUHARDO III MTE

Date Location Name of Respondent or Type of Focus Group Team Member(s) Hedayat Islam, STC AG/L; Shikha Biswas, STC Life Mike, Jeanne & Skills; Jahangir Alam, STC PSE; Abdur Rashid, Mark Environment; CARE SHO3 HQ Feb 13 Dhaka Md. Sazzad Hossain, Monetization & Program Golam Support Manager Mohammad Shohedul Alam Matabbar, STC‐ Golam Commodity Zubaid Rahman, Partnership Operations & Resources & Ratan Kumar Debnath, Grants & Mike Partnership Manager, CARE SHO3 HQ Anowarul Haque, Director – Extreme Rural Poverty Mike & Mark Program, CARE Bangladesh Zakir Hussain, Assistant VP, Head of EBEK Mike Operation and Social Payment, Bank of Asia Shah Newaz, Director & SHOUHARDO Focal Point, Mike POPI Mannan Mazumdar, STC‐Governance Golam Shejuti Hayat, Advocacy Manager Golam Walter Mwasaa, CoP, SHOUHARDO III Golam & Mark Humaira Aziz, Director, Women and Girls Jeanne, Rukhsana Feb 14 Dhaka Empowerment, CARE Syeda Ashrafiz Zaharia, Senior Technical Coordinator, Women’s Empowerment, Jeanne, Rukhsana SHOUHARDO III, CARE Khan Touhid Parvez, STC, HHN, CARE Rukhsana Rashid, I&EO, CARE Rukhsana Afrhill Rances, Senior Team Leader, Knowledge Management, Advocacy and Learning, SHO III, Mark & Jeanne CARE Raquibul Hassan, Senior Monitoring and Evaluation Jeanne Coordinator, SHO III, CARE M.Abdur Rashid General Manager, Md. Abdul Mark Rayhan Bhulyan Asst. Mgr., Lal Teer STC Private Sector Engagement, Jahangir Mark Tom Love & Mohammad Nurnobi, Project Mike, Mark, Jeanne Management Specialist, USAID Bangladesh & Golam H.J.M. Kamal, Deputy CoP, SHOUHARDO III, Dhaka Mike Ehsanur Rahman, Executive Director, Dhaka MIke Feb 15 Dhaka Ahsania Mission, Dhaka Firoz Khan, Project leader &Mohammod Ilyas, Mike Nutrition Sensitive Food Systems Officer, World Fish , Dhaka Walter Mwasaa, Chief of Party, Shouhardo III, CARE Jeanne

133

SHOUHARDO III MTE Methodology Report 17 July 2018

Mhd. Sazzad Hussein, Monetization and Program Jeanne Support Manager, SHO III, CARE STC Resilience – Shafiqur Rahman Mark STC Infrastucture – Anjar Alam Mark Senior M&E Coordinator, Md. Raquibal Hasan Mark Feb 16 Sirajganj Travel to Sirajganj Abdul Salam, Project Manager, NDP Mike & later Golam Belkuchi NDP Zahurul Islam, Commodity Officer, NDP, Belkuchi Golam Office, Belkuchi Bashudev Sharker, Field Facilitator (FF) NDP, Khaled Shofiqul Islam TO Livelihoods Mark FGD with UpzDMC (Female – 3; Male – 5) Golam Belkuchi Upazilla Dr. Md. Mahbub Hossain, UH&FPO, Belkuchi Sadar Offices Khaled Upazila Health Complex FGD with Youth Group (Female – 8; Male – 2) Golam FGD with VDC (Female – 7; Male – 2) Golam Dalua Madhya, Jeanne FGD: Male Group (10 Fathers) Feb 17 Belkuchi Union, Khaled Belkuchi Upazilla Site Visit: Observed WASH facilities in the village Khaled EKATA FGD‐ 10 Females Jeanne VDC FGD – 7 females, 2 males Jeanne FGD with IGA FFBS (9 women) Mike FGD with CHD FFBS (11 women) Mike Chandra Para, Khadeja, CAV for Chandra Para Mike Rajapur Union, FGD – VDC (4 male & 3 female) Mark Belkuchi Upazilla FGD ‐ Youth who received training (3 male & 2 Mark female) FGD ‐ Maize value chain (22 male & 1 female) Mark Mr. Mohammad Hasib Sarkar, Assistant Commissioner (Land); Dr. Abdus Samad, Upz Shajadpur Livestock Officer; Engineer Ahmed Rafique, Upz Golam Upazilla Engineer; and Mr. Mohammad Azharul Islam, Upz Youth Development Officer FGD with Union Parishad at Kaijuri Union (Female‐ Kaijhuri Union Golam 0; Male‐ 7) FGD with CHD FFBS (10 participants) Khaled EKATA FGD – 11 Females Jeanne & Khaled Feb 18 Couples Session – 1 male, 1 female Jeanne Sthal Naohata, IGA Group – 12 Females Jeanne Sthal Union, Community Empowerment Volunteer (CEV) Jeanne Chauhali Upazila FGD – VDC (5 male & 8 female) Mark & Khaled FGD ‐ CFW participants (homestead raising) (4 male Mark & 2 female) Seed dealer (Imran) Mark FGD with VSLA, 12 women Mike Khamar Shanila, FGD with Sweet Gourd FFBS, 14 women Mike Rupabati Union, Khurshida Akter, FF, Khamar Shanila, NDP Mike

134

SHOUHARDO III MTE Methodology Report 17 July 2018

Shahjadpur Upazila Char Gudhai Aziza Khatun, VDC Member, Char Gudha Bari Golam Bari, Kaihuri FGD with Maize FFBS, 10 men Mike Union, Shahjadpur Fisheries Group, 5 men Mike Upazilla Sharmina Khatun, Technical Manager, Private Mike & Mark Sector Engagement, Char Region, CARE Asaduzzaman, Regional Commodity Logistics Golam Manager, Regional Office, CARE Visit to Sirajganj District Warehouse Golam Nazim Uddin, Regional Senior Technical Manager, Golam Kishoreganj Regional Office, CARE Sirajganj Zakir – M&E TO Mark Kaleda ‐ DCRM TO Mark Mike, Jeanne & Sajeda Begum, Regional Coordinator, CARE Feb 19 Golam TO‐ HHN, NDP, Fenshi Begum Khaled Siddik – Gender Focal Person Jeanne Nobi, WorldFish Technical Officer, Sirajganj Mike Jamalpur Ataur Rahman Ansari‐ PM ESDO Mark & Jeanne Nishi, TO Women’s Empowerment, Manowar, APM Technical Mgr, & Mahbub, TO Ag & Livelihoods, Mike SKS Gaibandha Mohammad Nazrul Islam, Infrastructure & Golam Environmental Officer, SKS Mohammad Mizanur Rahman Baksi, DCRM, SKS Golam Sherina Akhter‐ Char Gobindee CEV Jeanne Jamalpur Hazrat Ali‐ Religious Leader, Char Kamaler Batti Jeanne Char Charia, EKATA Group – 11 Females Jeanne Palbandha IGA COG FGD‐ 10 Females Jeanne Union, Islampur FGD – VDC (4 male & 6 female) Mark Upazila FGD – members of SHO I VDC (2 male & 1 female) Mark CHD COG FGD – 12 Females Jeanne Adolescent Girls – 9 Females Jeanne Char Gobindi, Samir Uddin maize trader Mark Feb 20 Char Goalini FGD ‐ School Based Teen Brigade (Pachbaria Mark Union, Islampur Adorsa) (20 male & 20 female) Upazila Field facilitator working with SBTB – Asfakul Islam Mark FDG – UzDMC (?) Mark Paschim Wapda FGD with Off‐Farm IGA Group, 8 men and 2 women Mike Bandh, Kapasia FGD with VSLA, 11 women Mike Union, Three CHD Participants, 3 women Mike Sundarganj FGD with VDC (Female‐8, Male‐2) Golam Upazila FGD with Youth Group (Female‐1, Male‐5) Golam

135

SHOUHARDO III MTE Methodology Report 17 July 2018

FGD with Mothers Group (Pregnant=3, lactating= 7) Khaled FGD with Maize FFBS, 13 female participants Mike Ton Gram, Mohd Abdul Rajak, Community Agriculture Mike Kapasia Union, Volunteer Sundarganj FGD with VDC (6 females, 2 males) Rukhsana & Khaled Upazila 3 Food Ration Participants Rukhsana & Khaled Sharifa Khatun, CHV, SKS Rukhsana & Khaled H.M. Golam Kibria, UNO, Sundarganj Golam Sundarganj Shahabuddin, Upazila Livestock Officer Golam Upazila Rupali Rani, Upazila Women Affairs Officer Golam Rashedul Islam, Upazila Agriculture Officer Golam EKATA FGD – 8 Females Jeanne Zula Para, NNPC – UP office 3 Females/12 males Jeanne Bagar Char VDC FGD – 5 Females/4 Males Jeanne Union, FGD – Youth received training (4 male & 3 female) Mark Bakshiganj FGD – UDMC (?) Mark Upazila Entrepreneur Union Digital Center ‐ Selim Mark FGD – goat group (11 female) Mark Char Kamlerbati, FGD (Non‐Participant Women) ‐ 7 Females Jeanne Sadhur Para EKATA FGD‐ 12 Females Jeanne Union, FGD – VDC (3 male & 5 female) Mark Bakshiganj FGD – FFBS maize (18 male) Mark Upazila FGD with Young Female Off‐Farm IGA Participants, Mike 3 women Feb 21 FGD with CHD FFBS, 7 women Mike FGD with Youth Non‐Participants, 3 men and 3 Mike Uttar Sanyasi women Paschim, EKATA Observation and FGD (14 participants), Rukhsana & Khaled Erendabari Shahar Bano CEV Union, Fulchari Mothers Group (11 participants), CHV Neelima Rukhsana & Khaled Upazila 1 lactating mother ‐ with 13 m baby, 1 lactating Rukhsana & Khaled mother with 6 m baby Neelima, CHV, SKS Rukhsana & Khaled FGD with VDC (Female‐3, Male‐3) Golam FGD with Youth Group (Female‐4, Male‐4) Golam FGD with Union Parishad Members (Mr. Fulchari Union Mohammad Golam Mostafa, Mr. Mohammad Golam Khaledur Rahman, Mr. Mohammad Abdul Kader) Rita Khatun‐ Technical Officer, Empowerment Jeanne Md. Aflab Hossain – Technical Officer, Agriculture Jamalpur Jeanne and Livelihoods Feb 22 Mahfuza Rahman – Technical Officer, DCRM Jeanne Ruhul Amin, SO‐TSR, CARE Gainbandha Mike & Golam Gaibandha Shariful Islam, Commodity Reporting Officer, CARE Golam Shankar, PM, SKS Golam

136

SHOUHARDO III MTE Methodology Report 17 July 2018

Manowar Hossain, APM Technical, SKS Golam Amir Ali, APM Finance & Operations, SKS Golam Shankar Kumar Roy, Project Manager SKS Rukhsana & Khaled Md Nazrul Islam, I&EO, SKS Rukhsana & Khaled Md. Ruhul Amin, Senior Officer, Technical Support Rukhsana & Khaled & Representation, CARE Dewan Morshed Kamal, Deputy Director Family Rukhsana & Khaled Planning, Gaibandha Atiqur Rahman, Principal, Technical Training Mike Center, Gaibandha Shankar, PM SHOUHARDO III & Rajab Ali, Head of Mike & Golam Programs, SKS Gaibandha FGD with Off‐Farm IGA Group, 7 women and 1 man Mike Abdul Rahim, Aziz, Abdul Goni, Village Leaders Mike Hasilkandi Hasilkandi, Tanvir, FF and Mahbuba, FS, SKS Mike Saghata Union, FGD with VDC (Female‐5, Male‐5) Golam Saghata Upazila FGD ‐ EKATA Group, 6 participants Rukhsana & Khaled FFBS CHD, 6 females, 1 male Rukhsana & Khaled Ratna, CHV, SKS Rukhsana & Khaled FGD with Saghata Union Parishad (Female‐2, Male‐ Saghata Union Golam 6) FGD with On‐Farm IGA FFBS, 13 women Mike FGD with VSLA, 13 women Mike Chhinaihat, FGD ‐ Adolescent group, 11 participants, females Rukhsana Chhinai Union, Najma CHV and Shefali CHV, MJSKS Rukhsana Rajarhat Upazila Sultana Begum, Pregnant mother Rukhsana FGD with VDC (Female‐5, Male‐5) Golam FGD with Youth Group (Female‐6, Male‐2) Golam Mahmudul Hassan, APM – Tech & Hassan Habib, Mike TO – Ag&L, MJSKS, Kurigram APM (Finance & Operation), Commodity Officer Golam Feb 23 Kurigram TO‐HHN, Fazlul Haq Rukhsana TM Sirajganj ‐ Md Aslam Hossain Talukder Rukhsana Sajjad Wahid TO M&E Mark Mosaddek Hussain TO DCRM Mark EKATA FGD ‐ 8 Females Jeanne Abason Para, CHD FGD – 11 Females Jeanne Shimulbari Ms Momina Khatun‐CEV Jeanne Union, Phulbari FGD – VDC (4 male & 5 female) Mark Upazila FGD ‐ Youth – non‐project participants (?) Mark KII – Field Facilitator ‐ Ubaid Mark Kirtaniatari, FGD with CHD FFBS, 10 women Mike Feb 24 Mohanganj FGD with Maize FFBS, 6 men Mike Union, Aminul Islam Batu, CAV Mike

137

SHOUHARDO III MTE Methodology Report 17 July 2018

Rajibpur Upazilla Observation of Courtyard session with pregnant (1) Rukhsana & Khaled and lactating mothers (9) Observation of GMP measurements ‐ CHV Shahana Rukhsana & Khaled Household visit counselling by CHV Shahana (45 Rukhsana days baby) Lactating mother (Fatima) Rukhsana Observations ‐ Handwashing facilities, Latrines Rukhsana Promod Chandra Sarkar, Field Supervisor, MJSKS Rukhsana Nurun Nabi, Field Supervisor, MJSKS Rukhsana FGD – VDC (2 male & 6 female) Mark FGD – members of SHO I VDC (2 male & 1 female) Mark Maliani, FGD with VDC (Female‐6, Male‐5) Golam Bamandanga EKATA Group – 10 Females Jeanne Union, Adolescent Females – 9 Females Jeanne Nageshwari CHD Group ‐ 10 Females Jeanne Upazila Shorifa ‐ CEV Jeanne Bamandanga FGD with Bamandanga UP (Female‐3, Male‐2) Golam Union Shantipur FGD with VDC (Female‐3, Male‐5) Golam Wabda, FGD with Youth Group (Female‐9, Male‐2) Golam Bhitarband Union, CAV (M), CEV (F), CHV (F) Jeanne Nageshwari Upazila Sabed Ali, PM, MJSKS, Kurigram Mike Mike, Golam & Sushil Kimar Das, SO‐TSR, CARE Hub‐Office Mark Md. Rafiqul Islam Selim, Deputy Director‐Local Golam Government, Kurigram Civil Surgeon Dr SM Aminul Islam Rukhsana & Khaled Terre Des Hommes Child Nutrition Centre Rukhsana & Khaled Discussion with Dr Tajul and team Mahmudul Hasan‐APM Tech. NDP Jeanne Kurigram Shyamak Chandra Sarkar‐ Director MJSKS Jeanne Irin Parveen – Program support officer – CARE Jeanne Feb 25 Kurigram Sabed Ali‐ PM MJSKS Kurigram Jeanne Shamima Akhter‐ Technical Officer, Empowerment Jeanne ‐ MJSKS staff Ms. Shefali Begum, CEV‐ Shantipur Wapda para Jeanne Shahana Akhter – District Women’s Affairs Officer Jeanne M&E and Learning Officer ‐ Firoz Alom Mark Laili Begum, Sewing Technical Trainer, Rajerhat Mike Rajerhat Upazilla Upazilla FGD with Sewing Training Youth Participants, 13 Mike girls& 3 boys Feb 26 Kishoreganj Anwarul Alam, TM‐Ag & Livelihoods, POPI Mike 138

SHOUHARDO III MTE Methodology Report 17 July 2018

Shofiqul Islam, Deputy Director, DAE Mike Motiul, Sales Representative, Lal Teer Mike Taufiqur Rahamn, PM Golam Zahirul Islam, Deputy Director‐Local Government, Golam Kishoreganj TO‐HHN, Sultana Akhtar Rukhsana & Khaled Dr Habibur Rahman, Civil Surgeon and Dr Mujibur Rukhsana & Khaled Rahman, Deputy Civil Surgeon Dulan Hossain‐ APM Tech. POPI Jeanne Mamun Ar Rashid‐ District Women Affairs Officer, Jeanne Kishoreganj Shabana Begum‐ CEV Jeanne Kaludia Nobrek, TO – DRCM Mark FGD with On‐Farm IGA Group, 7 women and 1 man Mike Borokanda, FGD with CHD FFBS, 8 women Mike Dhaki Union, Alamgir, Community Agriculture Volunteer Mike Mithamoin EKATA group – 6 Females Jeanne Upazilla Couples Session – 5 Females / 2 Males Jeanne Adolescent Girls – 6 Females Jeanne Austragram Golam Feb 27 FGD with Astragram UP (Female‐2, Male‐5) Union FGD with Youth Group (Female‐9, Male‐5) Golam Rangpur Hati, FGD: VSLA Group (15 female participants) Mark & Rukhsana Austagram FGD: Grandmothers Group (6) Rukhsana & Khaled Union, Meeting with School Based Teen Brigade (SBTB) (6 Mark & Rukhsana Austagram boys, 8 girls) Upazila FGD – VDC (5 male & 6 female) Mark FGD with a CHD FFBS with Ducks, 9 women Mike FGD with the VSLA, 7 women Mike FGD with a Chili FFBS, 3 men and 1 woman Mike Saharmul, FGD with VDC, Female‐7, Male‐5 Golam Karpasha Union, FGD with Youth Group, Female‐5, Male‐5 Golam Nikli Upazilla Asma Begum‐ CEV Jeanne Adolescent Females – 11 Females Jeanne IGA Off Farm Group – 5 Females/ 1 Males Jeanne Karpasha Union FGD with Karpasa UP, Female‐0, Male‐5 Golam Feb 28 Kishoreganj, Masud, TM Fisheries, SHOUHARDO 3, Haor Region, Mike CARE Regional WorldFish Office Md. Abdus Sattar Fakir, Food Monitor, CARE Golam Observation: Adolescent girls EKATA Group (11 Rukhsana girls) with CEV Shehlina and CHV Mahmuda Simla, Observation: GMP session with CHV Mahmuda Rukhsana Badla Union, and Field Facilitator Ramizuddin Itna Upazila Observation: Counselling of pregnant mother Rukhsana Naisha by CHV Mahmuda FGD – youth, project non‐participants, Mark

139

SHOUHARDO III MTE Methodology Report 17 July 2018

Including union disaster volunteer (6 male & 3 female) FGD ‐ IGA off‐farm (6 male & 2 female) Mark Kishoreganj, Subrata, Haor Regional Coordinator, CARE Mike CARE Office Towfique, PM, POPI Mike Kishroeganj, PM POPI, Mr Toufique Rukhsana POPI Office TO‐DCRM, (Kloutiya Nokrek) Rukhsana Tarun Barua‐ Field Supervisor Jeanne Upazila Health & Family Planning Officer, (Dr Asad March 1 Rukhsana Din Mahmud) Senior Engineer, DPHE, Nikli‐ (Md Aminur Rahman Kishoreganj Rukhsana Bhuiyan) Focus group – goat trader, bamboo fish trap Mark makers, chili trader (4 male) Golam, Mark and Nikli Upazila FGD with Nikli UzDMC, Female‐1, Male‐7 Jeanne Halim, TO Ag & Livelihoods Mike Latif, PM, DAM, Aslam, APM‐T, DAM, Hurunur Mike Rashid, SO‐TCR, CARE Sunamganj, Mizanur Rahman, Commodity Officer Golam DAM Office Visit to Sunamganj Warehouse Golam Warehouse Officer and Assistant Officer,Male‐2 Golam Afifun Nahar ‐Technical Officer Purpose 4 Jeanne TO ag and livelihoods ‐ Halim Mark Kazi Mohua Momtaz, Upazila Nirbahi Officer, Golam Dowarabazar Upazila Sunamganj Fazlur Rahman, Mark Dr. Belal Husain, District Livestock Officer Mark Natun Krishna FGD Rice FFBS, 11 men Mike March 2 Nagor, FGD with On‐Farm IGA FFBS, 7 women & 2 men Mike Pandergaon FGD – Youth group (4 male & 9 female) Mark Union, Dowarabazar Md. Mizanul Rahman, Field Facilitator in Mark Upazilla FGD: Youth Group, 5 males, 3 female Rukhsana & Jeanne Observation: Tubewell, Handwashing Station, Rukhsana Bag Mara, Latrines Bougla Bazar Field Facilitator (Morzina) Rukhsana Union, TO‐HHN (Rashida) Rukhsana Dowarabazar Observation: Household visit by CHV and FF Rukhsana Upazila Monitoring Officer (Taslima) Rukhsana Husbands of EKATA group ‐ 12 Males Jeanne FGD VSLA – 16 Females Jeanne Chattervuj, FGD VSLA, 10 women Mike March 3 Dakshin Baradal FGD CHD FFBS, 6 women and 3 men Mike

140

SHOUHARDO III MTE Methodology Report 17 July 2018

Union, Tahirpur FGD with VDC, Female‐4; Male‐5 Golam Upazilla FGD with Youth Group, Female‐5; Male‐3 Golam EKATA Group – 15 Females Jeanne Shipa Akhtar‐ CEV Jeanne Shymal Talukdar‐ FF Jeanne Dakshin Baradal FGD with UP Chairman and Members, Female‐1; Golam Union Male ‐4 Field Supervisor ‐ Asma Rukhsana Cherar Gaon, Lactating mother ‐ Reshma Rukhsana Uttar Sreepur Observation: Counselling for Complementary Rukhsana Union, Tahirpur Feeding by new CHV (Amina) Upazila SO‐TSR ‐ Harun Rukhsana March 5 Dhaka Walter Mwasaa, SHOUHARDO III Chief of Party Mike H.J.M. Kamal, DCOP, SHOUHARDO III Golam Amitav Sarkar, Joint Secretary and Julia Moin, March 6 Dhaka Mark & Golam Deputy Secretary, LG Division, MLG Mahbub Hossain, Additional Secretary, LG Division Mark & Golam

141