World Renew and FH Terms of Reference for Program Evaluation of Brightening the future Orphan and Vulnerable Children Project (Phase 4) In three districts of , (, and )

Preamble Funded by the Canadian Food Grains Bank (CFGB) and World Renew, FH Ethiopia (FHE) has been implementing a two year Orphan and Vulnerable Children (OVC) program in three districts of South Gondar zone, Amhara National Regional State (ANRS). The project which will be closed by December 31, 2019 now requires an external evaluation. This Terms of Reference (ToR) is, therefore, prepared to help interested consultants grasp basic information and requirements about the upcoming evaluation process of the project. The ToR highlights the project description, project goal and objectives, the purpose of the evaluation, objectives, qualification for interested consultants along with roles and responsibilities of the awarder and the awardee. The ToR also outlines timeline and deliverables.

1. Background Founded in 1971, FH operates in 26 countries. FH began operations in Ethiopia in 1985, and has been implementing multi-sectoral programs across the Region. Currently, FHE manages food security, education, water and sanitation, health and nutrition, OVC projects, child focused community transformation and emergency relief and rehabilitation programs, among others. It operates in five regions: Amhara, SNNPR, Oromia, Benishangul Gumuz and Tigray with over 850 staff. FHE strives to work with communities to improve their living conditions through sustainable and participatory development programs.

World Renew is an agency of the Christian Reformed Church in North America (CRCNA), and is a Christian faith-based nonprofit working in international development and disaster response. World Renew has home offices in Burlington, Ontario, Canada, and Grand Rapids, Michigan, USA and local staff and partners working in 30 countries around the world. World Renew has enjoyed 35 years of partnership with CFGB effectively responding to hunger worldwide.

The Canadian Foodgrains Bank (CFGB) is a partnership of 15 Canadian churches and church-based agencies working in conjunction with local partners to end global hunger by supporting international programs to meet immediate food needs, reduce malnutrition, and achieve sustainable food security. CFGB is one of two primary channels for the Government of Canada’s funding for food assistance.

2. Project Description This project titled “Brightening The Future Of Orphans And Vulnerable Children” is a two year (January 1, 2018 to December 31, 2019) project funded by CFGB and World Renew, that has been implemented in three adjacent district towns (Lay Gayint/ town, Tach Gayint/Arbgebya town and Simada/Wogeda town) of South Gondar Zone of the Amhara National Regional State (ANRS). This program has been supported by CFGB and World Renew continuously since February 2007. FHE has completed 3

phases, which were similar to the current project with some exceptions. It is currently in the fourth phase and now requires an external evaluation as it draws to a close. The project’s objectives are to provide food and care support for the physical and social development of 1400 Orphans and Vulnerable Children affected by HIV/AIDS; as well as improve livelihood opportunities for their households.

The project location in general is a designated chronic food insecure zone. There has also been a high number of Orphan and Vulnerable Children (OVC) owing to the prevalence of HIV infections in the area in the past. The project was designed to be holistic in nature, focusing on four areas of support (food, education, health and psychosocial) for 1400 OVC (direct beneficiaries) in the three districts. The project also includes components to focus on stakeholder capacity building as well as livelihood development for the care-giving families through self-help groups and small-scale income generating activities.

3. Project Goal and objectives: Project goal/ultimate Outcome: Improved livelihood opportunities of household with children affected and infected by HIV/AIDS.

Intermediate and immediate outcomes of the project: 1) Maintained OVCs school performance a. Maintained adequate food consumption for OVCs b. Maintained school attendance 2) Increased income of the household with OVCs a. Increased participation of OVCs/guardians in income generating activities (IGAs) 3) Maintained involvement of local partners in OVC care and support a. Improved participation of the community volunteers in OVC care and support b. Increased participation of the Community Care Coalition members in supporting OVC

4. Purpose of the Final Evaluation The purpose of the evaluation is to assess the performance of the project including the extent to which the interventions (which include community participation, food assistance, income generating activities, education assistance, psychosocial support, and health and hygiene promotion) as well as technical support and revolving loans have attributed to improved livelihood opportunities of households with OVC affected by HIV/AIDS. Based on this critical assessment, the evaluation will provide recommendations on the way forward, identifying the most effective interventions to be focused on in the next and final project phase in order to improve the odds of sustainability.

5. Objectives of the Final Evaluation The specific objectives of the evaluation are to: 1) Critically assess and measure the extent to which the project outcomes are achieved as well as the sustainable replication of the project outcomes 2) Determine whether or not the project outcomes, indicators, activities and inputs have contributed to achieving the project goal and sustainability of the goal and final outcomes.

3) Examine the strengths, weaknesses, challenges, opportunities, and best practices of the project design, implementation and sustainability of impacts. 4) Explore whether the project addresses gender gaps in comparison to the TRAIN gender analysis done for the region; and recommend opportunities for integration of gender elements in the next project phase (ranging from gender sensitive to, transformative to gender justice).

To this end some of the overarching questions to be answered through the evaluation process are: 1) To what extent have the SACCO, community volunteers and CCC contributed to the project outcomes? How relevant and effective have the SACCOs, community volunteers and Community Care Coalitions been to ensuring project sustainability (i.e. lasting benefits)? 2) Which project activities need to be ramped up to enable sustainability of the project outcomes and goal? What are the enabling factors and barriers? 3) Has the project attributed to achieving the project goal? What factors make the project design the best solution? Which parts of the project design need improvement, redesign or termination in the next project phase? 4) How culturally, socially and economically acceptable are the project activities to the beneficiaries with respect to their values and priorities? 5) How cost efficient is the intervention in its use of resources? 6) To what extent did the intervention incorporate gender aspects into its design and implementation? What gender based impacts were seen? What opportunities are present for further gender integration?

6. Methodology The consultant/evaluation team should consider appropriate quantitative and qualitative/participatory methods in the design of their evaluation methodology. The quantitative data will be collected from samples of households and secondary sources such as periodical project reports. Using flexible or semi- structured data collection tools, qualitative information will be collected through focus group discussions, discussion with partner offices, in-depth or key informant interviews and field observation. Such discussion groups must include beneficiaries, families/guardians, Community Care Coalition, Voluntary Care Givers, Voluntary Teachers, and SACCOs. Key informants will include technical support staff, relevant stakeholder and subject experts (e.g. gender or credit and loans groups as deemed relevant).

7. Qualifications and Skills: Criteria for Selection The consultant is required to have rich research experience and technical background in designing and evaluating economic livelihoods programs. Expertise in related practices for economic livelihood improvement is essential - Savings and Loan Associations, market linkages, micro-finance, and related cooperative approaches. Practical experience on OVC managed and HIV related programs/projects is also desirable.

In reality, the final evaluation process is a dynamic process that calls for demonstrated ability to cater to the needs and achieve expected results by potential consultants. It requires strong pragmatic experience in

the field realizing the ever increasing quality demanded from such service providers. World Renew and FHE would like to invite consultants with the following minimum qualifications to take part: ● Eligible lead consultants should comprise of professional/s in the area of public health, economics, demography, sociology, gender studies or other relevant social science fields of study with proven work experience; ● A minimum of Post Graduate degree (MSc., MPH or MA.) is required; ● 5 or more years of experience in the consulting industry/ in conducting external evaluations related to livelihood improvement (Savings and Loan Associations, market linkages, micro- finance, and related cooperative approaches) and community organizing; ● Multi-disciplinary team displaying strong statistical background in quantitative and qualitative data collection, analysis, interpretation and reporting; ● Practical experience on OVC and HIV related programs/projects is preferred. ● Proven capacity to apply and integrate gender elements; ● Knowledge of community development principles and results-based frameworks; ● Understanding of the culture of Amhara Region in Ethiopia is desirable; ● Interpersonal skills and sensitivity while conducting research; ● Should submit at least three certificates for successful completion of previous assignments of similar nature which has to be provided in writing along with his/her resume (the evidence should include the areas/organizations of experience, year, legal letters of recommendations or other valuable evidence);

8. Scope of the work and key tasks and responsibilities of the consultant: As a general guide, the consultant(s) will lead and coordinate the final evaluation survey and is responsible for the whole process including designing of data collection tools, training of field data collectors, data cleaning, data entry into relevant statistical software, data analysis, interpretation and report writing. The consultant will also be responsible for providing translations of tools, discussions etc. in Amharic the regional language. The major roles under the responsibility of the consulting firm include 1) Review pertinent documents such as the OVC proposal, annual reports, baseline survey and other relevant documents to come up with information needs to be addressed through the evaluation, 2) Carry on an in-depth key informant interviews1, Focus Group Discussions (FGD)2, consultative meeting with stakeholders including: OVC, guardians/families, Community Care Coalition, Voluntary Teachers, Voluntary care givers, appropriate community and government representatives3 (at Regional, Zonal and district level), non- governmental organizations working on SACCOs, or community organizing in OVC or HIV/AIDS affected communities in the same districts, and development partner (CFGB) former representative in Addis Ababa.

1 Key informant interviews (KII) with government representatives, FH staff and other relevant NGOs to be conducted in each of the three districts/woreda as well as at the Zone level 2 Separate FGD are expected to be held in each of the three districts/woreda as well as at the Zone level for the different stakeholders. Separate FGD’s for men and women should also be held where relevant (e.g. at the household level). 3 Will be determined in consultation with FH health department

3) Conduct household surveys to collect quantitative data using representative sample sizes in each of the Woreda/district (note- the sample size is to be calculated based on 1187 beneficiary households and proportionally representative of each woreda) 4) Assess if the objectives of the program were met making sure that the information collected are in line with the project objectives/outcome and impact level indicators set in the project document 5) Assess the perception of the stakeholders about the achievements of the project 6) Determine the sustainability of the project results 7) Identify challenges and the way forward in discussion with stakeholders 8) Choose ideal software for data entry and create data entry template for the questionnaire, with which to collect quantitative data for this evaluation. 9) Recruit and train field supervisors and data collectors 10) With the objective of acquainting the data collection tool with the field staff, conduct pretest of the questionnaire to incorporate comments coming up from the process 11) Lead the field data collection while ensuring data quality 12) Enter the data into the computer system and, clean and analyze the data using statistical software and interpret and write the evaluation report. 13) Critically analyze core findings of quantitative survey, compare, contrast and explain in combination with the final quantitative survey result 14) Present preliminary findings to WR, FHE senior leadership and local staff to get input on findings and recommendations. 15) Conduct a validation workshop with project stakeholders to verify initial findings and test feasibility of potential recommendations 16) Finalize and submit the draft evaluation report which will include the final results of the analysis 17) Incorporate feedback from FHE and other stakeholders the consultant will prepare and submit the final evaluation report. 18) Keep FHE and World Renew informed for major evaluation process on a regular basis

9. Logistics World Renew via FHE will cover the cost of transportation to and from the project sites and within the study areas. The consultant will cover all other costs related to the evaluation including multiplication of the questionnaire, stationery, lodging and accommodation, hiring and training costs related to the survey. The consultant will also cover the per-diem expenses of enumerators and their supervisors. Moreover, the consultant will use his/her own computer and related facilities to write and produce reports.

10. Deliverables: In accordance with the proposed implementation schedule, the consultant shall submit the following deliverables, 1) Inception document: submitted by the consultant whose technical proposal is eligible. The report shall cover the detailed methodology and the tools including the survey checklists and

respondent categories and numbers, etc. The inception report has to be completed and approved prior to the departure to do the field work. This report should be submitted in two hard copies and electronically. 2) Presentation of preliminary findings: presented to World Renew and FHE senior leadership team and appropriate staff and incorporate relevant comments and recommendations in the final evaluation report 3) One day validation workshop: organized by FHE to present the findings to all relevant stakeholders 4) Draft report: submitted in two hard copies with an electronic copy 5) Feedback on draft report: provided by FHE, WR to be incorporated into the final evaluation report 6) Final Evaluation report: submitted in three original hard copies together with an electronic copy by the date specified in the agreement.

The final evaluation report shall be in English and the main text should be written in Times New Roman, font size 12 and 1.5 space and should not exceed 42 pages. The consultant is expected to follow this outline in the evaluation report writing: a) Cover page b) Table of Content c) Acronyms d) List of figures and tables e) Executive summary (Maximum 2 pages) f) Main text ● Introduction or background describing the project (max 2 pages) ● Description of the evaluation area (Max 1 page) ● Objective and scope of the evaluation (max 2 pages) ● Methodology (max 2 pages) ● Limitations (max 1 page) ● Overall accomplishment (max 4 pages) ● Findings narration (max 5 Pages) ● Project outcome/impacts (max 5 pages) ● Relevance, efficiency, effectiveness impact and ownership and sustainability of the project (max 5 page) ● Challenges faced (maximum of one page) ● Strength and weakness of the project (max 4 pages) ● Lessons learnt (max 2 pages) g) Conclusions and recommendations – (max 3 pages) The conclusions and recommendation should be presented based on the analysis of the relevance, effectiveness, efficiency, gender approach and preliminary impact and sustainability issues. It should clearly show the important lessons that need to be drawn for the future, and highlight the most effective activities to be incorporated into the final project phase. The

recommendations should also be feasible both technically and financially (max 3 pages). h) Annexes – no page limits

11. Guiding principles 1) The consultant is expected to abide by core values and principles as well as sign necessary codes of conduct, and code of ethics for evaluators and undertake the entire assignment within the OVC project domain. 2) FHE will be responsible for coordinating overall field level security in Ethiopia. The evaluation team will follow FHE’s security and protocols and recommendations and guidance regarding security precautions in Ethiopia. 3) Without the permission of World Renew and FHE he/she is not allowed to share any information with external bodies and all documents prepared and all data and information gathered by the consultant are the property of the clients and the consultant shall not be entitled either directly or indirectly to make use of such documents for other purposes without the prior written consent of the clients (World Renew and FHE). 4) All writings, books, articles, artwork, computer programs, databases, source and object codes, and other material of any nature whatsoever produced in the course of this assignment produced in whole or in part by the consultant in the course of his/her service to the clients (World Renew and FHE) shall be considered a work made for hire, or otherwise, and therefore World Renew and FHE property. 5) 6) Abide by Child protection and safeguarding policies.

12. Requirement for preparation of the proposal Preparation of the Technical Proposal Proposals will be evaluated according to the criteria in this ToR, and therefore the Technical Proposal should include the following information: 1) Clearly defined evaluation methodology including data collection, analysis, interpretation, reporting, work plan and structure of the evaluation team 2) A discussion of work done for similar clients, with a list of recent references 3) Attach curriculum vitae of each individual/professional team member to be assigned for the work and of the senior officer who will be responsible for supervising the team 4) Valuable evidence that clearly shows the experience of the firm/ team members regarding the subject matter

13. Preparation of the Evaluation proposal budget Submit a detailed budget for the proposal in a separate document describing the details of costs, what are expected for the execution of the final evaluation works; excluding the transportation costs, which will be covered in full by World Renew.

14. Selection process Selection of the consulting firm will be done as per World Renew’s procurement guidelines and the winner will be selected through the existing procedures via committee comprising of staff from CFGB, World Renew and FHE.

15. Timeline overview of the evaluation process: The evaluation consultant will commence the contract mid-August, 2019. The consultant will conduct data collection at the project sites during September and adjust and coordinate accordingly to accommodate for Ethiopian holidays during that time. A first draft and presentation of major findings and recommendations; validation workshop and first draft is expected no later than the third week of October and the final report will be due in the last week of October. Both soft and hard copies of the report will be submitted to FHE and World Renew for joint review.