TERMS OF REFERENCE
For the External Evaluation of ACF’s
Humanitarian assistance to the populations affected by the Syrian conflict in Syria and the neighbouring countries
Programme Funded by ECHO
Contract Reference
ECHO/SYR/BUD/2013/91039
This version of the ToR is subject to change and must be validated by ECHO prior to signing of the contract
22-09-2014
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
1. CONTRACTUAL DETAILS OF THE EVALUATION
1.1. Key Evaluation Dates
Expected Start Date: 20-10-2014 End Date: 30-11-2014 Submission of Draft Report 25 Calendar days after start of the evaluation Submission of Final Report 10 Calendar days after revision of the draft report
1.2. Language of the Evaluation
Language Requirements for the Evaluation: English Language of the Report: English
1.3. Workplan & Timetable
Activities Working Days Preparatory works and final work plan with ACF-UK HQ 2 Develop evaluation tools and methodology 2 Briefing and evaluation framework analysis with ACF-Spain Technical Advisors 1 Briefing with ACF Lebanon/Syria country team 1 Project data collection and analysis 2 Field Visits in Lebanon and Key Informants Interviews 4 Visit to ACF Syria and Key informants Interviews tbc 2 Data analysis and draft of the evaluation report 8 Learning Workshop 1 Submission of final report 10 Total 33
1.4. Budget for the Evaluation
The consultant is responsible for personal insurance during the evaluation. The consultant will also provide any necessary materials (including laptops) required for the evaluation.
2. DETAILS OF THE PROGRAMME
Name of the Programme: Humanitarian assistance to the populations affected by the Syrian conflict in Syria and the neighbouring countries Location: West Bekaa, Central Bekaa, South Lebanon - Lebanon Syria (section 2.1) Ketem and Ne’Ayymeh villages, Bani Obaid District, Irbid Governorate –Jordan (operations closed July 2014) Starting Date: 1st of October 2013 End Date: 31st of October 2014 (including 3 months NCE)
2.1. Map of Programme Area
Page 2 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
Syria: Map –project locations
Kindly refer to Annex Kindly refer to Annex 1. Table on ACF intervention in Syria
Lebanon:
West Bekaa: El Marj, Houch Harime, Istabl, Rachaya, Kirbet Rouha, Mdoukha, Rafid, Akaba;
Page 3 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
Central Bekaa: Aali en Nahri, Faour, Fayda, Majdel, Anjar, Rayak, Taalabaya, Terbol; South Lebanon: Tyre, Nabatiyeh
2.2. Programme Overview ACF is implementing the Action “Humanitarian assistance to the populations affected by the Syrian conflict in Syria and the neighbouring countries” in Syria, Lebanon and Jordan.
The Action aims to “contribute to save lives and provide a life with dignity to the affected populations of the Syrian Crisis” in the targeted areas. It was started on 1st October 2013 and will continue until 31st of October 2014 based on the 3 month NCE submitted and approved by ECHO.
Syria
Action in Syria is being implemented in partnership with SARC (Syrian Arab Red Crescent), Ministry of Water Resources (MoWR) and in a closed coordination with MoLA (Ministry of Local Administration) and MoE (Ministry of Education). ACF still in the process of signing MoU (Memorandum of Understanding) with MoLA and renewal of MoU with MoE. So far, ACF Syrian mission was able to get written approval from MoLA to conduct rehabilitation in seven collective shelters and approval for well drilling in 7 schools of Hassakeh governorate from DoE while waiting for MoUs to be signed.
ACF started its activities in the Syria in year of 2009 through a humanitarian assistance to Iraqi refugees in North Easter part of Syria in a partnership with SARC and MoE. To date, ACF has implemented number of Actions, mainly in Food Security, Water and Sanitation, livelihood, Vocational training provision, shelter and schools rehabilitation, both in emergency and development Actions. ACF received funding from DG ECHO for the intervention in the Syria in 2012 and 2013 and as of July 2014 ACF Syria mission is implementing five actions funded by different donors (ECHO, SDC, AECID, DFADT, and UNICEF). This Action aims to improve the living conditions of the IDPs, IDP/refugee returnees and host families and communities affected by the crisis in Syria through immediate relief assistance provision. Immediate provision of basic relief assistance in food basket, WASH services, NFI and small scale rehabilitation of shelter facilities and schools will increase the coping mechanism of the affected people and the access to basic services that are limited and/or not available due to the crisis. ACF is working closely with SARC, MoWR, MOE and MoLA ensuring that all activities follow the country strategy, respect their requirements, whilst also ensuring the needs of the beneficiaries are met and that the regulations of the donor agency are followed.
Location Identification of locations for this action is coordinated with SARC and relevant Ministries (partners of ACF) based on the needs/gaps identified during the surveys and by relevant sector stakeholders. MoLA is in charge of shelter related activities and coordinating with stakeholder, including identification of locations, assessment, identification of shelters needs and planning. ACF coordinates with MoLA and re-validate the shelter needs based on the available information. MoE is in charge of the schools related activities and already identified damaged schools. ACF coordinates and develops join plan to identify priority schools for rehabilitation. SARC is the organization in charge of registering the IDPs and is the main body coordinating the distribution of
Page 4 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference relief items retains all data and records on the families having benefited from the different distributions and the needs based on the locations. Therefore, ACF in a partnership with above mentioned agencies are involved in the selection of location and identification of beneficiaries. The following governorates are targeted: Hassakeh, Deraa, Tartus, Alleppo, Rural Damascus, Homs, Hamas, Idlib. ACF is keeping flexibility towards location due to security context and access.
Beneficiaries1 Based on the Modification request submitted to donor agency, new estimated number of targeted people in Syria is 500.000 individuals which are counted as direct beneficiaries under this action. The action is targeting IDPs/affected people and/or returnee IDPs or refugees. The following table shows the estimated number of direct beneficiaries in each of the following sector targeted: Sector Beneficiaries WaSH (including water trucking) 300,000 beneficiaries Food Security 150,000 beneficiaries Shelter/NFI 50,000 beneficiaries
An estimated 1,045,000 individuals are expected to indirectly benefit from the: Improved access to safe and adequate water supply, environmental sanitation and hygiene practices Improved access to shelters and schools
Partners involved in the Action The following partners and governmental agencies are involved in to the action: SARC (Syrian Arab Red Crescent), MoWR (Ministry of Water Resources) -DoWR (directorate of Water Resources) at the governorates level MoE (Ministry of Education) -DoE (Directorate of Education) at the governorates level MoLA (Ministry of Local Administration) _so far there is no MoU signed, but good cooperation and coordination has been established -DoTS (Directorate of Technical Services) at the governorate level
Lebanon
In Lebanon, ACF ran a full WaSH assessment in June 2013. The sample identified included 3908 households in tented settlements in North, Central and West Bekaa Valley, 845 HH in houses, unfinished houses and collective shelters. A specific feasibility study on potential CfW activities was also conducted in South Lebanon. The methodology included individual interviews with stakeholders, secondary data review and focus group discussions with resident refugees.
The main findings from the assessments outlined significant gaps in WaSH response: over 29% of refugees living in tented settlements reported water borne diseases, 21% of beneficiaries did not
1 Please note that beneficiary numbers refer to the entire Action/project
Page 5 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference had access to drinkable water and 78% did not had access to adequate sanitation facilities. In South Lebanon, refugees were facing lack in livelihood opportunities, limited access to unconditional cash programs with over 70% of the HH interviewed living on a minimum expenditure basket.
To tackle the identified needs, ACF proposed to improve living conditions of Syrian Refugees in Lebanon settled in vulnerable conditions through provision of water, sanitation and Hygiene, and through set up of cash for work opportunities (Result 2).
A comprehensive package of WaSH Relief assistance for Syrian Refugees in the most vulnerable identified locations was set up. Activities encompassed provision of 1) water trucking, water tanks, water filters, water storage items enhanced by permanent water analysis; 2) Provision of emergency latrines hand washing points, cleaning kits, soak away pits, garbage containers, support to garbage sorting, dislodgment services and hygiene kits distribution.
Furthermore, taking in consideration the level of vulnerability of particular host communities, and the stress suffered from the Refugee influx, ACF has decided to cope with lack of access to water for all affected populations (including Host Communities) by supporting water infrastructure works increasing accessibility in particular affected municipalities. For that purpose, ACF has coordinated with Municipalities and respective Water Establishments to assess the areas of intervention.
In the South, Refugees and Host Communities have benefited from regular Cash for Work activities, augmenting the livelihoods of vulnerable refugees, as well as improving living conditions of Host Communities. The scope and planning of CfW activities were discussed and validated by the Municipal authorities and MoUs have been signed to promote stability and define obligations of parties. Furthermore, ACF has piloted an Income Generating Activities program in the South of Lebanon intended to target Women in particular.
Beneficiaries
The total number of direct beneficiaries for both regions is 39.000. The provisional number of direct beneficiaries reached was: 16.000 in Bekaa Valley (West Bekaa, Central Bekaa, Rachaya) benefiting from the WaSH and Hygiene Promotion service provision in Refugee Informal Settlements, 20.000 beneficiaries benefiting from the WaSH Infrastructure constructions, and about 3.000 in South Lebanon (Tyre and Nabatyieh) from the Food Security and Livelihood Promotion program;
Beneficiaries from the WaSH direct assistance were chosen in coordination with UNHCR and Sector Working Groups in order to use resources efficiently and avoid overlapping. During 2013 and partially 2014, refugees in informal settlements would benefit from blanket assistance for WaSH services according to Country Minimum Standard in Humanitarian Assistance (defined by Wash Working Group); Furthermore, information was cross-checked on the basis of the guidelines provided by WFP and UNHCR under the Vulnerability Assessment Guidelines June 2013. For the CfW component the beneficiaries were selected according to the WFP’s scoring system, obtained from an initial vulnerability assessment. All non-skilled labour participants were selected in coordination with the Local Authorities, with the aim of maximizing visibility whilst minimizing risk to the municipalities.
Partners involved on the Action: - UNHCR as coordination agency;
Page 6 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
- INGOs and Local NGOs; - Municipalities; - Bekaa Water Establishment; - ECHO (Donor)
Under the contract agreement between ACF and the donor, European Commission Humanitarian Office (ECHO), it was agreed that ACF will conduct an external evaluation at the end of action. The external evaluation will cover the 13 months of the implementation period from October 1st, 2013 to October 31st of 2014. The main purpose is to evaluate the achievement of the objectives to date and propose recommendations for future interventions in the region and specifically considering complexity of the context.
Jordan
ACF has closed its operations in that particular operational area (with no presence and no team) since end of July 2014. Given that ACF is no longer present in the area, the modality of the evaluation would be one of interviews with the technical coordinators involved in the implementation of the programme, review of operational data and reports, interview with ECHO Jordan team and if possible some interviews with other stakeholders in the operational area (other NOGs, beneficiaries, etc).
2.3. General Objective
To contribute to save lives and provide a life with dignity to the affected populations of the Syrian crisis.
2.4. Specific Objectives/Results
Specific Objective: To improve the living conditions of displaced, refugees and hosting communities affected by the Syrian crisis;
Result 1 (Syria): Living Conditions of the IDPs and Host families and communities affected by the crisis in Syria are improved through a better coverage of basic needs;
Result 2 (Lebanon): Living conditions of Syrian Refugees in Lebanon settled in vulnerable conditions are improved through provision of water, sanitation and hygiene and through set up of cash for work opportunities;
Result 3 (Jordan): Living conditions of Syrian Refugees and vulnerable Jordanians are improved through financial support, wash infrastructures rehabilitation and psychosocial support;
2.5. Programme Activities Result 1 Activity 1.1 Procurement, logistic arrangement and distribution of food basket
Page 7 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
Activity 1.2 Procurement, logistic arrangement and distribution of NFI and winterization kits Activity 1.3 Rehabilitation and/or repair of collective and/or private shelter sites, host family shelters and schools Activity 1.4 Rehabilitation and/or repair of collective and/or private shelter sites, host family shelters and schools Activity 1.5 Rehabilitation and/or repair of water and sanitation facilities at collective and/or private shelter sites and community level Activity 1.6 Support to shelter management/maintenance (Public, private and host family)
Result 2 Activity 2.1: Provision of water trucking, water tanks, water filters, household water storage items, water analysis; Activity 2.2: Provision of emergency latrines, hand washing points, latrine cleaning kits, soak-away pits, garbage containers, support to garbage sorting, dislodgement services, hygiene kits distribution; Activity 2.3: Rehabilitation works in selected municipalities; Activity 2.4: Hygiene promotion sessions, nutrition and feeding practices & drama based activities etc distribution of IEC material, organization of hygiene related events with different audiences Activity 2.5: Set up of cash for work activities;
Result 3 Activity 3.1 Continuous identification and registration of the households in need of livelihood provisioning; WASH access improvement and/or psychosocial support. Activity 3.2 Provision of monthly cash grant contributing to reduce pressure on the household economy during winter months Activity 3.3 Rehabilitation of water and sanitation facilities and promotion of water management and hygiene adapted practices Activity 3.4 Psychosocial support through group sessions and recreational activities Activity 3.5 Development and implementation of ongoing / continuous monitoring mechanisms on processes and impact on targeted households and communities to readjust activities.
3. AIM OF THE EVALUATION
3.1. Target User(s) of the Evaluation
ACF Implementing HQ WaSH Advisor, Nutrition Advisor, FSL Advisor, Actions Officer, Quality Officer, Desk Officer, Operations Director. Field Level Country Directors, Program Coordinators (FSL, WaSH) and action officers, Technical experts, Emergency action manager Other ECHO
3.2. Objective(s) of the Evaluation
The objective of the evaluation and planned results will enable ACF to review the action design, implementation strategy and measure the action impact, draw lessons learnt to support the definition of future proposals.
The analysis and findings of the evaluation should be based on a review of action documents, Key Informant Interviews and site visits, supported with substantial information/data.
Page 8 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
Objectives:
1. To evaluate the quality, pertinence, relevance and appropriateness of the action taking in consideration the needs analysis, participation mechanisms and project design; 2. To assess ACF capacity to respond to the emergency and accomplish the proposed results; 3. To assess the impact of the response delivered as well as the lessons learnt; 4. Provide practical and realistic recommendations to improve ACF emergency response capacity; 5. Evaluate how ACF is able to maintain its principle and mandate throughout the action implementation and increase coverage (scale, sector, outreach, support in cross line areas etc.)
The evaluation is expected to come out with analysis based on a set of criteria defined under the Scope of the Evaluation (Section 3.3). The local context (political and security context, cultural values) will be taken into account throughout the evaluation.
3.3. Scope of the Evaluation
The topline questions (in bold below) will serve to focus the evaluative work on those 5 areas. The evaluator will be required to produce an inception report during the preparation stage in which they will detail what questions they will pursue in order to comprehensively cover the questions and additional points below. This will be shared with the evaluation manager (ELA Unit), the evaluation focal point (ACF-Lebanon & Syria) and the desk (ACF-Spain), who will provide feedback to ensure all necessary areas are covered. 1. Quality of the Action
a) Was the action design appropriate to contribute to the objectives and achieve the proposed results?
When determining the quality of Action design, the following should be taken into account:
The quality of the Action preparation and design; Level of participation of relevant stakeholders and beneficiaries in the preparation and design; The quality of the internal logic of the Action design. How practical and consistent are: the activities in order to contribute to the intended results, the results to contribute to the specific objectives, the purpose to contribute to the overall objectives, the indicators and finally how practical and consistent are the risks and assumptions identified.
2. Impact
The impact evaluation should examine and evaluate the contribution of the action to the overall objective, i.e., the effect of the Action on the wider sector objectives (as summarized in the 2.3 Overall Objective and 2.4 Specific Objective parts), and on the achievement of the objectives of the donor’s sector/priorities in Food Security, Water and Sanitation, Shelter/NFI. The evaluator should take into account the unintended impact aside from those anticipated in the intervention framework.
The following key questions should be taken into account when assessing action impact:
Page 9 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
a) Are the action indicators relevant to analyze data or measuring impact? b) Intended impact: What is the contribution of the Action to the overall objective and specific objective; c) Unintended impact: What unintended (positive or negative) consequences did the action provoke? (E.g. impact on other sectors, on donor policy objectives) Why did they arise?
The impact of the Action should be evaluated (at all levels) in relation to all the components of the Actions within the Syria, Lebanon and Jordan context and within the existing policies, restrictions and provisions.
3. Sustainability
The sustainability of the Action should be assessed and answer the following:
b) Will there be (and why) continued positive outcomes as a result of the action? c) Technical sustainability: Is the response methodology being used/promoted by the action justifiable? d) Institutional sustainability: Is the Action supported by local institutions? Are capacity- building activities being effectively carried out? Is there a plan for the phase out strategy and the handover of any management responsibilities? e) Cultural and social sustainability: Is the Action well integrated with local social and cultural conditions? Are requirements of local ownership satisfied? f) Environmental sustainability: Is the intervention compatible with a sustainable use of natural resources? Is it harmful to the natural environment? g) Was the assistance provided in a way that took account of the longer-term usage?
4. Relevance / Appropriateness
This criterion is to determine how appropriate the Action is to beneficiary needs as well as national and donor priorities. The following should be addressed and assessed during the evaluation:
a) Were the actions undertaken appropriate in the context of the needs of the targeted population and the context of the situation? Are there still basic needs to address? b) Was the assistance appropriate in relation to the customs, practices of the affected population? c) Were the activities appropriate for each level (national, governorate, households, Schools and Shelters)? d) Were stakeholder’s recommendations integrated prior to the action and comment/feedback are followed during the implementation? e) Does the Action support/contribute to the implementation of the national/governorate governmental priorities? f) The priorities, objectives and guiding principles of the Donors were followed? g) Were the most vulnerable people targeted or did they indirectly benefit from the activities? h) Were all stakeholders including beneficiaries and non-beneficiaries aware of action objectives and targeting criteria?
5. Coherence
Examine and evaluate whether the components of the Action were optimally integrated and are coherent with the overall country response;
Page 10 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
a) What steps were taken by ACF and partners to ensure the coordination and integration of the different components of the action in their own areas of responsibility? b) Did ACF ensured coordination and integration of activities undertaken by several sector stakeholders intervening in common areas? c) How the Action design took into consideration the previous action in the current areas of intervention?
6. Efficiency
Evaluate if the Action resources are used in the best way possible. What could be done differently to improve implementation for an acceptable and efficient cost? Have the planned activities been achieved? Has the Action obtained the planned results? At least the four following aspects should be taken into account while determining the efficiency of an Action:
a) Timeliness: Are the means being provided on time? Are the activities being implemented on time? Is the time consumed reasonable and proportional to the results obtained? b) Financial progress: Are the means being provided within budget? Is the cost of the activities reasonable and proportional to the results obtained? Are the accountability systems adequate in identifying/deterring corrupt practices? c) Physical progress: Is the relevant information on Action results (OVIs) being collected and used? d) Is the Action reaching its measurable results, according to the OVIs? e) Quality progress: Assess the quality, expertise and capacity of work of the Action staff in the sector, both consortium partners and their local partners involved in the implementation? Is the quality of the means good enough? Is the quality of the results good enough? f) Has the action implemented and achieved its results are targets following the standards/procedure agreed and established for the action?
7. Effectiveness
The examination of the effectiveness of the Action should answer the question: are the Action results successfully contributing to the achievement of the overall Action objective?
This should include the assessment of any evidence that the Action’s intended beneficiaries are indeed benefiting from the Action. To do this, various questions need to be answered by checking the purpose OVIs to see how the Action is performing in terms of accomplishing them. The following assessments should be made:
a) Comparison of the actual schedule and completion of the activities compared to the initial plan; b) What were the reasons for any deviation from planned activities and how could this be improved for future interventions? c) What has been the effect of deviations from the planned Action? d) Have the expected benefits been delivered?
8. Involvement / participation
The analysis of involvement/participation should take in consideration the level of engagement of different stakeholders from the assessment stage until project closure.
a) How the different responsibilities and duties have been identified with National Partners?
Page 11 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
b) How are national partners implementing the activities/responsibilities assigned? Are they responsive enough and have the technical capacity to implement the action activities? c) What was the level of national stakeholder’s participation in Action design, implementation and monitoring?
9. Monitoring
a) How ACF monitors and evaluates the progress and impact of the Action? b) How adequate are the monitoring tools and systems in place? c) Is the monitoring approach participative? Are the feedback systems in place? d) How have lessons learned from monitoring and evaluation been incorporated into the Action?
10. Cross-Cutting Issues/Themes
Assess and analyze how the Action has performed in addressing the cross-cutting issues/themes identified to be mainstreamed during implementation, such as:
a) The action addressed the needs of most vulnerable groups such as elderly and single women headed families? b) Were the activities implemented to address the living condition of infants and elderly to cope with the crisis and cover the basic needs?
3.4. Evaluation Criteria
ACF subscribes to the Development Assistance Committee (DAC) criteria for evaluation: Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and Efficiency. ACF also promotes systematic analysis of the monitoring system and cross cutting issues (gender etc.) External evaluations are not expected to use DAC criteria to structure the report but should adhere to the standards in data analysis and reporting. Nevertheless in every case the evaluator will be expected to use the following table to rank the performance of the overall intervention using the DAC criteria. The table should be included as an annex to the report.
Criteria Rating Rationale (1 low, 5 high) 1 2 3 4 5 Impact Sustainability Coherence Coverage Relevance/Appropriateness Effectiveness Efficiency
3.5. Best Practices
Page 12 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
The evaluation is expected to provide at least one (if not more) key example of Best Practice from the action/Action. This example should relate to coordination processes or systems (including at Consortium Coordination level) and / or to innovation and / or new technology promoted amongst the partners and adapted to the intervention context and should be potentially applicable to other emergency contexts in the country. The examples of Best Practice should be presented in the Main Body of the report and/or the annexes using the following table:
Title of Best Practice (max 30 words)
Innovative Features & Key (What makes the selected practice different?) characteristics Practical/Specific Recommendations for (How can the selected practice be replicated more widely?) Roll Out
3.6. Evaluation Outputs
The result of this evaluation should be presented in a written final evaluation report.
Debriefing with partners and Donor at Lebanon and Jordan: One with Donor representative One with ACF mission representatives (Country Director and Technical coordinator) Debriefing with Donor representative if requested;
3.7. Methodology
3.7.1. Briefing
Prior to the evaluation taking place, the evaluator is expected to attend a briefing at HQ level, and at field level (Lebanon possibly due to limited access to Syria) with the Country Director and/or the relevant technical focal point. Briefings by telephone must be agreed in advance.
3.7.2. Field activities
Consultants are expected to collect an appropriate range of data. This includes (but not limited to): Indirect information: Interviews with local representatives; interviews with action staff expatriate and national staff; humanitarian agencies, donor representatives and other stakeholders. For indirect data collection, standard and participatory evaluation methods are expected to be used. Considering context of Syria, interviews can be conducted by phone, skype and meetings organized in Lebanon. Secondary information analysis: including the following documentation, which will be provided to the consultant upon signing of the contract: Action proposals, annual plans and annual reports Action Activity progress reports, technical assessment reports, surveys reports and other studies conducted under the Action MoUs / approval letters, contracts, Minutes of meetings. Action monitoring data or of any other relevant statistical data
3.7.3. Report
Page 13 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
The report shall follow the following format. Cover Page Table of Contents Executive Summary: must be a standalone summary, describing the Action, main findings of the evaluation, and conclusions and recommendations. This will be no more than 2 pages in length. Main Body: The main body of the report shall elaborate the points listed in the Executive Summary. It will include references to the methodology used for the evaluation and the context of the action. In particular, for each key conclusion there should be a corresponding recommendation. Recommendations should be as realistic, operational and pragmatic as possible, considering the context of the country; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally and in the commissioning HQ. Annexes: Listed and correctly numbered. Format for the main body of the report is: o List of Acronyms o Contents o Background Information o Methodology o Evaluation Challenges and Limitations o Findings & Analysis of Action Results o Lessons Learned and Recommendations o Conclusions o Annex I (Best Practices) o Annex II (DAC-based Rating Table)
The report should be submitted in English. The report should not be longer than 35 pages (excluding annexes). The draft report should be submitted no later than 25 days after start of the evaluation. The final report will be submitted no later than the end date of the consultancy contract. Annexes to the report will be accepted in the working language of the country and Action subject to the evaluation.
3.7.4. Debriefing & Learning Workshop
The evaluator should facilitate a learning workshop (in Beirut or Amman):
To present the draft report and the findings of the evaluation to the ACF Mission and other national stakeholders and donor representative at regional level. To gather feedback on the findings and build consensus on recommendations. To develop action-oriented workshop statements on lessons learned and proposed improvements for the future.
3.7.5. Debriefing with ACF HQ
The evaluator should provide a debriefing with the relevant ACF HQ on her/his draft report, and on the main findings, conclusions and recommendations of the evaluation. Relevant comments should be incorporated in the final report.
3.7.6. Rules of Engagement
Page 14 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
The basic prerequisite of the ethical and professional behavior of the evaluation team will be as follows: Anonymous and confidentiality – The evaluation should respect the right of the person to collect information ensuring anonymous and confidentiality. Responsibility – Any disagreement or difference in the opinions that could occur among the evaluation team members or between them and the intervention responsibilities in relation with the conclusions and / or recommendations will be mentioned in the report. Any affirmation should be supported by the team or leave evidence of the disagreement of the team. Integrity – The evaluators should take responsibility to ask questions not mentioned specifically in the ToR if necessary to get a more complete analysis of the intervention. Independence – The evaluation team should guarantee its Independence from the evaluated intervention, not being involved in its execution or management or any other of its component. Incidence – In case of any problem during any phase of the evaluation, ACF will be notified immediately. Validations of the information – The evaluation team guarantee the truth of the information compiled for the report writing and is final instance are responsible of the information presented in the evaluation report. Evaluation report – The release of the compiled information and of the final report is ACF and ECHO’s prerogative. Submission of reports – In case of delay in the reports submission, or if the quality of the reports is poor, penalties forecasted in the contract clause will apply.
In any case, ACF and ECHO reserve the right to run the evaluation and/or decide on its distinct aspects.
4. PROFILE OF THE EVALUATOR
Knowledge in WaSH, FSL Emergency Actions; Significant field experience in the evaluation of humanitarian emergency response; Significant experience in coordination, design, implementation, monitoring and evaluation. Good communications skills and experience of workshop facilitation; Ability to write clear and useful reports (may be required to produce examples of previous work); Fluent in English; Experience working and /or good understanding of Syria crisis context, including Lebanon; Understanding of donor requirements; Ability to manage the available time and resources and to work to tight deadlines; Independence from the parties involved.
5. RIGHTS
The ownership of the draft and final documentation belong to the agency (ACF) and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except ACF before the delivery by ACF of the final document to the donor.
ACF is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. ACF is likely to share the results of the evaluation with the following groups: Donor(s)
Page 15 of 16
ACF Evaluation Policy & Guideline Annex I. Terms of Reference
National Governmental partners Various co-ordination bodies
Intellectual Property Rights All documentation related to the Assignment (whether or not in the course of the evaluator’s duties) shall remain the sole and exclusive property of ACF.
List of acronyms: ACF _ Action Contra el Hambre_Spain (ACF International) AECID_ Spanish Agency for International Cooperation and Development DAC _Development Assistance Committee DFADT_Department of Foreign Affairs, Trade and Development DoE _Directorate of Education DoTS _Directorate of Technical Services DoWR _Directorate of Water Resources ECHO_European Commission Humanitarian Aid and Civil protection Office FSL _Food Security and LivelihoodHQ_ Headquarters IDPs_ Internally Displaced Persons MoE _Ministry of Education MoLA _Ministry of Local Administration MoU_ Memorandum of Understanding MoWR _Ministry of Water Resources NCE-No Cost Extension NFI_ None Food Items OVIs_Objectively Verified Indicators SARC _ Syrian Arab Red Crescent SDC_ Swiss Agency for Development and Cooperation SY_Syria ToR_Terms of reference UNICEF_The United Nations Children Fund WaSH _Water Supply and Hygiene
Page 16 of 16