Terms of Reference for Evaluation of PBF Country Project Portfolios

Peacebuilding Fund (PBF)

Guidance Note 5.3

Terms of Reference for evaluation of PBF country project portfolios

Terms of Reference (TOR) for evaluation of PBF country project portfolio

This TOR is a sample, which can be modified for the specific project or portfolio at hand.

I. BACKGROUND

Briefly outline the status of current UN Efforts in Support of Peacebuilding and PBF status in Country/ Region X. - What is the current conflict situation in the country?

- What are key features of Governments roadmap for peacebuilding?

- What are focus areas of bi- and multilateral interventions (UN agencies, IFIs, donors) to support governments efforts in peacebuilding?

- What is the justification for PBF country engagement and strategic entry points?

II. PURPOSE AND USE OF EVALUATION

Define the purpose and use of the evaluation.

- Why this evaluation at this point of time?

- Is the main purpose of the evaluation learning or accountability? - Will the evaluation be used to decide on future funding? Or to provide input to new strategy at country level (e.g. second PBF tranche) or global level for better strategic positioning?

- Programme specific: amount of funding, and risks tied to the intervention.

- Are there any specific events that have triggered the evaluation (unveiling of corruption, results that run counter to intentions in the intervention, new research being released, the conflict situation is worsening in country)?

III. OBJECTIVES AND SCOPE OF EVALUATION

Outline the specific objectives and proposed scope of the valuation. These should be consistent with the purpose and uses outlined above. They must also be realistic and consistent with available funding/ situation on the ground. Evaluation objectives may include assessment of:

- the status of results achievements at mid-term or the end of funding ?

- some or all of the project activity's objectives?

- implementation strategies, processes and performance of fund users?

- the programme’s underlying assumptions and theories of change?

Which OECD DAC evaluation criteria are to be used in the evaluation:

- impact, relevance, sustainability, efficiency, or effectiveness, ownership of results?

What are additional areas of interest that should be assessed in the evaluation?

- PBF strategic positioning at country level

- PBF’s added value for national efforts to conaolidate the overall peacebuilding processes

- PBF organizational performance to respond quickly to country request

- Urgency of gaps (financial and / or peacebuilding gaps)?

- Catalytic in terms of additional financial commitments (either from donor or national governments) or unleashing peace relevant processes? - Risk taking / innovative approaches?

SCOPE, TIMEFRAME AND OBJECTIVES OF PROJECT/PORTFOLIO TO BE EVALUATED

Provide a brief description of the project/ portfolio of projects which is/are being assessed.

- Specify issues to be covered, budget and funds spent,

- the time period to be evaluated,

- types of activities,

- geographical coverage,

- target groups, as well as other elements of the conflict prevention and peace building intervention addressed, such as contextual issues.

- Specify scope of any existing programme reviews, self-assessments of the implementation status, reporting cycle (half year, annual), major findings (internal and external data sources).

IV. EVALUATION METHODOLOGY/ APPROACH:

- Specify what method will be used in the evaluation? How should the evaluation be conducted, via what process and steps, etc.? Will there be an inception phase? What will the level of stakeholder involvement in the evaluation process be?

- Undertake a thorough conflict analysis and then draft an inception report describing how they will answer the key evaluation questions.

- Triangulate available data sources (programme’s self-evaluations, donors’ and country reports from PBF and MPTFO (via gateway). - Consult with national authorities, funding recipient and implementation agencies (or JSC), programme staff, donors and visit each project site for focus group discussions with the final beneficiaries.

- Start and end with a workshop to brief these parties on the purpose and objectives of the mission as well as on the findings at the end to ensure high ownership and full transparency of the whole evaluation process.

- Address ethical behavior in conflict environments and provide guidance on safety and logistics. Due to safety concerns, the evaluation team will conduct field visits in Regions x, but not in Region x. For Region which cannot be visited, evaluators will instead meet with proxies in the capital and collect data from a recent study xxx carried out in regions xx. The visit should take place during xx.

V. EVALUATION PRINCIPLES AND STANDARDS:

- PBSO will be responsible to provide relevant standards and principles to the team leader of the evaluation mission.

- The team leader outlines in his inception report the standards applied for the evaluation process in particular for the conduct of perceptions surveys.

VI. MANAGEMENT ARRANGEMENTS AND QUALITY ASSURANCE PROCESS.

- Who will be in charge of managing each task and providing oversight to the evaluation? Who will the evaluation team report to? Is there a need to establish a steering mechanism for the evaluation? Who will be responsible for ensuring information sharing among team members? Who will be involved in drawing and assessing conclusions? Will they be public or confidential? Will they be published or placed on the internet?

- Will the reports and conclusions be checked? What quality control systems will be used? VII. EVALUATION TEAM COMPOSITION AND REQUIREMENTS

- What will be the team composition for the evaluation? What expertise do they need to have? What is the desired size and composition of the team? What time commitment is involved in terms of person-hours? What types of individuals are needed for this particular evaluation in this particular context?

- The team should include expertise in both conflict analysis and assessment as well as in the specific thematic areas of the PBF country engagement. They must have a solid experience in the evaluation of programme effectiveness within the peacebuilding context. There should be a team lead (40 days), at least two national experts (30 days each) and up to two research assistants (30 days total).

VIII. BUDGET AND SCHEDULE.

- How will the evaluation be funded? Have there been arrangements made for security costs or other additional costs associated with working in a conflict environment? Are funds available for conflict analysis? (Bids may also be accepted and then compared to establish the appropriate funding needed.)

- When will the evaluation be conducted? When will it start and when is the final report expected?

- What criteria will be applied to disburse funding?

- Table showing team composition, number of days, and schedule.

IX. KEY EVALUATION DELIVERABLES

- List what the key deliverables will be and when each is due.

- The consultants will produce the following:

 Preliminary assessment (Project Fact Sheets). PBSO will provide the basic inputs into the Project Fact Sheets, with assistance from the national M&E expert at country level. These will be reviewed by the Team Leader prior to the commencement of field visits and discussions with the RUNOs.  Inception Report: based on a desk review of the relevant documentation, including the Project Fact Sheets, a concise inception report will be developed by the evaluation team (max 5 pages). It will outline the proposed evaluation steps and provide a detailed description of the envisaged methodology, its parameters, assumptions and an explanation as to why this is the most appropriate way forward. The report will also include a work plan with associated timetable, sites to be visited and will be presented to the PBSO for comments and approval.  Inception meeting: prior to conducting the field visits, the team will meet with the focal persons of the participating UN organizations to discuss the process, methodology and questions/issues to be address in the evaluation  Debriefing Note: prior to the Team Leader’s departure from xxx, a debriefing will be given to the RC/UNCT, national counterparts, implementation partners and donors to review a first draft of conclusions and recommendations. A short two to three page debriefing note and power point presentation will support the verbal presentation.  Draft Report: the draft report will be submitted to the PBSO, who will be responsible for co-coordinating the feedback process with the recipient agencies. Expected deadline: xxxx  Final Report: the final report will be approved by PBSO in close consultation with the UN RC office after reception of comments from the country team. The teamleader will be in charge to incorporate to the largest extent possible the comments from the PBSO, the Government and the UN Country Office, while preserving his independent views as an evaluator. The response from the UNCT should be part of the Annex. The report should not exceed 30 – 35 pages, not including appendices or the Executive Summary. It should provide lessons learnt and a clear evidence basis for its conclusions; all recommendations should be actionable. The independent team will have editorial control of the final report. Expected deadline: xxxx.

- The outline of the evaluation report must be compliant with the following format, and will be submitted to Government, PBSO, and the United Nations, containing at least the following information:  Executive Summary of the evaluation findings (max 3 pages; with reference to: Purpose of the evaluation, findings on overall programme relevance and effectiveness, recommendations and lessons learnt)  Background  Assessment of conflict drivers / causes to be addressed  Assessment of theory of changes and expected results (for the entire portfolio and per sector intervention)  Key Achievements of the PBF supported projects against OECD/DAC evaluation criteria for the overall portfolio and per PBF supported project. Particular attention will be paid to PBF’s responsiveness to urgent funding gaps, risk taking, catalytic effects and, overall cost effectiveness / value for money)  Challenges and Lessons Learnt  Recommendations on how to improve programme effectiveness.  Annexes: Individual project fact sheets to assess project based contributions to peacebuilding process (see PBF standard; reference to project relevant PMP indicators and OECD/DAC evaluation criteria with particular attention to cost effectiveness)  One or two concrete stories or examples of success and failure

For the desk review, the following documents will need to be reviewed:  PBF project documents (Business Plan, PMP, Annual Report SG 2011, documents related to measuring catalytic effects; relevant government policy documents; PBF Progress reports submitted by the UN Recipient Agencies via MPTFO gateway).  Relevant other background documentation (e.g. conflict analyses; performance reviews etc)