TERMS OF REFERENCE Development of a Monitoring and Evaluation Framework for the SMART Health Care Facilities in the Eastern Caribbean Project Phase II

1.0 Introduction

The Smart Health Care Facilities in the Eastern Caribbean Project Phase II supported by the UK Department for International Development (DfID) will be implemented in four (4) selected countries namely Grenada, Saint Lucia, St. Vincent and the Grenadines and Dominica over a period of three (3) years commencing May 2015.

The aim of the project is to support the Governments of the most vulnerable Caribbean countries to assess and prioritize vulnerability reduction investments of their health facilities. Health facilities will all be assessed for disaster safety, water and energy improvements providing a road-map for investment and results incorporated within the national risk exposure database of each country. At least 12 health facilities (3 health facilities in each of the selected countries) will be retrofitted to help reduce the downtime and potential damage to the facility in the event of a disaster as well as reduce operational expenditures with water and energy management improvements and ensure energy auto-sufficiency to support continuity of healthcare delivery in the event of a major disaster.

The project is based on a highly successful pilot initiative, Phase I of the SMART Health Care Facilities in the Caribbean. In Phase II, PAHO will continue to work in partnership with the Ministry of Health (MOH) in each target country. The organization, through its long standing presence and technical cooperation with the countries of the Caribbean and its record of major success, including the Safe Hospitals initiative which was launched in the Region and is now a global programme and the SMART Hospitals Phase I, has built credibility and a great working relationship with its member states.

As part of the project, a monitoring and evaluation framework will be developed for the three year project period to fulfil both learning and an accountability purpose.

The primary recipients of services will be PAHO and the WHO. Deliverables will be used to inform the development of the programme, demonstrate results and inform the development of policies.

2.0 Objective of the Assignment

A Service Provider 1 will be contracted to develop a comprehensive monitoring and evaluation (M&) framework and implementation plan to be used for assessing and understanding the progress of the project outputs and outcomes. It will also establish baselines.

The Service Provider will provide the roadmap to monitor and evaluate the project’s contribution to achieving the desired impact of reducing mortality, morbidity and economic losses in the Eastern Caribbean by strengthening health care facilities against disasters and climate change.

1 A team of at least two consultants Implementation progress, programme reach, and (intended and unintended) effects on end users will be assessed as well as success in building capacity and influencing policy. The effectiveness of the programme to mobilize continued investment in protecting critical infrastructure and long term sustainability as well as replication in other countries will also be evaluated.

3.0 of Services

The Service Provider will be expected to carry out and deliver on the following tasks: a. Review project documents including the log frame, annual targets and indicators, theory of change and existing monitoring tools. b. Develop a comprehensive monitoring and evaluation framework and road map for implementation for the project:  The framework should outline performance indicators (output and outcomes) with clear definitions, milestones, data collections strategies, and frequency of collection. It should include methodologies for measuring these and who is responsible for collection. It should also clearly demonstrate how the impacts outlined will be measured and evidence for the evaluations will be generated. Systematic disaggregation of data including by sex, disability, geographical location and income status will be important throughout. Views from intended beneficiaries should be sought and fed into the evaluation  The framework should take into consideration national M&E systems, draw on existing data available and ensure new data collection proposed is complementary to existing systems and data is made available to national stakeholders as far as possible. . Set and measure the baseline. d. Develop the evaluation strategy that draws on OECD-DAC evaluation criteria related to relevance, effectiveness, efficiency, impact and sustainability, coverage and coherence and the programme’s Theory of Change. Evaluation methods to test and explore impacts2 should be specified. The Evaluation should gather robust evidence about how, where and why (or why not) the programme was effective and achieved results. Knowledge, Attitude and Practice (KAP) changes will be important to capture. The evaluation strategy should correspond to the project’s operation context, learning and operational needs, as well as donor requirements. It should also specify how evaluative data will be captured. e. Develop draft terms of reference for conducting the independent mid-term review and final evaluation.

4.0 Deliverables

The Service Provider will provide a final report that includes 1. A comprehensive M&E framework for the 3-year project including a detailed implementation plan for monitoring activities, an evaluation strategy as well as design and information requirements for the evaluation. 2. A baseline assessment and plan to address any information gaps through survey or other relevant data collection tools. 3. Terms of Reference for independent external mid-term and final evaluation. 4. Findings gathered during the framework development and review of existing M&E systems, with recommendations.

2 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67427/design-method- impact-.pdf

5.0 Timeframe

It is anticipated that the Assignment will be completed within a total of six-eight (6-8) weeks from the date of the signing of the contract between PAHO and the Service Provider. All work must be completed by 15th November 2015.

6.0 Co-ordination of visits or interviews

The Service Provider will be responsible for arranging their own logistics, including meetings, transport and accommodation if required. It is not envisioned that the Service Provider will need to travel to target countries. PAHO will compile background documents and data collected for facility selection. They will also facilitate contact with stakeholders including national focal points to ascertain what data sets are available and beneficiary groups. All relevant expenses will be covered by the contract budget.

7.0 Payment

The Service Provider will be paid for the amount agreed between PAHO and the Service Provider with payments made directly to the Service Provider as follows: a. 10% on completion and submission of the work plan, detailing how Assignment will be accomplished with realistic timelines. b. 40% on completion of baseline plan with survey and relevant data collection tools. c. 50% on completion and acceptance of comprehensive M&E framework, the implementation guideline/plan on its rollout, and submission of final report with recommendations.

Deliverables will be reviewed and certified as satisfactory by the PAHO Project Officer and DfID. They must be submitted in both electronic version in relevant Microsoft Office format and hard/printed copy.

8.0 Skills and Qualifications The Service Provider should have knowledge and experience in the following areas: a) Design and delivery of robust, relevant and timely evaluation strategies and reviews of development interventions using qualitative and quantitative methods; b) Expertise and experience of designing and applying robust and appropriate performance monitoring and results frameworks (including expertise and experience in indicator development, testing and data collection / analysis); c) Monitoring and evaluation of large, complex, long term programmes d) Generating data to demonstrate programme effects for different segments of the population e) Reviews and evaluations in the Caribbean Region and a demonstrated understanding of political economy issues in the region f) Using reviews and evaluation as a tool for lesson learning g) Experience of review and evaluation of health sector, healthcare services at the national level or national entities in disaster risk reduction would be an asset as well as Climate Change adaptation and low carbon development programmes in low and middle incomes countries h) Good communication skills, both orally and in writing. i) Fluency in written and spoken English

9.0 Contract award criteria

The contract will be awarded based on the following:

# Description Weighting Main criteria (60 marks total) a Quality of suggested consultant/consultants team to undertake all aspects 15 of the assignment. b Adherence to ToR’s specifications and related requirements: a clear 25 understanding of required deliverables and robust, appropriate proposed approach e Displayed capacity for similar high quality work on data collection, 15 evidence-gathering, and evaluation. Experience working in Caribbean region, in the health sector, or on climate change or disaster risk reduction will be an advantage. e Communication, Writing skills and language proficiency 5 Commercial criteria (40 marks total) f Competitive fee rates and expenses in relation to the market and 25 demonstration of Value for Money. g Clear and effective financial plan to deliver output based deliverables and 5 key performance measures h Financial approach and methodology for ensuring the requirements will 10 be delivered on time and in line with agreed costs, highlighting any financial risks. Total 100