USAID/ZAMBIA SYSTEMS FOR BETTER HEALTH
ACTIVITY MONITORING AND EVALUATION PLAN (VERSION 2)
August 2016
This publication was produced for review by the United States Agency for International Development. It was prepared by Abt Associates for the USAID Systems for Better Health activity.
SBH M&E Plan
Contract/Project No.: Task Order No. AID611-TO-16-00001 Contract No. AID-OAA-I-14-00032 GUC Mechanism
Submitted to: William Kanweka, Contracting Officer’s Representative USAID/Zambia
Prepared by: Abt Associates
In collaboration with: American College of Nurse-Midwives Akros Inc. BroadReach Institute for Training and Education Initiatives Inc. Imperial Health Sciences Save the Children
DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government
USAID Systems for Better Health ▌pg. i SBH M&E Plan
Table of Contents
Acronyms ...... iv
1. Introduction ...... 6 1.1 Context ...... 6 1.2 Development Hypothesis ...... 7 1.3 Systems for Better Health Results Framework ...... 7 1.4 Geographic Focus ...... 10
2. Guiding Principles for SBH Monitoring and Evaluation ...... 11 2.1 Guiding Principles ...... 11 2.2 Comprehensive and Flexible Approach ...... 11 2.3 Gender and Equity ...... 12
3. Theory of Change and Implementation Logic Model ...... 13 3.1 SBH Theory of Change ...... 13 3.2 Rationale for Indicator Selection...... 15
4. Plan for SBH Monitoring and Evaluation ...... 16 4.1 SBH Research, Monitoring, and Evaluation Team ...... 16 4.2 Routine Monitoring ...... 17 4.3 Data Sources, Data Collection and Management ...... 18 4.4 Data Quality Assurance ...... 20 4.5 Data Flow ...... 23 4.6 Data Use, Dissemination and Reporting ...... 24 4.7 Schedule for Performance Monitoring Tasks ...... 25
5. Research and Evaluation Plan ...... 28 5.1 Baseline Documentation ...... 28 5.2 SBH Research and Evaluation Plan ...... 28 5.3 Abt Research Standards ...... 29 5.4 Abt Institutional Review Board and Ethical Reviews...... 30 5.5 USAID Open Data Policy ...... 30
USAID Systems for Better Health ▌pg. ii SBH M&E Plan
6. RME Budget...... 31
7. Indicators and Targets ...... 32
Annex 1: Summary of Indicator Revisions ...... 40
Annex 2: List of Target Facilities by Province and District ...... 44
Annex 3: Detailed Theory of Change by Result ...... 47
Annex 4: SBH Performance Indicator Reference Sheets (PIRS) ...... 53
Annex 5: Data Quality Assessment Checklist and Recommended Procedures ...... 138
Annex 6: SBH RME Year 1 Work Plan ...... 142
USAID Systems for Better Health ▌pg. iii
Acronyms
AMEP Activity Monitoring and Evaluation Plan APAS Annual Performance Assessment System CB Capacity Building CBD Community Based Distributor CDCS Country Development Cooperation Strategy CHA Community Health Assistant CHW Community Health Worker CHV Community Health Volunteer CPR Contraceptive Prevalence Rate CTC Abt Associates’ Client Technology Center DATIM Data for Accountability, Transparency, and Impact DDL Data Development Library (USG) DHIS2 District Health Information System Version 2 DHMT District Health Management Team DQA Data quality assurance or data quality audit EHT Environmental Health Technician GMCSP Governance and Management Capacity Strengthening Plan (MOH) GRZ Government of the Republic of Zambia HF Health Facility HMIS Health Management Information System HRH Human Resources for Health HRIS Human Resources Information System MandE Abt Associates’ Monitoring and Evaluation System M&E Monitoring and Evaluation MOH Ministry of Health OR Operations Research PA Performance Assessment PEPFAR United States President’s Emergency Plan for AIDS Relief
USAID Systems for Better Health ▌pg. iv
PFM Public Financial Management PIRS Performance Indicator Reference Sheet PMP Performance Monitoring Plan PQA Project Quality Assurance RF Results Framework RME Research, Monitoring and Evaluation SBH Systems for Better Health Project SOP Standard Operating Procedure TOC Theory of Change TWG Technical Working Group TSS Technical Support Supervision USAID United States Agency for International Development USG United States Government
USAID Systems for Better Health ▌pg. v
1. Introduction
This is a revised Activity Monitoring and Evaluation Plan (AMEP) for the United States Agency for International Development’s (USAID) five-year Systems for Better Health (SBH) in Zambia activity1. See Annex 1 for an overview of changes. SBH seeks to improve health outcomes for Zambians by strengthening systems that underpin the delivery of high quality health services. The activity also seeks to increase the utilization of high impact health interventions. SBH is being implemented by Abt Associates Inc. (Abt) partnered by subcontractors that include Akros Inc., American College of Nurse-Midwives, BroadReach Institute for Training and Education, Imperial Health Sciences, Initiatives Inc., and Save the Children to implement special segments in the SBH program from October 2015 to October 2020. To achieve its goal, SBH provides technical, financial, logistical, and administrative assistance to the Government of the Republic of Zambia (GRZ) at the national, provincial, and district levels. SBH strengthen the capacity of non-governmental and community-based Box 1. Zambia Health Sector—the basics organizations (NGOs and CBOs) to foster healthy behaviors among community recipients and deliver selected health The following summary is extracted from NHSP: services in remote areas. Since 1991, Zambia’s approach to health sector organisation has focused on decentralization of planning, 1.1 Context management, and resources to the district level where health services are delivered. This approach has inevitably called for broader participation of all the key stakeholders, Zambia has made promising gains against several key health particularly the communities in the governance of the indicators in recent years, although challenges remain2. Over health sector. The MOH is responsible for the overall the past decade, Zambia has made steady reductions in child coordination and management of the health sector in mortality (MTR 2014) with under-five mortality rate declining Zambia. In order to facilitate efficient and effective coordination, the following sector coordination structures steadily from 197 per 1,000 live births (1996 Zambia have been established at national, provincial, district, and Demographic and Health Survey (ZDHS) to 75 per 1,000 community levels: (2013/14 ZDHS). Similar reductions were achieved in infant mortality rates, which dropped from 109 per 1,000 live births National Level: The MOH Headquarters is responsible for overall coordination and management of the health in 1996 to 45 per 1,000 in 2013/14 (ZDHS). These rates bring sector. Zambia close to meeting the National Health Strategic Plan (NHSP) targets for under-five and infant mortality rates, 63 Provincial Level: Provincial Health Offices (PHOs) are per 1,000 and 38 per 1,000 live births respectively. Over the responsible for coordinating health service delivery in their respective provinces. past decade, the GRZ, with financial and technical support from the donor community, has invested heavily in District Level: District Health Offices (DHOs) are development of policies, strategies and guidelines, and responsible for coordinating health service delivery at the programmatic support. The investment was made to ensure district level, in primary care facilities, and first-level hospitals. progress in health program areas particularly HIV/AIDS, family planning, maternal and newborn health, nutrition, and child Community Level: At community level, Neighbourhood survival. Health Committees (NHCs) have been established to facilitate linkages between the communities and the health Despite the recent gains, Zambia’s health sector is challenged system. by the major tasks of effectively mobilizing and channeling financial resources and disseminating and implementing new or revised strategies and guidelines. The country’s health sector also is challenged by
1 The AMEP Version 1 was submitted in January 2016. This revised AMEP reflects the following changes: Final geographic targeting details (targeted districts and facilities); refinements to selected indicator definitions, based on data availability and other issues; updated baseline values; and final performance targets for the life of the project.
2 Zambia Demographic and Health Survey, 2013/2014, Lusaka
USAID Systems for Better Health ▌pg. 6
deploying and managing adequate numbers of skilled providers, particularly in the more remote, rural districts and facilities, making use of accurate health information to inform planning and management of service delivery, and engaging the citizenry in support of community level health activities.
1.2 Development Hypothesis
In alignment with USAID/Zambia Country Development Cooperation Strategy (CDCS) 2011-2015, USAID/Zambia designed the SBH project to assist the Ministry of Health (MOH) to address the critical challenges and bottlenecks to achieving improved health outcomes. USAID/Zambia’s development hypothesis for SBH states: “All programs aimed at empowering Zambia for improved health are premised upon the underlying hypothesis that improved health service delivery, combined with stronger and more accountable institutions, and increased awareness of healthy practices and healthy lifestyles, will result in healthier Zambians.” — (USAID/Zambia) Building upon USAID’s assumptions, it is our conviction that stronger performance of Zambia’s health system will support sustainable, effective and high quality health services. When high quality health services are available, accessible, and utilized, Zambians will achieve and sustain better health outcomes.
1.3 Systems for Better Health Results Framework
The foundation of the SBH project approach is a comprehensive results framework that depicts the causal pathway of our project from inputs, activities, and outputs to desired outcomes and ultimate impact. Complementing the SBH results framework is a theory of change that describes the critical assumptions we have made about the context and operating environment, as well as the causal relationship among SBH interventions. The SBH theory of change describes the considerations and rationale through which the program is designed to achieve its intended results. Together, the results framework and theory of change serve as the foundation to rationalize and describe how SBH contributes to the achievements of GRZ national health goals and objectives and how SBH lead to results in USAID/Zambia’s Country Development Cooperation Strategy (CDCS), specifically in support of DO3/IR3.2 and its sub-intermediate results, and DO2/Sub-IR 2.4.1. The SBH Results Framework forms a roadmap for the project’s M&E and is used routinely to validate assumptions and ensure that all activities effectively contribute to our intended results. SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels. To achieve this ambitious goal, SBH has three higher level results that align with the mandated Project Tasks by which the contract is organized:
Result 1: Strengthened MOH stewardship of the health sector Result 2: Strengthened program management capacities of provincial and district health teams Result 3: Improved capacities of MOH and community-based organizations to increase quality, availability and use of priority health services, and promote better health through prevention and healthy behaviors, at the community level in targeted districts.
USAID Systems for Better Health ▌pg. 7
These results are interrelated and interdependent. The SBH Results Framework (see Figure 1) demonstrates the ways in which various SBH results and sub-results link to one another. The achievement of all three results is contingent upon a cross-cutting implementation approach adopted by the project to improve capacity and influence accountability enhancements at all levels of the health system. The achievement of Result 1 and its sub-results create an enabling environment that supports the achievement of Results 2 and 3. Furthermore, the combined results of Results 1 and 2 lead to the project’s ability to achieve Result 3. Each result is supported by several sub-level results, as described in more detail below. The SBH Results Framework reflects the organization of the Zambian health sector and context of decentralization (see Box 1). The SBH project is expected to achieve the following outcomes: 1. Effective national level health stewardship 2. Improved capacities of provincial and district health teams to perform program management functions 3. Increased quality and availability of priority health services at the community level in targeted districts 4. Increased utilization of key public health interventions.
SBH outcomes are measured, in part, through the following priority indicators and targets:
Increase retention of HIV patients on antiretroviral therapy to 85 percent Couple years of protection in targeted districts increased by 10% Proportion of deliveries with assistance from a medically trained provider increased by 20 percent in targeted districts Increase the proportion of fully immunized children aged 12 to 23 months to at least 80 percent As outlined above, the highest level indicators for measuring SBH contributions towards the project goal will measure changes in health service use in targeted districts. Over time, increased use of priority health services will lead to better health outcomes. The project Activity Monitoring and Evaluation Plan (AMEP) includes both outcome and output indicators for each interim result (by program task) associated with systems strengthening, service quality, and service utilization improvements. It is hypothesized that improved utilization of priority health services is to contribute to better health among Zambians. An underlying assumption is that SBH’s contributions is being complemented by the MOH and other health sector partners through effective health prevention and promotion activities to advance better health. The SBH Result Framework (see Figure 1) demonstrates how SBH will progress toward its intended results and contribute to USAID/Zambia’s strategic direction as outlined in the CDCS. By the end of the project, we will have clear documentation of positive change in service utilization within SBH targeted districts and evidence of strengthened systems to sustainably support the delivery and utilization of high quality health services. The SBH theory of change (see Section 3) further explains how we envision the attribution and contribution of SBH systems strengthening, capacity building, community engagement, and information generation and use of activities for improvements at the service delivery and health outcomes level.
USAID Systems for Better Health ▌pg. 8 SBH M&E Plan
Figure 1. Systems for Better Health Results Framework
USAID Systems for Better Health ▌pg. 9
1.4 Geographic Focus
SBH leads improvements at all levels of the health system. Task 1 activities prioritize national level improvements and systems strengthening. Tasks 2 and 3 prioritize the provincial, district, and facility levels. In terms of geographic focus, SBH works in a total of five provinces (Central, Copperbelt, Eastern, Lusaka, and Southern). Within each selected province, SBH target four districts, for a total of 20 target districts, as shown in Table 1. SBH worked in collaboration with its MOH Table 1. SBH Geographic Priorities counterparts to carry out a Target comprehensive district selection Province SBH Target District Phase 1 Districts process during Year 1 Quarter 2. Chisamba SBH will implement Task 2 and 3 Kabwe Kabwe Central activities using a phased approach. Kapiri Mposhi During Phase 1, SBH has begun to implement activities 10 targeted Mkushi Mkushi districts (see Table 1). This initial Chililabombwe phase of assistance will culminate Kitwe Kitwe in a gradual withdrawal of Copperbelt intensive support and Mufulira Mufulira “graduation,” of the supported Chingola districts when agreed-on performance targets are met and Chipata Chipata DMOs and facilities demonstrate Nyimba improved capacity to Eastern independently maintain a high level Petauke Petauke of performance. As SBH reduces Lundazi support from the initial set of Lusaka Lusaka target districts, we begin intensive Shibuyunji Shibuyunji support to the second set of 10 Lusaka districts. Phasing support to SBH Luangwa districts allow for cost-effective, targeted use of project resources Chirundu to strengthen health systems Chikankata3 performance and health outcomes Gwembe in all targeted districts during the Southern life of the project. Monze Monze Livingstone Livingstone After the selection of the target districts, approximately 15 health facilities and their catchment communities within each phase 1district were selected targeting up to 150 health facilities listed in Annex 2.
USAID Systems for Better Health ▌pg. 10
2. Guiding Principles for SBH Monitoring and Evaluation
2.1 Guiding Principles
The SBH AMEP is a comprehensive management tool to serve both internal and external audiences over the life of the project. The SBH team uses the AMEP as a continuous performance management to ensure excellence, quality, integrity and efficiency throughout implementation and achievement of results. Also, the project team use the SBH AMEP to make mid-course corrections and adjustments as needed, provide lessons learned, and share new evidence to both internal and external stakeholders. The SBH AMEP provides information necessary to:
Clarify the intended results and ultimate impact the project aims to have. Specify how the project measures its contributions toward these intended results. Provide critical assumptions about the operating environment, context, and other factors (internal and external) that may affect the project’s performance. Guide implementation of M&E over the life of project, with clear roles and responsibilities (sustaining consistent M&E despite any personnel changes). Ensure integrity and quality of data and information generated, analyzed, and used by SBH for management and strategic decision making and external sharing.
2.2 Comprehensive and Flexible Approach
The SBH AMEP includes a comprehensive results framework, performance indicators, performance indicator reference sheets, plans for research and evaluation, M&E processes and tools, data quality assurance procedures, and mechanisms for the utilization and dissemination of performance information. Abt has built on lessons learned through our extensive experience monitoring health systems globally and in Zambia under the Zambia Integrated Systems Strengthening Project (ZISSP), Scaling Up Family Planning (SUFP) and Africa Indoor Residual Spray Project (AIRS). The SBH team will use innovative M&E tools and systems to improve the efficiency of routine performance monitoring and adopt a decentralized approach to M&E to reinforce ownership of results monitoring among all project staff. Over the life of the project, the SBH Research, Monitoring and Evaluation (RME) team will pursue a harmonized M&E approach with other health sector partners in Zambia to ensure complementarity and reduce inefficiencies or duplication across partners. With our flexible approach to M&E, SBH uses the AMEP as a dynamic management tool to facilitate adjustments; for example, when we must accommodate changes in scope, project priorities or expected results, or when refinements are made to interventions based on formative or implementation research findings. We operationalize this flexibility in the following ways:
Results-Oriented Work Planning. The SBH Senior Management Team (SMT) develops SBH’s annual project work plan in a consultative way, results-based strategy sessions. SBH use performance data to plan ahead and assess the expected contributions of all proposed annual work plan activities to the SBH results framework. This important collaborative examination of knowledge and lessons learned is a critical element of the performance management cycle, and the project team uses this exercise to refine and customize project interventions over time and ensure results are achieved.
USAID Systems for Better Health ▌pg. 11
Routine Review. SBH reviews the AMEP annually to ensure the ongoing relevance and utility of its performance indicators and evaluation plan to internal and external audiences so that all stakeholders have the correct data to inform planning, implementation, resource allocation, and other decisions. In alignment with SBH annual work planning, the RME team leads consultations across the project to review the AMEP and identify any potential needs to update or refine the project measurement approach or performance indicators. The RME team continuously identifies new measurement opportunities and initiates the development of new research plans as appropriate. This review occur annually during the development of yearly work plans, as well as when any major program scope or targeting changes may occur that affect the AMEP directly. Prioritized Research and Evaluation. The RME team updates the SBH research and evaluation plan to inform project design considerations. The team’s plan also substantiate SBH’s contributions towards a stronger health system in Zambia, better use of key health services, and improved health among Zambians. To support efficient and timely results-based performance management, the SBH project has developed a database system to facilitate data management, analysis and reporting. The customized database supports project performance management needs tracking and performance against the AMEP indicator framework. With MandE’s robust Results Framework functionality, the project team is be able to link all project activities to specific results and generate dynamic and customizable report with real-time updates so SMT and project staff can review progress and guide decision- making. Over time, the database will facilitate a decentralized approach to monitoring and evaluation by enabling direct data entry and interpretation at all levels of the project.
2.3 Gender and Equity
The SBH Gender Advisor coordinate with project technical staff across all domains to implement our proposed strategies for integrating gender and equity considerations into all project activities to foster equity in access and outcomes. Our AMEP includes specific measures of gender and female empowerment and disaggregate M&E data by several key attributes to help assess project performance towards addressing gender and equity. These attributes include gender, age, province/district, urban/rural, education level, and income. Further, the SBH RME Director ensures that gender and other equity considerations are integrated into the design and analysis plans for all assessments, evaluations and studies, and other data collection efforts. Through the project’s internal performance review processes, the RME Director analyze all relevant performance data to actively monitor and identify any gaps or unintended consequences that reflect inequity in program execution, participation, or outcomes. In such instances, the team work immediately to develop preventive and/or corrective action.
USAID Systems for Better Health ▌pg. 12
3. Theory of Change and Implementation Logic Model
3.1 SBH Theory of Change
The SBH theory of change aims to describe the context for SBH, including the conditions that may shape our ability to influence change and the underlying pathways through which we influence the expected changes (knowledge, skills, behaviors, attitudes, beliefs or practices). Further, we summarize our assumptions about how we intend to influence the desired improvements within Zambia’s health system and the ways in which we create the conditions to reach and sustain our desired goal and results. The Systems for Better Health project is designed to support health systems strengthening to ensure that the Zambian health system effectively provides accessible high quality health services that the population can use, leading to better health. SBH’s overall theory of change maintains that sustained capacity in the health system demands an enabling environment supported by policies, guidelines and financing mechanisms that foster equity, access, and accountability (Result 1); organizational systems and processes that assist managers and providers implement the health agenda at every level, coordinate activities between levels, and manage the delivery of health care (Results 1 and 2); and skilled individuals who have technical, managerial and advocacy skills, and knowledge to ensure quality service delivery (Results 2 and 3). Successful resolution of systems and human capacity issues within Zambia’s health sector will result in more effective national stewardship of the financial, human, and programmatic resources in the health sector; enhanced capacity at provincial, district, facility, and community levels for management and delivery of quality services; and increased engagement of communities in mobilization, planning, and advocacy for health. By improving the GRZ’s capacity-both institutional capacity and human capacity-at all levels, the health sector will be able to better plan, monitor, and deliver programs and services. The improved availability and use of high quality services will lead to greater utilization of high quality services for better health outcomes, a reduction in the burden of disease, and improved livelihoods for Zambians. Figure 2 demonstrates the links between the SBH implementation model (technical approach) and the theory of change. A more detailed overview of our theory of change is provided in Annex 3.
USAID Systems for Better Health ▌pg. 13 SBH M&E Plan
Figure 2: Systems for Better Health Theory of Change and Implementation Model
Systems for Better Health in Zambia: Implementation Model SBH Impact and SBH Outputs SBH Outcomes (inputs and intervention) Sustainability
Strengthen health Strengthen program management capacities at provincial Increase capacity of GRZ and CBOs to SBH activities stewardship by MOH and district level improve quality, availability and use and strategies at at national level (Task (Task 2) of priority health services at Strong enabling environment each level to 1) community level in targeted districts and accountability promotes strengthen (Task 3) and sustains health systems improvements Effective national level health National Province District Facility Community
s stewardship n o i s i c
Policies, strategies, guidelines, plans (1.3) Sufficient financial resources e
Systems and procedures to support linkages between d communities and facilities for health interventions (3.2) are allocated appropriately Information systems to collect, store, analyze information (1.3) m Systems and policies r
and effectively o
f Improved health n
Systems and procedures to conduct performance assessment, technical support i Domestic resource Increased outcomes for supervision, and other QI (2.2) o t mobilization and utilization of key Zambians revenue collection (1.2) n o HRH i public health through strong t e Financing and financial Improved capacities s planning a u interventions systems that
Managers have the skills and management and m of provincial and r
d Public financial management (1.2) o n manage tools to plan, manage and underpin the f
a district health teams
ment n i Such as: o monitor health programs delivery and use i
(1.1) y to perform program t Capacity to generate and use information for decisions (1.3) l ART a effectively e of quality, high r management FP e m i
n impact health Management capacity t Deliveries by trained
e functions d
g Capacity to manage, plan, supervise, monitor health interventions (2.1)
Management services at
n providers n and technical a o Immunization district and i e l t capacity to plan a Capacity to supervise and improve quality of health interventions (2.2) and implement b Nutrition community a i m Health facilities and workers community l r
Technical capacity e levels. o
health programs r f Technical capacity are qualified with the right
among CHVs and f n (3.1) I o
Technical capacity to deliver quality health services (2.2) CBOs to deliver skills and held accountable for quality health e delivery of high quality s u
interventions (3.1)
service delivery e n i t
Stakeholder u Increased quality and Engagement and collaboration between facilities and o engagement R availability of priority communities (3.2) Strong linkages between health services at the health facilities and community level in communities targeted districts
Communities are active partners in health and holding health sector actors accountable
USAID Systems for Better Health ▌pg. 14 SBH M&E Plan
3.2 Rationale for Indicator Selection
SBH used the underlying results framework and theory of change to identify and select performance metrics to measure implementation of the project, progress towards its intended results, and the effects of SBH interventions. Indicators were identified for each level of the results framework and are mapped to the respective result and sub-result. The SBH AMEP includes a selection of indicators associated with:
USG, including State Department/Standard “F” PEPFAR Globally-accepted entities (e.g., World Health Organization, other) Custom, project-specific.
USAID Systems for Better Health ▌ pg. 15 SBH M&E Plan
4. Plan for SBH Monitoring and Evaluation
4.1 SBH Research, Monitoring, and Evaluation Team
The SBH Research, Monitoring and Evaluation team falls under the leadership of the SBH Chief of Party. The RME team manages and implements the AMEP, and is comprised of the Research, Monitoring and Evaluation (RME) Director and a team of technical staff. The team structure is summarized in Table 2 below.
Table 2. SBH RME Team, Roles and Responsibilities
Team Member Core Roles and Responsibilities Reporting Structure
Director -Research Oversees and develops the integrated AMEP with Reports to: COP Monitoring and results framework, performance monitoring plan, Evaluation and research and evaluation plan. Supervises: Develops appropriate M&E systems and tools Senior Manager M&E Prof. Amusaa Inambao necessary to routinely collect, analyze, and present program data for SBH staff members, partners, and M&E Team (indirect) donors, and for preparing monthly and quarterly HMIS Specialists (seconded program indicator reports. to MOH) Provides routine data analysis to enable staff and management to monitor program performance and systems strengthening; coordinates quarterly, semi- annual, and annual data analysis and reporting for all the program indicators. Leads development of SBH research and evaluation plan, including oversight of all research activities (e.g., leadership and quality assurance of research/studies). Supervises SBH RME team to ensure quality and timeliness of all M&E functions. Senior Manager Manages the M&E database, systematically collecting, Reports to: Director - RM&E Monitoring and analyzing, and presenting program data for SBH staff, Director Evaluation partners, and donors, and for preparing monthly and quarterly program indicator reports. Supervises: Leads capacity building among M&E team in Benson Bwalya M&E Analyst collecting data in line with systems and procedures. M&E Assistant Supervises the M&E Analyst, M&E Assistant, and Surveillance Assistant. Surveillance Officer M&E Analyst Supports the Reports to: Snr Manager-RM&E to Reports to: Snr Manager systematically collect, analyze, and present program data. Abel Muwowo Facilitates field and remote supervision, training of
health facility reporting in target districts, planning, coordination, development. Provides oversight of a community-level surveillance platform. Liaises with M&E staff within provincial and central channels of MOH, District Environmental Health Technicians (EHTs), District Health Management Teams (DHMTs), and the Community Health Workers.
USAID Systems for Better Health ▌ pg. 16 SBH M&E Plan
Research Specialist Supports the Reports to: Snr Manager-RM&E in Reports to: Director Research Officer designing of all SBH research concept, evaluation, Monitoring and Evaluation Kakompe Kalenga data analysis and report writing Participate in the design, execution and control of research study. Coordinate and conduct the data processing and analysis. Manage research field work for both quantitative and qualitative work Liaises with all the partners at different level in data collection of routine data. M&E Assistant Supports the Snr Manager-RM&E to systematically Reports to: Snr Manager collect, analyze, and present program data. Paul Chirwa Facilitates field and remote supervision and training of health facility reporting in target districts.
Assists in the planning, coordination, development, and oversight of a community-level surveillance platform. Surveillance Officer Supports technical interventions to strengthen Reports to: Snr Manager national health information and surveillance systems. TBD Helps strengthen capacity for disease surveillance, data-driven decision-making, M&E, and geographic information systems. Supports, guides, and builds capacity for MOH staff in data monitoring, evaluation, and decision-making through use of the DHIS2. Strengthens systems and processes to improve data quality. Works with MOH staff to streamline the flow of quality and relevant data to critical stakeholders in the decision-making and program implementation process.
The SBH RME team liaises with SBH program staff to ensure a coordinated and streamlined effort to RME across the project. The COP maintains overall responsibility for ensuring high quality, timely, and responsive RME for the project.
4.2 Routine Monitoring
The SBH RME team conducts routine monitoring of SBH implementation and results. This includes; ongoing tracking of activity implementation; routine monitoring include the following:
Continual tracking of activity implementation (work plan monitoring and tracking) Quarterly performance reviews, including development and submission of SBH quarterly progress reports in alignment with the contractual reporting timelines. Quarterly progress reports will provide a comprehensive overview on the status of implementation, results achieved during the reporting period and to date, success stories, challenges and opportunities, and other programmatic highlights. Internal consultations and performance reviews, led by the SBH RME team, to ensure ongoing reflection on implementation status and project progress. The internal review serves as a
USAID Systems for Better Health ▌ pg. 17 SBH M&E Plan
continuous process of updating program staff on activities completed and achievements made against targets. SBH Monthly Updates highlight major activities, accomplishments and/or results to date. The monthly updates highlight upcoming STTA, core activities of the previous month, and projected activities/events. Semi-annual and cumulative performance summaries and participation in USAID/Zambia portfolio review processes. M&E data submission to appropriate USG systems, including the Data for Accountability, Transparency, and Impact System (DATIM) or Development Data Library (DDL), as appropriate. The SBH M&E team ensures that appropriate SBH M&E data is submitted to relevant systems either quarterly, semi-annually or annually per PEPFAR and other reporting requirements. The M&E team also submits any research data to the DDL as contractually required.
4.3 Data Sources, Data Collection and Management
The following sections outline the types of data sources, data flows, data collection and data management approaches that SBH has put in place to ensure high quality performance monitoring over the life of project. In addition, we describe the data quality assurance mechanisms and tools we use to ensure the reliability, integrity and consistency of our data from the time it is collected through use and submission in any product or reporting documentation. Data Sources and Data Collection To meet SBH performance monitoring, evaluation, and reporting needs, in alignment with USAID/Zambia and USG partners, SBH uses the following approaches for data collection and analysis. SBH use a mix of existing data and project-generated data. All data sources are specified in the illustrative Performance Indicator Tracking Table and Performance Indicator Reference Sheets.
USAID Systems for Better Health ▌ pg. 18 SBH M&E Plan
Existing Data Sources and Reporting Systems. SBH data collection and management processes for routine reporting on indicators draw heavily on service availability and utilization statistics from the existing national Health Management Information System Box 2. Supporting Health Sector M&E in Zambia (HMIS), the District Health Information System Version 2 (DHIS2) platform. SBH leverages information The SBH project supports institutional and human generated through the HRIS once it has been rolled capacity building efforts that will improve the quality of information generated and used by Zambia’s health out to SBH target districts. (See Box 2 for examples of sector. SBH interventions will improve the quality, use, how SBH strengthen the quality of data generated and reliability of DHIS2 and HRIS. Specifically, we are through these existing data sources.) SBH works with working to improve DHIS2 data availability through the MOH to obtain access to extract required innovative mHealth applications for decentralized data information from DHIS2 routinely, to enable ongoing entry, which will enhance the reliability of DHIS2 as a source for timely data. Simultaneously, we will build review of relevant data to inform project strategies and sustainable capacity of health information functions performance. within Zambia’s health system to generate quality data. We also will support more systemic data quality In addition, SBH rely on review of standard assurance processes within the MOH. SBH also government documentation, including national, supports enhancement of the HRIS that will enable it to provincial and district level financial reports, annual be used as a reliable source of information within the health sector. work plans/action plans, data quality reports, and others. SBH project staff collects copies of relevant These technical interventions strengthen capacities reports from SBH MOH counterparts, as within the health sector to generate and use high appropriate. quality information, but also improve SBH’s ability to reliably use routine information for our own M&E Selected outcome indicators rely on data from periodic purposes. national surveys, including the Zambia Demographic and Health Survey (ZDHS). Project Data: SBH measure several process, output, and outcome indicators through project-specific information sources, including training registers, pre- and post-training evaluations, mentoring records, and other sources. The project’s framework for sustainable capacity and complementary performance metrics will provide a regular source of information for measuring and analyzing changes in institutional capacity at various levels. SBH plans to use an Excel-based Capacity and Performance tool for self-assessments and scoring at periodic intervals, allowing for easy, direct, and systematic analysis by stakeholders. This tool will also allow rapid uploading into the SBH M&E database for customized programming (e.g., to identify targeted technical assistance needs and make decisions about “graduation”) and reporting on real progress over time. SBH RME Team developed data collection tools and standard operating procedures (SOPs). In addition, the team will develop flow charts for all required project data, including detailed training records (capturing type of training, objectives, locations, participant details, and attendance documentation), Clinical Care Specialist (CCS) mentoring forms, HS Specialist mentoring forms, Technical Support Supervision (TSS) reports. SOPs outline the specific requirements for the use of each tool, including roles and responsibilities, data flows, timelines, and other parameters. The SBH M&E Lead is responsible for ensuring compliance with the SOPs and routinely checked the completeness of standard forms. The SBH M&E Team issues a certificate to the program staff upon verifying completion of the information. In addition, SBH plans to issue grants to non-governmental organizations, community-based organizations and other entities through a ‘grants under contract’ mechanism. The RME team works closely with the SBH grants team to ensure that all grantees develop appropriate monitoring and evaluation plans to enable them to collect and submit routine grant-related performance data, and contribute (as appropriate) data to selected SBH indicators. The RME team plans to develop grant specific M&E processes and tools, and train grantee staff on M&E issues as necessary.
USAID Systems for Better Health ▌ pg. 19 SBH M&E Plan
SBH plans to carry out several assessments, evaluations and studies to collect information over the life of the project (including base-line, midline and end-line data for select indicators). (The AMEP provides a brief overview of initial ideas for potential studies in the Research and Evaluation Plan in Section 5). Data Management As noted, SBH is using a customized database to support efficient and timely results-based performance management. This system is being implemented in a phased approach of decentralized M&E in terms of direct data entry by program staff at different levels. and 2M&E database use is focused within the RME team and selected program teams located in the SBH Lusaka office. Starting in Year 3, SBH plans to roll out decentralized direct data entry to SBH provincial and district level staff). The SBH database will enable the generation of dashboards to facilitate unique performance views for different types of users (such as combined indicator views for specific targeted districts or facilities). The database will also ensure proper alignment of relevant indicator and data characteristics (such as disaggregation elements) with USAID and PEPFAR requirements.
4.4 Data Quality Assurance
Data quality for project M&E is often defined by a combination of dimensions, as outlined in Table 3. The SBH RME team use a comprehensive data management processes and establish measures to ensure the accuracy, reliability, completeness, precision, timeliness, and integrity of data.
Table 3. Key Dimensions of Data Quality
Data Quality Description Dimension Accuracy Accurate data correctly measure what they are intended to measure. Accurate data have minimal errors (e.g., transcription, sampling, or recall errors). Reliability Reliable data are measured and collected consistently. The protocols do not change based on who is using them or how often they are used. Completeness Complete data represent the entire list of eligible persons or units, not just a part of the list; no data are missing. Incomplete data do not reflect the real situation. Precision Precise data are detailed and thus are not ambiguous. Timeliness Timely data are current and available on time and on a schedule that makes the data useful (i.e., the data are recent enough to be useful for decision making). Integrity Data with integrity are protected from deliberate bias or manipulation for political or personal reasons. Source: Adapted from MEASURE Evaluation, Data Quality Assurance Tool for Program-Level Indicators, USAID/PEPFAR, January 2007. The M&E Team Lead developed forms and guidelines (tools, SOPs) to standardize data collection and record use by SBH staff and partners. This includes the routine and mandatory use of clear operating definitions as delineated in Performance Indicator Reference Sheets (PIRS, see Annex 4). SBH data management systems, including MandE have internal mechanisms that ensure data quality, including standardized data collection tools, data entry fields with restricted values to prevent typos/invalid data entry, and routine internal data audits against source documentation by the M&E Team.
USAID Systems for Better Health ▌ pg. 20 SBH M&E Plan
SBH developed data controls processes which are followed during and after data entry to assure accuracy. The M&E team continuously checks hard copy forms for completeness, consistency, and accuracy before data is entered into the MandE system. The M&E team routinely performs post-data entry verification to ensure that all the entries are correctly done. Entries are checked for duplication, completeness, among others. The M&E Analyst and Assistant produce summary reports which are shared with the program staff for discussions on data completeness, inconsistencies, or other identified data quality challenges. The team conducts structured data quality assessments (DQAs) on a sample of indicators each quarter and uses the results to address data quality issues. To ensure quality of data from external sources, SBH follows Zambia’s MOH Data Quality Audit Guidelines in a sample of sites to conduct at least twice-yearly reviews of HMIS data to validate the data against the source registers and aggregation forms. The primary purpose of data verification and quality audit are to validate the data coming through from the program staff members and the HMIS. In addition, the MOH Health Information Officer and RME team participate in the data quality audit, and thus the audit strengthen the data collection process. The MOH offices and health facilities are visited by the M&E team/auditors to verify and validate the source documents for completeness and consistency as prescribed in the HMIS. SBH recognizes that measuring the success and improving program activities depends on strong M&E systems that produce quality data related to program implementation. The SBH RME team uses various types of DQA mechanisms to ensure high quality data (see Table 4).
USAID Systems for Better Health ▌ pg. 21 SBH M&E Plan
Table 4. Types of DQA
Type of DQA Purpose Timing Examples
Data Quality Processes/protocols Before and during Performance indicator reference sheet Assurance integrated into project project implementation (PIRS) with full operational definition of M&E processes and indicator terms, precise methods for system that are designed data collection that ensure consistent to ensure that a specific application of the indicator over time activity (i.e., data and by different people. collection or data entry) is properly carried out Protocols for data collection staff. to ensure quality. Standardized data collection forms that have been tested and edited to ensure proper design and ease of comprehension by data collectors. Standardized M&E system, database, or data entry template in Excel or other system. Ideally the tool would have logic checks or locks to ensure high-quality data entry.
Data Quality Activities/processes During implementation Period reviews of data collection forms Control designed to evaluate by the M&E officer or other assigned collected and entered staff to ensure project staff are data and provide completing them correctly (such as corrective feedback. training registers). Systematic verification of data entered into a system, database, or file. This entails using the primary data collection form to cross-check the data as entered into the electronic file. For data entry, a data quality control best practice is the use of double data entry to cross verifies large data sets.
Data Quality Post-activity (or post- Following data collection Field visits by M&E or supervisory staff Audit data collection) and/ or activity to cross-check data collected at the site evaluation of data quality completion of implementation. with the aim of verifying Examples: data quality and gathering lessons learned to inform Interviewing health providers to confirm improvements. that they received mentoring reported by field staff. Visiting a sample of health facilities to check source data records against summary data forms completed by partner staff.
SBH uses USAID and MOH standard data audit checklists during data audits (see Annex 5). Standard Performance Indicator Reference Sheets (PIRS) include precise indicator definitions, data sources, frequency of data collection, person responsible of data collection among other variables and serve as an important reference tool to ensure consistent operationalization of indicators over time.
USAID Systems for Better Health ▌ pg. 22 SBH M&E Plan
In addition to conducting routine and periodic data quality audits as part of our standard M&E mandate, SBH minimizes potential data quality issues at two different levels, i.e., HMIS (DHIS 2) and project- generated data, summarized below.
HMIS Data. SBH recognizes that HMIS has occasional data quality challenges such as incompleteness, timeliness, and accuracy on some indicators. SBH supports institutional and human capacity building efforts that improve the quality of information generated and used by Zambia’s health sector. SBH plans to work with the MOH HMIS team to improve DHIS2 data availability through establishment of appropriate data access policies and applications for decentralized data entry, to enhance the reliability and timely availability of data. These interventions will improve both the quality and use of health information systems (as highlighted in Box 2). Project-Generated Data. In collaboration with SBH program staff, the RME team developed standard data collection tools such as training registration/mentorship forms, daily attendance sheets, PA, and District Records to capture all data.
The PIRS and SOPs provides detail on the exact parameters and timelines by which data needs to be collected and reported to ensure that SBH approach is standardized over time.
Box 3. Abt Associates Project Quality Assurance Mechanisms
Abt’s Project Quality Assurance (PQA) program, which has been in place since 2000, provides the structure and planning with respect to overall quality control of all data, products, and deliverables. An assigned PQA Advisor, usually the Portfolio Manager, prepares and implements a quality assurance plan for the life of the project. The PQA Advisor provides on-time substantive review and comment on technical documents, reports, project work plans, annual and quarterly reports, scopes of work for short-term technical assistance, and other materials as requested by the Chief of Party.
PQA is not just deliverable-based, it is integrated into the entire course of the project, taking into account the dynamic and changing environments of our projects. PQA is an integral part of good project execution and risk management standards for quality work as well as a means for ensuring cost-effective management. Activities and issues related to quality assurance are a standing agenda item during Abt’s quarterly project management reviews. Abt’s Quality Assurance Council, established in 2011, provides basic training for PQA advisors and maintains a quality assurance checklist applicable to all Abt projects. These are reviewed and updated quarterly and entered into a project management database for International Health Division oversight. Newly revised PQA tools will be rolled out to all Abt projects as they are available.
Abt Associates maintains a rigorous policy on data security routines and protocols, which are established in our field offices with assistance from Abt’s Information Technology Services Center. Abt provides annual training to all staff on data security and ensures that field offices are in compliance with all procedures. The SBH RME Director will maintain data security and integrity, in collaboration with Abt home office. 4.5 Data Flow
The RME team conducts continuous comprehensive training to SBH staff all M&E related requirements. MandE will be rolled out to the SBH team in a phased approach. In the initial stage during Year 1, the system is being implemented by the RME team and rolled out to senior management and other program team leaders. During this initial period, data is generated by SBH project staff and submitted to the M&E Analyst in Lusaka in accordance with established SOPs (differs by data and indicator type). The M&E Analyst supports the SBH technical teams to ensure all necessary information is collected and submitted. This data is then entered into the database system centrally by the M&E Assistant. The M&E Analyst works with SBH program staff to resolve any data quality issues or inconsistencies, and generates certificates of completion. The RME team intends to retain all source documentation for data quality control and audit purposes.
USAID Systems for Better Health ▌ pg. 23 SBH M&E Plan
To assure timely and accurate quarterly, semi-annual and annual reports, the RME Team established firm deadlines by which data must be submitted. Data verification is conducted on a continual basis to minimize delays and ensure the MandE system is kept up-to-date.
Box 4. Training SBH Staff to Ensure Data Consistency and Quality
4.6 Data Use, Dissemination and Following approval of the SBH AMEP by USAID/Zambia, the Reporting RME Team will finalize the development of internal SBH tools and procedures for routine data generation and data management. A tool is under development and will be rolled Data Use and Dissemination out at the SBH staff semi-annual review in August 2016.
SBH builds on Abt’s proven and effective approach to The Team will design and conduct a one-day training for all SBH the use and dissemination of program results and staff on the AMEP and all required M&E processes, to increase findings to a wide range of counterparts and the ability of all project staff to identify, collect, analyze, and stakeholders. Existing strategies have be refined and report on high-quality performance monitoring data. The training will establish the roles and responsibilities of each staff in elevated, through the use of a custom database and supporting M&E needs. Data storage and security requirements, improvements to DHIS2, to provide efficient access and data quality assurance planning will also be addressed. to high-quality, timely information. Primary users of project information include the SBH team, central In addition, programme staff in selected target districts and the M&E analyst have been trained in the use of DHIS2 (through level MOH stakeholders, Provincial health teams, MOH-led training, as well as internal SBH partner staff District Health Management Teams, facility heads, resources). grantees and other health sector entities; USAID, PEPFAR, and other implementing partners; communities and traditional leaders; and the broader public health community. SBH ensures timely dissemination of information in the following ways:
Feedback and evidence-based planning at all levels. The SBH work planning and review cycle ensures adequate engagement and harmonization of project technical plans with counterpart planning cycles and priorities. Our team work closely with national, provincial and district level partners during implementation and therefore shares results, lessons learned, and performance challenges on an ongoing basis. Documenting progress and achievements: SBH progress, lessons learned, and key findings are communicated to USAID, MOH counterparts, and other partners (as appropriate) through monthly project reviews, quarterly reports, and annual work-planning. Sharing lessons learned and new evidence: SBH RME Director and senior technical staff coordinate with the MOH on national dissemination of lessons learned and new evidence, via semi- annual/annual ministry planning and review meetings, donor coordination meetings, quarterly partner coordination meetings, regional meetings, and/or TWGs or other collaboration mechanisms. Opportunities for dissemination beyond Zambia are being explored, through submission of abstracts to relevant international technical meetings and conferences. As appropriate, SBH provides technical support for development of journal articles on projects findings of interest to an international technical audience. Routine Reporting SBH uses a routine reporting schedule with deadlines and review guidelines, based on the SBH Contract requirements. SBH technical leads contribute to preparation of annual documentation and dissemination plans produced in concert with the annual work plan. These include all technical deliverables and other dissemination products for sharing project experience and results, such as scientific publications, technical briefs, tool kits, study reports, and final products.
USAID Systems for Better Health ▌ pg. 24 SBH M&E Plan
SBH use efficient processes to respond to USG reporting requirements as follows.
Table 5. SBH Routine Reporting Requirements
USG Reporting Requirement Frequency/Timing Quarterly Progress Reports Quarterly: Q1/January 31, Q2/April 30, Q3/July 31 Annual Report Annually (includes Q4 reporting): October 31 Submission of results to DATIM, and other relevant USG Per reporting schedules systems (as appropriate) Final Progress Report Once: 45 calendar days prior to EOP Portfolio Review Presentations Twice yearly PEPFPAR Semi-Annual and Annual Performance Reporting Twice yearly (SAPR and APR) U.S. Foreign Assistance Performance Plan and Report Annually Reports of all research and studies Varies: Within 30-90 days of completion Submission of research data to DDL Per required timeframe Ad hoc performance requests As necessary
As part of its routine reporting requirements, the SBH RME Team submit/upload appropriate data to the Data for Accountability, Transparency, and Impact System (DATIM) or Development Data Library (DDL), as appropriate. The SBH M&E team ensures that appropriate SBH M&E data is submitted to DATIM per required PEPFAR reporting parameters. The M&E team submits any research data to the DDL as contractually required.
4.7 Schedule for Performance Monitoring Tasks
To complement this comprehensive AMEP to guide the overall approach to M&E over the life of project, detailed M&E operational plans are developed each year as part of the SBH project work plan development process to guide the implementation of M&E activities. (See Annex 6 for a summary of the Year 1 RME work plan.) Table 6 below presents an overview of routine RME tasks. Following completion of each annual work plan, the RME team develops a more detailed team-level operational plan that describes the activities, timelines, responsible parties, and budget for relevant M&E activities. The RME operational plan supports the AMEP and hence the ability of the RME team to work effectively and efficiently to achieve the expected results of all or given project M&E activities. The AMEP defines roles and responsibilities of team members is to strengthen communication and outlines RME activity specifics including for data collection, data quality assurance, and reporting for strong internal management and accountability.
USAID Systems for Better Health ▌ pg. 25 SBH M&E Plan
Table 6. SBH Performance Management Task Schedule
Key M&E Task Timing Person Responsible Ongoing Data Collection SBH Indicators Ongoing and customized, per PIRS SBH program staff SBH M&E Senior Manager Data Cleaning and Data Analysis Data entry Ongoing and customized, per PIRS and SBH program staff SOPs SBH M&E Analyst, SBH M&E Assistant Data cleaning Quarterly SBH M&E Analyst, SBH M&E Assistant Data analysis (routine M&E data) Monthly, Quarterly RME Team Internal Results Review Quarterly Reviews Quarterly RME Director, M&E Senior Annual Work Plan Development Annually (September-October) Manager, SMT Ad hoc TBD Reporting Quarterly and Annual Reports Q1(Oct-Dec) – Due January 31 RME Director, M&E Senior Q2 (Jan-Mar) – Due April 30 Manager Q3 (Apr-Jun) – Due July 31 Annually/Q4 (Oct-Sep) – Due Oct 31 Data submission to DATIM or other USG Per reporting requirements systems PEPFAR Reporting SAPR (May) APR (November) Portfolio Review Presentations Semi-Annually U.S. Foreign Assistance Performance Plan Annually and Report Other, as requested or required TBD Data Quality Assessments Sample of indicators and sites each Quarterly M&E Senior Manager quarter Other (TBD) Special Assessments Baseline February - July RME Director Research Research protocols for all Ongoing/as needed studies/evaluations Ethical review Ongoing/as needed Research reports Within 30 days of completion RME Director Data uploaded to Data Development As necessary, in accordance with RME Director Library required timeframes Review and Update of AMEP (as necessary) Annual Work Plan Development September–October RME Director, M&E Senior Manager, COP Other, as necessary TBD RME Director, M&E Senior Manager
USAID Systems for Better Health ▌ pg. 26 SBH M&E Plan
Key M&E Task Timing Person Responsible Final Progress Report 45 calendar days prior to EOP RME Director, M&E Senior Manager, COP
USAID Systems for Better Health ▌ pg. 27 SBH M&E Plan
5. Research and Evaluation Plan
Research and evaluation are important components of the SBH AMEP. Targeted surveys and studies support the following functions: 1) provide baseline, mid-point and/or end-of-project data for select indicators (such as availability of services in SBH districts) 2) generate a better understanding of the operating context through formative research to inform interventions and generate lessons for stakeholders (such as provider attitudes and behaviors) 3) Study the outcomes and attribution of innovative approaches though experimental or quasi- experimental methods. SBH regularly collaborate with health sector partners (e.g., through stakeholder and partner coordination mechanisms, TWGs,) to identify synergies and opportunities for research and data collection collaboration among partners.
5.1 Baseline Documentation
To enable specific monitoring and evaluation of SBH efforts in targeted districts and facilities, SBH designed and conducted a baseline assessment and analyzed data on key metrics related to the project focus areas. The baseline comprised of a combination of existing data and self-reported data by the PMOs and DMOs. Existing data sources include the heath management information system/District Health Information 2 (HMIS/DHIS 2), HRIS, Ministry of Finance, MOH documentation (e.g., provincial, district, or facility action plans, and other sources). Additional information for some indicators was collected from the PEPFAR DATIM. Georeferenced details have been added to the current data base through the geo-mapping of health facilities in the current MOH List of Health Facilities in Zambia report.
5.2 SBH Research and Evaluation Plan
SBH developed an initial research and evaluation plan, using rapid assessments, contextual mapping and a deep-dive analysis of existing health sector approach. The initial list of potential research topics were selected based on their relevance to the project’s strategic priorities, and will be discussed with USAID/Zambia and MOH counterparts. In addition SBH aims to identify other research opportunities during the course of the project. The initial SBH research topics include:
Establishing an integrated capacity building interventions model to increase quality of services and improve health outcomes Evaluation of the impact of quality improvement efforts in targeted facilities Determining impact of performance assessment on improvements in quality of service delivery at the facility and community levels. Determining the impact of the use of HMIS on the quality of planning developed at various levels of health service management.
USAID Systems for Better Health ▌ pg. 28 SBH M&E Plan
Determining the effects of strengthening coordination among different specific interventions on improvement of health care service delivery at facility level. Determining the effectiveness of SBH interventions through contracted partners. (for example, pilot interventions such as Kitwe HIV and community HMIS pilots). Assessment of the effectiveness of ART adherence and retention in selected provinces in Zambia. Determining impact of quality assurance in design, development and delivery of HIV/AIDS training program on performance of service providers at service delivery level. Assessing effects of verticalization of HIV/AIDS and other programs on health systems interventions improvements.
5.3 Abt Research Standards
Implementation of the above research topics will be implemented under Abt. Associates robust research standards and rigor, requiring that appropriate research protocols and other requirements are developed an approved by relevant competent authorities. Furthermore, ethical clearance by the Home Office and Local Ethical clearance bodies will be sought for each study, where this is a requirement. Basic elements of a standard Abt-supported research protocol are highlighted in Box 5.
Box 5. Standard Elements of a Comprehensive Research Protocol
Executive Summary
Background and Context
Rationale for Research
Literature Review
Specific Research Questions
Conceptual Framework (hypothesis)
Description of Interventions (project interventions that influence outcomes of interest)
Study Design (methodology)
Power Calculations (as appropriate per method)
Sampling Plan
Study Instruments
Plans for Data Collection and Data Entry
Data Analysis Plan
Ethical Considerations and Risk (including IRB requirements and process)
Expected Deliverables and Dissemination Plan
Timeline and Budget
USAID Systems for Better Health ▌ pg. 29 SBH M&E Plan
5.4 Abt Institutional Review Board and Ethical Reviews
All research studies conducted by Abt, our SBH partners, or any subcontractors must be reviewed by Abt’s internal Institutional Review Board (IRB). In addition, SBH works closely with the Ministry of Health Directorate of Disease Control and Surveillance as it develops research concepts and complies with ethical review requirements in Zambia. Abt maintains our own IRB to protect human subjects participating in research conducted by Abt or our subcontractors. The Abt IRB:
Conducts prospective reviews of proposed research Monitors research for the purpose of safeguarding research participants’ rights and welfare Ensures that our research meets the highest ethical standards and receives fair, timely, and collegial review by an independent panel Coordinates with the IT Service Center, IT Security, and Contract Operations via Abt’s Information Risk Management team to protect the confidentiality of study participants’ data. All Abt-supported research studies include clear consent processes as well as a data security plan that specifies how data collection instruments (such as paper questionnaires or mobile-based data) will be securely transmitted and stored. The SBH RME Director ensures appropriate IRB and ethical reviews and approvals for all research and evaluation efforts conducted by the project. In addition, all project directors and any SBH program staff and/or researchers who participate in research with human subjects must complete Abt’s online IRB training prior to the start of data collection.
5.5 USAID Open Data Policy
Any studies undertaken by SBH will have to be in accordance with the Open Government Policy of the U.S. Government (USG), USAID and USG agencies and institutions require that all quantitative data collected for the purpose of evaluation must be uploaded and stored at a central database. The SBH RME Director ensures SBH compliance with USAID Open Data Policy requirements and submission of data to the Data Development Library (DDL).
USAID Systems for Better Health ▌ pg. 30 SBH M&E Plan
6. RME Budget
SBH maintain a budget for research, monitoring, and evaluation of approximately five to seven percent of the total five year budget. For Year 1, SBH was budgeted at US$527,644 (approximately 4 percent of the Year 1 budget) to support RME staffing, routine M&E activities, baseline, MandE system customization and the launch of selected research activities.
USAID Systems for Better Health ▌ pg. 31 SBH M&E Plan
7. Indicators and Targets
The SBH AMEP provides a set of process, output, outcome, and impact indicators that collectively demonstrate the project’s performance and contributions to improved health indicators. The Performance Indicator Tracking Table (PITT), presented in Table 7, includes a summary of SBH performance monitoring indicators. Performance Indicator Reference Sheets are provided in Annex 4. Most of the indicator elements of the AMEP in this revised version have been determined and are to be confirmed pending the finalization of the final indicators matrix. These have been listed in the baseline values and targets listed in the tables that follow.
USAID Systems for Better Health ▌ pg. 32 SBH M&E Plan
Table 7. Systems for Better Health in Zambia Performance Indicator Tracking Table
Relationships to select strategies/ frameworks (including ZNHSP; USAID/Zambia CDCS; US Government Standard Foreign Assistance Indicators (USG/F); PEPFAR Next Generation Indicators; Feed the Future Indicators) are specified in the “Links” column. Although indicators are presented by task in this table, several indicators provide information relevant across tasks and sub-task/results and such relationships are highlighted in the cross-reference column. SBH will use an integrated M&E approach to cross-reference SBH indicators appropriately in analysis and reporting.
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting SBH Goal: Improved health outcomes for Zambians by strengthening systems that underpin the delivery of high quality health services and increasing the utilization of high impact health interventions at district and community level. SBH Impact and Outcome Level Indicators: Proportion of deliveries with assistance from a ZNHSP; 1 23.1% 23.1% 29.1% 33.2% 38.1% 43.1% 43.1% Ministry of Health-HMIS Quarterly medically trained provider CDCS in targeted districts Couple years of protection in targeted districts 2 CDCS 268,071 268,071 277, 322 430,217 440,141 446,360 1,862,111 Ministry of Health-HMIS Quarterly
Percent of children aged Ministry of Health-HMIS 12 to 23 months fully CDCS Semi- 4 94% > 90% > 90% > 90% > 90% > 94% > 94% and Central Statistics immunized in targeted Annual ZNHSP Census 2010 Projection districts Percentage of adults and children known to be alive PEPFAR MER and on treatment 12 5 TX-RET; 79% >79% >82% >84% >86% >90% >90% PEPFAR DATIM Quarterly months after initiation of 3.1.1-78 antiretroviral therapy in targeted districts 4 SBH Outcome and Output Level Indicators: Task 1. Design, implement and monitor national level interventions to strengthen health stewardship by MOH [Result 1: Strengthened health stewardship of MOH ] Sub-task 1.1: Strengthen Human Resource Planning and Management
4 The SBH contracts include the following metric “ARV retention rates in targeted districts”, which will be measured by this standard indicator per USG and PEPFAR MER indicator definitions.
USAID Systems for Better Health ▌ pg. 33 SBH M&E Plan
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting Number of new health care workers who graduated from a pre- Indicator Institutional records, service training institution #H2.1.N/ project records 8 or program as a result of 3.1.1-81; 0 120 450 450 450 200 1,670 Annual PEPFAR-supported PEPFAR HRH- Register from the strengthening efforts, PRE training institutions within the reporting period Percent of targeted provincial and district level 40% PMO Staff Retention Report Semi- 9 health offices that have SBH 65% 80% 70% 85% 90% 90% 40% DMO and Appraisal Forms Annual access to and routinely use HRIS Proportion of PMO and DMO personnel who have 32.2% 32.2% received an annual PMOs PMO Staff Retention Report 10 SBH 35% 50% 70% 80% 80% Annual performance appraisal in 33.3%DMO 33.3% and Appraisal Forms the past year in targeted s DMO districts Number of non-finance based incentives to attract health professionals in 11 SBH 0 0 1 2 -- -- 3 SBH records Annual remote areas identified, costed, and submitted for review by GRZ Sub-task 1.2: Improve health care financing and public financial management Percent of national government expenditure USG, ZNHSP, 7% Ministry of Health – 13 7% 10% 11% 11.5% 12% 12% Annual on health out of general SBH (2015)5 Budget Status Report government expenditure Number of months per MOH Financial reports year in which targeted (central, disaggregated by 16 districts receive monthly SBH 9.3 8 9 10 11 12 12 Annual district) District records funding as per approved (approved budget) budget (within 30 days)
5 Republic of Zambia Ministry of Health and World Health Organization Regional Office for Africa. Zambia Rapid Health Sector Performance Assessment, October 2015.
USAID Systems for Better Health ▌ pg. 34 SBH M&E Plan
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting Proportion of funds disbursed to targeted District Financial Annual 17 districts out of total SBH 68.7% 70% 80% 90% 90% 95% 95% Annual Report approved annual budget of districts Proportion of funds disbursed (by targeted District Financial Annual 18 districts) to health facilities SBH 55.9% 55.9% > 56% > 56% > 57% > 58% > 58% Annual Report out of their approved total annual budget
Sub-task 1.3: Strengthen MOH capacity to oversee delivery of key health programs.
SBH documentation, Number of improvements MOH 19 to laws, policies, strategies, USG 0 0 7 2 2 0 11 Annual legal/policy/strategy/guide regulations, or guidelines line documents Percent of targeted 26 facilities submitting HMIS ZNHSP 65% 65% 75% 80% 85% 90% 90% Ministry of Health-HMIS Quarterly reports in a timely manner Task 2: Design, implement and monitor effective interventions to strengthen program management capacities of provincial and district health teams [Result 2: Strengthened program management capacities of provincial and district health teams]
Sub-task 2.1: Strengthen program management capacity of provincial and district health teams
Percent of target provinces /districts that SBH records, capacity Baseline, demonstrate capacity to framework tools, 6 SBH TBD 20% 40% 60% 80% 100% 100% Midline, sustainably plan, manage TSS/mentoring tools, EOP and oversee accessible HMIS/DHIS2 high quality health services Percent of targeted facilities in target districts SBH records, capacity that demonstrate capacity framework tools, Baseline, 7 to deliver high quality SBH TBD 0 30% 50% 70% 85% 85% TSS/mentoring Midline, health services and engage monitoring tools, EOP with communities in their HMIS/DHIS2 catchment area.
Capacity score of targeted Capacity strengthening 23 SBH TBD TBD TBD TBD TBD TBD TBD Semi- provinces and districts tool, SBH records Annual (to
USAID Systems for Better Health ▌ pg. 35 SBH M&E Plan
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting be confirmed) Percent of targeted 40% 40% 100% 100% 100% 100% 100% provinces/districts that conducted at least two Province Province Province Province Province Province Province Province and District 24 RFTOP Annual reviews of their annual 90% 90% 100% 100% 100% 100% 100% Action Plans action plans, in the past District District District District District District District year 100% 100% 100% 100% 100% 100% 100% Percent of targeted PMOs Province, Province Province Province Province Province Province HMIS/DHIS2, SBH 25 / DMOs that use HMIS SBH Quarterly 40% records data routinely 40% 100% 100% 100% 100% 100% District Districts District District District District District Number of health Quarterly, managers and providers SBH, ZMLA M&E Semi- 27 SBH 0 0 250 0 0 0 250 trained in management and Training register annually, leadership Annually Percent of target facilities that did not experience a HMIS/DHIS2; logistics 28.a stock-out of Coartem SBH 93.3% 93.3% 100% 100% 100% 100% 100% management information Quarterly (ACT) during the system reporting period Percent of target facilities that did not experience a HMIS/DHIS2; logistics 28.b stock-out of contraceptive SBH 73% 73% 80% 80% 90% 100% 100% management information Quarterly (injectable)during the system reporting period Sub-task 2.2: Improve technical capacity of provincial and district health teams to deliver quality health services in facilities Number and percent of children who received Indicator # Ministry of Health HMIS Semi- 30 DPT3 vaccine by 12 94% >90 >90 >90 >90 >90 >90 3.1.6-61 (HMIS CHN3-045) Annual months of age in targeted districts Percent of HIV-positive pregnant women who PMTCT_ARV 31 received antiretroviral _NAT, 3.1.1- TBD 90% 90% 90% 90% 90% 90% HMIS/DHIS2/SmartCare Quarterly treatment to reduce risk 86 of mother-to-child-
USAID Systems for Better Health ▌ pg. 36 SBH M&E Plan
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting transmission in targeted districts Percentage of targeted DMOs that have completed their semi- 70% PA 80% PA annual performance 40% PA 40% PA 60% PA 60% PA 80% PA PA Monitoring Tools, 32 SBH 70% 80% TSS Annual assessment and technical 10% TSS 10% TSS 60% TSS 60% TSS 80% TSS SBH records supportive supervision TSS visits to facilities in their catchment area Percent of targeted DMOs that demonstrate specific HMIS/DHIS2, PA performance 33 SBH 90% 90% 100% 100% 100% 100% 100% Monitoring Tools, SBH Annual improvements since their records previous performance assessment
60% - 90% 90% TSS 90% TSS 90% TSS Percent of targeted 57.5% - TSS TSS TSS 90% TSS primary care facilities that 85% 90 100% 41% - 41% - 100% 100% TSS Monitoring Tools, Semi- 34 have received technical RFTOP, SBH Clinical Clinical Clinical Clinical Clinical Clinical Clinical Mentorship forms Annual supportive supervision or Mentorin Mentori Mentori Mentoring Mentori Mentoring clinical mentoring g ng Mentori ng ng ng Percent of targeted facilities in targeted districts that have up-to- SBH records, 35 SBH 0% 0% 60% 80% 100% 100% 100% Quarterly date and gender sensitive mentor/TSS records job aids for HIV, FP, MNCH, and/or nutrition Percent of targeted facilities in targeted districts that have initiated SBH records, Meeting 36 SBH, PEPFAR 0% 0% 50% 50% 70% 85% 85% Annual QI projects in ART, minutes, QI Reports PMTCT, MC, FP, child health and nutrition, or maternal health services
USAID Systems for Better Health ▌ pg. 37 SBH M&E Plan
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting with documented process results6
Number of people trained Indicator 37 in child health and 0 120 1,500 1,500 1,500 100 4,720 SBH Training Registers Quarterly #3.1.9-1 nutrition7 Number of people trained 38 in maternal and newborn USG/F 0 60 450 500 570 0 1,580 SBH Training Registers Quarterly health Number of people trained in family 39 USG/F 0 40 100 150 150 0 440 SBH Training Registers Quarterly planning/reproductive health Number of new health care workers who TSS and Mentoring successfully completed an Indicator 40 0 150 500 500 1,000 500 2,650 Tools, SBH Training Quarterly in-service training program #3.1.1-84 Registers within the reporting period Task 3: Provide technical and financial assistance to the GRZ and community-based organizations to increase the quality, availability and use of priority health services at the community level in targeted districts [Result 3: Improved capacities of MOH and community-based organizations to increase quality, availability and use of priority health services, and promote better health through prevention and healthy behaviors, at the community level in targeted districts] Sub-task 3.1: Improve capacity to deliver quality health services at the community level Number of CHAs that receive routine supervision Mentorship forms, PA, 43 (from health facility staff) SBH 0 0 70 150 150 50 420 TSS report or trip Quarterly in target facility catchment report, grantee records areas
6 This indicator incorporates PEPFAR indicator “Percentage of PEPFAR-supported clinical service sites with QI activities implemented that address clinical HIV program processes or outcomes and have documented process results in the last 6 months”
7 In alignment with USG indicator guidance, the indicators 37, 38 and 39 will cover the following: health professionals, primary health care workers, community health workers, volunteers, and non- health personnel.
USAID Systems for Better Health ▌ pg. 38 SBH M&E Plan
Targets/Milestones Frequency No. Indicator Links Baseline Data Source of data Year 1 Year 2 Year 3 Year 4 Year 5 EOP reporting Number of community volunteers (including SMAGs, others) that have SBH Delivery and Goods 44 received equipment to SBH 0 0 912 912 0 0 1,824 Quarterly Receive Notes deliver priority community health services in targeted facility catchment areas Sub-task 3.2: Strengthen linkages between the community and facility for key health interventions Number of active beneficiaries served by PEPFAR OVC programs 46 for children and families PEPFAR 0 0 1500 TBD TBD TBD TBD SBH Data Tracking Tool Quarterly affected by HIV/AIDS in target areas
Percent of target health facilities with a functional SBH records, facility/QI 48 facility/community-level SBH 0 0 80% 60%8 80% 80% 80% Quarterly documentation Quality Improvement (QI) committee Sub-task 3.3: Implement community level Social and Behavioral Change Communication (SBCC) interventions to increase utilization of high impact health services
Percent of children under five years of age who >80% in each Indicator # Semi- 49 received Vitamin A from 171% 171% >80% >80% >80% >80% targeted HIMS/DHIS 2 3.1.9.2-3 Annually USG-supported programs district in target districts Notes: HRIS: Human Resources Information System EOP: End of Project ZNHSP: Zambia National Health Strategic Plan 2011-2015 HMIS: National Health Management Information System ZDHS: Zambia Demographic Health Survey DHIS2: District Health Information System Version 2.0 ZHWRS: Zambia Health Worker Retention Scheme
8 The target dips in Year 3 because interventions will be initiated in a new cohort of districts during this period, per the phased approach. In reporting, we will disaggregate results by phase cohort.
USAID Systems for Better Health ▌ pg. 39 SBH M&E Plan
Annex 1: Summary of Indicator Revisions
Current Indicator Number and Modifications from Previous AMEP Rationale for Change Effective Date of Change Statement Modern contraceptive use is typically measured through population-based Original indicator (AMEP Version 1): SBH 5/1/2016. This change is survey methods. To enable SBH to SBH 2: Couple years of protection in 2: Modern contraceptive use in targeted reflected in SBH contract capture changes to contraceptive targeted districts districts. modification AID-OAA-I-14- coverage more routinely, in discussion 00032/AID-611-TO-16-00001. with USAID/Zambia, the original This indicator was replaced with “Couple Modification No. 02 indicator was replaced with CYP. CYP years of protection in targeted districts” can be tracked using routine data captured through HMIS/DHIS2. This indicator was deleted from the 5/1/2016. This change is SBH AMEP as a result of changes in reflected in SBH contract Original indicator (AMEP Version 1): SBH the SBH scope of work and the modification AID-OAA-I-14- 3: Prevalence of stunting among children deletion of "Sub-task 3.3: Implement 00032/AID-611-TO-16-00001. under five years old in five targeted high impact community level Nil Modification No. 02 districts. Nutrition and Social and
Behavioral Change Communication
This indicator was deleted. (SBCC) interventions". This deletion
was jointly agreed with
USAID/Zambia. Original indicator (AMEP Version 1): SBH 7: Percent of target facilities in target 5/1/2016. This change is SBH 7: Percent of targeted facilities in districts that demonstrate capacity to After careful review of the indicator, reflected in SBH contract target districts that demonstrate capacity sustainably deliver quality health services the indicator wording was refined to modification AID-OAA-I-14- to deliver high quality health services and and engage with empowered communities support more feasible measurement. 00032/AID-611-TO-16-00001. engage with communities in their to improve access and use of priority Modification No. 02 catchment area. services.
This indicator was revised to read:
"Percent of targeted facilities in target
districts that demonstrate capacity to
deliver high quality health services and
engage with communities in their catchment area."
USAID Systems for Better Health ▌ pg. 40 SBH M&E Plan
Original indicator (AMEP Version 1): SBH 5/1/2016. This change is SBH 25: Percent of targeted PMOs / 25: Percent of targeted PMOs/DMOs that After careful review of the indicator, reflected in SBH contract DMOs that use HMIS data routinely have access to program and health and current functionality and use of modification AID-OAA-I-14- indicator performance reports (generated DHIS2 in Zambia, the indicator was 00032/AID-611-TO-16-00001. through DHIS2). updated to more accurately reflect Modification No. 02 the status of information use in This indicator was revised to read: Zambia. “Percent of targeted PMOs / DMOs that use HMIS data routinely” Original indicator (AMEP Version 1): SBH 29: Percentage of CBDs trained by SBH in 5/1/2016. This change is target districts who submitted reports Due to potential data collection reflected in SBH contract Nil monthly and received commodities when challenges, SBH proposed to delete modification AID-OAA-I-14- needed. this indicator. 00032/AID-611-TO-16-00001. Modification No. 02 This indicator was deleted. 5/1/2016. This change is This indicator was deleted due to Original indicator (AMEP Version 1): SBH reflected in SBH contract changes in SBH scope of work and the 42: Number of children reached with modification AID-OAA-I-14- removal of "Sub-task 3.3: Implement nutrition information and services through 00032/AID-611-TO-16-00001. Nil high impact community level USG-supported programs. Modification No. 02 Nutrition and Social andBehavioral
Change Communication (SBCC) This indicator was deleted. interventions". This deletion was
jointly agreed with USAID/Zambia. This indicator was deleted in an effort to streamline the number of 5/1/2016. This change is indicators in the SBH AMEP. Since the reflected in SBH contract Original indicator (AMEP Version 1): SBH AMEP includes another similar metric modification AID-OAA-I-14- 45: Number of HIV positive children and on ART coverage (“Percentage of 00032/AID-611-TO-16-00001. adolescents on ART in targeted areas. adults and children known to be alive Modification No. 02 and on treatment 12 months after
This indicator was deleted. initiation of antiretroviral therapy in
targeted districts"), this indicator was
removed. This deletion was jointly agreed with USAID/Zambia.
USAID Systems for Better Health ▌ pg. 41 SBH M&E Plan
The revision was made to Original indicator (AMEP Version 1): SBH 5/1/2016. This change is accommodate differing 36: Percent of targeted facilities in reflected in SBH contract implementation timelines among targeted districts that have initiated QI modification AID-OAA-I-14- facilities. projects in ART, PMTCT, MC, FP, child 00032/AID-611-TO-16-00001.
health and nutrition, or maternal health Modification No. 02 SBH 36: Percent of targeted facilities in services with documented process results targeted districts that have initiated QI in past 6 months. projects in ART, PMTCT, MC, FP, child
health and nutrition, or maternal health This indicator was refined to read: services with documented process results "Percent of targeted facilities in targeted
districts that have initiated QI projects in
ART, PMTCT, MC, FP, child health and
nutrition, or maternal health services with
documented process results".
Original indicator (AMEP Version 1): SBH On request by USAID/Zambia, this 5/1/2016. This change is SBH 46: Number of active beneficiaries 46: Number of HIV positive children and indicator was replaced to reflect the reflected in SBH contract served by PEPFAR OVC programs for adolescents with active case management currently used PEFPAR indicator. This modification AID-OAA-I-14- children and families affected by HIV/AIDS plans that include social services within was jointly agreed with 00032/AID-611-TO-16-00001. in target areas the community in targeted areas. USAID/Zambia to ensure SBH is Modification No. 02 responsive to PEFPAR reporting This indicator was replaced "Number of requirements. , active beneficiaries served by PEPFAR OVC programs for children and families affected by HIV/AIDS in target areas”. SBH determined that this indicator Original indicator (AMEP Version 1): SBH was not going to provide meaningful 5/1/2016. This change is 47: Percent of target health facilities with information since community action reflected in SBH contract community action plans in targeted Nil plans are already a routine part of the modification AID-OAA-I-14- districts. health facility planning process 00032/AID-611-TO-16-00001.
(therefore all facilities are likely to Modification No. 02 This indicator was deleted. already have community action plans).
USAID Systems for Better Health ▌ pg. 42 SBH M&E Plan
This indicator was deleted from the Original indicator (AMEP Version 1): SBH SBH AMEP as a result of changes in 50: the SBH scope of work and the 5/1/2016. This change is a) Prevalence of children 6-23 months deletion of "Sub-task 3.3: Implement reflected in SBH contract receiving a minimum adequate diet high impact community level modificationAID-OAA-I-14- b) Women’s dietary diversity score Nutrition and Social and 00032/AID-611-TO-16-00001. c) Prevalence of underweight women of Behavioral Change Communication Modification No. 02 reproductive age (SBCC) interventions". These d) Prevalence of households with Nil deletions were jointly agreed with moderate or severe hunger USAID/Zambia. e) Prevalence of exclusive breastfeeding of
children under six months of age
Indicators SBH 50a-e were deleted.
USAID Systems for Better Health ▌ pg. 43 SBH M&E Plan
Annex 2: List of Target Facilities by Province and District
SBH Phase 1 Target Districts and Health Facilities
Provinces Central
Districts Kabwe Mkushi Bwacha Changilo (Chengelo) Chowa Chisanaga Kabwe General Hospital Fiwila Kalwelwe Kabengeshi Kang'omba Kakushi Kasanda Luanshimba Kasavasa Mikunku Target Facilities Kawama Miloso Katondo Milungwe Makululu Mkushi District Hospital Mukobeko Township Mulungwe Munga Munsakamba Natuseko Nambo Ngungu Nkolonga Pollen Twatasha Number 15 15 Provinces Copperbelt Districts Kitwe Mufulira Buchi Main Butondo Chavuma Clinic 1 Chimwemwe Clinic 2 Garnatone Clinic 3 Itimpi Luansobe Kakolo Kamuchanga Clinic Kawama Kamuchanga Hospital Target Facilities Kwacha Kansuswa Kitwe Central Hospital Mokambo Luangwa Murundu Mindolo 1 Mutundu Mulenga Ronald Ross General Hospital Ndeke Village Hospital Taung-up Wusakile Twatasha Zambia National Service 14 Miles Number 15 15
USAID Systems for Better Health ▌ pg. 44 SBH M&E Plan
Provinces Eastern Districts Chipata Petauke Chikando Chisenjere Chinunda Kakwiya Chipangali Kalindawalo Chiparamba Luamphande Chipata General Hospital Manyane Chizenje Mawanda Jerusalem Merwe Target Facilities Kamulaza Minga Hospital Kapata Mumbi Kasenengwa Mwanjabanthu Kwenje Nyamphande Madzimoyo Petauke District Hospital Mkanda Petauke Urban Muzeyi Ukwimi A Mwami Hospital UkwimiB Number 15 15 Provinces Lusaka Districts Lusaka Shibuyunji Bauleni Kapyanga Chainda Mukulaikwa Chawama Mwembezhi Chelstone Nampundwe Chilenje Shabasonje Chipata Sichobo George Target Facilities Kalingalinga Levy Mwanawasa Hospital Kamwala Kanyama Matero Reference Mutendere Ng'ombe University Teaching Hospital Number 15 6 Provinces Southern Districts Livingstone Monze Boma Bweengwa Target Facilities Dambwa North Chikuni Hospital
USAID Systems for Better Health ▌ pg. 45 SBH M&E Plan
Hilcrest Chisekesi Libuyu Hakunkula Linda Kanundwa Livingstone Central Hospital Katimba Mahatima Ghandi Keembe Maramba Luyaba Mosi-o-tunya Manungu Nakatindi Monze Mission Hospital Police Clinic Monze Urban Prison Clinic Moomba St Josephs Nampeyo Zambia Air Force Njola Mwanza Zambia Army Siatontola Number 15 15
USAID Systems for Better Health ▌ pg. 46
Annex 3: Detailed Theory of Change by Result
This Annex describes in more detail the theory of change and assumptions within the SBH results structure. It also explains how the SBH implementation model lead to the changes we aim to influence in order to reach our intended results (see Section 3, Figures 1 and 2).
Result 1: Strengthened health stewardship of MOH Evidence shows that a strong health system is necessary for high level service availability, quality, and utilization. Evidence also shows that stewardship is a prerequisite for a flourishing system and is the cornerstone of system performance. SBH implements strategies that build institutional and individual capacities and systems to assist the MOH to be a better steward of the Zambian health system. Improved stewardship of Zambia’s health system by MOH leaders is dependent on improvements in two critical components of the country’s health system-(1) human resources planning, management, and mobilization and (2) management of finances. Both support and enable improvements in service delivery. SBH implement carefully designed technical support interventions, leading to more effective functioning of these systems, along with enhanced capacity of national level leaders and managers to oversee delivery of key health programs. Result 1 is in direct alignment with priorities outlined in the Zambia National Health Policy and reinforces the recognition that systems improvements must occur to enable improved health in Zambia. Result 1 is supported by three sub-results, which separate institutional capacity building according to core health system functions: human resources (sub-result 1.1), health financing and public financial management (sub-result 1.2), and oversight of service delivery (sub-result 1.3). Only through the combination of improved systems functions will the GRZ be able to ensure its capacity to implement high quality and sustainable health and HIV/AIDS programs. SBH combine system strengthening support with skill building interventions to ensure that tools, processes, and systems are harmonized and the workforce has the right mix of capacities to manage and deliver quality health services. This approach to developing sustained capacity foster effective stewardship of the health system by MOH leaders and enable managers and health providers at each level of the system to be accountable for service delivery performance and health outcomes. MOH leaders use improved systems, analyze information to address critical bottlenecks, and mobilize and deploy resources (financial and human) strategically to achieve objectives. Sub-Result 1.1: Strengthened human resource planning and management The Abt team support human resources teams within the MOH at the central level to implement existing strategies and systems and develop new strategies. SBH interventions in the areas of health financing (see sub-result 1.2) will enable the GRZ to sustainably avail resources to adequately staff and retain health workers. We will assess the appropriateness of results based financing (RBF) interventions to help generate the necessary incentives to motivate health workers and improve their performance. To enable Zambia’s health system to successfully support the delivery of high quality health services, SBH believes that:
The HR information system (HRIS) and performance management package (PMP) must be fully implemented to enable managers at all levels to use evidence to make or advocate for decisions related to planning and allocation of resources, and hold individuals and entities accountable for delivering on stated performance goals. Qualified health providers must be distributed rationally, with specific emphasis on placing trained providers in rural and remote areas.
47
Health workers must have practical and clinical skills to ensure effective delivery of high quality health services. Zambian health workers require enhanced capacities at the pre-service and in-service levels so that all providers have up-to-date competencies to meet needs in Zambia. SBH support the MOH to make or advocate for decisions related to addressing these issues. Sub-Result 1.2: Improved health care financing and public financial management Releasing public funds and budget execution in a timely manner is a key challenge for the Zambian health sector. While the MOH’s Governance and Management Capacity Strengthening Plan (GMCSP) and the MOF’s broader Public Financial Management (PFM) reform are improving financial management of the health sector, it is important to strengthen evidence-based planning and budgeting, financial management at the local level, and the system for reporting and auditing to ensure accountability and transparency. There is urgent need for a comprehensive and up to-date health financing strategy that reflects current health financing needs and takes Zambia’s socio-economic trajectory into consideration. SBH works with the MOH to implement a national health financing strategy that supports the mobilization of resources for the sector, effective allocation and management of budget, and roll out of strategic initiatives such as performance-based financing and social health insurance. Improved health financing supports the achievement of all other SBH Results and sub-results. In the absence of strong health financing, improvements among all other functions of the health system will be hindered, including service delivery. Sub-Result 1.3: Strengthened MOH capacity to oversee delivery of key health programs The ultimate aim of the health system is to ensure good health through access and availability of high quality health services. Although dependent on all other health system building blocks, service delivery functions include those that touch people directly. This sub-result is closely inter-related to sub-results 1.1 and 1.2. The MOH must have the capacity to cohesively bring together its decisions in all areas of the health system (including human resources, health financing, information, and logistics/supplies) to ensure that health services are delivered efficiently and effectively. In the past few years, there have been several major changes at the Ministry level on how the health sector is managed. For many years, health sector management was split into two ministries (Ministry of Health and Ministry of Community Development, Maternal Child Health) which led to some disruptions within health systems and structures that limited effective oversight. This organization recently reverted back to a single MOH structure, though challenges remain. There are insufficient resources and limited management oversight to properly roll out and ensure implementation of national policy guidance, programs, and services to lower levels of the health system. SBH tries to strengthen the enabling environment to deliver priority health programs and services. SBH knows that certain improvements will enable stronger performance across the system as a whole, such as when:
Ministry leaders and decision-makers have the skills, systems/processes, accountability mechanisms, and mandate to effectively oversee and support operationalization of policies and guidelines at subnational levels. Decisions are supported by the use of timely, accurate and user friendly-data. The health sector is successfully adapting policies and clinical guidelines to include current best and promising practices. And there is functional collaboration across sub-sectors to ensure optimal integration of services at the primary level.
48
In addition, SBH recognizes the importance of using information to guide decisions and plans to strengthen in-depth analysis and use of information by decision-makers to further emphasize continual performance improvements and increased accountability among stakeholders. With more robust tools and reports, the MOH managers at all levels will be enable to monitor performance and address what they see with corrective actions, resource allocation, selective capacity building, and other types of decisions that support positive change.
Result 2: Strengthened program management capacities of provincial and district health teams In support of Zambia’s decentralized governance approach, SBH recognizes that enabling provincial, district and facility teams to manage programs and services more effectively is equally important to the stewardship improvements at the national level. This is complemented by a central MOH that holds lower levels accountable for their performance (result 1). SBH’s theory of change contends that sustained improvements in management and delivery of quality health services will be achieved when:
Managers and health providers are able to monitor their progress against performance goals, identify systems barriers and capacity gaps, and access the technical and financial resources required to support continuous improvement (linked to result 1.3 above). Managers at all levels focus on production of more accurate, gender disaggregated data from existing sources, use the improved data to allocate resources, and hold individuals and teams in provinces, districts, and health facilities accountable for performance and results. PMOs and DCMOs have the capacity to effectively coordinate the inputs and activities of health implementing partners and stakeholders, ensuring that partner plans and allocation of resources are aligned with provincial and district plans. Quality Improvement Teams are functioning fully, supported by integrated application of the Performance Assessment, Technical Support Supervision (TSS) and Clinical Mentoring systems, along with needs-based in-service training. SBH designed the interventions under Sub-tasks 2.1 and 2.2 to equip health managers and service providers at province, district, and facility levels with the necessary skills, processes, and tools to deliver high quality HIV, FP, MNCH and nutrition services in targeted districts. Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams At provincial and district levels, managers do not consistently receive the capacity building and skills development required to support them in carrying out their vast responsibilities in managing health programs. Although the health sector has well-established systems for annual planning, performance assessment, and HMIS, there has been insufficient focus on ensuring these processes are applied rigorously or on helping managers utilize them in an integrated manner. This has mitigated the ability to use performance as an effective tool to ensure accountability within the health sector. SBH improves management capabilities in targeted provinces and districts to both ensure effective management of health programs therefore ensuring that resources are used appropriately and decisions are made to support the delivery of priority health services at the facility and community levels. By building a stronger culture of active data use for performance management, SBH helps to unlock more of the management power of performance assessment by strengthening the process and enabling routine use of information for better oversight and management at various levels. With improved competencies in using information to make decisions and better tools to measure performance routinely, stakeholders can both formulate decisions more effectively and ensure mutual accountability for strong performance, including the achievement of service delivery and health outcome targets. These improvements are envisaged to create a
49
stronger sense of ownership at all levels and lead to improved performance across programmatic and clinical areas. Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities SBH understands that improved capacity for program management is only one element of strengthening performance. While provincial and district managers require stronger management and accountability capacities (sub-result 2.1), provincial and district teams also need the technical and clinical skills to drive improved health outcomes. SBH link management and clinical capacity building interventions and ensure they are driven by performance data to enable strategic targeting of clinical training and skills and competencies are continually reinforced through supportive oversight and mentoring. Recognizing the inefficiencies of improving individual capacities of health workers if they are unable to provide services at their facilities due to lack of equipment or supplies, SBH ensures that effective procurement planning occurs to ensure that facilities have the equipment required to provide selected priority health services. Result 3: Improved capacities of MOH and community-based organizations to increase quality, availability and use of priority health services, and promote better health through prevention and healthy behaviors, at the community level in targeted districts Effective inclusion of communities in the health system is essential to achieving Zambia’s health sector goals. SBH community level interventions are designed to achieve enhanced ownership of health programs by communities in targeted districts in support of increased availability and use of quality health services. SBH’s theory of change contends that results will be improved when:
Communities are fully engaged in health planning where men, women and youth are empowered to prioritize their own health issues, identify locally-owned solutions, and access resources to implement these solutions. Community structures, such as Neighborhood Health Committees (NHCs) and Safe Motherhood Action Groups (SMAGs), include both women and men as members who: Serve as effective links between communities and the health system Access the resources and tools required to generate support for community health activities Hold health facilities accountable for delivery of appropriate, quality services and outreach. Community health assistants and volunteers are equipped with the necessary skills, tools, and incentives to effectively promote preventive and promotive care and positive health seeking behaviors. Accurate, timely data on disease incidence, health behaviors, and service delivery flows from communities into the HMIS and is used at each level of the system-community included to monitor performance and allocate resources. Community level MNCH, HIV, nutrition, and family planning interventions are effectively integrated and are delivered in an equitable, culturally appropriate manner and effective referral mechanisms from community to facility level are functioning. If SBH assists the MOH effectively achieve the results under sub-results 3.1, 3.2, and 3.3, this will bring improved and higher quality HIV, MNCH, FP, and nutrition services closer to the community and influence the demand and utilization of these services.
50
Sub-Result 3.1: Improved capacity to deliver quality health services at the community level Significant understaffing and lack of resources for transport limit the ability of providers to conduct routine community outreach activities, such as growth monitoring and antenatal care services. There is also infrequent monitoring of the quality of services provided by community volunteers and Community Health Assistants (CHAs), and facility-community partnerships remain week constraining efforts to cohesively prioritize and address health issues, including those that influence health-promotive and health-seeking behaviors. Even if broad health systems improvements under Results 1 and 2 are achieved, desired improvements in the access and use of health services by Zambians will not reach their full potential without changes at the community level. SBH use innovative approaches to more effectively reinforce the work of community volunteers and CHAs through improved engagement of communities in health planning and improved and integrated supportive supervision approaches. Further, SBH work closely with the Zambia Support for an AIDS- Free Era project to expand community-level HIV services to increase the number and geographic coverage of ART treatment adherence supporters and lay counselors, in support of the government’s long-term vision of decentralization of HIV care and treatment services into communities. SBH knows that it is critical to strengthen the engagement of the community level in the planning, management, and delivery of health services to reach desired improvements in access and use of priority services within targeted districts. Sub-Result 3.2: Strengthened linkages between the community and facility for key health interventions Health Center Advisory Committees (HCACs) are faced with limited opportunities for training and orientation. DMOs do not have an established position for community-level coordination and capacity- building and annual budgets cannot support establishment /revitalization of NHCs in every Zambian community. Training of community level actors is not consistently paired with activities that link trained volunteers/organizations into systems of mentorship, supervision, referral and reporting. The health system lacks a standardized referral system for CHAs/CBOs to refer clients to facilities and to communicate feedback from facilities to community to support follow up to clients. SBH elevate the community planning and health information systems through enhancing training and supervision of CHVs, establishing trained IEC/BCC Committees, engaging traditional leaders, and involving local NGOs and CBOs in health promotion and planning. Our periodic reviews of health performance provide data that communities use to hold the health system and community members accountable to health actions. SBH build upon the strengthened linkages to expand the QI system to health facility/community level. QI project implementation strengthen the ability of community members to access, understand, and use health data to hold health facility staff accountable to delivery of quality, equitable services. The collaboration between health workers and community members on QI projects further build collective ownership of health outcomes. Sub-Result 3.3: Implement community level Social and Behavioral Change Communication (SBCC) interventions to increase utilization of high impact health services In Zambia, under-nutrition has persistently affected children under the age of five years. In five Feed the Future (FTF) districts, linkages between food security and health at the community level is complicated by the wide set of players, policies, strategies, and donor initiatives related to nutrition and nutrition- related programs.
51
SBH supports scale-up of evidence-based nutrition interventions in five FTF target districts. To allay the existing challenge of cross-sectoral coordination, SBH supports the National Food and Nutrition Commission (NFNC) and the Ministry of Health to revise, or develop new clinical, program, or operational guidelines, training materials, and job aides for nutrition in consultation with the Nutrition Technical Working Groups. The contractor must support the NFNC and the Ministry of Health to integrate nutrition into other health program guidelines and training materials. In addition, the contractor must support development and implementation of the NFNC’s 2016-2022 Strategic Plan and provide technical and material assistance to strengthen M&E systems at the NFNC. The contractor must assist the NFNC and MOH to build the capacity of central level personnel to design and implement nutrition interventions. The contractor must support the NFNC and the MOH to coordinate nutrition partners and to strengthen coordination with the Ministry of Agriculture. Knowledge and awareness among the population is critical to positively influence health prevention, promotion and care seeking behaviors. While many other SBH results focus on improving the supply- side functions of ensuring access to high quality services, SBH also supports capacity building in social and behavior change communications (SBCC) interventions among stakeholders to ensure that the health sector can effectively influence the demand for high quality health services. SBH support social mobilization and health communication activities in targeted communities, expand service provision and data collection at community level through HIV, family planning, child health and nutrition volunteers, and support implementation of community health action plans.
52
Annex 4: SBH Performance Indicator Reference Sheets (PIRS)
53
SBH Impact and Outcome Level Indicators
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 1: Proportion of deliveries with assistance from a medically trained provider in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): Deliveries conducted by medically trained providers: In alignment with MOH definitions, this include deliveries conducted by trained providers including: Deliveries by skilled personnel (Midwife/Obstetrician): This is a count of all women whose delivery was done under the supervision of either a midwife or gynaecologist/obstetrician. and Deliveries by other skilled personnel (Nurses, CO, ML, MO): This is a count of all women whose delivery was done under the supervision of a nurse, clinical officer, medical licentiate or medical officer who cannot be classified as midwife, obstetrician or gynaecologist. These are health workers who have been educated and trained in the skills needed to manage normal (uncomplicated) pregnancies, childbirth and the immediate postnatal period, and in the identification, management and referral of complications in women and new- born.
In targeted districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: Deliveries by medically trained providers (Deliveries conducted by registered midwives and doctors in facility). Denominator: Expected deliveries
HMIS/DHIS2 Indicator Code: HMIS IRH4-025, IRH4-030, IRH4-035 Unit of Measure: Proportion of deliveries Disaggregated by: District, facility, and cadre Justification & Management Utility: Deliveries by medically trained providers is an important metric because skilled personnel are be able to recognize complications and intervene appropriately (treatment/referral). An increase in deliveries by appropriately trained providers contributes towards reduction in maternal and infant mortality. This indicator measures the proportion of deliveries conducted by medically trained health providers in targeted districts. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS 2 Data Source: SBH obtain data through HMIS/DHIS2, though the source documentation includes HMIS/DHIS2- Obstetric Tally or Activity Sheet HIA2. Method of Data Acquisition by USAID: The M&E Team extracts data from HMIS/DHIS 2 on a quarterly basis. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, part of routine reporting Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka
54
DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Potential limitations exist in relying on the HMIS as the primary data source; data quality may vary. Actions Taken or Planned to Address Data Limitations: The SBH M&E team works closely with the Ministry of Health M&E staff, including periodic data quality assessments. SBH also identify data anomalies and work with the MOH to conduct data validation exercises. Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Continual Reporting of Data: PMP data tables and narratives are used in reporting data. NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 23.1% ne 2016 23.1% 2017 29.1% 2018 33.2% 2019 38.1% 2020 43.1% EOP 43.1% THIS PIR SHEET WAS LAST UPDATED ON 24th 2016.
55
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved; Name of Indicator: SBH 2: Couple years of protection in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Couple Years of Protection - The estimated protection provided by family planning (FP) services during a one-year period, based upon the volume of all contraceptives sold or distributed free of charge to clients during that period
The CYP is calculated by multiplying the quantity of each method distributed to clients by a conversion factor, to yield an estimate of the duration of contraceptive protection provided per unit of that method (Wishik and Chen, 1973; Stover, Bertrand, and Shelton, 2000). The CYPs for each method are then summed over all methods to obtain a total CYP figure.
The USAID-endorsed CYP conversion factors for each method are:
Method CYP Condoms 120 units per CYP Oral Contraceptives 15 cycles per CYP Medroxyprogesterone, DMPA 4 doses per CYP injection Norethisterone enanthate injection 6 doses per CYP Implants - 3 year 2.5 CYP per implant - 4 year 3.2 CYP per implant - 5 year 3.8 CYP per implant IUCD (Copper-T 380-A) 4.6 CYP per IUD inserted Sterilisation: 10
Target districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: N/A
Denominator: N/A
HMIS/DHIS2 Codes: IRH3-050, IRH3-055, IRH3-060, IRH3-065, IRH3-070, IRH3-075, IRH3-080, IRH3-085, IRH3- 090, IRH3-095. Unit of Measure: Couple years of protection Disaggregated by: Geographic (district) Justification & Management Utility: CYP provides an estimate of coverage and an indication of the volume of program activity. SBH and USAID use it to monitor progress in the delivery of contraceptive services in areas supported by the project. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS 2 Data Source: HMIS/DHIS; USAID-approved CYP conversion factors
56
Method of Data Acquisition by USAID: The M&E Team have access to the HMIS/DHIS 2 and extract service data on quarterly basis. CYP will be calculated annually. Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): TBD Actions Taken or Planned to Address Data Limitations: This indicator is disaggregated by the type of contraceptive. Date of Future Data Quality Assessments: Annually Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator. Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Annually Reporting of Data: PMP data tables and narratives used in reporting data. NOTES Notes: Baselines/Targets and Others This indicator changed from AMEP Version 1 to AMEP Version 2. The original indicator included in AMEP Version 1 was “modern contraceptive use in targeted districts”. This was replaced with “CYP in targeted districts” to enable monitoring with routine HMIS data. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 268,071 ne 2016 268,071 2017 277, 322 2018 430,217 2019 440,141 2020 446,360 EOP 1,862,111 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
57
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 4: Percent of children aged 12 to 23 months fully immunized in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): According to WHO, a child is considered fully vaccinated if he or she has received a Bacille Calmette- Guerin (BCG) vaccination against tuberculosis; three doses of vaccine to prevent diphtheria, pertussis, and tetanus (DPT); at least three doses of polio vaccine; and one dose of measles vaccine. These vaccinations should be received during the first year of life.
Targeted Districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Fully Immunized: Per the MOH HMIS procedural manual, this is a count of children who have received all the required antigens before their first birthday. The standard series of immunization is BCG, DPT 1-3, OPV 0-3, PCV 1-3, RV 1-2 and measles.
Numerator: Number of children aged 12 to 23 months fully immunized in target districts Denominator: Number of children aged 12 to 23 months in target districts
HMIS/DHIS2: CHN3-080 Unit of Measure: Percent Disaggregated by: District, facility Justification & Management Utility: Vaccination coverage is an important metric because when a larger proportion of children is immunized the chances of immunizable disease outbreaks, for example measles, are reduced, and more children are protected against such diseases. The goal is to ensure survival and healthy development of children through maintenance of a high level of immunization. This indicator measure the proportion of children aged 12 to 23 months who are fully immunized in targeted districts. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS 2 Data Source: HMIS/DHIS 2, CSO Method of Data Acquisition by USAID: The M&E Team have access to the HMIS/DHIS 2 and run the report on a monthly basis. Frequency and Timing of Data Acquisition by USAID: This indicator is reported after every six months of the latest child health week. However, routinely monthly data are compiled for internal use and checked regularly to monitor performance and trends. Estimated Cost of Data Acquisition: Negligible, part of routine reporting Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD
58
Known Data Limitations and Significance (if any): The denominator used for this indicator is population data from Central Statistics Office (CSO) which is considered as official. This denominator often times creates some challenges during reporting period because the numerator is found to be higher than the denominator especially at district and province level. Actions Taken or Planned to Address Data Limitations: None because the CSO data is the official data which is used to track this indicator. However, facility head count data is used by the districts and the provinces. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Semi-annually and annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseline -- 94% 2016 2017
2018 Targets for each district will be determined 2019 > 90% in each targeted according to the baseline of each district. The increment by the end of the project period 2020 district should be no less than 94% for all districts
EOP > 94% in each targeted district THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
59
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Sub-Intermediate Result: 3.2.1 Service delivery improved Name of Indicator: SBH 5: Percentage of adults and children known to be alive and on treatment 12 months after initiation of antiretroviral therapy in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 PEFPAR MER TX-RET; 3.1.1-78 DESCRIPTION Precise Definition(s):
Note: In the SBH contract, the metric is worded “antiretroviral therapy retention rate in targeted districts.” Per OP and PEPFAR MER definitions, SBH will operationalize this by measuring the percentage of adults and children known to be alive and on treatment 12 months after initiation of antiretroviral therapy.
Targeted Districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: Number of adults and children who are still alive and on treatment at 12 months after initiating ART
Denominator: Total number of adults and children who initiated ART in the 12 months prior to the beginning of the reporting period, including those who have died, those who have stopped ART, and those lost to follow-up
Explanation of Numerator: The numerator requires that adult and pediatric patients must be alive and on ART at 12 months following their initiation of treatment. For a comprehensive understanding of survival, the following data must be collected: a) Number of adults and children in the ART start-up groups initiating ART at 12 months prior to the end of the reporting period (denominator) b) Number of adults and children still alive and on ART at 12 months after initiating treatment (numerator).
The reporting period is defined as a continuous 12-month period that has ended within a pre-defined number of months from the submission of the report. The pre-defined number of months can be determined by PEPFAR or national reporting requirements. If the PEPFAR reporting period is October 1, 2014 to September 30, 2015, countries will calculate this indicator by using all patients who started ART any time during the 12-month period from October 1, 2013 to September 30, 2014. A 12-month outcome is defined as the outcome, i.e., whether the patient is still alive and on ART, dead or lost to follow-up, 12 months after starting. For example, patients who started ART during August 2014 will have reached their 12-month outcomes in August 2015 (e.g., August 4, 2014-August 3, 2015). Patients who started ART during January 2014 will have reached their 12-month outcomes in January 2015.
The numerator does not require patients to have been on ART continuously for the 12-month period. Patients may be included in the numerator (and denominator) if they have missed an appointment or drug pick-up or temporarily stopped treatment during the 12 months since initiating treatment, as long as they are recorded as still being on treatment at month 12.
Explanation of Denominator: The denominator is the total number of adults and children in the (monthly) ART start-up groups who initiated ART at a point 12 months prior to the beginning of the reporting period, regardless of their 12-month outcome. (i.e., died, LTFU, stopped); this includes those “New” on ART as well as those who “Transferred In” if they have a cohort-start date within the reporting period of interest. At the facility level, the Transfers Out (TO) are be taken out of the denominator
60
as well as the numerator. It is assumed that if a patient transfers out from an ART facility, that patient will be a “transfer in” at a new ART facility. Logically, facilities and programs may visualize this calculation of the denominator as the facility or program is no longer responsible for an ART patient who has officially transferred out to another ART facility. For example, for the reporting period October 1, 2014 to September 30, 2015, this includes all patients who started ART during the 12-month period from October 1, 2013 to September 30, 2014. This includes all patients, both those on ART as well as those who are dead, have stopped treatment or are lost to follow-up at month 12. Again the denominator includes patients that have transferred in (and their initiation date is known) and excludes patients that transferred out during the time period.
This indicator should NOT be estimated. This indicator should be calculated directly from information gathered in standard cohort ART registers or tabular analysis from electronic patient level databases.
Country teams should ensure that all sites are reporting on the same 12 ART start-up groups. Only sites that have been operational for at least 24 months prior to the end of the reporting period should report, so that all sites report on the same 12 ART start-up groups. PEPFAR country teams may use the USG FY reporting period as the timeframe for the 12 ART start-up groups. Country teams should record how many ART sites are reporting on this indicator and seek to ensure reporting among all eligible ART sites (i.e., operational for 24 months) by the end of FY 2015.
Sites are encouraged to disaggregate retention by health status at initiation (e.g., CD4 count or WHO stage) to review the retention of every ART start up group on a continuous basis, to summarize the data at regular intervals (e.g., monthly), and to use this information to improve follow-up and retention of patients.
HMIS CODES: HMIS HIV4-330 and HIV4-340 Unit of Measure: Number of adults and children Disaggregated by: Geographic (district), age, sex Justification & Management Utility: This indicator measures the proportion of individuals who have retained on antiretroviral therapy (ART). ART is viewed by the scientific community and PEPFAR not only as essential for decreasing morbidity and mortality, but also as a highly effective approach to prevent HIV transmission. Death and loss to follow-up are the two highest causes of patient attrition from ART, especially in the first few months after initiating on ART. High retention is one important measure of program success, specifically in reducing morbidity and mortality, and is a proxy for overall quality of the ART program. Monitoring the program level retention is a critical quality of service indicator at the site, national and PEPFAR program levels as it can highlight barriers to health seeking behaviors and/or gaps in access to and provision of health services. This indicator is also important for long- term sustainability of the ART programs. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS 2 Data Source: HMIS/DHIS2, ART registers/databases Cohort/group analysis forms// SmartCare Method of Data Acquisition by USAID: The M&E Team have access to the HMIS/DHIS 2 and run the report on a monthly basis. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: N/A Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): The HMIS/DHIS 2 does not capture all the age disaggregation required by PEPFAR for this indicator. Actions Taken or Planned to Address Data Limitations: Date of Future Data Quality Assessments: TBD
61
Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine HMIS/DHIS 2 data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly, semi-annually, and annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 79% ne 2016 >79% 2017 >82% 2018 >84% 2019 >86% 2020 >90% EOP >90% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
62
Task 1: Design, implement and monitor national level interventions to strengthen health stewardship by MOH and MCDMCH
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 8: Number of new health care workers who graduated from a pre-service training institution or program as a result of PEPFAR-supported strengthening efforts, within the reporting period SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.1: Strengthened human resource planning and management Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 Indicator #H2.1.N/ 3.1.1-81; PEPFAR HRH-PRE DESCRIPTION Precise Definition(s): Pre-Service Training: Training under this indicator is defined as “pre-service” training—the training of “new” health care workers (see definition below). All training must occur prior to the individual entering the health workforce in his or her new position. In order to count, the duration of training must meet or exceed a minimum of six months. A pre-service training program must be nationally accredited, or at the minimum meet national and international standards. The program must also have specific learning objectives, a course curriculum, expected knowledge, skills, and competencies to be gained by participants, as well as documented minimum requirements for course completion. The duration and intensity of training will vary by cadre; however, all training programs should have at a minimum the criteria listed above.
Graduated: Individuals may be in training over many reporting periods; however, only participants who have graduated their training will be counted. Graduation may be documented by diploma or certificate. Individuals not meeting these documented requirements will not be counted in this indicator.
Health care workers: Refers to individuals involved in safeguarding and contributing to the prevention, promotion and protection of the health of the population (both professional and auxiliary-professionals)
The categories below describe the different types of health workers to be considered under this indicator. This not an exhaustive list of all health workers and position titles may vary from country to country. For the purposes of this indicator, health workers include the following: 1. Clinical health workers: Clinical health workers play clinical roles in direct service delivery and patient care: a) Clinical professionals, including doctors, nurses, midwives, laboratory scientists, pharmacists, social workers, medical technologists, and psychologists, and they usually have a tertiary education and most countries have a formal method of certifying their qualifications; b) Clinical officers, medical and nursing assistants, lab and pharmacy technicians, auxiliary nurses, auxiliary midwives, T&C counselors, and they should have completed a diploma or certificate program according to a standardized or accredited curriculum and support or substitute for university-trained professionals. 2. Non-clinical health workers: Non-clinical workers do not play clinical roles in a health care setting but rather include workers in a health ministry, hospital and facility administrators, managers, monitoring and evaluation advisors, epidemiologists and other professional staff critical to health service delivery and program support.
New health care workers: These are people who have graduated from the pre -service program. A health worker who advances to a higher cadre (e.g., a clinical assistant who completes training to become a clinical officer) shall be counted as a “new” health worker for the purposes of this indicator.
Pre-service training institution: Pre-service training institutions are university-based or affiliated schools of medicine, nursing, public health, social work, laboratory science, pharmacy, and other health-related fields. Non-professional or paraprofessional training would be any accredited and nationally recognized pre-service program that is a requirement for this cadre’s entry into the workforce.
63
Support: The type of support will either be direct or indirect.
Number: The number is the sum of new health care workers from the host country who graduated from a pre-service training institution within the reporting period with full or partial support. Individuals may be in pre-service training over several years, but will not count as graduated until they have completed their program. Local pre-service institutions may support other host country nationals under their program but those graduates should not be included in a country’s report on this indicator.
Numerator: A count of the number of new health care workers who graduated from a pre-service training institution or program in the reporting period Denominator: N/A Unit of Measure: Number of people Disaggregated by: Sex, cadre Justification & Management Utility: Zambia faces high crises in terms of Human Resource in the health sector. SBH supports three midwifery schools and other training institutions for the Community Health Workers (CHA) either directly or indirectly to increase the number of graduates from pre-service training institutions within the reporting period. The support will vary from one school to the other based on the school needs. Some of the support include, buying of equipment, learning materials etc. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Registration, graduation list Data Source: Institutional records, project records MOH Human Resource Information Systems (HRIS), training institutions, or accredited institution which is receiving support from SBH. Method of Data Acquisition by USAID: Annual report Frequency and Timing of Data Acquisition by USAID: Annually; data is collected and aggregated in time for PEPFAR reporting cycles. Estimated Cost of Data Acquisition: Negligible, part of routine reporting Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): The project depends on the quality of data at the source institutions, which can vary. At times, obtaining this data from institutions directly is challenging because of confidentiality. This indicator does not measure the quality of the pre-service training, nor does it measure the outcomes of the training in terms of the competencies of individuals trained, nor their job performance. This indicator does not measure the placement or retention in the health workforce of trained individuals. Actions Taken or Planned to Address Data Limitations: SBH developed a list of all these institutions from which data is sourced. A letter is written to the Ministry of Health and Ministry of Education or any other relevant authorities/board which are responsible for supervising the institutions to seek permission of accessing the names of graduates. SBH count all graduates who during the period of SBH support to a particular institution. A simple standard data collection tool was developed which the institutions are asked to fill in. Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: In close collaboration with Ministry of Health, the relevant training institutions, the M&E staff conducts post data quality verification for this indicator. This is done by verifying the names of the graduates and the names of those who were enrolled at the beginning of the course. Routine DQA approaches are done to ensure that data submitted data is correct. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator
64
Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Semi-annually and annually Reporting of Data: SBH routine program reports, data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseline -- 0 2016 120 2017 450 2018 450 2019 450 2020 200 EOP 1,670 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
65
Performance Indicator Reference Sheet Name of Development Objective: DO 3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 9: Percent of targeted provincial and district level health offices that have access to and routinely use HRIS SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.1: Strengthened human resource planning and management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Access to: This will mean that HROs have computers and have been trained in HRIS software.
HRIS: Human Resources Information System
Routinely use: The use of evidence-based decision making to recruit, reduce staffing shortages, attrition, and inequity of deployment
Targeted Provincial and District Level Health Offices: These are the provincial and district medical offices in SBH targeted provinces and districts.
Targeted Provincial: Central, Copperbelt, Eastern, Lusaka and Southern
Targeted Districts: Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: Number of target provincial/district medical offices that report accessing and using HRIS in the past three months
Denominator: All SBH-targeted provincial and/or district medical offices Unit of Measure: Percentage Disaggregated by: Geographic (province) Justification & Management Utility: Zambia struggles with high vacancy rates and insufficient educational capacity to produce the number of health workers required to meet current and projected needs. SBH supports the provincial/district medical offices to develop and implement the Human Resource Information System (HRIS) and technical assistance through the Human Resource Specialist (HRS) in the targeted provinces and districts. SBH Health System (HS) Specialists collaborates with the Provincial and/ or District Human Resource Officers (HROs) to lead training, organize data entry, procure computers for HRIS, and produce routine reports. At the subnational level, the ministry is not yet using the HRIS to guide decisions on recruitment, training, distribution of personnel. This information will help in utilization resource and guide the recruitment and distribution of human personnel. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Capacity Assessment Reports Data Source: Capacity Assessment Reports Method of Data Acquisition by USAID: The SBH M&E team works closely with the HRS at province/ district and HSS specialist in collecting this data. Frequency and Timing of Data Acquisition by USAID: Semi-annually
66
Estimated Cost of Data Acquisition: Negligible, part of routine reporting by HRS at province /district and HSS specialist or Capacity Building Assessments Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): This indicator is tracked routinely by HRS and HSS specialist measures the provincial offices and district offices that have access to HRIS. This requires further review by the M&E team and necessary documentation to verify that HROs are using HRIS in decision making. Actions Taken or Planned to Address Data Limitations: Routine data verifications and data quality assurance exercises Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approach is done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly, semi-annually, and annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 40% PMO ne 40% DMO 2016 65% 2017 80% 2018 70% 2019 85% 2020 90% EOP 90% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
67
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 10: Proportion of PMO and DMO personnel who have received an annual performance appraisal in the past year in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.1: Strengthened human resource planning and management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
PMO and DMO personnel: This means the Provincial Medical Officer and the District Medical Officers
Annual performance appraisal: MOH Annual Performance Appraisals are conducted annually, during the period May-July to review performance during the previous year.
Targeted provinces: Eastern, Central, Copperbelt, Luapula, Lusaka and Southern
Targeted districts; This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: Number of PMO/DMO personnel that received an annual performance appraisal in the reporting period Denominator: Total number of PMO or DMO personnel in targeted provinces and districts Unit of Measure: Percentage of personnel Disaggregated by: Province, district, and sex Justification & Management Utility: Ministry of Health staff is supposed to receive performance appraisal at different times to measure their performance in order to improve service delivery. However, this is not done consistently. SBH through the HRH specialist(s) developed a strategy to orient health facility personnel on how to use the Performance Management Package, with a focus on the Annual Performance Appraisal System (APAS). SBH district-level HS Specialists works with the HROs to measure the extent to which PMO and DMO are receiving annual performance appraisals through the use of the Annual Performance Appraisal System (APAS). PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: SBH data collection tools Data Source: SBH records, Performance Appraisal Records, other record Method of Data Acquisition by USAID: The SBH M&E team works closely with the HRS and HSS specialist in collecting this data by requesting Performance Assessment Reports from Provincial Health. Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES
68
Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Annual Performance Appraisals have no fixed dates on which they are conducted and often times not done consistently. This creates some problems in terms of timing of reporting. Actions Taken or Planned to Address Data Limitations: Through our HRH and HSS specialist, deliberate efforts are made so tentative calendar dates have been developed with the HROs to ensure that performance appraisals are done with respective cadres at the arranged dates. Date of Future Data Quality Assessments: Annually Procedures for Future Data Quality Assessments: In close collaboration with HRO, Program Staff, SBH M&E team conduct annual reviews of the performance appraisal forms for this indicator, in alignment with the SBH M&E Plan. Routine DQA approach is done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes -- 32.2% PMO Baseline 33.3% DMO 32.2% PMO 2016 33.3% DMO 2017 35% 2018 50% 2019 70% 2020 80% EOP 80% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
69
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 11: Number of non-finance based incentives to attract health professionals in remote areas identified, costed, and submitted for review by GRZ SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH and MCDMCH SBH Sub-Result 1.2: Improved health care financing and public financial management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Non-finance based incentive: This includes non-monetary based incentives, such as additional training, mentoring or supervision, subsidies for medicines or other goods, uniforms or other clothing, food, and training, transportation, or others.
Identified, costed and submitted: Options for non-financial incentives are identified, costed, and submitted to the MOH for consideration.
Remote areas: According to the Zambia Health Workers Retention Scheme Guideline, remote areas means that there should be a greater degree of difficulty in reaching the facility due to bad or seasonally impassable gravel or dirt roads and/or having to use a boat to get to the facility for health care. (Rural Health Centres along tarred roads do not qualify.)
Numerator: Number of non-finance incentives identified, costed and submitted for review Denominator: N/A Unit of Measure: Number of non-finance based incentive proposals Disaggregated by: N/A Justification & Management Utility: The MOH is currently facing a number of challenges to retain health professional in remote area despite providing rural hardship monetary incentives. Traditionally non-finance based incentives such as certificates of recognition, training, supervision, mentorship, availability of equipment, material support (accommodation, transport) have been known to attract health professionals in remote areas. SBH identify viable options for the provision of non-financial incentives in remote areas. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: SBH project documentation Data Source: SBH records Method of Data Acquisition by USAID: Document review and interpretation Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): N/A Actions Taken or Planned to Address Data Limitations: N/A
70
Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0 No activities planned for Year 1 2017 1 2018 2 2019 -- No activities planned for Year 2 2020 -- No activities planned for Year 5 EOP 3 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
71
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 13: Percent of national government expenditure on health out of general government expenditure SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.2: Improved health care financing and public financial management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
General Government Expenditure on Health: The sum of outlays by government entities to purchase health care services and goods. It comprises the outlays on health by all levels of government, social security agencies, and direct expenditure by parastatals and public firms. Besides domestic funds, it also includes external resources (mainly as grants passing through the government or loans channeled through the national budget).
General Government Expenditure: Includes consolidated direct outlays and indirect outlays (e.g., subsidies to producers, transfers to households), including capital of all levels of government, social security institutions, autonomous bodies, and other extra budgetary funds.
Numerator: General government expenditure on health Denominator: General government expenditure Unit of Measure: Percentage Disaggregated by: Health program/priority (including HIV/AIDS and other priorities in health i.e. nutrition if possible)
Justification & Management Utility: Zambia’s health sector is underfinanced, and the level of financing from donors is unsustainable. Zambia’s draft National Health Accounts (MOH, 2013) states that nearly 40% of total health expenditures were donor-financed in 2010. Other key challenges relate to timely release of public funds by the Ministry of Finance (MOF) and effective budget execution.
SBH measure the percent of national government expenditure on health against the general government expenditure. The Health Finance and Planning Specialist will lead interventions to finalize Zambia’s national Healthcare Financing Strategy and provide support for key components of this strategy.
PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Data extracted from routine statistics/reports of Ministry of Finance Data Source: Annual Government Financial Report by MOF or Central Ministry Method of Data Acquisition by USAID: The SBH M&E team works closely with the Health Finance in collecting this data by requesting from the Ministry of Finance and Ministry of Health Central Office. Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: None, data source from existing/routine government data Individual Responsible at USAID: COR, Systems for Better Health (SBH) Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES
72
Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): SBH depends on the level of detail available in Ministry of Finance expenditure reports. Timing of release of data is outside of project control. Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Annually Procedures for Future Data Quality Assessments: Routine DQA approach is done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 7% ne 2016 7% 2017 10% 2018 11% 2019 11.5% 2020 12% EOP 12% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
73
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 16: Number of months per year in which targeted districts receive monthly funding as per approved budget (within 30 days) SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.2: Improved health care financing and public financial management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Districts: This includes the specific districts selected by SBH for priority interventions (to be determined in Year 1 Quarter 2).
Receive monthly funding: The receipt of funding at the district level
Per approved budget: The amount budgeted for the month
Within the month: The monthly funding was received during the relevant/expected monthly time period. Unit of Measure: Number of months Disaggregated by: Geographic (province/district) Justification & Management Utility: Budget execution rates can be affected because of low levels of budget release to health facilities by districts. On the other hand, low level of disbursement by districts can be affected by delays in flow of funding from MOF to MOH, and then to provinces and districts. Indicators 16, 17 and 18 provide measures against these issues. SBH strengthen evidence-based planning and budgeting, build skills in financial management, and improve reporting and auditing to ensure accountability and transparency. This indicator measure out the 12 months in year, how many months the district received timely funding in alignment with their budget. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Budget and financial report review by SBH district staff Data Source: MOH Financial reports; District records (approved budget) Method of Data Acquisition by USAID: SBH district level staff obtain district budget and financial reports Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: None Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): SBH rely on the availability and completeness of MOH and MOF documentation. Actions Taken or Planned to Address Data Limitations: SBH district level staff collaborates closely with MOH to ensure access to documentation. Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING
74
Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 9.3 ne 2016 8 2017 9 2018 10 2019 11 2020 12 EOP 12 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
75
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 17: Proportion of funds disbursed to targeted districts out of total approved annual budget of districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.2: Improved health care financing and public financial management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): This indicator will measure, by district, the proportion of funds disbursed to the district level out of the total approved annual district budget.
Target Districts: Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Funds disbursed to districts: The actual amount disbursed to the district level
Total approved annual budget: Total approved budget for the specific district for the fiscal year
Numerator: Total amount of funds disbursed to the district Denominator: Total approved annual district budget Unit of Measure: Proportion Disaggregated by: Geographic (province/district) Justification & Management Utility: Budget execution rates can be affected because of low levels of budget release to health facilities by districts. On the other hand, low level of disbursement by districts can be affected by delays or erratic funding from MOF to MOH and then to provinces and districts.
SBH through the Health Finance and Planning Specialist plans to strengthen evidence-based planning and budgeting, build skills in financial management, and improve reporting and auditing to ensure accountability and transparency. This indicator will measure the proportion of funds disbursed to districts out of total approved annual budget of districts. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Document review Data Source: MOH Financial reports (central, disaggregated by district); District records (approved budget) Method of Data Acquisition by USAID: Document review and analysis Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD
76
Known Data Limitations and Significance (if any): SBH rely on availability of financial data from the GRZ (disbursement and budget data). Actions Taken or Planned to Address Data Limitations: SBH collaborates with MOH counterparts to identify and obtain necessary data. Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 68.7% ne 2016 70% 2017 80% 2018 90% 2019 90% 2020 95% EOP 95% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
77
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 18: Proportion of funds disbursed (by targeted districts) to health facilities out of their approved total annual budget SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH SBH Sub-Result 1.2: Improved health care financing and public financial management Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): This indicator will focus on SBH-targeted districts.
Funds disbursed by districts to health facilities: Total amount disbursed to health facilities within the annual reporting period
Approved total annual budget: MOH-approved annual district budget
Numerator: Actual amount of funds disbursed to health facilities during the year Denominator: Total approved annual district budget Unit of Measure: Proportion Disaggregated by: Geographic (province) Justification & Management Utility: Budget execution rates can be affected because of low levels of budget release to health facilities by districts. On the other hand, low level of disbursement by districts can be affected by delays in flow of funding from MOF to MOH and MCDMCH, and then to provinces and districts. SBH through the Health Finance and Planning Specialist strengthen evidence-based planning and budgeting, build skills in financial management, and improve reporting and auditing to ensure accountability and transparency. This indicator will try to measure proportion of funds disbursed (by districts) to health facilities out of their approved total annual budget. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Document review of financial reports, budgets Data Source: MOH Financial reports (central, disaggregated by district) District records (approved budget); Facility reports; HMIS/DHIS2 Method of Data Acquisition by USAID: Review and analysis of existing reports Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, TBD Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): SBH depend on the timely availability of relevant documentation to report on this indicator. Actions Taken or Planned to Address Data Limitations: TBD Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING
78
Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 55.9% ne 2016 55.9% 2017 >56% 2018 >56% 2019 >57% 2020 >58% EOP >58% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
79
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 19: Number of improvements to laws, policies, strategies, regulations, or guidelines SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH and MCDMCH SBH Sub-Result 1.3: Strengthened MOH and MCDMCH capacity to oversee delivery of key health programs Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): (In alignment with USG indicator definitions)
Improvements: Laws, policies, regulations, guidelines, or systems have been changed/revised/updated.
Laws, policies, strategies, regulations, guidelines: Laws, policies, strategies, regulations and guidelines include those developed with USG-assistance and formally endorsed by government, non-government, civil society and/or private sector stakeholders at the local, national, or international scale, with the intent to strengthen health service delivery, including quality, access, and sustainability. Unit of Measure: Number of improvements Disaggregated by: Type of document, program areas Justification & Management Utility: SBH works with the MOH in the different program areas to strengthen systems by improving laws, policies, regulations, guidelines, or systems. The main goal of the indicator is to track the number of improved laws, policies, regulation, guidelines, or systems. SBH works with MOH to identify such laws, policies, and regulations, guidelines to assist the MOH to adopt, implement, or institutionalize them in order to strengthen the health system. This indicator measures progress in improving the enabling environment for strengthening health systems and improving access and use of high quality priority health services. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Routine monitoring of project interventions/contributions Data Source: SBH documentation, MOH legal/policy/strategy/guideline documents Method of Data Acquisition by USAID: In close collaboration with program staff, SBH M&E team will work closely with the program staff for this indicator, in alignment with the SBH M&E Plan. Frequency and Timing of Data Acquisition by USAID: Annual Estimated Cost of Data Acquisition: Negligible Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Program staff may not capture all the improvements to laws, policies, regulations, guidelines, or systems that the project may influence or contributes. Actions Taken or Planned to Address Data Limitations: SBH institutionalize routine tracking mechanisms to capture relevant activities that contribute to this indicator. SBH actively participate in various TWGs to ensure it captures relevant policy level work that feeds into the indicator. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING
80
Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0 2017 7 2018 2 2019 2 2020 0 EOP 11 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
81
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.2 Health systems and accountability strengthened Name of Indicator: SBH 26: Percent of targeted facilities submitting HMIS reports in a timely manner SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 1: Strengthened health stewardship of MOH and MCDMCH SBH Sub-Result 1.3: Strengthened MOH capacity to oversee delivery of key health programs Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 HMIS Manual(2008) #INF 20 DESCRIPTION Precise Definition(s):
Targeted facilities: This includes the specific facilities within the targeted districts selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Targeted district: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Submitting HMIS reports in a timely manner: Reporting unit data timeliness rate within targeted geographic area
Numerator: Number of facilities (reporting unit) within a targeted geographic area (province/district) with data submitted to next level within time limit (as specified in data flow policy)
Denominator: All facilities (reporting units) expected to submit data within a targeted geographic area (province/district)
This indicator is defined per existing HMIS guidelines, data flow requirements and timing/schedules of the MOH. SBH will use DHIS2 reports on reporting rates and timeliness rates. Unit of Measure: Percentage Disaggregated by: Geographic (Province/District)
82
Justification & Management Utility: Generation and use of information is an increasingly important element of a strong and functioning health system. SBH works with the MOH to improve capacities to generate and use health information at various levels to inform decisions. One element of quality health information is ensuring that data is submitted to the HMIS as expected and on time. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Data extraction of reporting rates from HMIS/ DHIS2 Data Source: HMIS/DHIS2 Method of Data Acquisition by USAID: The M&E Team has access to the HMIS/DHIS 2 and run the report on a monthly basis. The team further verifies the data with the MOH M&E Team. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, relies on existing data Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): TBD Actions Taken or Planned to Address Data Limitations: TBD Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others
PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 65% ne 2016 65% 2017 75% 2018 80% 2019 85% 2020 90% EOP 90% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
83
Task 2: Design, implement and monitor effective interventions to strengthen program management capacities of provincial and district health teams
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 6: Percent of target provinces /districts that demonstrate capacity to sustainably plan, manage and oversee accessible high quality health services SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Target provinces: SBH will work in five provinces, including Southern, Lusaka, Central, Copperbelt and Eastern.
Target districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Capacity to sustainably plan, manage and oversee high quality health services: Several weighted measures will be used to determine the capacity of the PMO/DMO teams to plan, manage and oversee health services effectively. However the final agreement on selection of appropriate indicators will be determined by agreement with USAID/Zambia and GRZ stakeholders.
Numerator: TBD Denominator: Total number of target provinces/districts Unit of Measure: Percent Disaggregated by: Geographic (province/district) Justification & Management Utility: This is a composite index indicator to measure comprehensive provincial and district capacity. Several weighted measures will be included that will collectively demonstrate a sustainable combination of strengthened systems, management and technical capacity, and broad engagement. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: SBH data analysis (data sources include those collected and used for other purposes) Data Source: SBH records, capacity framework tools, TSS/mentoring tools, HMIS/DHIS2 Method of Data Acquisition by USAID: Team requests Performance Assessment Reports from Provincial Health Offices. Frequency and Timing of Data Acquisition by USAID: Baseline, Midline, EOP Estimated Cost of Data Acquisition: Negligible, analysis of data collected for other purposes Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Data limitations will be assessed once the components of the composite index are finalized. Actions Taken or Planned to Address Data Limitations: TBD
84
Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Periodic review Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- TBD ne 2016 20% 2017 40% 2018 60% 2019 80% 2020 100% EOP 100% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
85
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 7: Percent of targeted facilities in target districts that demonstrate capacity to deliver high quality health services and engage with communities in their catchment area. SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
This is a composite index indicator to measure comprehensive facility level capacity.
Target districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Target facilities: This includes the specific facilities within the targeted districts selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Capacity to sustainably deliver quality health services and engage with communities: Several weighted measures will be included to determine the appropriate combination of performance elements for measuring change in performance of the systems. The methodology for the composite index indicator will be finalized with USAID/Zambia and GRZ stakeholders.
Numerator: TBD Denominator: Total number of targeted facilities in targeted districts Unit of Measure: Percent Disaggregated by: Geography (province/district) Justification & Management Utility: This is a composite index indicator to measure comprehensive facility level capacity. Illustrative sub-indicators that may be included are: Evidence that priority policy/standards/procedures have been disseminated and are in use at facility level; minimum facility capacity score for selected performance domains; availability of trained staff; availability of a community action plan; broad engagement of communities; documented improvement in selected service utilization indicator (such as number of institutional deliveries) over a period of time; or others.
86
PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Varied, depending on selection of measures Data Source: SBH records, capacity framework tools, TSS/mentoring tools, monitoring tools, HMIS/DHIS2 Method of Data Acquisition by USAID: TBD Frequency and Timing of Data Acquisition by USAID: Baseline, Midline, EOP Estimated Cost of Data Acquisition: TBD (expected to be minimal) Individual Responsible at USAID: COR, Systems for Better Health (SBH) Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): TBD Actions Taken or Planned to Address Data Limitations: TBD Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others.
The indicator language was updated in May 2016, as follows. The original (AMEP Version 1) indicator SBH 7: Percent of target facilities in target districts that demonstrate capacity to sustainably deliver quality health services and engage with empowered communities to improve access and use of priority services was updated. The current indicator statement reads: “ "Percent of targeted facilities in target districts that demonstrate capacity to deliver high quality health services and engage with communities in their catchment area" PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- TBD ne 2016 0 2017 30% 2018 50% 2019 70% 2020 85% EOP 85% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
87
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 23: Capacity score of targeted provinces and districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Capacity score: SBH will use a capacity assessment tool at multiple levels to routinely measure capacity against pre- determined/pre-defined performance standards (to be determined in collaboration with MOH during Year 1 Q3). The analysis to assist in identification of indictors has been has been collected through the completed capacity assessments. These will be used to generate ‘scores’ by performance domain, as well as a total ‘score.’ Specific performance domains include: Planning Systems and Capacity; Information System Management; Financial Management; Human Resources Management; Supply and Logistics Management; Health Programs Management; Laboratory and Support Capacity; Community Mobilization.
Targeted provinces: Central, Copperbelt, Eastern, Lusaka and Southern
Targeted districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata. Unit of Measure: Absolute score Disaggregated by: Geographic (province/district); performance domain Justification & Management Utility: A fundamental strategy of SBH to achieve and sustain its intended results is to build capacity at all levels. SBH will help the MOH to routinely measure its own capacity against agreed standards within the priority performance domains (see above). SBH will design and conduct the assessments in collaboration with the MOH Directors and program officers. These assessments will consider the structures, processes, tools, and people skills necessary to design and implement new guidance documents. The assessment will enable SBH to develop a strategy for capacity building that focuses on the design and implementation of guidance documents.
Scores will be calculated at least biannually to allow for regular analysis of change/improvement over time. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Capacity assessment tools, reports Data Source: Capacity strengthening tool, SBH records Method of Data Acquisition by USAID: SBH will work in collaboration with MOH in conducting routine capacity assessments. Frequency and Timing of Data Acquisition by USAID: Semi-annually (to be confirmed) Estimated Cost of Data Acquisition: Negligible, part of SBH technical intervention/programming Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES
88
Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Dependent on routine implementation of self-assessment process at various levels Actions Taken or Planned to Address Data Limitations: SBH technical support and capacity building in use of tools; development of standard processes/guidelines in self-assessment, Date of Future Data Quality Assessments: Semi-annually Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Semi-annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others Baseline will be collected during first round of capacity assessments expected in Year 1 Q 4. Targets will be developed and refined once baseline data is analyzed. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baselin -- TBD e 2016 TBD 2017 TBD 2018 TBD 2019 TBD 2020 TBD EOP TBD THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
89
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 24: Percent of targeted provinces/districts that conducted at least two reviews of their annual action plans, in the past year SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Targeted provinces: Central, Copperbelt, Eastern, Lusaka and Southern
Targeted districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Conducted reviews of annual action plans: Evidence that the province/district formally reviewed its annual action plan (documented by record review, meeting minutes)
At least two reviews: Action plan reviews done twice in the last 12 months
Numerator: Number of provinces/districts that conducted at least 2 reviews of annual action plan
Denominator: Total number of targeted provinces/districts Unit of Measure: Percent of provinces/districts Disaggregated by: Geographic (province/district) Justification & Management Utility: SBH provides both technical and financial support review of the annual action plans. The purpose of the review of the action plan is performance tracking with subsequent adjustment the plans. This indicator measures the proportion of provinces/districts that conducted at least two reviews of their annual action plans, in the past year. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Performance Assessment Reports; other documentation Data Source: SBH records, district documentation Method of Data Acquisition by USAID: SBH provincial/district level staff requests relevant documentation from Provincial and District Health Offices Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): None
90
Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Annually Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others N/A PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 40% Province ne 90% District 40% Province 2016 90% District 2017 100% 2018 100% 2019 100% 2020 100% EOP 100% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
91
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 25: Percent of targeted PMOs/DMOs that use HMIS data routinely SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): This indicator measures the level of access to HMIS and DHIS2. The indicator shall be stated as the percentage of personnel given access to the system, as well as percentage of staff able to access the HMIS and utilize data for planning and decision making.
Targeted PMO: Provincial Medical Office within the SBH targeted provinces (Central, Copperbelt, Eastern, Lusaka and Southern)
Targeted DMO: District Medical Offices within the specific districts selected by SBH for priority interventions. This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata
Use HMIS data routinely: Use of HMIS will be measured as PMO or DMO use of DHIS2 data for planning and decision making.
Numerator: Number of PMO/DMO that report use of DHIS2 in the reporting period Denominator: Total number of targeted PMOs/DMOs Unit of Measure: Percent Disaggregated by: Geographic (province) Justification & Management Utility: Availability and use of information for decision making is an important element of health system strengthening in Zambia. Ensuring that stakeholders have access to timely information is critical. SBH supports various interventions to improve the generation and use of high quality data, and will promote routine use of this data by decision makers. This indicator seeks to measure the level of accessibility of data among important stakeholder groups. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: SBH data collection tool for PMO and DMO personnel to collect information on HMIS use Data Source: SBH records; self-reported information from PMO/DMO Method of Data Acquisition by USAID: Data review Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD
92
Known Data Limitations and Significance (if any): Depends on reliability of DHIS2 usage metrics and availability to track usage by user level (province, district). Actions Taken or Planned to Address Data Limitations: SBH technical interventions aim to ensure generation and use of high quality data, and works with appropriate partners/counterparts to ensure that accurate HMIS usage information is generated and available. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: TBD PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others
This indicator was revised from the original wording in AMEP Version 1. The original wording was “SBH 25: Percent of targeted PMOs/DMOs that have access to program and health indicator performance reports (generated through DHIS2)”. It was updated to read “Percent of targeted PMOs / DMOs that use HMIS data routinely” in May 2016 for inclusion in AMEP Version 2. PERFORMANCE INDICATOR VALUES Year Target Actual Notes -- 100% Province, Baseline 40% District 2016 100% Province, 40% District 2017 100% Province, 100% District 2018 100% Province, 100% District 100% Province, 100% District During this year, SBH will be working in 2019 20 districts and the base will change 2020 100% Province, 100% District EOP 100% Province, 100% District THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
93
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 27: Number of health managers and providers trained in management and leadership SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): SBH partner BRITE developed the Management and Leadership Academy (MLA) concept and developed a curriculum in collaboration with the MOH under the Zambia Integrated Systems Strengthening Project (ZISSP). Using the MLA curriculum, SBH will provide in-service training to the MOH managers/supervisors in target provincial, district and hospital/ facility level to increase their capacity to perform their management functions effectively.
Management and Leadership: The Management and Leadership Academy (MLA) is a training program in management and leadership targeting MOH managers/ supervisors at central, provincial, district, and hospital/facility levels. The curriculum consists of six modules, spread over four 2-3 day workshops. To enhance learning, small group mentoring sessions focusing on the same topics covered in workshops are provided after each workshop.
Trained: MLA participants are expected to complete 4 training workshops over the course of 12 months; a person is trained when one meets this criteria. A person is counted as having successfully completed each workshop if he/she attends at least 75% of the duration of each workshop with a specific subject, area, theme or topic. For example, if a person attended two of the four workshops, that person should be counted two times.
Numerator: The number of MOH managers/supervisors that are trained (per defined criteria) through MLA Denominator: N/A Unit of Measure: Number of people Disaggregated by: Geographic (province/district); cadre; sex Justification & Management Utility: The MLA activities include managers/supervisors in targeted provinces, districts and facilities. This indicator tracks the number of managers who complete the MLA courses. Training through the MLA is expected to lead to improved management, leadership, planning of resources and service delivery. This indicator focuses on MLA participants and measures the participants who complete the full each training module. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Training register Data Source: SBH, ZMLA M&E Training register Method of Data Acquisition by USAID: In close collaboration with program staff, SBH M&E team works closely with the program staff under BRITE for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly, Semi-annually, Annually Estimated Cost of Data Acquisition: Negligible, part of project activity tracking Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD
94
Known Data Limitations and Significance (if any): Any training indicator has the fundamental problem of only capturing all the training provided. The quality or the outcome of the training (whether the trainees learned any skills or whether they use the taught skills) are NOT measured with this indicator. There is also some risk that some individuals who did not attend 75% of the workshop may be counted as being trained. Actions Taken or Planned to Address Data Limitations: A standard data collection tool has been designed which is used to capture this data. This is complemented with the daily attendance sheet which is signed by the participants. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts quarterly data quality assessments in alignment with the SBH M&E Plan. Routine DQA approach is ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Quarterly, semi-annually, and annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0 2017 250 2018 0 No management and leadership training planned in Year 3-5 2019 0 2020 0 EOP 250 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
95
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 28.a. Percent of target facilities that did not experience a stock-out of Coartem (ACT) during the reporting period SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Target facilities: This includes the specific facilities within targeted districts selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Stock out: Coartem (ACT) have been out of stock at ANY time during the reporting period.
Reporting period: SBH will measure this indicator on a quarterly basis.
Numerator: Number of target facilities that did not experience Coartem (ACT) stock-out in the reporting period Denominator: Total number of target facilities
HMIS/DHIS2 Code: DRG1-005 DRG1-095 (HIA3) Unit of Measure: Percentage Disaggregated by: Geographic (province/district) Justification & Management Utility: This indicator complements the work of PEPFAR supply chain management system to strengthen the pharmacy professional and build skills to diagnose and solve problems related to stock-out and excesses. This information is used for facility level ordering and reporting. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Data extraction from HMIS/DHIS2, logistics management information system Data Source: HMIS/DHIS2; logistics management information system Method of Data Acquisition by USAID: Data review Frequency and Timing of Data Acquisition by USAID: Quarterly
96
Estimated Cost of Data Acquisition: Negligible, relies on existing data Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Delays in data availability may present challenges. Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team and Supply Chain Specialist conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes Baselines/Targets and Others
PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 93.3% ne 2016 93.3% 2017 100% 2018 100% 2019 100% 2020 100% EOP 100% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
97
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 28.b: Percent of target facilities that did not experience a stock-out of contraceptive (injectable)during the reporting period SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.1: Strengthened program management capacity of provincial and district health teams Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Target facilities: This includes the specific facilities within targeted districts selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Stock out: Contraceptive (injectable) has been out of stock at ANY time during the reporting period.
Reporting period: SBH will measure this indicator on a quarterly basis.
Numerator: Number of target facilities that did not experience a stock-out of contraceptive (injectable) in the reporting period Denominator: Total number of target facilities
HMIS/DHIS2 Code: DRG1-095 (HIA3) Unit of Measure: Percentage Disaggregated by: Geographic (province/district) Justification & Management Utility: This indicator will complement the work of PEPFAR supply chain management system to strengthen the pharmacy professional and build skills to diagnose and solve problems related to stock-out and excesses. This information will be used for facility level ordering and reporting. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Data extraction from HMIS/DHIS2, logistics management information system Data Source: HMIS/DHIS2; logistics management information system Method of Data Acquisition by USAID: Data review
98
Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, relies on existing data Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Delays in data availability may present challenges. Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team will conduct routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others
PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 73% ne 2016 73% 2017 80% 2018 80% 2019 90% 2020 100% EOP 100% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
99
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 30: Number and percent of children who received DPT3 vaccine by 12 months of age in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) __2016 – 20 Indicator # 3.1.6-61 DESCRIPTION Precise Definition(s): DPT refers to a class of combination vaccines against three infectious diseases in humans: diphtheria, pertussis (whooping cough) and tetanus. The vaccine components include diphtheria and tetanus toxoids, and killed whole cells of the organism that causes pertussis. To be fully protected, children must receive three doses of the vaccine, administered at the ages of one month, one month and a half, and three months.
Children who received DPT3 vaccine by 12 months of age: Number of children aged 12 to 23 months of age who received the third dose of DPT (Diphtheria, Pertussis, Tetanus) vaccine by 12 months of age
Targeted districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: Number of children 12-23 month of age who received third dose of DPT (Diphtheria, Pertussis, Tetanus)-containing vaccine by 12 months of age within targeted districts. Denominator: Number of living children age 12-23 months in targeted districts
HMIS Code: HMIS CHN3-045 Unit of Measure: Number of children who receive DPT3 by 12 months of age, and Percent of children who received DPT3 vaccine by 12 months of age Disaggregated by: Rural/urban; Geography (province/district) Justification & Management Utility: This indicator is used to report the percentage of children receiving the final dose (DPT3), which is an important gauge of how well countries are providing childhood immunization coverage. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS /DHIS2 Data Source: HMIS/DHIS2 (HMIS CHN3-045) Method of Data Acquisition by USAID: TBD Frequency and Timing of Data Acquisition by USAID: Semi-annually Estimated Cost of Data Acquisition: Negligible, relies on existing data source Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD
100
Known Data Limitations and Significance (if any): Reliant on routine statistics and timing of reporting this indicator. This data is collected through the HMIS/DHIS 2. This indicator requires the same child to be tracked over a 12-month period, which is difficult to do in many of the settings in which USAID works. A child might actually get three doses, but not all at the same site. There is also a data limitation challenge given that the denominator relies on Central Statistics Office data, which can differ from facility/catchment area population counts, leading to coverage rates that can exceed 100%. Actions Taken or Planned to Address Data Limitations: This data is collected through the HMIS/DHIS 2 and the data is made available 3 months after every quarter. This data is reported three months after every quarter. Date of Future Data Quality Assessments: Semi-annually Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 94% ne 2016 >90 2017 >90 2018 >90 2019 >90 2020 >90 EOP >90 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
101
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 31: Percent of HIV-positive pregnant women who received antiretroviral treatment to reduce risk of mother-to-child-transmission in targeted districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 PMTCT_ARV_NAT, 3.1.1-86 DESCRIPTION Precise Definition(s): a) This indicator allows countries to monitor: 1) the coverage of antiretroviral given to HIV-positive pregnant women. It is recognized that due to the way in which data is collected and reported in many countries, some level of duplication may be inevitable. Additionally, there may be over or undercounting of certain regimens based on data collection methodologies. 1. PEPFAR is directly interacting with the patient or beneficiary in response to their health (physical, psychological) care needs by providing key staff and/or essential commodities for routine service delivery. For women receiving PMTCT services, this can include: ongoing procurement of critical commodities, such as test kits, ARVs, or lab commodities, or funding for salaries of HCW. Staff who are responsible for the completeness and quality of routine patient records (paper or electronic) can be counted here; staff who exclusively fulfill MOH and donor reporting requirements cannot be counted. b) AND/OR 2. PEPFAR provides an established presence at and/or routinized, frequent (at least quarterly) support to those services at the point of service delivery. For PMTCT services, this ongoing support for service delivery improvement can include: training of PMTCT service providers, clinical mentoring and supportive supervision of PMTCT service sites, infrastructure/renovation of facilities, support of PMTCT service data collection, reporting, data quality, QI/QA of PMTCT services support, ARV consumption forecasting and supply management, support of lab clinical monitoring of patients, supporting patient follow-up/retention, support of mother mentoring programs.
Targeted districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Numerator: The number of HIV-positive pregnant women who received antiretrovirals for prophylaxis or treatment during pregnancy or during labor and delivery (L&D) (HMIS HIV2-135) Denominator: Number of HIV-positive pregnant women identified in the reporting period (HMIS HIV2-065)
Details per PEPFAR Guidance: Numerator: The number of HIV-positive pregnant women who received antiretrovirals for prophylaxis or treatment during pregnancy or during labor and delivery (L&D)
The numerator can be summed from categories a through d below: a) Number of pregnant women with unknown HIV status attending ANC who received an HIV test and result during the current pregnancy b) Pregnant women with known HIV infection attending ANC for a new pregnancy c) Number of pregnant women with unknown HIV status attending L&D who received an HIV test and result during their current pregnancy
102
d) Women with unknown HIV status attending postpartum services within 72 hours of delivery who were tested for the first time in the current pregnancy and received results.
Denominator: Number of HIV-positive pregnant women identified in the reporting period (including known HIV- positive at entry) Unit of Measure: Number of HIV positive pregnant women Disaggregated by: By regimen type, as appropriate to local MOH policy); Geographic (district) c) Justification & Management Utility: This indicator measures the provision and coverage of antiretroviral prophylaxis and treatment, by regimen type, for HIV-positive pregnant women in order to: 1. Identify progress toward the global goals of increasing ARV coverage (prophylaxis and treatment) among pregnant women living with HIV 2. Assess progress toward implementing more efficacious PMTCT ARV regimens 3. Determine the coverage of HIV+ pregnant women on ARV prophylaxis and ART for life among all HIV+ pregnant women identified 4. Provide data for models estimating the country-specific and global impact of PMTCT programs. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS 2 Data Source: HMIS/DHIS2/SmartCare Method of Data Acquisition by USAID: Routine reporting Frequency and Timing of Data Acquisition by USAID: Data is collected continuously at the facility level. Data should be aggregated in time for PEPFAR reporting cycles. In addition, USG country teams may request periodic aggregation, i.e., quarterly. Estimated Cost of Data Acquisition: Negligible, based on existing routine data Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): This indicator enables the USG PEPFAR team to monitor trends in HIV testing among pregnant women and uptake of testing at USG-funded sites. The points at which drop-outs occur during the testing and counseling process and the reasons why they occur are not captured by this indicator. The HMIS/DHIS 2 does not capture all the age desegregation required for this indicator.
There is a risk of double counting with this indicator, as a pregnant woman could be tested multiple times during ANC or, L&D, and postpartum. This is particularly true when pregnant women get re-tested according to some national guidelines or when they seek testing in different facilities, or when they come to the L&D without documentation of their test. Actions Taken or Planned to Address Data Limitations: While not feasible to avoid double counting entirely, a standard data collection and reporting system has been put in place to minimize it, such as using patient held and facility held ANC records to document that testing took place and only counting and reporting the last test with a definitive result, or the previously known HIV-infected status. This tool will be designed to ensure that all the disaggregation is captured. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator
103
Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications
PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- TBD ne 2016 90% 2017 90% 2018 90% 2019 90% 2020 90% EOP 90% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
104
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2.1 Service delivery improved Name of Indicator: SBH 32: Percentage of targeted DMOs that have completed their semi-annual performance assessment and technical supportive supervision visits to facilities in their catchment area SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Targeted DMO: District Medical Office within the specific districts selected by SBH for priority interventions. This will includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata
Completed semi-annual performance assessment: The DMOs have assessed the facilities using the performance assessment tool that measures facility performance.
Technical supportive supervision visits: DMOs will visit the facilities at least once to review the performance of the facilities.
Facilities in catchment area: This includes the specific facilities selected by SBH for priority interventions (to be determined in Year 1 Quarter 2).
Numerator: Total number of targeted DMOs that complete semi-annual performance assessment and TSS per above criteria to facilities Denominator: Total number of targeted DMOs Unit of Measure: Percentage of DMOs Disaggregated by: Geographic (Province/District) Justification & Management Utility: SBH supports the MOH to improve the routine use of routine performance assessment processes, as a mechanism to reinforce accountability, and ultimately help lead to improved performance. This indicator provide an important gauge on the degree of institutionalization of important MOH performance improvement approaches. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: PA reports and TSS records Data Source: PA Monitoring Tools, SBH records Method of Data Acquisition by USAID: Team requests Performance Assessment Reports from Provincial Health Offices Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES
105
Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): N/A Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 40% Semi-annual appraisal ne 10% TSS 2016 40% PA, 10% TSS 2017 60% PA, 60% TSS 2018 60% PA, 60% TSS 2019 70% PA, 70% TSS 2020 80% PA, 80% TSS EOP 80% PA, 80% TSS THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
106
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 33: Percent of targeted DMOs that demonstrate specific performance improvements since their previous performance assessment SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Targeted DMO: District Medical Office within the specific districts selected by SBH for priority interventions. This will includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata
Demonstrate specific performance improvement: SBH monitor performance in the following program indicators: . Proportion of targeted DMOs that showed improvement in relation to previous performance assessment . Proportion of targeted DMOs that implemented the action plan developed during the previous performance review . Proportion of targeted DMOs that held semi-annual performance review meetings with their health facilities . Proportion of target districts that invite partners to their planning and semi-annual review meetings.
Since their previous performance assessment: Performance assessment is conducted twice yearly.
Numerator: Number of targeted DMOs that demonstrate performance improvements since their previous performance assessment Denominator: Total number of targeted DMOs Unit of Measure: Percentage of DMOs Disaggregated by: Geographic (province) Justification & Management Utility: SBH will provide technical, financial, logistical and administrative assistance to the MOH at the national, provincial, and district levels, and to community-based organizations to conduct effective performance assessment (PA) and technical support supervision (TSS) visits to health providers in order to address problems that hinder the delivery of high quality HIV prevention, care and treatment services at the facility level. This indicator serves to demonstrate the extent to which PA is being actively used to identify and solve problems. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS 2, PA results analysis Data Source: HMIS/DHIS2, PA Monitoring Tools, SBH records Method of Data Acquisition by USAID: Team requests Performance Assessment Reports Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka
107
DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Actions Taken or Planned to Address Data Limitations: Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team will conduct routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 90% ne 2016 90% 2017 100% 2018 100% 2019 100% 2020 100% EOP 100% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
108
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 34: Percent of targeted primary care facilities that have received technical supportive supervision or clinical mentoring SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Targeted primary care facilities: This includes the specific primary care facilities selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Received technical support supervision or clinical mentorship: Documented completion of TSS or clinical mentoring visit
Clinical staff include the following health workers: d) Doctors, nurses, midwives, laboratory scientists, pharmacists, EHTs, social workers, medical equipment technologists, lab and pharmacy technicians and psychologists.
Numerator: Number of targeted primary care facilities that received TSS or clinical mentoring for clinical staff at least once in previous sixth-month period Denominator: Total number of targeted primary care facilities Unit of Measure: Percentage of facilities Disaggregated by: Geographic (province/district) Justification & Management Utility: SBH provides financial and technical support to target districts and health facilities to improve service delivery and quality of care. This indicator measures proportion of targeted primary care facilities that have received appropriate supports during the reporting period in alignment with MOH-supported TSS and clinical mentoring approaches. PLAN FOR DATA ACQUISITION BY USAID
109
Data Collection Method: TSS and Mentorship Forms Data Source: TSS Monitoring Tools, Mentorship forms Method of Data Acquisition by USAID: Team requests Performance Assessment Reports and Mentorship Forms from Provincial Health Offices Frequency and Timing of Data Acquisition by USAID: Semi-Annual Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): None Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Semi-annually Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications Baseline and targets will be defined as proportion of districts that have been targeted for data quality assessment in Y1Q2. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli 57.5% - TSS ne 41% - clinical mentoring 2016 60% - TSS, 41% - Clinical Mentoring 2017 90% TSS, 85% Clinical Mentoring 2018 90% TSS, 90% Clinical Mentoring 2019 90% TSS, 100% Clinical Mentoring 2020 90% TSS, 100% Clinical Mentoring EOP 90% TSS, 100% Clinical Mentoring THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
110
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 35: Percent of targeted facilities in targeted districts that have up-to-date and gender sensitive job aids for HIV, FP, MNCH, and/or nutrition SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Targeted facilities: This includes the specific facilities selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Targeted Districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Up-to-date: This will mean that the job aid are current and in line with the standard guidelines/policies.
Gender sensitive: This means gender inclusiveness and equity into job aids and other materials.
Job aids for HIV, FP, MNCH and/or nutrition: These are tools which provide guidance on service delivery. (to count for this indicator, the facility job aids must be both up-to-date and gender sensitive).
Numerator: Percent of targeted facilities that have up-to-date and gender sensitive job aids Denominator: Percent of targeted facilities in targeted districts Unit of Measure: Percentage of facilities Disaggregated by: Topic Justification & Management Utility: SBH will work with MOH to ensure that job aids are current, and also include appropriate guidance on gender and equity. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Document review and analysis of TSS and mentoring documentation
111
Data Source: SBH records, mentor/TSS records Method of Data Acquisition by USAID: Document review Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): None Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Annually Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team will conduct routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others This indicator measures SBH interventions/outputs, so baseline is N/A. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0% ne 2016 0% 2017 60% 2018 80% 2019 100% 2020 100% EOP 100% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
112
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 36: Percent of targeted facilities in targeted districts that have initiated QI projects in ART, PMTCT, MC, FP, child health and nutrition, or maternal health services with documented process results. SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Targeted Facilities: This includes the specific facilities selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Targeted Districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata.
Initiated QI project: A facility has an established QI committee that is implementing at least one QI project to improve ART, PMTCT, FP, maternal health, child health, and nutrition services.
Documented process results: Demonstrated through existence of QI meeting minutes, QI project reports or other formal documentation that explains progress of the QI project(s) over time
Numerator: Number of targeted facilities in targeted districts that imitated QI projects and have documented process results in the past six months Denominator: Total number of targeted facilities in targeted districts Unit of Measure: Percentage Disaggregated by: Topic, district, facility
113
Justification & Management Utility: SBH provides financial and technical support for QI committees and capacity building in QI methods and performance improvement approaches. This is important to improve the quality of care and service delivery at facility level. This indicator will measure the proportion of targeted facilities in targeted districts that have initiated QI projects in ART, PMTCT, MC, FP, child health and nutrition, or maternal health services and are documenting the results of the QI project(s). PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Document review of QI reports, meeting minutes, other documentation Data Source: SBH records, Meeting minutes, QI Reports Method of Data Acquisition by USAID: Team requests Performance Assessment Reports from Provincial Health Offices Frequency and Timing of Data Acquisition by USAID: Annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Limitations include the availability of required documents (e.g. meeting minutes, reports) that SBH can use to verify the criteria of functionality. Actions Taken or Planned to Address Data Limitations: A standard data collection tool is used to document functionality Date of Future Data Quality Assessments: Annually Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others This indicator was refined. In AMEP Version 1, the original wording of SBH 36 was “Percent of targeted facilities in targeted districts that have initiated QI projects in ART, PMTCT, MC, FP, child health and nutrition, or maternal health services with documented process results in past 6 months”.
This indicator was refined to now read "Percent of targeted facilities in targeted districts that have initiated QI projects in ART, PMTCT, MC, FP, child health and nutrition, or maternal health services with documented process results" in May 2016.
PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0% ne 2016 0% 2017 50%
114
2018 50% 2019 70% 2020 85% EOP 85% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
115
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 37: Number of people trained in child health and nutrition SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 Indicator #3.1.9-1 DESCRIPTION Precise Definition(s): Number of people (health professionals, primary health care workers, trained in child health care and child nutrition through United States Government (USG)-supported programs during the reporting year. This indicator will also count those people who receive mentorship in child health and nutrition.
People: Health professionals, primary health care workers,, non-health personnel (agriculture sector)
Trained: Means the participant has attended 75% of the training
Mentorship: Mentorship will be provided on site/at facility to those people who have been either trained by SBH or any other implementing partners.
Child health and nutrition: Training topics may include, ((3) Vitamin A supplementation and de-worming, (4) improve Infant and Young Child Feeding (IYCF), and Integrated Management of Severe Acute Malnutrition (IMAM)
Numerator: Number of people trained in child health and/or nutrition through SBH support Denominator: N/A Unit of Measure: Number of people Disaggregated by: Sex; cadre; geographic (province/district), training type Justification & Management Utility: SBH goal is to contribute to reducing childhood morbidity and mortality by strengthening the capacity of the MOH to coordinate, plan, implement, monitor, national child health and nutrition program interventions. This indicator tracks the number of people trained and mentored in child health and nutrition. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Activity Training Reports Data Source: SBH Training Registers Method of Data Acquisition by USAID: In close collaboration with program staff, SBH M&E team work together to collect the data for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES
116
Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): If the same person received multiple trainings, s/he is counted several times, which is acceptable for this indicator because it counts all instances of people being trained. Any training indicator has the fundamental problem of only capturing the training provided. The quality of neither the training nor the outcome of the training (whether the trainees learned any skills or whether they will use the taught skills) is measured with these indicators. There is also some risk that some individuals who did not attend 75% of the workshop will be counted as being trained. Actions Taken or Planned to Address Data Limitations: A standard data collection tool has been designed which will be used to capture this data. This is complemented with the daily attendance sheet which is signed by the participants. A standard MOH mentorship forms are used to track those mentored Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA are done ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications This indicator serves as a measure of SBH interventions/outputs, therefore baseline is 0. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 120 2017 1,500 2018 1,500 2019 1,500 2020 100 EOP 4,720 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
117
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 38: Number of people trained in maternal and newborn health SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): IIP indicator tracks the number of health care providers (professionals, primary health care workers, community health workers, volunteers) trained in maternal and/or newborn health. It includes both those providers trained in EmONC and those trained as Safe Motherhood Action Groups (SMAGs) trainers and SMAG members.
For EmONC, training occurs in two sessions—theory and practical. For a participant to be considered to have completed the training, he/she must participate at least 75% of the time in each training session.
For the SMAG community volunteers, the training is done in one session; similarly to the count as having been trained, participants must complete 75% of the session.
People: Includes health care workers and community members. SBH works with Safe Motherhood Project 360+. SBH is printing the training material while the Safe Motherhood Project 360+ is doing the training. SBH will track the community members (SMAGs) through Safe Motherhood Project 360+.
Trained: Means the participant has attended 75% of the duration of the training
Numerator: Number of people trained in maternal and newborn health Denominator: N/A Unit of Measure: Number of people Disaggregated by: Province, District, Facility, Sex and type of cadre of (health worker, community health worker) Justification & Management Utility: SBH train health care providers and prints the training materials which are used to community volunteers while Safe Motherhood Project 360+ is doing the training. Maternal/newborn health is an important health priority for the GRZ, USAID and other partners. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: SBH Training Registers Data Source: SBH Training Registers Method of Data Acquisition by USAID: In close collaboration with program staff, SBH M&E team works together to Community Health collect the data for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, routine project data collection/activity tracking Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES
118
Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): The EmONC training has been designed in two phases comprised of theory and practical training. The training has been designed in such a way that the theory training is done at different location and at different times in that the training is not done sequentially. Therefore, not all the people who attended the theory attend the practical training. The challenge for tracking this indicator is that a participant has to attend at least 75% of the time in each of the two trainings, i.e., theory and practical to be counted as being trained.
The other problem is that, if the same person received multiple trainings, s/he is counted several times, which is acceptable for this indicator because it counts all instances of people being trained. Any training indicator has the fundamental problem of only capturing the training provided. The quality of neither the training nor the outcome of the training (whether the trainees learned any skills or whether they use the taught skills) are measured with these indicators. Actions Taken or Planned to Address Data Limitations: For EmONC training participants who attend theory training are given an opportunity to attend the practical training at any other time when they are available. This makes this indicator easy to track because the program staff make all the necessary arranged through an established system that those who attend the theory sessions are also given an opportunity to attend the practical trainings.
A standard data collection tool has been designed which is used to capture this data. This is complemented with the daily attendance sheet which is signed by the participants to ensure that they attended 75% of the training. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications This indicator provides a measure of SBH intervention/outputs, therefore baseline is 0. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseline -- 0 2016 60 2017 450 2018 500 2019 570 2020 0 No training in MNCH planned in Year 5 EOP 1,580 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
119
120
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 39: Number of people trained in family planning/reproductive health SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s): Number of people (health professionals, primary health care workers, community health workers, volunteers, non- health personnel) trained in family planning and reproductive health through United States Government (USG)- supported programs during the reporting year
People: Health professionals, primary health care workers, community health workers, volunteers, non-health personnel
Trained: Means the participant has attended 75% of the training
Numerator: Number of people
Denominator: N/A
Unit of Measure: Number of people Disaggregated by: Province, District, Facility, Sex and type of cadre of (health worker, community health worker) Justification & Management Utility: SBH seeks to build capacity of MOH, province, district, training institutions personnel, and community health volunteers in FP/RH counseling and service delivery. One strategy is to increase the use of FP/RH is to increase the number of sites that offer FP counseling and/or services. An increased contraceptive prevalence rate reduce the unmet need for FP, the number of unintended pregnancies, the number of abortions, and neonatal, infant, child and maternal mortality and morbidity. This indicator tracks the number of people trained in FP/RH counseling, service delivery. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Training Registers Data Source: SBH Training Registers Method of Data Acquisition by USAID: In close collaboration with program staff, SBH M&E team work together to collect the data for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD
121
Known Data Limitations and Significance (if any): The challenge for tracking this indicator is that a participant has to attend at least 75% of the time. The other problem is that if the same person received multiple trainings, s/he is counted several times, which is acceptable for this indicator because it counts all instances of people being trained. Any training indicator has the fundamental problem of only capturing the training provided. The quality of neither the training nor the outcome of the training (whether the trainees learned any skills or whether they use the taught skills) are measured with these indicators. Actions Taken or Planned to Address Data Limitations: A standard data collection tool will be designed which will be used to capture this data. This will be complemented with the daily attendance sheet which is signed by the participants to ensure that they attended 75% of the training. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above. Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others This indicator serves as a measure of SBH interventions/outputs, therefore baseline is 0. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 40 2017 100 2018 150 2019 150 2020 0 No training in FP/RH planned in Year 5 EOP 440 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
122
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 40: Number of new health care workers who successfully completed an in-service training program within the reporting period SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 2: Strengthened program management capacities of provincial and district health teams SBH Sub-Result 2.2: Improved technical capacity of provincial and district health teams to deliver quality health services in facilities Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 Indicator #3.1.1-84 DESCRIPTION Precise Definition(s): In–service training: In-service training can occur through structured learning and follow-up activities, or through less structured means, to solve problems or fill identified performance gaps. In-service training can consist of short non- degree technical courses in academic or in other settings, non-academic seminars, workshops, on-the-job learning experiences, observational study tours, or distance learning exercises or interventions.
An in-service training program must meet national or international standards and have specific learning objectives, a course curriculum, expected knowledge, skills, and competencies to be gained by participants, as well as documented minimum requirements for course completion. The duration and intensity of training will vary by cadre; however, all training programs should have at a minimum the criteria listed above. This indicator is distinct and separate from the indicator for pre-service training and education a health care worker may be counted under both indicators ONLY if that worker has completed pre-service training and education distinct and separate from their in-service training in the same reporting period.
SBH conducts in-service training through Clinical Care mentorship/Technical Support Supervision. Specific program areas of focus include PMTCT, Pediatric ART, Adult ART, Counseling and Testing, Male Circumcision, Laboratory, Blood Safety, Clinical Care and Quality Improvement and other services. HSS topics may include training in Planning, Performance Management Package, Financial Management, Budgeting, WISN, CHA program, Gender, Human Resource Information, Strategic Information, and data audit and M&E training.
Health Care Workers: Refers to individuals involved in safeguarding and contributing to the prevention, promotion and protection of the health of the population (both professional and auxiliary-professionals). The categories below describe the different types of health workers to be considered under this indicator. This is not an exhaustive list of all health workers and position titles may vary from country to country.
For the purposes of this indicator, health workers include the following: Clinical staff include the following health workers: a) Doctors, nurses, midwives, laboratory scientists, pharmacists, EHTs, social workers, medical equipment technologists, lab and pharmacy technicians and psychologists. b) Non-clinical workers do not play clinical roles in a health care setting but rather include workers in a health ministry, hospital and facility administrators, managers, planners, monitoring and evaluation advisors, epidemiologists, staff from community radio stations, community groups (SMAGs, NHCs), drama groups, health promotion workers, religious and traditional leaders and other professional staff critical to health service delivery and program support.
Successfully completed: A person is counted as having been trained if he/ she participates in a workshop, course, or session, sponsored by SBH (in whole or in part), with a specific training subject, area, theme or topic.
If a person is trained in more than one topic, s/he is counted for each instance/topic. Individuals being mentored in a mentorship program/session that covers more than one program area should be counted separately in each of the
123
respective areas. For example, a person who completes mentorship on PMTCT and on Pediatric ART in the same workshop/training/side by side mentorship will be counted twice. Additional notes from USAID guidance for this specific indicator: Types of In-service Training: 1. Continuing education: Education/training offered to current providers to either update or add new knowledge and skills. While in-service training is often limited to practitioners in the public sector and/or managed by the Ministry of Health (or similar entity), continuing education is often used to describe education/training that is provided by other sources, such as professional associations, that reaches private sector practitioners and which can be linked to re-licensure and/or certification. 2. On-the-job training: Instruction in a specific task or skill is provided via mentoring by a practitioner using explanations, demonstration, practice and feedback. On-the-job training may be combined with academic or technical training to provide a practical experience component. 3. Computer based training: An interactive learning experience in which the computer provides most of the stimuli, the learner responds, and the computer analyzes the responses and provides feedback to the learner.
Training: Training is a learning activity taking place in-country, a third country, or in the U.S. in a setting predominantly intended for teaching or facilitating the development of certain knowledge, skills or attitudes of the participants with formally designated instructors or lead persons, learning objectives, and outcomes, conducted full-time or intermittently. Training refers to training or retraining of individuals and must follow a curriculum with stated (documented) objectives and/or expected competencies. Training may include traditional, class-room type approaches to training as well as on the job or “hands-on” training such as clinical mentoring or structured supervision so long as the following three criteria are met: a. Training objectives are clearly defined and documented b. Participation in training is documented (e.g. through sign-in sheets or some other type of auditable training c. The program clearly defines what it means to complete training (e.g., attend at least four days of a five-day workshop, achieve stated key competencies, score a defined minimum score on post-test exam). For example, that person should be counted four times. If a person repeats the same training course, he/she should not be counted twice. Please count the staff/volunteers of your organization who were trained, as well as any additional individuals (e.g., from a different organization) that you may have trained in a USG-supported training course that your organization implemented. Only participants who complete the full training course should be counted. An individual should only be counted once they have completed the training. Individuals that are mid-way through a training course should be counted in the next reporting period. Individuals attending more than one training in a particular program area during a reporting period should only be counted once. Individuals participating in training that covers more than one program area may be counted in each of the respective areas.
Number: The number is the sum of health care workers who successfully completed an in-service training program within the reporting period with full or partial PEPFAR support. Individuals will not count as having successfully completed their training unless they meet the minimum requirements as defined by international or national standards. In the absence of international or national standards, the minimum requirement will be determined by the PEPFAR country team. Any individual involved in safeguarding and contributing to the prevention, promotion, and protection of the health of the population may be counted in this in-service training indicator. Refer to the pre-service training indicators #H2.1.D and #H2.2.D for illustrative, but not exhaustive, examples of the types of workers one might include. This in-service training indicator includes health workers as illustrated in indicator #H2.1.D and community health and para-social workers as illustrated in #H2.2.D. There are no specific exclusions to this in-service training indicator #H2.3.D.
Components most often consist of drill-and practice, tutorial, or simulation activities offered alone or as supplements to learner where learners work on their own. It uses a range of mechanisms such as self-guided lesson plans, mailings, radio, and computer based activities. Usually it is tied to an educational facility and uses sequential instructional material that is corrected by the instructor. Regardless of methodologies chosen, it requires motivation on the part of the learner and regular feedback on the part of the learning institution. It can also be used for pre-service education.
Numerator: The number of health care workers who successfully completed an in-service training program and number of MOH managers/supervisors that complete each of four management and leadership academy sessions.
124
Denominator: N/A Unit of Measure: Number of persons Disaggregated by: Type of training/mentorship; geographic (province/district); sex; program area Justification & Management Utility: Building the capacity of individuals is an important strategy of SBH. This indicator tracks the number of people that have received in-service training in order to build their capacity to support the delivery of high quality health services. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Activity Training Reports Data Source: TSS and Mentoring Tools, SBH Training Registers Method of Data Acquisition by USAID: Team requests Performance Assessment Reports from Provincial Health Offices Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): This indicator does not measure the quality of the training, nor does it measure the outcomes of the training in terms of the competencies of individuals trained, nor their job performance. This indicator does not measure the placement or retention in the health workforce of trained individuals. Although training is an essential component of human resources for health, programs should plan it in the context of effective human resources management and an overall HRH strategy. Actions Taken or Planned to Address Data Limitations: SBH measure individual capacity strengthening through complementary methods, including capacity building self-assessments, post-training surveys, and other approaches. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others This indicator serves as a measure of SBH interventions/outputs, therefore baseline is 0. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 150 2017 500 2018 500
125
2019 1,000 2020 500 EOP 2,650 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
126
Task 3: Provide technical and financial assistance to the GRZ and community-based organizations to increase the quality, availability and use of priority health services at the community level in targeted districts
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 43: Number of CHAs that receive routine supervision (from health facility staff) in target facility catchment areas SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 3: Increased quality, availability and use of priority health services at the community level in targeted districts SBH Sub-Result 3.1: Improved capacity to deliver quality health services at the community level Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
CHA: Community Health Assistants
Receive routine supervision: This indicator counts the number of CHAs who received mentorship, PA or technical support supervision from health facility staff during the reporting period.
Health facility staff: Health facility staff include CHA supervisors, staff involved in PA and TSS from different levels of the health system which will include facility, hospital, district and provincial.
Target facility: This includes the specific facilities selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola). Unit of Measure: Number of CHA Disaggregated by: Geographic (province/district, sex, facility) Justification & Management Utility: System level challenges, including staff shortages, reduced monitoring of the quality of services provided by CHAs. CHAs are equipped with the tools and incentives to promote preventive care, positive health seeking behaviors and selected curative services. Supervision of CHAs are strengthened the institutionalization of the CHA strategy and build capacity amongst the CHAs in service delivery. This indicator measures the number of CHAs that receive routine supervision (from health facility staff) in target facility. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Mentorship forms, PA, TSS report or trip report
127
Data Source: Mentorship forms, PA, TSS report or trip report, grantee records Method of Data Acquisition by USAID: In close collaboration with program staff, SBH M&E team works with together Community Health Specialist to collect the data using the mentorship forms/PA, TSS report for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, part of routine reporting by program staff Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): At times, PA/TSS is not conducted routinely as planned because of financial limitations. Some of the CHA work at health posts and do not have immediate supervisors meaning they will not receive any of mentorship, PA or TSS. Actions Taken or Planned to Address Data Limitations: An integrated supervision system has been created to increase health facility coordination and supervision of CHAs. Deliberate efforts are made using existing data to target facilities which do not receive supervision. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team will conduct routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0 2017 70 2018 150 2019 150 2020 50 EOP 420 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
128
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 44: Number of community volunteers (including SMAGs, others) that have received equipment to deliver priority community health services in targeted facility catchment areas SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 3: Increased quality, availability and use of priority health services at the community level in targeted districts SBH Sub-Result 3.1: Improved capacity to deliver quality health services at the community level Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Community volunteers: Includes community health volunteers, members of SMAGs, nutrition groups, others
Received equipment: SBH will procure equipment in target districts. Equipment may include MUAC tapes and family planning kits, bicycles, bags, and other equipment/supplies.
Numerator: Number of Community volunteers that receive equipment Denominator: N/A Unit of Measure: Number of community volunteers Disaggregated by: Geographic (province/district); sex Justification & Management Utility: SBH will procure and supply necessary equipment to enable community health volunteers provide priority services. The project collaborate with MOH and Zambia Medical Stores Limited (MSL) to assure procurement plans include the necessary medical equipment kits for provision community health services in targeted facility catchment areas. Community volunteers are trained and equipped by the grant recipients. SBH will require grantees to report on this indicator. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: SBH Delivery and Goods Receive Notes Data Source: SBH Delivery and Goods Receive Notes Method of Data Acquisition by USAID: In close collaboration with program staff-administration department, SBH M&E team will work together to collect the data using the delivery and goods receive notes for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches will be done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly Estimated Cost of Data Acquisition: Negligible, part of routine reporting by program staff Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): N/A Actions Taken or Planned to Address Data Limitations: N/A Date of Future Data Quality Assessments: Quarterly
129
Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, SBH staff, SBH M&E team will conduct quarterly data quality assessments in alignment with the SBH M&E Plan. Routine DQA approaches will ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0 No relevant work expected in Year 1 2017 912 2018 912 2019 0 2020 0 EOP 1,824 THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
130
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 46: Number of active beneficiaries receiving support from PEPFAR OVC programs to access HIV services SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 3: Increased quality, availability and use of priority health services at the community level in targeted districts SBH Sub-Result 3.2: Strengthened linkages between the community and facility for key health interventions Is this a Performance Plan and Report indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
PEPFAR OVC programs serve children who may already be infected by HIV or are at high risk of becoming HIV infected, either through mother-to-child transmission or sexual transmission. It is important for OVC programs to link potentially HIV-exposed infants and/or their caregivers to PMTCT programs and to connect children of all ages, particularly adolescents, to HIV testing and counseling. Furthermore, it is critical for OVC programs to connect HIV- infected children with treatment, counseling and support services.
This is a direct output indicator that measures the percent of children and/or caregivers who are referred to and enabled to access clinical HIV-related services. The data obtained from this indicator can inform cross-program planning including PMTCT, pediatric and adult HIV testing, and treatment programs.
This indicator will enable PEPFAR headquarters to: 1. Gain a basic, but essential, understanding of the support OVC programs provide to their beneficiaries to enable them to access HIV-related services. 2. Provide important information to stakeholders about the OVC contribution to the clinical goals of the PEPFAR continuum of response. 3. Triangulate data with other OVC output indicators to track levels of OVC services provided across all PEPFAR countries over time.
At the country level, this indicator will enable PEPFAR country teams, governments, implementing partners, and other in-country counterparts to: • Assess the extent to which OVC support services link to clinical HIV services. o Identify programmatic gaps by analyzing the number and ages of people receiving support to access clinical services o Estimate the reach of services in particular geographic areas. • Advocate for greater resources and technical assistance to enable OVC programs to contribute to the continuum of response.
Targeted Areas: This measure is specifically related to a planned SBH pilot program in area (Kitwe).
Numerator: Number of active beneficiaries receiving support from PEPFAR OVC programs to access HIV services Denominator: Number of active beneficiaries served by PEPFAR OVC programs for children and families affected by HIV/AIDS (OVC_SERV) Unit of Measure: Number of children and adolescents Disaggregated by: Age, sex, geographic (province/district) Justification & Management Utility:
131
PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Pending design of SBH Pilot Design and M&E plan Data Source: SBH Pilot M&E Plan Method of Data Acquisition by USAID: In close collaboration with Ministry of Ministry, SBH M&E team will work together with the program staff from its partner Save the Children using a standard facility assessment tool for this indicator, in alignment with the SBH M&E Plan. Routine DQA approaches will be done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: TBD Estimated Cost of Data Acquisition: TBD Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): TBD Actions Taken or Planned to Address Data Limitations: TBD Date of Future Data Quality Assessments: TBD Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team will conduct routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: TBD Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others This original indicator (AMEP Version 1), “SBH 46: Number of HIV positive children and adolescents with active case management plans that include social services within the community in targeted areas” was replaced with “Number of active beneficiaries served by PEPFAR OVC programs for children and families affected by HIV/AIDS in target areas” in May 2016. PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0 2017 1,500 2018 TBD 2019 TBD 2020 TBD EOP TBD THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
132
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 48: Percent of target health facilities with a functional facility/community-level Quality Improvement (QI) committee SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 3: Increased quality, availability and use of priority health services at the community level in targeted districts SBH Sub-Result 3.2: Strengthened linkages between the community and facility for key health interventions Is this a Performance Plan and Report Indicator? No X Yes Reporting Year(s) 2015-2020 DESCRIPTION Precise Definition(s):
Target health facilities: In targeted facilities: This includes the specific facilities selected by SBH for priority interventions (Kabwe: Bwacha, Chowa, Kabwe General Hospital, Kalwelwe, Kang'omba, Kasanda, Kasavasa, Kawama, Katondo, Makululu, Mukebeko Township, Munga, Natuseko, Ngungu, Pollen. Mkushi: Changilo, Chisanaga, Fiwila, Kabengeshi, Kakushi, Luanshimba, Mikunku, Miloso, Milungwe, Mkushi District Hospital, Mulungwe, Munsakamba, Nambo, Nkolonga, Twatasha. Kitwe: Buchi Main, Chavuma, Chimwemwe, Garnatone, Itimpi, Kakolo, Kawama, Kwacha, Kitwe Central Hospital, Luangwa, Mindolo I, Mulenga, Ndeke Main Clinic, Wusakile, Zambia National Service. Mufulira: Butondo, Clinic 1, Clinic 2, Clinic 3, Luansobe, Kamuchanga Clinic, Kamuchanga Hospital, Kansuswa, Mokambo, Murundu, Mutundu, Ronald Ross General Hospital, Taung-up, Twatasha, 14 Miles. Chipata: Chikando, Chinunda, Chipangali, Chiparamba, Chipata General Hospital, Chizenje, Jerusalem, Kamulaza, Kapata, Kasenengwa, Kwenje, Madzimoyo, Mkanda, Muzeyi, Mwami Hospital. Petauke: Chisenjere, Kakwiya, Kalindawalo, Luamphande, Manyane, Mawanda, Merwe, Minga Hospital, Mumbi, Mwanjabanthu, Nyamphande, Petauke District Hospital, Petauke Urban, Ukwimi A, Ukwimi B. Lusaka: Bauleni, Chainda, Chawama, Chelstone, Chilenje, Chipata, George, Kalingalinga, Levy Mwanawasa Hospital, Kamwala, Kanyama, Matero Reference, Mutendere, Ng'ombe, University Teaching Hospital. Shibuyunji: Kapyanga, Mukulaikwa, Mwembezhi, Nampundwe, Shabasonje, Sichobo. Livingstone: Boma, Dambwa North, Hilcrest, Libuyu, Linda, Livingstone Central Hospital, Mahatima Ghandi, Maramba, Mosi-o-tunya, Nakatindi, Police Clinic, Prison Clinic, St Josephs, Zambia Air Force, Zambia Army. Monze: Bweengwa, Chikuni Hospital, Chisekesi, Hakunkula, Kanundwa, Katimba, Keembe, Luyaba, Manungu, Monze Mission Hospital, Monze Urban, Moomba, Nampeyo, Njola Mwanza, Siatontola).
Functional: The QI committee are counted as being functional if it meets if meets at least three of the following criteria: . Meetings documented with action items or minutes at least two times per quarter. . CCT includes at least one clinical mentor trained in each of the following SBH priority health areas (high-impact service), HIV/AIDS, Family Planning, EmONC related activities. . The CCT includes members with experience in each of the following: pharmaceutical logistical management, nursing care, diagnostic services, health information, and clinical care. . The facility has QI project. . Minimal requirements for gender sensitive balance and participation of women’s groups. . Numerator: Total number of targeted facilities with functional facility/community-level QI committee Denominator: Total number of targeted facilities Unit of Measure: Percentage of facilities Disaggregated by: Geographic (province/district/ facility)
133
Justification & Management Utility: SBH support the establishment of QI committees in the targeted areas at all levels. The QI committees identify the gaps in management and health service delivery to address these gaps through the implementation of QI projects and clinical mentorship. To effectively implement and coordinate mentorship at all levels, SBH collaborates with the PHO/DHO to establish multi-disciplinary Clinical Care Teams (CCTs). Members of the CCTs provide appropriate mentorship to health facility staff in clinical case management, and also help strengthen systems such as pharmaceutical management, nursing care, diagnostic services and health information. This promote the establishment of functional QIs in the targeted facilities. This indicator measures the proportion of target health facilities with a functional9 facility/community-level QI committee. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: Document review at facility level, i.e. minutes, mentorship forms Data Source: SBH records, facility/QI documentation Method of Data Acquisition by USAID: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts routine data quality assessments, in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. Frequency and Timing of Data Acquisition by USAID: Quarterly, semi-annually, and annually Estimated Cost of Data Acquisition: Negligible, part of routine reporting by Provincial and District Health Offices Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): Limitations include the unavailability of required documents (e.g. meeting minutes) that SBH can use to verify the criteria to measure functionality. Actions Taken or Planned to Address Data Limitations: A standard data collection tool has been designed to ensure this indicated is tracked in a standard way. Date of Future Data Quality Assessments: Quarterly Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team conducts this indicator using standard facility tools in alignment with the SBH M&E Plan. Routine DQA approaches to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Quarterly, semi-annually, and annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 0 ne 2016 0%
134
2017 80% 2018 60% 2019 80% 2020 80% EOP 80% THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
135
Performance Indicator Reference Sheet Name of Development Objective: DO3 Human capital improved Name of Intermediate Result: IR 3.2 Health status improved Name of Indicator: SBH 49: Percent of children under five years of age who received Vitamin A from USG- supported programs in target districts SBH Goal: Improved health outcomes for Zambians through systems that underpin the delivery and use of quality, high impact health services at district and community levels SBH Result 3: Increased quality, availability and use of priority health services at the community level in targeted districts SBH Sub-Result 3.3: Implement community level Social and Behavioral Change Communication (SBCC) interventions to increase utilization of high impact health services Is this a Performance Plan and Report Indicator? No Yes X Reporting Year(s) 2015-2020 Indicator # 3.1.9.2-3 DESCRIPTION Precise Definition(s): This indicator will measure the percent of children under five years of age who received Vitamin A through SBH supported programs in targeted districts.
Targeted districts: This includes Chipata, Petauke, Lundazi, Nyimba, Kabwe, Mkushi, Chisamba, Kapiri Mposhi, Kitwe, Mufulira, Chingola, Chililabombwe, Shibuyunji, Lusaka, Luangwa, Chirundu, Livingstone, Monze, Gwembe, Chikankata
Numerator: Number of children under five who received Vitamin A supplementation through a SBH-supported program in the last six months Denominator: Number of children under 5
HMIS Code: HMIS CHN2-065 + CHN2-070 Unit of Measure: Number of children Disaggregated by: Urban and rural; Geographic (province/district) Justification & Management Utility: Vitamin A supplementation reduces risk of under-five mortality by about one-fourth among children deficient in this micronutrient. The number of children under five years of age who received Vitamin A from USG-supported programs in the last six months from the time this data is collected is captured. In order to best affect Vitamin-A deficiency, children need two rounds of coverage in one year. In order not to double count children, data is reported annually and capture the number of children who received Vitamin A in the last six months.
Ideally children receive two doses within one year. Vitamin A supplementation reduces risk of under-five mortality by about one-fourth among children deficient in this micronutrient. This indicator allows the tracking of the number of children who receive vitamin A in target districts. In order to best affect Vitamin-A deficiency, children need two rounds of coverage in one year. PLAN FOR DATA ACQUISITION BY USAID Data Collection Method: HMIS/DHIS2. Data is reported annually, three months after the end of the year (due to the time lag with which this data becomes available from the HMIS). Data Source: HMIS/DHIS2 Method of Data Acquisition by USAID: HMIS/DHIS 2 Frequency and Timing of Data Acquisition by USAID: This indicator is reported after every six months of the latest child health week. Estimated Cost of Data Acquisition: Negligible, relies on routine statistics
136
Individual Responsible at USAID: COR, Systems for Better Health Individual Responsible for Providing Data to USAID: COP, Systems for Better Health Location of Data Storage: SBH head office, Lusaka DATA QUALITY ISSUES Date of Initial Data Quality Assessment: TBD Known Data Limitations and Significance (if any): There is a potential data limitation as the population head count can provide a higher denominator compared to CSO numbers. Actions Taken or Planned to Address Data Limitations: In order not to double count children, data will only be reported annually and capture the number of children who received Vitamin A in the last six months Date of Future Data Quality Assessments: Semi-Annual Procedures for Future Data Quality Assessments: In close collaboration with USAID and Ministry of Health, M&E staff, SBH M&E team use the HMIS/DHIS2 to track this indicator in alignment with the SBH M&E Plan. Routine DQA approaches are done to ensure that data used by SBH for performance monitoring and decision making is sound and reliable. PLAN FOR DATA ANALYSIS, REVIEW, & REPORTING Data Analysis: Compare targets to actual performance and review trends of this indicator Presentation of Data: Tables and graphs showing targets and actual, disaggregated as noted above Review of Data: Annually Reporting of Data: PMP data tables and narratives NOTES Notes: Baselines/Targets and Others including modifications PERFORMANCE INDICATOR VALUES Year Target Actual Notes Baseli -- 171% ne 2016 171% 2017 >80% 2018 >80% 2019 >80% 2020 >80% EOP >80% in each targeted district THIS PIR SHEET WAS LAST UPDATED ON: 24-JULY-16
137
Annex 5: Data Quality Assessment Checklist and Recommended Procedures
The following Data Quality Assessment Checklist is provided as an illustrative example of a DQA tool to be used by SBH. SBH will modify the tool to meet project specific needs.
USAID Mission or Operating Unit Name: Title of Performance Indicator: [Indicator should be copied directly from the Performance Indicator Reference Sheet]
Linkage to Foreign Assistance Standardized Program Structure, if applicable (i.e., Program Area, Element):
Result This Indicator Measures [For USAID only] (i.e., Specify the Development Objective, Intermediate Result, or Project Purpose):
Data Source(s): [Information can be copied directly from the Performance Indicator Reference Sheet]
Partner or Contractor Who Provided the Data: [It is recommended that this checklist is completed for each partner that contributes data to an indicator. It should state in the contract or grant that it is the prime’s responsibility to ensure the data quality of sub-contractors or sub grantees.]
Period for Which the Data Are Being Reported:
Is This Indicator a Standard or Custom Indicator? ____ Standard Foreign Assistance Indicator ____ Custom (created by the OU; not standard) Data Quality Assessment methodology: [Describe here or attach to this checklist the methods and procedures for assessing the quality of the indicator data, e.g., reviewing data collection procedures and documentation, interviewing those responsible for data analysis, checking a sample of the data for errors]
Date(s) of Assessment:
Assessment Team Members:
USAID Mission/OU Verification of DQA Team Leader Officer Approval X______
138
YES NO COMMENTS VALIDITY-Data should clearly and adequately represent the intended result.
1 Does the information collected measure what it is supposed to measure? (e.g., a valid measure of overall nutrition is healthy variation in diet; age is not a valid measure of overall health.) 2 Do results collected fall within a plausible range?
3 Is there reasonable assurance that the data collection methods being used do not produce systematically biased data (e.g., consistently over- or under-counting)? 4 Are sound research methods being used to collect the data? RELIABILITY-Data should reflect stable and consistent data collection processes and analysis methods over time. 1 When the same data collection method is used to measure/observe the same thing multiple times, is the same result produced each time? (e.g., a ruler used over and over always indicates the same length for an inch.) 2 Are data collection and analysis methods documented in writing and being used to ensure the same procedures are followed each time? TIMELINESS-Data should be available at a useful frequency, should be current, and should be timely enough to influence management decision making. 1 Are data available frequently enough to inform program management decisions? 2 Are the data reported the most current practically available? 3 Are the data reported as soon as possible after collection? PRECISION-Data have a sufficient level of detail to permit management decision making; e.g., the margin of error is less than the anticipated change. 1 Is the margin of error less than the expected change being measured? (e.g., if a change of only two percent is expected and the margin of error in a survey used to collect the data is +/- five percent, then the tool is not precise enough to detect the change.) 2 Has the margin of error been reported along with the data? (Only applicable to results obtained through statistical samples.) 3 Is the data collection method/tool being used to collect the data fine-tuned or exact enough to register the expected change? (e.g., a yardstick may not be a precise enough tool to measure a change of a few millimeters.)
139
INTEGRITY-Data collected should have safeguards to minimize the risk of transcription error or data manipulation. 1 Are procedures or safeguards in place to minimize data transcription errors? 3 Is there independence in key data collection, management, and assessment procedures? 3 Are mechanisms in place to prevent unauthorized changes to the data? SUMMARY Based on the assessment relative to the five standards, what is the overall conclusion regarding the quality of the data?
Significance of limitations (if any):
Actions needed to address limitations prior to the next DQA (given level of USG control over data):
IF NO DATA ARE AVAILABLE FOR THE COMMENTS INDICATOR If no recent relevant data are available for this indicator, why not? What concrete actions are now being taken to collect and report these data as soon as possible? When will data be reported?
Recommendations for Conducting Data Quality Assessments
Data Quality (DQ) assessor make sure that they understands the precise definition of the indicator by checking the Performance Indicator Reference Sheet. Please address any issues of ambiguity before the DQA is conducted. DQ assessors have a copy of the methodology for data collection in hand before assessing the indicator. For USAID Missions, this information should be in the PMP’s Performance Indicator Reference Sheets for each indicator. Each indicator should have a written description of how the data being assessed should be collected. Each implementing partner use a copy of the method of data collection in its files and documented evidence that it is collecting the data according to the methodology. DQ assessor record the names and titles of all individuals involved in the assessment. Does the implementing partner have documented evidence that it has verified the data that has been reported? Partners should be able to provided USAID with documents (process/person conducting the verification/field visit dates/persons met/activities visited) which demonstrates that they have
140
verified the data that was reported. Note: Verification by the partners should be an ongoing process. The DQ assessors are able to review the implementing partner files/records against the methodology for data collection laid out in the PMP (for USAID Missions only). Any data quality concerns should be documented. The DQ include a summary of significant limitations found. A plan of action, including timelines and responsibilities, for addressing the limitations should be made.
141
Annex 6: SBH RME Year 1 Work Plan
The following pages summarize key research, monitoring, and evaluation activities for Year 1. Please see the SBH Year 1 Work plan document for details.
142
SBH M&E Plan
2016 SBH RME Year 1 Workplan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Research, Monitoring and Total Budget $527,644 Evaluation 50000 Routine Project Monitoring and Evaluation 50001 Finalize SBH M&E Plan. The X X M&E team finalized the activity monitoring and evaluation plan (AMEP) for submission to USAID by January 15, 2016. The comprehensive AMEP provide a framework for measuring SBH results at the output, outcome, and impact levels and cohesively measure progress of project activities and contributions towards intended results. A revised M&E Plan is being submitted to USAID/Zambia by August 9, 2016 once baseline data collection is complete, with updated baseline and targets. 50002 Routine internal review of X X AMEP and indicators. In alignment with SBH annual work planning, the M&E team will lead consultations with the SBH team to review the SBH M&E approach and indicator pool to identify any potential needs to update or refine the project measurement approach. The M&E team also identifies new measurement opportunities and initiates the development of new research plans as appropriate. (in alignment with Year 2 work planning)
USAID Systems for Better Health ▌ pg. 143 SBH M&E Plan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 50003 Develop and launch X X X X X X X X comprehensive M&E system. SBH uses Abt’s enterprise monitoring and evaluation system, MandE. The M&E team works with the Abt CTC team to customize the system to meet SBH requirements. Once completed, the M&E team will train SBH staff on the use of the system and begin using it for routine data collection (MandE Work Plan Tracking and Indicator Data Repository Setup) by February 25, 2016. 50004 Design and conduct SBH X X X X X X baseline. To enable specific monitoring and evaluation of SBH effects in targeted districts and facilities, the M&E team designed and conducted a baseline assessment to collect and analyze data on key metrics related to the project focus areas. The baseline is comprised of a combination of existing data and information collected through SBH data collection efforts. Existing data sources include HMIS/DHIS2, HRIS, DSO, ZDHS, NHA, MOH documentation (e.g., provincial/ district/facility action plans), and other sources. Additional information was collected from the district and facility levels once agreed upon. The baseline design was finalized after the SBH geographic targeting decisions were agreed upon. (STTA) A SBH Baseline report was submitted to USAID/Zambia by August 5, 2016.
USAID Systems for Better Health ▌ pg. 144 SBH M&E Plan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 50005 Engage with MOH on DHIS2. X X The M&E Team Leader collaborate with the MOH and facilitated SBH access to routine HMIS data. This includes the training of selected SBH staff (M&E and program staff) on the Zambia DHIS2 system. (The SBH M&E team also contribute to relevant activities to support DHIS2 under Task 1.3) 50006 Organize quarterly X X X performance review meetings and prepare quarterly performance data for review and planning 50007 Organize the annual X performance review meeting and prepare annual performance data for review and planning 50008 Conduct SBH routine X X X X performance monitoring. The SBH M&E team conducts routine monitoring of SBH implementation and results. This include routine tracking of activity implementation; comprehensive results reviews on a quarterly basis (including development of SBH quarterly progress reports in alignment with USG reporting timelines: Q1 report due January 31, 2016; Q2 report due April 30, 2016; Q3 report due July 31, 2016); and consolidated review of annual results (including preparation of Year 1 annual report due October 31, 2016). The SBH M&E team organize internal consultations and performance reviews to ensure ongoing reflection on implementation status and project progress.
USAID Systems for Better Health ▌ pg. 145 SBH M&E Plan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 50009 Prepare SBH Monthly X X X X X X X X X X X X Updates. In alignment with contractual requirements, the SBH M&E Team prepare monthly updates to highlight major activities, accomplishments, and/or results to date. The monthly updates include upcoming STTA, core activities of the previous month, and highlights of projected activities/events. 50010 Contribute and participate in X X USAID/Zambia Portfolio Reviews. The SBH M&E team prepare semi-annual and cumulative performance summaries and participate in USAID/Zambia portfolio review processes. In addition, SBH M&E team ensures that appropriate SBH M&E data is submitted in DATIM semi-annually (within 15 days of USAID/Zambia portfolio review meetings/presentations). 50011 Conduct Quarterly Data X X X Quality Assessments. SBH M&E team conducts quarterly data quality assessments on a sample of selected performance indicators in alignment with the SBH M&E Plan. 51000 Design and conduct research and evaluation to measure SBH contributions towards results
USAID Systems for Better Health ▌ pg. 146 SBH M&E Plan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 51001 Develop a SBH Research and X X X Evaluation Plan. The Research and M&E Director lead a collaborative process to define research and evaluation plans. He uses rapid assessments, contextual mapping exercises, and a deep-dive analysis of existing health sector data during the first year of the project to develop a SBH research and evaluation plan. SBH use targeted surveys and studies to: 1) provide baseline, mid-point and/or end-of-project data for select indicators (such as availability of services in SBH districts); 2) generate a better understanding of the operating context through formative research to inform interventions and generate lessons for stakeholders (such as provider attitudes and behaviors); and 3) study the outcomes and attribution of innovative approaches (including pilots) though robust methods. During the first quarter, SBH M&E team collaborate with health sector partners (e.g., through stakeholder and partner coordination mechanisms, TWGs) to identify synergies and opportunities for research and data collection collaboration among partners. 51002 Initiate SBH research studies. Per the SBH Research and Evaluation Plan, the M&E team develops concept notes, research protocols, and implementation plans for any research studies to be launched in Year 1. 51003 Design measurement X X X X X X approach for SBH pilots: As SBH develops plans to test innovative interventions through pilot approaches, SBH Research
USAID Systems for Better Health ▌ pg. 147 SBH M&E Plan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC and M&E Director collaborates with the relevant SBH program team to design and implement evaluative approaches to measure the outcomes and effects of pilot programs. 51004 Assessment of the effectiveness of ART adherence and retention in selected provinces in Zambia. SBH RME team undertake the assessment of ART adherence and retention in SBH target provinces to determine effectiveness of structures and mechanisms established to promote ART adherence and retention for the purpose of strengthening follow-up of patients on treatment. This will entail assessment of various mechanisms and review of other assessment reports on the subject to help SBH strengthen HIV/AIDS management. 51005 Determining quality improvements in the design, development and delivery of HIV/AIDS program management and training. SBH (RME) team will design a study to assess quality assurance measures undertaken in the design and development of HIV/AIDS training and management programs to determine how quality assurance impacts on performance of management and training interventions. This is important for SBH's capacity strengthening interventions as it will identify performance gaps resulting from inappropriate design of interventions for HIV/AIDS management
USAID Systems for Better Health ▌ pg. 148 SBH M&E Plan
Activity Number Specific Activities Year 1 Budget JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 51006 Assessing impact of verticalization of HIV/AIDS on health systems strengthening. 52000 Collaborate with Health Sector M&E Efforts (link to activities under 1.3)
52001 SBH participation in health X X X X X X X X X X X X sector M&E mechanisms. The SBH M&E team actively participate in the national M&E TWG (by being part of the secretariat), collaborate with health sector M&E partners, and participate in relevant activities as identified. SBH participation include specifically supporting the M&E TWG to conceptualize a pilot for community HMIS, as well as other initiatives that are relevant to SBH results areas. 52002 Support MOH M&E related X X X activities. The SBH M&E team will support the MOH, as appropriate, to complete critical monitoring, evaluation, and data analysis related efforts. In Year 1, SBH may contribute to the finalization of the updated HMIS procedure manuals, updated 2014 facility listing report, and other relevant products as appropriate.
USAID Systems for Better Health ▌ pg. 149 SBH M&E Plan
USAID Systems for Better Health ▌ pg. 150