QUALITY ASSURANCE PROJECT EVALUATION REPORT The Zambia Quality Assurance Program Final Evaluation Center for Human Services • 7200 Wisconsin Avenue, Suite 600 • Bethesda, MD 20814-4811 • USA ���������� ������������������������� ���������������� ���������������������� ��������������������� ������������������� ������������� ��������������� ������������ ������������������ ������������������������� �������������������������������� ����������������������� ������������������������������ ����������������� ��������������� �������� The authors are indebted to the Central Board of An evaluation of this nature could be threatening to Health (CBoH) staff for their assistance throughout the all stakeholders of the Quality Assurance Program in mission, both at a technical level and for secretarial Zambia, although the findings should relieve part of the assistance, physical facilities, transportation, and equip­ stress. The intentions of the evaluation team were to ment made available throughout the mission. Without draw lessons that could benefit both Zambia and other the efforts of the CBoH, the logistics would have un­ countries, as objectively as possible. In doing so, doubtedly been more challenging, and the output less achievements are recognized, and remaining challenges likely in the time available. Senior officials’ willingness are identified. Successes are praised, whereas failures to make themselves available at all times is recognized and shortcomings are addressed through constructive with considerable gratitude. The atmosphere of collabo­ recommendations. This evaluation report is intended to ration that prevailed throughout the mission illustrates motivate all those who continue to work for a higher well the partnership spirit in which it was designed. level of quality of care and health services in Zambia. The evaluation team acknowledges the support Despite its efforts to validate findings and their provided by the USAID mission in Lusaka in funding interpretation, the evaluation team is aware of the risk this evaluation and thanks the Agency for providing the of reporting inaccurate data and missing other parts of Quality Assurance Project with the opportunity to relevant information. The evaluation team takes full contribute to the success of the Health Sector Reform in responsibility for such omissions and inaccuracies; the Zambia. The evaluation team’s work was supported by views expressed in this report reflect the opinion of the USAID under Contract No. HRN-C-00-96-90013 to the evaluation team only and no other institution. Center for Human Services. The team felt privileged to be associated with such an important undertaking and would like to give its special thanks to Dr. Eddie Limbambala, Dr. Mwendaweli �������������������� Maboshe, Ms. Joyce Tembo, and Mr. Albert Mulungu Bouchet, B., C. Lumbwe, J. Øvretveit, E. Msidi, M. for facilitating the mission. We are also grateful for Francisco, M. Lyby, and E. Goodrich. 2000. The Mr. Augrey Mwenda, CBoH, and Ms. Pay Sayer, Zambia Quality Assurance Project: Final evaluation. URC-CHS, for their essential contributions. Evaluation Report 1(2). Published for the United States Agency for International Development (USAID) by the Quality Assurance Project (QAP): Bethesda, Maryland. ����������������� The Evaluation Report series presents the findings, recommendations, and lessons learned of completed quality assurance (QA) evaluations in countries with long-term QA programs. An electronic copy of this publication may be found at [email protected]. ��������������������������������������� ����������������� List of Tables and Figures .............................................................................................................................................. iv List of Appendices .......................................................................................................................................................... v Abbreviations and Acronyms ......................................................................................................................................... vi Abstract ....................................................................................................................................................................... vii Executive Summary ....................................................................................................................................................... ix A. Introduction ..................................................................................................................................................... ix B. Findings and Recommendations: General ........................................................................................................ ix C. Findings and Recommendations: Development of Standards ........................................................................... ix D. Findings and Recommendations: Communication of Standards ........................................................................ x E. Findings and Recommendations: Assessment of Quality Performance .............................................................. x F. Findings and Recommendations: Problem-Solving Teams ............................................................................... xi G. Findings and Recommendations: QAP Support Systems ................................................................................. xii I. Introduction ............................................................................................................................................................. 1 A. Purpose of the Evaluation ................................................................................................................................. 1 B. Methodology .................................................................................................................................................... 1 C. Structure of the Report ..................................................................................................................................... 2 II. Background: Quality Assurance in Zambia .............................................................................................................. 2 A. The Relevance of QA for Zambia ..................................................................................................................... 2 B. Zambia’s QA Activities .................................................................................................................................... 3 III. Achievements and Challenges ................................................................................................................................. 4 A. Standards of Quality for Zambia’s Healthcare System ....................................................................................... 4 B. Evaluation of Problem-Solving Teams ............................................................................................................ 17 C. Evaluation of the QA Support Systems ........................................................................................................... 25 D. The Role of Other Structures and Sectors in QA Activities ............................................................................. 32 IV. Detailed Recommendations for Strengthening QAP ............................................................................................... 34 A. General Recommendations ............................................................................................................................. 34 B. Recommendations on Standards of Care ......................................................................................................... 35 C. Recommendations on Performance Monitoring .............................................................................................. 36 D. Recommendations on QA Activities ............................................................................................................... 36 E. Recommendations Regarding the QA Support Systems .................................................................................. 37 F. Prioritizing the Recommendations .................................................................................................................. 38 V. Conclusion ............................................................................................................................................................ 39 Work Cited ................................................................................................................................................................... 40 �������������������������� Table 1–1: Summary of the Main Recommendations of the Evaluation Team ............................................................ xv Figure 1–1: CBoH Organization Chart ......................................................................................................................... 3 Table 4–1: Changes in Provider Performance with Training and with Time ............................................................... 10 Table 4–2: Changes in Provider Performance with Clinical Supervision .................................................................... 10 Table 4–3: Features of Supervision Checklists .......................................................................................................... 12 Table 4–4: Types of Data Collected on Health Center Performance ..........................................................................