QUALITY

ASSURANCE

PROJECT

TECHNICAL REPORT SUMMARY

The QAP/BASICS Joint Project

Center for Human Services • 7200 Wisconsin Avenue, Suite 600 • Bethesda, MD 20814-4811 • USA The Quality Assurance Project (QAP) is funded by the U.S. Agency for International Development (USAID), under Contract Number HRN-C-00-96-90013. QAP serves countries eligible for USAID assistance, USAID Missions and Bureaus, and other agencies and nongovernmental organizations that cooperate with USAID. The QAP team consists of prime contractor Center for Human Services (CHS), Joint Commission Resources, Inc. (JCI), Johns Hopkins University School of Hygiene and Public Health (JHSPH), Johns Hopkins Center for Communication Programs (JHU/CCP), and the Johns Hopkins Program for International Education in Reproductive Health (JHPIEGO). It provides comprehensive, leading-edge technical expertise in the design, management, and implementation of quality assurance programs in developing countries. CHS, the nonprofit affiliate of University Research Co., LLC (URC), provides technical assistance and research for the design, management, improvement, and monitoring of healthcare systems and service delivery in over 30 countries. T E C H N I C A L R E P O R T S U M M A R Y

The Niger QAP/BASICS Joint Project Abstract

This summary presents the objectives, findings, lessons learned, and recommendations of the fall 1998 evaluation of a quality assurance (QA) program in Niger, a French-speaking Table of Contents country in West Africa with a population of 10 million. The I. Introduction ...... 1 Nigerien QA Project (QAP) began A. Events Influencing Nigerien Healthcare in the Late 1990s ...... 1 in the region in 1993 and in 1997 merged with another B. The Evaluation of the QAP/BASICS Project ...... 3 healthcare program—Basic C. Healthcare Environment Pre-QAP ...... 4 Support for Institutionalizing II. TAP Objective Number 1 ...... 5 Child Survival (BASICS)—to A. Defining Quality ...... 5 become the joint QAP/BASICS B. Immediate Results of the Supervisory System ...... 6 project. Later, the Konni District (Tahoua) and the Boboye District C. Improving Quality ...... 7 (Dosso) were selected for QAP/ III. TAP Objective Number 2 ...... 9 IMCI (Integrated Management of A. Introducing IMCI in Niger ...... 10 Childhood Illness) interventions; B. Local Implementation ...... 10 this was the first time in the C. Results of Introducing IMCI in a QA Environment ...... 10 history of international health interventions that IMCI was D. Dissemination of Information on the Nigerien QAP ...... 12 introduced in an environment IV. Cost Analysis of the Niger QAP/BASICS Project ...... 12 where QA practices were in A. Sustainability Costs ...... 13 place. B. Basic QA and IMCI Training Costs ...... 13 Important lessons learned from C. Training of Trainers ...... 13 the evaluation of the first five D. Team-Based Problem Solving ...... 13 years of Nigerien QAP and QAP/ E. Supervision and Coaching ...... 14 BASICS relate to the savings and F. Supervision Checklist ...... 14 benefits derived from introducing IMCI in a QA environment, the G. Quality Meetings ...... 14 ability of healthcare providers to V. Lessons Learned ...... 14 learn and adapt QA principles A. For Niger ...... 15 and implement solutions, the B. For the International Health Community ...... 16 development of teamwork among C. Main Factors Limiting QA ...... 16 QA-trained staff, and the importance of coaching and VI. Recommendations ...... 17 meetings to support and sustain A. Establishment of Standards ...... 17 QA activities. The story of QA in B. Supervision and Monitoring of Standards ...... 17 Tahoua stands out as an C. Coaching and Quarterly Meetings ...... 17 excellent example of how QA D. Problem Solving ...... 17 activities can improve the quality of care, even in the face of E. Sustainability ...... 17 severe resource constraints. F. Dissemination and Documentation ...... 18 Acknowledgement

The authors wish to express their appreciation to Youssef Tawfik, MD, MPH; Xavier Crespin, MD, MPH; and Mamoudou Djingarey, MD, MPH, for their participation on the evaluation team. The evaluation team is indebted to the staff of the QAP/BASICS project for their technical assistance throughout the mission and for the secretarial help, transportation, and equipment they made available. The team also appreciates the willingness of the departmental and district health authorities in the Tahoua and Dosso to be available at all times. The evaluation team acknowledges the support provided by the USAID Mission in in funding this final evaluation and thanks the agency for providing the Quality Assurance Project with the opportunity to contribute to the success of the QAP/BASICS project in Tahoua and Dosso. The evaluation team’s work was supported through the Center for Human Services (USAID Contract No. HRN-C-00-96- 90013) and the BASICS Project (USAID Contract No. HRN 6006-C-00- 3031-00).

Recommended citation

Legros, S., E. Goodrich, and H. Abdallah. 2000. “The Niger QAP/ BASICS Joint Project.” Technical Report Summary 1(1). Published for the U.S. Agency for International Development (USAID) by the Quality Assurance Project (QAP): Bethesda, Maryland.

About this series

The Technical Report Summary series provides concise descriptions and background information on the Quality Assurance Project’s technical intervention results, evaluation results, and other project activities. This summary is derived from a longer report that was presented to Nigerien and U.S. officials immediately after the evalua- tion. For a copy of the longer report, write to [email protected] or call 301-941-8532. For more information on the evaluation of Niger’s QAP/BASICS project, please contact [email protected]. The Niger QAP/BASICS Joint Project

Stephane Legros, Elizabeth Goodrich, and Hany Abdallah

Missions, and field-based cooperat- ing agencies. The Center for Human Services (CHS) has operated QAP since its inception. I. Introduction During its first five years, QAP developed national and regional QA Based in Bethesda, MD, the Quality programs, stimulated QA interven- Assurance Project (QAP) helps tions in 16 countries, and conducted developing countries improve the training activities in another eight. Acronyms and Abbreviations quality of their healthcare services Since the start of its second five- by introducing quality assurance year phase in 1996, QAP has been ARI Acute respiratory infection (QA) tools and methods. Such tools placing greater emphasis on (e.g., checklists, flowcharts, cause- BASICS Basic Support for demonstrating the cost-effectiveness and-effect diagrams) and methods Institutionalizing Child Survival of QA interventions. Specifically, it is (e.g., standard setting, problem seeking to: (a) determine the cost- BPS Basic Package of (Healthcare) solving) ultimately enable a effectiveness of QA in national Services healthcare system to improve the programs, (b) document the impact health outcomes of the population CCM Combined Case Management and cost-effectiveness of QA served. QA has gained widespread through operations research, and CHS Center for Human Services acceptance in healthcare settings, (c) establish accreditation and other particularly as part of health sector IMCI Integrated Management of regulatory mechanisms. In 1993, reforms that seek to deliver cost- Childhood Illnesses QAP began to assist the Ministry of effective, equitable, and high-quality Public Health (MPH) in Niger in ISQua International Society for Quality services. While many countries have institutionalizing QA methods in its in Health Care articulated goals for healthcare healthcare system. That program is quality, ineffective structures and MPH Ministry of Public Health termed “the Nigerian QAP” or “the processes often hinder service QAP” hereafter. QA Quality Assurance delivery. QAP is working to improve those structures and processes by QAP Quality Assurance Project introducing QA interventions. RAHWP Rapid Assessment of Health A. Events Influencing The U.S. Agency for International Worker Performance Nigerien Healthcare in the Development (USAID) created QAP Late 1990s TAP Technical Action Plan in 1990 to improve the quality of healthcare, population, and nutrition The Nigerien QAP experience was USAID United States Agency for services in developing countries affected by numerous events that International Development through technical support to took place between its introduction WHO World Health Organization healthcare facilities, USAID in 1993 and the evaluation in 1998.

The Niger QAP/BASICS Joint Project (TRS) ■ 1 Figure 1 Those events included the imple- Factors Influencing Healthcare Utilization in Tahoua mentation of two other major health- related programs and several events outside the public health sector that Purchase Income affected that sector, including a crop Massive power linked to related to exodus 1997 harvest 1998 harvest failure-driven exodus that resulted in reductions in healthcare utilization. – – + + QAP Figure 1 shows several of these Health since events, and Table 1 is a timeline of Direct cost –/ 1993 + Center significant dates in the implementa- recovery beginning utilization + tion of the QAP and other major in 1998 in Tahoua CCM in 1995 + then IMCI in healthcare developments. Descrip- mid-1998 tions of some developments follow: UNICEF + and ALAFIA Complaint/Treatment programs – The QAP started in the Tahoua Strategies in 1993 – Department of Niger in April 1993:1 The Nigerien QAP sought to provide Vertical training for Mother and Child the government of Niger with both Healthcare/Extended Immunization Coverage technical and operational assistance to improve the delivery of critical Note: ALAFIA is a healthcare program sponsored by Germany. primary healthcare services in a limited demonstration region. This Table 1 effort first aimed to identify and address the most prevalent illness- Timetable of Major Healthcare Developments related problems of this population and then determine the resources QA Year BASICS and Other necessary to deliver a basic QA introduced in Tahoua at the health center 1993 Cost recovery introduced in limited areas package of services (BPS) to and district levels prevent and/or solve those problems. QA training begins 1994 QA training of health center supervisors 1995 BASICS introduced After completing this introductory work, the QAP introduced a quality QA conference in Tahoua CCM introduced in Boboye and Say Districts assurance system of management QA training continues 1996 for healthcare delivery through: QA introduced in Boboye (a) clarifying, communicating, and monitoring clinical and management Training of trainers 1997 Cost recovery starts to be introduced in Konni, standards; (b) QA training; and Illéla and Boboye Start of joint QAP/BASICS project (c) implementing and supporting a Introduction of supervisory checklist process for preventing and correct- ing problems. In addition, the QAP st 1 RAHWP sought to improve the quality of 2nd RAHWP 1998 Cost recovery continues to be introduced in family healthcare service by increas- Konni, Illéla, and Boboye International QA conference in Niamey ing effective coverage of the IMCI introduced in Konni and Boboye population with preventive services, Evaluation improving case management of the 3rd RAHWP most prevalent conditions threaten-

1 Niger is divided into seven political regions or “departments,” and the term is used in this report in reference to a region and not to a functional division of an organization. “Districts” are subdivisions of departments.

2 ■ The Niger QAP/BASICS Joint Project (TRS) ing women and children, and In 1995 an independent, USAID- interventions that IMCI was intro- strengthening core support services. funded program called BASICS duced in an environment where QA While improving health outcomes, (Basic Support for Institutionalizing practices were in place. the QAP aimed to demonstrate the Child Survival)2 began: effectiveness and feasibility of BASICS supported the MPH in In the fall of 1998 the evaluation quality management strategies by introducing Combined Case took place: assessing the application of QA Management (CCM) for children At that time, QAP/BASICS was interventions, analyzing their impact, in primary healthcare facilities.3 ongoing in Tahoua Department and and disseminating the findings that Implemented within limited areas, Boboye. The evaluation team also resulted. CCM revised national policies for visited non-QAP facilities for diarrheal diseases, acute respiratory comparison. A QAP resident advisor directed infection (ARI), and malaria. in-country activities and worked directly with the Tahoua Department In 1996 the QAP extended QA Healthcare Director, his staff, and B. The Evaluation of the beyond Tahoua to the Boboye district healthcare personnel to QAP/BASICS Project District in the : accomplish project objectives. This step would support the mid- In late 1998, USAID hired an 1998 introduction of the Integrated evaluation team to review and In 1993 cost recovery began in Management of Childhood Illness assess the QAP/BASICS project. The limited fashion in Tahoua and then (IMCI) clinical guidelines. team consisted of four doctors who evolved into different forms as it Developed by the World Health specialize in public healthcare—one spread to other regions: Organization (WHO), IMCI was later from the Quality Assurance Project Cost recovery requires patients to incorporated as part of BASICS in in Bethesda, Maryland, one from pay for part of their health services Boboye. BASICS, and two from the Nigerien and medications and thus effects MPH—and a healthcare program their decisions of whether to seek In early 1997 the QAP and BASICS assistant from USAID. healthcare for themselves or their merged to become the QAP/BASICS dependents. In Niger, funds raised Just before the evaluation, the team joint project in Niger: through cost recovery were used to attended a USAID-funded interna- QAP/BASICS focused on the replenish supplies of medications. tional conference in Niamey, Niger, possibility of revising the approach This improved the healthcare where the results of the QAP/BASICS to the improvement of child system’s ability to provide services project were presented by those healthcare services at the district and medications, which also involved in its implementation. Next, level. The Konni District (Tahoua) influenced decision making. The the evaluation team reviewed all and the Boboye District (Dosso) evaluation assessed the effects of QAP-related documents and visited were selected for QAP/IMCI cost recovery in the districts of two department-level health director- interventions on the basis of the Konni, Illéla, and Boboye, where it ates, six district headquarters (which availability of essential medicines was implemented in late 1997 and provide administrative and manage- and the interest shown by staff. This early 1998. ment support for district hospitals development is important to the and healthcare centers), and 13 international health community primary healthcare centers (health because it was the first time in the centers). The team interviewed history of international health about 60 individuals in all—from the

2 BASICS provides technical leadership to reduce child mortality and illness worldwide. It applies cost-effective, child-survival interventions in six programmatic areas: sustaining immunization programs, integrated management of childhood illness (IMCI), strengthening the link between nutrition and health, promoting and sustaining healthy behaviors, improving techniques for monitoring and evaluation, and establishing public/private partnerships. 3 Primary healthcare facilities in Niger are generally small clinics staffed by a nurse or less-trained healthcare provider. These facilities generally serve as the initial point of contact between a patient and health services; cases that cannot be treated at the health centers are referred to hospitals and specialist facilities.

The Niger QAP/BASICS Joint Project (TRS) ■ 3 Nigerien healthcare system, the of project performance and results; information and solve problems. In QAP, BASICS, and other interna- their efforts resulted in a broad addition, health managers and staff tional staff—and conducted two overview of QA accomplishments in had neither the knowledge nor the focus groups of women who used Niger. The team recognizes that tools to solve problems or improve the healthcare facilities in seeking factors and programs other than delivery systems. care for themselves and their QAP/BASICS may have influenced Not only were many of the health children. Interviews were guided by those accomplishments. centers lacking the essential a multilevel questionnaire covering equipment to provide a basic different aspects of QA. The team package of services, but the presented a preliminary report to the C. Healthcare Environment concept of a BPS was merely a MPH General Secretary and other Pre-QAP vague desire in the minds of a few partners on the last day of the high-level managers who had evaluation to validate its findings Before QA activities began in 1993, neither the information to design nor and recommendations. clinical standards were unknown to Nigerien healthcare providers (see the means to implement one. The evaluation team used the USAID Table 2). Health workers had no Another example of the poor state of 1997/98 Technical Action Plan (TAP) understanding of or experience with healthcare in Niger before the QAP as its guiding document. Written by working in teams. Regional supervi- is the fact that cleanliness was USAID, the TAP set the objectives sory competence was lacking, and largely unsatisfactory in most for the QAP/BASICS project. Those no district supervisory teams facilities. objectives are presented in Chap- existed. Healthcare workers could Table 3 presents the baseline ters II and III. receive training only by means of indicators and departmental The goals of the evaluation were to: vertical processes—no regular objectives that were in place as (a) assess QAP/BASICS’ success in meetings were scheduled to share Niger’s QAP began. achieving those objectives, (b) summarize the lessons learned from introducing IMCI in a QA environ- Table 2 ment, (c) present a cost analysis for Existing Standards for Patient Management and Health Center sustaining QAP/IMCI in Niger after the cessation of QAP/BASICS, and Administration (d) develop recommendations to help the project expand into other Department Unit Standards and Guidelines Source Year areas of Niger and the surrounding Child Health and Development Mother and Child Healthcare/ MPH 1993 countries that are, or may become, Family Planning interested in introducing IMCI in a QA environment. Handbook of Clinical Procedures MPH 1995 for tuberculosis, malaria, and The evaluation team focused on: malnutrition cases (a) documenting the benefits of QA institutionalization in the healthcare Complaint/Treatment Strategies MPH 1993, revised integration process, (b) document- in 1998 ing programs that increased Combined Case Management BASICS/MPH 1995 healthcare workers’ qualifications and motivation, and (c) evaluating IMCI WHO/MPH/BASICS Mid-1998 how standards and tools were Health Center Management Standard management procedures QAP/MPH 1996, revised introduced to achieve the integration manual (administration, finance in 1998 of QA and IMCI approaches. organization, and health policy) The evaluation team does not claim to have made an exhaustive analysis

4 ■ The Niger QAP/BASICS Joint Project (TRS) Table 3 effectiveness of the QAP’s efforts to BPS Baseline Indicators: 1993, 1997, and Objectives strengthen cost recovery. This section presents the team’s findings (Tahoua Department; in Percentages) in these areas as they relate to the three aspects of QA: defining Indicators 1993 1997 Department Objective quality, assessing quality, and improving quality. Utilization rate of curative services (total population) 30 29.8 50 Infant consultation coverage rate (0–5 km.) 58 75.6 85 A. Defining Quality Prenatal consultation coverage rate (0–5 km.) 63 76.6 85 Standards are essential to achieving Nutritional rehabilitation recovery rate 14 24.8 25 customer satisfaction, but the Nutritional rehabilitation drop-out rate 15 16 10 evaluation team found that Nigerien health workers had difficulty manag- Family planning utilization rate (total population) 5 14 7 ing the numerous clinical standards BCG coverage (0–11 months) 35 76 100 (listed in Table 2) that had been developed by the Nigerien govern- Fixed coverage rate for measles (0–11 months) 24 55 80 ment, Tahoua Department manag- Fixed coverage set rate of DTCP/3 (0–11 months) 63 68 90 ers, and donor partners. In addition, some clinical protocols conflict with Fixed return rate anti-tetanus vaccine 2/1 74 73 90 others, causing confusion for Source: Official Tahoua data healthcare providers. NB: These indicators were calculated from demographic data from the general population census Standards of healthcare may be (1988) and revised according to the annual population growth rate in the Tahoua Department and communicated to health workers calculation methods recommended by Niger’s national health information system. through several means: preservice Demographic Data: training, inservice training, meetings of health workers and/or supervi- Children aged 0–11 months represent 4.7 percent of the population. sors, the dissemination of publica- Expected pregnancies represent 52/1000 of the population. tions bearing the standards, Women of childbearing age represent 22 percent of the population. on-the-job training through supervi- sion, job aids, or any combination of these. The evaluation team found that through six regional (to West II. TAP Objective activities in the Konni, Illéla, and Africa), national, and local training Boboye districts. sessions held in Niger in 1998, 39 Number 1 Nigerien health staff received Institutionalize quality assurance in For this objective, the evaluation training in the IMCI clinical guide- two districts in Tahoua Department team determined the degree of lines. This number includes six who 4 and one in Dosso Department. institutionalization of QA in two received training of trainers. Through Strengthen the supervision teams’ districts in Tahoua and one in Dosso, documentation and other evidence, management capabilities, revitalize measuring the capabilities of the the evaluation team found that 14 problem-solving teams, and support supervision teams and the problem- out of 18 health centers have at least and strengthen cost-recovery solving teams, and considering the two IMCI-trained staff.

4 “Institutionalization” can be described as follows: When QA activities are formally and functionally incorporated into the structure of a healthcare system or organization; are consistently implemented; and are supported by a culture of quality, as reflected in organizational values and policies that advocate quality care.

The Niger QAP/BASICS Joint Project (TRS) ■ 5 Other standards were usually 1. Development and Activities evaluation, department staff had communicated during quarterly of the Supervisory System made several information collection meetings and supervisory visits. The The Quality Council initiated the visits, which may have resulted in QAP did not initially plan specific following measures: the kinds of improvements one training for the implementation of the would expect from a supervisory ■ national standards it developed. Training in supervisory tech- visit. Supervisory visits by district niques (each district has at least staff of health centers were being four supervisors to guide and held as planned—every three B. Assessing Quality assist health center problem- months—and had helped the health solving teams) centers acquire the technical and Monitoring and supervision systems ■ financial support they needed, work together to ensure that District-level management teams which improved operations and staff healthcare providers use standards of two doctors, a manager, a motivation. appropriately. To measure the extent communicator, and an epidemi- of their use and their effectiveness, ologist, all trained in QA, Health workers had been trained to indicators must be established and coaching, and/or supervision ensure a functinal transport system: records kept, and supervisors must (successfully formed in six of Healthcare providers received ensure that the records are used for eight districts of Tahoua) training in maintaining equipment supervision (to improve provider ■ Health worker training in techni- and managing inventory, and the car performance). Supervisory activities, cal equipment and inventory pool system was strengthened, including monitoring the use of management, and a stock although it was still not fully ad- standards, were largely nonexistent management system equate at the time of the evaluation. before the QAP began in Niger. The ■ Standards were being developed concept of supervision implied an Improvements in the car pool and implemented: authoritarian nature and a vertical system Management standards and approach. ■ Management standards and a standards for the delivery of the BPS Within six months after the QAP BPS had been drafted, adopted, and began, a Quality Council formed in disseminated. Tahoua. Comprised of department- A supervisory checklist had been level healthcare managers, its 2. Immediate Results of the developed: objective is to guide, coordinate, Supervisory System The checklist has sections that help and follow all actions relating to While the ultimate goal of the QAP supervisors monitor health worker quality improvement. It identified is to improve health outcomes, performance in following standards supervision as the main vehicle to reaching it requires intermediate relating to IMCI; prenatal, postnatal, introduce and strengthen QA in steps. The Niger QAP’s success in and infant consultations; immuniza- Niger’s healthcare system. It called improving health outcomes through tion coverage, etc. It also covers for the creation of a supervisory improved supervision would rely on cost recovery, community participa- system, which began with a docu- several intermediate achievements. tion, monthly reports, and other ment that lists several elements The evaluation team reviewed the administrative reporting. From essential to supervision, such as a progress made and found as interviews with those who use the definition of supervision, objectives follows: checklist, the evaluators concluded of supervision, and methodology. The supervisory process was not that its strengths are that it is easy to Thus began the journey from an fully operational: use and archive and that its most authoritarian style of supervision It calls for supervisory visits by the important sections apply to IMCI and toward a team approach. The department of each district every six community participation. Its weak- following details the activities and months and by the districts of each nesses are that it is too long, its cost results of the supervisory system. of their health centers every three recovery section does not request months. While department visits the same information as the cost were still irregular at the time of the recovery system in Tahoua, and that

6 ■ The Niger QAP/BASICS Joint Project (TRS) sometimes the activities to be meetings also allow for the system- work—a health center, district monitored do not occur while the atic review of health outcome hospital, or administrative office— supervisor is visiting the health monitors (which, in turn, helps and participate in the development center. develop data for future quality and continuation of a “problem- improvement activities), the adoption solving” team. Teams tend to have A rapid assessment of health and communication of standards, three to eight members; composition worker performance (RAHWP) had and training and self-evaluation. varies, depending on who has been also been developed: trained and the needs and re- The RAHWP helps supervisors sources of the workplace. A health determine the quality of health center team might include a head services, especially those related to C. Improving Quality nurse, an assistant nurse, a tradi- IMCI. Using the RAHWP, supervisors Once supervision is in place to tional birth attendant, and the monitor health workers’ abilities to monitor the quality of healthcare director of the district office. QAP perform activities like greeting a services, the next step in improving recommends that community caretaker appropriately, taking a customer satisfaction is identifying members also participate, although patient’s history, examining children problems that hinder improvements they generally are not trained (and according to IMCI, classifying the in measurable healthcare outcomes were not in Niger). In Niger, continu- disease, identifying treatment, and finding solutions to those ous team membership was a verifying a child’s vaccination and problems. QAP advocates the use of problem because of high staff nutritional status, and counseling teams of healthcare providers to turnover. mothers. The evaluation team found implement this process. It also that the RAHWP helps workers instructs providers in using QA Identifying problems: improve their performance if results methods, such as planning and While QAP trains health providers to are discussed with them. This tool’s defining quality, quality monitoring, use data gathering and analysis to weaknesses are that it does not and quality improvement to identify identify problems, most of the cover health services other than problems and implement solutions. workers interviewed by the evalua- child health, that health workers are tion team used brainstorming To begin, Niger´s QAP trained over not accustomed to such intense instead. Brainstorming is also taught 520 health workers between 1993 scrutiny, and that it is time consum- by QAP but is appropriately used for and 1998. Courses included basic ing. other steps in problem solving. QAP QA training (400 trainees), coaching teaches providers to use a selection Managerial meetings were being (52), supervision (94), and IMCI (39). matrix to select a problem once held: Training was given every year from several are identified, and this Meetings of staff who engage in 1993 to 1998, with the largest method was used in 90 percent of supervisory activities to support QA number trained in 1996. The the cases discussed with the are held at two levels: (a) district evaluation team felt that the high evaluation team.5 Criteria used for head doctors and staff, and (b) density of trained staff contributed to selection usually included frequency health center managers of a district the level of teamwork among of the problem, its impact on other and their supervision team. The problem-solving teams and made activities, and its importance and meetings provide a forum for problem solving possible despite impact on service use. reflecting on and exchanging less-than-optimal mobility. information among health centers in The evaluation team found that 120 a single district or department. As problem-solving cycles had been part of the information exchange, 1. Problem-Solving Process started and were either continuing or participants benchmark successful After training, those trained are had been concluded at the time of problem-solving activities. The expected to return to their place of the evaluation. Another 27 had been

5 The selection matrix is a consensus development technique where a group of people who are familiar with the problems at hand are asked individually to weigh the advantages and disadvantages of solving each of those problems. Through various scoring techniques, individual preferences are combined to establish a group preference.

The Niger QAP/BASICS Joint Project (TRS) ■ 7 abandoned for reasons unknown problem and is especially important department staff would use part of it to the team. All cycles addressed if initial results from the problem to solve a problem in a day’s time. problems relating to the BPS, solving are unsatisfactory. The meaning that problems relating to evaluation team found that a review logistics or health center manage- was infrequently done; teams 3. The Coaching Mechanism ment failed to receive the attention tended to press on to a second A coaching mechanism was estab- they probably warranted. The most problem rather than assess the lished to assist the problem-solving frequently addressed problems impact of their solution. Alternatively, teams by providing support and related to either nutritional training they would analyze their solution but facilitation. Coaches tend to be and rehabilitation or family planning. not document their findings. district managers and healthcare Describing a problem: providers who have received training in both basic QA and The evaluation team found that 2. Results of Problem-Solving coaching. Coaching results in closer most teams had difficulty at first in follow-up of the teams’ work and wording the description of their Teams provides the support, such as problems, but all teams acquired The evaluation team found several advocacy for resources, necessary this skill with experience. constraints to problem-solving efforts, including the improper or to implement solutions. This mecha- Problem analysis: insufficient use of QA tools, conflict- nism is especially important during a The evaluators also found that the ing schedules, and lack of coach- team’s first cycle and at the begin- Nigerien problem-solving teams ing. On the other hand, the team ning of subsequent cycles. At the performed problem analysis well, found appreciable improvements in time of the evaluation, there were 24 especially with respect to using teamwork that can serve as a active coaches in Tahoua and four in cause-and-effect diagrams. How- foundation for improved healthcare. Dosso. The evaluation team con- ever, the teams had difficulties Particularly significant were the cluded that coaches significantly collecting data and never used motivating and participatory atmo- contribute to the problem-solving Pareto or control diagrams, which sphere for health workers, including process by encouraging teams, that QAP explains and recommends for having doctors shift their focus to they should be stronger at the this step. patients’ expectations and satisfac- department level, and that they Identifying, applying, and analyzing tion, regular brainstorming to solve should meet to share experiences. solutions: problems, commitment on the part of The evaluation team rated Nigerien workers to solve problems at the health providers highly in identifying lowest level possible, and health 4. Process Design/Redesign viable solutions and using selective center management that seeks to Process redesign—where healthcare criteria, such as cost and ease of improve the quality of healthcare. providers go beyond problem application. In applying solutions, Drug-stocking and cost-recovery solving to restructure a system or providers engaged in several types systems were implemented to process—is still formative in Niger, of activities, such as raising the organize the essential generic drugs but the evaluators were impressed public’s awareness and organizing inventory, sustain routine support with what they discovered. Although medical home visits. Local solutions activities, and finance restocking Nigerien healthcare managers and to problems were usually found (cost recovery and drug supply providers had no training or expo- without significant outside help, management are described below in sure to process redesign, there was although occasionally the commu- the chapter on the second TAP evidence of spontaneous develop- nity was asked to help build a facility objective). ment of these skills. annex or contribute food for nutrition At the department level, the evalua- For example, providers realized presentations. The team felt that, tion team reported finding spontane- during their problem-solving generally, the teams should ask for ous problem-solving practices. For sessions that it would be more community help more often. example, rather than implement the efficient to offer all healthcare Solution analysis helps teams entire problem-solving process, services to patients during a single evaluate their success in solving a visit, rather than have patients return

8 ■ The Niger QAP/BASICS Joint Project (TRS) to the health center on the day the costs (prescriptions are less the healthcare system in the pilot needed service was scheduled. The expensive under cost recovery, area become a model for implement- concept of integrating services—a where people buy medications ing IMCI: Lessons learned from that process redesign—was suggested through the public health system effort were to be disseminated and implemented on a fairly wide rather than through a private throughout Niger and the surround- scale in the pilot areas. pharmacist). The evaluation team ing countries. The team also notes that healthcare services in evaluated the impact of the QA Similarly, providers realized that new Niger generally remain too expen- environment on health workers’ skills mothers should have postnatal visits sive, especially for rural families and in IMCI. As noted above, in 1995 and implemented the idea on their during the rainy season. The team BASICS worked with the Nigerien own initiative. These examples was pleased to see that patients, MPH to introduce Combined Case resulted directly from QA practice now having to pay for services, are Management for children in primary and show that providers are going demanding better quality, which is healthcare facilities. CCM helped beyond simple problem solving to expected to continuously improve Niger revise policies for the treat- address faulty processes. those services. ment of diarrheal diseases, ARI, and malaria, thus reducing missed To make a comparison, the team opportunities to identify sick chil- 5. Results Analysis visited a health center that was not dren. The evaluation team found improve- part of the QAP/BASICS project or ments in most of the BPS indicators, the cost recovery system. It found In early 1997, the newly merged although other influences, such as that the facility without these QAP/BASICS project began to focus the implementation of cost recovery programs lacked many valuable on revising the approach to the and a crop failure-driven exodus, features. It saw less organized improvement of child healthcare hindered those improvements. For offering of services and activity services at the district level. The example, the use of curative planning, no monitoring of follow-up Konni District (Tahoua) and the services rose from 30 percent in indicators, no use of data, no Boboye District (Dosso Department) 1993 to 37 percent in 1995, but then awareness of any changes in health were selected for QAP/IMCI inter- dropped to just under 30 percent in outcomes, no provision of prescrip- ventions on the basis of the avail- 1997. The team believes that tions, and no integration of services. ability of essential medicines (in part improvements to the healthcare because of cost recovery) and the system, some of them created by interest shown by staff. IMCI focuses QAP/BASICS, caused the increase III. TAP Objective on five conditions that cause child and that by forces outside QAP/ Number 2 mortality: diarrhea, ARI, malaria, BASICS caused the decrease, measles, and malnutrition; BASICS although this cannot be proven. Review, test, and distribute in Niger was USAID’s main vehicle for Other indicator data are in Table 3. and the surrounding countries a introducing IMCI in the late 1990s. 6 One of the objectives of the evalua- The improvements in the indicators model to improve IMCI at the tion was to make recommendations after the setbacks may have been district level. Improve the health that would help QAP/BASICS caused by a growing satisfaction workers’ skills in IMCI. Disseminate expand into other departments of with healthcare services among the in Niger and the surrounding Niger and into neighboring countries Nigerien people. This in turn may region the materials produced and interested in introducing IMCI in an have been caused by: (a) improved lessons learned from the QAP/IMCI environment that has a quality availability of medicines (cost approach. assurance system. recovery income is used to resupply medicines/drugs), (b) the integration For this objective, the evaluation of services, and/or (c) lower medical team looked at the Nigerien QAP’s success in having certain aspects of

6 That is, the healthcare system in Niger where QAP/BASICS was implemented would serve as a model for neighboring regions in the management of childhood illnesses.

The Niger QAP/BASICS Joint Project (TRS) ■ 9 A. Introducing IMCI in place, thanks to the QAP and IMCI. Utilization rates before and after cost Niger In addition, the pilot QA/IMCI recovery suggest that it caused a program taught international and decrease in usage. Figure 2 shows The QAP/BASICS project provided governmental healthcare planners changes in utilization for three technical and managerial assistance the specifics of implementing IMCI healthcare services in Konni before in introducing IMCI into the Nigerien standards in a QA environment, and after cost recovery was imple- health system, including: including the provision of training in mented. And, as mentioned above, ■ Planning, through a working IMCI skills and use of quality cost recovery can be beyond the group, to adapt the IMCI generic7 improvement tools and methods. financial means of some families. standards to Niger’s specific needs 1. Cost Recovery and Drug 2. Health Indicators ■ Funding all 1998 workshops for Supply Management QAP/BASICS has improved the district managers in orientation, While cost recovery was in effect in ability of health staff to collect, QA training of trainers, and basic both Konni and Boboye, the evalua- analyze, and use data. The evalua- QA training of health workers in tion team found that they were not tion team was particularly impressed Konni and Boboye equally successful in developing with Konni District staff’s analytical ■ Orienting MPH decision makers, and implementing drug supply ability to monitor health service international partners, prominent systems. In Konni, the stockkeepers indicators (rates of family planning, pediatricians, and health workers were successfully managing prenatal, and neonatal visits, and in the pilot districts supplies, using payments from vaccinations) and to detect malnutri- clients to maintain stocks. In tion, tuberculosis, and leprosy. ■ Adapting national standards for Boboye, however, the team found vertical programs and the generic several missed opportunities that IMCI standards to account for were causing shortages. For C. Results of Introducing Niger’s relatively low rates of instance, first-line antibiotics malaria and parasitic infestation IMCI in a QA Environment required for IMCI were generally and to reduce the signs for The evaluation team looked at available but second-line antibiotics evaluating dehydration several factors that contributed to were lacking. the successful introduction of IMCI ■ Training to communicate the IMCI While the availability of essential in Niger’s districts with QA pro- standards medicines improved remarkably with cesses in place. It found that in ■ Organizing the training events cost recovery, its impact on the use Tahoua, where department and of services warrants caution.

B. Local Implementation Figure 2 The evaluation did not directly Utilization of Services in Konni District demonstrate that savings resulted from introducing IMCI in a QA (1997 and 1998; January 1 to June 30) environment. However, the evalua- tion team does believe that greater New Patients Under 17828 compliance to IMCI standards was a Five Years 11943 direct result of QA and that QA Infant 17964 facilities should reap savings Consultation 15457 through a reduced use of medicines and personnel time. Also, better- Prenatal 8160 1997 quality child care standards were in Consultation 6935 1998

7 WHO has devised “generic” IMCI standards; they are adapted to fit the situation of each country where they are implemented.

10 ■ The Niger QAP/BASICS Joint Project (TRS) district supervisory teams were Table 4 trained in both QA and IMCI, Results of RAHWPs supervisors are able to provide technical support to primary (1997 and 1998; in Percentages) healthcare workers. The team is concerned that IMCI quality could Konni Illéla erode if QA-trained supervisors are Indicators Oct. 97 June 98 Oct. 97 June 98 transferred out of QA districts and n = 78 n = 41 n = 81 n = 32 into those where QA has not been implemented. Because the evalua- Children checked for general danger signs 49 25 51 63 tion occurred just three months after Children correctly examined 54 28 19 49 IMCI was introduced at the health center level, the team could only Children correctly treated 81 83 77 78 tentatively assess the impact QA Mothers adequately counseled for the treatment 52 73 50 69 was having on IMCI’s effectiveness. of their sick child The following summarizes some of the findings. Mothers counseled for signs that merit bringing 75 83 33 50 their child back Children having their nutritional status checked 93 45 6 16 1. Supervision The supervision checklist discussed Mothers who knew at least two general danger signs 43 83 79 53 with the first TAP objective evolved Mothers who know how to give the prescribed 73 83 40 80 with the introduction of IMCI to medicine to their sick child ensure adequate monitoring of health workers’ IMCI skills. (The sections of the checklist that monitor maintenance of essential medicines/ second time this tool was used, a resources (e.g., breaks in the cold drugs and compliance with the cost higher percentage of mothers chain caused by lack of refrigerator recovery system were also updated received adequate counseling for fuel) nor detect defects in staff to conform to changes in those treating their children. On the other clinical skills, although it can programs.) hand, some indicators either did not continuously improve efforts to help improve or declined between the solve these kinds of problems. The rapid assessment of health first and second use (Table 4). worker performance (also discussed with the first TAP objective) focuses 3. Findings mainly on IMCI and is conducted 2. Quality Improvement While the evaluation team recog- through observation of health All IMCI-trained staff in Konni and nizes that IMCI had just been workers while they examine young Boboye had also had QA training, implemented in Niger, it did release children. It also guides supervisors including problem solving. The team the following tentative findings: in interviewing mothers to assess discovered that this training enabled their satisfaction with services and ■ Trained staff are capable of staff to recognize problems with the effectiveness of health worker applying the IMCI clinical malnourished children and monitor communications. The evaluation guidelines to examine and treat their progress. This resulted in team found the RAHWP valuable for children, and they can check impressive improvements in the monitoring performance and children’s vaccination and indicators measuring dropout rates generating data to improve services. nutrition status. in nutrition care. Still, serious The team also saw that feedback concerns remain: QA can neither ■ Many of the items necessary for from the assessment can foster solve problems resulting from lack of IMCI to be successful were improved worker performance. The

The Niger QAP/BASICS Joint Project (TRS) ■ 11 missing because of financial as effective referral centers supervisory activities and quarterly restraints: no health center visited because they lack needed meetings, the RAHWP, and measur- had a respiratory rate counter; equipment, such as an oxygen ing client satisfaction. Boboye had no IMCI forms; and mask (Konni). The QAP staff participated in several the urban health center in Konni ■ Some mothers received international and regional confer- had no baby scale. insufficient instruction in using ences, study tours, visits, and ■ Health workers have to complete prescriptions. projects, including the International five different records for each sick Society for Quality in Health Care ■ Cold chain problems and failures child: the IMCI forms, the National (ISQua) conferences from 1993 to to provide vaccinations are Health Information System tally 1995, projects in Burkina Faso, undermining IMCI’s effectiveness. notebook, the Daily Register Cameroon, and Togo, a visit by an Book, the Cost Recovery Regis- Ivory Coast delegation to Tahoua, a ter, and the Card of Care. This workshop and training in Rwanda, a results in inefficiencies. D. Dissemination of visit by African experts to Brazzaville Information on the Nigerien to define national QA strategies and ■ Health workers do not know what QAP programs, and a QA workshop in to do with the completed IMCI Niamey for French- and Portuguese- forms, undermining the program’s Before the QAP/BASICS merger, the speaking countries. effectiveness in monitoring trends Nigerien QAP disseminated informa- in child health. tion on its progress and experience, The QAP staff also participated in a reaching health providers and one-week study tour on QAP/IMCI in ■ Some completed IMCI forms managers both inside and outside Mali and collaborated with the indicate that staff fail to complete the country. At a QA conference in BASICS regional office in Senegal. the nutrition section and/or to December 1995 in Tahoua with over All of these activities contributed to refer children with malnutrition. 200 people in attendance, present- the broadcast of information on the ■ Some health staff need closer ers from all over the country dis- Nigerien QAP experience. The supervision to complete the IMCI cussed results from the problem- evaluation team found that case forms correctly. solving teams. Two-and-a-half years studies, lessons learned, develop- later, the MPH visited all the centers ment of standards, procedural ■ Only 11 percent of expected involved, which resulted first in a improvements, activity analyses, and follow-up visits actually occurred; statement of intent to create a all other QA documentation can be caretakers may not be able to national QA program, and ultimately duplicated in Niger and other comply with IMCI standards that in a national QA policy (this had not African Francophone countries encourage their return for a been implemented at the time of the follow-up visit. evaluation because of resource ■ A similar discrepancy exists restraints). In addition, through IV. Cost Analysis of the between children who exhibited a quarterly meetings in Tahoua, staff in general danger sign and should regions beyond the pilot areas have Niger QAP/BASICS have been referred to a hospital begun to learn about and generate Project and those who actually were. In demand for QA. The evaluation team analyzed the one set of records, only 9 percent In October 1998, which was after the of children with a general danger operational costs of implementing merger and just prior to the evalua- and maintaining essential program sign were referred. Either staff tion, the international conference in failed to record the referrals or activities. The analysis focused on Niamey attracted 200 participants, costs incurred at the facility, district, they are reluctant to refer cases, many from Niger and the surround- knowing that it is hard for caretak- and department levels, relying ing region. Presentations addressed mostly on historical data from QAP/ ers to go to the hospital because the basic concepts of the QA of transportation difficulties. BASICS. They did not measure the approach, the child survival full cost of technical assistance or ■ The Konni and Boboye District approach, QA rationale, the project-level costs and overhead, so Hospitals are not prepared to act problem-solving experience, IMCI,

12 ■ The Niger QAP/BASICS Joint Project (TRS) estimates do not reflect the alloca- Figure 3 tion of indirect costs related to the Breakdown of QA Training Costs administrative infrastructure. Also, government-set per diems were high compared to devaluating salaries, Per Diem (54%) so personnel costs are higher for activities that involve a per diem Honorarium (training, supervision, and coaching) Local Technical Assistance (5%) than they are for salary-based (7%) activities (problem solving). The Transport Refreshments Training Kits & Supplies (10%) (19%) former activities involve travel and (5%) consequently require per diems. In Based on four training sessions held between February and April, 1998 addition, several factors limit the application of these figures to Niger: government policies, the high cost diems for participants and trainers, future. Such training is provided to of fuel in Niger, and the fact that honoraria for trainers, transportation, selected individuals after they spending was influenced by a supplies, and refreshments. IMCI complete the basic QA training. The budget. Lastly, the team was unable training also included accommoda- evaluation found that in Niger the to obtain sufficient data to link the tions and site rental. The team cost ranged from $200 to $260 per costs of these activities with their determined that it cost about $230 to trainee. effectiveness. train one provider in basic QA and $430 in IMCI (based on seven days A. Sustainability Costs of QA training and 11 of IMCI D. Team-Based Problem training). More than 50 percent of Solving The evaluation determined that a the training costs was for per diems; department with the size and Figure 3 and Figure 4 show the Costs from team-based problem structure of Tahoua would need to percentages of the costs of the solving accrue mostly from meetings invest about 3.1 cents per capita per various components of QA and IMCI and collecting data, and the atten- year to sustain essential QA activi- training. dant administrative costs of these ties of training, supervision, coach- activities. Problem-solving teams in ing, meetings, and IMCI. This figure Niger typically have three to eight is the equivalent of 7.7 cents per C. Training of Trainers members. Meetings average one to person within the catchment area of two hours per week, and cycles Training of trainers in QA helps a QA health center. The projected average three months. Costs varied sustain program goals well into the level of investment capitalizes on from $78 to $130. the other QA activities built into the design of the program, including the development of standards and Figure 4 quality monitoring. Breakdown of IMCI Training Costs

B. Basic QA and IMCI Per Diem & Honorarium Training Costs (56%) Other Materials The evaluation team concluded that Accomodations (2%) training, accounting for nearly 60 (4%) Hall Hire percent of total costs, constituted Training Kits & Supplies (10%) Transport the most significant cost category of (7%) (21%) all the essential QAP/BASICS activities. Training costs include per Based on two training sessions held between August and September, 1998

The Niger QAP/BASICS Joint Project (TRS) ■ 13 Data collection and supply costs tion of health centers in a district, V. Lessons Learned had significant cost components of but ranged from $20 to $35 per visit. per diems to data collectors (ap- The presence of both the QAP and proximately 30 percent), transporta- BASICS in Niger presented a unique tion or fuel costs (23 percent), and F. Supervision Checklist opportunity for USAID and its markers (20 percent). Survey forms, partners to merge the two projects, The evaluation team estimated the flip charts, and a record book improving IMCI’s effectiveness by cost of replicating the checklist, accounted for just over 25 percent. introducing it into a QA environment. which would entail buying a binder, The evaluation team noted that In addition to the obvious benefit of copying the checklist, and minimal these estimates are approximate reducing operational costs—which training in using the checklist. This and would vary and that “safety the evaluation team believes to be breaks down to approximately $14 stocking” (where teams request considerable though per center to purchase a binder, $20 more supplies than necessary) unquantifiable—by combining the per year per center to make copies tends to inflate costs. projects, Niger benefited from the of the checklist, and $33–42 per synergistic effect of supporting IMCI supervisor for training. with quality assurance. For West E. Supervision and Africa and the international commu- Coaching Activities nity, it demonstrated a new ap- G. Quality Meetings proach for introducing IMCI, an Having been instigated by the QAP, The team estimated meeting costs approach that starts with strengthen- supervision and coaching are new and found that per diems, transpor- ing health support systems well costs to the Nigerien health system. tation, and refreshments account for before IMCI is introduced. The evaluation tried to capture all substantial portions. The QAP/ In selected areas, QA interventions the costs associated with these BASICS project set a cap on the were conducted at least a year activities. The team notes that under- amount that would be reimbursed for before IMCI training began and or over-reporting of the number of meetings, which probably influenced resulted in a significant improvement coaching visits may have occurred the amount spent on them. because of idiosyncrasies in the in supervision and the availability of reporting process. District meetings are typically held essential drugs (also facilitated by quarterly, attended by 18 staff, and cost recovery). The project suc- Coaching activities incur costs when last two days; this costs about $200 ceeded in creating both a QA coaches visit health centers to per quarter. Department meetings supervisory system and trained participate in problem solving. are also usually quarterly; attract 24 supervisory teams to provide Typically, two coaches would visit individuals from the districts, technical support to healthcare staff. two centers a day (each center is department office, and MPH; and Peer coaching helped transform the scheduled for a monthly visit). In last three days; they total $3000 per traditional authoritarian style of addition to the per diem paid to quarter. Quality Council meetings supervision into a supportive, each coach are the costs of a driver are bimonthly and attract 24 people problem-solving approach. and gas. Two districts reported from a single area; they total $120 Between 1993 and 1998, the project having completed the coaching per meeting. schedule and served as the basis trained 400 healthcare workers in for estimating costs. Averages problem solving, resulting in the ranged from $15 to $20 per visit, formation of over 77 problem-solving with per diems accounting for 5–15 teams. About 120 problem-solving percent and gas 70–80 percent. cycles were ongoing or had been completed at the time of the evalua- All scheduled supervisory visits tion, improving the delivery of were reportedly conducted in four prenatal and postnatal care, infant districts (Tahoua, Konni, Illéla, and consultations, family planning, Madaoua). Estimates vary by immunization coverage, and district, depending on the distribu- nutritional rehabilitation.

14 ■ The Niger QAP/BASICS Joint Project (TRS) The project also introduced a quality healthcare. Problem solving different records for each child. number of training tools that contrib- has had a significant, positive Integrating these forms could lead to uted to the support of IMCI. The impact on the staff’s ability to better health worker performance. program framework provided improve the quality of services at the While healthcare providers are now indicators and offered a new health center and district levels. capable of checking a child’s environment for managing The project also introduced tools vaccination status and have inte- healthcare organizations. Healthcare that contributed to the support of grated this service into daily workers changed their behavior, IMCI. The RAHWP proved valuable activities, problems in maintaining improved their performance, not only in monitoring health worker the cold chain are limiting the developed a sense of teamwork, performance, but also in improving it delivery of effective vaccinations. and experienced increased motiva- through timely feedback. Also, the This has a twofold downside. First, tion. supervision checklist is a helpful, children are not being vaccinated The pilot of the combined QAP/IMCI comprehensive tool, albeit too long. because health workers realize that approach demonstrates that a break in the cold chain has Because of problem-solving training healthcare support systems can be spoiled the vaccines, or workers are and practice, health providers can strengthened well before IMCI providing vaccinations without identify and solve problems. They training begins. This combined QAP/ knowing that the vaccines have now have the skills necessary to IMCI approach is efficient and can deteriorated. Either way, the popula- collect, analyze, and use data; if be used as a benchmark for other tion is unprotected. Second, IMCI sustained, this will have a long-term Francophone African countries records are incorrect, because staff impact on healthcare services in considering a launch of IMCI. don’t realize that some vaccines Niger. The problems identified have spoiled, and they are unknow- usually addressed issues connected ingly reporting ineffective vaccina- to integration of BPS-related activi- tions. An investment in maintenance A. For Niger ties, such as immunization. Using and new equipment could solve The merger of the QAP and BASICS data collected by the problem- these problems. in Niger created a new way to solving teams themselves rather implement IMCI that starts with than from the national data system Results show that health workers interventions—such as the supervi- would likely expand problem-solving and caretakers are having difficulty sory checklist and drug stock activities beyond the BPS. complying with IMCI standards management—to improve the quality regarding referral of very sick Improved stocking of essential of healthcare. These interventions children and the need to return to drugs/medicines, boosted by cost support IMCI, improving its viability the health facility for follow-up visits. recovery, resulted in lower cost, and effectiveness. Previously, IMCI This difficulty, caused by Niger’s more reliably available drugs/ had never been introduced with dispersed population, presents a medicines, improved decision- such support. In selected areas of challenge not only to the health making on the part of consumers, Niger, QA interventions had been delivery system but also to IMCI and ultimately, greater continuity of implemented a year or more before standards. Further adaptation may care and integration of services. IMCI training began and resulted in be needed to adjust the require- significant improvements. The evaluation team learned that ments for referral and follow-up visits several essential items were missing for this population. The joint project brought many from some health centers, including improvements to the healthcare The team discovered some troubling IMCI forms, respiratory rate system in the target areas. First, it aspects of healthcare service counters, and baby scales. If created trained supervisory teams delivery in the target areas. The allowed to continue, this will have a that can provide technical support to district hospitals in Konni and detrimental impact on IMCI’s health facility staff. The concept of Boboye are ineffective referral effectiveness. coaching facilitated the develop- centers because equipment and ment of a supportive, team-based In addition to the IMCI form, health trained staff are lacking. This means approach to problem solving and workers have to complete four that children are not receiving

The Niger QAP/BASICS Joint Project (TRS) ■ 15 appropriate care and that the improved the quality of healthcare The sometimes dramatic improve- facilities cannot follow the IMCI and services, although progress ment in coverage and utilization protocols. Furthermore, exit inter- varies. The greatest weakness of the rates—along with user satisfaction— views with mothers at these hospi- teams is their problem analysis, following problem solving shows that tals revealed that health providers specifically data collection and problems are not always caused by should improve their communica- interpretation. a lack of means, but sometimes by tions regarding prescriptions and poor service organization or man- Quality teams have greatest impact nutrition. agement problems. when they are supported by: (a) a supervision and coaching system Cost recovery had a positive impact that the district manages, and (b) a on the QA approach and IMCI. B. For the International district coordination structure like Indeed, low-cost essential generic Health Community the Quality Council. drug availability stimulated continu- ity of care and activity integration. The project enabled Niger to play a Discussion opportunities within the After a slight decrease when cost leadership role in demonstrating an problem-solving cycle generated recovery began, utilization rates effective new way to introduce IMCI. favorable group dynamics and increased significantly in most Healthcare workers from many West teamwork. Experience sharing is an cases. African countries have participated indispensable part of the system, in IMCI training in Niger, and Niger’s and the emergence of teamwork will experience in QA and IMCI was likely improve healthcare services in presented at a number of regional the target areas for a long time to C. Main Factors Limiting conferences and meetings. Consid- come. Information sharing by QA ering the large number of trained coaches helps capture successful The evaluation team identified agents and trainers, prepared ideas from the health centers. Those several factors that will limit the materials, existing documentation, ideas can then be disseminated to success of QA unless improvements and the success of the international other facilities that might have are made, including: conference in October 1998, the similar problems. project’s replication in Francophone ■ Personnel mobility and motivation Africa should be rapid. Further, the The integration of BPS activities is a ■ Old refrigerators, maintenance project’s benefits could spread to good example of process redesign problems, gas shortages, other continents. The supervision that all interviewed healthcare unreliable car/motorbike pools system’s launch with the RAHWP, workers noted as a product of QA supervision checklist, and coaching efforts. An illustration of this is the ■ Inaccurate and/or out-of-date methods were important experi- new practice of providing immuniza- demographic data ments that could benefit other QA tion every day at all the health ■ Access to healthcare limited by projects in Africa, Latin America, centers the evaluation team visited. the dispersed nature of the and Asia. Problem-solving cycles average six population Several aspects of the joint project months in length and range from ■ Confusion of health workers experience should serve as guide- three months to a year. Regular caused by too many sets of posts for those who will implement coaching, if performed systemati- clinical standards similar endeavors in the future. First cally—as is the case at the health and foremost, the evaluation centers in the target areas— ■ Unmet salary payments for health determined that the QA approach shortens this cycle. workers, which were often helped improve the effectiveness of Community participation varies, compensated with QAP per diem IMCI by creating favorable condi- depending on the health center. payments for supervision, tions, such as activity integration, a Since commitment from the commu- coaching, and training activities referral system, and regular follow- nity leads to longer-lasting solutions, ■ Time away from health centers to up of activities. the health centers should do more to attend training sessions Using the problem-solving process, integrate community involvement the quality teams have consistently into their QA activities.

16 ■ The Niger QAP/BASICS Joint Project (TRS) VI. Recommendations The departmental level of the department-level coaches to healthcare system should regularly continually update their knowledge. The following is a summary of update the supervisory system to Periodic meetings should be recommendations developed by the adapt to changes in IMCI and other scheduled at all levels on the evaluation team. healthcare programs. The depart- functioning of management teams. ments should promote the use of the In addition, quarterly meetings central supervisory checklist should continue so that healthcare (especially the patient interview A. Establishment of workers will have ongoing training, section) to reinforce their supervi- Standards motivation, and opportunities to sion of the districts. Semiannual evaluate themselves, share experi- The MPH should establish an official supervisory visits should include the ences, and set benchmarks. inventory of existing, consistent RAHWP. All supervisors should be standards on child healthcare and trained in coaching techniques. integrated health center manage- ment and release this inventory as a Supervisors should combine D. Problem Solving supervisory and coaching visits in reference. It should also ensure that Teams at the national, departmental, order to save time and money and the new IMCI standards effectively and district levels should conduct promote optimal supervision. replace the previous childcare problem-solving cycles regularly to Healthcare workers’ performance management standards. improve support systems, retain should be monitored closely to personnel, and prepare to learn The MPH should create standards to ensure that they complete forms and other, higher-level QA techniques, improve the referral system and provide therapeutic and nutritional such as quality design. Districts strengthen district hospitals so that advice. Supervisors should provide should systematically encourage they will be effective centers for workers with continuous feedback community participation within the treating referral patients. on their performance assessment problem-solving teams. The MPH should also expand the results as quickly as possible to training of managers and healthcare enable improved performance. workers in drug stock management. The supervisory system should E. Sustainability It should also improve procurement create and support a follow-up The MPH should support the rapid (especially for IMCI forms, chronom- system for requests and suggested implementation of a QA curriculum eters, and baby scales) and the solutions. A focal point for such in training schools and support and functionality of the cold chain by requests and suggestions from each distribute QA documents, including repairing or replacing old or dam- health center should be created at a QA skills and methodologies aged refrigerators. the departmental and district levels. guide. The archival system for completed IMCI forms should be improved at The MPH should investigate meth- the district level to facilitate data C. Coaching and Quarterly ods to use cost recovery to finance analysis and service evaluation. Meetings essential QA activities. It should also explain and show the QA cost/ Personnel transfers of QA-trained Coaches should receive regular, activity system to various sponsors. staff should be encouraged only ongoing QA training. They should be Other healthcare partners in each within areas where QA is in place. trained in data collection, process- district should be made aware of ing, and interpretation techniques QAP/IMCI activities. (given the importance of these skills B. Supervision and in monitoring activities). Monthly On a departmental level, the MPH Monitoring of Standards coaching activities should be should establish a transportation conducted at the department and system—including cars in working The MPH should incorporate the district levels when new problem- condition, available spare parts, and quality indicators used in the solving cycles occur. Guidance trained mechanics—to ensure the RAHWP into the national data should be provided to enable viability of key QA activities (e.g., system.

The Niger QAP/BASICS Joint Project (TRS) ■ 17 coaching, supervision, or transfers F. Dissemination and Tahoua and Boboye. Supervisors required by IMCI). Documentation should train departmental and district supervision teams in opera- Districts should encourage local Each level of the healthcare system tional research. Additionally, service- participation in covering costs for should regularly coordinate opera- provider satisfaction has not been fundamental QA activities (e.g., fuel tional research activities to dissemi- documented and should be pursued and food expenses, supplies for nate results achieved. Regular in further QAP/IMCI studies. quarterly meetings, coaching and forums to share QA experiences supervisory activities, and transpor- should be established. A sample tation). A plan should be developed group should be created to do to transfer responsibility for the operations research using the QAP/ project to other healthcare partners. IMCI experiences and progress in

18 ■ The Niger QAP/BASICS Joint Project (TRS)