PERU CHILD SURVIAL XVI Departments of Cajamarca & La Libertad REDESS FAO-A-00-00-00030-00 October 1, 2000 - September 30, 2004 FINAL EVALUATION December 2004 Consultant: Renee Charleston CARE HQ Contact: CARE Peru Contact: Joan Jennings, MPH Carlos Cardenas Senior Technical Advisor: Child Health Country Director CARE USA CARE Peru 151 Ellis Street NE Aparto 11-0628 Atlanta, GA 30303 Lima 11, Peru Tel: 404-979-9413 Tel: 51-14-317430 Email: [email protected] Email:[email protected] ACRONYMS ANA Analysis of Learning Needs (Analisis de Necesidades de Aprendizaje) AOP Annual Operating Plan (POA in Spanish) ASOACS CHA Association ARI Acute Respiratory Infection BCC Behavior Change Communication CDD Control of Diarrheal Disease CHA Community Health Agent (includes promoters, TBAs, women leaders) CHP Community Health Promoter (Volunteer) COACS CHA Committee CODECO Community Development Committee CS Child Survival CSHGP Child Survival and Health Grant Program DCM Diarrheal Case Management DIP Detailed Implementation Plan ENLACE Previous CARE Peru CS project 1996-2000. FE Final Evaluation FOGEL Facilitating Local Government Participation in Health HC Health Center HH/C IMCI Household/Community Integrated Management of Childhood Illnesses HIS Health Information System HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficient Syndrome HQ CARE Headquarters in Atlanta IEC Information, Education, Communication IMCI Integrated Management of Childhood Illnesses KPC Knowledge, Practice, and Coverage Survey LQAS Lot Quality Assurance Sampling MNC Maternal Newborn Care MOH Ministry of Health (MINSA) MTE Mid Term Evaluation NGO Non-Governmental Organization ORS Oral Rehydration Solution PCM Pneumonia Case Management PVO Private Voluntary Organization SIVICS Community Health Information System TBA Traditional Birth Attendant TOT Training of Trainers TT Tetanus Toxoid USAID United States Agency for International Development WRA Women of Reproductive Age Table of Contents A. Summary . .1 B. Assessment of Results and Impact of the Program. 4 1. Results: Summary Chart . 4 2. Results: Technical Approach . 5 3. Results: Cross-cutting approaches . 15 a. Community Mobilization. 15 b. Coordination with Government Structures. 21 c. Communication for Behavior Change. 24 d. Capacity Building Approach. 28 i. Strengthening the PVO Organization. 29 ii. Strengthening Local Partner Organizations. 31 iii. Health Facilities Strengthening. 33 iv. Strengthening Health Worker Performance. 34 v. Training. 36 e. Sustainability Strategy. 39 C. Program Management. 42 1. Planning. 42 2. Staff Training. 42 3. Supervision of Program Staff. 42 4. Human Resources and Staff Management. 43 5. Financial Management . 43 6. Logistics. 44 7. Information Management. 44 8. Technical and Administrative Support. 48 9. Management Lessons Learned. 49 D. Conclusions and Lessons Learned. 50 E. Results Highlight . 53 ATTACHMENTS: A. Evaluation Team Members B. Evaluation Assessment Methodology C. List of persons interviewed and contacted D. Final KPC Report E. Project Data Sheet A. Summary CARE Peru received a Child Survival XVI grant to implement a four year project (10/00-9/04) in two northern Departments, La Libertad and Cajamarca. REDESS is a follow-on project to the previous ENLACE CS project 1996-2000. Many of the strategies which were proven effective during ENLACE were modified, improved, and implemented as part of REDESS. The replication of these interventions in a different geographic area was successfully accomplished. The underlying goal of these projects has gone beyond implementing technically sound health interventions to a broader scope of changing social structure-within communities and local governments. REDESS received a four month no-cost extension (through 1/05). A participatory Final Evaluation was conducted during 12/04 by a 14- person team of representatives from MOH central, regional and local offices, CARE Peru and headquarters, and an external evaluator. The project goal of improved maternal-child health results from reaching four intermediate results through a partnering relationship between CARE Peru, the Ministry of Health (MOH), Community Health Agents (CHAs), local governments and community organizations within a national process of Health Sector Reform and decentralization. These changes have focused on health promotion, increased local control of resources and decision making, and strengthening of strategic alliances among all stakeholders. CARE has positioned itself to be ahead of these changes, developing models for local control and networks before legislation has mandated them. The institutionalization of interventions is a key element of sustainability. IR 1 Improved coverage and quality of services provided by health networks REDESS influenced both the demand for services and the supply of quality services. The focus was not on improving the technical capacity of the MOH, rather on improving community outreach and management skills. IR 2 Increased caretaker knowledge and practice REDESS did not reach many preset targets for changing knowledge, in part because indicators were complex and measured multiple answers. The project did make significant changes in practices, for example: Ø Increased from 48% to 78% mothers who seek appropriate medical treatment for children age 0 to 23 months with signs of pneumonia Ø Increased from 55% to 84% women with an obstetrical complication who are treated by a health professional Ø Increased from 56% to 81% women with 4 or more prenatal care visits 1 Ø Increased from 33% to 83% children 0 - 6 months exclusively breastfed The approach utilized to stimulate behavior change through campaigns using a variety of methods and media, support groups, and education sessions was an effective way to improve the health of the population. IR 3 Improved quality and coverage of care provided by CHAs. The implementation by CHAs of the referral system, community support groups, and emergency evacuation plan have increased demand for services, linkages between Health Centers and communities have been strengthened by improving MOH staff skills and providing opportunities for coordination. IR 4 Strengthened relationships between CHA associations, governments and civil society. The emphasis was to strengthen the structure within which technical interventions are implemented. This structure consists of CHAs, CHA and community organizations, MOH staff, local governments, and consensus building committees. CHA organizations have gained the skills necessary for self-management and for collaborating with the MOH and other community organizations. Community committees have given people a voice and mechanism for reaching consensus within an organized community. REDESS has been able to successfully meet all intermediate results, or make substantial progress in completing them. The strength of this project was not the technical focus, (divided between maternal newborn care 40% and 60% integrated IMCI (nutrition 30%, pneumonia case management 15%, control of diarrheal disease 15%) but on the cross cutting approaches which were implemented. The first two strategies are expected to be adopted by program partners for wider scale applicability: s Community Health Information System (SIVICS) including community mapping of vulnerable groups, census, and referral system,is a key tool for strengthening the link between communities and the health sector. s Structure, organization, and operational skills of the CHA organizations s Strengthening of community development committees, CHA organizations, and local government structures and the linkages united them. s Health facility strengthening through improved organization and training of health staff, including IMCI training Sustainability can be viewed based on four principles- permanent behavior change- KPC survey results show an increase in knowledge, practices, and coverage; supportive structures- REDESS has woven together a structure to support health activities; links with permanent institutions-The strength 2 of this project has been the focus in this area that is very much in step with current national political restructuring, and financial support- REDESS has encouraged financial sustainability at various levels. The inclusion of partners and communities throughout the entire process foments excellent community ownership and pride in the work accomplished. The project took on the very difficult task of weaving together a structure capable of sustaining improved health practices and services. The improvements seen to date are the results of the sum of training at various levels. The fabric that has been woven is still very fragile, but the project has been able to successfully improve capacity, leaving human resources capable of continuing, and improving upon, health activities. REDESS has left excellent models for decentralization and increased citizen participation, but these models need to be systematized and replicated in other areas, including within other sectors as a model for grassroots development, CARE Peru and partners are actively seeking funding to support scaling up. Key lessons learned during project implementation include: s Information collection is not a valid activity unless that information is used for decision making. All projects need to create a culture of information use. s Implement principal activities during the first two years of the project, and use the last two tears for monitoring and supervision s Documentation and marketing of successful models is a long