4.2.1.1. Knowledge of Contraceptive Methods 42 4.2.1.2

Lists of tables III Lists of Figures V ACRONYMS VI Acknowledgment VII Evaluators VIII Executive summary IX

I. INTRODUCTION 1 1.1. Background 1 1.2. Context of RH/FP in Oromia Region 4 1.3. Program Description 6 1.3.1. Objective of the Project 6 1. General Objective: 6 2. Specific Objectives of ODA program: 6 1.3.2. Program Components 7 1.4. Justification of the Evaluation 9 1.5. Stakeholder Analysis 9

II. OBJECTIVE OF THE EVALUATION 11 2.1 General Objective 11 2.2. Specific objectives of the evaluation 11 2.3 Evaluation Questions 11

III. EVALUATION METHODOLOGY 15 3.1. Study area and Period 15 3.2 Evaluation Design and Data collection Methods 15 3.3. Sample size and sampling technique 16 3.4. Data management and analysis Limitations of the study 18 3.5. Ethical considerations 19

IV. RESULTS AND DISCUSSIONS 21 4.1. Degree of Program Implementation 21 4.1.2. Level of stakeholders’ involvement 22 4.1.4. Supervisory Support 24 4.1.5. Capacity Building Activities 24 4.1.6. Role of CBRHAs and their acceptance by the community 26 4.1.7. Information, Education, and Communication (IEC) 29

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) II

4.1.7.2. Provisions of Information and Education 30 4.1.7.3. Exposure to Family Planning messages 31 4.1.7.4. Organize/Supporting in and out-of-school youth clubs 32 4.1.8. Provisions of Contraceptives 33 4.1.8.1. Amount of contraceptive methods and Couple-year protection (CYP) generated 33 4.1.8.2. New contraceptive users 34 4.1.8.3. Contraceptive Acceptance Rate: 36 4.1.8.4. Perceived Quality of services and Client Satisfaction 37 4.1.8.5. Informed choice 40 4.2. Outcome of the project 40 4.2. 2. Family Planning 42 4.2.1.1. Knowledge of contraceptive methods 42 4.2.1.2. Use of contraceptive methods 44 4.2. 2. Age at first marriage 51 4.2.3. Utilization of Maternal Health services 51 4.2.3.1. Antenatal care 52 4.2.3.2. Delivery care 54 4.2.3.3. Postnatal Care (PNC) 56

V. CONCLUSION AND RECOMMENDATION 63 5.1. Conclusion 63 5.2. Recommendation 65

VI. REFERENCES 69

VII. ANNEXES 73 Annex I. List of documents received from ODA RH/FP project coordinating office and used for document review. 73 Annex II. Ranking of Woredas Based on their Performance 76 Annex III. Discussion guide used to facilitate of Focus Group Discussions with CBRHAs 78 Annex IV. Discussion guide for Focus Group Discussion with Adolescents 81 Annex V. Guide for Expert Interview – Health Offices 83 Annex VI. Guide for Expert Interview – ODA Coordinators 87 Annex VII. In-depth interview guide for health workers 90

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) III

LISTS OF TABLES

Table 1 List of sampled woredas per zone and distribution of samples to zones and woredas, June 2009. 17

Table 2 Planned vs. Achieved selected Capacity Building Activities goals, 2000-2008, Ethiopia, 2009. 26

Table 3 Percentage distribution of Availability of CBRHAs and type of service they provide, by project zones, Ethiopia, 2009 . 27

Table 4 IEC materials distributed to support IEC activity, 2000-2008, Ethiopia, 2009. 29

Table 5 Total number of encounters with people on RH through IEC Activities by project zones and year; 2000-2008, Ethiopia, 2009.. 30

Table 6 Total number of Contraceptive methods distributed and CYP Generated by Method and project zone; 2000-2008, Ethiopia, 2009.. 34

Table 7 Cumulative Number of New Contraceptive users by method and year; 2000-2008, Ethiopia, 2009. 35

Table 8 Information related to Perceived quality of services and client satisfaction towards RH/FP service provided, Ethiopia, 2009. 38

Table 9 Socio-demographic and socioeconomic characteristics of participants, Ethiopia, 2009.: 41

Table 10 Percentage of all women and currently married women who know any contraceptive method, by specific method, Ethiopia, 2009. 43

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) IV

Table 11 Percentage distribution of married women by type of contraceptive methods used in the last five years, Ethiopia, 2009. 46

Table 12 Percentage of all and currently married women currently using contraceptive method by specific method, age, zone, and level of education, Ethiopia, 2009. 47

Table 13 Reasons for discontinuation of contraceptives and Future Intention of contraceptive use, Ethiopia, 2009. 51

Table 14 Distribution of women who had a live birth in the past five years and received ANC by provider during pregnancy for the most recent birth, Ethiopia, 2009. 52

Table 15 Percent of women who had a live birth in the five years by the timing of the first visit and number of ANC visits for the most recent birth, Ethiopia, 2009. 53

Table 16 Comparison of Women Receiving ANC and Number of visits made during EDHS 2005, EGFHIE 2008, and current Evaluation, Ethiopia, 2009 54

Table 17 Percent of place of delivery and person providing assistance during delivery of live births in the five years by zone and level of education, Ethiopia, 2009. 55

Table 18 Percent of mothers delivered in the last five years who received PNC and time after delivery the first PNC received for the last live birth, Ethiopia, 2009. 57

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) V

LISTS OF FIGURES

Figure 1. An Evaluation framework used to assess the implementation of the program and the achievement of operational outcomes, Ethiopia, 2009. 15

Figure 2. Number of project Woredas by zone and year, 2000-2008, Ethiopia, 2009. 22

Figure 3. Percentage distribution of source of FP information among women of reproductive age group, Ethiopia, 2009 31

Figure 4. Trends of new contraceptive users by method and year; 2000-2008, Ethiopia, 2009. 36

Figure 5. Baseline, plan and achievement of Contraceptive Acceptance rate by project zone; 2000-2008, Ethiopia, 2009 37

Figure 6. Percent of respondents who indicated the project has helped them or their family by zones, Ethiopia, 2009 . 39

Figure 7. Percentage distribution of ever use of contraceptive methods among all women and currently married woman by zone, Ethiopia, 2009. 44

Figure 8. Percentage of all women and currently married women who have used any contraceptive method in the last five years by zones; Ethiopia, 2009. 45

Figure 9. Trends in Current Use of Contraception by type of methods used, 1999-2009, Ethiopia, 2009. 48

Figure 10. Percentage distribution of sources of contraceptives among current users, Ethiopia, 2009. 49

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) VI

ACRONYMS

AIDS Acquired Immunodeficiency Syndrome ANC Antenatal care ARHS Adolescent Reproductive Health Service CBD Community-based distribution CBRHA Community Based Reproductive Health Agent CPR Contraceptive Prevalence Rate CSA Central Statistical Agency CYP Couple-year protection DHS Demographic and Health Survey EDHS Ethiopian Demographic and Health Survey EGFHIE Ethiopian Global Fund Health Impact Evaluation FGD Focus Group Discussion FP Family Planning HEWs Health Extension Workers HIV Human Immunodeficiency Virus HTP Harmful Traditional Practice ICPD International Conference on Population and Development IDI In-depth Interview IEC Information, Education and Communication IUD Intra Uterine Devise LAM Lactational Amenorrhea Method MCH Maternal and Child Health NGOs Non-Governmental Organizations ODA Oromia Development Association OCP Oral Contraceptive Pills PNC Postnatal care REB Regional Education Bureau RH Reproductive Health RHB Regional Health Bureau SRH Sexual and Reproductive Health STDs Sexually Transmitted Diseases STIs Sexually Transmitted Infections TFR Total Fertility Rate TOT Training of Trainers TTBA Trained Traditional Birth Attendant UN United Nations VSC Voluntary Surgical Contraception WoHO Woreda Health Office ZHD Zonal Health Department

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) VII

ACKNOWLEDGMENT

The Evaluation Team gratefully acknowledges the David & Lucile Packard Foundation and Oromia Development Association RH project for supporting this evaluation. Particularly our appreciation goes to Ato Sahlu Haile, W/ro Yemisrach Belayneh from David & Lucile Packard Foundation and Dr Mulugeta Hawas manager of ODA RH project previous.

The Evaluation Team wishes to thankfully acknowledge the Zonal, and Woreda ODA RH project offices and staffs that participated in the evaluation process and sharing information and their views.

The Principal Investigator thanks Dr. Girma Azene, Dr. Carla Decotelli, W/t Dildile Yohannes, Ato Dimitri Tibebe and Ato Bisrat Birhanu for their invaluable assistance for the completion of the study.

Finally, we highly appreciate all the field coordinators, facilitators, and data collectors and, more importantly, the survey respondents, who were critical to the success of this evaluation.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) VIII

EVALUATORS

Dr. Wuleta Lemma: Principal Investigator, Assistant Professor Co-Director, Center for Global Health Equity School of Public Health and Tropical Medicine Tulane University 1440 Canal St. Ste 1191 New Orleans, LA 70112 Email: [email protected]

Moreira dos Santos, Elizabeth: Consultant Evaluator, Full Professor and Researcher at National School of Public Health/ FIOCRUZ, Rio de Janeiro, Brazil

Jemal Aliy (MD, MPH): Consultant Evaluator, (TUTAPE)

Dr. Girma Taye: Consultant Evaluator, (TUTAPE)

Mr. Yibeltal Kifle: Consultant Evaluator, Lecturer Jimma University, Ethiopia

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) IX

EXECUTIVE SUMMARY

Introduction: The Oromia Development Association, with a technical and financial support from the David & Lucile Packard Foundation, started the reproductive health/family planning project in selected zones of Oromia Regional state in the beginning of 2000. The major objective of the project is to increase access to reproductive health/family planning services, improve utilization of family planning services and to strengthen institutional capacity. The components of the Project are distribution of non-clinical contraceptives through community- based reproductive health agents (CBRHAs) linked by referral and backup support to nearby health facilities, provision of IEC on different RH issues to create awareness and bring about behavioral change, promotion of ARH in-and out of school, youth clubs (art, poetry, and sports), and peer-to-peer education, and capacity building of different levels of the health care system.

Objective: The objective of this evaluation is to assess the strengths, weaknesses and results of the Packard Foundation funded RH/FP project implemented by ODA in four zones of the Oromia National Regional State.

Methods: A mix of data collection and analysis methods are used to meet the overall objective of the evaluation. These entail review of records and reports of ODA for secondary data, collection of both qualitative and quantitative data using unstructured and structured questionnaires.

Result: The evaluation result shows high community involvements in all stages of the project, adequate linkages to health services for referral and clinical service, and availability of supervision. However the supervision lacks regularity, the training materials are not in local language, the training is of short duration and lacks adequate attention on counseling process, development and design of IEC lacks proper process. Nearly 100% of the planned health facilities are equipped, 80% of the planned health workers and 99% of the planned CBRHAs are trained and all planned youth clubs are supported. There is marked improvement in the knowledge and service utilization. Knowledge of at least one method of contraception is 91 percent for all women and 93 percent for currently married women. The current use of contraceptive method among currently married women is 38.9 percent. But majority of the current contraceptive users are using clinical contraceptives. Sixty five percent of mothers received antenatal care from health professionals, 13 percent of births were delivered in a health facility, and 23 percent received postnatal care.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) X

Conclusion: The evaluation found that the process of project implementation was in the proper way, the planned different capacity building activities of the health facilities and health workers, CBRHAs, and youth clubs were accomplished. This evaluation shows a significant increase in the knowledge and practice of family planning methods and utilization of maternal health services among women in the project areas. There is a significant increase in the ever use and current use of contraceptive methods among both all women and currently married women and there is a significant increase in ANC, delivery care, and PNC. However there is a shift of contraceptive acceptance from non-clinical contraceptive to the clinical contraceptive.

Recommendation: A regular and supportive supervision with proper feedback, development of training materials in local language, provision of training for adequate period with adequate attention on counseling process, proper design and development of IEC materials with special emphasis to condom promotion are recommended. Further, the shift of contraceptive acceptance from non-clinical contraceptive to the clinical contraceptive and the initiative of the government to expand community based health services through HEWs shows that distribution of non-clinical contraceptives by CBRHAs is less feasible. Building a stronger bond between HEWs and CBRHAs, using the CBRHAs as model families and expanding their role on health promotion and community education are recommended.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) I

INTRODUCTION 1

I. INTRODUCTION 1.1. BACKGROUND

Globally there is a wide disparity between developed and developing countries in terms of general health and reproductive health service and the health outcomes. Women in developing countries have less access to reproductive health services such as antenatal care (ANC), delivery care, postnatal care (PNC), and family planning (FP) and are less likely to seek out and utilize the available health services to better meet their own reproductive health goals including the that of safe motherhood (1).

The poor health outcomes of women in developing nations is related to different socio-economic factors (low status of women in the household, limited education, unemployment), socio-cultural factors (early marriage, marriage by abduction, female genital cutting), and the lack of health care services. Lack of access to sexual and reproductive health (SRH) services and information contributes to high levels of morbidity and mortality for largely preventable SRH problems. Restrictions on information about sexuality, contraception, prevention and healthcare, limit people’s ability to make choices regarding their own sexual and reproductive health and rights (2).

In 1994, the UN International Conference on Population and Development (ICPD) in Cairo placed Sexual and Reproductive Health and Rights (SRHR) and gender equity at the center of the international agenda. In September 2006, as a result of advocacy by international and national non-governmental organizations (NGOs), the United Nations (UN) General Assembly finally adopted the target of universal access to reproductive health (3, 4).

Reproductive health is defined as a state of complete physical, mental, and social well being, and not merely the absence of disease or infirmity, in all matters related to the reproductive system including its functions and processes and reproductive health care(3). Reproductive health consists of the following components:

• Quality family planning counseling, information, education, communication and services;

• Prenatal care, safe delivery and post natal care, including breast feeding;

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 2

• Prevention and treatment of infertility;

• Prevention and management of complications of unsafe abortion;

• Safe abortion services, where not against the law;

• Treatment of reproductive tract infections, sexually transmitted diseases and other conditions of the reproductive system;

• Information and counseling on human sexuality, responsible parenthood and sexual and reproductive health;

• Active discouragement of harmful practices, such as female genital cutting;

• Referral for additional services related to family planning, pregnancy, delivery and abortion complications, infertility, reproductive tract infections, sexually transmitted diseases and HIV/AIDS, and cancers of the reproductive system, including breast cancer.

Despite various national and international initiatives to improve maternal health, more than half a million women from developing countries die each year as a result of complications related to pregnancy and child birth [1,2]. With approximately 247,000 maternal deaths per year, sub-Saharan Africa shares nearly half of the toll despite accounting for less than 12 percent of the world population [5].

In most of sub-Saharan Africa (SSA), coverage of health service is low; the health system is built around curative services and is mostly concentrated in urban areas. Consequently, in most countries, less than half of the population, mostly those living in major urban centers, have access to a health facility.

It takes a rural woman in Africa a long time to weigh her reproductive options. She has to take into consideration not only her personal feelings but those of her husband and the extended family. Only after a lot of thinking and consultations is she able to finally decide to see a health professional. In order to do so, she must travel long distances and give up her household responsibilities for the day.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 3

In areas where the health service coverage and source of information for the reproductive health services is low and hence access to services is limited, especially in rural areas, evidence suggests that community-based initiatives have proven to be a good alternative service delivery mechanism. These initiatives have been used extensively and have worked successfully in the area of family planning. Community-based distribution (CBD) of family planning is one of the most common strategies used to reach populations with limited access to services, using non-professional local workers, sometimes volunteers, who live in or visit communities to provide services that a woman otherwise would have to travel to obtain [6]. CBD also involves sharing knowledge about the importance of family planning services and the proper use of family planning; this information may alter couples’ fertility intentions, a factor that influences the demand for family planning services [7]. Overall, the CBD strategy assumes that lack of convenient access to contraceptives represents a substantial barrier to meeting the needs of family planning users.

The 2007 Population and Housing Census results show that the population of Ethiopia grew at an average annual rate of 2.6 percent between 1994 and 2007. The highest annual growth rate for the period 1994-2007 is observed for Gambella Region (4.1 percent), followed by Benishangul-Gumuz (3 percent), SNNP and Oromia (2.9 percent). In 2007, the total population was 73.92 million, with 84 percent of the population living in rural areas (8).

A comparison of the total fertility rates (TFRs) calculated from the 2000 and 2005 Ethiopian Demographic and Health Surveys (DHS) shows little change for the country as a whole between 2000 and 2006 (5.5 births versus 5.4 births, respectively). The level of fertility is significantly lower in urban areas (TFR of 2.4) compared to rural areas (TFR of 6.0). Fertility is highest in the Oromia Region (6.2 births per woman) and lowest in Addis Ababa (1.4 births per woman) (9). On the other hand, the contraceptive prevalence rates (CPRs) increased between 2000 and 2005 from 8.1% to 14.7%,while the unmet needs for family planning during the same periods were reported as 36% and 34%, respectively (9, 10).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 4

As can be observed in reports from the World Health Organization (11, 12), World Bank (13), the Ethiopian Demographic and Health Surveys (EDHS) and other similar reports (9, 14, 15), Ethiopia has poor health status relative to other low- income countries, even within SSA. Poor nutritional status, infections, and a high fertility rate, together with low levels of access to reproductive health and emergency obstetric services, contribute to one of the highest maternal mortality ratios and infant mortality rates in the world. The maternal mortality ratio of the country was 871 and 673 per 100,000 live births in 2000 and 2005, respectively (9, 10). The infant mortality rate as reported by the EDHS was 77 per 1000 live births in 2005 (9). This is one of the highest in the world. The life expectancy at birth is only 49 years (10) indicating one of the lowest survival rates.

This situation is further aggravated by the high population growth. Young people constitute one third of the total population in Ethiopia (8). This implies profound reproductive health needs. The major reproductive health problems faced by the young population in the country are gender inequality, early marriage, female genital cutting, unwanted pregnancy, closely spaced pregnancy, unsafe abortion, and Sexually Transmitted Infections (STIs) including HIV/AIDS (16). Utilization of maternal service in Ethiopia is very low. Twenty-eight percent of mothers received antenatal care from health professionals (doctor, nurse, or midwife), six percent gave delivery in the health facility, and only five percent received postnatal care within the critical first two days after the delivery (9).

1.2. CONTEXT OF RH/FP IN OROMIA REGION

Oromia national regional state where the project was operating and the present study was conducted is an area of reproductive health concerns. The Oromia region of Ethiopia has a population of about 27.16 million (8). The health service coverage of the region for health stations and health centers is about 44.2 % while coverage including health posts as well is 73.1%. (17). The four zones of Oromia National Regional State(Jimma, East Wollega, Illubabor and West Wollega) where the project was operating and the present study was conducted consist of about 29 percent of the population of the entire Oromia region (8).

The infant mortality rate has improved from 2000 to 2005 116 to 76 per 1000

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 5

live births based on the EDHS (9,10). The region has the highest total fertility rate in Ethiopia of 6.2 births in 2005 and the annual population growth rate is estimated at 2.9 percent (9,10). Utilization of maternal health service is low in the region. According to the reports of the DHS 2005, the contraceptive prevalence rate (among married women) of this region was 13.6 percent. Twenty-five percent of mothers received antenatal care from health professionals, only 4.8 percent of births are delivered with the assistance of a trained health professional(a doctor, nurse, or midwife), and only 4 percent received postnatal care within the critical first two days after the delivery (9).

The region’s population is dominated by young people – over 67 percent of its population is under 25 years of age, an age group that has profound reproductive health needs (8). The many sexual and reproductive health problems faced by youth in the region include early marriage, female genital cutting, unplanned pregnancies, closely spaced pregnancies, abortion, sexually transmitted infections (STIs), and HIV/AIDS. Lack of education, unemployment, and extreme poverty exacerbates and perpetuate the reproductive health problems faced by Ethiopian youth.

According to a baseline study of reproductive health issues in Oromia in 1999, approximately 19 percent of all ever-married women in the region were married before the age of 15 and about one-half of all ever-married women entered in to marital union between the ages of 15-17. The estimated total fertility rate for the whole region was 6.4 per women and can be broken down to 6.2 percent for Illubabor, 5.5 percent for Jimma, 4.2 percent for East Wollega, and 3.1 percent for West Wollega (18).

Knowledge of at least one modern family planning method for the four zones of Oromia where the study is being conducted was 45.2 percent and only 15 percent of interviewed women had ever used at least one method of contraception. The same survey reported that only 8.4 percent of women who were not pregnant (or who were not sure about their pregnancy status) were reported to be using contraceptive methods at the time of the survey. There was some variation in contraceptive use by zone as 14.4 percent in Illubabor, 8.6 percent in Jimma, 9.4 percent in East Wollega, and 15.1 percent in West Wollega used contraceptives at the time of the survey (18). At the beginning of 2000, the Oromia Development Association (a membership-

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 6

based non-governmental organization established in 1993), expanded its social services by embracing reproductive health/family planning programs in selected zones of Oromia national regional state The main objective of the project was to ensure couples and individuals have access to FP and Reproductive Health services in order to meet their needs and to reduce the incidence of high-risk pregnancies, morbidity and mortality and to strengthen institutional capacity. The project was supported technically and financially by the David & Lucile Packard Foundation in three cycles. These cycles are defined by time and geographic focus of the project. The first cycle encompassed 2000-2002 (East Wollega and Jimma); the second cycle was between 2003-2005 (West Wollega and Illubabor zones, East Wollega and Jimma on consolidation phase) and the third was 2006 through the end of 2008 (West Wollega and Illubabor).

1.3. PROGRAM DESCRIPTION 1.3.1. OBJECTIVE OF THE PROJECT

1. General Objective:

The major objective of the project is to increase access to reproductive health/ family planning services, improve utilization of family planning services and to strengthen institutional capacity. 2. Specific Objectives of ODA program:

1. To increase the CPR (contraceptive prevalence rate)

2. To train nurses or health officers in teaching methods and reinforce the community based reproductive health agent curriculum, which will eventually lead to the training of other CBRHAs in their catchment area

3. To motivate the peripheral health workers and strengthen their capacity.

4. To increase access to reproductive health information to at least 90% of the project sites

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 7

5. To upgrade the knowledge and improve the skills of all peripheral health workers (CBRHA Supervisors) in family planning

6. To equip selected health facilities in the project sites with essential equipment and supplies in the project areas

7. To improve the amount and contraceptive mix in all delivery sites

The need and profile of the project changes with time and the objectives of the project also needed to be changed accordingly. However, the changes made to the objectives were not well explained in documents received from ODA.

1.3.2. PROGRAM COMPONENTS Contraceptive provision

The main component of this project is to distribute non-clinical contraceptives through community-based reproductive health agents (CBRHAs) by going door- to-door. This community-based contraceptive distribution is linked by referral and backup support to nearby health facilities.

Information, Education and Communication (IEC)

One of the major objectives of the project is to address the community with different RH information and to create awareness and bring about behavioral change. Home visits, individual and group discussions at public gatherings, schools and community based organizations such as Idir & Ikub, coffee ceremonies, faith-based institutions and others were reported as major ways of information dissemination through community health agents. Usually the topics covered including family planning methods, STI and HIV, maternal and child health (ANC, delivery, postnatal care, breast feeding/nutrition), immunization, traditional reproduction practices, and control of diarrheal disease. The IEC activities are supported by print media, such as leaflets, posters, magazines, newspapers and others.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 8

Adolescent Reproductive Health

Adolescent reproductive health is addressed using a multifaceted approach that encompasses promotion of RH in and out of school, at youth clubs (art, poetry, and sports), and through peer-to-peer education. The peer-to-peer approach is guided by a manual, which addresses adolescent sexuality, life skills education, prevention of HIV/AIDS/STI, and prevention of unwanted pregnancy, among others

Capacity Building

The project works to strengthen different levels of the health care system in the project site by improving organizational and managerial effectiveness and ensuring sustainability of the services.

Program Monitoring and Supervision

The project has developed standards for recording, reporting, and supervision based on different level staff that can effectively monitor the implementation of project activities.

Project Management and Coordination

The project management is directly accountable to the General Manager of the Association. The project has established offices at the regional, zonal, and Woreda levels, which are responsible to plan, coordinate, monitor and evaluate the project at their levels. The offices is staffed with a project manager, IEC Officer, RH Officer, and chief accountant at the Regional Level, a project coordinator and RH officers at zonal levels, and a woreda coordinator at the Woreda levels.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 9

Formation and strengthening of committees at various levels were undertaken to effectively implement coordination. An advisory committee at the regional level and steering committees at zonal, and Woreda levels were established. Each committee is chaired by the administrative officials at the respective levels, while ODA is serving as secretariat. Members are drawn from relevant government and non-government sectors such as Labor and Social Affairs, Education, Health, Agriculture, Finance, Women Affairs, NGO representatives, and ODA representatives. Coordination committees are primarily responsible to review annual plans, monitor implementation and facilitate networking through their regular meetings.

1.4. JUSTIFICATION OF THE EVALUATION

With technical and financial assistance from the David & Lucile Packard Foundation, ODA has been implementing RH/FP programs since June 2000 in the western zones of Oromia. The major objectives of the program are to increase access to reproductive health services and to expand family planning options in an effort to slow the population growth in the region. An evaluation of the program after eight years is essential to identify major expected and unexpected outcomes. The program will be evaluated to ensure that successful activities and strategies can be reproduced and unsuccessful activities can be improved.

1.5. STAKEHOLDER ANALYSIS

Stakeholders are individuals, groups, or organizations that have significant interest on how well a program functions. It is essential to involve stakeholders because without them, findings of any evaluation will have little chance of being used for program improvement. The role of major stakeholders in the program is described and their interest in the evaluation is assessed through a series of discussions with managers of the Packard Foundation Country Office and the Oromia Development Association.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) II

OBJECTIVEOF THEEVALUATION 11

II. OBJECTIVE OF THE EVALUATION 2.1 GENERAL OBJECTIVE

To assess the strengths, weaknesses and results of the Packard Foundation funded RH/FP project implemented by ODA in four zones of the Oromia National Regional State.

2.2. SPECIFIC OBJECTIVES OF THE EVALUATION

1. To assess the timely provision of resources to service providers

2. To assess the level of implementation of project activities as compared to what the project planned to do

3. To assess the quality of services that has been delivered by CBRHAs

4. To identify the role of the CBRHAs on the provision of RH services in the project site

5. To assess the knowledge, attitude and practice of the population in the project site on FH/FP services

6. To draw lessons learnt for future improvement of the program

2.3 EVALUATION QUESTIONS

1. What is the level of implementation of the project?

• Timely availability of inputs (Uninterrupted supply of contraceptives) and execution of activities (trainings, procurement, distribution) during the project life

• The relationship between different actors in the project (ODA, health structures, and community leaders)

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 12

• Quality of services (compliance with technical standards)

• Amount of products/services produced versus planned outputs(number of beneficiaries served)

2. Is there evidence that the program contributed to change in the RH indicators in the project site?

• Knowledge and practice of women on FP and maternal health services

• Utilization of FP and maternal health services

3. What were the facilitators and barriers to the implementation of the project and the achievement of its expected outcomes?

4. What were the non-expected outcomes of the project?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 13

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) III

EVALUATION METHODOLOGY 15

III. EVALUATION METHODOLOGY

3.1. STUDY AREA

The evaluation was conducted in four zones of Oromia National Regional State where the project was implemented: Jimma, East Wollega, Illubabor and West Wollega.

3.2 EVALUATION DESIGN AND DATA COLLECTION METHODS

The evaluation design encompassed a blend of several techniques to assess the implementation of the program and the achievement of operational outcomes. The overall framework is depicted in figure 1. It focuses on the core activities and expected outcomes of the program.

Figure 1 An Evaluation framework used to assess the implementation of the program and the achievement of operational outcomes, Ethiopia, 2009.

Availability Resources:  Number of local schools and youth centres equipped  Operational Outcomes at  Amount of FP commodities and Outcomes: target population medical supplies distributed  Geographic coverage  Timely supply of contraceptives  increased Increased CPR methods knowledge of Increased CYP  Availability of supervision contraceptive  Number of people reached with IEC methods  Number of IEC materials distributed  Increase obstetric  Number of health facilities equipped coverage  Number of CBRHAs trained and deployed

Sustainability:

 Satisfaction of clients on variety for choice on existing FP methods  Satisfaction of the clients on the handling by service providers  Existence of community organized committees to foster RH activities  Existence of linkage with the formal health sector  Acceptability of the project by local community

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 16

Various data collection strategies were used in the evaluation; these included a review of service statistics from records, reports of ODA, and collection of both qualitative and quantitative data. A total of 16 focus group discussions (FGDs) were held with CBRHAs (8 with female and 8 male groups), 6 FGDs with Youth adolescent reproductive health services (ARHS) club members, and in-depth interviews (IDI) were administered to 64 experts working at different levels (ODA coordinators (4 zonal and 8 woreda), health administrative structures (4 zonal and 8 woreda), health workers (8), and health extension workers (32)). Additionally, a structured questionnaire was administered to a total of 904 women of reproductive age groups.

3.3. SAMPLE SIZE AND SAMPLING TECHNIQUE

Determination of the sample size and the sampling technique was as following: Sample size

Sample households representative of each of the four zones was calculated by using the assumptions and parameters from the project records. Since there were multiple variables in the study on which there is no population data that can be used for sample size determination, the population proportion (P) was considered as 50% to calculate the minimum required sample size. This allows calculating the largest value for the minimum required sample size. As multistage sampling was applied, a design effect of 2 was assumed to adjust for the possible loss of precision due to the design. Taking zonal population size differences into consideration, slight upward adjustment was made to the minimum sample size determined by the formula shown below. The total sample size was distributed to the zones proportional to their population size.

N= (Z)2P(1-P)/e2 Where Z=z_value at 5% probability level, p=proportion of people assumed received the services, and e ( margin of error)=5%

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 17

Selection of sample woredas and kebeles

To select sample Woredas for the study, all project woredas were categorized as high, medium, and low performers based on performance criteria developed by the evaluation team together with ODA RH/FP coordinating office (Annex II). Based on this, from each of the three zones except Jimma, it was decided to randomly select one Woreda from each of the high and low performers. As the population of the Jimma zone is almost double that of the other zones, it was decided to select four woredas from Jimma zone, two from each of the high and low performers. Accordingly, a total of ten woredas were selected from all zones; two woredas from each of the three zones and four from Jimma zone based on population size (table 1). A total of 36 kebeles were sampled; the woredas were selected based on probability proportional to population size and the kebeles sampled were selected randomly from the list of kebeles in the specific woredas (table 1). The intention to select woredas according to implementer perception was to provide for a differential comparison and to check if individual insights expresses sound evidence.

Table 1 List of sampled woredas per zone and distribution of samples to zones and woredas, June 2009.

Number Number Name Sample Total Name of Population Performance of House of of size/ # of selected (in million) category holds per selected Zones zone woredas woreda woreda Kebeles

East Jimma Haro High 100 4 1.23 180 18 Wollega Abay Chomen Low 80 3 Seka Chekorsa High 108 4

Gomma High 92 4 Jimma 2.49 350 20 Manna Low 83 3 Tiro Afeta Low 67 3

West Ayra Gulliso High 51 2 1.35 190 19 Wollega Najo Low 139 6 Bedele Dabo High 105 4 Illubabor 1.27 180 24 Chawaka Low 75 3

TOTAL 6.34 900 81 10 900 36

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 18

3.4. DATA MANAGEMENT AND ANALYSIS

Data collectors and supervisors used in this evaluation are high level professionals who were trained on the collection of the data. Each day, the collected data was checked by supervisors and survey coordinators for quality and completeness. The qualitative data collected through field notes and tape record transcribed daily to fair notes, translated to English and then coded and analyzed using thematic and interpretive approach, and finally the contents were summarized and reported in narrative way. Analyses of the quantitative data were done using SPSS version 16.0. Descriptive statistics and statistical tests including chi-square test, analysis of variance and fitting of relationships between family planning and maternal health indicators were performed. Finally comparisons were made between the presumed high and low performing woredas based on the observed evidence.

Limitations of the study

The household survey includes only females of reproductive age (15-49 years). Due to financial and time constraints, male household members were excluded from the survey and therefore, it was not possible to assess their knowledge, attitudes and practices.

Initially, it was planned cover project area and non- project area in the evaluation to compare the result of program outcome. Due to lack of finance and time constraints, we took presumed high performing and low performing Woredas. But there are discrepancies between the presumed high performing and low performing Woredas (based on the stakeholder perceptions) and observed high performing and low performing Woredas(based on the household survey) . The details are presented in the respective section of the results.

Data on quality of counseling services such as adequacy of physical environment for individual counseling, demonstration of appropriate counseling techniques, coverage of essential information during counseling sessions, and development of socio-emotional support should have been collected through observation of a counseling session by an expert observer or mystery client. But due to time constraints, data related to quality of service delivery through CBRHAs was collected only by administering interview to the study participants. The issue of financial expenditure was not addressed in the evaluation.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 19

3.5. ETHICAL CONSIDERATIONS

Appropriate ethical clearance was obtained from Tulane University IRB. Verbal consent was obtained from the rest of the constituents including Packard Foundation, ODA, Zonal health departments (ZHDs), Woreda Health Offices (WoHOs), and health facilities during the period of data collection. Individuals interviewed also gave verbal consent. Furthermore, any personal identifiers of respondents were excluded from all the data collection tools.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) IV

RESULTSAND DISCUSSIONS 21

IV. RESULTS AND DISCUSSIONS

The results of the evaluation are organized in two parts: (1) Degree of program implementation which is related to the process and level of implementation of the major planned activities of the project; and (2) the program outcomes.

4.1. DEGREE OF PROGRAM IMPLEMENTATION

Degree of program implementation measures the process and level of implementation of the major planned activities of the project.

4.1.1. GEOGRAPHIC COVERAGE:

Over the last eight years, a total of 57 woredas were covered by the project; 13 Woredas from each of Jimma, East Wollega and Illubabor and 18 Woredas from West Wollega zone in a phased approach. All the 26 Woredas from Jimma and East Wollega were covered during the first phase of the project while the 31 from Illubabor and West Wollega were covered during the second phase of the project. The project achieved 100 percent of its plan to cover the 57 Woredas.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 22

Figure 2 Number of project Woredas by zone and year, 2000-2008, Ethiopia, 2009.

# of woredas covered 6

0 2000 2001 2002 2003 2004 2005 2006 2007 2008

Jimma East Wollega Illubabor West Wollega

4.1.2. LEVEL OF STAKEHOLDERS’ INVOLVEMENT

Proper stakeholder involvement is crucial for the success and sustainability of the program. The project documents indicate establishment of project steering committees and involvement of the community representative in the project planning and implementation. This is confirmed by interviewed project coordinators and FGD participants who agreed involvement of stakeholders drawn from relevant government and non-government sectors such as Labor and Social Affairs, Education, Health, Agriculture, Finance, Women Affairs, NGO representatives, and religious institutions. They reported this involvement had helped them to get acceptance in the community, to pass RH/FP message in public meetings, and even religious gatherings. But the participants from health sector have reported that there were occasions where there were joint planning but separated implementation as well as occasions with no joint planning but connected implementation.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 23

4.1.3. RELATION OF THE PROJECT WITH PUBLIC SECTOR AND COLLABORATION WITH HEWS

To be effective, any community-based health services should have a good relationship and linkages with the formal health sector that would provide technical, supervisory, and referral support. During the FGD with CBRHAs, participants discussed the relationship between the ODA RH/FP project and the public health sector such as WorHO, HC/HP. The focus group participants agreed that the ODA RH/FP project and the public health sector work together at health centers and clinics in the area. The community based reproductive health agents said they are supervised by health extension workers from nearby health facilities, refer cases to those facilities, and the health centers uses the CBRHAs for implementation and follow up of some services.

However, while the majority of FGD participants agreed that they were able to refer clients to the formal health sector, some CBRHAs expressed that there were occasionally issues where clients had to wait extended periods of time before receiving services. Further, there were friction between HEWs and CBRHAs on the modalities and cost recovery of contraceptives. The CBRHAs were used to sell the contraceptives while the same contraceptives are provided free of charge by HEWs. Despite the fact that the issue was solved, it indicates that there is a need for joint planning for complimentary interventions.

According to experts interviewed at ZHDs and WorHOs as well as HEWs, the collaboration between HEWs and CBRHAs covers not only reproductive health but also other community based health programs. For long term contraceptive services, CBRHAs refer their clients to health posts where HEWs deliver services and if there is shortage of supplies at health posts. Additionally, HEWs are getting strong support from CBRHAs for different activities including public education, promoting latrine construction, community mobilization for vaccination, immunization, and environmental sanitation. Some of the CBRHAs are also selected as model families. Data from FGD suggest that when asked about the scope of their work female CBRHAs tend to clearly list RH issues. However males are broader in their responses. This “natural” male expansion of the subject might be related to the difficulties that male CBRHAs have to address RH when talking to female clients.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 24

4.1.4. SUPERVISORY SUPPORT

The activities of community based reproductive agents (CBRHAs) should be continuously supervised by such as a mid-wife, a family planning nurse, a program coordinator or other staff members and they should be supported in managing any problems as they occur. The majority of participants discussed they had been receiving supervision, mostly from the woreda project coordinators, health workers, or both. Supervisor support included suggestions regarding their daily activities, responses to difficult questions raised by the public, and support on record keeping. Supervisor support also included a variety of different types of feedback. Nevertheless, some participants reported that there are some supervisors who are not as supportive. These supervisors were reported to only appear monthly to take the records and compile reports without providing any technical support. Additionally, some participants reported that supervision has been weak recently.

4.1.5. CAPACITY BUILDING ACTIVITIES

Capacity building of the project implementers specifically of the health facilities and health workers and ODA RH/FP staffs were among the project outputs planned so as to manage the project effectively.

A. Capacity building of the health facilities and health workers

Equipping health facilities, training of the health personnel, and ensuring a continuous supply of contraceptives were among the planned project outputs. The project planned to equip 300 health facilities over the duration of the project with different medical equipment related to RH/FP services; to date 99% of the targeted health facilities received this equipment (table 3). The major medical equipment distributed to the health facilities included delivery kits, sphygmomanometers, stethoscopes, adult weighing scales, loop insertion & removal sets, examination tables, and delivery bed areas.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 25

As part of human capacity building activities, different types of trainings were planned and provided to health workers in order to improve their knowledge on different types of contraceptives methods and to train them as trainers and supervisors for referral backup. Based on program reports that were conducted during the duration of the project, 80 % of the health personnel had received training on long-term contraceptive methods and/or training of trainers. However, due to issues regarding the training of long-term contraception care, health workers were not able to ensure continuity using long-term contraceptive methods in the health facilities. This lack of trained personnel to remove already inserted long-term contraceptives was reported as a major problem faced during the project because of the phase out by experts both from the project and the health sector.

B. Capacity building of the CBRHAs

The foundation of the community based RH/FP services are the CBRHAs who are expected to provide non-clinical contraceptives (pills and condoms) at the community level and educate the community on selected reproductive issues. The CBRHAs, usually village women and men, need to be trained on the basic concepts of family planning, how each method must be used, side effects and precautions for each method, how to keep simple records, and how to report the information to their supervisor. Information related to training of CBRHAs was collected through document review, IDIs, and FGDs.

The reviewed project documents showed, prior to deployment, nearly all of the CBRHAs had received training on community based RH services for 15 days in all Woredas of the project zones in the last eight years. Additionally, refresher trainings were given to 1190 CBRHAs who had served the project for more than one year to update their knowledge and skills (Table 2).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 26

Table 2: Planned vs. Achieved selected Capacity Building Activities goals, 2000-2008, Ethiopia, 2009.

Ser # Activities Planned Achieved % 1 Equipping health facilities 300 297 99 TOT on FP methods/Training of health workers 2 537 427 80 on contraceptive technology 3 Deployment and training of CBRHAs 2400 2386 99

Interview with Zonal and Woreda Health Office experts and FGDs with CBRHAs reported that the trainings were of good quality with appropriate trainers who were experts in the field. The trainings used interactive and participatory learning methods such as demonstrations, models, charts, and pictures relevant to FP and RH, group discussions, and role-playing which was essential for training the CBRHAs. However one major limitation of the trainings was that training materials were prepared in English and Amharic and most of the CBRHAs were not able to understand either language. There were some complains about protection of the material during the raining season. CBRHAS refer that they lost materials because of humidity and rain water. Additionally, the majority of the CBRHAs FGD participants reported the duration of training was too short for both the basic and refresher courses and that there was an absence of a training manual. Some respondents felt that the refresher trainings were not well planned, and that while they are organized every year in some Woredas, they are not consistently planned in others.

4.1.6. ROLE OF CBRHAS AND THEIR ACCEPTANCE BY THE COMMUNITY Role of CBRHAs

The project documents stated that major role of the CBRHA is educating and hiring of clients, provision of non-clinical contraceptives and referring those in need of clinical contraceptives to the nearest health facilities as well as educating the community on prevention of STIs/HIV and AIDS, promotion of MCH, and on harmful traditional practices (HTPs). Additionally, many CBRHAs conduct work outside of family planning services including malaria control activities, immunization services, outbreak notification and control, and hygiene and sanitation.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 27

Furthermore, to assess the impact that the CBHRAs had in the community, community members/study participants were asked if they knew someone who teaches the community and conducts outreach on RH and FP . Additionally they were asked if they had every received a particular service from the CBRHA including health education, condoms, oral contraceptive pills (OCP), a referral for family planning services and/or a referral for another service. Just over half (54.8%) of all respondents reported to know someone who teaches the community on RH and FP, although there was significant zonal variation (p<0.05) by project zone where 69.4% of respondents in Illbabor knew of a CBRHAs while only 33.9% knew in East Wollega. Overall, 43.7% of respondents reported to have received some service from this person; the most common service was health education (33.8%), followed by OCP (18.6%), and referral for FP services (12.7%). Very few (less than 3% overall) received condoms from the CBRHAs (table 3).

Table 3 Percentage distribution of Availability of CBRHAs and type of service they provide, by project zones, Ethiopia, 2009 .

East West Variables Jimma (%) Illubabor (%) All (%) Wollega (%) Wollega (%)

Know CBRHAs 33.9 51.7 66.5 69.4 54.8

Received some service from 29 40.3 61.3 46.7 43.7 CBRHAs Health 27.3 30.3 40.8 40 33.8 education Condoms 3.8 1.7 3.7 1.7 2.5 distribution OCP 11.5 18.3 24.6 20 18.6 Referral for 10.9 5.1 36.1 4.4 12.7 FP services Referral for other 9.8 7.7 3.7 7.8 7.3 services Total number 183 350 191 180 904

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 28

Acceptability of CBRHAs by the community

Acceptability of the service providers is one of the factors that affect utilization of services; services less accepted by the community are usually less utilized. Information collected through expert interviews shows that CBRHAs have strong community acceptance. Experts from different health structures stated that the CBRHAs are community elected service providers, have good community acceptance and trust, and even demonstrated this by convincing community members, especially religious leaders, who were very resistant against use of family planning.

Some of the factors that contributed to the observed level of acceptability, as mentioned by HEWs and MCH experts at health centers include:

• the selection of CBRHAs from the community and involvement of community members during selection

• the presence of referral linkage between CBRHAs and Health Facilities which increased the credibility of the CBRHAs advice

The gender of the CBRHA plays a role in the dissemination of RH/FP services. Some participants explained that female CBHRAs can easily reach the community and provide necessary services since women may be more open to discuss their reproductive health issue with a female rather than a male providers. On the other hand, some claimed that male providers can help elicit change easier since they have the advantage of being able to convince the husband or partner. According to the document review, the majority (70 %) of the CBRHAs were male with overall male-to- female ratio of 2.6-to 1 (3.27-to-1 for Jimma and East Wollega and 1.63-to-1 in Illubabor and west Wollega). The reason for the low number of female CBRHAs is attributed to inability to get female CBRHAs that could fulfill the minimum criteria set for employment which are: Resident, married, preferably female with at least one child, volunteer, Age between 20- 50 years, previously or currently practicing contraception, has a good mastery of the local language, culture and tradition, proven ability to read and write, and who have no other “official” responsibilities in the community. Participants who responded to have known and received services from the CBRHAs were asked about the sex of CBRHA they met. In addition respondents were asked whether

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 29

they are comfortable to discuss reproductive health issues with male CBRHA and whether he was involved and convinced the husbands or partners. 58% of participants responded they received services from male CBRHAs. Among them, 47 percent reported they are comfortable to discuss with male CBRHA and 41 percent reported that he was involved and convinced their husband or partner.

4.1.7. INFORMATION, EDUCATION, AND COMMUNICATION (IEC)

The information, education, and communication (IEC) component of the project aimed to present to the community different RH information and to create awareness to bring about behavioral change. The project documents showed IEC activities were supported by the distribution of different IEC materials such as posters, brochures, booklets, flipcharts, leaflets, magazines, newspapers and que- cards. Data compiled from the project reports on distribution of IEC materials and the numbers of people addressed, and information generated through FGD with youth ARHS club member is presented below.

4.1.7.1. DISTRIBUTION OF IEC MATERIALS

The program report of the organization shows distribution of several types of different IEC materials to support IEC activities (table 4).

Table 4 IEC materials distributed to support IEC activity, 2000-2008, Ethiopia, 2009.

Type Amount (pieces) Posters (five types on 5 subjects) 449,448 Brochures (five types on 5 subjects) 41,193 Booklets (two types on 2 subjects) 2,200 Flipcharts 20,760 Leaflets 82,015 Magazines 2,279 Newspaper 15,356 Youth to youth newspapers 5,400 Que-cards 608

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 30

4.1.7.2. PROVISIONS OF INFORMATION AND EDUCATION

All the project zones reported poviding information and education about selected components of reproductive health in the project sites to a large number of people. The number of individuals (including those encountered for more than one period, as shown in Table 5 addressed on RH through IEC activities in each project zones were increased from 1.4 million in 2000/2001 to more than 12 million in 2007/2008. More than 55 million individual encounters were reported during this period, ranging from 6.3 million in Illubabor to nearly 21 million in East Wollega. This shows that community members were informed more than four times on different RH components each year during the project period.

However, there is a paucity of effectiveness of the IEC approach used by the project for several reasons. Disseminating information at public gatherings, schools and community-based organizations such as Idir and Ikub by community health agents might help for advocacy but is less effective for conveying a deep understanding of the issues and behavioral change. Further discussions with ODA officials revealed that the project is not developing its own IEC materials but collects and distributes all IEC materials on FP/RH developed by other organizations. There is no in-depth analysis of the problem and no pre-test of the materials for its comprehension, aesthetic appeal, and cultural acceptability.

Table 5 Total number of encounters with people on RH through IEC Activities by project zones and year; 2000-2008, Ethiopia, 2009.

Project years E. Wollega Jimma W. Wollega Illubabor Total 2000-2001 498,365 914,337 0 0 1,412,702 2001-2002 709,298 1,434,357 0 0 2,143,655 2002-2003 1,549,953 1,000,756 0 0 2,550,709 2003-2004 3,129,500 2,353,961 598,580 247,657 6,329,698 2004-2005 2,719,908 2,428,425 1,648,279 1,205,996 8,002,608 2005-2006 3,121,674 3,176,200 2,590,396 1,629,500 10,517,770 2006-2007 3,964,719 3,925,850 2,742,685 1,512,305 12,145,559 2007-2008 5,220,916 2,459,083 2,757,023 1,715,599 12,152,621 Total 20,914,333 17,692,969 10,336,963 6,311,057 55,255,322

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 31

4.1.7.3. EXPOSURE TO FAMILY PLANNING MESSAGES

Exposure to family planning messages widens the horizon of understanding on issues related to contraceptive use and helps in the realization of its importance in achieving desired family size. Study participants were asked to list the most common sources of information regarding family planning for them. The result shows CBRHAs are the most frequent source of information on FP in all the project zones (40 % overall) except in East Wollega, followed by health extension workers (35 %), and Radio (28 %) (figure 2).

Figure 3 Percentage distribution of source of FP information among women of reproductive age group, Ethiopia, 2009

30 Percent

0 East Wollega Jimma West Wollega Illubabor All Project zones Radio Television Print Media* CBRHAs HEWs Health Workers Friends/Family Others

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 32

4.1.7.4. ORGANIZE/SUPPORTING IN AND OUT-OF-SCHOOL YOUTH CLUBS

Adolescent and youth constitute a large portion of the population and this group has special needs that require careful approaches to meet their unmet reproductive health needs. It was planned to establish new youth clubs and to support already available youth clubs where the youths can educate themselves and the community using peer-to-peer learning approaches through entertainment programs and demonstrations. A total of 73 youth clubs were organized or supported, 33 out-of- school and 40 in-school youth clubs across the four zones. FGDs conducted with youth adolescent reproductive health (ARH) club members revealed that the clubs have benefited them a lot in learning about RH and is an appropriate approach for this group of population. The clubs foster open discussions and free expression of ideas among the youth. Youth said the clubs are better than others even than formal schools and health facilities because there are better demonstrations at the youth clubs that foster open discussion and allow them to freely express their ideas and ask questions in a peer environment. The study participants agreed the approach has helped adolescents and the community to understand reproductive health issues such as using family planning methods, awareness on female genital cutting and other HTPs, and knowledge of modes of transmission of HIV and reduction in stigma associated with lack of correct knowledge. The IEC activities which are based on peer-to-peer learning approaches were supported by different IEC materials.

This study also identifies some gaps in the implementation of adolescent reproductive health programs. These include the following:

• The ARHS clubs and services are limited only in small towns and the majority of rural youths are inaccessible.

• There is a gap in the process of development of IEC materials. Most of IEC materials used are developed elsewhere and brought to the youths for use.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 33

4.1.8. PROVISIONS OF CONTRACEPTIVES

Community health agents based on community distribution strategy provide non- clinical contraceptives including condoms and but refer clients who need clinical contraceptives (injectibles, Norplant, IUCD, and VSC). Prior to the introduction of the health extension program, CBRHAs were the only community based contraceptive providers.

4.1.8.1. AMOUNT OF CONTRACEPTIVE METHODS AND COUPLE-YEAR PROTECTION (CYP) GENERATED

Timely availability of contraceptive methods is critical for continuity of service provisions. According to the project reports, over the project years more than 9.3 million different contraceptives were provided to clients by the community health agents and health facilities. Pills, injectables, and condoms were the major types of contraceptive methods distributed. Participants of FGD and IDI confirm the timely and continuous supply of contraceptives by ODA RH/FP project.

Couple year protection is a measure representing the total number of years of contraceptive protection provided by a method. For each method, the CYP is calculated by taking the number of units distributed and dividing that number by a factor representing the number of units needed or estimated to protect a couple for one year. Over the project years, more than 1.1 million CYP were generated in the four zones. Jimma and Illubabor zones recorded for maximum and minimum CYP. Injectables accounted for more than half of the total CYP generated (table 6).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 34

Table 6 Total number of Contraceptive methods distributed and CYP Generated by Method and project zone; 2000-2008, Ethiopia, 2009.

Methods Distributed Zones Pills Condoms Injectables Norplants IUCD East Wollega 2,423,263 597,854 810,921 5,112 227 Jimma 2,540,873 505,698 799,306 7,989 473 West Wollega 430,642 31,253 444,572 2,453 455 Illubabor 398,368 2,449 276,227 2,252 61 Total 5,793,145 1,137,255 2,403,551 17,806 1,216 CYP Generated by Method Pills Condoms Injectables Norplants IUCD Total East Wollega 186,405 3,986 202,730 17,627 783 411,531 Jimma 195,452 3,371 199,826 27,547 1,631 427,827 West Wollega 33,126 208 111,143 8,460 1,569 154,507 Illubabor 30,644 16 69,057 7,766 210 107,693 Total 445,627 7,582 600,888 61,400 4,193 1,119,689

4.1.8.2. NEW CONTRACEPTIVE USERS

Over the eight project years more than 733,000 new contraceptive users have benefited from the project in the four zones. About 55% of the cumulative new users over the project years were oral contraceptive pill (OCP) and condom users who were provided with the services by CBRHAs while 45% were users of clinical contraceptive methods (injectibles, Norplant, IUCD and VSC). Overall OCP (49 percent) and injectibles hormones (43 percent) were the two commonly used methods, followed by condoms (5.5%) and Norplant (2.4%). More than 64 percent of the cumulative new users over the project years are reported from Jimma and East Wollega (Jimma 35 percent and East Wollega 29 percent; while West Wollega and Illubabor combined accounts for 36 percent (table 7).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 35

Table 7 Cumulative Number of New Contraceptive users by method and year; 2000-2008, Ethiopia, 2009.

2000 2001 2002 2003 2004 2005 2006 2007 2008

Methods

OCP 7,949 32,356 67,177 90,400 133,796 181,586 242,985 300,831 359,331

Condoms 1,765 7,134 12,879 16,243 20,381 24,000 29,400 35,964 40,621

Injectibles 681 10,397 26,875 43,150 75,146 124,952 170,600 239,781 313,324 IUCD 0 0 0 0 58 405 579 986 1,201 Norplant 0 0 0 5 192 2,163 8,129 12,231 17,642 VSC 0 0 0 0 48 296 550 933 996

Total 10,395 49,887 106,931 149,798 229,621 333,402 452,243 590,726 733,116

Zones East 5,932 26,075 54,434 66,646 90,732 111,638 145,616 179,575 212,418 Wollega Jimma 4,463 23,812 52,497 68,499 99,218 135,201 166,952 205,963 258,600 West 0 0 0 8,435 22,507 45,242 77,714 117,296 151,385 Wollega Illubabor 0 0 0 6,218 17,164 41,321 61,961 87,892 110,713

Total 10,395 49,887 106,931 149,798 229,621 333,402 452,243 590,726 733,116

Comparison of trends of new users by type of contraceptive methods over the years shows a decline in OCP and condom users and an increase in injectibles and Norplant users. Between 2000 and 2008, OCP users declined by 35 percent (from 76.5 percent to 41 percent) and condom users declined by 14 percent (from 17 percent to 3.3 percent), while injectible users increased by 45 percent (6.6 percent to 52 percent) and Norplant users by 4 percent (fig 3.) However, there is no significant correlation between total number of people addressed about FP and total number of condoms distributed during the same year (p=0.462).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 36

Figure 4 Trends of new contraceptive users by method and year; 2000-2008, Ethiopia, 2009.

90.00%

80.00%

70.00%

60.00%

50.00%

40.00%

30.00%

20.00%

10.00%

0.00% 2000 2001 2002 2003 2004 2005 2006 2007 2008 Year OCP Condoms Injectibles IUCD Norplant VSC

4.1.8.3. CONTRACEPTIVE ACCEPTANCE RATE:

The major outcome of project was the increase in the CPR from 6 percent in 2000 to 25 percent in 2005 and to 40 percent in 2008 in Jimma and East Wollega, and from 7.5 percent in 2002 to 23 percent in 2005 and to 40 percent in 2008 in Illubabor and West Wollega. Data compiled from the reports on CPR shows that the CPR of Jimma and East Wollega for 2005 and 2008 were 26.5 percent and 35 percent respectively while it was 18.7 percent during 2005 and 33.6 percent in 2008 for Illubabor and West Wollega.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 37

Figure 5 Baseline, plan and achievement of Contraceptive Acceptance rate by project zone; 2000-2008, Ethiopia, 2009

45 40 40 40 35 35 33.6

30 26.5 25 25 23 20 18.7 Percent

10 7.5 6 5

0 Jimma & E.Wollega Illubabor & W.Wollega Project zones

Baseline plan 2005 Achieved 2005 plan 2008 Achieved 2008

4.1.8.4. PERCEIVED QUALITY OF SERVICES AND CLIENT SATISFACTION

Perceived quality of services and client satisfaction towards services received are important for service utilization. Services with good perceived quality and services that satisfy clients are more likely to be used than services with poor perceived quality and with poor satisfaction. Data on perceived quality of services and client satisfaction towards RH/FP service provided through CBRHAs were collected in the survey. Study participants were asked questions whether they received the information and services they need and their feelings about having adequate privacy during consultation with the CBRHA and confidentiality of the information.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 38

Overall, nearly 6 out of 10 respondents indicated to have received some information and services they needed. About half reported to have had adequate privacy during consultations with the CBRHA and to have felt that the information they shared with CBRHA was kept confidential.

Communicating effectively with clients and potential clients and provision of quality counseling service is essential not only to recruitment but also to retaining clients. Study participants were asked a number of questions, including whether their consultation time with CBRHAs were adequate and regarding interactions they had while with the CBRHAs.

The result shows that only 57 percent of the respondents agreed on adequacy of the consultation time. However, there is evidence that the counseling process is generally poor. Because majority of respondents complained that the CBRHAs do not listen to their concern, do not let them ask questions, and were not satisfied with the responses of CBRHA to the questions they raised.

This might be related with lack of adequate training on counseling process and that each CBRHAs are expected to access more than 250 households. Compared with the other zones, respondents from West Wollega are in better position in terms of all these indicators (table 8).

Table 8 Information related to Perceived quality of services and client satisfaction towards RH/ FP service provided, Ethiopia, 2009.

East West Jimma Illubabor Indicators All (%) Wollega (%) Wollega (%) (%) (%) Received any information or services 64.2 55.4 74.4 45.2 63.1 they need Have adequate privacy during 50.9 48.2 55.6 35.7 48.1 consultation with the CBRHA? Information shared is kept confidential 28.3 52.5 61.5 46.4 50.6

Adequacy of consultation time 54.7 51.7 60.6 59.5 56.5

CBRHA listen to their concerns 9.4 34.8 39.3 19 29.4

CBRHA let them to ask questions 3.8 10.6 25.6 11.9 14.4

Satisfied with response of CBRHA to 3.8 9.2 25.6 10.7 13.7 the questions Total number of respondent (N) 180 350 190 180 900

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 39

Across all project zones, 85 percent of women surveyed believed that the project has helped their family (fig 6). Some of the benefits to their family include adequate knowledge of FP services, how to prevent diseases including STDs, and accessibility to services near their community.

Figure 6 Percent of respondents who indicated the project has helped them or their family by zones, Ethiopia, 2009 .

100 99.1

90 84.8 79.2 82.1 80 76.6

Percent 40

0 East Wollega Jimma West Wollega Illubabor All

Project zone

Furthermore the study participants were asked open ended questions if they could suggest any improvement to the RH/FP services provided by CBRHAs. Using female CBRHA, more involvement of their male partners (husband), more intense and frequent health education, using health professionals, and free supply of contraceptives are some of the mentioned points as improvement.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 40

4.1.8.5. INFORMED CHOICE

Providing adequate information and comprehensive advice to ensure informed choices about the methods adopted is very important to continue the methods and or seek medical support. Current users of various modern contraceptive methods were asked whether at the time they were adopting a particular method they were informed about the possible side effects or problems they might have with the method and what to do if they experienced side effects. 61 percent of the respondents reported they were informed about the possible side effects or problems they might have with the method and 51 percent of the respondents reported they were informed what to do if they experienced side effects. Although EDHS and this study cannot be compared directly for reasons of methodology and level of representation, it can help as a reference to which other related studies can be compared. This is significantly higher than the EDHS 2005 that reported 4 percent of users were informed about the side effects or problems associated with the method, 30 percent were informed about what to do if they experienced side effects. This shows that the level of informed choices about consequences of contraceptive in these communities is by far higher than the national estimate.

4.2. OUTCOME OF THE PROJECT

The major objective of the project is to increase access to reproductive health/ family planning services and to improve utilization of family planning and maternal health services. Data on knowledge and practice of family planning and utilization of maternal health services were collected from women of reproductive age groups using structured questionnaire.

4.2.1. SOCIO-DEMOGRAPHIC CHARACTERISTICS OF RESPONDENTS

All study participants were women between the ages of 15-49 years, with mean age of 28.5 (SE=0.28). The majorities were married (81.6 %), had no education (59 %), belonged to the Oromo ethnic group (89.8 %), and were farmers (54.5 %) by occupation. Muslims(48.2%), Protestants(27.0%) and Orthodox Christians(24.6%) together account for more than 99 % of the participants. (Table 9).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 41

Table 9 Socio-demographic and socioeconomic characteristics of participants, Ethiopia, 2009.:

Age group Frequency Percentage 15-19 114 12.6 20-24 169 18.7 25-29 200 22 30-34 159 17.6 35-39 129 14.3 40-44 90 10 45-49 39 4.3 Total 900 100 Marital status Married 738 81.6 Divorced 15 1.7 Widowed 42 4.6 Separated 8 0.9 Never married 88 9.7 Total 891 100 Highest level of education No education 533 59 Able to read & write 144 16 Primary and above 221 24.4 Total 898 100 Religion Muslim 436 48.2 Protestant 244 27.0 Orthodox 222 24.6 Others 1 0.1 Total 903 100 Ethnicity Oromo 812 89.8 Amhara 55 6.1 Gurage 7 0.8 Others 30 3.32 Total 904 100 Occupation House wife 231 25.6 Farmer 493 54.5 Merchant 76 8.4 Student 50 5.5 Government employ 12 1.4 Others 11 1.2 Total 873 100

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 42

4.2. 2. FAMILY PLANNING

Data on contraceptive knowledge, attitudes and behavior was collected from sample women in reproductive age groups. Comparisons are made, where appropriate, with findings from the 2000 & 2005 EDHS and with the baseline survey to evaluate changes over the project years.

4.2.1.1. KNOWLEDGE OF CONTRACEPTIVE METHODS

Acquiring knowledge about family planning is an important step towards gaining access to and using a suitable contraceptive method in a timely and effective manner. Individuals who have adequate information about the available methods of contraception are better able to make choices about planning their families. Thus, one of the main objectives of this evaluation was to obtain information on knowledge of family planning methods among women in the reproductive age group. Data on knowledge of contraception was collected in two ways. First, respondents were asked to mention all the methods of contraception that they had heard of spontaneously. For methods not mentioned spontaneously, the interviewer described and probed for whether the respondent recognized it.

Information was collected for six modern contraceptive methods: pill, IUD, injectables, implants, condoms, and lactational amenorrhea method (LAM), and three traditional methods (rhythm, periodic abstinence and withdrawal). Table 10 shows knowledge of contraception among all women age 15-49 and among those who are currently married. Knowledge of contraceptive methods is high with 91 percent of all women and 93 percent of currently married women knowing at least one method of contraception. Modern methods are more widely known than traditional methods. For example, 93 percent of currently married women know of a modern method, and only half of them (47 percent) know of a traditional method. The pill is the most widely known method (90.5 percent), followed closely by injectables (88.8 percent).

The mean number of methods known is a rough indicator of the breadth of knowledge of family planning methods. The mean number of methods known is 3.88 for currently married women and 3.84 for all women.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 43

Although EDHS and this study cannot be compared directly for reasons of methodology and level of representation, it can help as a reference to which other related studies can be compared. The result therefore shows that the level of knowledge of contraceptive demonstrated in the four zones is slightly higher than the result reported by EDHS 2000 and 2005. Knowledge of any modern method among currently married women was 85 percent in EDHS 2000 and 87 percent in EDHS 2005. However, knowledge about condom in this part of the country seems to be less compared to the national reports.

Table 10 Percentage of all women and currently married women who know any contraceptive method, by specific method, Ethiopia, 2009.

Currently Method All Women married Women

Any Method 91.5 93 Any modern method 91.5 93 Pill 89.4 90.5 IUD 20.1 20.7 Injectables 87.2 88.8 Condom 32.5 29.5 Implants 54.8 57.2 Lactational amenorrhea (LAM) 23.5 24.1 Any traditional method 46.8 47.3 Rhythm 22.3 22.4 Withdrawal 11.2 11.1 Sexual abstinence 43.3 43.6 Mean number of methods known 3.84 3.88 Total number 891 738

Knowledge of modern contraceptive methods is also found to be significantly associated with presence of radio in the household (p=0.006). Those who own radio are more likely to know more than one method. Furthermore participants of the qualitative survey agreed the community members have got adequate knowledge and are showing signs of behavior change even in the other components of reproductive health including HTP and HIV/AIDS. They stated an increase in number of individuals undergoing HIV testing prior to marriage; a reduction in wife inheritance, polygamy, and female genital cutting

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 44

4.2.1.2. USE OF CONTRACEPTIVE METHODS

4.2.1.2.1. EVER USE OF CONTRACEPTIVE METHODS

Ever use of contraception provides a measure of the cumulative experience of a population with family planning. Data was collected on ever use of contraceptive methods from women by asking respondents whether they had ever used any contraception. Among respondents who answered this question, 49.2 percent of the women overall and 56.4 percent of currently married women have ever used a family planning method. The results shows zonal variations in ever use of contraception. Women from West Wollega and Illubabor zones are more likely to use contraception compared to those from East Wollega and Jimma Zones (fig 6). Compared with EDHS 2005, ever use of contraceptive method is higher among both all women and currently married women which was 13% for all women and 17% for currently married women.

Figure 7. Percentage distribution of ever use of contraceptive methods among all women and currently married woman by zone, Ethiopia, 2009.

Percent 40

0 East Wollega Jimma West Wollega Illubabor All Project Zones All Women Currently Married Women

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 45

4.2.1.2.2. USE OF CONTRACEPTIVE METHODS IN THE LAST FIVE YEARS

Use of contraception in recent years, the last five years specifically, is an indicator of recent exposure or access to family planning and reproductive health services. Figure 7 presents use of contraception in the last five years among all women and currently married women by zone. The data indicates that 40.2 percent of all women surveyed and 46.9 percent of currently married women have used a method in the past five years. With in the last five years, women from West Wollega and Illubabor zones were more likely to have used contraception compared to those from East Wollega and Jimma Zones. Figure 8. Percentage of all women and currently married women who have used any contraceptive method in the last five years by zones; Ethiopia, 2009. 80 70.7 70 63.9 60 53.9 50 46.9 44.4 40.2 40 35.7 35.2 Percent 31.7 29.4 30

0 East Wollega Jimma West Wollega Illubabor All Zones All Women Currently Married Women In the last five years women were more likely to have used a modern method than a traditional method. Among currently married women who have used contraception with in the last five years, 96 percent used a modern method compared with 4 percent who used a traditional method. Injectables were the most commonly used modern method (54 percent) followed by pills (34 percent) (table 11).

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 46

Table 11 Percentage distribution of married women by type of contraceptive methods used in the last five years, Ethiopia, 2009.

East Wollega Jimma West Wollega Illubabor All Pill 34.0 40.7 31.8 30.2 34.2 IUD 1.1 1.4 2.9 0.0 1.6 Injectable 50.0 45.7 57.2 62.3 53.8 Condom 0.0 2.9 0.6 0.9 1.2 Implant 2.1 5.7 4.0 4.7 4.3 LAM 2.1 0.0 1.2 0.0 0.8 Rhythm 4.3 0.7 1.7 0.0 1.6 WD 4.3 2.1 0.6 0.9 1.8 Abstinence 2.1 0.7 0.0 0.9 0.8 Total (N) 180 350 190 180 900

4.2.1.2.3. CURRENT USE OF CONTRACEPTIVE METHODS

The current level of contraceptive use is a measure of the actual contraceptive practice at the time of the survey. It takes into account all use of contraceptions, whether the concern of the user is permanent cessation of childbearing or a desire to space births. Current use of family planning services provides insight into one of the principal determinants of fertility. It also serves to assess the success of family planning programs.

Contraceptive use among all women and currently married women is presented in table 12 by age group. The contraceptive prevalence rate for married women who are currently using a method of family planning in the project zones was 38.9 percent while it was 33.4 percent for all women. Almost all of these users are using modern methods. Seventy seven percent of married women who are currently using a method of family planning are using injectables followed by the pill (20 percent). There is substantial variation in current use by zone. Current use is highest in West Wollega (55.7 percent) and Illubabor (46.1 percent) and lowest in the East Wollega zone (24.2 percent). The result from household survey shows paradoxical to the result of document review that depicted contraceptive acceptance rate of 35% for Jimma and East Wollega and 33.6% for Illubabor and West Wollega. Women from West Wollega and Illubabor are more likely to use currently than those in Jimma and East Wollega.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 47

Table 12 Percentage of all and currently married women currently using contraceptive method by specific method, age, zone, and level of education, Ethiopia, 2009.

Current Use of contraceptive methods ALL WOMEN Any Pill IUD Injectable Condom Implant LAM VSC Age modern (%) (%) (%) (%) (%) (%) (%) method (%) 15-19 14.9 6.1 0.9 7.9 0 0 0 0 20-24 41.4 9.5 0 33.1 0.6 0.6 0 0 25-29 45 7.5 0.5 36 0 2 0 0 30-34 32.7 5 0.6 25.2 0.6 1.3 0 0 35-39 40.3 7.8 1.6 30.2 0 3.1 0 0 40-44 12.2 3.3 0 7.8 0 1.1 0 0 45-49 17.9 2.6 0 15.4 0 0 0 0 Zone East Wollega 21.9 4.9 0.5 16.4 0 0 0 0 Jimma 27.4 7.7 0.6 18 0.6 0.9 0 0 West Wollega 50.8 6.8 1 41.9 0 3.1 0 0 Illubabor 38.3 6.1 0 32.8 0 1.7 0 0 Level of Education No education 31.3 6 0.4 23.8 0 1.1 0 0 Only read & 40.9 6.9 0.7 31.2 0 2.1 0 0 write Primary+ 38.4 8.1 0.9 27.1 0.9 1.4 0 0 Total 33.4 6.6 0.6 25.7 0.2 1.3 0 0 CURRENTLY MARRIED WOMEN 15-19 36.6 14.6 2.4 19.5 0 0 0 0 20-24 44.8 10.3 0 35.9 0.7 0.7 0 0 25-29 45.7 7 0.5 37.1 0 2.2 0 0 30-34 34.5 5.4 0.7 26.4 0.7 1.4 0 0 35-39 42.5 8.3 1.7 31.7 0 3.3 0 0 40-44 15.6 4.7 0 9.4 0 1.6 0 0 45-49 22.6 3.2 0 19.4 0 0 0 0 Zone East Wollega 24.2 5.1 0.6 18.5 0 0 0 0 Jimma 33.3 9.9 0.7 21.2 0.7 1.1 0 0 West Wollega 55.7 7.2 1.2 46.1 0 3.6 0 0 Illubabor 46.1 6.4 0 40.4 0 2.1 0 0 Level of Education No education 35.1 6.6 0.4 26.8 0 1.3 0 0 Only read & 46.8 8.1 0.8 35.5 0 2.4 0 0 write Primary+ 51 10.5 1.3 35.9 1.3 2 0 0 Total 38.9 7.6 0.7 29.9 0.3 1.6 0 0

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 48

4.2.1.2.4. TRENDS OF CURRENT USE

Results of this study were compared with baseline survey conducted in 1999, the EDHS 2000 and 2005, and EGFHIE (19). Current use of contraceptive methods showed nearly five fold increase in the 10 year period between the 1999 baseline survey and the current survey from nearly 8 percent to 39 percent, nearly 2.5 times between 2005 EDHS and the current survey from 13.6 percent to 39 percent, and 4 percent from EGFHIE 2008 to the current evaluation (figure 8).

Figure 9. Trends in Current Use of Contraception by type of methods used, 1999-2009, Ethiopia, 2009.

40 38.9 38.9

35 35 34.4

30 29.9 27.7 25

Percent 20

15 13.6 12.9 10 8.4 8.6 7.6 6.6 6.4 5 4.3 4.2 3 3.4 2.4 1.9 1.6 0.7 0 0.4 0.1 0.20.2 0.30.4 0.1 0 0.3 Any Method Any Modem Pill IUD Injectables Condom Method Method

Baseline study 1999 EDHS 2000 EDHS 2005 EGFHIE 2008 Current evaluation 2009

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 49

4.2.1.2.5. SOURCE OF FAMILY PLANNING METHODS

Information on sources of modern contraceptives is useful for family planning managers and implementers. Women who reported using a modern method of contraception at the time of the survey were asked where they obtained the method the last time.

As shown in figure 10, majority of the current users (77.9 percent) obtain methods from the government health facilities such as health stations/clinics (50.4%), health centers (21.8%), and hospitals (5.6%). The CBRHAs also play a major role in distributing contraceptives, being the source for 19 percent of the users. The result of source of contraceptive methods among the current users is consistent with EDHS 2005 that reported 80 percent of current users obtained methods from the public sector. But percentage of current users who obtained the method from the CBRHAs shows a significant increase from 1.4 percent reported by EDHS 2005 to 19 percent in this study.

Figure 10. Percentage distribution of sources of contraceptives among current users, Ethiopia, 2009.

Gov. Hospital, 5.6 CBRHAs, 18.6

Pharmacy/drug vendor, 1.5 Health center, 21.8

Private HF, 1.8

Health Post/station, 50.4

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 50

4.2.1.2.6. CONTRACEPTIVE DISCONTINUATION AND INTENTION OF FUTURE USE

Couples can realize their reproductive goals only when they use contraceptive methods continuously. A major concern for family planning program managers is discontinuation of methods. In the evaluation, those who ever have used but currently are not using contraceptives were asked the reasons for any discontinuation. The desire to become pregnant is the most prominent reason for contraceptive discontinuation (42 percent), followed by health concerns (32 percent) and side effects (10 percent). When viewed across the age category, similar proportions of respondents voiced health concerns, but only those between 20 and 40 reported to discontinue the contraceptive due to side effects. It is however encouraging to observe that reasons for discontinuation such as a disagreement by the spouse, lack of access, and non-affordability were very rare among respondents. This is evidence that partly supports the impact of Packard foundation and other partners interventions.

Intention to use a method of contraception is an important indicator of the potential demand for family planning services. Currently married women who were not using contraception at the time of the survey were asked about their intention to use family planning methods in the future. The majority (72 percent) of currently married women who were not using any contraception at the time of the survey said that they intend to use a family planning method sometime in the future. Twenty percent do not intend to use any method, while 4 percent were unsure of their intention. Those women who do not intend to use any method in the future listed the desire for having as many children as possible, fear of side effects, and partner disapproval as the reasons for not having intention of future use.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 51

Table 13 Reasons for discontinuation of contraceptives and future intention of contraceptive use, Ethiopia, 2009.

Reasons for discontinuation of contraceptives Frequency Percent Wanted to become pregnant 39 42 Health concerns 30 32 Side effects 9 10 Husband away 5 5 Partner disapproved 2 2 Other 8 9 Total 93 100 Future Intention of contraceptive use Frequency Percent Yes 67 72 No 19 20 Unsure 4 4 Non response/missing 3 3 Total 93 100

4.2. 2. AGE AT FIRST MARRIAGE

A baseline study of reproductive health issues in Oromia in 1999 showed about 19 percent of all ever-married women in the region were married before the age of 15 and about half of all ever-married women entered in to marital union between the ages of 15-17. Data on age at first marriage was collected to see if there is any change in the last eight years. In this study, 11 percent of all married women were married before the age of 15 and 45 percent all ever-married women married between the ages of 15-17, demonstrating a mild reduction.

4.2.3. UTILIZATION OF MATERNAL HEALTH SERVICES

Provision of IEC to the community on maternal health care services such as ANC, delivery and post natal care are some components of the IEC activities of the project that aim to create awareness and increase service utilization. Data on utilization of maternal health care services such as ANC, Delivery and post natal care were collected from study participants to see if there was any increase in service utilization.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 52

4.2.3.1. ANTENATAL CARE

Antenatal care (ANC) coverage is based on the type of provider, number of ANC visits, and stage of pregnancy at the time of the first visit as well as content of services and information provided during ANC. In the household survey information on ANC coverage was obtained from women who had a birth in the five years preceding the survey. For women with two or more live births during the five- year period, data refered to the most recent birth only. Out of those interviewed 64% of the women reported to have given birth in the last five years. The majority (94 percent) of them gave birth once or twice; 61 percent once and 33 percent twice. The mean number of births during these period was 1.47 (SD=0.662). Sixty five percent of mothers received antenatal care from health professionals (doctor, nurse, mid-wife) for their most recent birth in the five years preceding the survey. Mothers from West Wollega reported exceptionally high rates of ANC utilization (table 14). There are significant differences in the use of antenatal care services by level of education and project zones (χ2=54.6, P<0.005). Those who have some education are more likely to use the services than those with no education.

Table 14 Distribution of women who had a live birth in the past five years and received ANC by provider during pregnancy for the most recent birth, Ethiopia, 2009.

Health No ANC Missing TTBA (%) Total workers (%) (%) (%) All 65 1.6 32.4 1 580 Education No education 53.9 1.7 43.3 1.1 356 Only read & Write 78.8 2.9 17.3 1 104 Primary+ 86.2 0.0 12.9 0.9 116 Zone East Wollega 59.5 1.5 38.9 0.0 131 Jimma 60.7 0.5 37.4 1.4 211 West Wollega 81.0 4.1 12.4 2.5 121 Illubabor 62.4 0.9 36.8 0.0 117

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 53

Timing and Number of Antenatal Visits

Antenatal care is more beneficial in preventing adverse pregnancy outcomes when it is sought early in the pregnancy and is continued through to delivery. Health professionals recommend that the first antenatal visit should occur within the first three months of pregnancy and continue on a monthly basis through the 28th week of pregnancy and fortnightly up to the 36th week (or until birth). WHO also recommends that under normal circumstances a woman without complications should have at least four ANC visits to obtain sufficient antenatal care.

Data on time of the first ANC visit and the total number of ANC visits was collected in the study. It was found that 1 in 5 women made their first antenatal care visit before the fourth month of pregnancy. The median duration of pregnancy for the first antenatal care visit is 5 months indicating that most of them start antenatal care at a relatively late stage of their pregnancy. About 35 percent women made four or more antenatal care visits while 32 percent of the women did not make any antenatal care visits during their entire pregnancy period (table 15).

Table 15 Percent of women who had a live birth in the five years by the timing of the first visit and number of ANC visits for the most recent birth, Ethiopia, 2009.

Time of first visit Percent None 32.4 1st three months 20.3 2nd three months 36.7 3rd three months 5.5 Missing/no response 5 Median months of pregnancy at 1st visit 5 Total 580

Total number of visits made Percent

None 32.4 1 3.4 2_3 25.3 4 and above 34.8 Missing/no response 4 Total 580

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 54

Trend in ANC attendance and number of visits made

The results of this study shows an increase in the proportion of women who received ANC from Skilled HWs and number of visits the mothers made compared with the results of EDHS 2005 and EGFHIE (table 16). Women who received ANC increased from 25 percent (EDHS 2005) and 40 percent (EGFHIE) to the current 65 percent. Further similar increment was observed for those respondents who reported to have visited Health facility 4 or more times, demonstrating that once again theses zones performed well above the national average of ANC attendance. Table 16 Comparison of women receiving ANC and number of visits made during EDHS 2005, EGFHIE 2008, and current Evaluation, Ethiopia, 2009 Current evaluation Types of providers EDHS EGFHIE 2005 (%) 2008 (%) 2009 (%)

Skilled HWs 27.6 47.9 65 No one 71.5 37.9 32 TBA 0.7 2.8 2 Total number 7307 2754 580 Current evaluation EDHS EGFHIE Total number of visit 2009 (%) 2005 (%) 2008 (%) None 71.5 37.9 32.4 1 4.6 2.4 3.4 2_3 11.3 16.6 25.3 4 and above 12.2 30.8 34.8 Missing/no response 0.4 12.3 4 Total number 7307 1371 580

4.2.3.2. DELIVERY CARE

Proper medical attention and hygienic conditions during delivery can reduce the risk of complications and infections that may cause the death or serious illness of the mother and the baby or both. Hence, an important component in the effort to reduce the health risks of mothers and children is to increase the proportion of babies delivered in a safe and clean environment and under the supervision of health professionals. Data on place of delivery and type of assistance during delivery was obtained for all births that occurred in the last five years.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 55

The majority of births (86 percent) were delivered at home; conversely, nearly thirteen percent of births were delivered in a health facilities. The proportion of births delivered in a health facility is exceptionally low in Illubabor (2.6 percent) while relatively high in West Wollega (32 percent), depicting disparity in the number of available service providers at their disposal or lack of awareness to use the services. There is also a moderate association between mother’s education and place of delivery. The proportion of births delivered in a health facility is 9 percent among uneducated mothers, compared with 27 percent among mothers with primary education (table 17).

Table 17 Percent of place of delivery and person providing assistance during delivery of live births in the five years by zone and level of education, Ethiopia, 2009.

Place of delivery

Health Facility Home Other Missing Total Public Private All 11.2 1.4 86.2 0.7 0.5 580 Zones East Wollega 9.2 0 89.3 0 1.5 131 Jimma 9 0 85.3 0.9 4.7 211 West Wollega 25.6 6.6 66.1 0.8 0.8 121 Illubabor 2.6 0 95.7 0.9 0.9 117 Level of Education No education 5.3 0.8 92.4 0.6 0.8 356 Only read & 15.4 2.9 80.8 1.0 0.0 104 Write Primary+ 24.1 1.7 73.3 0.9 0.0 116 Person providing assistance during delivery Health HEWs TBA Relative No one Missing Total Workers All 14.5 2.8 25.7 50.5 6.2 0.3 580 Zones East Wollega 9.2 3.9 3.1 80.2 3.1 0.8 131 Jimma 11.8 2.4 20.5 55 9.5 0.5 211 West Wollega 34.7 0.8 29.7 28.9 5.8 0 121 Illubabor 3.4 5.2 55.5 31.6 4.3 0 117 Level of Education No education 7.0 2.8 26.4 55.1 8.4 0.3 356 Only read & 24.0 1.9 21.2 50.0 2.9 0.0 104 Write Primary+ 27.6 3.4 27.6 37.9 2.6 0.9 116

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 56

Obstetric care from a trained provider during delivery is recognized as critical for the reduction of maternal and neonatal mortality. Births delivered at home are usually more likely to be delivered without assistance from a health professional, whereas births delivered at a health facility are more likely to be delivered by a trained health professional.

As shown in table 18, about 15 percent of births are delivered with the assistance of a trained health professional, that is, a doctor, nurse, or midwife and 26 percent are delivered by the traditional birth attendant. The majority of births are attended by a relative or some other person (51 percent). Six percent of all births are delivered without any type of assistance at all. More than one in three births (34.7 percent) in West Wollega were assisted by a trained health professional, compared with only 3.4 percent of births in Illubabor. Additionally, 80 percent of births in East Wollega were delivered with the help of a relative or some other person, compared with 29 percent of births in West Wollega. Births to women with at least primary education are almost three times (28 percent) more likely to receive delivery assistance from a health professional than births to women with no education (7 percent).

4.2.3.3. POSTNATAL CARE (PNC)

A large proportion of maternal and neonatal deaths occur during the 48 hours after delivery.

Thus, postnatal care is important for both the mother and the child to treat complications arising from the delivery, as well as to provide the mother with important information on how to care for herself and her child. Safe motherhood programs have recently increased emphasis on the importance of postnatal care, recommending that all women receive a check on their health within two days of delivery. To assess the extent of postnatal care utilization, respondents were asked for the last birth in the last five years whether they had received a health check after the delivery and the timing of the first check. Nearly 23 percent received postnatal care. Of all respondents who received postnatal care, only 11 percent received the care within the first two days of birth. Mothers in East Wollega were less likely to receive PNC compared to other project zones. Utilization of PNC is found to be significantly associated with education level of the women (p<0.005). Those who have some education are more likely to use than those with no education.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 57

Table 18 Percent of mothers delivered in the last five years who received PNC and time after delivery the first PNC received for the last live birth, Ethiopia, 2009.

Don’t Within 4 to 23 3 to 41 Received 2 days know/ Total (%) 4 hours hours (%) days (%) (%) (%) missing (%) All 22.9 7.4 3.6 0.5 7.1 81.4 580 Zones East Wollega 11.5 6.1 0.8 0.8 2.3 90 131 Jimma 22.7 6.6 2.8 0 10.8 79.8 211 West Wollega 34.7 14.9 10.7 0.8 6.6 67.0 121.0 Illubabor 23.9 2.6 0.9 0.9 6.8 88.8 117.0 Level of Education No education 15.7 2.2 3.1 0.6 5.6 88.5 356 Only read & 23.1 13.5 4.8 1 2.9 77.9 104 Write Primary+ 43.1 16.4 4.3 0 15.5 63.8 116

Unexpected outcome of the program (positive and Negative)

The evaluation also tried to asses whether there were unexpected outcome of the project on the other community health services related initiatives.

• Involvement of CBRHAs in multidisciplinary (other social and development) activities such as environmental sanitation; child immunization and tracing EPI defaulters, advocacy on gender issues, and other similar community health event. This has boosted the effort of HEWs in reaching more community.

• The IEC activity of the CBRHAs has led to greater awareness about RH issue and to utilize the health services.

• The other positive outcome is increased referral linkage of mothers, children, and others to get family planning, STI and HIV, ANC, delivery, post-natal care, and immunization services.

• Most of the CBRHAs are greatly changed their own health behavior and can be used as community health promoters (model households) for HEP

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 58

However there are reported negative outcome as well, such as:

• duplication of efforts between CBRHAs and HEWs and

• Competitive attitude of CBRHAs among themselves and with HEWs that mainly related with collection and retention of service charges.

The following four tables display a comparison between the presumed performance of the studied woredas as per stakeholder perceptions and the degree of achievement of outcomes compiled from the community survey.

Almost all the presumed low performance Woredas in each zone are performing better than the presumed low performing Woredas except Gomma Woreda compared by knowledge of Contraceptive methods (table 19). Similarly all the presumed low performance Woredas had achieved better than those categorized as high in terms of pregnant women who received ANC from Health Workers(table 21). Comparison by use of Contraceptive methods (table 20), delivery in health facility, delivery assisted by skilled birth attendants, and who received Postnatal care gave mixed pictures. The impressing issue in this result came for institutional delivery, delivery assisted by skilled birth attendants, and who received postnatal care (table 22). Percentage of mothers who received PNC in Tiro afeta woreda raised by 8 fold compared to the institutional delivery and delivery assisted by skilled birth attendants while similar picture were not observed in other woredas. This might be due to the fact that, these mothers might get the services not for their own sake but while visiting health facilities for other purposes such as for vaccinating their children.

Table 19. Comparison of High and low performance woredas by Knowledge of Contraceptive methods Any Performance Total Zones Woreda modern Pills Injectables Category Number method East Jimma Haro High 88 83 81 100 Wollega Abay Chomen Low 96 96 93 83 Seka Chekorsa High 80 78 76 108 Tiro Afeta Low 82 82 82 66 Jimma Gomma High 99 97 92 92 Manna Low 82 80 80 84 West Ayra Gulliso High 94 94 90 52 Wollega Najo Low 95 94 91 139 Bedele Dabo High 97 95 92 129 Illubabor Chawaka Low 100 96 96 51 904

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 59

Table 20. Comparison of High and low performance woreda by Ever-use, Use in the last five years and current use of Contraceptive methods

In CPR Performance Ever Total Zones Woreda last 5 among Category use Number years married

Jimma Haro High 36 29 26 100 East Wollega Abay Chomen Low 47 35 22 83 Seka Chekorsa High 34 20 31 108 Tiro Afeta Low 24 18 16 66 Jimma Gomma High 60 41 49 92 Manna Low 52 37 33 84 Ayra Gulliso High 62 54 30 52 West Wollega Najo Low 73 68 65.8 139 Bedele Dabo High 40 36 40 129 Illubabor Chawaka Low 65 65 57 51 904

Table 21. Comparison of High and low performance woredas by Percentage of pregnant women who received ANC from Health Workers

Performance ANC by Total Zones Woreda Category HWs Number Jimma Haro High 58 71 East Wollega Abay Chomen Low 70 60 Seka Chekorsa High 33 63 Tiro Afeta Low 38 37 Jimma Gomma High 80 56 Manna Low 87 55 West Ayra Gulliso High 68 41 Wollega Najo Low 88 80 Bedele Dabo High 63 72 Illubabor Chawaka Low 62 45 580

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 60

Table 22. Comparison of High and low performance woredas by percent of pregnant mothers who delivered in health facility (DHF), Delivery assisted by skilled birth attendants(DABA), and who received Postnatal care (PNC)

Performance received Total Zones Woreda DHF DABA Category PNC Number

East Jimma Haro High 7 14 11 71 Wollega Abay Chomen Low 12 12 12 60 Seka Chekorsa High 2 3 5 63 Tiro Afeta Low 3 3 24 37 Jimma Gomma High 18 27 34 56 Manna Low 13 22 31 55

West Ayra Gulliso High 12 20 17 41 Wollega Najo Low 43 44 44 80 Bedele Dabo High 4 10 26 72 Illubabor Chawaka Low 0 7 10 45 580

As it can be observed there are not only convergent and divergent classifications, but also, some woredas that were classified as high performance showed poorer indicators than those classified as low performance. This simple exercise highlights the importance of monitoring the program with evidences, since subjective judgment might lead managers and donors to misleading conclusions.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 61

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) V

CONCLUSIONAND RECOMMENDATION 63

V. CONCLUSION AND RECOMMENDATION

5.1. CONCLUSION

The evaluation found that the process of project implementation was conducted in the proper way as there was high community involvements in all stages of the project; agents were selected from and by the communities in which they live and serve; adequate linkages existed to health services for referral and clinical services, and the availability of supervision was available. However some problems were identified related to the planning process of the program and the way the supervision was undertaken. The planning seems to be guided by the availability of the funds and there were gaps in the way the supervision was conducted as it was not always regular and supportive. Additionally evidence from health professionals showed that there were occasions where there is joint planning but no joint implementation and at times where there is no joint planning but connected implementation. This suggests that there exist two different implementation strategies that might affect sustainability of the project.

The study found that most of the different capacity building activities of the health facilities and health workers, ODA RH/FP staffs, CBRHAs, and the establishment and support of in-and out of school youth clubs was accomplished. Training of the CBRHAs used practical methods which were appropriate for the community based agents. Although substantial efforts were observed in capacity building, some limitations remain to be a significant inhibitor of greater success.

• The language in which the training material prepared are not proper,

• The duration of the training for CBRHAs were for 15 days, which the CBRHAs claimed was too short,

• Lack of adequate training on counseling process that is manifested with poor quality household consultation and counseling service provided by the CBRHAs.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 64

Large numbers of people were reported to be reached through IEC activities on selected components of reproductive health. The result of household survey also showed that CBRHAs being the most important source of information on FP in all the project zones. However, there is a paucity of effectiveness of the IEC approach. The project is not developing its own IEC materials but collects and distributes all IEC materials on FP/RH developed by other organization. There is no an in-depth analysis of the problem and no pre-test of the materials for its comprehension, aesthetic appeal, and cultural acceptability.

ODA’s ARH clubs have benefited youths a lot in RH and is an appropriate approach for this group of population. It fosters open discussions and free expression of ideas among the youth. However, these ARHS clubs and services are limited only to small towns and the majority rural youths are inaccessible.

This evaluation shows a considerable increase in the knowledge and practice of family planning methods among women in the project areas. There is a considerable increase in the ever use and current use of contraceptive methods among both all women and currently married women and high proportion of women used the contraceptives within the last five years. Furthermore, the proportion of women who married before the age of 15 and those who married between the ages of 15- 17 years showed reduction by 5% in the project zones.

However, knowledge about condom is low, even less than the national estimates of EDHS 2005. Further, ever use and current use of contraceptive methods as well as utilization of maternal health services is higher in Women from West Wollega and Illubabor than those in Jimma and East Wollega. This might be related to the fact that the CBRHAs in these two zones might have been fade up with the job as they have been serving voluntarily for more than eight years compared to the others who served less.

The evaluation reveals a large increase in utilization of maternal health services such as antenatal care, delivery care, and postnatal care. Even though there is a significant improvement, the proportion of mothers who gave birth in health facility is still low, only 13 percent of births were delivered in a health facility. This study also shows that uneducated women were less likely to use health facilities for antenatal care, delivery assistance, and postnatal care.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 65

Analysis of both the service reports and household survey shows a shift in the family planning demand from non-clinical contraceptive to the clinical contraceptive. Comparison of trend of new users by type of contraceptive methods over the years (based on service reports) shows a decline trend in OCP and Condom users but an increasing trend in Injectibles and Norplant users. The survey result also shows similar situation. Therefore this is an indication that distribution of non-clinical contraceptives by CBRHAs would be of little use, hence it would be better if the CBRHAs focused on health promotion in the community.

5.2. RECOMMENDATION

1. The success of this project is mainly related to high level of stakeholders’ involvement; particularly the local community involved in all stages of the project and this has to be replicated in all other community based initiatives.

2. The planning and monitoring processes should have baseline information and measurable targets against which progresses could be compared.

3. Supervision to the CBRHAs is found to get less regular and less supportive and become more of control. A regular and supportive supervision with proper feedback, which motivate agents to perform better, is recommended.

4. The training materials for CBRHAs training were found to be in English and Amharic while most of the CBRHAs can not understand both. It is recommend that training and training materials for any community based agents and volunteers prepared in local language.

5. The IEC activities that aim to bring about the intended behavioral change using different channels should be supported by production of IEC materials based on an in-depth analysis of the problem, pre-test of the materials for its comprehension, aesthetic appeal, and cultural acceptability. Further the program needs to make clear the rationale for the intended behavioral change.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 66

6. ARHS clubs and services are limited only in small towns and the majority rural youths are inaccessible. All adolescent and youth should have access to the same information and resources so that it will help to achieve the reproductive health related goals. Such services should possibly be expanded to all the rural youths.

7. Knowledge about condom in this part of the country is less than the national estimates which might have serious implication on family planning, STI and prevention of HIV. Therefore, promotion of condom use shall be one of the IEC activities in these zones.

8. There are variations between zones in the outcome indicators; Zones where the project operated for longer periods achieved lower compared to the others. This might be related to low moral and fatigue of the CBRHAs as they are serving voluntarily for more than eight years compared to the others. We recommend the organization better recognize that exclusively volunteer schemes do not work and seek concrete motivating schemes for the CBRHAs.

9. Quality consultation and counseling skill is crucial to recruit and retain clients. The poor quality household consultation and counseling service provided by the CBRHAs is related to lack of adequate training on counseling process. Therefore, adequate training that equips the community health workers to communicate effectively with clients and potential clients is recommended.

10. The declined acceptance of non-clinical contraceptive and shift to the clinical contraceptive shows that distribution of non-clinical contraceptives by CBRHAs is less feasible. These dictates it would be better if the CBRHAs focused on education of the community on the defined community based health services in collaboration with the Health extension workers.

11. This study found that CBRHAs are important supporters of HEWs evenwith the competitive attitude they have on some of their activities. Using them as model families will have a great potential to sustain the contribution of these experienced community members for the provision of community based RH services in the absence of external support. Though this could have been considered as one of the major activities during the consolidation phase of the project, still there are opportunities to build strong bond between HEWs and CBRHAs.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 67

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) VI

REFERENCES 69

VI. REFERENCES

1. Ronsmans C, Graham WJ: Maternal mortality: who, when, where and why. Lancet. 2006, 368(9542):1189-1200. PubMed Abstract | Publisher Full Text

2. World Health Organization: The world health report 2005: Make every mother and child count. Geneva: WHO; 2005.

3. United Nations: Programme of Action. United Nations International Conference on Population and Development. UNFPA; 1994.

4. United Nations: The Millenium Development Goals Report 2006. New York 2006.

5. World Health Organization: Maternal Mortality in 2000: Estimates Developed by WHO, UNICEF and UNFPA. World Health Organization; 2004.

6. Katz KR, West CG, Doumbia F, Kane F: Increasing access to family planning services in rural Mali through community-based distribution. Int Fam Plan Perspect 1998, 24:104-110.

7. Gillespie D, Ahmed S, Tsui A, Radloff S: Unwanted fertility among the poor: an inequity? Bull World Health Organ 2007, 85:100-1007. PubMed Abstract

8. Central Statistical Agency. Summary and statistical report of the 2007 Population and Housing Census, FDRE Ethiopia Population Census Commission. Addis Ababa, Ethiopia; December 2008

9. Central Statistical Agency, ORC Macro. Ethiopia 2005 Demographic and Heath Survey. Addis Ababa, Ethiopia; Calverton, Maryland; 2006.

10. Central Statistical Agency, ORC Macro. Ethiopia 2000 Demographic and Heath Survey. Addis Ababa, Ethiopia; Calverton, Maryland; 2001.

11. World population Data Sheet of the Population reference Bureau. Washington DC. 2007.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 70

12. Betran AP. Methodological issues in the measurement of maternal mortality and morbidity. Department of making pregnancy safer, WHO. 2004.

13. World Bank. MDG: Reduce by three-quarters, between 1990 and 2015, the maternal mortality ratio. Development Report. 2003. pp. 208–212.

14. Regassa N. Socio-economic Correlates of High Fertility among Low Contraceptive Communities of Southern Ethiopia. J Hum Ecol. 2007;21:203– 213.

15. Fantahun M, Kumb S, Degu G, Kebede Y, Admassu M, Haile W, Hailu S., Dabat Rural Health Project, North West Ethiopia. Report of the baseline survey. Ethiop J Health Dev. 2001;15

16. FMOH, HSDP III

17. Planning and Programming Department, Ministry of Health of Ethiopia. Health and health related indicators, 1999 EC (2006/07). Addis Ababa; 2008.

18. A study of reproductive health issues and knowledge, attitude, and practice of family planning methods, Oromia regional state, Ethiopia in 1999; February 2001

19. FMOH and EHNRI, Ethiopian Global Fund Health Impact Evaluation, Addis Ababa, Ethiopia, 2008

20. FMOH, Essential health Service Package of Ethiopia. Addis Ababa, Ethiopia, 2005.

21. Planning and Programming Department, Ministry of Health Of Ethiopia. Health and health related indicators, 2000 EC (2007/08). Addis Ababa; 2007.

22. FMOH, Family Planning Extension Package. Addis Ababa, Ethiopia, 2003

23. FMOH, Adolescent Reproductive Health Extension Package. Addis Ababa, Ethiopia, 2003

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 71

24. FMOH, Maternal and Child Health Extension Package. Addis Ababa, Ethiopia, 2003

25. FMOH, HIV/AIDS and Tuberculosis Prevention and Control Extension Package. Addis Ababa, Ethiopia, 2004

26. FMOH, Health Education and Communication Extension Package. Addis Ababa, Ethiopia, 2004

27. FMOH /EHNRI, Impact Evaluation of Ethiopia’s National Response to HIV/ AIDS, Tuberculosis and Malaria. Addis Ababa, Ethiopia, 2008

28. An Evaluation of Oromia Development Association (ODA) Reproductive Health/Family Planning Project, Proposal to the David and Lucile Packard Foundation, TUTAPE, September, 2008

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) VII

ANNEXES 73

VII. ANNEXES

ANNEX I. List of documents received from ODA RH/FP project coordinating office and used for document review.

The project proposals, plans, and reports are the compiled (are the aggregate) by Regional ODA RH/FP coordinating office. No separate zonal plans and reports but the regional plans and reports shows zonal breakdown. No document for Woreda level plan and reports. The documents received and used for evaluation are categorized below.

Project proposals

• Project development plan for integrated community based RH program in Jimma and east Wollega zones for January 2000-December 2004, September 1999

• Proposal to the David and Lucile Packard Foundation for Oromia Development Association In The Expansion And Consolidation of Integrated Community Based Program in Four Zones of Oromia Regional State Second Submission, March 2002

• ODA-CBRH Project October 2004- September 2005

• ODA-CBRH Project January 01, 2006 - December 2008

• Proposal on Project Amendment (January –September 2008)a nine month reproductive health/family-planning project amendment to be implemented in 49 Woredas of Oromia national regional state by Oromia Development Association

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 74

Annual POA

• Annual Plan of Action for East Wollega and Jimma Zones for 2001

• Annual Plan of Action for the year 2003-2004, August 2003.

• Annual Plan of Action for the Year 2006

Project reports

• July-December 2000 (for Jimma & East Wollega zones only)

• January 01-June 01(for Jimma & East Wollega zones only)

• July 01-December 01(for Jimma & East Wollega zones only)

• January 02-June 02(for Jimma & East Wollega zones only)

• July 02-December 02(for Jimma & East Wollega zones only)

• January 03-June 03(for Jimma & East Wollega zones only)

• July 03-December 03(for Jimma & East Wollega zones only)

• January 04-June 04 (for four Zones)

• July 04-December 04 (for four Zones)

• January 05-June 05 (for Illubabor & West Wollega Zones)

• July 05-December 05(for Illubabor & West Wollega Zones)

• January 06-June 06(for Illubabor & West Wollega Zones)

• July 06-June 07(for Illubabor & West Wollega Zones)

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 75

• July 07-June 08(for Illubabor & West Wollega Zones)

• October 2004-June 2008-Compiled (for East Wollega zone only)

• October 2004-June 2008-Compiled (for Jimma zone only)

Trainers manual

• Trainers manual: Training in Management Information System (MIS) For Community Based Reproductive Health Program

• Facilitative supervision Trainers manual

Related researches undertaken in the area

• A study of Reproductive Health Issues and knowledge, attitude and practice of family planning methods, Oromia Regional state, February, 2001.

• A study of Reproductive Health Issues and knowledge, attitude and practice of family planning methods, Amhara Regional state, 2000.

• ODA-RH/FP project-Background paper-April 19, 2008

• Report of baseline survey on Female genital mutilation in Seka Chekorsa district, September 2006

• Community based RH survey, 2004 (no cover page), 2004

• Trends in Youth Reproductive Health in Ethiopia, 2000 and 2005, April 2008

• Improving Lives and Slowing Population Growth, Six years of International Grant making in Reproductive Health

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 76

ANNEX II. Ranking of woredas based on their performance by ODA for the purpose of this evaluation

West Wollega Zone Sr Name of Woreda Performance Category No 1 Anfilo 2 Seyo 3 Ayra Guliso 4 Jarso Higher 5 Gawo Dale 6 Begi 7 Dale Sadi 8 Ganji 9 Haru 10 Mana Sibu 11 Bodji Medium 12 Lalo Kile 13 Nole Kaba 14 Jima Gidami 15 Najo 16 Hawa Welel Lower 17 Yubdo 18 Jima Horo Illubabor Zone Sr Name of Woreda Performance Category No 1 Metu 2 Chora 3 Bedele Dabo Higher 4 Darimu 5 Borecha 6 Halu Bure 7 Yayu Hurumu 8 Alge Sachi Medium 9 Dedesa

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 77

10 Gechi 11 Ale Didu Lower 12 Chawaka 13 Dega Meko Jimma Zone Sr Name of Woreda Performance Category No 1 Seka Chekorsa 2 Dedo Higher 3 Gomma 4 Kersa 5 Gera 6 Limmu Seka 7 Limmu Kossa Medium 8 Sentema 9 Omo Nada 10 Sokoru 11 Sigmo 12 Tiro Afeta Lower 13 Manna East Wollega Zone Sr Name of Woreda Performance Category No 1 Jimma Arjo 2 Jimma Horro Higher 3 Nunu Kumba 4 Jimma Genet 5 Gida Kiremu 6 Limmu 7 Abe Dongoro Medium 8 Guduru 9 Jimma Rare 10 Abay Chomen 11 Jardega Jarte Lower 12 Ibantu 13 Amuru

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 78

ANNEX III. Discussion guide used to facilitate of Focus Group Discussions with CBRHAs Introduction

Dear FGD coordinator and facilitator,

Funded by David & Lucie Packard Foundation, Oromia Development Association has been implementing a Reproductive Health/Family Planning Project in four Zones of Oromia National Regional State. To learn from past experiences and improve future project planning and implementation, the foundation has realized the need to conduct an evaluation. The objective of the evaluation is to assess the strengths, weaknesses and results of the Packard Foundation funded RH/ FP project implemented by ODA in four zones of the Oromia National Regional State.

Community Based Reproductive Health Agents who participated in the project are identified as key informants about the project implementation. As partof the methods the evaluation requires to assess the perception of CBRHAs on the program and on factors that facilitate and are barriers to the achievement of the objectives.

Thus, the objectives of the FGD are: to generate information on existent status of CBRHA; to look at availability of RH services; and to assess facilitators and barriers to their expected activities. With this purpose, the main themes of discussion are identified to be used as a guide for facilitation.

The themes are just to initiate and motivate discussion, please encourage participation supporting those with difficulties to express themselves.

Please remember that every session needs to be recorded. This session is expected to last from one to one and half an hour so have batteries ready for a two hour session.

Your role is to facilitate and stimulate the discussion among the members. Make sure that your moderator takes notes on everything important that happens in the session including characteristics of the setting, participant body language, behavior and personnel characteristics of participants etc.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 79

Themes for Focus Group Discussion

Theme 1: Can you describe to us your every day activities related to the RH program?

Probe: Expected program activities, such as: home visits (four a year to each household); conduct at least one GD per week (minimum twice a month 50 participants per group discussion); provide non clinical contraceptives through door-to-door visits; refer clients to nearby HF (long acting and permanent methods and medical and gynecological consultations; interact to HEW)

Theme 2: Can you describe to us your training activities? What you see as major strengths and weakness?

Probe: Who provide it, frequency, how it is provided; what is the best thing about it, the worst;

Theme 3: Can you describe to us your routine of supervision?

Probe: Who provide it, frequency, how it is provided; what is the best thing about it, the worst;

Theme 4: Can you describe to us what you see as major problem to your activities as CBRHA?

Probe: Lack or shortage of IEC materials; lack or shortage of Contraceptives; lack of supervision, referral, supplies, community acceptance; client’s fear of judgment, client’s shyness and discomfort (of being seen ); lack of matching between sex and age of CBRH and client’s age and sex; sex education (biased to female only);

Theme 5: How do you see relationship between the project and the public health sector?

Probe: Are you working togther with the public health sector for the referrals? How does it work? How is your relationship with the local HEWs related to RH issues? Do you meet regularly with HEWs? Do the HEWs accept your referral?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 80

Theme 6: Can you describe to us how you see the community’s acceptance to your activities as CBRHA?

Probe: Excellent, good, no. Language, matching between sex and age of CBRHA, client’s age and sex, religious resistance, networking; community support; range of FP methods provided.

Theme 7: What do you feel is the result of this CBRH project in your kebeles? Probe: decrease in unwanted pregnancies; increase in the interval between pregnancies; increase of CPR; decrease in STD, increase in ANC coverage.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 81

ANNEX IV. Discussion guide for Focus Group Discussion with Adolescents

Funded with the support of David and Lucy Packard Foundation, Oromia Development Association (ODA) has been implementing a Reproductive Health/ Family Planning project in four zones of Oromia National Regional State. To learn from past experiences and to improve future project planning and implementation, the foundation has realized the need to conduct an evaluation. The objective of the evaluation is to assess the strengths and the weakness and achievements of the Packard Foundation funded RH/FP project implemented by ODA in four zones of the Oromia National Regional State.

As part of the methodology the evaluation requires to assess the perception of Adolescents (both in school and out of school) on the program and on factors that facilitate and are barriers to the implementation of the objectives. Thus, the objectives of focus group discussion are: to generate information on the implementation of adolescent reproductive health programs mainly prevention of STI, HIV and AIDS, other reproductive health components such as family planning.

Dear FGD facilitators

Please ensure the following condition

• The Focus Group Discussion should with the ADOLESCENTS and not with the “managers” of the youth program.

• Each FGD has not more than ten participants

• Ensure all participants filled the Focus Group Discussion Report Form

• There should be a separate FGD for male and female

• Encourage maximum participation of all group members

• Record the discussion using tape recorder and at the same time document responses

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 82

Focus Group Discussion Report Form Name of Zone: ______Name of Woreda:______FGD #:______Sex of FGD group: 1. Female 2. Male Date FGD conducted: ______FGD Start Time:______FGD End Time: ______Name of FGD Moderator:______Name of FGD Note Taker: ______Number of participants: ______

Comments and Observations:

1. Can you describe what the ODA supported ARHS is? Probe: Prevention of STI, HIV and AIDS, other reproductive health

2. What do you think about the materials used? Probe: creativeness of media, Do they have the chance to share their ideas to change the materials or not?)

3. Why do you come to the youth clubs? Is it easier to hear about FP/RH here than in the health facility? Why?

4. How do you perceive the accessibility of youth (adolescent) services?

5. Tell me what you get from program (something you have learned from the adolescent reproductive health programs) that you can apply to your life.

6. What do you think is the contribution of this project for you and the community? (in creating environments supportive of behavior change, if it makes any difference and if so in what ways, etc?)

7. Do you discuss issues/information you acquired through the ODA ARHS with your family members such as parents or with your brothers/sisters

8. What component do you like most? Why?

9. What components do you disliked most? Why?

10. What are the barriers and facilitator of this program?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 83

ANNEX V. Guide for Expert Interview –Health Offices Introduction:

Good morning/afternoon; I am ______from Tulane University - Ethiopia. Funded by David & Lucie Packard Foundation, Oromia Development Association has been implementing a Reproductive Health/Family Planning Project in collaboration with the public health sector in four Zones of Oromia National Regional State. One of the Zones/Woredas is ______. To learn from past experiences and improve future project planning and implementation, the foundation has realized the need to conduct an evaluation. The objective of the evaluation is to assess the strengths, weaknesses and results of the RH/FP project in four zones of the Oromia National Regional State.

Experts who participated in the project, both from the public health sector and project coordination office, are identified as key informants about the project implementation. Therefore, I would like to ask you some questions. None of these questions are personal and your responses will be kept confidential. I will be recording our conversations to facilitate the data analysis process and the recorded tape will be accessed only for this purpose.

Instruction for Interviewer

• Use the above introduction to introduce yourself and start the interview.

• This is only an interview guide; allow the interviewee to speak whatever is important to him/her regarding the issue that is raised.

• Record the interview using tape recorder and at the same time document responses.

• After each interview, revise your notes and listen to the tape to prepare a final report of the interview.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 84

Identification

• Name of Interviewee ______

• Organization ______Health Office

• Responsibility ______

• For how long have you participated in this project? ______

• Are you currently participating in this project? ______

Interview

1. Involvement of stakeholders: How was your involvement in the planning and implementation of the project? Do you think that others who are relevant have been involved in the planning process?

___(ODA, Zonal Health Bureaus, Woreda health Offices (professionals), Health Centers (professionals), HEWs, Community Leaders, Local Religious Leaders, students, educational bureau, NGOs, university partners, etc) Trainings:

a Have you been trained as a trainer of CBRHAs? 1. Yes 2. No

b Would you please describe the quality of the training you took? (probe for duration, frequency/refresher trainings, content, trainers, training materials, accommodation)

c Have you participated in training CBRHAs? 1. Yes 2. No

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 85

d Would you please describe the quality of the training given to CBRHAs? (probe for duration, frequency/refresher trainings, content, trainers, training materials, accommodation)

2. Resources: how do you describe the adequacy and timely availability of resources for CBRHA during the project period? (probe for types of contraceptives available, stock out, communication about logistics)

3. Would you please tell us the role of CBRHAs in RH service provision in your catchment? In what specific activities are they engaged in?

4. How was the acceptability and utilization of services provided by CBRHAs? What are the limiting factors that determine the utilization of services provided by CBRHAs?

5. Could you please comment on the quality of RH/FP services that CBRHAs have been providing to the community?

6. Referral system

a How do you rate the referral linkage between CBRHAs and Health Posts/ Health Centers for family planning services? 1. Very strong 2. Strong 3. Not bad 4. Poor 5. Very poor

b Would you please explain why? What do you think are the factors that made the referral linkage so?

7. Supervision: How do you provide supervision to CBRHAs? (probe for who supervises, frequency of supervision, methods, training roles, supportiveness)

8. Project effectiveness: How do you describe the role of this project in improving access to RH/FP services to the rural community? How do you see the appropriateness of having CBRHAs in the presence of HEWs?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 86

9. In general, what were the strengths of this project that need to be duplicated in the future?

10. What were the weaknesses of this project that need to be improved in future similar projects?

11. What were the factors that contributed to the successes and/or failures of the project?

12. What do you recommend for future improvement of similar projects?

13. Any other thing you want to speak about the project

Name of Interviewer ______

Signature ______

Date of Interview ____/_____/______

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 87

ANNEX VI. Guide for expert interview –ODA coordinators Introduction:

• Use the above introduction to introduce yourself and start the interview. • This is only an interview guide; allow the interviewee to speak whatever is important to him/her regarding the issue that is raised. • Record the interview using tape recorder and at the same time document responses. • After each interview, revise your notes and listen to the tape to prepare a final report of the interview. Identification

1. Name of Interviewee ______2. Organization ______3. Responsibility ______4. For how long have you been working for this project? ______

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 88

Interview

a. Involvement of stakeholders: How was stakeholders’ involvement in the planning and implementation of the project? Do you think that all who are relevant have been involved in the planning process? How?

b. Trainings:

a. Have you been trained on reproductive health (as a trainer, supervisor, coordinator) of CBRHAs? i. Yes ii. No

b. Would you please describe the quality of the training you took? (probe for duration, frequency/refresher trainings, content, trainers, training materials, accommodation)

c. What was your role in training CBRHAs?

d. Would you please describe the quality of the training given to CBRHAs? (probe for duration, frequency/refresher trainings, content, trainers, training materials, accommodation)

c. Resources: how do you describe the adequacy and timely availability of resources for CBRHAs during the project period? (probe for types of contraceptives available, stock out, communication about logistics)

d. Would you please tell us the role of CBRHAs in RH service provision in your catchment? In what specific activities are they engaged in?

e. How was the acceptability and utilization of services provided by CBRHAs? What are the limiting factors that determine the utilization of services provided by CBRHAs?

f. Could you please comment on the quality of RH/FP services that CBRHAs have been providing to the community?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 89

g. Referral system

a. How do you rate the referral linkage between CBRHAs and Health Posts/ Health Centers for family planning services? i. Very strong ii. Strong iii. Not bad iv. Poor v. Very poor b. Would you please explain why? What do you think are the factors that made the referral linkage so? c. Supervision: How do you provide supervision to CBRHAs? (probe for who supervises, frequency of supervision, methods, training roles, supportiveness) d. Project effectiveness: How do you describe the role of this project in improving access to RH/FP services to the rural community? How do you see the appropriateness of having CBRHAs in the presence of HEWs? e. In general, what were the strengths of this project that need to be duplicated in the future? f. What were the weaknesses of this project that need to be improved in future similar projects? g. What were the factors that contributed to the successes and/or failures of the project? h. What do you recommend for future improvement of similar projects? i. Any other thing you want to speak about the project

Name of Interviewer ______Signature ______

Date of Interview ____/_____/______

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 90

ANNEX VII. In-depth interview guide for health workers Introduction

Good morning/afternoon, my name is ______and I represent Tulane University-Ethiopia. We are conducting Evaluation of ODA-RH/FP program within (______Zone/woreda). The purpose of this Evaluation is to learn the level of implementation of ODA-RH/FP program that can be used to improve the program in the future. This information will help us understand the strengths, weaknesses and results of the Packard Foundation funded RH/FP project implemented by ODA in four zones of the Oromia National Regional State.

Regarding this, I would like to ask you some questions. None of the questions are personal, and the answers you give will not be shown to anyone. They will only assist us in learning more about the level of implementation of ODA-RH/FP program.

Do I have your permission to continue? (Circle one.) 1 = Yes 2 = No (End the interview)

(Interviewer arranges for a private setting to conduct interview)

Time Start: ______

Time End: ______

Questionnaire ID # Address: Name of Zone/ Woreda ______/______Name of Health Facility Type of Health Facility A. Health Center, B. Hospital Telephone number of the Health Facility Date of Data Collection Name of Data Collector Signature Name of Supervisor Signature

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 91

1. Trainings:

a. Have you ever been received training of trainers of ODA-RH/FP programs’ teaching methods based on CBRHA curriculum? 1. Yes 2. No

b. Would you please describe the quality of the training you took? (probe for duration of the training, frequency/refresher trainings, content of the training, who were trainers, training materials, accommodation)

c. Have you participated in the training of the CBRHAs in your catchment’s area? 1. Yes 2. No

d. Would you please describe the quality of the training given to CBRHAs? (probe for duration, frequency/refresher trainings, content, trainers, training materials, accommodation)

2. supervision

a. Do you supervise CBRHAs? 1. Yes 2. No

b. Would you please describe the quality of Supervision to CBRHAs? (probe for frequency, supervision checklist, feedback system, institutionalization,)

c. comment on the positive side of the supervisory system

d. comment on what to improve on the current supervisory system

3. Referral system

a. Does this facility have referral linkage with CBRHAs for FP/RH activities? 1. Yes 2. No

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 92

4. Would you tell me how the referral system is arranged and working? (probe for two way referral, modality [using referral paper/slip, verbally/orally], special arrangement, feedback, logistic arrangement etc)

a. comment on the positive side of the current referral system

b. comment on what to improve on the current referral system

5. In general, what were the strengths of this project that need to be duplicated in the future?

6. What were the weaknesses of this project that need to be improved in future similar projects?

7. What were the factors that contributed to the successes and/or failures of the project?

8. What do you recommend for future improvement of similar projects?

9. Any other thing you want to speak about the project

10. What do you feel about having CBRHAs at the community level during the period when the health system is deploying two HEWs for each Kebele? Do you feel that it is appropriate or just duplication of effort? Why? How?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 93

ANNEX VIII. Questionnaire for household survey

Introduction

Good morning/afternoon, my name is ______and I represent Tulane University-Ethiopia. We are conducting Evaluation of ODA-RH/FP program within (______Zone/Woreda).

The purpose of this survey is to learn what people in the reproductive age group understand about reproduction, contraception, sexually transmitted infections, HIV and AIDS. This information will help us understand the health needs of people in the reproductive age group and the strengths, weaknesses and results of the Packard Foundation funded RH/FP project implemented by ODA in four zones of the Oromia National Regional State.

Regarding this, I would like to ask you some questions. Some of the questions are personal, but the answers you give will not be shown to anyone and no names will be in the report. They will only assist us in learning more about how the reproductive health program works. We especially want your answers because if everyone who is selected participates, our information will be more useful and will tell us how people think about the program.

Do I have your permission to continue? (Circle one.)

1 = Yes 2 = No (End the interview)

This Questionnaire is to be administered to a female household member who aged 15-49 years. Ask for a female household member whose age is more than 15 years but less than 49 years. (Those who are younger than 15 years or older than 49 years cannot be interviewed)

(Interviewer arranges for a private setting to conduct interview)

Time Start: ______

Time End:______

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 94

A. GENERAL: ONLY FOR DATA COLLECTORS AND SUPERVISORS

Questionnaire ID #

Address: Zone/Woreda/ Kebele/H.# ______/______/______/_____

Area of resident a. Rural b. Urban

Date of Data Collection

Name of Data Collector

Signature of Data Collector

Name of supervisor

Date data checked

Signature of supervisor

Completed=1 Partially completed=2 Questionnaire Status Interrupted=3

B. SOCIO-DEMOGRAPHIC INFORMATION

Instruction: Read the questions audible and circle the appropriate response; for questions 1 and 3, circle the appropriate response and enter age in years.

I would like to ask you some questions related to your socio-demographic information.

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 95

SN QUESTIONS CODING CATEGORIES

What is your age? 1. Age of respondent in year’s _____ 1 2. Don’t know 1. Married 2. Divorced 2 Marital status 3. Widowed 4. Separated 5. Never married (Skip to Q. 4) If married, ask what was her age at first 1. Age at first marriage in year’s _____ 3 marriage? 2. Don’t know

1. Age at first intercourse in year’s 4 What was your age at first intercourse? _____ 2. Don’t know

1. No education 2. Only writing and reading What is the highest level of Education of the 5 3. Primary respondent 4. Secondary 5. Higher

1. Orthodox 2. Catholic 6 What is your religion? 3. Protestant 4. Muslim 5. Other (specify)______

1. Oromo 2. Amhara 3. Guragie What is your Ethnicity? 7 4. Sidam 5. Tigrie 6. Welaita 7. Other (specify)______

1. Housewife 8 What is your role/ Position in the household? 2. Household head 3. Other Household member

Do you or some one in the household have 1. Yes 9 Radio? 2. No

10 What is your Occupation? Occupation______

1. Yes 11 Do you have cropping land? 2. No

1. Yes 12 Do you have grazing land? 2. No 1. Yes 13 Do you have farm animals? 2. No

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 96

C. RH/FP

C.1. FP/ CONTRACEPTION METHOD

Instruction: Read the questionnaire audible. Circle the appropriate response for the structured questions and write the response of an interviewee in clear handwriting for the open ended questions.

Now, I would like to ask you some questions about FP/ contraception method. I would like to remind you that the information you provide will remain strictly confidential,

SN QUESTIONS CODING CATEGORIES

Have you ever heard of ways to prevent 1 Yes 14 pregnancy? 2 No (skip to Q 26)

Ask all the methods of contraception (ways to prevent pregnancy) that she had heard of and 15 probe for methods not mentioned spontaneously. (Circle the appropriate response for all listed)

Methods Yes No Not sure 1. Pill 1 2 3 2. IUD 1 2 3 3. Injectables 1 2 3 4. Condom 1 2 3 5. Implants 1 2 3 6. Lactational amenorrhea (LAM) 1 2 3 7. Rhythm 1 2 3 8. Withdrawal 1 2 3 9. Sexual abstinence 1 2 3

Have you ever used of any contraception 1 Yes 16 method? 2 No (Skip to Q. 24)

Have you ever used of any contraception 1 Yes 17 method within the last five years? 2 No (Skip to Q. 24)

1. Pill 2. IUD 3. Injectables If yes, would you tell me all the methods of 4. Condom contraception that you have used within the 18 5. Implants last five years? 6. Lactational amenorrhoea (LAM) (Circle all that apply) 7. Rhythm 8. Withdrawal 9. Sexual abstinence Currently, are you using any contraception 1 Yes (ask Q 20-to-22, but Skip Q 23-to 25) 19 method? 2 No (Skip to Q. 23)

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 97

SN QUESTIONS CODING CATEGORIES

1. Pill 2. IUD Would you tell me the methods of 3. Injectables 20 contraception that you are using Currently? 4. Condom 5. Implants 6. Lactational amenorrhoea (LAM) 7. Female sterilization

(If, the current method is either Pill, IUD, Injectables or Implants) ask if she were informed 21 about the following two issues at the time she were adopting a particular method:

1 Yes Were you informed about the possible side effects 21a 2 No or problems you may have with the method? 3 Don’t remember

1 Yes Were you informed what to do if you experienced 21.b 2 No side effects? 3 Don’t remember

1. Government hospital 2. Government health centre 3. Government health post/ clinic Would you tell me where you obtained the 4. CBRHAs (CBD worker) method the last time?(Most recent source of 22 5. Other public method) 6. Private hospital/clinic 7. Pharmacy/drug shop or vendor 8. NGO Health facility 9. Shop

1. Became pregnant while using 2. Wanted to become pregnant 3. Husband/partner disapproved 4. Side effects 5. Health concerns 6. Lack of access/too far If she is not using contraceptive currently 23 7. Inconvenient to use (Respond no to Q 19), ask why she discontinued? 8. Infrequent sex/husband away 9. Cost too much 10. Method not available 11. Difficult to get pregnant /menopausal 12. Marital dissolution/separation 13. Other

If she is not using contraceptive currently 1 Yes, she is Intends to use Skip to Q26 24 (Respond no to Q. 19), ask about here intention to 2 Unsure Skip to Q26 use family planning methods in the future. 3 No, she does not intend to use

25 If (Respond no to Q 24), ask why? ______

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 98

SN QUESTIONS CODING CATEGORIES

1. Just before the period begins, Could you tell me if there are certain days 2. During the period, between two menstrual periods when a woman is 3. Right after the period ends, 26 more likely to become pregnant if she has sexual 4. Halfway between the two periods intercourse? 5. Other (specify) 6. Don’t know 1. Radio 2. Television What are the most common sources of 3. Newspaper/magazine/ brochure information regarding Family planning for you? 4. CBRHAs 27 5. HEWs (circle all the sources she responded) 6. Friends & Family 7. Others (specify) 8. None

C2. MATERNAL CARE

Instruction: Questions 28-38 are all about Antenatal, Delivery, and postnatal care. Read the questions audible and Circle the appropriate response.

Now, I would like to ask you some questions about Antenatal, Delivery, and postnatal care. I would like to remind you that the information you provide will remain strictly confidential,

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 99

C2.1. ANTENATAL CARE

SN QUESTIONS CODING CATEGORIES Have you given birth to a child 1 Yes 28 within the last five years? 2 No Skip to Q. 30 What is the total number of 1. Number of children_____ 29 children (sons and daughters) 98 No response born to you in the last five years? When was your last (most recent) ______Date of last pregnancy 30 pregnancy? 1. Yes, health worker Have you attended ANC (during 31 2. Yes, TTBA most recent birth)? 3. No 1. Number of months pregnant at time of first If attended, which month of ANC visit ____ 32 pregnancy you first made ANC 2. Don’t remember visit? 98 no response)

If attended, what was the total 1. Number of total ANC visit ____ 33 number ANC visits you made 2. Don’t remember/ during this pregnancy? 98 No response

C2.2. DELIVERY CARE

SN QUESTIONS CODING CATEGORIES

When was the date your last (most _____ Date of last birth 34 recent) birth?

1. Home Where did you give birth to your 2. Health facility (hospital/ HC) 35 (most recent birth) last baby? 3. Health facility (private clinic) 4. No children 5. Other

1. Health workers (doctor, nurse, or midwife) During delivery to your (most recent 2. traditional birth attendant 36 birth) last baby, who assisted you? 3. a relative or some other person 4. No assistance at all. 5. Don’t know/ Don’t remember

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 100

C2.3. POSTNATAL CARE

SN QUESTIONS CODING CATEGORIES

Have you received a health check after the 1 Yes 37 delivery for your most recent birth from a 2 No Skip to Q. 39 health worker? 3 Don’t remember Skip to Q. 39

1. Less than 4 hours What was the time after delivery you have 2. 4-23 hours 38 the first postnatal checkup 3. 2 days 4. 3-41 days 5. Don’t remember

D. CBRHA ACTIVITIES

Instruction: Read the questionnaire audible. Circle the appropriate response for the structured questions and write the response of an interviewee in clear handwriting for the open ended questions.

Now, I will ask you questions related to the activities of CBRHAs in this community.

SN QUESTIONS CODING CATEGORIES

Do you know someone who teaches the community on RH/FP and distribute contraceptives? 1. Yes 39 (Check this is not to mean HEWs, but 2. No Skip to End of questionnaire CBRHAs who teach the community on FP and distribute contraception’s)

Do you think that these persons can teach 1. Yes 40 and provide family planning services to the 2. No community

If the answer to the above question is “No” Why do you think is so? Reason ______41 ______

Have you ever had received service from a 1. Yes 42 CBRHA? 2. No Skip to End of questionnaire

1. Health education 2. Condoms What services did you receive from this 43 3. OCP person 4. Referral for FP services 5. Referral for other illnesses

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 101

SN QUESTIONS CODING CATEGORIES 1. Male 44 Would tell me the sex of this person? 2. Female Skip to Q. 46 If the sex of the CBRHA is male, ask Questions # 58 and 59 and write here response in the space provided.

Are you comfortable to discuss your reproductive health issue with male CBRHA? ______45 ______

Is he able to convince and involve your husband / 1. Yes 46 partner in the family planning? 2. No

1. He/she make home visit Where do you usually get this individual? 2 On the community meeting (churches, mosques, 47 (Do not read the list, but probe by asking, Idir, etc) “Any other?” and circle all that apply) 3 Other specify)______

1. Weekly 2. Monthly How often did this person visit your home 3. Quarterly 48 to provide you the RH/FP services? 4. Semi-annually 5. Annually 6. Other: (specify) ______7. Don’t Remember

When was your last contact with this 49 ______days ago person?

Would you say you were satisfied with the 1. Satisfied Skip to Q. 52 CBRHA visit to your home, or were you 2. Dissatisfied 50 dissatisfied with the CBRHA visit to your 3. Other: ______home?

Why were you dissatisfied with CBRHA visit to your home? (explain) ______51 ______

If you could suggest improvements to the services provided, what would it be? (Itemize) 1. ______52 2.______3.______

1 Yes Did you receive the information and 2 No 53 services that you wanted on the last 3 Partially CBRHA visit to your home? 4 Can’t recall 98 No response

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA) 102

SN QUESTIONS CODING CATEGORIES

1. Individually How do the CBRHA counsel you in cases 2. In group 54 where there is another woman (who is living 3. Don’t remember together or a guest) in your house? 4. 98 No response

1 Too short How do you think the length of time of 2 Too long 55 your consultation with the CBRHA? 3 About right 4 Don’t remember During the last visit of CBRHA, did you 1 Yes have any concerns about family planning 56 2 No Skip to Q.58 or other health issues that you wanted to 3 Don’t remember Skip to Q. 58 discuss with the CBRHA? 1 Yes Did the CBRHA listen to your concerns to 57 2 No your satisfaction? 3 Don’t remember 1 Yes During the last visit of CBRHA, did you 58 2 No Skip to Q. 61 have any questions you wanted to ask? 3 Don’t remember Skip to Q. 61

1 Yes 59 Did the CBRHA let you ask the questions? 2 No Skip to Q. 61 3 Don’t remember Skip to Q. 61 1 Yes Did the CBRHA respond to your questions 60 2 No to your satisfaction? 3 Don’t remember In your opinion, did you have enough 1 Yes 61 privacy during your consultation with the 2 No CBRHA? Do you believe that the information you 1 Yes 62 shared about yourself with the CBRHA will 2 No be kept confidential? 3 Don’t know Have you ever been referred to the nearest 1 Yes 63 health facility by the CBRHA? 2 No Skip to Q 66

1. gave referral paper/slip and told to go there 2. told verbally/orally to go there but no referral What was the referral system the CBRHA 64 paper/slip used? 3. took me there personally 4. don’t remember

1. Readily accept the referral and gave the How the health worker at health facility did service 65 received you? 2. Accept the referral but told to wait her turn 3. Did not accept the referral

Do you think this project had helped you 1 Yes 66 and your family? 2 No

67 If yes, would you tell me how the project helped you and your family?

Evaluation of Reproductive Health/Family Planning Project Of Oromia Development Association (ODA)