FINAL REPORT Mid-Term Evaluation of the Romania Family Planning Project (Project No. 186-0002) IQC NO. AEP-0085-I-00-2060-00 Delivery Order No. 4 Commissioned by: ENI/EUR Bureau for Europe and New Independent States United States Agency for International Development Washington, D. C. 20523 Prepared by: Amil Kumar, Nancy Langer, Dr. Louise Tyrer Submitted by: DATEX, INC. January 14, 1994 The views and opinions expressed herein are those of the evaluation team, and are not necessarily those held by Datex, Inc., nor USAID. ACRONYMS BTB Breakthrough Bleeding C&L Coopers & Lybrand CDC Centers for Disease Control CEDPA Centre for Development and Population Activities FPMV Family Planning Movement of Vranchea GP General Practitioner physician IE&C Information, education & communication IPPF International Planned Parenthood Federation IUD Intra Uterine Device JHPIEGO Johns Hopkins Program for International Education in Reproductive Health MIS Management information systems MOH Ministry of Health MSCI Margaret Sanger Center International NGO Non-Governmental Organization NIS Newly Independent States Ob/Gyn Obstetrics & Gynecological physician OC Oral contraceptive PCS Population Communication Services PVO Private Voluntary Organization RFP Romanian Family Planning Project RHS Romanian Health Survey SECS Society for Education in Contraception and Sexuality STD Sexually transmitted disease TA Technical assistance TOT Training of trainers UNDP United Nations Development Program UNFPA United Nations Population Fund USAID/ENI USAID/Bureau for Europe and the Newly Independent States USAID U.S. Agency for International Development WHO World Health Organization WIM Women in Management TABLE OF CONTENTS EXECUTIVE SUMMARY ................................................ i SECTION ONE: BACKGROUND ......................................... 1 1.0 Romanian Social, Economic and Political History as it Relates to the Romanian Family Planning Project ....................................... 1 2.0 Evaluation Methodology ....................................... 3 SECTION TWO: PROGRAMMATIC DEVELOPMENT ........................... 4 2.1 The Quality of Clinical Services ................................. 4 2.1.1 Major Findings (Results) . ............................... 4 2.1.1.1 Choice . ................................. 4 2.1.1.2 Disseminatior ofinformation .................... 4 2.1.1.3 Quality of clinicalcare . ...................... 5 2.1.1.4 Organizationand location of clinicalservices ........ 5 2.1.2 Significant Accomplishments and Impacts ...................... 5 2.1.3 Conclusions .. ....................................... 5 2.1.4 Recommendations ...................................... 6 2.1.5 Lessons Learned .. .................................... 7 2.2 New Programmatic Activities and Areas to Develop ................... 7 2.2.1 Information, Education & Communication . .................... 7 2.2.1.1 Findings. ................................ 7 2.2.1.2 Conclusions . ............................. 8 2.2.1.3 Recommendations ........................... 8 2.2.3 NGO as Public Advocate . ............................... 8 2.2.3.1 Findings. ................................ 8 2.2.3.2 Conclusions . ............................. 9 2.2.3.3 Recommendations ........................... 9 SECTION THREE: PROJECT MANAGEMENT SYSTEMS AND PROCESSES .......... 10 3.1 Management Tools and Processes ................................ 10 3.1.1 Findings ........................................... 10 3.1.1.1 Needs-Assessment .. ......................... 10 3.1.1.2 Definition ofMeasurableObjectives andDevelopment of Framework .. ............................. 11 3.1.1.3 Initial andAnnual Workplans ................. 11 3.1.1.4 OrganizationalChartfor Project Structure ......... 11 3.1.1.5 Job Descriptions(JD). ...................... 11 3.1.1.6 Scopes of Work (SOWs) and Reports of Short Term TA . 12 3.1.1.7 Qu)terly Reports and Annual Reviews and Reports .... 12 3.1.1.8 Contractsand Reviews for Field Staff ............ 13 3.1.2 Conclusions .. ...................................... 13 3.1.3 Recommendations ..................................... 13 3.2 Management and Programmatic Relationships and Communication ......... 13 3.2.1 Findings .. ......................................... 13 3.2.2 Conclusions .. ...................................... 14 3.2.3 Recommendations ..................................... 15 3.3 Institutional and Professional Development of the Family Planning NGOs ..... 15 3.3.1 Findings ............. ............................ .. 15 3.3.1.1 Professional Development ..................... 15 3.3.1.2 Training.. ............................... 15 3.3.1.3 Standardizationof terms ...................... 16 3.3.1.4 Development of clinical manuals and curricula ....... 16 3.3.1.5 Resources and literature ..................... 16 3.3.1.6 Study tours and internationaltraining ............ 17 3.3.2 Conclusions ......................................... 17 3.3.3 Recommendations ................................... .. 18 3.4 Significant Accomplishments ................................. .. 18 3.5 Lessons Learned ........................................... 19 SECTION FOUR: FINANCIAL MANAGEMENT AND ANALYSIS ................ 19 4.1 Financial Reporting and Financial Management ...................... 19 4.1.1 Findings ........................................... 19 4.1.1.1 ExpenditureLevels . ........................ 19 4.1.1.2 FinancialReporting and Analysis as a Management Tool . .... .. ... ... .. ... 20 4.1.1.3 CEDPA's Analysis of SubgranteeFinancialReports .... 21 4.1.1.4 Subgrantees'Analysisof Their Own FinancialReports . 21 4.1.2 Conclusions .. ...................................... 22 4.1.3 Recommendations ..................................... 22 4.2 Technical Assistance in Financial and Cormmodities Management .......... 23 4.2.1 Findings ........................................... 23 4.2.1.1 FinancialManagement ....................... 24 4.2.1.2 Management Information Systems (MIS) ............ 24 4.2.1.3 Commodities Logistics Training ................. 25 4.2.1.4 Coopers & Lybrand, Local Consulting............. 25 4.2.2 Conclusions .. ...................................... 25 4.2.3 Recommendations ..................................... 25 4.3 Sustainability ............................................. 26 4.3.1 Current Context and Progress to Date ........................ 26 4.3.2 Findings and Constraints to Sustainability..................... 27 4.3.3 Conclusions .. ...................................... 28 4.3.4 Recommendations ..................................... 28 4.4 Currency Management and Recognition of Inflation .................... 28 4.4.1 Findings .. ......................................... 28 4.4.1.2 GrantDenomination . ....................... 29 4.4.1.3 Effects of Inflation . ........................ 29 4.4.2 Conclusions .. ...................................... 29 4.4.3 Recommendations ..................................... 29 SECTION FIVE: FUTURE GRANT ACTIVITIES ............................. 30 5.1 Overall Findings and Recommendations ............................ 30 5.2 CEDPA Activities and Responsibilities . .......................... 31 5.3 Other Contractors' Activities and Responsibilities ..................... 31 5.4 USAID Activities and Responsibilities ............................. 31 5.5 Funding for Remaining Project Activities ........................... 32 5.6 Other Donor Activity .. ..................................... 32 ANNEXES Annex One: Scope of Work Annex Two: Documents Reviewed Annex Three: People Contacted Annex Four: Disbursements Made to Subgrantees Annex Five: Evaluation of Medical/Clinical Services Annex Six: Management Tools and Processes Annex Seven: Summary of Grant History and Development Annex Eight: Map of Romania Annex Nine: List of Computer hardware and Software Annex Ten: List of Recommendations EXECUTIVE SUMMARY 'Project Background Romania was ruled by the Communist regime of Nicolae Ceausescu for 24 years from 1965 to 1989. His strongly pronatalist government outlawed abortion and family planning. These policies prompted widespread illegal abortions. The abortions were inexpensive but dangerous and resulted in the highest maternal mortality rate in Eastern Europe. Following Ceausescu's assas iation, "Western" reporters were invited into the country. The maternal mortality rate and vast numbers of institutionalized children prompted inter.mational horror and the U.S. Congress immediately earmarked $3 million, $1.5 million for each of these problems. In 1991, The Centre for Development and Population Activities (CEDPA) was awarded a $1.5 million grant by the U.S. Agency for International Development (USAID) to implement the Romanian Family Planning (RFP) Project (Grant No. EUR-0002-G-00-1016-O0) for eighteen (18) months. The goal was "to establishi a pilot family planning service delivcery system ipthe private sector '. The project purpose was to "improve the health and well-being of the women of Romania through family planning service delivery" ' . It was determined that objectives would best be achie~ved by funding existing family planning Non-Governmental Organizations (NGOs) to establish model famiiy planning clinics. The grant was ultimately extended to $4.9 million for five (5) years, to go through early 1996. CEDPA currently funds three Romanian NGOs, the Society for Education in Contraception and Sexuality (SECS), Family Planning Movement of Vranchea (FPMV), and Youth-for-Youth. SECS is the largest NGO and runs ten clinics