ATION REPORT Olj the CONTRACEPTIVE PREVALENCE STUDIES PROJECT AMY ONG TSUI 1?
Total Page:16
File Type:pdf, Size:1020Kb
EVALt!ATION REPORT OlJ THE CONTRACEPTIVE PREVALENCE STUDIES PROJECT A Report Prepared By: AMY ONG TSUI 1?h.D. JAY H. GGASS~~, Ph.D. JOHN A. ROSS, Ph.D. During The Period: JuLY 19-29, 1983 Supported By The: U. S. AGENCY FOr- IN'rEHNATIOi'U\L DEVELOPHENT (ADSS) AID/DSPE-C-0053 AUTHORIZA':L'ION~ Ltr. AID/DS/POP~ 1/6/84 Assgn. No. 582209 ACKNOWLEDGMENTS The evaluation team is grateful for the timely assistance and cooperation extended by Westinghouse Health Systems and the Office of Population at AID towards assembling the information needed to complete this evaluation and for the supporting services of the American Public Health Association in coordinating the evaluation. -i- CONTENTS Page ACKNO~'lLEDGMENTS . • . • . • . • . i AB B REV I AT ION S. .. vii LISTOFTABLES •.•......•...•.•..•...•...•..••••.••.•............• ix I . SUMMARY ...... • ••••• 90 • D •••• " • • • • • • • • • • • • • • • • • • • • • • • • • • 1 II. EVALUATION METHODOLOGY ......... .. 7 Composition of Team ........... · ... 7 Dates and Places of Evaluation .• . • • 0 • • e • ... 7 Materials Evaluated .. • • • • iii • • 8 Basic Questions ..... .0. ....... • • • • • • • • • • 8 III. EXTERNAL FACTORS. • • • fI · .... 13 country Select ion .. 13 Funding Certainty ...... .. 13 IV. INPUTS ...... • • ., • 0 • • • • • .... 17 Contractor Inputs. 17 AID Inputs 22 V. OUTPUTS .. .' ..... 25 Further Development of Survey Documentation .. 25 Selection of Participating Countries. · .. 26 Subcontractor Selection. 27 Field Opera t ions .............. 29 Monitoring Field Operations .... 29 Data Processing and Analysis. 30 Further Analysi s ........... 0 •• 33 VI. PURPOSE ..•.•••.•••••••••.••• 0 • 0 •••••••••••• 0 ••••• 0 •••••• 35 VII. GOAL OF THE PROJECT .................................... 37 VIII. BENEFICIARIES ..... 39 Data Produced and Available for Family Planning Program Management ..... 39 Use of the Survey Results for Family Planning Program Management . ..... 41 "Further Analyses", A Potential Source of Useful " Re sui t s ......... 41 IX. LESSONS LEARNED ...... ........... 43 Project Performance and Design. 43 Validity of Assumptions Underlying the Project. 44 Future Assistance and Further Analysis ..... 45 TABLES Previous :1?age -iii- APPENDICES A Persons Contacted During Evaluation of CPS II Project (Exclusive of Field Site Visits) B-1 Report on Field visit to Barbados, August 11-12, 1983 by Amy Ong Tsui B-2 Report on Field Visit to Egypt, September 2-9, 1983 by Jay H. Glasser C CPS II Project Organization D CPS II Status Summary Report E CPS Publication Status Report F Further Analysis LIST OF ABBREVIATIONS AID Agency for International Development CDC Center for Disease Control CPS Contraceptive Prevalence Survey eTO Cognizant Technical Officer LDC Less Developed Country PAHO Pan American Health Organization POPLAB Laboratories for Population Statistics RFP Request for Proposals WFS World Fertility Survey -vii- LIST OF TABLES 1. Level of Effort and Travel Spent on CPS II by Professional Staff 2. Country Survey Monitors, Country Responsibility, Academic and Language Background, and Year Joined CPS II 3. Measurement of ·CPS II Recruitment Effort by Region 4. List of Trip Reports CPS II 5. Regional CPS II Seminars 6. Comparison of End-of-·Project, Current and Anticipated Final Statuses for CPS II -ix- SUMMARY The evaluation team finds that the contractor has, in large measure, acted in good faith and with its best effort to meet the essential requirements of contract number AID/DSPE-C-0052; qualifications to this, together with recommendations for the t.cansition to the "follow-up" project, are provided. Project Background and Current Situation The objective of the Contraceptive Prevalence Studies Project II (CPS II) is to carry out repetitive sample surveys of contraceptive use prevalence in 25-30 develcping countries, with active public and/or private sector family planning programs, and to publish the resulting data for use by LDC family planning administrators and by AID. The scope of work requires the contractor, yvestinghouse Health Systems, to provide technical and financial assistance to countries that wish to conduct national or subnational surveys in order to obtain information on current levels of contraceptive use and attitudes toward contraception. The contractor assists participating countries in all technical aspects of designing and implpmenting of CPS and provides financial inputs where required as part of its effort to institutionalize the capability to conduct contraceptive prevalence studies. The contractor also arranges for conferences and workshops at which the methodology and findings of the surveys are presented to interested audiences, including planners, policymakers, and other administrators. This evaluation was carried out during the fourth year of project operations, between July 19-29, 1983. Following a debriefing by AID, the evaluation team discussed and reviewed the contract outputs and related project materials at the contractor's offices in Columbid, Maryland. Two overseas field visits to Barbados (August 11-12) and Egypt (September 2-9) yielded additional insights into the contractor's performnce in arranging for CPSs in these countries. CPS II is a five-year, incrementally funded contract whi~n began September 30, 1979, and is scheduled to expire September 29, 1984. The total contract value is $11,499,447 of which $9,700,000 had been obligated at the time of the project evaluation. An ddditional $1,600,000 will be forthcoming from some AID regional bureaus and local missions. At the time of the field evaluation a potential shortfall in funding created some concern for project administration and completion. At report submission (mid-Septc~ber), only about $180,000 of the contract total had not been committed (counting non-Washington contributions). 1 Progress Toward Contract Objectives The contract envisioned approximately 60 surveys fielded, with 21 being first, 24 second and 14 third round surveys. At th end of the CPS II project, it is likely that 32 surveys in 30 countries will have been completed, of which 24 will be first and six second round. As of June 30, 1983, 14 surveys had been completed and eighteen were ongoing. Eight country reports had been published. The evaluation finds that first-round surveys have received more emphasis than repetitive surveys. Only three countries have carried out both first and second round surveys within this project period, and in all three the sample has been changed. Also, the evaluation team is concerned about the forthcoming backlog of 20-some country reports to be produced in the final year of the project. It may not be possible to complete these without incurring an extension of the contract period. Although a higher volume of surveys was planned for in the project design, the expectation may not be realistic for a five-year contract. The evaluation team found tha~ the contractor is putting forth its best effort, and an impressiv~ ;ne, in meeting contract requirements and furthering the larger goals of AID's population assistance program. The shortfall in survey numbers can be partially attributed to factors in the project environment over which the co~tractor has little control. Problems Impacting on Contract Performance Two problems have impacted on contract performance. One 1S the precedence given to AID programmatic i~terests and political considerations in the selection and recruitment of countries for the survey program. This limits the field of eligible participants and the distribution of available survey capabilities, as in the recent focus on African and Middle Eastern countries. A second problem is uncertainty about the full funding of the contract. A demand for contraceptive prevalence surveys has been generated which cannot be met due to curtailed resources. The funding constraints experienced in the fourth year have necessarily limited the number of new survey starts and have jeopardized the timely completion of the reports from ongoing surveys. Major Findings and Recommendations The major findings and recommendations of this evaluation are listed here. 1. Inputs. Contractor inputs (staffing pattern, organizatio~support services and project management) have been ad~quate in meeting the needs of this project, with some improvement possible. 2 Recommendation: Staffing should be at maximum early in the project to avoid delays in start-up. Recommendation: The contractor should maintain frequent and reqular monitoring of its fiscal position relative to contract funding for early anticipation of needed financial inputs. AID inputs have been substantial and well received by the contractor. However, some local missions have not shown much interest and have not cleared survey projects. In some cases this lack of interest has detracted from the full recruitment process. 2. Outputs. The contractor has relinquished control of the survey instrument, in order to permit evaluation of local service programs. The pressure to design and conduct a survey rapidly, however, has occasionally interfered with an unhurried consideration of what useful data can be collected when a survey opportunity presents itself. Supplemental questions (modules) have been included on an ad-hoc basis. Recommendation: The supplemental questions or modules should be assembled in a systematic form for country and external review. country recruitment has been vigorous with 80 percent of the eligible large LDCs contacted and 68 percent of those interested becoming participants in the survey program. Recommendation: