USER'S GUIDE ON PLANNING AND MANAGING A QUALITY SURVEY IN REPRODUCTIVE HEALTH PROGRAMS Prepared by Alison Ellis, MSH/Boston Dr. Ersin Topçuoglu, MSH/Ankara August 2002 Management and Leadership Program Cooperative Agreement No. HRN-A-00-00-00014-00 Management Sciences for Health 891 Centre Street Boston, MA 02130 1 TABLE OF CONTENTS Abbreviations ............................................................................................................. 3 Introduction ............................................................................................................... 4 1. Preparation 1.1 Setting the date for the quality survey ................................................... 8 1.2 Preparing the data collection tools ............................................................. 9 1.3 Setting up the survey headquarters ...............................................................12 1.4 Selecting the facilities ................................................................................. 13 1.5 Staffing ………………………………………………………………..…. 16 1.6 Training ………………………………………………………………….. 16 1.7 Managing the survey process…………………………… …………... 19 2. Fieldwork 2.1 Data collection .......................................................................................... 20 2.2 Supervision ............................................................................................... 23 2.3 Data Entry .................................................................................................. 25 3. Analysis & Feedback 3.1 Data analysis ............................................................................................. 27 3.2 Feedback to policy makers ......................................................................... 28 3.3 Feedback to survey staff and service providers ............................................ 30 3.4 Reporting .................................................................................................... 31 4. Cost & Level of Effort 4.1 Cost figures and analysis for the quality surveys...................................... 34 4.2 Level of Effort for the quality surveys .................................................... 36 Conclusion ...................................................................................................... 39 Appendices: Appendix I: Performance Standards for Quality of Care Appendix II: Facility Checklist for Family Planning Facility/Unit Appendix IIIa: Exit Interview for Family Planning Clients Appendix IIIB: Exit Interview Questionnaire for Postabortion Clients Appendix IIIC: Exit Interview Questionnaire for Postpartum Clients Appendix IV: Interviewer Training Agenda 2 Abbreviations CA Cooperating Agency FP Family planning HQ Headquarters IEC Information, education, and communication ICPD International Conference on Population and Development MCH/FP Maternal and child health/Family planning M&E Monitoring and evaluation MOH Ministry of Health MSH Management Sciences for Health NGO Non-governmental organization PC Personal computer PHD Provincial Health Directorate RH Reproductive health QC Quality of care SPSS Statistical Package for Social Sciences SSK Social Insurance Organization USAID United States Agency for International Development UNFPA United Nations Fund for Population Activities UNICEF United Nations International Children's Education Fund 3 Introduction Purpose of the Manual This User's Guide on planning and managing a quality survey has been prepared for two reasons: · to document the experience and lessons of the Management Sciences for Health (MSH) Turkey program in implementing seven quality surveys in four provinces during the period 1998-2001; and · to provide other international health programs with a sufficient level of detailed information to replicate similar surveys to improve the performance and management of family planning (FP) and other reproductive health (RH) programs. The manual is organized under the following headings: preparation, fieldwork, analysis and feedback, and cost and level of effort. In each section, detailed information on the steps involved are provided, including some stories from the Turkey program's experience, and key lessons learned. Data collection tools, sample training agenda/content areas, and sample formats for reports and other management tools are provided in the body of the manual or in appendices. The following two sections provide background information on the importance of monitoring quality of care, and the rationale for designing and implementing quality surveys in Turkey. Importance of Monitoring Quality of Care: Global and MSH Perspectives The issue of quality of care (QC) in family planning and reproductive health programs (FP/RH) gained worldwide prominence nearly a decade ago with the Bruce/Jain framework on quality of care. The framework outlines six elements that define quality of care in FP programs: choice of methods, information given to clients, technical competence, interpersonal relations, follow-up and continuity mechanisms, and the appropriate constellation of services. 1. The importance of quality of care in FP/RH services was further reinforced at the 1994 International Conference on Population and Development (ICPD) held in Cairo as well as the ICPD+5 Conference held in New York in 1999. In 1994 ICPD Conference the focus on FP efforts shifted to a more comprehensive "reproductive health" approach which calls for client-oriented, quality services that empower a woman to make an informed choice about RH care in an environment of dignity and respect. This focus was reinforced at the 1999 ICPD+5 Conference. The quality of care initiative has struck a respondent chord in countries around the world. At the individual level, improved quality of care in FP programs ensures women and men of respectful treatment by technically competent providers; it is intended to empower them to make choices consistent with their own reproductive intentions. At the aggregate level, greater satisfaction with services should translate into greater contraceptive adoptions and continuation rates, thus increasing contraceptive prevalence. Improved quality of care also may prompt women to act on their intentions to seek services in areas where there is fragile demand for contraception. From policy makers and managers at the country level to donor agencies at the international level, there is consensus that delivering quality services to women seeking FP and other RH services is an important objective. Initiatives to improve quality of care in FP/RH are supported at the policy level, 1 Bruce, Judith, "Fundamental Elements of the Quality of Care: A Simple Framework", Working Paper No. 1, The Population Council, May 1989. 4 given the post-Cairo focus on the individual needs of women. Total Quality Management (TQM) and Continuous Quality Improvement (CQI) are approaches that have taken hold around the world. Many programs are shifting from a focus on the number of clients served (which may lead to high discontinuation rates) to better serving the comprehensive needs of their clients. In addition to effects at the individual level, higher quality services also may lead to changes in behavior in the catchment areas surrounding service delivery points as word gets around of improved quality of services provided. As a result, while putting the needs of the woman first, programs can both satisfy the client and increase coverage. Measuring quality on a routine basis is extremely important for several reasons. Highlighting QC demonstrates to staff that it is an important component of the program and thus sets the standard for staff performance. In cases where an intervention to improve QC is in place, quality of care can be measured over two points in time, not only to determine the effectiveness of the intervention, but also to inform future program strategies. Improved QC is of tremendous importance to consumers who are the first to benefit from better services and who may be further encouraged to meet their reproductive intentions as a result of quality services received.2 Why is quality important for MSH? In addition to the rationale and events noted above, service quality is integral to MSH's mission, notably MSH's aim to "increase the effectiveness, efficiency, and sustainability of health and family planning services by improving their management, promoting access to these services, and influencing public policy." 3 An Approach to Monitoring Quality of Care in the Turkey Family Planning Program With the implementation of the United States Government Performance and Results Act of 1993, USAID-funded projects are now under added pressure to provide quantitative measures of project performance. While many see reporting requirements as a burden on activities in the population and health sector, routine project monitoring can be designed to meet the needs of both local project managers and higher government authorities. Monitoring and evaluation (M&E) plans that adhere to a set of accepted best practices are easy to implement and yield data that can be used to continually improve project performance. Simple, state- of-the-art M&E plans enable projects to make data-based decisions regarding public health interventions and also provide government agencies with evidence-based program outcomes. In order to better track progress and evaluate improvements in the quality of family planning services in Turkey, the USAID Turkey program implemented an innovative M&E system. The M&E plan incorporated several accepted