Prepared Under HNPSP of the Ministry of Health and Family Welfare
ECONOMIC EVALUATION OF DEMAND-SIDE FINANCING (DSF) PROGRAM FOR MATERNAL HEALTH IN BANGLADESH
February 2010
Recommended Citation: Hatt, Laurel, Ha Nguyen, Nancy Sloan, Sara Miner, Obiko Magvanjav, Asha Sharma, Jamil Chowdhury, Rezwana Chowdhury, Dipika Paul, Mursaleena Islam, and Hong Wang. February 2010. Economic Evaluation of Demand-Side Financing (DSF) for Maternal Health in Bangladesh. Bethesda, MD: Review, Analysis and Assessment of Issues Related to Health Care Financing and Health Economics in Bangladesh, Abt Associates Inc.
Contract/Project No.: 81107242 / 03-2255.2-001-00
Submitted to: Helga Piechulek and Atia Hossain GTZ-Bangladesh House 10/A, Road 90, Gulshan-2 Dhaka 1212, Bangladesh
Abt Associates Inc. 4550 Montgomery Avenue, Suite 800 North Bethesda, Maryland 20814 Tel: 301.347.5000. Fax: 301.913.9061 www.abtassociates.com
In collaboration with:
RTM International 581, Shewrapara, Begum Rokeya Sharoni, Mirpur Dhaka 1216, Bangladesh
2 Economic Evaluation of DSF Voucher Program in Bangladesh ECONOMIC EVALUATION OF DEMAND- SIDE FINANCING (DSF) FOR MATERNAL HEALTH IN BANGLADESH
DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) GmbH.
CONTENTS
Acronyms...... xiii
Acknowledgments...... xv
Executive Summary...... xvii
1. Introduction...... 1 1.1 Objectives of the evaluation...... 1 1.2 Organization of the report ...... 2 2. Background and literature review...... 3 2.1 Maternal health in Bangladesh...... 3 2.2 International and local experiences with Demand-Side Financing (DSF) programs ...... 4 2.3 The DSF maternal health voucher program in Bangladesh...... 7 2.3.1 Pilot DSF program areas...... 7 2.3.2 Funding and management structure...... 7 2.3.3 Eligibility criteria...... 8 2.3.4 Beneficiary identification process ...... 8 2.3.5 Voucher distribution...... 9 2.3.6 Participating health care providers...... 9 2.3.7 Voucher benefits to consumers...... 9 2.3.8 Provider reimbursement and incentives...... 10 2.3.9 Distribution of cash and in-kind incentives to consumers and providers...... 11 2.4 Summary of findings from 2005 baseline report ...... 12 3. Methods...... 15 3.1 Overview of data collection methods...... 15 3.2 Instrument preparation and pre-testing ...... 15 3.3 Selection of upazilas for the evaluation...... 16 3.4 Selection of respondents...... 19 3.5 Field data collection procedures...... 20 3.5.1 Hiring and training data collectors ...... 20 3.5.2 Informed consent procedures...... 20 3.5.3 Qualitative data collection...... 20 3.5.4 Phase 1 household survey data collection...... 20 3.5.5 Phase 2 household survey data collection...... 21 3.5.6 Administrative data collection ...... 21
iii 3.5.7 Summary of completed data collection ...... 21 3.6 Transcription, translation and data entry...... 23 3.7 Household survey processing...... 23 3.7.1 Sampling weights ...... 23 3.7.2 Comparison of phase 1 and phase 2 household survey data...... 24 3.8 Analysis ...... 25 3.8.1 Variable construction ...... 25 3.8.2 Analytic approach...... 25 3.9 Limitations ...... 26 4. Findings Part 1: Voucher Operations...... 29 4.1 Voucher program design...... 29 4.1.1 Targeting and identification of the poor...... 29 4.1.2 Incentives for consumers and providers ...... 30 4.1.3 Involvement of NGO and Private Sector Facilities...... 30 4.2 Implementation and logistics ...... 31 4.2.1 Informing consumers about the DSF program...... 31 4.2.2 Identifying eligible beneficiaries...... 34 4.2.3 Voucher distribution...... 39 4.2.4 Payment of cash and in-kind benefits to consumers ..... 44 4.2.5 Service provision under the voucher program ...... 48 4.2.6 Monitoring and evaluation systems ...... 51 4.3 Financial management and accounting ...... 53 4.3.1 Financial management...... 53 4.3.2 Financial sustainability of scheme...... 55 4.4 Institutional arrangements and governance ...... 56 4.4.1 DSF program organizational structures and systems.... 56 4.4.2 Responsiveness to stakeholders and consumers ...... 57 4.5 Summary of key findings...... 58 5. Findings Part 2: Effect of the DSF program on the demand side...... 61 5.1 Household survey sample description ...... 61 5.2 Bi-variate analyses ...... 65 5.2.1 Antenatal care...... 65 5.2.2 Delivery care...... 67 5.2.3 Postnatal care ...... 70 5.2.4 Subanalysis: ANC, delivery care, and PNC among voucher recipients ...... 71 5.2.5 Subanalysis: Pre-post comparison of previous births .... 73 5.2.6 Out-of-pocket (OOP) expenditures...... 74 iv Economic Evaluation of DSF Voucher Program in Bangladesh 5.2.7 Subanalysis: Differential Impacts of the DSF program according to poverty status ...... 76 5.2.8 Beneficiary use of cash incentive ...... 81 5.3 Multivariate analyses ...... 82 5.4 Qualitative findings...... 87 5.4.1 Beneficiary opinions about the voucher program...... 87 5.4.2 Key informant opinions about the voucher program.... 88 5.4.3 Provider perceptions of DSF program impacts on beneficiaries ...... 89 5.4.4 Perceived effect on use of other programs and services 89 5.5 Program cost and cost-effectiveness analyses ...... 90 5.5.1 total program cost in 16 upazilas ...... 90 5.5.2 Average cost per voucher distributed...... 91 5.5.3 cost-Effectiveness analysis...... 91 5.6 Summary of key findings...... 93 6. Findings Part 3: Effect of the DSF program on the supply side...... 97 6.1 Results from Facility Quality Assessment...... 97 6.1.1 Sample description...... 97 6.1.2 MCH/FP Services Provided in The Year Prior to Assessment ...... 98 6.1.3 Staffing...... 100 6.1.4 Registers, Educational Materials and Sessions ...... 102 6.1.5 Referral Capacity...... 102 6.1.6 Infrastructure...... 103 6.1.7 WHO Mother Baby Package Assessment of Supplies, Equipment, Laboratory and Medications Capacity...... 104 6.1.8 Provider Training, Background, Knowledge and Skills 107 6.1.9 Key Findings from the facility assessment: Capacity for EOC and ENC Management ...... 111 6.2 Household survey results...... 111 6.3 Qualitative results: Perceived effects on providers ...... 114 6.3.1 Beneficiary perceptions of quality of care...... 114 6.3.2 Key Informant perceptions of changes in service quality 115 6.3.3 Key informant perceptions of impacts on non-DSF programs ...... 115 6.3.4 Provider satisfaction ...... 116 6.4 Summary of key findings...... 117 7. Discussion and recommendations ...... 119
v 7.1 Effects of the DSF program on maternal health service utilization and expenditures...... 119 7.2 Effects on service provision...... 120 7.3 Voucher distribution and targeting...... 121 7.4 Funds disbursement and financial management...... 123 7.5 Input shortages ...... 123 7.6 Monitoring and evaluation ...... 124 7.7 Program benefits and incentives ...... 124 7.8 Impact on other programs...... 125 7.9 Next steps for evaluation and scale-up...... 125 Annex A: Villages Visited for PHASE 1 Household Survey .. 127
Annex B: Villages Visited for PHASE 2 Household Survey... 131
Annex C: Additional tables ...... 133
Annex D: Bibliography...... 149
LIST OF TABLES
Table 2.1. Reimbursements and incentives to government providers... 10 Table 2.2. Breakdown of incentives for Basic maternal health services, by service provider (Tk.)...... 10 Table 2.3. Breakdown of incentives for maternal health Complications, by service provider...... 11 Table 3.1. Summary of upazilas selected for evaluation...... 17 Table 3.2. Planned and completed data collection...... 21 Table 3.3: Comparison of phase 1 and phase 2 samples ...... 24 Table 3.4: Qualified providers for ANC, delivery, and post partum care ...... 25 Table 4.1. Awareness of voucher program, by intervention type (%) ... 32 Table 4.2. Extent to which women were told about what the voucher pays for, by intervention type (%)...... 33 Table 4.3. Percentage of women who received a voucher by land ownership, asset ownership, and monthly expenditures (%) ...... 37 Table 4.4. Vouchers distributed as percentage of targets, April 2007- August 2009...... 39 Table 4.5. Receipt of cash and in-kind benefits among those who received a voucher booklet (%) ...... 45 Table 4.6. Receipt of cash and in-kind benefits among those who received a voucher booklet, by place of delivery(%) ...... 45 Table 4.7. Receipt of cash by delivery date and type of intervention (%) ...... 46 Table 4.8. Beneficiary difficulties in obtaining transport stipends (%) .... 46 Table 4.9. Beneficiary difficulties in receiving cash incentive for nutritious food (%) ...... 46
vi Economic Evaluation of DSF Voucher Program in Bangladesh Table 5.1. General background of the study sample (%)...... 61 Table 5.2. General background of study sample, Comparing universal and means-tested intervention groups (%)...... 62 Table 5.3. Household Socio-economic characteristics, Comparing control and Pooled intervention groups (%)...... 63 Table 5.4. Household Socio-economic characteristics, Comparing means-tested and universal groups (%)...... 64 Table 5.5. Presence of EOC UHCs in upazilas sampled for the household survey...... 65 Table 5.6. Location of most recent delivery, by intervention type (%) 69 Table 5.7. Location of most recent delivery among voucher recipients (%)...... 69 Table 5.8. Summary of key maternal health service utilization indicators (%)...... 70 Table 5.9. Voucher use for ANC, delivery, and PNC by intervention type (%) ...... 71 Table 5.10. Delivery care outcomes among next-to-last births, before and after voucher introduction ...... 73 Table 5.11. OOP expenditure on delivery by intervention type (Taka) 75 Table 5.12: OOP expenditure on delivery by type of delivery and intervention type (Taka)...... 75 Table 5.13. Selected maternal health services utilization, by quintile (%) ...... 77 Table 5.14. OOP expenditure on delivery by poverty status and intervention type (Taka)...... 79 Table 5.15. OOP expenditure on delivery types by poverty status and intervention type (Taka)...... 80 Table 5.16. PNC costs by poverty status (Taka)...... 80 Table 5.17. Beneficiary use of cash incentives for food (%)...... 81 Table 5.18. Probit regressions: The effects of the DSF program on the likelihood of delivery with a qualified provider and institutional delivery...... 83 Table 5.19. The effects of the DSF program on selected key maternal service utilization outcomes ...... 84 Table 5.20. The effects of the DSF program on selected maternal expenditure outcomes...... 85 Table 5.21. The effects of the DSF program on key outcomes, comparing intervention and control areas ...... 86 Table 5.22. Difference-in-differences estimates of the DSF effect on selected maternal health service utilization outcomes, comparing pre- and post-intervention births...... 87 Table 5.23. calculation of the average cost per voucher distributed ..... 91 Table 5.24. Calculation of the number of deliveries with qualified attendant attributable to the DSF program...... 92 Table 5.25. Cost effectiveness ratio for DSF program compared to control...... 93 Table 6.1. Total number of respondents available for interview during facility quality assessment, by respondent type ...... 97 Table 6.2. Characteristics of assessed facilities...... 98 Table 6.3. Average number of maternal health patients in 2008 ...... 98
vii Table 6.4. Key maternal and neonatal health indicators...... 99 Table 6.5. Average number of key personnel on staff, per facility...... 100 Table 6.6. Average number of vacancies per facility in 2008...... 101 Table 6.7. Average facility scores: Sufficiency of staffing to provide maternity and newborn care, according to MotherBaby Package criteria ...... 102 Table 6.8. Emergency transportation systems ...... 103 Table 6.9. Presence of essential medications for treating maternal health conditions ...... 107 Table 6.10. EOC knowledge and skills assessment Guidelines ...... 108 Table 6.11. Characteristics of most senior physician, obstetrician or gynecologist ...... 109 Table 6.12. Characteristics of most senior nurse ...... 110 Table 6.13. Selected quality indicators for neonatal care (%)...... 112 Table 6.14. Percentage of women who reported having “big problems” with care received during her first ANC visit (%)...... 113 Table 6.15. Percentage of women who reported having “big problems” with care received during her postnatal care encounter (%)...... 114 Annex Table 1. Source of information about voucher program, by intervention type (%) ...... 133 Annex Table 2. Percentage of women who received vouchers, by intervention type (%) ...... 133 Annex Table 3. Percentage of women who received vouchers by wealth quintile and intervention type (%)...... 134 Annex Table 4. Number of days after listing when beneficiaries received voucher (%) ...... 134 Annex Table 5. Main reported voucher distributors (%)...... 134 Annex Table 6. ANC utilization among all women by intervention type ...... 135 Annex Table 6B. ANC utilization among all women, control vs. intervention (%) ...... 135 Annex Table 7. Percent of women who received antenatal care from a qualified provider by intervention type(%) ...... 135 Annex Table 7B. Percent of women who received antenatal care from a qualified provider, control vs. intervention (%)...... 135 Annex Table 8. Delivery care for births taking place in the 6 months preceding the survey...... 136 Annex Table 8B. Delivery care for births taking place in the 6 months preceding the survey, control vs. intervention (%) ...... 136 Annex Table 9. Skill level of birth attendant for home deliveries, by intervention type (%) ...... 136 Annex Table 9B. Skill level of birth attendant for home deliveries, control vs. intervention (%) ...... 136 Annex Table 10. Summary of key maternal health service utilization indicators (%) ...... 137 Annex Table 11. Place of delivery among voucher recipients by intervention type (%) ...... 137 Annex Table 12. Place of delivery among voucher non-recipients by intervention type (%) ...... 138
viii Economic Evaluation of DSF Voucher Program in Bangladesh Annex Table 13. OOP expenditure on first ANC visit by poverty status (Taka)...... 138 Annex Table 13B. OOP expenditure on first ANC visit by poverty status (Taka)...... 138 Annex Table 14. Total OOP expenditure associated with pregnancy and delivery care by poverty status (in Taka)...... 138 Annex Table 14B. Total OOP expenditure associated with pregnancy and delivery care by poverty status (Taka) ...... 139 Annex Table 15. Key maternal and neonatal health indicators (analysis of EOC facilities only)...... 139 Annex Table 16. Average number of FP services provided in 2009 ....139 Annex Table 17. Average number of support and other staff per facility ...... 139 Annex Table 18. Capacity to provide Emergency Obstetric Care (EOC) ...... 140 Annex Table 19. Percentage of facilities with registers and guidelines available ...... 140 Annex Table 20. Percentage of facilities with pictorial educational materials available...... 140 Annex Table 21. Percentage of facilities with charts available ...... 141 Annex Table 22. Percentage of facilities with leaflets or handouts available ...... 141 Annex Table 23. Educational sessions for mothers ...... 141 Annex Table 24. Availability of EmOC services at referral facility...... 141 Annex Table 25. Who makes travel arrangements in case of obstetric emergency...... 142 Annex Table 26. Referral procedures and patterns...... 142 Annex Table 27. Indicators of adequate facility infrastructure: General ...... 142 Annex Table 28. Indicators of adequate facility infrastructure: Labor and delivery rooms ...... 143 Annex Table 29. Average availability of required supplies to provide essential maternal health services, according to Mother-Baby package criteria (%)...... 143 Annex Table 30. Average availability of equipment recommended by the Mother-Baby Package (%)...... 144 Annex Table 31. Presence of adequate laboratory capacity for essential maternal health services...... 144 Annex Table 32. Proportion of most senior health facility staff with Emergency Obstetric and Emergency Neonatal Care training .....144 Annex Table 33. Indicators of physician knowledge...... 145 Annex Table 34. Percent of physicians that currently conduct the following procedures ...... 146 Annex Table 35. Indicators of nurse knowledge...... 147 Annex Table 36. Percent of nurses that currently conduct the following procedures...... 147 Annex Table 37. Routine checks performed during the first ANC visit (%)...... 148
ix LIST OF FIGURES
Figure 4.1. Source of Information about Voucher Program, By Intervention Type (%)...... 32 Figure 4.2. Percentage of women who received vouchers, among different eligibility subgroups ...... 34 Figure 4.3. Eligibility characteristics of voucher recipients...... 35 Figure 4.4. Percentage of women who received vouchers by wealth quintile and intervention type (%)...... 35 Figure 4.5. Wealth quintile distribution of voucher recipients in universal and means-tested upazilas...... 36 Figure 4.6. Percentages of voucher recipients who met various poverty criteria ...... 37 Figure 4.7. Vouchers distributed per month vs. targets, by type of intervention, April 2007 - August 2009 ...... 40 Figure 4.8. Vouchers distributed as percentage of targets, by upazila ... 41 Figure 4.9. Pregnancy gestation at the time of voucher receipt...... 42 Figure 4.10. Number of days after listing when beneficiaries received voucher (%) ...... 42 Figure 4.11. Main reported voucher distributors (%)...... 43 Figure 4.12. Vouchers distributed and ANC services provided under the DSF program, April 2007-August 2009...... 48 Figure 4.13. Vouchers distributed and delivery and PNC services provided under the DSF program, April 2007-August 2009...... 49 Figure 5.1. ANC utilization among all women by intervention type ...... 66 Figure 5.2. Percent of women who received antenatal care from a qualified provider by type of study site (%)...... 67 Figure 5.3. Delivery care for births taking place in the 6 months preceding the survey...... 68 Figure 5.4. Skill level of birth attendant for home deliveries by intervention type (%) ...... 70 Figure 5.5. Place of delivery among voucher recipients by intervention type (%) ...... 72 Figure 5.6. Place of delivery among voucher Non-recipients by intervention type (%) ...... 73 Figure 5.7. OOP expenditure on ANC, Delivery care, and PNC (in Taka)...... 74 Figure 5.8. Total OOP expenditure on pregnancy and delivery care (Taka)...... 76 FIgure 5.9. OOP expenditure on first ANC visit by poverty status (Taka)...... 78 Figure 5.10. OOP expenditure on delivery by poverty status and intervention type (Taka)...... 78 Figure 5.11. Total OOP expenditure associated with pregnancy and delivery care by poverty status (in Taka)...... 81 Figure 6.1. Average number of deliveries and PNC visits in 2008...... 99 Figure 6.2. Capacity to provide Emergency Obstetric Care (EOC).....101 x Economic Evaluation of DSF Voucher Program in Bangladesh Figure 6.3. Average availability of required supplies to provide essential maternal health services, according to WHO Mother-Baby package criteria ...... 105 Figure 6.4. Average availability of equipment recommended by the WHO Mother-Baby Package...... 106 Figure 6.5. Proportion of most senior health facility staff with Emergency Obstetric and Emergency Neonatal Care training...... 108 Figure 6.6. Routine checks performed during the first ANC visit (%).112
xi
ACRONYMS
ANC Antenatal Care APH Antepartum Hemorrhage CSBA Community Skilled Birth Attendant DfID Department for International Development (United Kingdom) DGFP Directorate General Family Planning DGHS Directorate General of Health Services DSF Demand-Side Financing EmOC Emergency Obstetric Care EPI Expanded Program on Immunization FGD Focus Group Discussion FWA Family Welfare Assistant FWV Family Welfare Visitor GTZ Gesellschaft für Technische Zusammenarbeit (Germany) GoB Government of Bangladesh HA Health Assistant HEU Health Economics Unit, MOHFW HNPSP Health Nutrition and Population Sector Program ICDDR,B International Centre for Diarrhoeal Disease Research, Bangladesh IRB Institutional Review Board KII Key Informant Interview MDG Millennium Development Goal MIS Management Information System MO Medical Officer MOHFW Ministry of Health and Family Welfare MO-MCH Medical Officer, Maternal and Child Health NGO Non-Governmental Organization PHC Primary Health Care PNC Postnatal Care PPH Postpartum hemorrhage PPS Probability Proportionate to Size
xiii QAO Quality Assurance Officer (retained by GTZ) RMO Residential Medical Officer RTM Research, Training and Management International UHC Upazila Health Complex UHFPO Upazila Health and Family Planning Officer UHFWC Union Health and Family Welfare Centre UK United Kingdom UNFPA United Nations Population Fund UNO Upazila Nirbahi Officer WHO World Health Organization
xiv Economic Evaluation of DSF Voucher Program in Bangladesh ACKNOWLEDGMENTS
This is the final report for Study A (Economic Evaluation of Demand-Side Financing for Maternal Health in Bangladesh) of the project “Review, Analysis and Assessment of Issues Related to Health Care Financing and Health Economics in Bangladesh,” funded by the Deutsche Gesellschaft für Technische Zusammenarbeit GmbH (GTZ), and prepared under HNPSP of the MOHFW. It is the product of the collaborative efforts of many individuals and organizations who contributed to the design, analysis, and completion of the study.
We are very grateful for the support and guidance provided by the Health Economics Unit (HEU) of the Government of Bangladesh’s Ministry of Health and Family Welfare (MOHFW), as well as the Directorate General of Health Services (DGHS) and Directorate General of Family Planning Services (DGFP). We gratefully acknowledge the role played by Dr. Shamim Ara Begum, ex-Joint Chief (Joint Secretary) and ex-Line Director, HEU in expediting the process of the study. We also gratefully acknowledge Dr. Md. Anwar Hossain Munshi, Joint Chief (Joint Secretary) and Line Director, HEU in facilitating dissemination of the study and hosting the dissemination workshop in February 2010. We also want to thank Mr. Abdul Mannan, Joint Chief, Planning, MOHFW and Mr. Balijur Rahman, Deputy Chief, HEU, MOHFW for their support. Mr. Md. Rafiqul Islam Khan, Deputy Secretary, HEU, was continuously supportive and extended helpful cooperation and guidance. All other colleagues at HEU were also extremely helpful at all stages of the study.
We are very thankful to Mr. Salam Khan, National DSF Coordinator, for his continuous support and guidance. Mr. Khan was extremely helpful in providing various program documents, data, and responding to multiple requests for information. We also want to thank all DSF Coordinators at the upazila level, who provided their time, information, and support for the project. We gratefully acknowledge the contributions, input and assistance of Dr. Frank Paulin, Medical Officer PHA, WHO. We thank Muhammad Ashraful Alam, DSF Coordinator, Banaigonj Upazila, Sylhet, for sharing the photos of voucher recipients featured on the cover of this report.
We are grateful to Mr. ABM Jahangir Alam, Director, PHC and Line Director DGHS and Dr. Jafar Ahmed Hakim, Director, MCH Services and Line Director, MCRH, DGFP for their support. We are also thankful to Dr. Nazrul Islam, former DPM for the DSF Program, who was very helpful during the design and implementation phase. We are in general very grateful to the MIS Units at DGHS and DGFP, who provided their support and data for the project.
We are very appreciative of the huge efforts and long hours put in by the RTM International team, which provided technical support in all study design, data collection, and data entry activities: Dr. Ahmed Al-Kabir (President), Mr. Syed Anwarul Islam, Mr. Nazmul Huq, Dr. Sharmin Sultana, Dr. Mustafiza Rushdi, Ms. Tasnuva Sultana, Ms. Dhiraj Kanti Chowdhury and Ms. Jobayda Fatema, among others. We also want to thank all members of the data collection, data entry, and data analysis teams for their hard work. We also gratefully acknowledge the efforts of Ricky Merino of Abt Associates who formatted the final report.
We acknowledge the contribution of all the Upazila Health Complexes, who provided their support and essential data for the project as well as allowing us to spend time interviewing their staff. We are grateful to DSF committee members and all health providers including facility managers, CSBAs,HAs,
xv HIs, FWVs, FWAs and other staff, who gave us their very valuable time. We are thankful to all individual respondents at household level and village dais, who also gave us their valuable time and provided us essential information in connection with the study.
Finally, we thank Dr. Helga Piechulek, Mr. Jean-Olivier Schmidt and Ms. Atia Hossain of GTZ for their support and assistance throughout the entire project effort. Without all this support at various levels, it would have been impossible to accomplish the task.
xvi Economic Evaluation of DSF Voucher Program in Bangladesh EXECUTIVE SUMMARY
This report presents findings from a comprehensive evaluation of the Demand-Side Financing (DSF) pilot maternal health voucher scheme in Bangladesh. The main objective of the DSF program is to accelerate progress toward Millennium Development Goal 5 (MDG 5) to improve maternal health, by stimulating increased utilization of safe maternal health services by poor pregnant women, including antenatal care (ANC), delivery by qualified providers, emergency obstetric and postnatal care (PNC). Poor pregnant women receive vouchers which entitle them to free maternal health services, transport subsidies, cash incentive for delivery with a qualified provider (either at home or at a designated facility), and a gift box. Providers receive incentives to distribute vouchers and to provide services covered by the vouchers.
The DSF voucher program is implemented by Bangladesh’s Ministry of Health and Family Welfare (MOHFW) using pooled funds co-financed by the World Bank, United Kingdom (UK),European Community, Germany, Sweden, Canada, Netherlands, and the United Nations Population Fund (UNFPA). The World Health Organization (WHO), with co-funding from the UK’s Department for International Development (DFID), provides technical assistance to the DSF program, including administrative and monitoring support through the posting of DSF coordinators to each DSF upazila. These organizers are overseen by a National DSF Coordinator, based in the national DSF cell in Dhaka.
This evaluation (conducted over June-December 2009) focuses on the 21 upazilas (subdistricts) where the program was functioning by mid-2007 and thus covers two years of experience with the DSF program. The evaluation compares DSF program intervention upazilas to matched control upazilas, in order to evaluate demand-side and supply-side impacts of the program, and also conducts a focused assessment of program operations in DSF upazilas.
DEMAND-SIDE IMPACT
The overall conclusion of this evaluation is that the DSF program has had an unprecedented positive effect on the utilization of maternal health services in the short time since its initiation. Demand-side impact was measured based on a household survey in 16 DSF upazilas and 16 matched control upazilas, covering 2208 women who delivered between February 1, 2009 and July 31, 2009. Key findings are that: The rate of deliveries attended by qualified providers is now more than twice as high in DSF program upazilas (64%) compared to control upazilas (27%). The rate of institutional deliveries is now twice as high in DSF program upazilas (38%) compared to control upazilas (19%). There is no statistically significant difference in the rate of C-sections between DSF areas (10%) and control areas (9%). Women in DSF program upazilas are significantly more likely to have at least 3 ANC visits (55%) than women in control upazilas (34%). Women in DSF upazilas are significantly more likely to seek PNC services (36%) than women in control upazilas (21%).
xvii Total out-of-pocket expenditures for ANC, PNC, and delivery care are significantly lower in DSF program upazilas (Tk. 1,441) than in control upazilas (Tk. 2,191).
These results (except for the rate of C-sectons) are statistically significant. The results are also substantiated by rigorous multivariate regression analysis, which adjusts for potentially confounding factors to confirm that the results are likely attributable to the DSF program. Since evidence shows that delivery with a skilled birth attendant reduces the risk of maternal death, these results are very promising, especially in the Bangladeshi context where maternal mortality ratios are very high. Reducing maternal deaths also saves newborn lives, and has substantial social and economic benefits to the family and society.
There is some indication that awareness of the importance of delivering with a qualified provider is nonetheless still limited, which raises concern about program sustainability and brings attention to the need for a targeted awareness-raising and behavior change campaign (BCC).
SUPPLY-SIDE IMPACT
Supply-side improvements have not consistently kept up with the increased utilization of maternal health services resulting from the DSF program. While the primary-level Upazila Health Complexes (UHCs) in program areas have received some additional funding in the form of a “seed fund,” the use of this fund for improving quality has varied across upazilas. Key findings are that: Overall, UHCs in DSF program areas are seeing a dramatically and significantly higher number of patients for voucher-covered services than UHCs in control areas. The incidence of stillbirths is significantly lower in DSF program UHCs than in control UHCs; the incidence of newborn deaths is substantially lower; and there is no statistically significant difference in maternal death rates between these facilities. Human resources shortages at health facilities present a serious challenge to implementing the DSF program. National and upazila-level key informants, providers, and individual women expressed concerns about staff shortages – which are contributing to long lines at health facilities and providers feeling overloaded. In general, UHCs in DSF areas possess a greater proportion of recommended supplies for normal and complicated labor and delivery; these facilities also have slightly more recommended equipment and medications than control facilities to manage hemorrhage, eclampsia and obstructed labor. More UHCs in the DSF upazilas surveyed (5 out of 8, compared to 2 out of 8 control UHCs) have received emergency obstetric care (EOC) upgrades, which likely contributes to this finding. About three-quarters of providers indicated that they enjoy working with the DSF program; their main complaints relate to inadequate compensation given the increased workload.
VOUCHER OPERATIONS Several aspects of voucher operations were assessed using qualitative interviews with key informants and providers, as well as the quantitative household survey. Key findings are summarized below.
Targeting and eligibility:
xviii Economic Evaluation of DSF Voucher Program in Bangladesh Overall, 71% of women in DSF upazilas received a voucher booklet. Limited targeting of poor women is occurring in those upazilas where voucher eligibility is intended to be means-tested. While women from poorer quintiles are significantly more likely to receive vouchers than women from wealthier quintiles, there is substantial leakage to women who do not meet land ownership criteria (18% of voucher recipients), asset ownership criteria (75% of voucher recipients) and income criteria (91% of voucher recipients). However, some poverty-related eligibility criteria are very narrow. While most women surveyed in means-tested upazilas would meet the land ownership criterion (79% are functionally landless), less than one-quarter own no productive assets, and only 7% reported household income less than Tk. 2,500 per month. Parity and contraceptive use restrictions are not being fully enforced. Thirteen percent (13%) of voucher recipients in “universal” program areas and 9% in means-tested areas were parity 3 or greater. Vouchers were distributed to approximately equal proportions of parity-2 contraceptive users and non-users. From April 2007 through August 2009, about 80% of the target number of vouchers were distributed. Means-tested upazilas are closer to achieving their targets (94%) than universal upazilas (71%).
Incentive payments and financial management: Delays in the disbursement of funds, both from the central level to the upazilas and from the upazila to the beneficiary or provider, are a serious challenge for the DSF program at an operational level. The Government of Bangladesh’s standard financial regulation that unused monies be returned to the Treasury at the end of the fiscal year presents an additional, serious disruption to voucher program operations. Among voucher recipients who delivered with a qualified provider, 69% in universal areas and 67% in means-tested areas reported receiving the cash incentive, by the time of the survey. (These rather low rates may be influenced by the fact that our survey timing coincided with the beginning of the new fiscal year, before DSF funds were transferred to the upazilas.)
Use of cash stipend: Most women who received the cash stipend for delivering with a qualified provider (Tk. 2,000) reported using it for food or medications, as intended by the DSF program design.
Role of the private sector: Few private and NGO facilities are part of the DSF program, either because facilities are not available or not interested, or because quality of care is considered to be below standard. Key informants felt that greater efforts should be made to involve private sector and NGO facilities in the voucher program, in order to stimulate competition and improve quality.
Monitoring:
xix Aside from external monitoring by DSF coordinators, who also perform program administrative duties, the monitoring and evaluation system for the DSF program is inadequate and not integrated with routine management information systems in health facilities. Specific monitoring of C-section rates seems to have kept the rates of surgical deliveries low; while monitoring to prevent abuse of the system is needed, it is important going forward to ensure that women needing C-sections have access to this care.
Other voucher operations: Shortages of supplies and medicines are an obstacle to smooth DSF program implementation. Many voucher recipients are still paying for medicines outside of government health facilities, due to stock-outs. The seed funds may be helping address these shortages but they have not solved the problem; cash incentives may help cover these out-of-pocket costs, but they are not provided at the time of service and therefore do not eliminate the financial barrier.
DSF PROGRAM COSTS
The average cost per voucher distributed (based upon the direct costs of the DSF program) is estimated to be US$ 41. In addition to program administrative costs, this cost includes incentives to pregnant women as well as to providers and facilities (through the seed fund).
DSF program benefits extend beyond safe motherhood and reduced maternal mortality. For example, health facilities and non-maternal health patients will benefit from any quality improvements at the DSF facilities. This is important to keep in mind when considering the benefits vs. cost of scaling up the DSF program.
KEY RECOMMENDATIONS
The report provides a range of recommendations; key ones are listed here.
Given how rapidly and extensively the provision of safe motherhood services improved in this pilot, and given the average voucher program cost of $41 per voucher distributed, we recommend expanding this program (with some modifications as discussed in the report) to other areas in Bangladesh. Continued investment in EOC upgrades is recommended. Greater efforts should be made to encourage facilities to use the seed fund for quality improvements, as well as to procure drugs, supplies, labor beds, and equipment. We recommend further advocacy to include private and NGO facilities (that meet quality standards) as this may increase the available supply of services for voucher beneficiaries, as well as potentially improve quality through competition While there is good overall awareness of the DSF program, information and behavior change communication campaigns should be launched to improve understanding of specific DSF benefits, as well as broader safe motherhood messages.
xx Economic Evaluation of DSF Voucher Program in Bangladesh We recommend further evaluation (if possible, involving randomization of new DSF areas) of the advantages and disadvantages of means-tested versus universal voucher eligibility. Since little poverty-related targeting is occurring and therefore targeting does not account for differences in rates of delivery with qualified providers or institutional delivery, we are unable to recommend one model over the other. We note that poverty-related targeting is difficult and costly, prone to leakage, and subject to potential corruption. However, expanding universal voucher provision will be costly as well. Further evidence is required. The current poverty eligibility criteria, if fully enforced, would limit DSF program eligibility in means-tested areas to a very small number of women. Policymakers may wish to consider relaxing the asset ownership and income criteria in particular. Whether or not eligibility criteria are revised, we recommend that greater emphasis be placed on standardized, transparent application of those criteria. This will likely require better guidance to health workers, better monitoring, and possible incentives for accurate targeting. Without such monitoring, corruption in the allocation of vouchers and incentives will always be of concern. The services covered by the DSF program should be expanded to explicitly include family planning counseling and essential neonatal care. We recommend adding a family planning page in the DSF voucher booklet to ensure that a provider signs off on providing family counseling after delivery. If a policy change is at all possible, remaining DSF funds at the end of the fiscal year should be allowed to stay at the upazila level and not be returned to the Treasury. Before the program is scaled up, the logistics of making transportation stipends and cash incentives available to women at the time they receive a service should be resolved. Continuous availability of funds at the upazila level is essential for ensuring this program’s success. Since Bangladesh faces a critical shortage of human resources in health, particularly in rural and remote areas, particular emphasis should be placed on ensuring staff availability at DSF facilities. This may require implementation of a human resources incentives program, such as was detailed by Luoma et al. (2010). The program will lose its momentum in the absence of qualified providers at the facility. More broadly, we recommend that the currently small DSF program office be expanded to serve as a full-fledged Voucher Management Agency. Several full-time technical staff will be needed to ensure smooth operations, track finances, monitor adherence to policy and program results, and report to the MOHFW. A Deputy Program Manager from DGHS can chair the DSF voucher management unit; however a full-time manager should be responsible for overseeing all activities. The role of the voucher management unit may be outsourced as it is in done in some other countries, but this should be carefully considered in the Bangladeshi context.
xxi
1. INTRODUCTION
As part of its Health, Nutrition and Population Sector Programme (HNPSP), Bangladesh’s Ministry of Health and Family Welfare (MOHFW) is implementing a pilot demand-side financing (DSF) maternal health voucher program in 33 upazilas (sub-districts) around the country. The program distributes vouchers to pregnant women entitling them to access free antenatal, delivery, emergency referral, and postpartum care services, as well as providing cash stipends for transportation and cash and in-kind incentives for delivering with a qualified health provider. The program also provides incentives to health care providers for identifying eligible women and providing maternal health services. When initially implemented in August 2006, women identified as extremely poor were eligible for vouchers; in 2008, eligibility was made universal in 9 upazilas. The objective of the program is to increase the use of qualified birth attendants and mitigate the financial costs of delivery, as part of Bangladesh’s efforts to reach Millennium Development Goal 5 and achieve a 75% reduction in maternal mortality by 2015.
The German Gesellschaft für Technische Zusammenarbeit GmbH (GTZ) provides technical assistance to the Government of Bangladesh, and to the MOHFW’s Health Economics Unit (HEU) in particular, related to health financing, health economics, and monitoring and evaluation processes. As part of this assistance, GTZ has funded a comprehensive evaluation of the DSF program, including an operational assessment and impact evaluation. GTZ contracted a project team led by Abt Associates Inc. (based in Bethesda, MD, USA) in partnership with Research, Training and Management International (RTM, based in Dhaka) to conduct the evaluation. This report summarizes the evaluation’s findings. Two other studies, prepared concurrently with this one, provides relevant data and recommendations to consider: one study focuses on costing of maternal health services, providing unit costs of key services as well as resource requrements for achieving MDG 5 (Chankova et al,, 2010); the second study recommends inentive designs to improve retention and performance of public sector doctors and nurses in rural and remote areas (Luoma et al., 2010).
1.1 OBJECTIVES OF THE EVALUATION
The objectives of the evaluation were as follows:
1. To comprehensively describe and assess voucher program operations, including: Issues related to program design (eligibility, benefits, incentives) Implementation and logistics (awareness raising, targeting beneficiaries, voucher distribution, payment of benefits to consumers and providers, provision of covered services, and monitoring and evaluation) Financial management of the program Institutional arrangements and governance Analysis of the number of vouchers distributed and benefits provided through the program.
Economic Evaluation of DSF Voucher Program in Bangladesh 1 2. To evaluate voucher program impacts on the “demand side”, specifically assessing impacts on: The rate of deliveries attended by a qualified health provider and/or in a health facility The rate of surgical deliveries Use of antenatal and postnatal care Out-of-pocket expenditures on covered maternal health services Poor vs. non-poor women, as defined by wealth quintiles based upon a household survey.
3. To evaluate voucher program impacts on the “supply side”, specifically assessing The effect of the program on provider behavior Volume of services provided Quality of care provided in voucher facilities (staffing, infrastructure, supplies, equipment, medicines, knowledge and skills) Impacts on other health programs, including family planning.
A pre-post analysis of the upazila-level aggregate impact of the DSF program on maternal health services use and health outcomes was originally specified in the terms of reference, but was not conducted due to data limitations. The evaluation primarily relies on matched control comparisons, given that pre- intervention baseline data are no longer available. Multiple quantitative and qualitative methods were used to obtain an in-depth, comprehensive overview of the program, its effectiveness, and its limitations. It is hoped that the results from this study will significantly contribute to the evidence base as the MOHFW considers future health financing options in Bangladesh and strategies for reducing maternal mortality.
1.2 ORGANIZATION OF THE REPORT
The report begins with a literature review on demand-side financing approaches around the world, followed by a detailed description of the Bangladesh DSF program (chapter 2). Methods for the evaluation are described in chapter 3. In chapter 4, we present our assessment of voucher operations. Chapter 5 summarizes findings on the impact of the DSF program on individual demand for maternal health services, while chapter 6 describes findings related to the supply-side impacts of the program. We discuss the main conclusions of the evaluation in chapter 7 and provide recommendations to policymakers in chapter 8. A number of annex tables are appended to the report for reference.
2 Economic Evaluation of DSF Voucher Program in Bangladesh 2. BACKGROUND AND LITERATURE REVIEW
2.1 MATERNAL HEALTH IN BANGLADESH
Bangladesh has made great strides over recent decades in improving maternal and child health indicators, especially with regards to infant and child mortality, total fertility rates, and immunization coverage. Maternal mortality ratios have also been declining, from 574 maternal deaths per 100,000 live births in 1991 to 320 in 2001 (NIPORT et al., 2003), although current estimates by different agencies put this in the range of 270-290. However, maternal mortality is still unacceptably high in Bangladesh – five times higher than in Sri Lanka or Vietnam and ten times higher than in Malaysia, for instance. According to the most recent Bangladesh Demographic and Health Survey (BDHS) from 2007, approximately 85% of all deliveries in Bangladesh took place at home as of 2007, and only 18% of births were assisted by a medically trained provider. Most women were not receiving antenatal or postnatal care services.
Millennium Development Goal 5 (MDG 5) requires Bangladesh to cut maternal mortality by 75% between 1990 and 2015, to 143 deaths per 100,000 live births. Achieving this target presents a severe challenge. To reduce maternal mortality, the government of Bangladesh has implemented a number of “supply side” or input-oriented efforts, including strengthening Essential Emergency Obstetric Care (EmOC) services in many health facilities, providing training to field-based health outreach workers (known as Family Welfare Assistants [FWAs] and Health Assistants [HAs]) to create a pool of Community-based Skilled Birth Attendants (CSBAs), and implementing the National Nutrition Program (NNP), which provides free food items for pregnant women in program areas for 150 days during pregnancy. Institutional delivery and skilled attendance at birth are the two indices most associated with improving maternal survival. EmOC services for pregnant women are now available in 59 out of 61 District Hospitals and in 132 out of 407 subdistrict-level Upazila Health Complexes (UHCs).
However, Bangladesh along with many governments around the world has acknowledged the need to address not only supply-side but also demand-side barriers that prevent the poor from seeking basic health care services (Gottret, Schieber, & Waters, 2008). These barriers can include lack of education about when to seek care, lack of information about what care is available, transportation barriers, intrahousehold and gender preferences, and financial barriers (Ensor, 2003). Evidence has indicated that supply-side subsidies often fail to reach the poor, since wealthier population groups are better positioned to take advantage of them. Demand-side financing approaches are being piloted in several countries as potential methods for overcoming some demand-side barriers. Demand-side financing of health services refers to channeling government health subsidies directly to consumers and thus transferring purchasing power to those who need it most. Evidence is emerging that demand-side financing can substantially improve targeting of the poor while overcoming financial, transportation, and information barriers.
To explore the potential of a demand-side approach for reducing maternal mortality in Bangladesh, the Ministry of Health and Family Welfare (MOHFW) and the Directorate General of Health Services (DGHS), with the technical assistance of the World Health Organization, launched a pilot maternal
Economic Evaluation of DSF Voucher Program in Bangladesh 3 health voucher scheme in 2004. The main objective of this program was to accelerate progress toward MDG 5 by stimulating increased utilization of maternal health services by poor pregnant women, including ANC, obstetric and post natal care. The voucher scheme is designed so that women can access safe delivery care at home or in institutions, necessary antenatal and post natal check ups, and antenatal- and delivery-related complication management through ensured referral services.
2.2 INTERNATIONAL AND LOCAL EXPERIENCES WITH DEMAND-SIDE FINANCING PROGRAMS
To date, the literature has shown that demand-side financing approaches can be effective complements to supply-side financing schemes, particularly for subsidizing the demand for known cost-effective services (Bhatia and Gorter, 2007).
One demand-side approach is the conditional cash transfer (CCT). CCTs provide financial support to poor families in exchange for certain behaviors that are beneficial to the household, such as bringing children for regular medical check-ups (Gottret, Schieber, & Waters, 2008). Overall, literature from several countries indicates that some CCT programs have been successful at motivating the poorest women to seek preventive and primary care for themselves and their children and at improving health status (Rawlings and Rubio, 2005; Lagarde, Haines & Palmer, 2007; Morris et al., 2004). In India, the implementation of a CCT scheme was associated with an increase in facility deliveries (Devadasan et al., 2008). In Mexico, the Oportunidades CCT program led to faster growth rates, reduced anemia, and lower illness rates among participating children (Gertler, 2004; Berhman and Hoddinott, 2005). An evaluation of a children’s nutritional CCT program in Colombia found that the program led to improved nutritional status, reduced morbidity, increased health facility visits, and increased consumption of protein and vegetables (Attanasio et al., 2005). However, a similar Brazilian nutritional CCT program was associated with a small negative effect on children’s nutritional status; this could have been a result of differences between intervention and control groups prior to the start of the program or of a belief that children would be excluded from the benefits if they gained weight (Morris et al., 2004b). In addition, the Opportunidades CCT in Mexico was shown to improve the quality of antenatal care (Barber and Gertler, 2009). Finally, CCT programs can also help families to repay household debt (Borghi et al., 2006).
One criticism of conditional cash transfer schemes is that they may not lead to increased access to services, since cash is transferred to recipients after the desired behavior has occurred (Borghi et al., 2006). Evidence on the long-term impact of CCTs is still limited, although evidence on Mexico’s 13-year effort is now becoming available (Fernald et al., 2008; Leroy et al., 2008). An additional challenge with CCTs is that they often face high operational and transactional costs (Bhatia & Gorter, 2007).
Another demand-side approach involves using vouchers to subsidize specific behaviors. Vouchers are like coupons that are given for free or sold to eligible participants, entitling them to specific health services from participating providers. While there is a paucity of rigorous, hypothesis-driven studies assessing the effectiveness of voucher schemes, the current evidence indicates that, when designed and implemented well, voucher programs can address some of the limitations that other demand-side financing schemes face. For instance, because vouchers are provided in advance of service utilization, they can help to overcome access barriers that CCTs may not address. Vouchers can be used to accurately target priority groups and areas for subsidization (Bhatia & Gorter, 2007). They have the potential to increase consumers’ choice of providers and improve quality of care by stimulating competition among providers for voucher clients, as well as increase the demand for services by simply informing clients about the program (Borghi et al., 2006; Bhatia & Gorter, 2007). Additionally, vouchers
4 Economic Evaluation of DSF Voucher Program in Bangladesh can reduce any incentive among providers to recommend and perform expensive and unnecessary services, because they only entitle recipients to a defined set of services (Bhatia & Gorter, 2007).
Vouchers are a particularly appealing approach for addressing demand-side barriers to maternal health services. The target population (pregnant women) is a vulnerable priority group that can be easily identified; a package of necessary maternal health services worthy of subsidy can be clearly defined; and the financial barriers to service use – particularly in the case of pregnancy complications – are substantial. Maternal health vouchers can be distributed not only for antenatal and delivery care, but for transportation and postpartum services as well. Because they are distributed in advance of delivery, they obviate the need for women to raise cash during labor and may improve birth planning. To date, there is preliminary evidence from a maternal health care voucher scheme in Kenya showing an increase in the number of deliveries with skilled birth attendants (Lenel and Griffith, 2007). A similar program in India has also been associated with an increased number of facility deliveries; some of the lessons from this program include the need for clearer guidelines about quality of care practices, standard definitions in obstetric care, and differential pricing for normal and complicated deliveries (Gorter and Bellows, 2008; UNFP, 2006).
A program in Cambodia has increased the use of vouchers for facility deliveries; however, the number of facility deliveries among those paying out-of-pocket has also increased, therefore making it difficult to distinguish the intervention effect from that of external factors (Horeman, Narin and Van Damme, 2008). Lessons from this study emphasized the need to properly identify eligible women and to account for other factors that discourage service use among women, such as the lack of transportation at night or a person to accompany them to the facility, and the lack of someone to tend to household chores while they are away (Horeman, Narin and Van Damme, 2008).
Vouchers are also being used to increase demand for reproductive health services more broadly. A family planning voucher scheme in Nicaragua was found to increase the quality of sexual and reproductive health care for adolescents (Meuwissen et al. 2006). Similarly, in Uganda, the distribution of vouchers led to an increase in the number of STI diagnoses at participating facilities (Lenel and Griffith, 2007). Kenya’s voucher program also subsidizes long-term and permanent contraception methods, but uptake of family planning (FP) vouchers has been much lower than uptake of safe motherhood vouchers (Kundu, 2007; Lenel & Griffith, 2007). A recent analysis of this program indicated that the FP services covered by the vouchers needed to be expanded to make them more appealing to target groups. The analysis also emphasized the importance of assessing the target population’s willingness-to-pay (since vouchers were sold rather than given away for free); accounting for providers’ preferences for services to be included in the package before the start of the program; and ensuring that reimbursement to providers for certain services is adequate (Patsika, O’Sullivan & Child, 2009).
In Bangladesh, there are experiences with at least three other voucher programs for maternal health services (aside from the DSF program evaluated here), although all are small in scale. One program implemented by the International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR,B) is the Commununity Health Project in Chakaria Upazila of Cox’s Bazaar district (Iqbal et al. 2009). Vouchers provided access to ICDDR,B operated village health posts, as well as to community-based SBAs trained through the project, and a doctor who visited the health post once a week. Women did not receive any cash incentives besides the voucher. Referrals were made for complicated cases. Over 2006 and 2007, among lowest two quintiles, ANC services doubled from 27.6% to 51.9%; deliveries with SBAs increased from 3.8% to 15.2%; and PNC services increased from 2.7% to 21.5%. SBAs were compensated directly based on voucher use which, along with the provision of free services for poor women, were likely to be key drivers in increasing utilization.
Economic Evaluation of DSF Voucher Program in Bangladesh 5 Another voucher scheme in Bangladesh, called Demand-Based Reproductive Health Commodity Project, was funded by the Canadian International Development Agency, through UNFPA. It was jointly piloted by and RTM International, ICDDR,B, and the Population Council, who also undertook an operations research study "Voucher Scheme for poor rural women to utilize pregnancy care", in two unions of Nabiganj Upazila of Habigonj district (RTM, 2009). Women received ANC, delivery and PNC services for free at designated public facilities, as well as an NGO and private facility at the district level. Women received Taka 500 for transport stipend (as in the MOHFW DSF program) but did receive any additional cash incentives. Providers received the same cash incentives as in the MOHFW DSF program. A key component of the program was behavior change communication through a Community Support Group, who played an active role in identifying the pregnant women and also community mobilization and awareness raising about services covered by the voucher program. Percent of women receiving at least 3 ANC services increased from 30 to 63 precent; delivery by a qualified provider increased from 5.5% to 22%; and institutional delivery incrase from 2.3% to 18%.
Marie Stopes Clinic Society, with financial assistance from the European Commission, is implementing a voucher scheme, alongside other interventions, in three areas: Shariatpur, Bhola, and Barisal districts (Marie Stopes Clinic Society, 2010). Selected pregnant women receive vouchers to access ANC services for free at Marie Stopes clinics and safe delivery services at designated public and private facilities. A transportation voucher is also provided to facilitate access. The program started in 2007, and to date voucher utilization rate is 67%-72%.
Further rigorous evaluation research is needed to identify whether voucher programs are successful at improving health outcomes in a cost-effective manner. To date, some of the challenges associated with vouchers include high administrative costs and some substantial management complexities (as with conditional cash transfers); risk of a black market emerging; the potential for the increased demand for subsidized services to overwhelm health facilities and overburden health personnel; and the potential to skew service provision towards voucher-subsidized services at the expense of other valued health services (Bhatia & Gorter, 2007; HEU, 2008). Concerns have also been raised about the sustainability of both vouchers and cash transfer programs, given that donor funding has supported almost all such programs to date.
Bangladesh’s pilot safe motherhood demand-side financing program, the subject of this evaluation report, aims to increase the demand for antenatal care, facility deliveries, and postpartum services among poor women through a combination of vouchers for services and transport and cash payments for institutional deliveries. A early rapid assessment of the program conducted in 2008 found that the utilization of maternal health services increased despite wide variation in program implementation across study areas (HEU, 2008). During interviews, some patients reported that the vouchers were helpful, but that they often paid out-of-pocket for medicines. Providers indicated that the incentives for participating in the program were small in comparison to the increased work load. These results suggested that scaling-up the voucher program would require more staff and supplies in order to meet the increased demand and to reach more people (HEU, 2008; UNFPA, 2006). The current evaluation aims to provide a more detailed and up-to-date picture of Bangladesh’s safe motherhood voucher program, including its operational effectiveness and impact on health outcomes.
In sum, the evidence to date suggests that voucher programs, when implemented well, can be effective at targeting the very poor for subsidizing demand and in improving service quality by stimulating competition among providers (Lenel and Griffith, 2007). The present study uses qualitative and quantitative data to analyze the impact of demand-side financing on the demand for maternal health services and on household expenditures among eligible women in Bangladesh, and will contribute to the literature on the effectiveness of voucher and cash transfer programs in resource-limited settings.
6 Economic Evaluation of DSF Voucher Program in Bangladesh 2.3 THE DSF MATERNAL HEALTH VOUCHER PROGRAM IN BANGLADESH
2.3.1 PILOT DSF PROGRAM AREAS
Bangladesh’s Demand Side Financing (DSF) program was initially launched in July 2004. Because of various complications including delays in the receipt of donor funds, the program was not implemented until August 2006, when it was only initiated in 2 upazilas (subdistricts).1 Between April and July 2007, DSF activities were initiated in 19 additional upazilas, and in 2008 an additional 12 upazilas were added. Currently, the program is operational in 33 upazilas across 31 districts, covering a population of 8.75 million as of the 2001 census (BBS 2001) which is currently estimated at 10.36 million people by the DSF program. This evaluation focuses on the 21 upazilas where the program was functioning by mid-2007, 9 of which have universal eligibility and 12 of which have means-tested eligibility for vouchers.
2.3.2 FUNDING AND MANAGEMENT STRUCTURE
The DSF voucher program is implemented by the MOHFW using pooled funds co-financed by the World Bank, United Kingdom, the European Community (EC), Germany, Sweden, Canada, Netherlands, and the United Nations Population Fund (UNFPA). The World Health Organization, with co-funding from the UK’s Department for International Development (DFID), provides technical assistance to the DSF program. This includes administrative and monitoring support through the posting of DSF organizers to each DSF upazila. These coordinators play a key role at the local level in assisting Upazila Health Complex (UHC)2 management to run the DSF program. They are overseen by a National DSF Coordinator, based in the national DSF cell in Dhaka.
Funds for the DSF program are transferred from the Central level to Sonali Bank accounts in each DSF upazila to cover incentive payments to providers and voucher beneficiaries, as well as the cost of procuring gift boxes for beneficiaries. There are four separate bank accounts at the upazila level: a beneficiary cash incentive account, a transportation stipend account, a provider incentives account, and a “seed fund” account. Funds are normally transferred twice in a fiscal year, with each installment typically covering payments for two quarters as an "advance;" this is settled after accounting for vouchers redeemed. Due to GOB public finance regulations, any unused funds are returned to the treasury at the end of the fiscal year, June 30. Funds for the new fiscal year generally do not arrive on July 1, as regular processing time can take up to a few months. During this time, the DSF upazilas do not have any funds with which to provide the voucher-covered benefits.
Each DSF pilot upazila was given a one-time lump sum budget of US$1,000, which was deposited into the “seed fund” account. This was to be used to improve the quality of care at UHCs, through infrastructure improvements or the purchase of medicines and equipment.
The National DSF Committee, chaired by the Health Minister, provides strategic and policy oversight of the entire program. The Secretary of the Ministry of Health and Family Welfare (MOHFW) is the designated Line Director for the program. The National DSF Program Implementation Committee, with
1 Bangladesh is organized into 6 Divisions, 64 Districts, 508 Upazilas (sub-districts), and 4,466 Unions (BBS 2008). Villages are the lowest geographical unit, below the union. 2 The Upazila Health Complex is the main subdistrict-level primary health care facility. Most also provide basic inpatient care services.
Economic Evaluation of DSF Voucher Program in Bangladesh 7 input from a Technical Subcommittee, is responsible for overall program operations. Below this, district-level Designation Bodies manage the selection and accreditation of participating health care providers in each pilot area. Upazila DSF Committees function as the financial and managerial core of the program, while Union DSF Committees assist with identifying eligible voucher recipients, distributing vouchers, and publicizing the program to the community.
The Resident Medical Officer (RMO) in each UHC is responsible for signing and approving vouchers and submitting them for reimbursement, while the Upazila Health and Family Planning Officer (UHFPO) is responsible for distributing incentive payments to women and service providers.
2.3.3 ELIGIBILITY CRITERIA
At its inception, the target beneficiaries of the DSF program were pregnant women on their first or second pregnancy who were considered extremely poor and vulnerable. In the 24 DSF upazilas that are “means-tested,” women meeting the following criteria are eligible for the voucher program:
Permanent residents of the union
Pregnant for the first or second time, and having used family planning prior to the second pregnancy
Functionally landless (owning less than 0.15 acres of land)
Earning extremely low and irregular income or no income (less than Tk. 2,500 per household per month)
Owning no productive assets, such as livestock, orchards, rickshaw or van.
In December 2007, by official government order, eligibility restrictions were relaxed in 9 “universal intervention” upazilas, making DSF benefits available to all permanent resident pregnant women of parity 1 or 2, regardless of poverty status.
2.3.4 BENEFICIARY IDENTIFICATION PROCESS
At the community level, eligible beneficiaries are identified by government field workers known as Family Welfare Assistants (FWAs, who are under the supervision of the Directorate General of Family Planning [DGFP]), and Health Assistants (HAs, who are supervised by the Directorate General of Health Services [DGHS]). Some of these FWAs and female HAs have received sufficient additional training (6 months) to qualify as Community Skilled Birth Attendants (CSBAs). FWAs, HAs, and CSBAs have their own demarcated working areas and a target number of households to which they make routine home visits. They prepare a preliminary list of eligible voucher recipients, ideally within women’s first trimester of pregnancy, and submit it to the local Union Parishad (council) member or Chairman, who verifies and approves the list. The list is supposed to be reviewed and approved as well at Union DSF Committee meetings, but often those meetings are not held regularly. In such cases, the field workers (FWA, HA, or CSBA) directly obtain approval from the Union Parishad Chairman. Then the list of eligible women’s names is forwarded to the upazila level, and is considered final. If an eligible woman does not receive a voucher, she may petition the Union DSF committee.
8 Economic Evaluation of DSF Voucher Program in Bangladesh 2.3.5 VOUCHER DISTRIBUTION
The voucher distribution system is not uniform across upazilas. At the outset of the pilot program, all voucher booklets were to be distributed from the UHC, and in some upazilas this is evidently still the standard practice. Most voucher booklets, however, are distributed to women at the community level by FWAs, HAs, and CSBAs. This is particularly common in areas that are farther from the UHC. Occasionally village dais (traditional birth attendants) have also reportedly distributed vouchers.
Women receive a booklet with separate “coupons” in triplicate for each covered service under the program. When a service is sought, providers keep two copies of the relevant voucher slip, one to submit for reimbursement and one to keep for documentation, and the third copy is returned to the woman.
2.3.6 PARTICIPATING HEALTH CARE PROVIDERS
As noted above, District Designation Bodies are responsible for selecting and accrediting eligible private and NGO providers in the pilot upazilas. Government facilities with the capacity to provide Basic and/or Comprehensive Essential Obstetric Care (EOC), including management of normal and problem pregnancies, surgical obstetrics, anesthesia, and blood transfusion (MotherCare policy brief), may be accredited to participate in the program. These include UHCs and Union Health and Family Welfare Centers (UHFWCs), the latter providing mainly antenatal, postnatal, and normal delivery care. Accredited referral facilities may include District Hospitals, Maternal and Child Welfare Centers (MCWCs), and government Medical College Hospitals. Government-certified CSBAs with at least 6 months of training may also provide ANC and delivery services at home.
NGO and private providers may also apply for accreditation. The Designation Body reviews whether they meet certain quality criteria (e.g., presence of operating theater and qualified providers) and determines whether they may participate. The inclusion of private and NGO facilities in the program is intended to stimulate competition among providers to attract voucher patients, in the hopes that such competition might lead to quality improvements. However, to date very few non-public facilities are participating in the voucher program. NGO and private providers are reimbursed at the same rate schedule as public providers, except that no funds are withheld for a seed fund, as with public providers.
2.3.7 VOUCHER BENEFITS TO CONSUMERS
The Bangladesh DSF voucher program provides the following benefits to eligible women:
3 antenatal care (ANC) check-ups
Safe delivery care in a health facility or at home with a qualified provider (such as CSBA)
Emergency care for obstetric complications, including Cesarean sections
1 postnatal care (PNC) check-up within 6 weeks of delivery
Tk. 500 for routine transport costs (up to Tk. 100 per health facility visit for 3 ANC, 1 delivery, and 1 PNC visit)
Tk. 500 for emergency transport to referral facility if needed
Economic Evaluation of DSF Voucher Program in Bangladesh 9 Tk. 2,000 cash incentive to mothers who deliver in health facilities or at home with a qualified provider, to be used for the purchase of nutritious food and medicines for the mother and infant
A gift box worth up to Tk. 500, including a bottle of Horlicks malted drink powder, a towel, a bar of soap, and two outfits for the newborn.
After distributing the voucher booklet, the distributor is to inform women about the participating facilities and providers from which they can obtain covered services. In case of referral for complications, pregnant women can access services at specific hospitals with a referral certificate. They are also entitled to Tk. 500 to cover the cost of ambulance transport, fuel, or other vehicle rental for referrals.
2.3.8 PROVIDER REIMBURSEMENT AND INCENTIVES
Health care facilities are reimbursed for providing voucher-covered services at fixed rates. Fifty percent of the reimbursement amount is deposited into the upazila “seed fund account”, while the remaining 50% is distributed to the government service providers who provided those services, as an incentive.
The following table displays the breakdown of reimbursements and incentives for each type of service:
TABLE 2.1. REIMBURSEMENTS AND INCENTIVES TO GOVERNMENT PROVIDERS
Description of services Total reimbursement Incentive payment amount (Tk.) (Tk.) Routine care Registration of eligible women 10 Two ANC blood tests (35*2)=70 Two ANC urine tests (35*2)=70 Three ANC check-ups (50*3)=150 (25*3) = 75 One PNC check-up 50 25 Normal delivery 300 150 Medicines 100 Obstetric complications Forceps delivery, manual removal of 1000 500 placenta, D&C, or vacuum extraction Medicines for management of eclampsia 1000 C-section 6000 3000
Incentives are distributed among providers according to the following charts (Table 2.2 shows routine services and Table 2.3 lists for complications):
TABLE 2.2. BREAKDOWN OF INCENTIVES FOR BASIC MATERNAL HEALTH SERVICES, BY SERVICE PROVIDER (TK.)
Service Regis- ANC ANC ANC Normal Normal PNC Daily provider tration check-up urine blood delivery delivery check-up voucher test test (home) (facility) program work FWA 10 HA 10
10 Economic Evaluation of DSF Voucher Program in Bangladesh Service Regis- ANC ANC ANC Normal Normal PNC Daily provider tration check-up urine blood delivery delivery check-up voucher test test (home) (facility) program work FWV 10 25 17.50 25 CSBA 10 25 17.50 75 25 Facility- 17.50 17.50 based clinician Doctor 25 60 25 Nurse 25 40 25 Aya/ward 25 boy Cleaner 25 UHC Clerk 30 / day TOTAL per 10 25 17.50 17.50 75 150 25 30 / day service across providers Note: For any one type of service, only one provider, who provides that service, will get the incentive payment; except for normal delivery, where the multiple providers listed all get incentive payments as shown.
TABLE 2.3. BREAKDOWN OF INCENTIVES FOR MATERNAL HEALTH COMPLICATIONS, BY SERVICE PROVIDER
Service provider Complications* C-section Doctor 300 Nurse 100 Aya/ward boy 50 100 * 2 Surgeon 1,100 Anesthetist 600 Operation assistant 500 Senior nurse 250 * 2 Cleaner 50 100 Total 500 3,000 *D&C, manual removal of placenta, forceps delivery, vacuum extraction. Note that providers do not receive any incentive payment for eclampsia management; the reimbursement amount of Tk. 1,000 is to be used for medication only.
2.3.9 DISTRIBUTION OF CASH AND IN-KIND INCENTIVES TO CONSUMERS AND PROVIDERS
Although originally it was hoped that women would receive cash incentives for transportation after each facility visit and the Tk. 2,000 stipend and a gift box before discharge from a health facility, in fact women do not typically receive cash when they receive the service. Usually, all voucher recipients must come to the health facility to receive their reimbursement on specific dates. This means payments are typically received with a delay of several months. There is some evidence from the field that some women receive cash through the FWAs, HAs, and CSBAs at their homes.
Economic Evaluation of DSF Voucher Program in Bangladesh 11 The UHFPO is responsible for distributing incentive payments to service providers. This also usually occurs monthly.
2.4 SUMMARY OF FINDINGS FROM 2005 BASELINE REPORT
In 2005, prior to the initiation of the DSF program, a baseline study was conducted in planned DSF intervention areas and matched control areas to assess the status of maternal health service provision, utilization, and expenditures. The study consisted of a document review, a household survey of pregnant women, and an inventory of service providers. Hard copies of the baseline report are available although the original data are not. For brevity, only the methods and a summary of key findings from the household survey of pregnant women are discussed below. These results should be used for general contextual information only and are not directly comparable to the current evaluation results.
The 21 proposed DSF upazilas were matched with control upazilas selected from the same districts, based on literacy rate, population density, poverty rate, and presence of a skilled birth attendant (SBA) program. A sample of 30 pregnant women from each upazila was estimated to be adequate for the household survey, for a total sample size of 1,267. Multistage random sampling was adopted; one union from each upazila was randomly selected and one village was selected randomly from each union. If the desired number of households with pregnant women could not be covered by one village, then one or more neighboring villages were selected to achieve the target sample size of 30 pregnant women from each upazila.
Pregnant women who would be eligible for the DSF program were sampled for the survey. Women were eligible if they were pregnant at the time of the baseline survey, owned less than 0.15 acres of land, earned less than 300 Taka per month per capita, and lacked productive assets.
Findings from the 2005 survey revealed that rates of maternal health service utilization were very low at baseline, in both intervention and control areas:
Pregnant women in control areas were more likely to have received any ANC (50% in control vs. 44% in intervention areas), and more likely to receive care from a Family Welfare Visitor (30% in control vs. 21% in intervention area).
Most births occurred at home. Slightly more currently pregnant women in control areas (14%) had their previous delivery in a facility, compared to the intervention areas (10%). Most deliveries were attended by friends or relatives (84% in control and 93% in intervention areas).
Women in intervention areas were more likely to have had a C-section for their previous delivery, but the rate was extremely low (2.3% in intervention vs. 0.5% in control areas)
Most women in both control and intervention areas planned to deliver their current child at home or at their parent’s home (94% in control and intervention areas).
About two-thirds of women (64% in intervention and 68% in control upazilas) practiced family planning after the first pregnancy or between the first and the current pregnancy, while one-third did not practice any method.
Average out-of-pocket expenditures for maternal health services (e.g. consultation fees, tests, medicines, transportation) were higher in DSF areas than in control areas.
12 Economic Evaluation of DSF Voucher Program in Bangladesh Note that the baseline survey methodology differed from this evaluation’s household survey methodology in several important ways. Different control upazilas were matched with the DSF upazilas. In the 2005 baseline survey pregnant women were interviewed, rather than women who had delivered in the previous 6 months as in the current survey. Consequently, baseline information about location and provider of delivery referred to the most recent previous pregnancy and thus did not have a specific reference period. As well, only women who met the poverty eligibility criteria were interviewed, rather than women of all socioeconomic groups.
Economic Evaluation of DSF Voucher Program in Bangladesh 13
3. METHODS
3.1 OVERVIEW OF DATA COLLECTION METHODS
We conducted a multidimensional assessment of the operations and impact of the Bangladesh Demand Side Financing (DSF) voucher scheme, using both qualitative and quantitative methodologies. The specific methods of the study were:
Literature review: To produce a background summary of international and local experiences with voucher programs
Administrative reports and secondary data: To conduct upazila-level assessments of voucher scheme operations
Key informant interviews: To understand the institutional and governance arrangements of the DSF voucher scheme including design, operational, and financial issues
Provider interviews: To understand the operations of the DSF voucher scheme, assess performance variation, and understand the role of incentives
Facility survey and quality checklist: To measure the resources available for and quality of Basic and Comprehensive Essential Obstetric Care
Household survey: To assess the individual-level impacts of the DSF voucher scheme on service utilization and expenditures
Focus group discussions (FGDs): To assess women’s satisfaction with the voucher scheme, their reasons for satisfaction or dissatisfaction, and reasons why some women do not use the voucher scheme.
3.2 INSTRUMENT PREPARATION AND PRE-TESTING
Abt and RTM experts collaborated closely in the design of all instruments, drawing on previously validated instruments, particularly those previously utilized in Bangladesh, wherever possible. Senior project staff reviewed all instruments for quality assurance purposes. A stakeholder technical consultation workshop was convened in Dhaka in June 2009 to share the project’s methodology, objectives, and data collection tools. Relevant stakeholders (including GTZ, the MOHFW’s Health Economics Unit, other representatives of MOHFW, and international HNPSP consortium members) provided input and their key recommendations were incorporated into the data collection protocols.
The draft study protocols were approved by Abt Associates’ Internal Review Board (IRB) in July 2009. The Bangladesh Directorate General of Health Services (DGHS) issued an official letter authorizing the team to conduct surveys with government health workers at their facilities. All new instrument questions and instructions (not derived from instruments previously utilized in Bangladesh) were
Economic Evaluation of DSF Voucher Program in Bangladesh 15 translated into Bangla by RTM, and senior RTM staff checked the quality of the Bangla translations against the original English versions. RTM staff then pretested all data collection instruments in one intervention upazila in July 2009. Key informant and provider interview protocols as well as the facility quality checklist were pretested in the Upazila Health Complex (UHC), while the household survey was pretested with eight women in a nearby village who were identified with the assistance of a local CSBA. Five of these women and one additional locally recruited woman also consented to participate in the pre-test of the focus group discussion protocol. Based on the results of the pretests, the tools were reviewed and revised by senior Abt researchers.
3.3 SELECTION OF UPAZILAS FOR THE EVALUATION
The twenty-one (21) pilot DSF upazilas (“intervention upazilas”) with the longest duration of voucher implementation were selected for this evaluation.3 Twenty-one (21) matched control upazilas were selected for comparison to the intervention upazilas. The matching criteria utilized were geographical proximity (location within the same district), number of beds in the Upazila Health Complex, and literacy rate. Literacy rate represents demand-side characteristics by proxying for socio-economic status; bed capacity represents supply-side characteristics in terms of capacity; and geographic proximity captures some unobservable differences. Table 3.1 below summarizes the upazilas selected in Study A and the different types of data collected in each area.
3 The DSF program was initiated in an additional 12 upazilas in 2008, but these areas were not included in our study sample due to limited program implementation time.
16 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 3.1. SUMMARY OF UPAZILAS SELECTED FOR EVALUATION
Intervention Upazilas Data Collection Instruments Data Collection Instruments
Assessment KII, Provider Matched of DSF interviews, Control operations FGDs, and Upazilas Assessment Start and Upazila HH Facility of Upazila HH Facility Division District Upazila U/M* Date level impact survey assessments level impact survey assessments Barisal Banaripara M Apr 2007 √ √ √ Babuganj √ √ √ Barisal Pathuakhali Kalapara M May 2007 √ √ - Galachipa √ √ - Comilla Daudkandi M May 2007 √ - - Chandina √ - - Chittagong Mirsharai M Jun 2007 √ - - Sitakunda √ - - Chittagong Noakhali Chatkhil M Jun 2007 √ √ - Senbagh √ √ - Cox's Bazar Ramu U Aug 2006 √ √ √ Chokoria √ √ √ Narsingdi Raipura M Mar 2007 √ - - Belabo √ - - Jamalpur Sarishabari M Apr 2007 √ - - Melandha √ - - Tangail Sakhipur M Apr 2007 √ √ √ Kalihati √ √ √ Dhaka Faridpur Bhanga M May 2007 √ √ - Boalmari √ √ - Kishoreganj Tarail U May 2007 √ √ - Karimganj √ √ - Manikganj Harirampur U Jun 2007 √ √ √ Shibalaya √ √ √ Jessore Chaugachhia M Apr 2007 √ √ √ Keshabpur √ √ √ Khulna Khulna Paikgachha M Apr 2007 √ √ - Terokhada √ √ - Kushtia Dalutpur U May 2007 √ √ - Mirpur √ √ - Joypurhat Khetlal U Apr 2007 √ - - Panchbibi √ - - Dinajpur Khanshama U Apr 2007 √ √ √ Birganj √ √ √ Rajshahi Sirajganj Shahjadpur U Apr 2007 √ √ √ Raiganj √ √ √ Nawabganj Shibganj U Apr 2007 √ √ - Gomastapur √ √ - Gaibandha Gobindganj U Apr 2007 √ √ - Sadullapur √ √ - Sylhet Habiganj Baniachong M Jun 2007 √ √ √ Lakhai √ √ √ Total sample 21 16 8 21 16 8 Note: 1. U = Universal Access; M = Mean-tested
Economic Evaluation of DSF Voucher Program in Bangladesh17
We selected 16 of the 21 intervention upazilas, and their matched controls, in which to conduct the household survey. These upazilas were selected purposively to include 8 universal and 8 means-tested upazilas and to ensure coverage of all divisions of the country.
Eight (8) of these 16 intervention upazilas were selected for further in-depth qualitative data collection (key informant interviews, provider interviews, facility quality assessments, and focus group discussions). The selection was done purposively, ensuring that half should be universal coverage and half means- tested upazilas and covering all divisions of the country. Facility quality assessments were also conducted in the 8 matched control upazilas.
3.4 SELECTION OF RESPONDENTS
Key informants: The DSF program in Bangladesh is managed by DSF committees at the central, district, upazila, and union levels. Key informants were drawn from these committees at each level in the 8 selected upazilas. Respondents represented senior health facility staff, local government representatives, voucher distributors, field health workers, and NGO and private sector representatives.
Providers: Providers at UHCs and UHFWCs who were involved in the voucher program were interviewed in the 8 selected upazilas. These included FWVs, FWAs, HAs, CSBAs, medical officers, nurses, and gynecologists.
Facilities for quality assessment: The same 8 upazilas selected for key informant and provider interviews were also selected for facility quality assessment. In addition, the facility quality assessment was also conducted in the corresponding 8 matched control upazilas for comparison.
Women who delivered in the prior 6 months (February 1 – July 31, 2009): The household survey sample in Phase 1 of data collection (over July-September 2009) was selected in three stages. First, in each upazila, three (3) unions reported by the UHC to have the largest number of deliveries during the reference period were selected. In each union, 2 villages with the highest estimated number of deliveries were similarly selected. The largest unions and villages were selected in order to ensure that an adequate sample of recently delivered women could be identified. In each village, eligible women who had delivered between February 1 – July 31, 2009 were identified with the help of CSBAs, FWAs, HAs, other health workers, and village residents.
In each of the selected upazilas, 23 women per union or 69 women total with deliveries in last 6 months were interviewed. In some unions, data collectors had difficulty identifying a sufficient sample of eligible women in the two sampled villages. In those instances, they visited a third village to complete the targeted number of interviews. The third village was the closest village from the one where requisite number of respondents was not available. Annex A provides a complete listing of villages included in the household survey in Phase 1.
Focus group discussants: Ten FGDs, one in each of 8 sampled intervention upazilas and 2 additional intervention upazilas, were conducted with both voucher recipients and non-recipients identified during the household survey. Eight to twelve (8-12) women were recruited for each discussion group, immediately after the household interview was completed.
Economic Evaluation of DSF Voucher Program in Bangladesh 19 3.5 FIELD DATA COLLECTION PROCEDURES
3.5.1 HIRING AND TRAINING DATA COLLECTORS
RTM International recruited data collectors with graduate or post-graduate degrees and data collection and supervision experience. Seven (7) days of classroom and field-based training were provided in Dhaka, in Daudkandi upazila (intervention) and in neighboring Chandina upazila (control). For quality assurance of field data collection, an external team of 10 quality assurance officers (QAOs) from GTZ also participated in the training as observers.
Data collectors were organized into 8 teams of 5 members each (4 interviewers and 1 supervisor). Two of the team members in each team were female; women conducted the household interviews while men conducted the provider interviews, key informant interviews, and other facility-level interviews. Field data collection was conducted in 2 phases, over July-September 2009 and November-December 2009.
3.5.2 INFORMED CONSENT PROCEDURES
Informed consent was obtained from each key informant, provider, household member, and focus group discussant prior to the interview, survey, or focus group discussion. The objectives of the survey and the risks and benefits of participation were explained, and the assurance of confidentiality was given to each potential respondent. For the household survey, informed consent was obtained from the household head as well as the woman who delivered during the prior six months.
3.5.3 QUALITATIVE DATA COLLECTION
For each type of qualitative data collection, one data collector took notes while a second conducted the interview or discussion. Key informant and provider interviews were not tape recorded, given widespread unwillingness in Bangladesh to be recorded and the difficulty in obtaining open, forthright responses from government employees even off the record. Focus group discussions were tape recorded.
3.5.4 PHASE 1 HOUSEHOLD SURVEY DATA COLLECTION
Survey teams visited a given intervention upazila and then its matched control upazila in succession, to ensure that comparable interviewing approaches were applied. Data collectors identified households with eligible women following the list they obtained from the nearest Upazila Health Complex, the local HA, FWA, or CSBA or village resident. Some respondents were temporarily resident in their parents’ home as is common for pregnant and postpartum women in Bangladesh, and may not have been permanent residents of the upazila. Informed consent for the interview was first obtained from the head of the household, who was then interviewed about household composition information. Next, the eligible woman’s informed consent was obtained, and she was then interviewed about her background, reproductive history, pregnancy and delivery. Household interviews took place in the respondents’ homes.
Data quality measures included on-the-spot field checks of completed instruments. Supervisors randomly checked 10% of selected questionnaires to ensure that the forms had been accurately filled out. Senior-level team members from RTM also visited the field sites and randomly checked 5% of the questionnaires.
20 Economic Evaluation of DSF Voucher Program in Bangladesh 3.5.5 PHASE 2 HOUSEHOLD SURVEY DATA COLLECTION
After the Phase 1 household survey, there were some concerns that eligible women had been missed because data collectors were targeting 11-12 women per village. Also, since data collectors relied on CSBAs, HAs, and village residents to identify eligible women, they were not using a comprehensive approach to identify respondents. In the case of CSBAs, who assisted with respondent identification, there were particular concerns that CSBAs were more likely to identify women who delivered institutionally, thus introducing potential selection bias.
For these and quality assurance purposes, a supplemental household survey (Phase 2) was conducted in November-December 2009 to assess whether any sample selection bias was present. A shorter version of the Phase 1 questionnaire was developed and used for this purpose. Abt Associates’ IRB approval for the additional data collection and the shortened protocol was obtained on November 13, 2009. We re- visited a stratified random sample of 40 out of 215 villages in the original household survey, including both control and intervention areas. All 16 of the originally visited intervention upazilas were re-visited, while 8 out of the 16 control areas were re-visited. Annex B contains a listing of these villages. Data collection supervisors were instructed to construct lists of all eligible women using input from all potential birth attendants in each village, including dais, HAs, FWAs, and CSBAs, as well as any members of women’s groups, to ensure more comprehensive identification of deliveries. All eligible respondents in those villages were re-interviewed or newly interviewed using the shortened form of the household questionnaire. Interviews were completed with 629 women, of whom 289 had been interviewed in Phase 1 and 340 were newly interviewed.
3.5.6 ADMINISTRATIVE DATA COLLECTION
We aimed to gather maternal and neonatal health service statistics from the health and family planning departments of each Upazila Health Complex, as well as service statistics from Community Skilled Birth Attendants (CSBAs). It was determined during field work that these data were limited and of poor quality in many upazilas.
3.5.7 SUMMARY OF COMPLETED DATA COLLECTION
Table 3.2 summarizes the extent to which data were collected as originally planned, by type of survey instrument.
TABLE 3.2. PLANNED AND COMPLETED DATA COLLECTION
No. Data collection Target Target Number Completed method respondents Number 1 Secondary data Central level MIS 21 upazilas (data 21 upazilas on voucher obtained centrally) distribution and Upazila level MIS 5 upazilas (data 5 upazilas use verified at upazila level) 2 Secondary data Upazila level MIS 42 upazilas 37 upazilas on health (limited data outcomes only) 3 Key informant Upazila-level interviews DSF Designation Civil Surgeon 8 7
Economic Evaluation of DSF Voucher Program in Bangladesh 21 No. Data collection Target Target Number Completed method respondents Number Committee (District (Chairman) Designated Body) NGO and Private 16 5 Sector Members Upazila DSF UNO (Chairman) 8 6 Committee Upazila Parishad 8 7 Member NGO Member 8 6 Union DSF Union Parishad 8 8 Committee Chairman FWV 8 7 FWA and/or HA if up to 16 14 female Upazila Health UHFPO 8 9 Complex RMO 8 8 MOMCH 8 5 DSF Coordinator 8 8 Central level (Dhaka) National DSF DG Health 1 1 Committee Director PHC 1 1 Director MCH 1 1 Services/ DGFP DSF Programme DPM DSF 1 1 Implementation Joint Chief Planning/ 1 1 Committee HEU National DSF 1 1 Coordinator TOTAL KEY INFORMANT INTERVIEWS: 110 to 118 96 4 Provider Upazila Health FWV 8 8 interviews Complex Most informed ANC 8 8 provider Senior Staff Nurse 8 8 MO or gynecologist 8 8 Union Health and FWV 8 8 Family Welfare CSBA or FWA 8 8 Centers (UHFWC) MO if female up to 8 3 TOTAL PROVIDER INTERVIEWS: 48 to 56 51 5 Focus group 8 to 12 eligible women per group 8 groups 10 groups, 88 discussions women 6 Facility quality (NA -- Health facility observation) 16 facilities 16 facilities checklists 7 Household surveys Phase 1 Women who delivered between February 1 2,208 2,208
22 Economic Evaluation of DSF Voucher Program in Bangladesh No. Data collection Target Target Number Completed method respondents Number and July 31, 2009 Phase 2 Women who delivered between February 1 Exhaustive sample in 629 and July 31, 2009 40 villages
Ninety (90) of the originally targeted 112 key informant interviews at the upazila level were conducted. Some targeted key informants were interviewed using the provider protocol rather than the KII protocol, since we could not over-burden the same individual with two interviews. In several upazilas, there were no NGO or private sector members in the District Designated Body. A few respondents were unavailable or out of town during the period of data collection.
3.6 TRANSCRIPTION, TRANSLATION AND DATA ENTRY
All qualitative interviews were translated from Bangla into English by transcribers. Transcribers were strictly instructed to make word-for-word transcripts without changing the meaning or making any additions to the notes that were collected in the field. Key informant and provider interviews were summarized and notes were translated into English. For focus group discussions, written transcripts were supplemented by tape recordings. Qualitative data were reviewed by senior researchers, key and other responses were categorized and coded, and the data were entered into NVivo software.
Administrative and facility quality data were entered in Microsoft Excel and double-checked by RTM staff. Microsoft Access databases were prepared for phase 1 and phase 2 household survey data entry. To ensure accurate data entry, 5% of questionnaires were double-entered. A random sample of entries was also manually checked against the hard copy questionnaires. Data cleaning was performed using Clipper software.
3.7 HOUSEHOLD SURVEY PROCESSING
3.7.1 SAMPLING WEIGHTS
To adjust for the fact that upazilas and unions were not sampled using probability proportionate to size (PPS) as originally designed, a sampling expert at Abt Associates estimated post-adjustment sampling weights using population data from the most recent (2001) census. Weights were assigned to selected women based on the population of unions and villages included in the sample, to approximate the probability of her inclusion had sampling been done PPS. All household survey results included in this report have been adjusted using these weights. Note that since the sample tends to disproportionately include villages with more women who delivered in the 6 months preceding the survey, the weighted estimates may not fully represent the whole country. They will accurately reflect the reality of the study sample. Furthermore, since sampling and data collection procedures were similar between the intervention and control upazilas, weighted statistics should give a reasonable estimate of the differences between the two groups in various background characteristics and outcomes.
Economic Evaluation of DSF Voucher Program in Bangladesh 23 3.7.2 COMPARISON OF PHASE 1 AND PHASE 2 HOUSEHOLD SURVEY DATA
To confirm that the first phase survey provided a valid sample for evaluating the DSF program, several options for comparing samples were employed:
Comparing 2,208 women from the first phase with 629 women from the second phase survey;
Comparing between 289 “second-time” interviewees and 340 “first-time” interviewees of the phase 2 sample; and
Comparing the phase 1 and phase 2 samples of only the 40 revisited villages.
In each scenario, samples were compared on all variables collected in the phase 2 survey, including background characteristics, voucher-related information for intervention areas, and key outcomes. The comparison used Chi2 tests for categorical variables and ANOVA for continuous variables, adjusted for sampling weights. In addition to comparing the full samples, we also compared information separately for control and intervention upazilas.
Table 3.3 presents the comparison results for selected key variables using option 1; comparison statistics for the full set of variables and for the other options are available from the authors upon request. In summary, the only noteworthy difference between the samples is that the percentage of women who reported being aware of the voucher program is higher in the old compared to the new sample. Conversely, the percentage of women who reported receiving a voucher is smaller in the new sample. However, there are no statistically significant differences (at the conventional p<0.05 level) in other voucher information, background characteristics, or outcomes among the set of variables presented in table 3.3. We therefore conclude that the phase 1 and phase 2 surveys are not systematically different as far as the outcomes of interest are concerned. The quantitative analysis presented throughout this report uses data from the first phase survey only.
TABLE 3.3: COMPARISON OF PHASE 1 AND PHASE 2 SAMPLES
Variables Phase 1 Phase 2 p-value (N=2208) (N=629) Background information Married (%) 99.3 98.7 0.436 Household size (persons) 5.6 5.7 0.865 Household monthly expenditure per capita (Taka) 1004 1048 0.265 Last birth was parity 1 or 2 (%) 73.3 71.2 0.829 Voucher information Having heard of voucher program (%) 87.5 96.6 0.002 Received voucher (%) 81.5 70.5 0.008 Received cash incentive for food (%) 62.9 65.3 0.689 Outcomes Had any ANC (%) 83.4 85.2 0.692 Had qualified provider for ANC1 (%) 74.6 66.8 0.112 Had qualified birth attendant (%) 45.6 46.1 0.886 Had institutional birth (%) 28.2 29.3 0.729 Had C-section (%) 9.8 9.0 0.578
24 Economic Evaluation of DSF Voucher Program in Bangladesh 3.8 ANALYSIS
3.8.1 VARIABLE CONSTRUCTION
Most variables used in the household survey analysis were created in a straightforward manner from the dataset, based upon the questionnaire. Two variables were constructed: wealth quintile and qualified provider. They are described in detail below.
Quintile: The wealth quintile variable indicates the respondent’s relative wealth status in comparison to other respondents in the survey. It was constructed using principal components analysis with information on ownership of land, assets, housing conditions, and monthly expenditures per capita. The assets included in the principal component analysis include radio, TV, telephone, refrigerator, almirah, table, chair, watch, bicycle, motorcycle, animal-drawn cart, car or truck, boat, rickshaw or van, and livestock. Housing condition variables include access to electricity and type of roof. The list of variables used for constructing quintiles in our survey is comparable to the list used in other standard household surveys, such as the DHS. However, since our sample consists of women who delivered 6 months preceding the survey in a nonrandom sample of DSF upazilas and matched controls, the constructed quintile variable does not correspond fully to the quintile variable reported in nationally representative surveys. Quintiles were developed in the full sample of women from control, universal, and means- tested areas; there were no statistically significant differences in the socio-economic rankings of these three areas.
Qualified provider: Providers of maternal health services were classified as “qualified” according to the BDHS (2007) report, as described in Table 3.4 below:
TABLE 3.4: QUALIFIED PROVIDERS FOR ANC, DELIVERY, AND POST PARTUM CARE
Providers ANC Delivery care Postpartum care Qualified doctor x x x Nurse x x x Midwife x x x Paramedic x x x FWV (Family welfare visitor) x x x CSBA (Community skilled birth attendant) x x x MA (Medical assistant) x x SACMO (Sub-assistant community medical officer) x x
Multiple indices were created from the facility quality checklist to assess quality of care based upon recommended analyses from the WHO Mother-Baby Package (for personnel, supplies, equipment, laboratory capacity and medications) and from the OBGuide assessment of personnel EOC/ENC background, and EOC knowledge and skills.
3.8.2 ANALYTIC APPROACH
Household survey data analysis was performed using Stata version 10.0 software. The bivariate analyses typically compare the control and intervention area, with intervention broken down into universal and means-tested in most cases. The comparisons use Chi-squared tests for categorical and ANOVA tests
Economic Evaluation of DSF Voucher Program in Bangladesh 25 for continuous variables. Multivariate probit regression analysis was used to estimate the adjusted effect of the DSF program on key binary outcomes (institutional delivery, delivery with qualified provider, C- section) controlling for potentially confounding factors. Multivariate linear regression was used with continuous outcomes (expenditures). Standard errors were adjusted for weights and for clustering at the village level. All results presented in this report are weighted.
Transcriptions from key informant interviews and provider interviews were analyzed using NVivo software. Focus group discussions were analyzed directly through theme coding while the facility quality assessment was analyzed using Excel.
3.9 LIMITATIONS
Initial delays in receiving approvals for this evaluation from MOHFW and HEU led to the delay in the initial national stakeholder workshop by almost two months. This delayed the initiation of all data collection activities and substantially compressed the already tight timeframe for field work, data entry, data cleaning, and analysis. Some field work had to be conducted during Ramadan, which is typically avoided when possible given the burden it places on interviewers and respondents. As well, the rainy season and continuous bad weather in some areas disrupted the process of locating and interviewing respondents. All of these meant that the data collection had to be conducted under heavy time pressure, and more than exhausted the level of effort estimated initially for data collection and data processing.
Based on the lack of adequate and quality secondary data at the upazila level, as understood after the data collection was completed, the Abt/RTM team decided not to pursue the evaluation of upazila-level impacts, after discussions with GTZ, as noted earlier. This is due to widespread lack of health outcomes and utilization data and the lack of pre-intervention data needed for trend analysis.
The Abt/RTM team placed extreme emphasis on careful adherence to rigorous sampling, interviewing, and data entry procedures. However, the original sampling design of using probability proportionate to size sampling of unions and villages was not implemented; rather, the unions and villages with the largest number of deliveries were selected in order to ensure identification of a sufficient sample in each area. This may have introduced some bias if larger villages are systematically different from smaller villages. The approach was identical in intervention and control areas, however, such that relative impact estimates remain valid. Post-adjustment sampling weights based on the most recent Bangladesh census (2001) were also constructed to reduce this potential bias.
Our sample may be slightly better educated than women in the 2007 BDHS, and rates of delivery with qualified providers in our control areas were higher than averages reported in the BDHS. However, as noted above this likely has to do with the fact that our sample includes women who delivered in the prior 6 months only, while the DHS includes all women of reproductive age (15-49) and is older on average, and estimates skilled attendance rates for births in the prior 5 years (2002-2007). Women in the DHS were also likely to be of higher parity than women in our sample due to the reference period for delivery (since women who delivered recently tend to be younger and have fewer children), and higher parity births are less likely to be attended by a qualified provider than lower parity births.
As described above, concerns were raised after the Phase 1 household survey data collection that women who had delivered with a qualified provider may have been more likely to be included in our sample, simply because the sample was identified with the help of government health workers (FWAs, HAs and CSBAs). However, as described in detail above, a supplemental data collection effort – which
26 Economic Evaluation of DSF Voucher Program in Bangladesh exhaustively identified eligible women in each village with the help of dais as well as medical providers – did not find evidence of selection bias in the sample.
Evaluations based on comparisons of baseline (pre-intervention) and follow-up data are preferred for identifying causal impact. A pre-post assessment was not possible in this case due to the absence of comparable pre-intervention baseline data. However, we do address this limitation by comparing births taking place before and after the program’s initiation reported by the women in the survey. For births before the program, we limit the sample to only those taking place after June 2004. This helps in lessening the recall bias for previous births, but reduces the sample size of births before the program. The small number of observations of births before the program (629) affects the power of our analysis.
The household survey missed some outcomes of interest, including the gender of the child born in the prior 5 months and infant mortality data. Mortality information was gathered in the facility quality assessment. Also missed was satisfaction with delivery care, while it was covered for ANC and PNC.
We were unable to tape record many key informant interviews, due to lack of consent from respondents. While extensive notes were taken, full transcripts were not always available for qualitative analysis. Focus group discussions were tape recorded, but the women participating all had small babies with them, and sound quality was sometimes poor as a result.
In the multivariate impact evaluation, we compare outcomes among women “living in the DSF upazilas” with women “living in the control upazilas” rather than among women “receiving a voucher” with women “not receiving a voucher.” This helps reduce but may not have eliminated potential self- selection bias in the process of obtaining vouchers, since as we note below, many ineligible women (on parity or poverty criteria) still reported receiving vouchers.
Although the sample of health facilities assessed under the Facility Quality Assessment included 8 of the 21 voucher Upazila Health Complexes and their matched controls, due to the limited number of facilities we did not always have sufficient statistical power to detect program-related differences.
Economic Evaluation of DSF Voucher Program in Bangladesh 27
4. FINDINGS PART 1: VOUCHER OPERATIONS
The first objective of our evaluation was to comprehensively describe and assess the operations of the voucher programs. This section presents qualitative findings from key informants at the upazila and national levels, providers, and focus group discussants, as well as quantitative data from administrative sources and the household survey.
4.1 VOUCHER PROGRAM DESIGN
4.1.1 TARGETING AND IDENTIFICATION OF THE POOR
In “universal” intervention upazilas, all pregnant women on their first or second pregnancy who are permanent residents of the upazila are eligible for the DSF program. Women on their second pregnancy must have used contraception in between pregnancies. In means-tested upazilas, eligibility is further limited to extremely poor women only: those whose family income is not more than Tk. 2,500 per month, who own less than 0.15 acres of land, and who do not receive income from a cow, poultry, fisheries, orchards, rickshaw, or van.
Key informants in the 8 sampled intervention upazilas (which included 4 means-tested and 4 universal upazilas) were asked whether they felt these eligibility criteria were reasonable. Almost all respondents from the universal areas felt that the criteria were appropriate. Among respondents from means-tested areas, about one-third supported the criteria while two-thirds found them to be unreasonable. The main criticisms were that the land ownership and income criteria were too strict. Representative quotes included:
“The land ownership criterion of 0.15 [acres] should be raised to 0.30, because people from the lowest economic quintile have more land.” – Civil Surgeon.
“Those who are considered to be ultra poor in the present context need more than Tk. 2,500 to survive, eating even only rice and pulses. … these ultra poor people are excluded from the program. That is why it is better to change it and raise the [income threshold] to Tk. 4000.” – other respondent.
Most central-level key informants echoed the perception that the poverty criteria were too strict in means-tested areas. Several suggested increasing the income threshold (one mentioned that Tk. 4,000 per month was more reasonable than Tk. 2,500) and one suggested doubling the maximum land ownership threshold from 0.15 to 0.30 acres. (See section 4.2.2 below for more analysis of eligibility criteria and targeting based on the household survey.)
Economic Evaluation of DSF Voucher Program in Bangladesh 29 4.1.2 INCENTIVES FOR CONSUMERS AND PROVIDERS
Key informants in the 8 intervention upazilas were asked whether they would suggest revising the benefits package covered by the DSF program, and if so how. About one in five respondents had no suggestions. Among others, the most common suggestion was that incentives paid to providers be increased – especially for RMOs and UHFPOs (who have substantial administrative responsibilities under the program) and for FWAs. About one in ten key informants suggested that incentives for administrators should be increased. One national-level key informant also felt that provider incentives ought to be increased. (Note that cash incentives for UHFPOs and RMOs are currently awaiting final government order.)
The next most common suggestion among local key informants was to eliminate the gift box provided to mothers who deliver in health facilities, and reallocate these resources towards increased cash incentives or additional covered services (such as neonatal care). This was suggested by all types of key informants from all 8 upazilas. A national-level key informant also disliked the gift boxes, noting that obtaining and distributing them in a timely manner is difficult. According to 2 National DSF Program Committee members, changes in the gift box incentive are already planned – the jar of Horlicks is to be replaced by a cotton sari for mothers.
Thirdly, key informants and providers at the upazila level suggested increasing cash benefits paid to mothers who deliver in health facilities, as well as increasing transportation allowances. Again, these responses were distributed across most of the upazilas sampled. Several providers noted that the incentives benefit women who live close to health facilities more, because their transport costs are lower. Two providers specifically suggested that the cash incentive should be increased to assist women to buy medicines that are not available in the health facility.
According to three central-level key informants, the DSF program is planning to change the way in which cash incentives are paid to women. While currently women receive a lump-sum payment of Tk. 2,000 after giving birth with a qualified provider – officially intended for the purchase of nutritious foods for the mother – stakeholders have noted that women need nutritious foods throughout their pregnancy. Instead, a Tk. 500 payment will now be made after each ANC visit and after delivery. This may have the added benefit of increasing uptake of prenatal care as well.
4.1.3 INVOLVEMENT OF NGO AND PRIVATE SECTOR FACILITIES
According to the DSF protocol (2008), each District Designation Body is responsible for selecting eligible public, private, and NGO providers to participate in the DSF scheme. These committees are tasked with identifying facilities capable of offering essential obstetric care (EOC) referral services and use a standardized normative checklist approach to ensure that participating facilities achieve minimum service quality levels. The District Designation Bodies are supposed to include at least one member from the NGO or private sector. However, in 3 of the 8 upazilas in which we conducted in-depth interviews, there were no private sector or NGO members on the District Designation Body. To date, few NGO or private facilities are participating in the pilot DSF program overall. In many upazilas there are reportedly no private clinics that meet the basic service standards for participating. “In the whole country only 10-12 [DSF] upazilas have NGO involvement; the rest of them do not,” noted one central- level key informant.
In our evaluation, key informants interviewed in 6 of the 8 sampled upazilas selected for qualitative data collection reported that there was at least one NGO or private facility participating in the voucher
30 Economic Evaluation of DSF Voucher Program in Bangladesh program in their upazila. When asked about the process for selecting NGO and private facilities, respondents noted that the main criteria for accreditation were the number and quality of qualified providers and support staff, the availability of drugs, and whether the facility was well-equipped:
“The selection is done by the District [Designation] committee, although approval is given to the NGOs through filling out the specific checklist from the Health Directorate in Dhaka. The NGOs are also selected on the basis of their strength. It is verified whether the NGO has adequate numbers of skilled staff, necessary equipment and infrastructure for providing ANC, delivery and PNC services.” -- DSF Coordinator.
National-level stakeholders felt that more private and NGO facilities should be involved, mainly to stimulate competition with the public sector in the hopes of improving the overall quality of services. Recent increases in reimbursements paid to private facilities have increased their willingness to participate. However, the mechanism for disbursing payments to nonpublic facilities is reportedly still a problem:
“It takes two months to release the fund money. It is a very lengthy procedure. … Arrangements should be made to disburse the money in a more timely manner.”
“If the private providers do not get their incentives at the right time they show negligence in their duty.”
Two individuals also expressed concern that the quality of services in private and NGO facilities is actually quite low, with many lacking any doctors or operating rooms:
“Without the help of private sector Government alone can not do this. If we engage private sector there will be competition so quality of the services will be improved.”
“It would be better to involve private NGOs but their quality should be improved. In many health centers, there is no doctor, no good operating room.”
4.2 IMPLEMENTATION AND LOGISTICS
4.2.1 INFORMING CONSUMERS ABOUT THE DSF PROGRAM
Table 4.1 provides information on the awareness of the voucher program reported by women in the household survey. Of the 1,104 women surveyed in the intervention areas, 86% in the universal and 89% in the means-tested areas reported that they have heard of the program. In general, women in the means-tested areas seemed to have more information about the program. For example, while in the means-tested areas, 71% of the women who were aware of the voucher program stated that they were told about the importance of having three ANC visits, the corresponding figure in the universal areas was only 40%. For “importance of delivery by a SBA,” the corresponding statistics are 26% and 7% respectively. These statistics might suggest that the means-tested areas did a better job in publicizing the voucher program. It is interesting that the percentage of respondents who reported hearing that it was important to deliver with a SBA (which is its key intended outcome) is much smaller than those who reported being told about the availability of a gift box.
Economic Evaluation of DSF Voucher Program in Bangladesh 31 TABLE 4.1. AWARENESS OF VOUCHER PROGRAM, BY INTERVENTION TYPE (%)
Awareness about voucher program Universal Means- p-value tested N =552 N= 552 Ever heard of voucher program 85.5 89.8 0.496 Specific information heard about the voucher program N = 498 N = 507 among those who had heard of it Availability of maternal health voucher 15.2 16.8 0.739 Importance of 3 ANC visits 40.4 71.0 0.001 Importance of delivery by SBA 6.6 25.9 <0.001 Importance of facility based delivery 26.0 39.3 0.146 Importance of 1 PNC 8.0 16.1 0.100 Reimbursement of each ANC visit 3.9 7.8 0.175 Reimbursement of delivery expenses by SBA 6.6 8.4 0.553 Reimbursement of delivery expenses at the facility 22.1 28.0 0.373 Availability of gift box for baby 59.6 67.9 0.205 Others 89.2 89.6 0.042 Note: The content of ‘Other’ information received cannot be disaggregated. Figure 4.1 and Annex Table 1 indicates who told the beneficiaries about the voucher program. It shows some notable differences between the universal and means-tested upazilas. For example, while FWAs and health assistants constitute the biggest source of information in the universal areas, in the means- tested areas, the CSBAs assume the most important role. In addition to health professionals, about 20%-24% of women also heard about the voucher from their relatives.
FIGURE 4.1. SOURCE OF INFORMATION ABOUT VOUCHER PROGRAM, BY INTERVENTION TYPE (%) 50
45
40
35
30
% 25
20
15
10
5
0 FWA HA Relatives CSBA Others
Universal Means-tested
We asked women whether they were told what the voucher pays for when they received the voucher. Table 4.2 indicates that significantly more women in the means-tested areas (91%) were told about what
32 Economic Evaluation of DSF Voucher Program in Bangladesh the voucher pays for than were women in the universal site (77%). The majority of women in both sites were told that transport costs for all three ANC visits, and delivery costs at a health facility, would be covered by the vouchers. Only 18% of women in universal upazilas, compared to 43% of women in means-tested areas, were told that the voucher could pay for home delivery with an SBA (p=0.002). Very few women were told the cost of an ambulance for emergency referral care would be covered. With the exception of information about home deliveries with an SBA, the study sites did not differ significantly in the types of information received by beneficiaries.
TABLE 4.2. EXTENT TO WHICH WOMEN WERE TOLD ABOUT WHAT THE VOUCHER PAYS FOR, BY INTERVENTION TYPE (%)
Information given to voucher recipients Universal Means- p-value tested N = 420 N = 427 Beneficiary was told about what the voucher pays for when given 76.9 90.7 0.013 the voucher Type of information the beneficiary was told Transport cost for 3 ANC will be paid 51.3 63.2 0.150 Delivery cost by SBA will be paid 17.7 42.6 0.002 Delivery cost in facility will be paid 61.2 70.8 0.204 Transport cost for PNC-1 will be paid 22.4 20.7 0.852 Cost for ambulance will be paid 1.3 3.5 0.107 Others 0.4 0.6 0.566
Key informants reported that consumers are informed about the voucher program through a variety of methods. Home visits or field-level contacts with FWAs and HAs were the most commonly identified means of creating awareness among potential beneficiaries. Women are also informed of the program during visits to health facilities, either for pregnancy care or other services. A smaller number of key informants mentioned that women receive information through local meetings or from community leaders.
Most key informants felt that this system for informing the community about the voucher program was effective. The involvement of local government officials and religious leaders through community meetings was highlighted as particularly useful, as were door-to-door visits by field health workers:
“Paying door-to-door visits by field workers to inform the community about voucher program is especially effective. This has resulted in an increasing trend of hospital-based delivery.” – FWA/HA.
“The courtyard meetings in the community with the mothers which are facilitated by the local government representatives … In the meetings, pregnant mothers and their guardians are informed and motivated about safe delivery. The difference between the delivery at home and the institutional delivery is explained in the meeting.” – DSF Coordinator.
A smaller group of key informants from across upazilas and respondent types felt that better communication efforts were needed. Some suggested increased use of media (posters, leaflets, videos, theater sketches); a few felt that field workers lacked motivation and training; and a few mentioned the need for greater involvement by community leaders.
Economic Evaluation of DSF Voucher Program in Bangladesh 33 “I suggest publicizing the benefits of the DSF program in the girl’s schools, village markets through People’s Theater, organizing refresher trainings for DSF committee members, and DSF- related training for youths and religious leaders.” – DSF Coordinator.
4.2.2 IDENTIFYING ELIGIBLE BENEFICIARIES
Voucher receipt was widespread among the women in our household survey sample, all of whom had given birth in the prior 6 months. According to the survey, almost an equal proportion of women received vouchers in both universal (70%) and means-tested areas (72%). Among women of parity 1 and 2 only, 80% in universal areas and 85% in means-tested areas received vouchers. Although only women with a parity of 1-2 were eligible to receive vouchers, between 30% and 40% of women of parity 3 or more received vouchers as well. Among women in our sample who were parity 2, those who did not practice family planning between their first and second births were just as likely to receive vouchers as those who reported practicing family planning. Figure 4.2 displays these results graphically, while Annex Table 2 presents the full table.
FIGURE 4.2. PERCENTAGE OF WOMEN WHO RECEIVED VOUCHERS, AMONG DIFFERENT ELIGIBILITY SUBGROUPS 100 90 80 70 60 % 50 40 30 20 10 0 Full sample Parity 1, 2 Parity 3+ Used Did not use contraceptive contraceptive Universal Means-tested
We also examined adherence to eligibility criteria by analyzing the characteristics of the women who received vouchers. Here, we find that the large majority of voucher recipients were women on their first or second pregnancy (87% in universal areas and 91% in means-tested areas). About 63% of parity- 2 voucher recipients in universal areas reported using contraception between pregnancies, compared to 76% of these women in means-tested areas.
34 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 4.3. ELIGIBILITY CHARACTERISTICS OF VOUCHER RECIPIENTS
100% 90% 80%
70% 60% 50% 40% 30%
20% 10% 0% Parity 1 or 2 Parity 1 or 2 FP use if parity 2 FP use if parity 2 (Universal) (Means-tested) (Universal) (Means-tested)
Eligible Ineligible
As shown in Figure 4.4 below and Annex Table 3, poorer women in the universal areas were somewhat more likely to receive a voucher than the richest women. However, the differences between the quintiles are not significant. In the means-tested sites, as intended, significantly more women in the bottom three quintiles (78%) received vouchers than did women in the richest quintile (49%). Still, the fact that nearly half of the women in the richest quintile obtained a voucher in the means-tested area raises a question about the effectiveness of poverty-related targeting.
FIGURE 4.4. PERCENTAGE OF WOMEN WHO RECEIVED VOUCHERS BY WEALTH QUINTILE AND INTERVENTION TYPE (%)
100
90
80
70
60 % 50 40
30
20 10
0 1 (poorest) 2 3 4 5 (richest) Quintile
Universal Means-tested
Economic Evaluation of DSF Voucher Program in Bangladesh 35 Again looking at voucher recipients only, a greater proportion of women in means-tested areas came from the poorest quintiles, while the quintile distribution of women in universal areas was relatively even (Figure 4.5).
FIGURE 4.5. WEALTH QUINTILE DISTRIBUTION OF VOUCHER RECIPIENTS IN UNIVERSAL AND MEANS-TESTED UPAZILAS
100%
90%
80%
70% Richest 60% Fourth 50% Middle Second 40% Poorest 30%
20%
10%
0% Universal Means-tested Table 4.3, column series 1 (below) shows the percent of the sample who met each of the poverty criteria for voucher eligibility. These criteria were relevant in means-tested areas only, but are shown for both universal and means-tested areas for comparison purposes. The table also shows the percentage of each eligibility subgroup who actually received a voucher (column series 2). Women in means-tested areas were eligible to receive a voucher if their household owned less than 0.15 acres of land, owned no productive assets (such as livestock, van or rickshaw, or orchards), and had a monthly household income less than Tk. 2,500. The relevant assets available in our survey included cows, goats, sheep, chickens, and ducks; and van or rickshaw. We use monthly expenditures to proxy for monthly income. According to the survey, the bulk of women in our sample owned less than 0.15 acres of land (63% in universal areas and 79% in means-tested areas). However, less than one-quarter of the sample owned no productive assets (23% in both areas), and only a small fraction reported household expenditures less than Tk. 2,500 (5% and 7% respectively).
As shown, nearly 75% of women from households owning less than 0.15 acres of land or no land reported receiving the voucher, while more than 63% of women from households owning more than 0.15 acres of land received a voucher – in both universal and means-tested areas. This targeting difference was statistically significant in the means-tested areas only. In universal upazilas, women who owned any productive assets were significantly more likely to receive a voucher than women who did not own assets. In means-tested areas, asset-eligible women received more vouchers than non-eligible women as expected, but the difference was nonsignificant. Among the very small number of women who met the income eligibility criterion, almost all received vouchers (97% in universal and 93% in means-tested areas). However, the majority of women with higher monthly household expenditures also received vouchers (69% and 71% respectively).
36 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 4.3. PERCENTAGE OF WOMEN WHO RECEIVED A VOUCHER BY LAND OWNERSHIP, ASSET OWNERSHIP, AND MONTHLY EXPENDITURES (%)
Poverty criteria for voucher Percent of sample that Percent of each eligibility in means-tested met each criterion subgroup who areas received a voucher (1) (2) Universal Means-tested Universal Means-tested N=552 N=552 N=552 N=552 No land or <0.15 acres 62.8 79.4 74.5 74.6 >0.15 acres 37.2 20.6 63.5 63.6 100.0 100.0 p=0.198 p=0.057
Owned no productive assets 23.3 22.8 65.7 78.3 Owned any productive assets 76.7 77.2 71.9 70.6 100.0 100.0 p=0.050 p=0.214
Monthly expenditures FIGURE 4.6. PERCENTAGES OF VOUCHER RECIPIENTS WHO MET VARIOUS POVERTY CRITERIA 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Land <0.15 Land <0.15 No No Income Income acres acres productive productive <2500 Tk. <2500 Tk. (Universal) (Means- assets assets (Universal) (Means- tested) (Universal) (Means- tested) tested) Met poverty criteria Did not meet poverty criteria Note: Poverty eligibility criteria were applicable only in means-tested areas, but results are presented here for both universal and means-tested areas for comparison purposes. Economic Evaluation of DSF Voucher Program in Bangladesh 37 Our qualitative data collection efforts also investigated issues with identifying and distributing vouchers to eligible beneficiaries. About four-fifths of the key informants we interviewed felt that voucher eligibility criteria were by and large adhered to. A smaller proportion of key informants, clustered primarily in 2 means-tested upazilas, reported that eligibility criteria were not being followed. The most common complaint was “nepotism” in the distribution of vouchers; this was clustered in one upazila. An additional reason, reported in two upazilas, was the absence of a local committee to conduct home visits for determining eligibility. “For selecting and certifying the poor, the local government makes politically biased decisions and is nepotistic. This deprives some of the real poor of the benefits. Local government has to more neutral and honest.” – NGO/private sector member. About three-fifths of providers we interviewed believed that eligible women were receiving vouchers. But some providers felt that eligible women were being missed, mainly because the selection process was not implemented properly and because relatives of influential people were more likely to be selected. A few providers noted that some women are not properly informed of eligibility criteria. One respondent noted that many women who came to their mother’s house for delivery were not resident long enough to obtain a voucher. “…those who are eligible are getting the voucher because our health workers are honest enough. For any unfair means the voucher is terminated.” – Medical Officer. “The selection of DSF women is not properly done. Cards are distributed among the relatives of influential persons. Monitoring is not properly done. The FWAs and CSBAs do not inform properly the poor women of the area about their eligibility to receive the voucher.” – FWV. In three of the four universal voucher upazilas, focus group discussants felt that both poor and nonpoor women were receiving vouchers. In one universal intervention upazila, respondents felt that richer women were more likely to receive vouchers. Comments such as “Those who have acquaintances and relatives in the hospital are getting cards” and “Those who have power are getting cards” were expressed. Respondents in most of the means-tested upazilas generally felt that the poor were receiving vouchers, although some noted that wealthier women were receiving vouchers in some cases. “All mothers irrespective of being their poor and rich have got cards.” [respondent in universal upazila] “…those who do not have land, only they got the card.” [respondent in means-tested upazila] Many women across the different focus groups mentioned that parity was an important eligibility criterion, and this engendered more negative reactions, particularly in means-tested areas: “Those who have more than two children did not receive cards. They are quite unhappy with this. Under this program, only [women] having two or one child are given cards.” [respondent in means-tested upazila] “Those who are eligible for vouchers are receiving vouchers. No one is left out because health workers take 20 [Taka] for issuing each voucher card. Those who are not getting vouchers know that they are not getting vouchers for having too many children.” [respondent in means- tested upazila] 38 Economic Evaluation of DSF Voucher Program in Bangladesh Respondents in one means-tested upazila felt that poor mothers with three children should be eligible for vouchers as well: “It will be better if mothers of three children are given voucher. … The extremely poor mothers, mothers of three children who are too poor and suffer from malnutrition should be covered under this service.” [respondent in means-tested upazila] One woman in a universal upazila noted that despite having more than two children, she received a voucher because her fetus was malpositioned and she did not have any money. 4.2.3 VOUCHER DISTRIBUTION Administrative data collected from the National DSF Coordinator indicate that between April 2007 and August 2009, a total of 244,733 voucher booklets were distributed in 21 DSF upazilas. This represents 80% of the target number of vouchers (304,147) that the DSF program aimed to distribute during that time. Means-tested upazilas distributed a higher percentage of their target vouchers than universal upazilas (Table 4.4). TABLE 4.4. VOUCHERS DISTRIBUTED AS PERCENTAGE OF TARGETS, APRIL 2007-AUGUST 2009 Intervention type Target number of Number of vouchers Percentage of target vouchers distributed Universal (N=12 upazilas) 180,217 127,562 71% Means-tested (N=9 upazilas) 123,930 116,171 94% TOTAL 304,147 243,733 80% The number of vouchers distributed on a monthly basis has varied widely since inception of the program. Figure 4.7 presents voucher distribution per month in universal and means-tested areas, along with targeted distribution levels. Economic Evaluation of DSF Voucher Program in Bangladesh 39 FIGURE 4.7. VOUCHERS DISTRIBUTED PER MONTH VS. TARGETS, BY TYPE OF INTERVENTION, APRIL 2007 - AUGUST 2009 12,000 2007 2008 2009 10,000 Distributed (U) 8,000 Target (U) 6,000 Target (M) 4,000 Distributed (M) 2,000 - A M J J A S O N D J F M A M J J A S O N D J F M A M J J Month Several factors may have contributed to these fluctuations. First, disbursements of funds for the DSF program have been delayed each year – for instance, in fiscal year July 2007-June 2008 upazilas received funds only in January 2008. Moreover, unspent funds must be returned to the Treasury by the end of June each year. This may help to explain the drop-off of voucher distribution in the latter half of each year. Third, incentives for public service providers were introduced in December 2007; this may also explain the surge in voucher activity in early 2008. Since the beginning of 2009, voucher distribution in means-tested areas appears to be staying relatively close to target levels, according to Figure 4.7. However, voucher distribution in universal areas appears to be quite consistently slightly below target levels. Figure 4.8 below breaks down the achievement to date of voucher distribution targets, by upazila. Universal upazilas have blackened bars. 40 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 4.8. VOUCHERS DISTRIBUTED AS PERCENTAGE OF TARGETS, BY UPAZILA Paikgachha Sarishabari Kalapara Sakhipur Chatkhil Bhanga Khanshama Tarail Daudkandi Chaugachhia Banaripara Khetlal Harirampur Shibganj Baniachong Shahjadpur Gobindganj Dalutpur Raipura Mirsharai Ramu 0% 20% 40% 60% 80% 100% 120% 140% 160% 180% Figure 4.9 below presents the woman’s reported pregnancy gestation at the time she received the voucher. More women in the means-tested sites received the voucher at an earlier stage in their pregnancy (between 3-5 months) while women in the universal sites were at a later stage in their pregnancy on average. According to protocol, women are supposed to be given a voucher booklet during the first trimester of pregnancy, but this occurred for only 5.4% in the universal area. The corresponding figure in the means-tested area is 11.3%. Economic Evaluation of DSF Voucher Program in Bangladesh 41 FIGURE 4.9. PREGNANCY GESTATION AT THE TIME OF VOUCHER RECEIPT 35 30 25 20 % 15 10 5 0 2 3 4 5 6 7 8 9 Gestational Age (months) Universal Means-tested According to protocol, eligible women are first identified by an FWA, HA or CSBA. After the list of eligible women is approved by the Union DSF committee, women may receive a voucher booklet. In Figure 4.10 (Annex Table 4), we see that most women in both the universal and means-tested sites (41% and 46%) received their vouchers less than 10 days after being identified. Very few women (<10%) had to wait longer than 2 months to receive the voucher booklet. FIGURE 4.10. NUMBER OF DAYS AFTER LISTING WHEN BENEFICIARIES RECEIVED VOUCHER (%) 50 45 40 35 30 % 25 20 15 10 5 0 <10 10-20 20-30 30-60 60-90 90+ Days received voucher after listing Universal Means-tested 42 Economic Evaluation of DSF Voucher Program in Bangladesh In the household survey, we asked women who received vouchers about who had distributed the voucher to them. Reported voucher distributors differed significantly between the universal and means- tested sites as shown in Figure 4.11 and Annex Table 5. In both sites, health assistants were equally likely to deliver vouchers (33% in universal and 34% in means-tested). However, 48% of women in the means-tested site received vouchers from a community skilled birth attendant, while only 29% of women in the universal site did. In both sites, women were least likely to receive the vouchers from family welfare assistants and others, however, 22% of women in the universal site received vouchers from FWAs compared with only 10% of women in the means-tested site. FIGURE 4.11. MAIN REPORTED VOUCHER DISTRIBUTORS (%) Universal Means-tested 8% 10% 16% 22% 34% 29% 48% 33% FWA Health Assistant (HA) CSBA Others Focus group discussants in three upazilas reported that they received their voucher booklet directly from a health field worker (FWA, CSBA, HA or nutrition worker). The health worker visited them at home: “Fazila apa [local FWA] enlisted us and distributed cards to us.” [respondent in means-tested upazila] In the other upazilas, respondents were more likely to report that the field worker identified them and “listed our names” but they had to obtain the voucher booklet from the local health facility (EPI center, UHFWC, UHC or hospital): “The nutrition workers and family planning workers of our area come to us with this proposal and listed our names. Later on, through them, we collected the voucher card from the EPI centre.” [respondent in universal upazila] Economic Evaluation of DSF Voucher Program in Bangladesh 43 “We heard about voucher card from our neighbors and got it from the hospital.” [respondent in means-tested upazila] Several focus group respondents noted that they became aware of the voucher program through their mothers-in-law or neighbors, though most reported being contacted by a field worker or learning about the program during a visit to a health facility for a check-up or illness. The main challenge in distributing vouchers mentioned by providers was handling pressure from ineligible recipients. This type of pressure was experienced especially by FWAs, CSBAs, and FWVs in all the sampled upazilas; one provider even noted that she had been threatened by an ineligible recipient. In two upazilas, a small number of providers mentioned that they had been pressured by local government officials to distribute vouchers to ineligible women. In contrast, one nurse reported that local government representatives were blocking the distribution of vouchers to eligible poor women. “The poor people who have more than two children Beneficiary receipt of transport stipends and cash incentives create pressure to get the voucher.” –FWA “Sometimes, even mothers of three children insist on [getting] the voucher. People who are not permanent residents also try to get the voucher card. Sometimes the influential people in society pressure us to get the voucher card.” – FWA “…the interference of the local [parishad] chairman does not allow us to give the card to all deserving beneficiaries.” – Senior staff nurse 4.2.4 PAYMENT OF CASH AND IN-KIND BENEFITS TO CONSUMERS Since the end of 2007, voucher recipients have been eligible to receive up to Tk. 500 in transportation subsidies – up to Tk. 100 for each of 3 ANC visits, Tk. 100 for institutional delivery, and Tk. 100 for postpartum care. This stipend is to be paid at the time of each visit to the health facility, although in practice many stipends were paid out in one lump sum after delivery. Voucher recipients who delivered in a facility or at home with a qualified provider were also eligible for a Tk. 2,000 cash stipend, to be used for the purchase of nutritious food. As shown in Table 4.5, over 60% of women in our household survey who received voucher booklets (regardless of actual service use) in both universal and means-tested sites also received cash incentives for nutritious food. Among voucher recipients who delivered with a qualified provider and were thus eligible for the cash incentive, 69% in universal areas and 67% in means-tested areas reported receiving the cash incentive. Fewer women who received vouchers (ranging from 4-19%) reported receiving transport stipends for ANC visits, delivery care, and postpartum care, and the likelihood did not differ significantly between the study sites.4 Women in the means-tested areas reported receiving larger cash stipends than women in universal areas. This likely reflects the common practice of paying out all transport and nutrition stipends in one lump sum, rather than in separate transactions. In fact, the low reported rate of receipt of transport stipends may reflect that women are not even be aware that the transport payment was included. Women in the means-tested sites were slightly more likely to receive a gift box; however, there was no significant difference in gift receipt between the groups. Some of the low rates are driven by the fact that the survey was conducted over July-August 2009 and covered 4 Note that the denominator for these statistics is all women who received a voucher booklet, and is not conditional upon using the particular service. 44 Economic Evaluation of DSF Voucher Program in Bangladesh deliveries between February 1, 2009 and July 31, 2009 – since the fiscal year starts on July 1 and funds were not available immediately, women who delivered in July and towards end of the last fiscal year did not receive their benefits until after November 2009 when funds were available (see Table 4.7 below). TABLE 4.5. RECEIPT OF CASH AND IN-KIND BENEFITS AMONG THOSE WHO RECEIVED A VOUCHER BOOKLET (%) Cash and in-kind benefits Universal Means- p-value tested N=420 N=427 Transport stipends First ANC visit 19.2 8.2 0.159 Second ANC visit 13.0 4.1 0.106 Third ANC visit 10.7 3.7 0.082 Delivery care 3.8 1.9 0.275 Postpartum care 3.8 0.4 0.004 Cash incentive for food 61.5 64.6 0.653 Gift box 49.3 57.8 0.367 Mean amount of cash incentive for food received (in Taka) 1886.3 2185.8 0.065 Note: The denominator N equals the number of women who received vouchers. Table 4.6 below breaks down the same statistics by place of delivery. As shown, women who delivered at home were less likely to receive transportation stipends not only for delivery, but also for antenatal and postpartum care. They were also statistically less likely to receive a gift box compared to women who delivered in a facility (43.3% versus 66.6%, p=0.001). TABLE 4.6. RECEIPT OF CASH AND IN-KIND BENEFITS AMONG THOSE WHO RECEIVED A VOUCHER BOOKLET, BY PLACE OF DELIVERY(%) Cash and in-kind benefits Home Facility p-value N=471 N=375 Transport stipends First ANC visit 10.1 19.2 0.100 Second ANC visit 5.2 13.6 0.012 Third ANC visit 2.9 13.3 <0.001 Delivery care 1.2 5.3 0.014 Postpartum care 0.1 4.0 0.043 Cash incentive for food 59.6 67.2 0.254 Gift box 43.3 66.6 0.001 Mean amount of cash incentive for food received (in Taka) 1924 2156 <0.001 Note: The denominator N equals the number of women who received vouchers. Some of these women may not have used each individual service. As shown in Table 4.7 below, there is a sharp difference in the probability of receiving cash benefits between women who delivered before and after June 2009. This is due to the end of the fiscal year and the requirement that unused funds be returned to the Treasury. New funds for the following fiscal year did not arrive at the upazila level until November 2009. Economic Evaluation of DSF Voucher Program in Bangladesh 45 TABLE 4.7. RECEIPT OF CASH BY DELIVERY DATE AND TYPE OF INTERVENTION (%) Cash receipt by date of last delivery Universal Means-tested N 420 N 427 Received cash if delivered before June 2009 73.3 83.1 Received cash if delivered during and after June 2009 3.4 0.7 P-value <0.001 <0.001 A small percentage of voucher recipients in our household survey reported problems with reimbursement for transport costs incurred while obtaining services. As shown in Table 4.8, very few women (<3%) of women in both sites had difficulties with reimbursement for ANC transportation costs or delivery transportation costs. TABLE 4.8. BENEFICIARY DIFFICULTIES IN OBTAINING TRANSPORT STIPENDS (%) Difficulty receiving reimbursement Universal Means-tested p-value amount promised for: N=420 N= 427 First ANC check up 2.5 0.1 <0.001 Second ANC check up 0.8 0.1 0.021 Third ANC check up 1.6 0.0 0.121 Delivery care 0.3 0.8 0.313 Note: The denominator N equals the number of women who received vouchers. Table 4.9 indicates that more women had problems with receiving the cash incentive for nutritional food. In the universal areas, 21% of women faced a problem with receiving the cash incentive while 9% of women in the means-tested areas did. When asked about specific problems faced, the majority of women in both sites mentioned that there was a delay in the reimbursement (20% in universal and 6% in means-tested, p=0.024). TABLE 4.9. BENEFICIARY DIFFICULTIES IN RECEIVING CASH INCENTIVE FOR NUTRITIOUS FOOD (%) Beneficiary difficulties with receiving incentives Universal Means-tested p-value N= 263 N =283 Difficulty receiving cash incentive for nutritional food 20.9 8.5 0.073 Type of problem Delay/did not receive immediate reimbursement 19.7 5.9 0.024 Received partial reimbursement 7.0 1.4 0.076 Did not receive any reimbursement 0.5 0.0 0.278 Had to pay extra 2.2 2.7 0.793 Other 0.0 0.3 0.300 Note: The denominator N equals the number of women who received the cash incentive. Focus group discussants reported mixed experiences regarding receipt of transport stipends. In three upazilas, respondents noted that the full Tk. 500 was paid at the same time as the Tk. 2000 cash payment after institutional delivery, rather than being paid at each visit. In one upazila respondents reported receiving the appropriate payments after each visit, and in two upazilas respondents had a variety of experiences – some received transport stipends while others did not. In the remaining four upazilas most respondents did not receive any stipends or were unaware that stipends were available. 46 Economic Evaluation of DSF Voucher Program in Bangladesh Some were informed that there was no money at the health facility for paying the stipends and they should return at a later date. Below is a sampling of responses: “I got Tk. 200 for transportation.” [respondent from means-tested upazila] “We never heard that transport allowance is given. We are hearing it from you now.” [respondent from universal upazila] “We asked about money; the doctor told that one can come to the hospital on foot, why do you need money?” [respondent from universal upazila] “When we ask for the money they tell us ‘we do not have money, come later’.” [respondent from means-tested upazila] Almost all focus group discussants reported receiving the cash incentive for delivering in a health facility. Only 7 respondents in 2 upazilas did not receive the payment. Beneficiary receipt of gift box Voucher recipients who deliver in a health facility or at home with an eligible provider (CSBA or FWV) are eligible to receive a gift box in addition to the cash stipend. The gift includes a towel and small bar of soap, two outfits for a newborn baby, and one large bottle of Horlicks (a powdered malted milk drink). As noted above, the DSF program is planning to replace the Horlicks with a sari for mothers in the near future. As reflected in Table 4.5 above, according to our household survey 49% of voucher recipients in universal areas, and 58% of recipients in means-tested areas, reported receiving a gift box. Gift boxes were very popular among the women who participated in the focus group discussions. Women were appreciative of the Horlicks, which they claimed gave them strength and helped them recover from the delivery. However, many noted the gift boxes were not provided immediately following delivery and women had to return to the health facility to obtain it at a later date. This delay meant that the baby outfit was usually too small for the infant by the time it was received. A few women suggested that a sari for the mother be provided instead. Many asked that the gift box be distributed in a more timely manner. In two upazilas, respondents had not received gift boxes and were upset about this; some did not know about the gifts. “I am very happy to receive the gift box. It will be better if the given dress sets could be a little bit larger in size.” [respondent from means-tested upazila] “It would be good if we [could] get gift boxes. We are poor people; we do not have the ability to purchase soap, Horlicks and towel.” [respondent from universal upazila] Provider difficulties in distribution of cash benefits Key informants from all 8 upazilas reported difficulties with the provision of cash and in-kind benefits to beneficiaries. The problems were particularly concentrated in two upazilas. The most common difficulties mentioned were long delays in receiving cash advances from the government, resulting in long delays in paying beneficiaries; and lack of sufficient administrative staff to distribute cash benefits, resulting in long lines at health facilities on distribution days. Economic Evaluation of DSF Voucher Program in Bangladesh 47 “Because of the lack of fund the beneficiaries cannot be reimbursed in a timely manner and a lot of questions have to be faced from them. The providers also lose interest in the work because of the delay getting the incentives.” – NGO/private sector member Central-level key informants noted that some upazilas only distribute cash incentives on certain days and infrequently, leading to crowding. They suggested that reimbursements be made “at the time of need” or more frequently. 4.2.5 SERVICE PROVISION UNDER THE VOUCHER PROGRAM Figure 4.12 below summarizes graphically the monthly trends in vouchers distributed and voucher- covered ANC services provided, according to data received from the National DSF Coordinator. Several conclusions can be drawn from this analysis. Trends in ANC utilization closely mirror trends in voucher distribution, as expected. The lag between voucher distribution and first ANC visit is quite small (approximately 1-2 months). However, not all women who receive voucher booklets seek ANC1. FIGURE 4.12. VOUCHERS DISTRIBUTED AND ANC SERVICES PROVIDED UNDER THE DSF PROGRAM, APRIL 2007-AUGUST 2009 16,000 2007 2008 2009 14,000 12,000 10,000 r e b m u n 8,000 l a t o T 6,000 4,000 2,000 0 A M J J A S O N D J F M A M J J A S O N D J F M A M J J A Month Vouchers distributed ANC1 ANC2 ANC3 Figure 4.13 below provides comparable information for safe deliveries, postnatal care, and C-sections provided under the voucher program. Here again, patterns in voucher distribution are somewhat mirrored by trends in safe deliveries, with a time lag of several months, as expected. Not all women who receive vouchers are receiving voucher-covered delivery care. Since the beginning of 2009, the average number of safe deliveries per month has remained around 6,000. Trends in PNC provision are 48 Economic Evaluation of DSF Voucher Program in Bangladesh nearly identical to trends in safe deliveries, with almost all women who have a voucher-covered delivery reportedly obtaining PNC. (Note that this trend is not consistent with findings from the household survey, presented in chapter 5.) The number of C-sections provided under the voucher program has been increasing, both in absolute terms and as a proportion of total deliveries. FIGURE 4.13. VOUCHERS DISTRIBUTED AND DELIVERY AND PNC SERVICES PROVIDED UNDER THE DSF PROGRAM, APRIL 2007-AUGUST 2009 16,000 2007 2008 2009 14,000 12,000 10,000 r e b m u n 8,000 l a t o T 6,000 4,000 2,000 0 A M J J A S O N D J F M A M J J A S O N D J F M A M J J A Month Vouchers distributed Safe deliveries PNC C-sections Key informant and provider perceptions regarding service provision capacity One widespread concern about interventions that stimulate demand for health services is that health facilities and health care providers will not be able to handle the influx of new patients. Simultaneously expanding supply-side capacity is an ongoing challenge for the Bangladesh DSF program, given the success it has had in increasing service utilization. Almost all of the upazila-level key informants interviewed expressed concerns about human resource shortages at health facilities, including shortages of clinical staff, shortages of administrative and support staff (cleaners, accountants, office administrators), and a lack of sufficient training for clinicians. They mentioned long lines among consumers seeking services and difficulties among staff in managing their workload: “The beneficiaries have to wait for a long time to receive ANC/PNC due to shortage of providers. … For providers, due to shortage of providers they have to spend more time on DSF voucher program. As a consequence they face problems in performing their day to day regular activities.” – FWA Economic Evaluation of DSF Voucher Program in Bangladesh 49 “We tell many pregnant women to come to the hospital in order to receive ANC-2 and ANC-3 services and sometimes it happens that many comes on the same date which results in a big crowd. Due to the shortage of sufficient manpower it takes a long time to provide the service. Staffs and doctors become very busy due to excessive patient load which results in delays in providing services.” – NGO/private sector member “If there is a crowd of ANC or PNC care seekers in the UHFWC and UHC, then the doctors, nurses, FWAs, and cleaners become really busy and it takes a long time to provide services. The number of skilled workers should be adequate.” – FWV A national-level key informant echoed these concerns, highlighting problems with the quality of services: “Our service quality is not that good. In private hospitals there is a shortage of qualified doctors and equipment. In Government hospitals there is a shortage of equipment. In some places there is not a single trolley available. Shortage of human resources is a big problem.” Service providers expressed similar concerns, with three out of four providers noting that human resource shortages limit smooth functioning of the DSF program. The shortages reportedly impact not only the direct provision of services to women, but affect the ability of the UHFPO and MO to manage DSF financial accounts and make payments to providers and beneficiaries in a timely manner. Some field workers noted that they were having difficulties registering beneficiaries and disseminating information about the voucher program. A few providers commented that there is a problem filling key vacant positions, specifically mentioning trained CSBAs and skilled accounting staff. Almost all providers interviewed stated that their workload had increased because of the voucher program. The nature of the increase was described as “double workload” (1 respondent), “increased number of deliveries” (4), “increased check-ups” (5), and “increased paperwork” (6). The most common suggestion to improve the situation was simply to hire more health workers and increase the efficiency of existing workers. Another suggestion was to increase DSF program participation by NGO providers: “Our work load is too much compared to before. This problem can be solved by providing manpower from the NGOs to assist our work.” – CSBA “More vaginal deliveries are taking place in our health complex since the voucher program started. As a result we have to follow up different problems of the newborns also. We cannot give enough time to the inpatient and outpatient departments as we are to concentrate on the voucher program. More efficient workforce should be appointed.” – Medical Officer Most providers noted that shortages of medicines, supplies and equipment hindered their ability to provide services to voucher recipients. When asked how they coped with these shortages, the most common response was that they encouraged patients to purchase prescribed medicines from an outside source. These purchases, of course, would not be subsidized by the DSF program. A gynecologist and a nurse in one upazila mentioned that they were using the “seed fund” to fund the purchase of new supplies. Others noted that they tried to make do with old equipment, and sometimes referred patients to other clinics that had the necessary inputs. “We do our work with the old equipment … and the problem of the medicines is also solved from the ‘seed fund’.” – Senior staff nurse 50 Economic Evaluation of DSF Voucher Program in Bangladesh “We ask the attendants of the pregnant mothers to buy medicines and other equipment from outside, or we have to procure them in advance on our own.” – CSBA “We cannot provide proper services due to shortages of medicines and instruments. In such situations we refer patients to the UHC.” – FWV Beneficiary perceptions regarding ability to access services Focus group discussants expressed widely varying perceptions of their ability to access services under the voucher program. In five of the ten FGDs, respondents emphatically stated that they did not face any problems in receiving services at health facilities or from health field workers. Service quality was reportedly very good. In fact, in one upazila respondents mentioned that voucher recipients received services from the hospital on a priority basis. In contrast, respondents in four upazilas noted substantial problems accessing services and in the remaining group responses were mixed. The main complaints were crowding, long wait times, and rude treatment by health care providers. “They did not check- up properly. We have to wait for a long time. If we want to know the reason for waiting then they behave roughly. … We do not receive service from hospital even after getting cards; they scold us and make us get out from the hospital if we do not take cards with us. … It is of no use even if we take the cards with us. The doctors and nurses talk in abusive language. Even they say what kind of trouble the Government has created?”[respondents in universal upazila] “The problem is that we have to wait to receive treatment because the number of patient is high.”[respondent in means-tested upazila] “When their acquaintances come they keep us waiting without providing check up services. They give priority to their known person to provide check up services.” [respondent in means- tested upazila] In two of these upazilas, respondents also reported that they did not know what services were provided under the voucher program: “We do not know what services are provided. Doctors do not tell us.” [respondent in means- tested upazila] “We do not know what services we can get after receiving cards.” [respondent in universal upazila] 4.2.6 MONITORING AND EVALUATION SYSTEMS There is limited capacity for monitoring DSF pilot program activities. To the extent that it is done, monitoring is primarily conducted by the WHO- and DFID-sponsored upazila-level DSF Coordinators, who have administrative responsibilities for the program. They collect voucher distribution and service utilization statistics, primarily for reporting to the National DSF Office.The information they collect is not integrated with the regular facility Management Information Systems (MIS). Upazila DSF Coordinators submit monthly reports to the National DSF Coordinator, who cross-checks reports during his field visits and monitors progress compared to targets. Currently there is no prescribed Economic Evaluation of DSF Voucher Program in Bangladesh 51 format for monitoring reports or any structured checklist for monitoring. Two central-level key informants noted that some monitoring is also done by the civil surgeons, UHFPOs, and upazila officers within the upazila DSF committees, but it was generally felt that the monitoring system was not effective. The central-level key informants suggested that the number of monitoring and evaluation staff be increased, and that the MOHFW (especially at the district and upazila levels) take on more monitoring responsibilities. One respondent suggested that CSBAs should be more involved in monitoring. Other suggestions included establishing a separate monitoring group, appointing a Deputy Program Manager (DPM) from the Directorate General of Health Services to be responsible for routine supervision and monitoring, establishing a systematic monitoring system, and strengthening MIS overall (including developing monitoring software, charts and checklists). When asked whether they thought the current system of monitoring the voucher program was efficient, about one-third of upazila-level key informants felt that it was not. These respondents were distributed across all 8 sampled upazilas. The most common suggestion for improving the monitoring system was to designate (and pay a salary to) a person specifically responsible for monitoring in each union or upazila. Another common suggestion was to provide some type of incentives for monitoring, though it was unclear exactly to whom these incentives should be paid. A third idea was to increase oversight at the field level by having more supervisors in the field. “Each union should have a monitor. If one monitor along with one or two Union DSF Committee members jointly does the monitoring, the monitoring situation can be improved.” -- Union parishad chairman “Separate manpower should be appointed for the monitoring at the central level.” – RMO, Upazila DSF committee Central-level stakeholders also expressed concern that little effective supervision is occurring at the field level, with only the WHO- and DFID-sponsored DSF Coordinators making field visits for supervision. Specifically, it was felt that supervision of the voucher distribution process needed to be improved, since the criteria for distributing vouchers were not being followed in some areas. However, almost all providers interviewed reported that their DSF activities were routinely supervised, usually by staff at the UHC. Most reported that supervision occurred monthly. Interestingly, in one upazila, 5 of the 6 providers interviewed stated that they were not subject to any monitoring. Monitoring private/NGO facilities According to key informants in the 6 upazilas where at least one private or NGO facility was participating, most of the private and NGO facilities reportedly do submit some type of monthly report detailing the number of voucher-covered services provided (ANC visits, deliveries, and PNC visits). Respondents in these upazilas reported that the DSF Coordinator, UHFPO, or RMO within a given government health facility was officially responsible for monitoring NGO and private facilities’ compliance with quality standards. Site visits were most commonly reported as the method for assessing compliance. The DSF Coordinator in one upazila stated, “I visit a specific safe delivery unit of the NGO at least once a month. I tally the number of submitted vouchers as per the coverage of the day and I discuss with a DSF mother by picking one of the submitted vouchers. None has violated the conditions yet.” 52 Economic Evaluation of DSF Voucher Program in Bangladesh However, NGO and private sector representatives in 1 of these upazilas noted that no quality monitoring of those facilities was conducted. 4.3 FINANCIAL MANAGEMENT AND ACCOUNTING 4.3.1 FINANCIAL MANAGEMENT As reported in the 2008 rapid assessment, there are still significant problems with delays in disbursement of central funds for the DSF program – funds for the cash incentives to be paid to women and for reimbursements and incentives to providers. More than half of all key informants (from all 8 sampled upazilas and representing all types of respondents) and several central-level key informants complained about these delays, noting that they diminish the impact of the voucher program on maternal health behaviors and reduce the willingness of providers to participate in the program. Facilities were forced to delay paying incentives to women until the funds were received. “The main problem is the late arrival of funds. The purpose for which financial benefits are given to mothers after delivery is thwarted because of delays in disbursement of money.” – NGO/private sector member “Because of the lack of funds the beneficiaries cannot be reimbursed timely and lot of questions have to be faced from them. The providers also lose interest in work because of the delay getting the incentives” – NGO/private sector member “It takes 3 months to get the advance money. This mechanism should be simpler.” – central- level key informant “As there are delays in the accounting of the money from one fiscal year, so there are also delays in the disbursement of the money for the next fiscal year.” – central-level key informant Central-level informants reported ongoing problems withdrawing money from the advance fund, including requirements for multiple layers of approval within the government and World Bank bureaucracies that slow disbursements: “It is difficult to draw money from the advance fund. We have to convince the AG [Accountant General]. It is then sent to the Finance Ministry with the signature of the Health Secretary which takes time. This problem occurs as there is no full-time DPM [Deputy Program Manager within the DGHS].” “We cannot ensure that money will be on time because of the system. We send a prayer for the advance, then it goes to the AG. From there it goes to the [Finance] Ministry. The cycle is too long. We want a simpler way. There have been so many discussions about this matter, but we have not gotten any results.” “The pooled fund mechanism is a very complicated method for disbursing funds. After releasing the first installment of funds from the Ministry, we have to meet the accounting formalities of the Line Director’s office and the AG’s office before asking for the second installment of funds. Economic Evaluation of DSF Voucher Program in Bangladesh 53 Then we send it to the World Bank, if they are satisfied with it then they will provide the same amount of money.” The “seed fund” accounts were generally perceived to be useful and appropriately used. These accounts are to be used for reimbursing public providers, paying FWAs and HAs for each woman registered in the voucher program, paying facility staff for daily DSF work, procuring drugs and supplies, and covering emergency referral transport costs. “The money of the seed fund is spent in procurement of medicines for the pregnant mothers, stationery, fuel, gift vouchers worth Tk. 500, and registration fees for the fieldworkers. Through the approval of the upazila committee, the seed fund is also used for procuring other relevant necessary things.” – DSF Coordinator Complaints were made, both at the upazila level and central level, about the fact that unused seed fund money had to be returned to the government at the end of the year. “They even take away the seed fund money at the end of the year. If they would not take the seed fund money or could provide us an advance, then we would be able to pay the money [to women] early. ” – UHFPO “At the end of one financial year if any money left we can not keep it for the next financial year. We have to send it back.” – Central-level key informant About half of the providers interviewed thought that the current provider reimbursement process was efficient, while the remainder did not. The main complaint was about delays in distributing payments. Less than half the providers interviewed receive incentive payments on a monthly basis, another third are paid every two months, and one in six is paid only twice per year. Notably, there was variation in the reported frequency of payment within the same upazilas. “For the providers this reimbursement method is only somewhat effective because the money is not disbursed at the right time. So we have to face problem for the lack of money. Not getting money at the right time is a problem.” – FWV “Payment is never made regularly. It is usually delayed by one to three months. The rest of the money is paid in the month of June [at the end of the fiscal year]. Payment is first made to health department personnel and then other providers get the payment.” – Senior staff nurse Several key informants mentioned a need for more staff to manage both the beneficiary and provider reimbursement process at the facility level. Overseeing this process is currently the responsibility of UHFPOs and RMOs, who have clinical responsibilities as well as DSF financial management duties. One MO interviewed suggested that skilled financial managers should be employed to handle voucher-related financial processes, since clinical staff do not have time or the necessary skills. This was echoed by central-level key informants, who noted that the process of disbursing funds is very time consuming: “If there were a separate person [responsible] for fund disbursement, the providers could do their job properly.” The second main complaint about provider payments was that compensation was insufficient given the amount of work. One CSBA also noted that CSBAs should also receive a transportation allowance: 54 Economic Evaluation of DSF Voucher Program in Bangladesh “In reality, the incentives that we get are not effective at all. When a pregnant woman calls us at night for delivery, we have to spend Tk. 50-60 as van fare. We spend Tk. 100 as travel cost, but get only Tk. 150 which I think as a problem. “ – CSBA A small number of key informants and providers noted that there is corruption within the DSF program financial management system at the upazila level, and requested better monitoring. Some upazila-level issues were already being addressed before these results. Two providers requested increased transparency in the payment of incentives to providers, and three suggested that provider payments should be made directly through banks rather than having cash transferred through the health facility. Similarly, a senior central-level key informant expressed concerns about the misappropriation of funds: “There is a likelihood of misappropriation of funds as beneficiaries are paid cash by health personnel instead receiving reimbursement through a bank. Ways should be devised so as to minimize this chance of misappropriation. In some cases women’s husbands may take away the cash, depriving women of the benefit they are supposed to have.” 4.3.2 FINANCIAL SUSTAINABILITY OF SCHEME Central-level key informants were asked whether and how the DSF program could be made sustainable. Responses varied, but it was hoped that donors would continue to support the program into the medium term, until the government of Bangladesh could provide greater financial support. One respondent felt that women’s maternal health seeking behavior is beginning to change, and hoped that over time they will become increasingly willing to seek services even without receiving payment. Another expressed concern about the enormous price tag on the voucher program, and suggested focusing the incentives on hard-to-reach and remote areas. “Those who are providing funding now, how long they will fund or whether the Government could do it remains a question. It is a matter of huge money. The expenditure should be minimized. If some arrangement of giving incentives to HTR areas or remote areas could be made then the program could possibly be made sustainable.” Most of the central-level key informants hoped that the DSF program could be expanded to other poverty-prone areas of the country. One hoped that donors would support this effort, while two thought the government should be able to support it, especially if smaller incentives were paid or if incentives were only provided to mothers. Several emphasized the importance of expanding the program gradually, with careful attention to the rational selection of the neediest upazilas. One official disagreed and seemed to question the DSF program concept more generally: “I personally feel that there is no need for such programs as paying money to people for taking a service. This type of program cannot be sustained in the end. The best solutions are (i) strengthening home delivery and (ii) equipping the upazila health complexes for delivering comprehensive EOC services.” Economic Evaluation of DSF Voucher Program in Bangladesh 55 4.4 INSTITUTIONAL ARRANGEMENTS AND GOVERNANCE 4.4.1 DSF PROGRAM ORGANIZATIONAL STRUCTURES AND SYSTEMS As described in the background section, the DSF program is managed by committees at several organizational levels. Centrally, the National DSF Committee has responsibility for overall voucher program strategy and policies, and the DSF Program Implementation Committee is tasked with addressing operational issues. A Technical Subcommittee advises the Program Implementation Committee and proposes operational guidelines. District-level Designation Bodies are responsible for selecting and accrediting participating providers in each intervention upazila. Below this, direct management of DSF program implementation is the responsibility of the Upazila DSF Committee (composed primarily of Upazila Health Complex medical staff and upazila government representatives) and the corresponding Union DSF Committees (headed by the Union Parishad Chairman and composed primarily of union government representatives and health workers such as FWVs and HAs). Committees at the upazila level are responsible for reimbursements to providers and submitting performance reports. Publicizing the scheme and identifying eligible women is the responsibility of union-level committee members. DSF Coordinators, who are funded by the WHO and DFID, are posted in UHCs that participate in the program. They are the only staff uniquely tasked with monitoring and supporting DSF activities; all other staff (UHFPOs, RMOs, field health workers) handle their DSF responsibilities on top of their routine clinical and management responsibilities. According to key informants at the upazila level, the frequency of different DSF committee meetings varies greatly. There is no “typical” frequency for District or Upazila DSF Committee meetings; responses included monthly, bimonthly, quarterly, annually, and rarely or never. Union DSF Committee meetings are reportedly held either bimonthly or rarely/never. Topics discussed reportedly include awareness-raising and publicity, voucher distribution, eligibility issues, drop-outs, services provided, seed fund issues, maternal health in general, NGO/private health center performance, workforce challenges, and annual financials. “The last meeting was held on June 30. … They discussed the home visits of FWVs, FWAs and HAs to see the patients; making mothers interested to come to hospitals; whether there is any problem in distributing vouchers; and whether the staff face any kinds of reactions.” – UHFPO At the national level, key informants reportedly uniformly that the National DSF Committee was not particularly functional and did not meet regularly, though it was supposed to meet every 6 months. It is supposed to discuss major policy matters, such as whether to expand the DSF program benefits to include neonatal health or to extend the program to other districts. The DSF Program Implementation Committee and Technical Subcommittee are reportedly more functional and active. About two-thirds of the key informants interviewed in this evaluation felt that this current organizational structure of the DSF program was effective, while one-third (distributed across all 8 upazilas) felt that it was somewhat or not effective. Some reasons for the perceived lack of effectiveness included shortages of human resources to manage the program and a need for increased monitoring. Upazila-level respondents did not provide much in-depth information on organizational challenges. Key informants were also asked how the committees at different levels were coordinated. This question was also difficult for respondents to answer. In general, responses indicated that the Upazila DSF committee serves as the coordination center, liaising with both the District-and Union-level bodies. 56 Economic Evaluation of DSF Voucher Program in Bangladesh Several respondents indicated that coordination occurs through regular meetings, phone calls, and letters. Central-level key informants felt that the organizational structure of the DSF program was only somewhat effective. One mentioned that the National DSF Program Implementation committee and the Technical Subcommittee do not meet regularly but only on an as-needed basis. Another noted that the District Designation Bodies also do not meet regularly and are not functional. A third respondent felt that local government representatives do not willingly attend DSF committee meetings. More generally, respondents noted that the lack of separate, designated staff to manage and implement the DSF program is a problem, given existing human resource shortages within the Bangladesh health system. It was suggested that someone be appointed specifically for monitoring program activities at the district and upazila levels. We asked key informants and providers to what extent local government representatives were involved in DSF program activities. Most felt that local government officials were highly or moderately involved, while a smaller number felt that they had limited involvement (generally concentrated in one universal upazila). Their main role in means-tested upazilas was reportedly certifying the poverty status and residential eligibility of pregnant women. Some noted that they generally play a role in publicizing and promoting the DSF program locally and in recruiting eligible women. “[Local government representatives] are 100% involved. They oversee whether the eligible poor women are getting the benefits of the voucher program at the time of registration and whether the eligible poor women are selected or not.” – DSF Coordinator “The representatives of the Local Government are involved with the voucher program. They are the chairmen of the [upazila] DSF committee or union committee. They verify the cards which are distributed at village level. They also inform the people about the vouchers.” – UHFPO, “They are involved to some extent. They only attend the monthly meeting otherwise they do not play any role.” – FWA/HA Time commitments Key informants and providers were asked how much time they spend per month on voucher-related activities. As expected, DSF Coordinators reported spending all of their time on the program. Some CSBAs and gynecologists also reported spending more than 150 hours per month on voucher activities. UHFPOs and Medical Officers reported spending about 60 hours per month, while FWVs, FWAs, and HAs spent between 45 and 55 hours per month. UNOs (who are the chairmen of the Upazila DSF committees) spent less than one day per month, while Union Parishad members spent about 24 hours per month. 4.4.2 RESPONSIVENESS TO STAKEHOLDERS AND CONSUMERS Women who participated in the focus group discussions had mixed opinions regarding the responsiveness of the voucher program to any consumer complaints. Discussants in six of the ten groups noted that they did not have any complaints about the program. Within these groups, women hypothesized that if they had a problem, they would inform their local FWA or CSBA: Economic Evaluation of DSF Voucher Program in Bangladesh 57 “We do not face any problem in receiving services. If we face any problem then we inform Fazila apa [FWA]. Problems get easily solved if Fazila apa discusses with the doctors about the problems.” [respondents in means-tested upazila] Women in the other four groups had more cynical responses, indicating that lodging complaints was useless, that they did not know where to go to make a complaint, and that they would either be ignored or treated rudely if they attempted to do so: “To whom and where we shall go for complaints? If we go to [DSF Committee] members or chairman then they tell that they do not know.” [respondent in universal upazila] “We have reported our complaint to the doctor but there was no outcome. … When we visit the doctors they do not give us any importance as we are poor.” [respondents in means-tested upazila] 4.5 SUMMARY OF KEY FINDINGS Appropriateness of eligibility criteria Many key informants, both at the local and national level, felt that income and land ownership criteria were too stringent in means-tested areas. According to our household survey, most women (63% in universal and 79% in means-tested areas) would meet the land ownership eligibility criterion. Less than one-quarter would meet the criteria of owning no productive assets. Only a small fraction of the sample had expenditures less than Tk. 2,500 (5% and 7% respectively). Incentives Some upazila-level key informants and providers suggested providing incentive payments to UHFPOs and RMOs, who have substantial administrative and management responsibilities under the DSF program. Field health workers requested that incentives for registering eligible voucher recipients be increased. Key informants and providers felt that the gift box should be eliminated, or at least revised. Some upazila-level key informants and providers suggested increasing cash payments to mothers. Role of the private sector National-level key informants felt that greater efforts should be made to involve private sector and NGO facilities in the voucher program, in order to stimulate competition and improve quality. However, concerns about the current quality of private facilities were also expressed. Voucher awareness Awareness of the voucher program is generally high: 86% of women in the universal and 89% in the means-tested areas who delivered in the previous 6 months reported that they had heard of the program. 58 Economic Evaluation of DSF Voucher Program in Bangladesh Women in the means-tested upazilas seemed to have received more specific information about the program and its benefits than women in the universal upazilas. Different health workers appear to the main information source about the voucher program in universal areas (FWAs and HAs) vs. means-tested areas (CSBAs). Less than half of the women surveyed in the means-tested areas, and less than one in five in the universal areas, were told that the voucher could pay for home delivery with a qualified provider. Voucher distribution 7 out of 10 women in our survey sample had received a voucher booklet during their most recent pregnancy. Surprisingly, this rate was about the same in both universal and means-tested upazilas. Parity and contraceptive use restrictions are not being enforced. Thirteen percent (universal) and 9% (means-tested) of voucher recipients were parity 3. Thirty-seven percent (universal) and 24% (means-tested) of parity-2 voucher recipients did not report using contraception before their second pregnancy. Little targeting is occurring according to poverty criteria in the means-tested areas. More than 63% of women from households owning more than 0.15 acres of land received a voucher. Seventy-one percent of women owning livestock, rickshaw or van received a voucher. Seventy-one percent of women with monthly household expenditures greater than Tk. 2,500 received a voucher. According to DSF targeting criteria, these women were ineligible. Voucher booklets are given to women quickly after they have been identified as eligible, but later in pregnancy than intended. Vouchers are getting to most women in means-tested areas when they are 4 or 5 months pregnant. In universal areas, they are getting to most women when they are 5 to 7 months pregnant. Providers, upazila-level informants, and focus group discussants in some areas expressed concern about corruption in the distribution of vouchers – particularly that relatives of influential people were more likely to receive vouchers. To date, about 80% of the target number of vouchers have been distributed. Means-tested upazilas are closer to achieving their targets (94%) than universal upazilas (71%). Payments to consumers and providers Few voucher recipients seem to be receiving the transport stipends. Transport stipends are typically not distributed at each visit, but more commonly at the same time as the cash incentive after delivery. Some women may not be aware that they received a transport subsidy if it was lumped together with the cash incentive. Among voucher recipients who delivered with a qualified provider, 69% in universal areas and 67% in means-tested areas reported receiving the cash incentive. Delays in the disbursement of funds, both from the central level to the upazilas and from the upazila to the beneficiary or provider, are a serious challenge for the DSF program at an operational level. Delays reduce provider willingness to identify voucher recipients and provide them with maternal Economic Evaluation of DSF Voucher Program in Bangladesh 59 health services. Delays reduce the credibility of the program in the eyes of beneficiaries and lessen the impact of the program on maternal health behaviors. Beneficiaries waste a lot of time waiting for payments on cash distribution days. The requirement that unused monies be returned to the Treasury at the end of the fiscal year presents an additional, serious disruption to program operations and impact. Financial management is challenging for UHFPOs and RMOs. Professional financial managers would reduce their burden and facilitate smoother payment processes. Service provision capacity Human resources shortages at health facilities present a serious challenge to implementing the DSF program. National and upazila-level key informants, providers, and individual women expressed concerns about staff shortages – which are leading to long lines at health facilities and providers who feel overloaded by their workload. This has implications for the long-term sustainability of the program, unless staffing is improved. Shortages of supplies and medicines are also an obstacle to smooth DSF program implementation. Many voucher recipients are still paying for medicines outside of government health facilities, due to stock-outs. The seed funds may be helping address these shortages but they have not solved the problem. Cash incentives may also help women afford these medicines, but typically the incentives arrive much later than the service is used and thus they do not overcome the potential financial barrier at the time of need. Monitoring and evaluation Aside from external monitoring by DSF coordinators who are sponsored by the WHO and DFID, the monitoring and evaluation system for the DSF program is inadequate and not integrated with routine management information systems in health facilities. Key informants suggested that the MOHFW become more involved in routine supervision and monitoring, and that specific individuals (at the central, upazila and union levels) be tasked with these responsibilities and paid a salary for this work. DSF Committee Meetings Upazila and Union DSF committees across the upazilas meet with different regularity; some rarely meet. There is limited coordination among the different DSF committees. 60 Economic Evaluation of DSF Voucher Program in Bangladesh 5. FINDINGS PART 2: EFFECT OF THE DSF PROGRAM ON THE DEMAND SIDE The second overarching objective of this evaluation was to assess the effect of the DSF program on individual women’s demand for maternal health services, including antenatal, delivery and postnatal care, as well as their out-of-pocket expenditures on such services. This section presents the main results of the household survey conducted from July-September 2009 with 2,208 women who had delivered in during the prior 6 months. Women in 8 universal and 8 means-tested DSF upazilas, as well as 16 matched control upazilas, were interviewed about their socio-demographic background, birth history, experience with the voucher program, and pregnancy and delivery care utilized. Section 5.1 begins with a description of the household sample. Section 5.2 summarizes bivariate analyses, while section 5.3 summarizes multivariate regression analyses. Section 5.4 includes relevant findings from key informant interviews and focus group discussions. Finally, section 5.5 presents cost-related analyses. 5.1 HOUSEHOLD SURVEY SAMPLE DESCRIPTION Table 5.1 provides information on the general background of the women in our sample, comparing those from control upazilas to those from intervention upazilas (means-tested and universal). To help in assessing the comparability of our data with national averages, we also include relevant statistics from the DHS 2007, where available. The average age of women in our sample is 24; women in the control area were very slightly older than women in the intervention areas. More than 70% of women interviewed had one or two previous births (parity 1 or 2); women in the intervention areas had slightly fewer births. Nearly all were married and most were Muslim. From 17% to 21% of the women reported having no education. Although not many women state that they normally read the newspaper or magazines, more than 40% of them do watch TV, suggesting that TV could be used to convey information about voucher program to women. TABLE 5.1. GENERAL BACKGROUND OF THE STUDY SAMPLE (%)5 Background information about Bangladesh Control Intervention p-value respondents 2007 DHS N=10,996 N= 1104 N= 1104 Mean age 30.6 24.6 23.7 <0.001 Married 92.7 99.6 98.9 0.143 Religion 0.706 Islam 91.0 91.9 90.5 Hindu 8.1 8.2 9.5 Total number of births ever had 0.078 5 Throughout this report, a p-value less than 0.05 is used as the conventional benchmark for judging whether the difference detected is statistically significant. Economic Evaluation of DSF Voucher Program in Bangladesh 61 Background information about Bangladesh Control Intervention p-value respondents 2007 DHS <=2 52.3 70.0 76.6 >2 47.7 30.0 23.4 Education 0.271 None 32.1 20.7 17.1 Primary 30.0 34.6 34.6 Secondary 30.0 40.0 43.7 College 6.0 3.8 2.5 Madrasha 1.6 1.0 2.1 Exposure to media Reads newspaper or magazine 6.6 9.1 6.9 0.336 Listens to radio 19.0 12.8 11.4 0.506 Watches television 46.8 40.2 44.4 0.410 Membership in local organizations Belongs to Grameen bank 9.1 12.7 10.2 0.337 Belongs to BRAC 6.8 7.2 6.8 0.772 Belongs to ASHA 11.1 11.6 8.7 0.238 Belongs to other organizations 10.4 n/a n/a -- Note: P-value is from Chi2 tests comparing intervention and control groups. Compared to the DHS, our sample appears to be more likely to be married, have fewer children, and be better educated. This is likely because we interviewed women who delivered in the prior 6 months; the DHS includes older women who would likely be less educated and have more children. Our household survey sample was also entirely rural, compared with 77% in the national DHS. Except for age, none of the characteristics of the sample is statistically significantly different between the control and intervention areas. This implies that the selection of matched controls was successful and women delivering within the six months preceding the survey in these areas are reasonably similar. Such balancing in background characteristics helps alleviate potential selection bias when comparing key outcomes between control and intervention areas. Table 5.2 compares the same background characteristics between universal and means-tested areas. Women in the means-tested areas are very slightly older than those in the universal areas. All other characteristics are not statistically different between these groups. TABLE 5.2. GENERAL BACKGROUND OF STUDY SAMPLE, COMPARING UNIVERSAL AND MEANS-TESTED INTERVENTION GROUPS (%) Background information Universal Means- p-value about respondents tested N= 552 N= 552 Mean age 23.4 24.0 0.057 Married 98.5 99.3 0.495 Religion 0.044 Islam 94.9 85.5 Hindu 5.2 14.5 Total number of births ever had <=2 76.1 77.1 0.615 62 Economic Evaluation of DSF Voucher Program in Bangladesh Background information Universal Means- p-value about respondents tested >2 23.9 22.9 Education None 17.8 16.3 0.840 Primary 32.4 37.3 Secondary 45.0 42.3 College 2.5 2.4 Madrasha 2.4 1.8 Exposure to media Reads newspaper or magazine 6.6 7.3 0.787 Listens to radio 10.4 12.5 0.487 Watches television 42.0 47.3 0.417 Membership in local organizations Belongs to Grameen bank 9.5 11.0 0.657 Belongs to BRAC 6.9 6.7 0.925 Belongs to ASHA 7.8 9.8 0.540 Table 5.3 summarizes the socioeconomic characteristics of the households in our sample by intervention status. Again, there are few differences between intervention and control groups. Households are not statistically significantly different in terms of ownership of valuable assets, including land, mobile phone, radio, television, and fridge. However, total household expenditures are significantly larger in the control area. The control group has a slightly higher percentage of women in the richest quintile compared to the intervention group. This suggests that the DSF areas in our sample are more economically disadvantaged than non-DSF areas. TABLE 5.3. HOUSEHOLD SOCIO-ECONOMIC CHARACTERISTICS, COMPARING CONTROL AND POOLED INTERVENTION GROUPS (%) Background information about respondents Control Intervention p-value Household ownership of assets Land 36.6 34.2 0.541 Electricity 42.9 41.1 0.738 Mobile phone 48.8 50.9 0.586 Radio 14.6 12.5 0.344 Television 27.3 24.7 0.497 Refrigerator 3.3 2.0 0.203 Wealth quintile 0.064 Poorest 20.9 19.2 Second 16.4 23.6 Middle 21.8 18.2 Fourth 18.7 21.5 Richest 22.3 17.6 Household head’s education 0.429 None 42.1 47.4 Primary 30.1 27.1 Secondary 22.8 22.0 College and higher 5.0 3.6 Economic Evaluation of DSF Voucher Program in Bangladesh 63 Background information about respondents Control Intervention p-value Household head is working 92.0 92.5 0.747 Monthly total expenditure per capita (Taka) 5663 5185 <0.001 Note: Total expenditure includes household spending on food, clothing, child care, health care, education, transportation, gas, electricity, and other miscellaneous items As noted in the methods section, the wealth quintile variable was constructed in our full sample using information on ownership of land, assets, and monthly expenditures per capita collected in our survey. Thus quintile reflects the wealth status of the woman relative to the sample, not to the general population of Bangladesh. Table 5.4 compares the universal and means-tested sites. Here, land ownership is statistically different. Interestingly, it is higher in the universal area than in the means-tested area. Another noteworthy observation is that household monthly expenditure per capita is also slightly lower in the means-tested compared to the universal DSF upazilas. The purported reason that the DSF program was originally made universal in selected upazilas was because of their greater overall poverty, so this finding raises a question about the selection criteria for universal upazilas. TABLE 5.4. HOUSEHOLD SOCIO-ECONOMIC CHARACTERISTICS, COMPARING MEANS- TESTED AND UNIVERSAL GROUPS (%) Background information about respondents Universal Means-tested p-value Household ownership of assets Land 41.5 25.8 0.009 Electricity 35.4 47.5 0.102 Mobile phone 46.6 55.9 0.053 Radio 10.6 14.7 0.190 Television 22.4 27.3 0.292 Refrigerator 2.3 1.7 0.604 Wealth quintile 0.573 Poorest 18.8 19.6 Second 21.4 26.1 Middle 17.5 19.0 Fourth 23.3 19.5 Richest 19.0 15.9 Household head’s education None 56.8 36.5 <0.001 Primary 21.1 34.1 Secondary 18.0 26.6 College and higher 4.2 2.9 Household head is working 92.4 92.6 0.945 Monthly total expenditure per capita (Taka) 5204 5164 <0.001 Note: Total expenditure includes household spending on food, clothing, child care, health care, education, transportation, gas, electricity, and other miscellaneous items An additional indicator of interest, particularly for the multivariate analyses presented later in this chapter, is the presence an Emergency Obstetric Care (EOC) facility in the upazila. EOC facilities are those which have been upgraded with the necessary equipment and supplies to conduct C-sections and other emergency interventions such as blood transfusions. The presence of an EOC facility might 64 Economic Evaluation of DSF Voucher Program in Bangladesh influence the key outcomes of interest in this analysis, independent of the DSF program, and as such is an important control variable in our analyses. Table 5.5 below indicates which upazilas had EOC- upgraded UHC facilities in our sample, as of 2008. TABLE 5.5. PRESENCE OF EOC UHCS IN UPAZILAS SAMPLED FOR THE HOUSEHOLD SURVEY Division District DSF U/M* EOC Control EOC Upazila facility? upazila facility? Barisal Barisal Banaripara M EOC Babuganj Pathuakhali Kalapara M EOC Galachipa Chittagong Noakhali Chatkhil M Senbagh Cox's Bazar Ramu U Chokoria Dhaka Tangail Sakhipur M EOC Kalihati Faridpur Bhanga M EOC Boalmari Kishoreganj Tarail U Karimganj EOC Manikganj Harirampur U EOC Shibalaya EOC Khulna Jessore Chowgacha M EOC Keshabpur Khulna Paikgachha M EOC Terokhada Kushtia Dalutpur U EOC Mirpur Rajshahi Dinajpur Khanshama U Birganj EOC Sirajganj Shahjadpur U EOC Raiganj Nawabganj Shibganj U EOC Gomastapur Gaibandha Gobindganj U EOC Sadullapur Sylhet Habiganj Baniachong M Lakhai DSF upazilas in our sample were substantially more likely to have EOC facilities – 11 out of 16 DSF upazilas had an EOC facility, compared with 3 control upazilas. This is not surprising given that access to emergency care was a criterion for introducing the DSF program. 5.2 BI-VARIATE ANALYSES In this section, we present cross-tabulations of the key outcome variables of interest (ANC, delivery care, PNC, and expenditures) by intervention group, and identify statistically significant correlations. We present results stratified by universal, means-tested and control groups for comparison purposes, although it is important to note that differences between universal and means-tested areas cannot be attributed to poverty-related targeting, given that little targeting actually occurred. (This issue is explored further in section 5.3 below.) Results for control versus intervention areas (universal and means-tested combined) are presented in the Annex. 5.2.1 ANTENATAL CARE Figure 5.1 and Annex Table 6 present the results for the utilization of antenatal care services among all women, by intervention group. (Annex Table 6B compares control to all intervention.) As shown, women in the universal and means-tested sites had significantly higher likelihood of having at least one ANC visit during their last pregnancy (92% and 91% respectively) than women in the control group (76%). All three groups are more likely to use ANC than the national average, which is 60% (DHS 2007). The percentage of women who had at least 3 ANC visits is 34% in the control group; in the means-tested group, 56% of women had at least 3 ANC visits, while 54% in the universal group did so Economic Evaluation of DSF Voucher Program in Bangladesh 65 (p=0.003). Significantly more women in the means-tested group (58%) had their first ANC within the first 5 months of pregnancy compared with the control (43%) as well as the universal group (38%). FIGURE 5.1. ANC UTILIZATION AMONG ALL WOMEN BY INTERVENTION TYPE 100 90 80 70 60 % 50 40 30 20 10 0 Had any ANC visit Had at least 3 ANCcheck- First ANC was within first 5 ups months of pregnancy Control Universal Means-tested Women in the means-tested area who sought antenatal care were significantly more likely to see a qualified provider than women in the universal or control areas. Figure 5.2 (Annex Table 7) indicates that at least 65% of women in the control area reported having the first ANC checkup (ANC1) from a qualified provider, compared to 76% in universal and 91% in means-tested upazilas (p=0.001). By way of comparison, according to the Bangladesh DHS 2007, 52% of women across Bangladesh received ANC at least once from a qualified provider. Conditioned on use of each service, the percentages of women seeing a qualified provider for ANC2 and ANC3 were each slightly higher than for ANC1. (Annex Table 7B compares control to all intervention.) 66 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 5.2. PERCENT OF WOMEN WHO RECEIVED ANTENATAL CARE FROM A QUALIFIED PROVIDER BY TYPE OF STUDY SITE (%) 100 90 80 70 60 % 50 40 30 20 10 0 First ANC visit Second ANC visit Third ANC visit Control Universal Means-tested 5.2.2 DELIVERY CARE The receipt of obstetric care from a medically qualified provider is essential for the reduction of maternal and neonatal mortality. Our results show that the DSF program is highly statistically significantly associated with increased rates of delivery with qualified providers. As shown in Figure 5.3 (Annex Table 8), only 27% of births in the control group were attended by a qualified provider in 20096 (which is higher than the national average of 18% recorded for 2007 [DHS 2007]). In the intervention areas, 58% of births in universal areas and 70% of births in means-tested areas were attended by a qualified provider. The differences are highly statistically significant (p<0.001). Moreover, while only 47% of complicated births in the control group were attended by a qualified provider, 87% and 82% of complicated births in the intervention and means-tested groups, respectively, received care from a qualified provider (p<0.001). (Annex Table 8B compares control to all intervention.) 6 Qualified providers include a qualified doctor, nurse, midwife, paramedic, family welfare visitor (FWV), or community skilled birth attendant (CSBA). In Bangladesh, medical assistants (MAs) and sub-assistant community medical officers (SACMOs) are considered medically trained providers for antenatal care and postnatal care, however, they are not considered medically trained providers for childbirth (DHS Bangladesh 2007). Economic Evaluation of DSF Voucher Program in Bangladesh 67 FIGURE 5.3. DELIVERY CARE FOR BIRTHS TAKING PLACE IN THE 6 MONTHS PRECEDING THE SURVEY 100 90 80 70 60 % 50 40 30 20 10 0 Qualified provider Qualified provider Institutional C-section (all births) (complicated delivery births) Control Universal Means-tested Notes: The difference among study areas in C-section is statistically significant at p=0.039. All other differences are significant at p<0.001. See Annex table 8 for details. For complicated births, sample sizes are 218 (control), 111 (universal), and 113 (means-tested). Qualified providers are Medical Trained Providers (MTPs), that is, a qualified doctor, nurse, midwife, paramedic, family welfare visitor (FWV), or community skilled birth attendant (CSBA). In addition, Figure 5.3 shows that 19% of births in the control group took place in a health facility. In the universal and means-tested groups, 44% and 30% of births took place in a facility. These results are statistically significantly different from the control group (p<0.001). These rates are also substantially higher than the nationwide 2007 figure, which was 15% (Bangladesh DHS, 2007). Figure 5.3 also presents information on the proportion of births delivered by Caesarean section. Delivery by C-section is often considered a proxy indicator of women’s access to skilled care (Bangladesh DHS 2007). As shown in Figure 5.3, 9% of women in the control group had a C-section at their last birth, which is comparable to the national average of 8% (Bangladesh DHS 2007). A slightly higher proportion of women in the universal group (13%) received C-sections while there was no difference between control and means-tested groups (8%). The difference between means-tested and universal areas was however statistically significant (p=0.01). It is unclear what accounts for the higher C-section rates in universal but not means-tested areas. Some increase in C-section rates is expected and desired as a result of the DSF program, in order to address unmet need for medically indicated surgical deliveries (e.g., in the case of obstructed labor or fetal distress). At the same time, concerns have been raised that the disparity in incentive payments to physicians conducting C-sections (Tk. 3,000 vs. Tk. 150 for vaginal deliveries) might put women at risk of unnecessary C-sections (or encourage facilities to report some normal deliveries as C-sections to obtain the incentive). The DSF Coordinators monitored C-section rates closely to prevent this, and indeed investigated some facilities where surgical deliveries suddenly spiked above 15%. The Directorate General of Health Services also issued several “show cause” orders, requesting an investigation of the medical necessity of certain C- sections. 68 Economic Evaluation of DSF Voucher Program in Bangladesh The distribution of facility-based births across public, private, and NGO facilities varied by intervention area (p<0.001). Table 5.6 indicates that more women in the control upazilas gave birth in private sector facilities (9%), which tend to charge higher fees, compared to women in universal (5%) and means-tested areas (2%). NGO facilities are most commonly used in the universal upazilas (7%) and rarely used in control or means-tested areas. Similar proportions of women delivered in public sector facilities in both universal (32%) and means-tested upazilas (28%), while only 9% of women in control areas gave birth in public facilities. TABLE 5.6. LOCATION OF MOST RECENT DELIVERY, BY INTERVENTION TYPE (%) Delivery location Control Universal Means- tested N=1104 N=552 N=552 Home 80.9 56.1 69.8 Public sector 9.0 31.6 28.0 NGO 0.3 7.0 0.0 Private sector 9.3 5.3 2.1 Other 0.6 0.1 0.1 TOTAL 100.0 100.0 100.0 Voucher recipients in universal and means-tested areas also sought delivery care in different locations (Table 5.7). Voucher recipients in universal areas were generally more likely to deliver in facilities, especially NGO and private facilities. TABLE 5.7. LOCATION OF MOST RECENT DELIVERY AMONG VOUCHER RECIPIENTS (%) Delivery location Universal Means- tested N=420 N=427 Home 47.5 67.1 Public sector 38.9 31.9 NGO 9.2 0.0 Private sector 4.4 1.0 Other 0.1 0.0 TOTAL 100.0 100.0 There is a large and statistically significant difference in the birth attendant assisting women who delivered at home in the control versus two intervention areas (Figure 5.4, Annex Table 9). While in the control area, 10% of home deliveries were attended by a qualified attendant, the corresponding figures in the universal and means-tested areas are 27% and 58% respectively. CSBAs play the major role in the means-tested areas; this is in line with the results presented above regarding their important role in distributing vouchers and voucher information in means-tested areas. (Annex Table 9B compares control to all intervention.) Economic Evaluation of DSF Voucher Program in Bangladesh 69 FIGURE 5.4. SKILL LEVEL OF BIRTH ATTENDANT FOR HOME DELIVERIES BY INTERVENTION TYPE (%) 100 90 80 70 60 % 50 40 30 20 10 0 Control Universal Means-tested CSBA Unqualified Other qualified 5.2.3 POSTNATAL CARE Table 5.8 presents a summary of postnatal care outcomes. For the purpose of examining statistical significance, we include the statistics on ANC and delivery care shown in the preceding figures. (The same table comparing control to pooled intervention groups is shown in Annex Table 10.) The table indicates that 20% of women in the control area reported having a PNC care visit. The rate is 1.5 and 2 times higher in the universal and means-tested areas. Conditioned on use of services, 73% of women in the control group received PNC from a qualified provider. In the intervention and means-tested groups, 84% and 90% of women, respectively, received PNC from a qualified provider. Compared with the control group, more women in both universal (54%) and means-tested groups (39%) sought PNC in the public sector than in the NGO or private sector. Note that 53% of women in the means-tested group received PNC at home, more than any other group. In addition, women in the control group were more likely to use a qualified (37%) or an unqualified private provider (15%) compared with the other two groups. TABLE 5.8. SUMMARY OF KEY MATERNAL HEALTH SERVICE UTILIZATION INDICATORS (%) Maternal health service utilization Control Universal Means- p-value tested ANC care Had any ANC visit 75.6 91.8 91.2 <0.001 Had at least 3 ANC check-ups 33.6 54.0 55.6 0.003 First ANC was within first 5 months of pregnancy 43.2 38.4 57.8 0.017 Provider was qualified at first ANC visit 64.9 75.7 90.7 0.001 70 Economic Evaluation of DSF Voucher Program in Bangladesh Maternal health service utilization Control Universal Means- p-value tested Provider was qualified at second ANC visit 65.2 72.3 92.7 0.001 Provider was qualified at third ANC visit 70.9 75.6 95.4 0.011 Delivery care Provider at last birth was qualified 27.1 58.1 70.2 <0.001 Provider at last complicated birth was qualified 47.3 87.0 82.4 <0.001 Birth took place in a facility 18.7 43.9 30.1 <0.001 Location of last birth Home 80.9 56.1 69.8 <0.001 Public sector 9.0 31.6 28.0 NGO 0.3 7.0 0.0 Qualified private 9.2 4.7 2.1 Other private 0.1 0.6 0.0 Other 0.6 0.1 0.1 Had C-section for last birth 9.1 12.9 7.6 0.039 Postnatal care Had any PNC visit 20.7 31.2 40.6 0.002 Provider was qualified for postnatal care within 6 72.9 84.4 89.7 0.008 weeks of birth Location of postnatal care Home 17.3 13.4 52.9 <0.001 Public sector 28.1 53.8 39.3 NGO 2.1 18.8 0.4 Qualified private 37.3 5.4 5.5 Unqualified private 15.2 8.7 1.9 5.2.4 SUBANALYSIS: ANC, DELIVERY CARE, AND PNC AMONG VOUCHER RECIPIENTS This sub-analysis looks at the percentage of voucher recipients only who used various services. Table 5.9 shows that more voucher recipients in the means-tested group used the voucher for a first, second, and third ANC visit than did women in the universal group, although the differences are non-significant. The use of voucher for delivery care is 20 percentage points higher in the means-tested than in the universal area, and the difference is statistically significant (p=0.003). The low percentage of voucher use for postnatal care and the third ANC visit reflects the fact that few women incurred these services in all study areas. TABLE 5.9. VOUCHER USE FOR ANC, DELIVERY, AND PNC BY INTERVENTION TYPE (%) Voucher use for ANC Universal Means-tested p-value N = 420 N = 427 First ANC visit 62.0 75.3 0.171 Second ANC visit 60.0 74.6 0.128 Third ANC visit 48.1 55.8 0.383 Use for delivery care 61.0 81.3 0.003 Use for postpartum care 26.3 40.1 0.10.8 Economic Evaluation of DSF Voucher Program in Bangladesh 71 Note: The denominator N represents the total number of women who received a voucher. We also looked at where voucher recipients delivered their babies. Figure 5.5 (Annex Table 11) reveals that voucher recipients in the universal area were more likely to have an institutional delivery than means-tested voucher recipients, and less likely to deliver at home with a qualified attendant. Interestingly, overall rates of delivery with a qualified provider among voucher recipients was substantially higher in means-tested areas than in universal areas. Twenty-seven percent of voucher holders in universal areas still delivered at home with an unqualified attendant, compared to 17% of means-tested voucher holders. FIGURE 5.5. PLACE OF DELIVERY AMONG VOUCHER RECIPIENTS BY INTERVENTION TYPE (%) 100% 80% 60% 40% 20% 0% Universal Means-tested At home with unqualified birth attendant At home with qualified birth attendant Institutional Note: The denominator N represents the total number of women who received a voucher. Figure 5.6 (Annex Table 12) examines the use of delivery care among voucher non-recipients. Voucher non-recipients are less likely to have an institutional delivery or deliver at home with a qualified attendant. In both universal and means-tested areas, voucher non-recipients were more likely to deliver at home with an unqualified attendant. 72 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 5.6. PLACE OF DELIVERY AMONG VOUCHER NON-RECIPIENTS BY INTERVENTION TYPE (%) 100% 80% 60% % 40% 20% 0% Universal Means-tested At home with unqualified birth attendant At home with qualified birth attendant Institutional delivery Notes: The denominator N represents the total number of women who did not receive a voucher. 5.2.5 SUBANALYSIS: PRE-POST COMPARISON OF PREVIOUS BIRTHS We examined maternal service utilization outcomes specifically for births taking place prior to the most recent birth reported by the study women. Among these, we distinguished between births that dated before and after the voucher program was launched. The timing of DSF launch was upazila-specific (ranging from August 2006 to June 2007 in our sample). In control upazilas, since there was no “launching” of the program, we use the matched intervention upazila’s timing as threshold for “before” and “after” voucher introduction. Among the 2,208 surveyed women, 653 next-to-last births were recorded, of which 512 took place before voucher introduction and 141 took place from April-August 2007 to February 2009 (prior to the start of our 6-month recall period for most recent births). Due to the small number of observations, we do not distinguish between universal and means-tested areas. TABLE 5.10.DELIVERY CARE OUTCOMES AMONG NEXT-TO-LAST BIRTHS, BEFORE AND AFTER VOUCHER INTRODUCTION Delivery Before voucher introduction After voucher introduction outcomes Control Intervention Control Intervention N=247 N=265 p- N=66 N=75 p-value value Qualified 14.0 19.1 0.372 14.0 33.3 0.089 provider Institutional 10.6 14.9 0.439 12.8 19.7 0.532 delivery C-section 4.0 3.4 0.789 11.4 2.2 0.018 Note: intervention comprises both universal and means-tested areas. Economic Evaluation of DSF Voucher Program in Bangladesh 73 Table 5.10 reveals that even before the voucher introduction, the intervention areas already had somewhat higher rates of institutional delivery and use of qualified birth attendants compared to the control area (15% versus 11% and 19% versus 14% respectively). Notably, the corresponding differences are larger in the post-voucher period. There was no change in rates of birth attendance by a qualified provider in the control area, while in intervention areas attendance by a qualified provider increased 19 percentage points. It is reassuring that although the post-voucher institutional delivery rate among intervention area residents (20%) is higher than the figure in the 2007 BDHS report (15%), the pre-voucher rate is similar. This implies that our sample is not far off the national average level. 5.2.6 OUT-OF-POCKET (OOP) EXPENDITURES A key outcome of interest for this evaluation was the impact of the DSF program on out-of-pocket expenditures for maternal health services. As shown in Figure 5.7, conditioned on use of ANC services, women in the universal and means-tested groups paid significantly less out-of-pocket for all 3 ANC check-ups compared with women in the control group. They also spent significantly less on out-of- pocket expenses for PNC than women in the control group. Although women in the means-tested group were more likely to use home-based PNC while women in the universal group mostly visited the public sector, women in the universal group were also more likely to use free NGO facilities for PNC, which might account for the reduced expenses. Note that, as expected, women in the control group spent the most on out-of-pocket expenses since they were more likely to use a private commercial provider. FIGURE 5.7. OOP EXPENDITURE ON ANC, DELIVERY CARE, AND PNC (IN TAKA) 1600 1400 1200 1000 Taka 800 600 400 200 0 First ANC visit Second ANC Third ANC visit Delivery care Postpartum visit care Control Universal Means-tested 74 Economic Evaluation of DSF Voucher Program in Bangladesh Note: The figures are presented for women who reported using each service. Out-of-pocket delivery costs incurred for the last birth are presented in Table 5.11. Women in the universal and means-tested groups spent significantly less on their most recent delivery than women in the control group (Tk. 945 and Tk. 896 versus Tk. 1480, respectively). The difference appears to be largely driven by payments to the provider and service fees. This is again due to greater use of private providers among women in the control upazilas; public sector providers do not officially charge any service fees. There were no statistically significant differences among the study groups in the total amount spent on lab tests, drugs and supplies either in the facility or outside, transportation, or other items. Note that the reported out-of-pocket expenditures were prior to receipt of any incentive payments, so it is not surprising that transportation costs are not lower in the intervention areas. TABLE 5.11. OOP EXPENDITURE ON DELIVERY BY INTERVENTION TYPE (TAKA) Expenditure item Control Universal Means-tested p-value N=1104 N=552 N=552 Payment to provider 352 152 117 <0.001 Card/Registration 1 0 0 0.318 Service fee 289 81 72 <0.001 Lab test/X-ray 33 9 22 <0.114 Medicine/supplies received from provider 179 174 116 <0.358 Medicine/supplies bought outside facility 389 340 362 <0.665 Round-trip transportation 102 106 73 <0.184 Gifts to delivery service provider 91 62 79 <0.002 Others 45 21 54 <0.164 Total 1480 945 896 <0.001 Table 5.12 compares delivery expenditures among the three sites for normal, complicated, and surgical deliveries. As shown, the universal and means-tested groups spent significantly less on all types of deliveries compared to the control group. TABLE 5.12: OOP EXPENDITURE ON DELIVERY BY TYPE OF DELIVERY AND INTERVENTION TYPE (TAKA) By delivery types Control Universal Means-tested p-value Normal 1045 (886) 567 (441) 585 (439) <0.001 Complicated 3448 (218) 2195 (111) 2089 (113) <0.001 C-section 10002 (110) 4483 (66) 4876 (52) 0.394 Note: figures in parentheses denote the number of women who had respective type of delivery. Tallying up all the reported expenses associated with the pregnancy and delivery care, women in the control group spent Tk. 2,191 on their last pregnancy and delivery, while women in the intervention (Tk. 1,427) and means-tested (Tk. 1,456) groups spent significantly less (p=0.001) (Figure 5.8). Economic Evaluation of DSF Voucher Program in Bangladesh 75 FIGURE 5.8. TOTAL OOP EXPENDITURE ON PREGNANCY AND DELIVERY CARE (TAKA) 2500 2000 1500 Taka 1000 500 0 Control Universal Means-tested Note: Values presented in parentheses are the total number of women who reported using each service. 5.2.7 SUBANALYSIS: DIFFERENTIAL IMPACTS OF THE DSF PROGRAM ACCORDING TO POVERTY STATUS To investigate whether the effects of the voucher program have been different among wealthier and poorer groups, we conducted several sub-analyses on the sample stratified into wealth quintile groups (Quintile 1 vs. Quintiles 2-5). Table 5.13 compares key utilization outcomes between the poorest quintile and the top 4 quintiles (henceforth referred to as non-poor for brevity) separately for the control and two intervention sites. This analysis reveals a rather consistent pattern: in the control upazilas, poor women were statistically less likely to utilize maternal health services compared to the non-poor – while in the two intervention areas, although the poor are still less likely to use these services than the non-poor, the differences are smaller and non-significant in most cases. The use of a qualified provider for complicated births was even statistically significantly higher among the poor than the non-poor in the universal area (98% versus 84%). These patterns suggest that voucher program may have reduced wealth disparities in access to maternal health services. 76 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 5.13. SELECTED MATERNAL HEALTH SERVICES UTILIZATION, BY QUINTILE (%) Control Universal Means-tested Quintile Quintile 2- Quintile Quintile 2- Quintile 1 Quintile 2-5 1 5 1 5 N=221 N=883 N=95 N=457 N=107 N=445 ANC Had any ANC 66.1 78.1 88.7 92.5 84.6 92.9 (0.004) (0.415) (0.124) Delivery care Provider was qualified 9.5 31.7 50.7 59.8 75.9 68.7 (<0.001) (0.208) (0.372) Provider was qualified 35.6 50.4 98.4 84.4 70.9 84.8 for complicated delivery1 (0.112) (0.012) (0.120) Facility birth 6.9 21.9 38.3 45.1 29.5 30.3 (<0.001) (0.367) (0.855) Had C-section 1.9 11.0 12.0 13.1 1.7 9.1 (<0.001) (0.881) (0.015) Postpartum care Had postpartum care 13.2 22.7 27.8 31.9 47.4 38.9 (0.021) (0.604) (0.373) Notes: For complicated delivery, N values are the following: control group quintile 1 (N=41) quintile 2-5 (N=177); universal group quintile 1 (17) quintile 2-5 (N=94); and means-tested group quintile 1 (22) quintile 2-5 (N=91). Figures in parentheses are p-values for the difference between quintile 1 and quintile 2-5 for each intervention type. Figure 5.9 (Annex Table 13) presents the average OOP expenditures for ANC1 incurred by poor and non-poor women. Poor women pay significantly less for the first ANC visit compared to the non-poor. This is true in both intervention and control areas. However, the gaps between poor and non-poor women are smaller in intervention areas. (Annex Table 13B compares control to all intervention.) Economic Evaluation of DSF Voucher Program in Bangladesh 77 FIGURE 5.9. OOP EXPENDITURE ON FIRST ANC VISIT BY POVERTY STATUS (TAKA) 600 500 400 Taka 300 200 100 0 Quintile 1 Quintiles 2-5 Control Universal Means-tested The total out-of-pocket delivery costs incurred by poor and non-poor women for the last birth are presented in Figure 5.10. Among the non-poor, controls paid substantially more than the intervention groups. Expenditures among the poor in the three areas were relatively similar, and were in fact highest in the universal upazilas. The relative reduction in expenditures associated with the DSF program is thus larger for the non-poor. FIGURE 5.10. OOP EXPENDITURE ON DELIVERY BY POVERTY STATUS AND INTERVENTION TYPE (TAKA) 2000 1800 1600 1400 1200 1000 Taka 800 600 400 200 0 Quintile 1 Quintiles 2-5 Control Universal Means-tested 78 Economic Evaluation of DSF Voucher Program in Bangladesh In Table 5.14, we examine the expenditure pattern by each group on detailed items. Poor women pay less on most items, including fee, medicines and supply, transportation, and gift. In the universal area, poor women incur major expenditures to buy medicines that are not available in the facility. This confirms the observation from the qualitative assessment, that health facilities may not have adequate stocks of drugs to address the huge increase in utilization among voucher beneficiaries. TABLE 5.14. OOP EXPENDITURE ON DELIVERY BY POVERTY STATUS AND INTERVENTION TYPE (TAKA) Expenditure item Control Universal Means-tested Quintile Quintile Quintile Quintile Quintile Quintile 1 2-5 1 2-5 1 2-5 N = 221 N = 883 N = 95 N = 457 N = 107 N = 445 Fee payment to provider 87 422 26 179 45 135 Card/Registration 0 0 0 0 0 0 Service fee 8 364 45 89 0 90 Lab test/X-ray 2 42 23 6 0 28 Medicine/supplies received 63 210 58 200 12 143 from provider Medicine/supplies bought 143 455 324 343 165 412 outside facility Round-trip transportation 19 125 57 117 28 85 Gifts to delivery service 79 94 34 68 37 89 provider Others 20 52 10 23 46 56 Total 421 1763 576 1026 334 1039 Notes: All differences between quintile 1 and quintiles 2-5 are significant at p<0.05, except for gift costs in the control group. Expenditure on ‘Other’ items includes total spent on items not listed above and total amount outstanding. Women in Quintile 1 are in lowest income and wealth quintile and women in Quintiles 2-5 are in all other quintiles, i.e. non-poor women. Table 5.15 compares delivery expenditures for normal, complicated, and surgical deliveries by poverty status and study groups. Women in quintile 1 spent significantly less on all types of deliveries compared to non-poor women in all groups. In addition, non-poor women in the means-tested and universal intervention areas spent less on each type of delivery than non-poor women from the control group. Poor women from the means-tested areas spent less than poor women in the control group on all types of deliveries; there were fewer differences between the universal and control groups for poor women, except for C-section costs. Economic Evaluation of DSF Voucher Program in Bangladesh 79 TABLE 5.15. OOP EXPENDITURE ON DELIVERY TYPES BY POVERTY STATUS AND INTERVENTION TYPE (TAKA) By delivery types Control Universal Means-tested Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 Normal 285 (180) 1256 (706) 374 (78) 609 (363) 277 (85) 666 (354) Complicated 1173 (41) 3938 (177) 1253 (17) 2400 (94) 591 (22) 2412 (91) C-section 5259 (7) 10214 (103) 2270 (3) 4931 (49) 440 (9) 5092 (57) Notes: Figures in parentheses denote the number of women who had respective type of delivery. All differences between quintile 1 and quintiles 2-5 are significant at p<0.05 except for normal delivery in the control group and for c-section delivery in the control and universal groups. Women in Quintile 1 are in lowest income and wealth quintile and women in Quintiles 2-5 are in all other quintiles, i.e. non-poor women . Table 5.16 shows the average OOP expenditure on postpartum care among women in the poorest quintile versus all others. Conditioned on service use, poor women spent less on postpartum care in both the control and means-tested areas. However, in the universal area, they paid more than the non- poor. The difference is statistically significant. Given that one outlier in expenditure has been removed from our sample, this result is counterintuitive. TABLE 5.16. PNC COSTS BY POVERTY STATUS (TAKA) Postpartum Control Universal Means-tested care costs by Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 type of study site N=34 N=223 N=35 N=190 N=45 N=183 Total spending on 559 649 363 179 206 302 postpartum care P-value <0.001 0.019 <0.001 Note: Figures apply to women who received PNC. Figure 5.11 (Annex Table 14) depicts the differential expenditure level on all maternal health services (ANC, delivery, and postpartum care) incurred by poor and non-poor women. While it shows that the poor pay less than the non-poor in both the control and the two intervention sites, the magnitude of the difference is much smaller in the intervention areas. Hence, the shrinking gap in utilization between the poor and the non-poor in the intervention areas described above also entails a shrinking gap in OOP expenditures. The voucher program, however, seems to be reducing costs more for the nonpoor women than for the poor women. (Annex Table 14B compares control to all intervention.) 80 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 5.11. TOTAL OOP EXPENDITURE ASSOCIATED WITH PREGNANCY AND DELIVERY CARE BY POVERTY STATUS (IN TAKA) 3000 2500 2000 Taka 1500 1000 500 0 Quintile 1 Quintiles 2-5 Control Universal Means-tested 5.2.8 BENEFICIARY USE OF CASH INCENTIVE Among women in our household survey who received the cash incentives, many responded that they used the cash to purchase nutritious food for themselves. As shown in Table 5.17, 48% of women in the intervention group and 41% of women in the means-tested group said they purchased food for themselves. The second most frequent use of the cash incentive in both groups, although slightly more likely in the means-tested group, was for purchasing medications and supplies for maternity/postpartum/ babies’ postnatal care (27% in the intervention group and 34% in the means-tested group). Finally, women in the means-tested group were slightly more likely to use the cash to buy food for their family than women in the intervention group (27% vs. 16%). TABLE 5.17. BENEFICIARY USE OF CASH INCENTIVES FOR FOOD (%) Use of cash incentive Universal Means-tested p-value N 263 N 287 Purchased nutritious food for herself 47.9 41.3 0.429 Purchased food for family 15.6 27.4 0.059 Purchased food for children 8.3 9.4 0.747 Purchased food for newborn 3.5 13.4 0.001 Purchased other household items 15.9 11.1 0.279 Spent money on transportation for 3.6 9.6 0.060 maternity/postpartum/babies postnatal care Spent money on transportation for other purposes 0.7 1.1 0.554 Spent money on medications/supplies for 26.9 33.9 0.266 maternity/postpartum/babies postnatal care Spent money on medications/supplies for other purposes 9.3 5.7 0.382 Others 10.9 8.3 0.633 Economic Evaluation of DSF Voucher Program in Bangladesh 81 Note: The denominator N is the total number of women who received the cash incentive When asked what the cash incentive was for and how they had used it, respondents in all ten focus groups knew that the incentive was for purchasing food, and almost all reported that they had used the money for this purpose. Milk, eggs, and fruit were listed as the suggested nutritious foods. Respondents in seven of the ten groups indicated that they had used the incentive for purchasing medicines. Other uses reported included investing in the family business, saving for the child’s education, paying family expenses, paying to visit a doctor for the mother or child, repairing the home, and paying off a loan. Comments included: “I purchased food for the family, medicine for children.” [respondent means-tested upazila] “I have bought extra food for the child, invested it in business and bought medicine.” [respondent means-tested upazila] “We did not receive money. We are poor; it could be useful if we would get the money.” [respondent means-tested upazila] 5.3 MULTIVARIATE ANALYSES The multivariate analyses seek to document the effects of the DSF program on selected key maternal health service utilization and expenditure outcomes, adjusted for potentially confounding factors. In most cases, the analysis refers to births taking place within the 6 months preceding the survey (most recent birth). We first used probit regression to estimate the effects of the DSF program – as measured by residing in an intervention upazila or means-tested upazila – on the probability of delivery with a qualified provider and the probability of delivery in an institution (Table 5.18). The figures presented are marginal effects and their robust standard errors are clustered at the village level. These multivariate models control for an array of factors that are potentially associated with delivery with a qualified provider, such as birth order, whether the delivery had complications, mother’s age and schooling, and other information on the household. Information on the husband’s education is not included since it is highly correlated with the household head’s education (the majority of the husbands are actually heads of the household). We also compare the outcomes for prior births among the same women which took place before the DSF program was in place, using a difference-in-differences regression analysis. 82 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 5.18. PROBIT REGRESSIONS: THE EFFECTS OF THE DSF PROGRAM ON THE LIKELIHOOD OF DELIVERY WITH A QUALIFIED PROVIDER AND INSTITUTIONAL DELIVERY Delivery with qualified provider Institutional delivery Variable Marginal effect Robust S.E. Marginal effect Robust S.E. Living in universal DSF upazila 0.418*** (0.055) 0.200*** (0.054) Living in means-tested DSF upazila 0.497*** (0.046) 0.050 (0.046) Living in upazila which has EOC facility -0.079 (0.052) 0.091** (0.039) Birth order: 2 -0.051 (0.044) -0.078** (0.032) Birth order: 3 -0.250*** (0.060) -0.136*** (0.044) Birth order: 4 -0.323*** (0.060) -0.143*** (0.055) Birth order: 5 -0.397*** (0.047) -0.246*** (0.057) Birth order: 6 and higher -0.161 (0.117) -0.066 (0.083) Complicated delivery 0.300*** (0.046) 0.283*** (0.042) Age 0.024* (0.012) 0.018** (0.008) Age squared 0.000 (0.000) -0.000** (0.000) Wealth quintile: second poorest 0.034 (0.057) -0.005 (0.034) Wealth quintile: middle 0.038 (0.045) -0.011 (0.037) Wealth quintile: second richest 0.038 (0.062) 0.012 (0.039) Wealth quintile: richest 0.155** (0.072) 0.110** (0.049) Schooling: primary -0.119** (0.053) -0.037 (0.040) Schooling: secondary -0.014 (0.061) 0.044 (0.044) Schooling: college 0.172 (0.112) 0.256*** (0.083) Schooling: Madrasha -0.122 (0.113) -0.032 (0.091) Religion: Islam -0.090* (0.054) -0.115** (0.046) Member of local women's organization -0.007 (0.042) -0.010 (0.028) Household size -0.011 (0.008) -0.015*** (0.006) Head's education: primary 0.135*** (0.046) 0.075*** (0.028) Head's education: secondary 0.166*** (0.058) 0.097*** (0.037) Head's education: college 0.219** (0.103) 0.078 (0.072) Head is working -0.214*** (0.075) -0.139** (0.060) Observations 2207 2201 Pseudo R-squared 0.242 0.235 Note: Figures presented are marginal effects estimated from the probit model and their clustered robust standard errors (in parentheses). *** significant at p<0.01; ** p<0.05; * p<0.1 Confirming the bivariate analysis, multivariate analysis reveals large and highly statistically significant effects of the DSF program on the likelihood of delivering with a qualified provider. On average, a woman living in a universal upazila has a probability of delivering with a qualified provider that is 42 percentage points higher than a woman living in a control area. The corresponding difference for the means-tested area is 50 percentage points. Results for institutional delivery reflect a different pattern. Here, living in a universal DSF upazila was associated with a 20 percentage point higher probability of delivering in a health facility than living in a control area, while living in a means-tested upazila had a much smaller and nonsignificant effect. In addition, the presence of an EOC facility in the upazila was associated with a significantly higher Economic Evaluation of DSF Voucher Program in Bangladesh 83 likelihood of institutional delivery. It is interesting that while the effect on institutional birth is stronger in the universal area, the effect of the DSF program on qualified birth attendance is larger among women living in means-tested areas. This corresponds with the bivariate results in Table 5.8 above. The marginal effects of other explanatory variables in these models are generally in the expected direction. For example, higher parity women are less likely to deliver with a qualified provider or in a facility. Women who experience complicated deliveries have a 30 percentage point higher probability of being assisted by a qualified birth attendant or seeking facility care. Women living in households with highly educated heads and wealthier women are more likely to be assisted by a qualified provider. Table 5.19 shows the effects of the DSF program on other key maternal service utilization outcomes. This table displays only the key predictor variables of interest – living in a universal or means-tested upazila and presence of an EOC facility in the upazila – although each regression controlled for all the other covariates listed in Table 5.15 as well. The estimates presented are marginal effects from the probit function, and can be interpreted as the percentage point increase in the probability of the outcome, comparing women in a category of interest to women in the reference category. TABLE 5.19. THE EFFECTS OF THE DSF PROGRAM ON SELECTED KEY MATERNAL SERVICE UTILIZATION OUTCOMES Variables Any ANC with Any PNC with C-section qualified qualified delivery provider provider Living in universal DSF upazila 0.178** 0.190*** 0.009 Living in means-tested DSF upazila 0.308*** 0.254*** -0.020 Living in upazila which has EOC facility 0.037 -0.080** 0.017 Observations 2207 2207 2207 (Pseudo) R-squared 0.150 0.123 0.223 Note: All models are adjusted for key covariates, not shown (birth order, delivery complications, age, wealth quintile, education, religion, membership in local organization, household size, household head’s education, and whether household head is working). These probit regression results reveal the large and significant effects of living in a DSF program area on receiving ANC or PNC from a qualified provider. Women living in means-tested upazilas had a 31 percentage point higher likelihood of receiving ANC from a qualified provider and a 25 percentage point greater likelihood of receiving PNC from a qualified provider than women in control areas. For women in the universal upazilas, the corresponding effects are 18 (ANC) and 19 percentage points (PNC). However, the voucher program seems to have no effect on the probability of C-section delivery. The rate of surgical deliveries is marginally significantly higher in universal areas than in means-tested areas (p=0.054). The presence of EOC facilities does not have a significant effect. Table 5.20 displays regression coefficients for two multivariate linear regressions on maternal health expenditures: out-of-pocket expenditures associated with the most recent delivery, and out-of-pocket expenditures associated with the last pregnancy, delivery care, and postpartum care. 84 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 5.20. THE EFFECTS OF THE DSF PROGRAM ON SELECTED MATERNAL EXPENDITURE OUTCOMES Variables Expenditure on Total expenditure delivery (Taka) associated with last pregnancy and birth Living in universal DSF upazila -448** -593** Living in means-tested DSF upazila -487*** -576** Living in upazila which has EOC -148 -274 facility Observations 2208 2208 (Pseudo) R-squared 0.200 0.224 Note: All models are adjusted for key covariates, not shown (birth order, delivery complications, age, wealth quintile, education, religion, membership in local organization, household size, household head’s education, and whether household head is working). Both models reveal strongly statistically significant effects of the DSF program. For OOP expenditures on the delivery only, on average a woman living in a DSF area paid roughly Tk. 450 to Tk. 490 less than a woman living in a non-DSF area. The difference constitutes nearly 10% of the total household per capita expenditure per month. The effect is even larger for total OOP expenditure that incorporates antenatal and postnatal care in addition to delivery. As described, the effects of the DSF program on key maternal health service utilization outcomes appear to differ between means-tested and universal upazilas. It is important to note, however, that these differences cannot be attributed to the application of means testing criteria – because (as discussed in chapter 4) there were few differences in the land ownership, asset ownership, and income characteristics of voucher recipients and non-recipients, or between voucher recipients in means tested versus universal areas. Targeting was not stringently enforced in means-tested areas; other differences between means-tested and universal areas must therefore be responsible for observed differences in rates of delivery with qualified providers and institutional delivery. One possible explanation for the different effects is that there is a greater supply of CSBAs in means- tested areas, facilitating home-based births with CSBAs in those areas. Using data we obtained from the Obstetrical and Gynaecological Society for Bangladesh, we confirmed that on a per-population basis, more CSBAs have indeed been trained in the means-tested areas than in universal or control areas since 2003. However, when we included the supply of CSBAs in our multivariate probit regressions, we found that this variable did not alter our main results. A greater supply of CSBAs did have a statistically significant (though very small) association with the likelihood of delivery with a qualified provider, but not with institutional delivery. Differences between means-tested and universal areas persisted. Given that poverty-related targeting is not the driver of observed differences between outcomes in means-tested and universal areas, it is therefore appropriate to present our key multivariate results pooling all DSF areas together and simply comparing intervention to control areas (Table 5.21). The interpretation of these marginal effects is thus “the percentage point increase in the probability of the outcome, comparing intervention to control areas.” Economic Evaluation of DSF Voucher Program in Bangladesh 85 TABLE 5.21. THE EFFECTS OF THE DSF PROGRAM ON KEY OUTCOMES, COMPARING INTERVENTION AND CONTROL AREAS Key outcomes Marginal effect of residing in an intervention area Delivery with qualified provider 0.462*** Institutional delivery 0.137*** C-section -0.006 Any ANC with qualified provider 0.253*** Any PNC with qualified provider 0.196*** Expenditure on delivery -468** Total expenditure on last pregnancy -584** and delivery Note: All models are adjusted for key covariates, not shown (birth order, delivery complications, age, wealth quintile, education, religion, membership in local organization, household size, household head’s education, and whether household head is working). Finally, an important question for impact evaluation of the DSF program using cross-sectional observations is whether the “effects” found just capture the inherent pre-existing differences between the intervention and control areas. Since the areas were not randomly selected, there are chances that they were already different before the DSF program was in place. For example, it could be the case that women in the intervention area were already more likely to seek formal delivery care irrespective of the program. From the statistics on background characteristics and on service utilization for births before the DSF program, we think this is an unlikely scenario. Table 5.22 addresses this concern by using a difference-in-differences estimation of the DSF effects on three outcomes for which data are available: institutional delivery, qualified birth attendant, and C-section. In this estimation, “post-DSF” refers to births taking place in the 6 months preceding the survey, which all occurred after the initiation of the program. “Not post-DSF” refers to the same women’s previous birth(s) that took place before the DSF program was in place in the upazila in question. The interaction term “DSF area*post DSF period” denotes the program effect – the difference in the outcome between the DSF and non-DSF areas comparing the pre- and post-DSF periods. 86 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 5.22. DIFFERENCE-IN-DIFFERENCES ESTIMATES OF THE DSF EFFECT ON SELECTED MATERNAL HEALTH SERVICE UTILIZATION OUTCOMES, COMPARING PRE- AND POST- INTERVENTION BIRTHS (1) (2) (3) Institutional delivery Qualified birth attendant C-section (Y/N) (Y/N) (Y/N) DSF area*post DSF period 0.202** 0.373*** 0.019 (0.082) (0.091) (0.037) DSF area 0.152 0.270 -0.032 (0.131) (0.172) (0.047) Post DSF period 0.056 0.139*** 0.057*** (0.036) (0.045) (0.012) Observations 2711 2719 2719 Pseudo R-squared 0.249 0.250 0.152 Note: All models are adjusted for characteristics deemed to stay stable during the time interval between the two births in question. They include household head’s education, woman’s education and religion, and upazila dummies. This analysis was not broken down by universal vs. means-tested areas due to the smaller sample size. As shown in Table 5.22, the interaction term is large and statistically significant for institutional births and births with qualified attendant, but not for C-sections. The results imply that births in the intervention upazilas have on average a 20.2 percentage point increase in the likelihood of being delivered in a facility over the pre- to post-DSF periods, relative to births in the control upazilas. The corresponding difference in terms of births with qualified attendants is 37.3 percentage points. Thus panel analysis results confirm what we find from the cross-sectional analysis. Table 5.22 also reveals that there was no difference in outcomes between the DSF and non-DSF upazilas, prior to DSF program initiation, because the coefficient on the variable “DSF area” is statistically insignificant. The positive and significant coefficient for “Post-DSF period” in models 2 and 3 (for qualified birth attendant and c-section) indicates that women in both areas have higher probability for these outcomes in the most recent births compared to births taking place before the beginning of 2007, when the DSF program was instituted. 5.4 QUALITATIVE FINDINGS 5.4.1 BENEFICIARY OPINIONS ABOUT THE VOUCHER PROGRAM Among women who participated in the ten focus group discussions, excitement and positive feelings about the voucher program were nearly universal. Women reported that they felt good about the program because they would get money, they could deliver in hospitals, and they could avoid risk of impoverishment due to the cost of a C-section. “We felt very good. Those who are poor are getting into the facilities.” [respondent in means- tested upazila] “…now the mothers in law are taking their pregnant daughters in law to the hospitals.” [respondent in means-tested upazila] Economic Evaluation of DSF Voucher Program in Bangladesh 87 Focus group discussants generally thought the voucher program was working. Women reported that maternal and neonatal mortality were decreasing, that more women were seeking antenatal care, and more women were going to hospitals for deliveries. Several women noted that women have become more aware of available health services than they were previously. “Previously many times pregnant mothers died due to failure to arrange money. It is not happening now.” [respondent in universal upazila] “All mothers were not aware of their own health care before. Especially, poor mothers would not come for check- ups due to financial crisis. Now, all have come to know about the services and are coming to receive services.” [respondent in means-tested upazila] When asked whether they personally had benefited from the program, most also felt that they had, though some respondents in two upazilas disagreed: “Everybody is getting benefited through this voucher program. Before the initiation of the program, we could not go to doctors due to financial crisis. But now we are getting services free of cost.” [respondent in universal upazila] “We are getting free services. We are getting money to buy medicine and nutritious food.” [respondent in means-tested upazila] “The services and the money or gift box that are provided through voucher should be provided properly. Then we will be properly benefited.” [respondent in means-tested upazila] In three upazilas, there was some indication that vouchers were somewhat more likely to be used for ANC services than delivery or postpartum services. In one means-tested upazila, for instance, respondents noted that voucher holders tend to seek the first ANC visit but not other covered services because the road to the hospital is very bad and health workers do not come to the subcenters regularly. Interestingly, there was universal agreement – across all respondents in all ten upazilas – regarding the positive impact the voucher program has had on the attitudes of husbands and in-laws towards seeking care for pregnant women. Without exception, women indicated that their husbands and mothers-in-law were more likely to support careseeking for antenatal, delivery and postnatal care from health facilities. “My mother-in-law took me to the hospital.” [respondent in universal upazila] “After the voucher program, the mindset of the in- laws or husband has changed. They think if free treatment can be obtained without spending money, then they do not have any problem to send us to the hospital.” [respondent in means-tested upazila] “Fathers in law, mothers in law and husbands do not resist.” [respondent in means-tested upazila] 5.4.2 KEY INFORMANT OPINIONS ABOUT THE VOUCHER PROGRAM Key informants at the upazila level were anxious to emphasize the positive impacts of the DSF program on beneficiaries. There was a widespread perception that the program is working to reduce maternal and infant mortality, and that the provision of incentives is alleviating poverty. A few illustrative quotes are shared below: 88 Economic Evaluation of DSF Voucher Program in Bangladesh “The rates of maternal deaths and neonatal deaths have decreased. … Had the money not been provided to the mothers, they would have to spend money (separately) for the deliveries. Thus, the DSF is playing role in reducing poverty.” – UHFPO “It contributes to poverty reduction. Women who have received vouchers otherwise would not have reported for medical check-ups for want of money. It is because of the voucher that they are able to go for check-ups. With the receipt of Tk. 2000, some of them are doing income- generating activities.” – Union Parishad Chairman “Formerly, women used to have their deliveries, assisted by unskilled traditional birth attendants that resulted in the deterioration of maternal and child health. Now that deliveries are being assisted by SBAs, maternal and child health are protected. The onetime cash received is helping the poor families take care of the mother and child after delivery.” – NGO/private sector member Central-level key informants echoed these perceptions, noting falling maternal and neonatal mortality rates, increasing service utilization, and increasing facility-based deliveries. They also felt that the DSF program is contributing to poverty reduction. “The poor people who are receiving money are very happy for this. This money is helping them a lot. Many women buy a goat with this money, and are generating income from that.” 5.4.3 PROVIDER PERCEPTIONS OF DSF PROGRAM IMPACTS ON BENEFICIARIES Providers almost universally felt that there has been a noticeable change in the health care-seeking behavior of the poor since the inception of the DSF program. Most felt that women were much more likely to proactively seek antenatal, delivery and postnatal care services. Some commented that women are more aware and seek more health information now. A few mentioned that husbands and family members also seem more aware. Many providers also stated that they thought maternal and neonatal deaths had decreased. Providers were asked to assess the extent to which beneficiaries “dropped out” or failed to seek covered voucher services after receiving a voucher booklet. About one quarter of providers thought that drop-outs did occur. It was reported that women were most likely to seek antenatal services, but that deliveries often still occurred at home and postnatal care might not be sought. Providers hypothesized that family and social barriers continued to present obstacles, and that transportation was an issue. Poor quality services and human resource shortages at facilities were also mentioned as possible reasons for drop-outs. 5.4.4 PERCEIVED EFFECT ON USE OF OTHER PROGRAMS AND SERVICES Among focus group discussants, the general perception was that the voucher program was not having much of an impact on family planning activities. Women reported that FWAs and HAs were continuing their regular activities as they had previously while also effectively distributing vouchers. They noted that the FWAs’ workload had definitely increased, but that they were benefiting from the incentives they receive for distributing vouchers. Economic Evaluation of DSF Voucher Program in Bangladesh 89 Respondents did not feel that the voucher program was encouraging women to have children in order to receive the cash incentive. The amount of money was thought to be very small. On the contrary, some respondents felt that the parity restriction on eligibility for vouchers was an added disincentive to having more than two children. A few illustrative quotes included the following: “No, we do not think so. Because to bring up a child it takes more money than the amount that we are getting for delivering [in a health facility].” [respondent in universal upazila] “Because of the voucher program, mothers are now not interested in more than two children.” [respondent in means-tested upazila] 5.5 PROGRAM COST AND COST-EFFECTIVENESS ANALYSES This section presents results of two simple analyses of the costs of the DSF program. The first calculates the average cost of the DSF program, measured as the program cost per voucher distributed. The second analysis estimates the incremental cost of an additional delivery with a qualified provider attributable to the DSF program, which is akin to a cost-effectiveness analysis. This study’s focus was on measuring the impact of the DSF program and on assessing operations of the program; it was not designed to be a comprehensive cost analysis study. However, the basic information presented here should be helpful as policymakers and development partners consider the costs and benefits of different programs, and make decisions about whether and how to scale up the DSF program. This analysis refers to the fiscal year covering July 2008 – June 2009 and the 16 DSF upazilas in which household survey data were collected. This allows comparison with the impact estimates presented earlier in section 5. The analysis also uses a parameter estimate from the multivariate regression analysis presented in section 5.3, estimated from the household survey data. An exchange rate of Taka 68.55 = US $1 is used throughout. 5.5.1 TOTAL PROGRAM COST IN 16 UPAZILAS The total program cost is estimated here based upon four components: i) MOHFW expenditures (using pooled donor funds) on DSF program incentives and subsidies; ii) the value of the proportion of MOHFW staff time spent on DSF program activities; iii) voucher printing costs; and iv) expenditures by the WHO and DFID on the DSF cell in Dhaka and at the upazila level, for program administration, coordination and monitoring. By “cost” we mean financial cost, or the direct costs of the DSF program. This does not include, for example, the capital costs borne by the MOHFW at UHCs, which are designated providers of the DSF program services. 1. MOHFW expenditures on DSF program incentives and subsidies (using pooled funds): Financial statements provide expenditures at the upazila level from the four DSF accounts. Total expenditure for 2008-2009 in the 16 upazilas analyzed here is Taka 235.12 million (US $ 3.43 million). 2. MOHFW staff time spent on DSF program activities: This calculation uses average time spent per month on DSF program activities by the Director, PHC at the central level, and by the UHFPO, RMO, and clerk at each Upazila Health Complex. These personnel are not compensated separately for the DSF program, and thus, their monthly earnings (salary plus allowances) are used to estimate the cost to the DSF program. The total cost to the program for the 16 upazilas is Taka 3.86 million (US $56,296). 90 Economic Evaluation of DSF Voucher Program in Bangladesh 3. Voucher printing costs: This was apportioned among 16 upazilas to get a total cost for 16 upazilas of Taka 1.27 million (US $ 18,586). 4. Expenditures on the DSF cell by WHO and DFID: These expenditures were classified as either capital or recurrent expenditures. The capital expenditure includes the annualized purchase value of vehicles, motorbikes, and computers. Recurrent expenditure is primarily salaries. Expenditures for half of each of calendar years 2008 and 2009 were used as an estimate for financial year 2008-2009. Since total DSF cell expenditures covered all 33 DSF upazilas, this was apportioned to 16 upazilas, yielding Taka 11 million (US $ 160,538). The total program cost for the 16 upazilas in 2008-2009 is thus Taka 251.26 million (US $ 3.67 million). 5.5.2 AVERAGE COST PER VOUCHER DISTRIBUTED The average cost per voucher distributed is calculated as the total expenditures on the DSF program in the 16 upazilas, as described above, divided by the total number of vouchers distributed in the same 16 upazilas. This yields an average cost of the DSF program of Taka 2,836 (US $ 41) per voucher distributed (see Table 5.23 below). TABLE 5.23. CALCULATION OF THE AVERAGE COST PER VOUCHER DISTRIBUTED Total program cost (in Taka) 251,259,700 Total number of vouchers distributed 88,601 Program cost per voucher distributed (in Taka) 2,836 Program cost per voucher distributed (in USD) 41 Note: Exchange rate used is Taka 68.55 = US $ 1 5.5.3 COST-EFFECTIVENESS ANALYSIS Cost-effectiveness analysis is typically used to compare among different programs or program features and answer the question: “Which option offers the most impact for the same unit cost?” In the current case, there is only one program in question, the DSF program. Ideally, we would use pre-DSF program cost and impact data to provide the comparison or reference case. In the absence of pre-program data, we looked at the cost of the DSF program per unit of outcome in DSF areas, compared to the outcome in control areas. The outcome is deliveries by a qualified provider attributable to the DSF program (e.g. compared to the control area using multivariate regression analysis). The analysis estimates an “incremental” cost- effectiveness ratio – cost per additional birth attended by a qualified provider attributable to the DSF program. The estimate of DSF program effect was based on two components: 1. Total number of deliveries with a qualified provider: The total number of expected deliveries in the 16 upazilas was estimated using a crude birth rate of 2.16 percent, applied to upazila total population estimates extrapolated from the most recent (2001) census. The proportion of deliveries with qualified providers observed in our household survey (64%) was used to estimate the total number of deliveries with a qualified provider (65,261) in the 16 upazilas (see Table 5.24 below). 2. Number of deliveries with a qualified provider attributable to the DSF program: We used multivariate probit regression to estimate the adjusted effect of the DSF program on the likelihood of delivery with a Economic Evaluation of DSF Voucher Program in Bangladesh 91 qualified provider (shown in section 5.3 above). The analysis presented there yielded a coefficient of 0.462, which when applied to the total number of deliveries, yields 46,880 deliveries with a qualified provider due to the DSF program (see Table 5.24 below). TABLE 5.24. CALCULATION OF THE NUMBER OF DELIVERIES WITH QUALIFIED ATTENDANT ATTRIBUTABLE TO THE DSF PROGRAM Estimated number of deliveries with qualified provider 65,261 Estimated program effect (probit regression marginal effect) Mean 0.462 95% confidence interval upper bound 0.551 95% confidence interval lower bound 0.373 Estimated number of deliveries with qualified attendant due to DSF program Mean 46,880 95% confidence interval upper bound 56,239 95% confidence interval lower bound 37,522 Note: Estimated number of deliveries with qualified provider was based on the estimated number of deliveries and the percent of deliveries with a qualified provider obtained from the household survey. The total program cost estimated above cannot be used for this analysis because the DSF program subsidizes several outcomes – three ANC visits, safe delivery, and postnatal care.It is necessary to derive an allocation ratio for our specific outcome of interest, that is, delivery with a qualified provider, and allocate a proportion of total expenditure on incentives to this specific outcome. We calculated this ratio by dividing the total nominal value of vouchers for delivery with a qualified provider (numerator) by the total nominal value of vouchers for all services (denominator). The nominal value is the total reimbursement to providers and beneficiaries according to voucher program regulations. The allocation ratio is approximately 90 percent – thus, approximately Taka 224.44 million of the total program cost is allocated to delivery with a qualified birth attendant, including care for complications. Table 5.25 summarizes the information described above and displays the estimated cost-effectiveness ratio. According to our calculations, each additional delivery with a qualified provider that can be attributed to the DSF program had a cost of roughly US $ 70. The 95% confidence interval for this estimate is of US $ 58 - 87. 92 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 5.25. COST EFFECTIVENESS RATIO FOR DSF PROGRAM COMPARED TO CONTROL Program cost attributed to delivery with qualified provider (Taka) 224,442,410 Program effectiveness (deliveries with qualified provider) Mean 46,880 95% confidence interval upper bound 56,239 95% confidence interval lower bound 37,522 Cost Effectiveness Ratio (Taka/delivery with qualified provider) Mean 4,788 95% confidence interval upper bound 3,991 95% confidence interval lower bound 5,982 Cost Effectiveness Ratio (USD/delivery with qualified provider) Mean 70 95% confidence interval upper bound 58 95% confidence interval lower bound 87 5.6 SUMMARY OF KEY FINDINGS Sample characteristics Our sample is better educated than the national average for reproductive-age women, according to the 2007 DHS. However, this is due at least in part to the fact that our sample is younger; the selection criteria included delivering in the prior 6 months. Intervention and control samples were well matched. In contrast to expectations, women in universal upazilas were more likely to own land than women in the means-tested upazilas. Antenatal care Women in the intervention areas were significantly more likely to have at least one ANC visit, and to have at least 3 visits, than women in the control area. Significantly more women in the means-tested group had their first ANC within the first 5 months of pregnancy compared with the control or the universal groups. Women in the means-tested area who sought antenatal care were significantly more likely to see a qualified provider than women in the universal or control areas. About two-thirds of voucher recipients sought any ANC. Delivery care The DSF program is strongly and significantly associated with higher rates of delivery with qualified providers. The likelihood of delivering with a qualified provider was more than twice as high in intervention areas as in the control upazilas. The effect was strongest for means-tested upazilas (70% of births), as compared to universal (58%) and control upazilas (27%). Multivariate regression analysis that controlled for potential confounders confirms these findings: residing in a DSF area was Economic Evaluation of DSF Voucher Program in Bangladesh 93 associated with a 46 percentage point increase in the probability of delivering with a qualified provider. A much greater proportion of complicated deliveries are attended by qualified providers in intervention than in control areas. The DSF program is strongly and significantly associated with higher rates of institutional deliveries. The effect was strongest for the universal upazilas (44% of births), as compared to the means-tested (30%) and control upazilas (19%). Multivariate analyses confirm these results: residing in a DSF area was associated with a 14 percentage point increase in the probability of institutional delivery. The DSF program is not significantly associated with higher rates of C-section deliveries in intervention areas. The rate of C-section deliveries is 13% in universal upazilas, 8% in means-tested upazilas, and 9% in control areas. Home deliveries were significantly more likely to be attended by a qualified provider in the means- tested (58%) and universal (27%) upazilas than in the control upazilas (10%). In both universal and means-tested areas, voucher non-recipients were more likely to deliver at home with an unqualified attendant. The pre-post difference-in-difference analysis confirms these cross-sectional results. The voucher program seems to be affecting delivery patterns differently in universal and means- tested areas. In means-tested areas, there is a greater proportion of home births with CSBAs, while in universal areas, there are more institutional births. However, since poverty-related targeting was not stringently applied in means-tested areas (and since the characteristics of voucher recipients were similar in both means-tested and universal areas), targeting cannot account for the differences in DSF program impacts. Means-tested upazilas have more Community Skilled Birth Attendants per population than universal or control upazilas, but this greater supply of CSBAs does not account for the differences in DSF program impacts in the two intervention areas. Residing in upazilas with an EOC-equipped health facility has an independent and statistically significant positive impact on the likelihood of institutional delivery. Postnatal Care The DSF program is strongly and significantly associated with higher rates of PNC visits. While 21% of women in the control area had a PNC visit, 31% of women in universal upazilas and 41% of women in means-tested upazilas received PNC. The PNC provider was more likely to a qualified professional in the intervention areas, as well. Out-of-pocket expenditures Women in DSF areas spent significantly less on antenatal care than women in control areas. Women in DSF areas spent significantly less on delivery care than women in control areas. 94 Economic Evaluation of DSF Voucher Program in Bangladesh Women in DSF areas spent significantly less on postnatal care than women in control areas. There was no difference in the amount that women in DSF areas spent on medicines purchased outside the facility, compared to women in control areas. Differential impacts by poverty status The poor/non-poor gap in delivery with a qualified provider was smaller in DSF areas than in control areas. In control areas, women from the top 80% of the wealth distribution were more than 3 times more likely to deliver with a qualified provider than women from the poorest 20%. In universal areas, this ratio was 1.2, while in means-tested areas the ratio was 0.9. In other words, the DSF program is associated with reduced inequalities in skilled birth attendance as well as absolute increases in rates of skilled birth attendance. Out-of-pocket expenditures on all types of pregnancy and birth care are lower for the non-poor in intervention areas than for the non-poor in control areas. However, total expenditures among the poor are about the same across all areas. Thus, inequalities in out-of-pocket expenditures are smaller in DSF areas primarily because spending among the non-poor is lower. This partly reflects the fact that the poorest women are still most likely to deliver at home in all areas. Use of cash stipend Most women who received the cash stipend (Tk. 2,000) reported using it appropriately, for food or medications. Perceived effects of the DSF program Key informants, providers, and focus group discussants consistently reported that they believe the voucher program is working, in terms of increased use of ANC, deliveries with qualified providers, and PNC. Program costs and cost-effectiveness The average cost of the DSF program per voucher distributed was estimated to be Taka 2,836 (US $ 41). Each additional delivery with a qualified provider that can be attributed to the DSF program cost roughly US $ 70. Economic Evaluation of DSF Voucher Program in Bangladesh 95 6. FINDINGS PART 3: EFFECT OF THE DSF PROGRAM ON THE SUPPLY SIDE Finally, the third overarching objective of this evaluation was to assess the impact of the DSF program on the provision of maternal health services. Here, we focus on results from the Facility Quality Assessment, including differences between voucher and control facilities in service volumes, health outcomes, staffing, infrastructure and supplies, and provider knowledge and skills. The household survey and provider and key informant interviews also provided information on provider-related impacts. 6.1 RESULTS FROM FACILITY QUALITY ASSESSMENT 6.1.1 SAMPLE DESCRIPTION Eight (8) Upazila Health Complexes (UHC) in DSF upazilas and 8 UHCs in matched control upazilas were selected for the facility quality assessment. Facilities were matched on hospital bed size to the extent possible. The facilities were similarly staffed on the day of the assessment, with slightly more professional clinicians (UFPOs, Physicians/Gynecologists/Medical Officers, Senior Staff Nurses, or Senior Family Welfare Visitors [FWVs]) available for interview in the voucher than control facilities, and slightly fewer staff of other types (Table 6.1). TABLE 6.1. TOTAL NUMBER OF RESPONDENTS AVAILABLE FOR INTERVIEW DURING FACILITY QUALITY ASSESSMENT, BY RESPONDENT TYPE Respondent type Intervention Control UHFPO 7 8 UFPO 3 2 Physician, Gynecologist, or Medical Officer 5 4 Medical Officer (MCH-FP) 1 1 Senior Staff Nurse 8 7 Senior FWV 6 4 Statistician 5 6 Store keeper 6 7 Other 3 4 Total 44 47 While the total hospital bed capacity was similar between voucher and control facilities (since it was one of the criteria for identifying matched control upazilas), the total number of maternity beds was substantially greater in the voucher than control facilities (Table 6.2 below). The average number of maternity beds was also substantially, though not statistically significantly, greater in the voucher than control facilities. The distances to the nearest UHFWC and hospital were also similar for control and Economic Evaluation of DSF Voucher Program in Bangladesh 97 intervention facilities. These data indicate that the matching criteria resulted in selection of fairly comparable facilities. One factor that differed between intervention and control facilities, as described in section 5.1 above, is whether they had been upgraded to provide emergency obstetric care (EOC). For this assessment, 5 out of 8 intervention and 2 out of 8 control facilities were EOC facilities. (For additional details, Table 5.5 in chapter 5 lists the specific upazilas with EOC facilities.) TABLE 6.2. CHARACTERISTICS OF ASSESSED FACILITIES Intervention Control p-value 31-bed capacity 75% 63% 0.59 50-bed capacity 25% 37% Total number of maternity beds (sum across all facilities) 83 61 Average number of maternity beds per facility 10.4 7.6 0.30 Average distance to nearest UHFWC (km) 4.0 4.0 1.00 Average distance to nearest hospital (km) 28.6 28.6 1.00 Facility upgraded to provide EOC 63% 25% 0.13 6.1.2 MCH/FP SERVICES PROVIDED IN THE YEAR PRIOR TO ASSESSMENT Table 6.3 and Figure 6.1 indicate that the voucher facilities have about 60% more maternal health contacts7 than control facilities on average. The greater number of maternity beds in voucher areas may thus reflect redistribution of beds based on increased use as a consequence of the voucher program. TABLE 6.3. AVERAGE NUMBER OF MATERNAL HEALTH PATIENTS IN 2008 Intervention Control p-value Number of maternal and newborn health contacts per month 653 410 0.32 Antenatal care visits per month 7,605 989 0.04 Deliveries per month 2,168 229 0.03 C-sections per month 263 10 0.01 Complicated deliveries per month 515 69 0.04 7 Defined as any visit by a pregnant woman or newborn less than 1 month of age. 98 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 6.1. AVERAGE NUMBER OF DELIVERIES AND PNC VISITS IN 2008 2,500 2,000 1,500 1,000 500 0 Deliveries C-sections Complicated Postnatal care deliveries visits Intervention Control Overall, UHCs in voucher areas are seeing a dramatically and significantly higher number of patients for voucher-covered services than UHCs in control areas. The average number of ANC visits during 2008 was much higher in voucher facilities than in control facilities, and this difference was statistically significant (7605 vs. 989, p=0.04). On average, the 8 voucher facilities provided care for 2,168 deliveries during the year, while the 8 control facilities provided care for 229 deliveries on average (p=0.03). Significantly more C-sections were performed per facility (263 vs. 10, p =0.01) and significantly more complicated deliveries were attended per facility (515 vs. 69, p=0.04) in voucher facilities as well (Table 6.3). However, the incidence of complicated deliveries was lower in voucher than control facilities, because so many more deliveries were attended at voucher area facilities (Table 6.4), while the incidence of C-sections was higher in voucher facilities: TABLE 6.4. KEY MATERNAL AND NEONATAL HEALTH INDICATORS Intervention Control p-value Complicated deliveries as percent of total deliveries 24% 30% <0.001 C-sections as a percent of total deliveries 12.1% 4.4% <0.001 Stillbirths as percent of total deliveries 1.54% 2.45% <0.001 Newborn deaths as percent of live births 0.05% 0.15% 0.15 Maternal deaths as percent of live births 0.05% 0.0% 0.42 As well, the incidence of stillbirths was significantly lower in voucher than in control facilities. The incidence of newborn deaths was substantially lower in intervention facilities but not significantly so. The incidence of maternal deaths was higher in the voucher than control facilities, but also not significantly so (0.05% in voucher facilities vs. 0% in control facilities, p=0.39). Note that our sample size is insufficient to accurately detect differences in rare events like maternal deaths. It is also likely that more maternal deaths in control areas were occurring in homes, since fewer women were presenting to facilities at all. Economic Evaluation of DSF Voucher Program in Bangladesh 99 Since more intervention than control facilities in our sample were EOC facilities, it is possible that EOC upgrades might account for better outcomes in intervention areas (rather than the DSF program itself). While we do not have a sufficiently large sample size to investigate this potential confounder thoroughly, we conducted a simple subanalysis limited to the 5 intervention EOC facilities and 2 EOC control facilities. The patterns observed in the table above persist in this subanalysis (see Annex Table 15): significantly higher C-section rates, lower stillbirth rates, and lower neonatal mortality rates in EOC intervention facilities than EOC control facilities, and no significant difference in maternal death rates. This implies that differences are not entirely attributable to EOC upgrades, but likely attributable to the DSF program as well. Annex table 16 indicates that voucher facilities provided more IUD and sterilization services in 2008, the year prior to the assessment. Voucher facilities provided fewer condom, oral contraceptive, and injectable contraceptive services than control facilities. Voucher and control facilities provided Norplant contraceptive services to a similar number of women. Although none of the differences were significant, these data confirm that voucher facilities provide proportionately more MCH services than control facilities. 6.1.3 STAFFING Table 6.5 indicates that voucher program facilities had slightly more obstetrician/gynecologists and anesthesiologists than control facilities. Conversely, voucher facilities had slightly fewer nurses, general practitioners (physicians), lab technicians and medical officers than control facilities. Voucher facilities had substantially and marginally significantly fewer FWVs than control. Each facility had an Expanded Programme for Immunisation Technician, whereas only 1 voucher and 1 control facility had a pediatrician. TABLE 6.5. AVERAGE NUMBER OF KEY PERSONNEL ON STAFF, PER FACILITY Intervention Control p-value Obstetrician & Gynecologist 0.9 0.8 0.66 Anaesthetist 0.5 0.4 0.64 General Physician 3.3 5.3 0.29 Nurse 8.5 9.9 0.56 FWV (FP) 2.3 5.0 0.09 MO (FP) 1.0 1.1 0.73 EPI Technician 1.0 1.0 -- Paediatrician 0.1 0.1 0.93 Lab Technician 1.6 2.0 0.27 There were different patterns of vacancies between voucher and control facilities. Table 6.6 summaries the number of vacancies per facility during 2008, by type of personnel. Voucher facilities had fewer vacancies for OB/GYNs than control facilities, but more vacancies for anesthetists, general physicians, nurses, and FWVs. 100 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 6.6. AVERAGE NUMBER OF VACANCIES PER FACILITY IN 2008 Intervention Control p-value Obstetrician & Gynecologist 0.0 0.3 0.17 Anaesthetist 1.6 0.3 0.39 General Physician 2.9 1.1 0.31 Nurse 3.6 1.5 0.13 FWV (FP) 1.6 0.3 0.39 MO (FP) 0.3 0.4 0.69 EPI Technician 0.1 0.0 0.35 Paediatrician 0.1 0.0 0.35 Lab Technician 0.0 0.5 0.03 Annex Table 17 indicates the facilities had similar amounts of support staff except for ayas and “other” staff. Only one control and no voucher facilities had a receptionist. All had at least one guard, sweeper, ward boy, statistician, store keeper, driver and cook. Figure 6.2 displays indicators of facilities’ capacity to handle obstetric emergencies; the corresponding table is presented in Annex Table 18. All but two voucher and two control facilities had personnel to handle obstetric emergencies through management or referral. Almost all (88%) voucher facilities had personnel qualified to conduct Caesarean sections and transfusions whereas only 57% could in control facilities (p=0.10). Most (7 of 8) voucher and (6 of 8) control facilities provide maternity services all of the time, including at night and weekends. All voucher facilities had a staff member always available at night compared with 75% of control facilities, however almost none live or stay on site for night duty. FIGURE 6.2. CAPACITY TO PROVIDE EMERGENCY OBSTETRIC CARE (EOC) 120% 100% 80% 60% 40% 20% 0% Personnel on site Facility has Facility has M aternity services Staff member Staff member can take care of all someone qualified someone qualified available all the time available at night available on site at obstetric to conduct C- to provideblood (incl night emergencies, sections transfusions nights/weekends) through management or referral Intervention Control Table 6.7 below shows the extent to which the facility staffing is sufficient to respond to maternity and newborn care according to the Mother Baby Package (MBP) criteria. Facilities received a score Economic Evaluation of DSF Voucher Program in Bangladesh 101 reflecting the extent to which they had all required staff to address specific maternal and newborn care needs. In general, voucher facilities are more adequately staffed to handle abortion complications, ANC, eclampsia, hemorrhage, normal labor and delivery, obstructed labor, postpartum care, sepsis and STDs than control facilities, although none of the differences are statistically significant. Many of these differences are substantial but not statistically significant given the limited number of facilities studies. Voucher facilities were similarly staffed as control facilities to provide anesthesia, and newborn care, and voucher facilities were slightly less well staffed to handle severe anemia. TABLE 6.7. AVERAGE FACILITY SCORES: SUFFICIENCY OF STAFFING TO PROVIDE MATERNITY AND NEWBORN CARE, ACCORDING TO MOTHERBABY PACKAGE CRITERIA Intervention Control p-value Abortion complications 0.84 0.69 0.29 Anesthesia 0.50 0.47 0.91 Severe anemia 1.00 1.13 0.35 ANC 0.97 0.88 0.23 Eclampsia 0.88 0.79 0.34 Haemorrhage 0.88 0.76 0.17 Basic newborn care 0.71 0.74 0.65 Normal labor and delivery 1.00 0.88 0.35 Obstructed labor 0.88 0.73 0.28 Postpartum care 1.00 0.95 0.35 Sepsis 0.97 0.81 0.25 Sexually transmitted infections 1.00 0.93 0.36 6.1.4 REGISTERS, EDUCATIONAL MATERIALS AND SESSIONS Annex table 19 summarizes the extent to which facilities had maternal health and family planning registers and protocols available. Voucher facilities had (seen or reported) more ANC and delivery registers, and more clinical management protocols for maternal health and family planning than control facilities. Control facilities however had more family planning ANC and delivery registries than voucher facilities, in essence balancing out the difference. Annex Tables 20 through 22 indicate whether the facilities had educational pictures, charts and leaflets or handouts for pregnancy warning signs, postpartum/newborn care or breastfeeding, FP, or sexually transmitted infections, in the Health and FP departments. There were two statistically significant differences in these indices; twice as many voucher facilities had postpartum/newborn care/breastfeeding leaflets and handouts whereas half as many voucher facilities had FP leaflets or handouts. Annex Table 23 indicates that voucher areas held educational sessions for mothers slightly (but not significantly) more frequently than control facilities. 6.1.5 REFERRAL CAPACITY Table 6.8 below indicates that slightly more voucher than control facilities have transportation for emergency obstetric care referral. All but two voucher and control facilities had a phone or radio for emergency obstetric care referral. All voucher facilities and 75% of control facilities have a car for obstetric emergencies; the two control sites without cars have an ambulance or use baby taxis for transferring obstetric emergencies. The time to reach a referral facility is nearly identical between voucher (56 minutes) and control (52 minutes) facilities using the ambulance, car or otherwise most common form of transportation. This is a significant amount of time, however, for women experiencing postpartum hemorrhage -- particularly for women delivering at home, presenting to the site and being 102 Economic Evaluation of DSF Voucher Program in Bangladesh referred upwards for care. While the time to reach the referral facility is similar, voucher facilities take an average of 33 minutes to prepare and transfer the patient compared with 53 minutes in control facilities (p=0.51). TABLE 6.8. EMERGENCY TRANSPORTATION SYSTEMS Intervention Control p-value Has transport system or ambulance in place to refer an obstetric 100% 88% 0.35 emergency Telephone or radio available 88% 75% 0.55 Average time to nearest referral facility using most common mode of 56.25 51.88 0.72 transport Available modes of transportation to referral facility: Car 100% 75% 0.17 Baby Taxi 75% 50% 0.33 Bus 75% 25% 0.05 Boat 0% 13% 0.35 Rickshaw 13% 0% 0.35 Other Transport 13% 38% 0.28 Average wait time before leaving the facility to be transferred to 33 53 0.51 referral facility (minutes) All upper level referral sites for voucher and control facilities have an obstetrician/gynecologist and a blood bank (Annex Table 24). All but one control facility upper level referral sites also have an anesthesiologist. In general, in both voucher and referral facilities, a nurse or family member usually arranges emergency obstetric transportation to an upper level referral facility (Annex Table 25), and it is usually a family member who accompanies the woman to the referral facility. All voucher and control facilities have referral slips and refer patients to upper level DH, MCWC, or MCH facilities (Annex Table 26). Similar numbers of patients are referred upwards from voucher and control facilities (19 vs. 22, p=0.73). All voucher and control facilities receive referrals. Slightly although not significantly more patients per month are referred to voucher than control facilities (9 vs. 6, p=0.64). 6.1.6 INFRASTRUCTURE Facilities were assessed on a variety of basic indicators of infrastructural quality and amenities. Most voucher and control facilities had citizens’ rights posted, charts of available medications, private examination and counseling areas, a storage area or cupboard for drugs and other supplies, toilets and refrigeration or cold storage for medications (Annex Table 27). All had ANC rooms, while voucher facilities tended to have more ANC rooms than control facilities (1.63 vs. 1.25 on average). All ANC exam rooms had adequate lighting except for one control facility. All had running tap water in the facility; the distance to the nearest water supply was shorter in voucher than control facilities. Of those reporting how the facility sterilizes its equipment, all but one control facility use autoclaves; the remaining control facility boils to sterilize equipment. Nearly two-thirds of voucher facilities have generators compared with only a quarter of control facilities. For labor and delivery, voucher and control facilities all have a similar number of labor rooms (one or two each) with adequate air flow and electricity (Annex Table 28). More voucher facility labor rooms have windows. None of the voucher or control facility labor rooms had cracked floors or walls, Economic Evaluation of DSF Voucher Program in Bangladesh 103 however fewer voucher than control labor room walls were dirty. All labor beds in the control facilities were reported to be in good condition compared with about half in the voucher facilities, likely due to the much greater use of labor beds in the voucher facilities. More voucher than control facilities had an operating table, an operation theatre for Cesarean section, or operation room portable lighting. Only three-quarters of facilities in both areas had labor room roofs/ceilings that provide protection from rain and sun. 6.1.7 WHO MOTHER BABY PACKAGE ASSESSMENT OF SUPPLIES, EQUIPMENT, LABORATORY AND MEDICATIONS CAPACITY The WHO Mother Baby Package (MBP) has defined sets of basic recommended supplies, equipment, lab capacity, and medications needed to handle essential maternal and newborn health services. This section describes the extent to which facilities had complete sets of these inputs. Supplies According to MBP criteria, voucher facilities on average possess a greater proportion of supplies to handle abortion complications than control facilities (Figure 6.3; full table displayed in Annex Table 29). However, voucher facilities have fewer MBP-recommended supplies to handle severe anemia than control facilities (29% vs. 46%) and fewer have antenatal supplies or cards; none of these differences is large or statistically significant. None of the facilities have the recommended supplies for managing sexually transmitted infections. In general, voucher facilities possess a greater proportion of MBP- recommended supplies for normal labor and delivery (100% vs. 73% complete), eclampsia (76% vs. 58%), hemorrhage (80% vs. 59%) and newborn care (48% vs. 42). None of the facilities had the MBP recommended supplies for sepsis. None of the differences were large except for normal labor and delivery, and none were statistically significant. 104 Economic Evaluation of DSF Voucher Program in Bangladesh FIGURE 6.3. AVERAGE AVAILABILITY OF REQUIRED SUPPLIES TO PROVIDE ESSENTIAL MATERNAL HEALTH SERVICES, ACCORDING TO WHO MOTHER-BABY PACKAGE CRITERIA 120% 100% 80% 60% 40% 20% 0% r s a e s s d i s e a i o i e g n e r e i s t s b r a o c t a p n i m i i p a t e h l c o e v r e i m a m v r t i r n s l d n a S r c s e l o r i a l n e e o n s c t o p o d m b e a i c E b t l r r e C A m t u c a e w r o H N t e v y e l f m c l s A e r n N a i b o S u O N x e S Intervention Control Equipment Voucher and control facilities have equivalent proportions of MBP-recommended equipment to manage abortion complications (60% vs. 58% complete), anemia (33% in both), and fewer voucher than control facilities have the recommended MBP ANC equipment (38% vs. 54%; Figure 6.4 and Annex Table 30). Voucher facilities have slightly more MBP recommended equipment than control facilities to manage hemorrhage and obstructed labor (63% vs. 53% for both). Voucher and control facilities have similar proportions of MBP recommended equipment to manage normal labor and delivery (both 0%), eclampsia (54% vs. 50% complete) and newborn care (60% vs. 68%). Voucher facilities have significantly less MBP recommended equipment than control facilities to handle sepsis (33% vs. 58%, p=0.03). None of the other differences were statistically significant. Economic Evaluation of DSF Voucher Program in Bangladesh 105 FIGURE 6.4. AVERAGE AVAILABILITY OF EQUIPMENT RECOMMENDED BY THE WHO MOTHER-BABY PACKAGE 80% 70% 60% 50% 40% 30% 20% 10% 0% Abortion Anemia ANC services Eclampsia Hemorrhage Obstructed labor Sepsis Newborn care complications Intervention Control Laboratory Capacity Voucher facilities have somewhat greater proportions of MBP-recommended laboratory capacity for abortion complications and severe anemia (both 48% vs. 38%; Annex Table 31), ANC (41% vs. 27%), eclampsia (56% vs. 38%) and sepsis (75% vs. 50%). None of the differences were statistically significant. There were no differences in MBP recommended laboratory capacity for hemorrhage or obstructed labor (all 63% complete). Medications Substantially though not statistically significantly more voucher than control facilities had the drug of choice (75% vs. 52%) and other alternative medications (66% vs. 52%) for management of abortion complication (Table 6.9). More voucher than control facilities also had more complete MBP- recommended medications for management of abortion complications (100% vs. 88%, assuming use of alternative less but still efficacious medications). Voucher facilities had less complete MBP- recommended ANC medications than control facilities (53% vs. 66%). None of the differences were statistically significant. There were no virtually differences in completeness of medications for severe anemia or STDs (including syphilis). More voucher than control facilities had more complete MBP recommended medications for eclampsia (51% v. 29%), hemorrhage (55% vs. 41%) and obstructed labor (45% vs. 28%; Table 6.9). More voucher than control facilities had the drug of choice for hemorrhage (55% vs. 41%) and none had alternative (less but still efficacious) medications for hemorrhage. There was virtually no difference in the proportion of MBP-recommended medications for normal labor and delivery, sepsis, postpartum or newborn care between voucher and control facilities. Nor was there any difference in the proportion of voucher or control facilities with the drug of choice for eclampsia. 106 Economic Evaluation of DSF Voucher Program in Bangladesh TABLE 6.9. PRESENCE OF ESSENTIAL MEDICATIONS FOR TREATING MATERNAL HEALTH CONDITIONS Intervention Control p-value Average proportion of MBP-recommended medications for: Management of abortion complications 0.66 0.52 0.10 Abortion complications: Drug of choice 0.75 0.38 0.15 Abortion complications: Essential (Drug of 1.00 0.88 0.35 choice or alternate drug) Severe anemia 0.63 0.67 0.68 ANC 0.53 0.66 0.23 STIs 0.80 0.78 0.82 Syphilis 0.42 0.46 0.77 Normal deliveries 0.41 0.38 0.82 Eclampsia 0.51 0.29 0.06 Eclampsia: Drug of choice 0.25 0.25 1.00 Eclampsia: Essential (Drug of choice or 1.00 0.88 0.35 alternate drug) Hemorrhage 0.55 0.41 0.06 Hemorrhage: Drug of choice 0.63 0.38 0.35 Hemorrhage: Essential (Drug of choice or 0.00 0.00 -- alternate drug) Obstructed labor 0.45 0.28 0.09 Sepsis 0.71 0.67 0.66 Postpartum care 0.81 0.88 0.69 Newborn care 0.44 0.49 0.72 6.1.8 PROVIDER TRAINING, BACKGROUND, KNOWLEDGE AND SKILLS At each facility, where available, the most obstetrically experienced physician and senior nurse were interviewed to qualify their EOC/ENC training, background, knowledge and skills. The most experienced professionals can serve as a resource and guide to others in the facility and affect quality of care (QOC). Systematic selection of the most qualified physicians and senior nurses for interview also best ensured sample comparability between voucher and control groups. In the eight voucher facilities, two obstetrician/gynecologists, two physicians and two FWVs were interviewed; there were no physicians present for interview in the remaining two facilities. In the control facilities, two obstetrician/gynecologists, four physicians (one of whom was the facility medical officer), and one FWV were interviewed; there was no physician present for interview in the remaining facility. More voucher facility physicians had Emergency Obstetric Care (EOC) and Essential Newborn Care (ENC) training than control facility physicians (Figure 6.5 and Annex Table 32). More voucher facility nurses had EOC training than control facility nurses. There was little or no difference in nurse ENC training or in EOC or ENC training of FWVs or other clinical staff between voucher and control areas. Economic Evaluation of DSF Voucher Program in Bangladesh 107 FIGURE 6.5. PROPORTION OF MOST SENIOR HEALTH FACILITY STAFF WITH EMERGENCY OBSTETRIC AND EMERGENCY NEONATAL CARE TRAINING 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Physicians: Physicians: Senior nurses: Senior nurses: FWVs: EOC FWVs: ENC EOC training ENC training EOC training ENC training training trainings Intervention Control Physician knowledge and skills to manage obstetric complications were also assessed using the OBGuide.xls (http://www.popcouncil.org/rh/palmtops.html) based on the American College of Nurse Midwives Life Saving Skills guidelines. This assessment qualifies EOC knowledge and skills on a 2 to 5 point scale as follows: TABLE 6.10. EOC KNOWLEDGE AND SKILLS ASSESSMENT GUIDELINES Knowledge Scale Scale Values Identification of antepartum hemorrhage (APH) 1-4 1=Good 2=Sufficient 3=Fair 4=Poor APH Action 1-5 1=Good Clinical (Asking) Identification of Infection 2=Sufficient Infection Action 3=Fair Eclampsia Action 4=Poor Identification of Dysfunctional (Obstructed) Labor 5=None (Very Poor) Identification of Postpartum Hemorrhage (PPH) PPH Action Infection Clinical Follow Up 3 1=Good Pregnancy Induced Hypertension Action 2= Fair 3=Poor PPH Urgent Action 2 1=Good 2=Poor 108 Economic Evaluation of DSF Voucher Program in Bangladesh Physicians As noted in the staffing section, voucher program facilities had slightly more obstetrician/gynecologists and anesthesiologists, but slightly fewer general practitioners (physicians) than control facilities (Table 6.5 above). Table 6.11 describes relevant characteristics of the most experienced/Senior OB/MD/Other of each facility assessed. TABLE 6.11. CHARACTERISTICS OF MOST SENIOR PHYSICIAN, OBSTETRICIAN OR GYNECOLOGIST Intervention Control p-value Average number of deliveries attended in past 6 months 151 26 0.20 Average salary per month (Tk.) 16,671 13,463 0.38 Happy in job 67% 71% 0.87 Plan to remain in current job 67% 86% 0.47 Physicians working in voucher facilities are conducting many more deliveries within a six month period than their control facility counterparts. The average monthly salary of the most experienced (in obstetrics) voucher facility physicians is also substantially though not significantly higher. While they appear to be equally happy in their positions (voucher 67% vs., control 71%), fewer voucher facility physicians plan to stay in the post (voucher 67% vs. control 86%). Physician knowledge was similar between voucher and control facilities for all indices except dysfunctional labor, where voucher physicians had fair to poor knowledge compared with control physicians who had sufficient to poor knowledge (Annex Table 33). Identification of antepartum hemorrhage (APH), APH action, clinical identification of infection, infection action and follow up, pregnancy induced hypertension action, eclampsia action, and postpartum hemorrhage (PPH) identification were fair to poor in both voucher and control facilities. It is possible that the knowledge assessment may suggest additional pretesting and modification of the assessment questions was necessary, and is not necessarily an accurate reflection of provider knowledge. The assessment of physicians’ EOC knowledge and skills may also be influenced by the two voucher and one control site where physicians were not available for evaluation. Most physicians have the knowledge to perform ANC risk screening, intravenous administration, check hemoglobin levels and conduct external bimanual compression of the uterus in both voucher and control facilities. Fewer have the knowledge to use partographs (voucher 67% vs. control 43%), conduct internal bimanual compression, repair cervical or vaginal lacerations, or conduct external or internal versions, menstrual extraction, reflex tests, speculum or bimanual exams. However, more voucher than control facility physicians had the knowledge to conduct manual removal of the placenta (100% vs. 71%), repair episiotomies (83% vs. 57%) and repair third to fourth degree lacerations (67% vs. 29%), all common and critical elements of EOC knowledge. None of the differences are statistically significant given the limited number of providers assessed. Less than half of the physicians in both voucher and control facilities actually conduct external bimanual compression, repair of vaginal lacerations, and none conduct bimanual exams (Annex Table 34). More voucher than control physicians’ repair third to fourth degree lacerations, conduct external and internal versions, vacuum and menstrual extractions and reflex testing. Fewer voucher than control physicians repair episiotomies and cervical lacerations and perform speculum exams. Economic Evaluation of DSF Voucher Program in Bangladesh 109 Senior Nurses As noted in the staffing section, voucher facilities had slightly fewer nurses than control facilities. Annex Table 32 indicates that more voucher facility nurses had EOC (88% vs. 50%) training than control facility nurses. There was little or no difference in nurse ENC training between voucher and control areas. Table 6.12 describes the background of the most experienced (obstetrically) nurse of each facility assessed. Senior nurses were available for interview in all of the eight voucher and eight control facilities. TABLE 6.12. CHARACTERISTICS OF MOST SENIOR NURSE Intervention Control p-value Average number of deliveries attended in past 6 months 67 18 0.20 Average salary per month (Tk.) 10,274 13,198 0.33 Happy in job 100% 100% 1.00 Plan to remain in current job 100% 88% 0.35 As do the most senior physicians, the most senior (in obstetrics) voucher facility nurses assist many more deliveries within a six-month period than their counterparts in control facilities. However, the voucher facility senior nurses receive less monthly compensation than the interviewed control facility senior nurses. They report being equally happy in their positions and slightly more voucher than control senior nurses interviewed plan to stay in the position. Senior nurse EOC knowledge was similar but generally slightly better in voucher than control facilities for all, and was generally good to fair for all indices except APH action (Annex Table 35). The senior nurses actually assist fewer deliveries than the physicians, yet their EOC knowledge seems somewhat better.As seen below, they may maintain better knowledge of certain EOC skills as they practice certain (preparation and assistance) EOC skills more frequently than physicians. Identification of APH, infection, eclampsia and PPH action and identification of dysfunctional labor and PPH was slightly better among voucher than control nurses. Voucher facility nurses had slightly poorer knowledge than control facility nurses about APH and urgent PPH action and about clinical assessment of infection. Most senior nurses have the knowledge to perform ANC risk screening, manually remove the placenta, administer IVs, check hemoglobin levels, conduct external and internal bimanual compression of the uterus, and assist Cesarean sections in both voucher and control facilities (Annex Table 35). More voucher than control senior nurses have the knowledge to use partographs (voucher 88% vs. control 50%); conduct internal bimanual compression; repair episiotomies or cervical, vaginal or third to fourth degree lacerations; or conduct menstrual extractions and speculum exams. Fewer voucher than control facility senior nurses had the knowledge to administer intravenous fluids, check hemoglobin, conduct external or internal vacuum extractions, reflex testing or bimanual exams. Sixty-seven percent (67%) and 88% of voucher and control senior nurses have the knowledge to conduct menstrual extraction and assist Cesarean deliveries, respectively. None of the differences are statistically significant given the limited number of providers assessed. Voucher facility senior nurses systematically conduct more EOC procedures than their counterparts in control facilities, particularly repair of cervical, vaginal, and third to fourth degree lacerations, and bimanual uterine compression (Annex Table 36). Few senior nurses conduct external versions, vacuum or menstrual extractions, or speculum exams. However over one-third (38%) of voucher facility senior nurses assist Cesarean delivery, compared with none in the control facilities (p=0.08). 110 Economic Evaluation of DSF Voucher Program in Bangladesh 6.1.9 KEY FINDINGS FROM THE FACILITY ASSESSMENT: CAPACITY FOR EOC AND ENC MANAGEMENT The voucher facilities assessed service a much larger volume of maternity patients than the control facilities, even though their overall bed capacity is similar. Thus the number of normal births, Cesarean and complicated deliveries is much greater at the voucher than control facilities. The voucher facilities have slightly more obstetric specialists or physician and nursing staff with EOC and/or ENC training. Although the disparity in physician knowledge is slight, voucher facilities are in general more adequately staffed to handle obstetric complications, even though more control facilities had operating theaters. Voucher facility registers and educational materials were more focused on maternity care compared with control facilities where there was greater focus on FP services. Transportation to upwards referrals was better in voucher than control facilities, and voucher facilities receive more EOC/ENC referrals than control facilities. In general, voucher facilities possess a greater proportion of MBP-recommended supplies for normal and complicated labor and delivery. Voucher facilities have slightly more MBP-recommended equipment than control facilities to manage hemorrhage and obstructed labor, which account for approximately one-third of maternal deaths and may account for as many perinatal deaths. More voucher than control facilities had more complete MBP recommended medications for eclampsia, hemorrhage and obstructed labor which jointly account for approximately half of maternal mortality. While the assessment of provider knowledge indicates some deficiencies, it is well documented that skills are better maintained with practice. In general, voucher facilities are better prepared to provide EOC and ENC services and are providing these services to significantly more maternity and neonatal patients than control facilities. Substantially more women, and thus more women with complications, are presenting (self-motivated or by referral) to voucher than control facilities and receiving timely care. Likely as a consequence of this, the number of still births, newborn and maternal deaths was higher in the voucher than control group, however the incidence of stillbirths and newborn mortality was substantially lower in the voucher than control facilities, while there is no statistically significant difference in maternal death rates. The maternal deaths observed in the voucher facilities likely reflects that more complicated cases are seen at these facilities due to the DSF program, and the difference may also be driven by the fact that fewer women in the control areas present for institutional delivery, and deaths associated with those deliveries occur at home. The observation that significantly more women are presenting to voucher facilities for delivery is consistent with and supportive of the household survey, which found institutional delivery rates in voucher areas were twice those observed in control areas, while institutional delivery rates in multiparous women prior to the voucher program were only 40% greater in voucher than control areas during that period. 6.2 HOUSEHOLD SURVEY RESULTS The household survey collected information on a few beneficiary-reported indicators of service quality, for antenatal, newborn, and postpartum care. Figure 6.6 and Annex Table 37 present information on routine checks performed during a woman’s first ANC visit during her last pregnancy. As shown, 87% of women in the control area, 82% of women in Economic Evaluation of DSF Voucher Program in Bangladesh 111 the universal area and 93% of women in the means-tested area had their blood pressure checked. The differences among groups were statistically significant. Women were significantly more likely to receive a blood test if they were in the universal or means-tested groups (37% and 35%, respectively) compared with women in the control group (25%). Twenty-six percent of women in the control group had their height measured while significantly more women in the intervention (48%) and means-tested (60%) groups did. Across the study sites, women were least likely to have an ultrasound. About 8% of women in the intervention and means-tested sites had an ultrasound while 15% of women in the control group did. FIGURE 6.6. ROUTINE CHECKS PERFORMED DURING THE FIRST ANC VISIT (%) 100 90 80 70 60 % 50 40 30 20 10 0 Height was Blood pressure Blood test taken Ultrasound taken measured checked Control Universal Means-tested Note: Results are applied to those incurring service only. Figures are shown only for those services where the difference among intervention areas was significant at p<0.05. Indicators that are not different at p<0.05 include: weight measure, urine test, abdomen exam, vaginal exam. Results for 2nd and 3rd ANC closely follow the patterns of 1st ANC. See Annex table 37 for more details. Table 6.13 below presents information on selected quality of neonatal care indicators for each study site. Women in the means-tested group were somewhat more likely to report that their baby was breastfed immediately or within 1 hour of birth (76%), while women in the control (68%) and intervention sites (69%) were equally likely to report the same outcome. The differences between the groups were non-significant. TABLE 6.13. SELECTED QUALITY INDICATORS FOR NEONATAL CARE (%) Quality of care indicators Control Universal Means- p-value tested N=1104 N=552 N=552 Breastfed baby immediately or within <1 hour of 68.1 68.6 76.3 0.288 birth Instrument was boiled before cutting cord 78.5 67.5 74.1 0.050 How long after birth was baby wiped and dried? (minutes) <10 84.5 85.7 88.9 0.591 10-20 10.7 9.4 8.2 112 Economic Evaluation of DSF Voucher Program in Bangladesh Quality of care indicators Control Universal Means- p-value tested >20 4.8 4.9 2.9 How long after birth was baby wrapped (minutes) <10 63.2 58.0 70.5 0.632 10-20 23.2 26.7 17.9 >20 13.7 15.4 11.6 Newborn was given colostrum immediately after 95.5 90.9 96.7 0.006 birth As shown in Table 6.13, from 68-79% of women in all study sites reported that the instrument for cutting the umbilical cord was boiled before use; the differences between the sites were marginally statistically significant. In addition, most women in all study sites reported that their baby was wiped and dried within the first 10 minutes of birth, and most women reported that the baby was wrapped within the first 10 minutes of birth; from 18-27% of women said the baby was wrapped within 10-20 minutes following birth. Over 90% of women in all study sites said their newborn was given colostrum immediately after birth, although women in the means-tested site (97%) were slightly more likely than those in the universal site (91%) to report the same outcome. In the household survey, women were asked whether they had any major problems with the ANC or PNC they received (Table 6.14). (Note that due to an oversight, the survey did not ask about major problems with delivery care.) About 20-23% of women reported that they had a ‘big’ problem with the time it took to wait, to complete the consultation, or to receive the test result for their first ANC visit. For all other components, women in the means-tested groups were least likely to report having a problem with any of the quality components (<10% of women in the means-tested group reported having a ‘big’ problem). TABLE 6.14. PERCENTAGE OF WOMEN WHO REPORTED HAVING “BIG PROBLEMS” WITH CARE RECEIVED DURING HER FIRST ANC VISIT (%) Have ‘big’ problem with Control Universal Means- p-value tested N=706 N=487 N=498 Times it takes to wait for the provider, complete 19.8 23.3 20.8 0.773 the service, or to receive the test result Ability to discuss 7.0 9.4 4.8 0.334 Amount of explanation 9.0 5.9 4.4 0.213 Quality of exam 10.6 4.4 6.9 0.115 Privacy 12.5 14.0 5.4 0.145 Medicine 19.1 14.5 8.8 0.258 Hours of service 3.4 5.8 5.5 0.417 Cleanliness 10.6 6.1 8.0 0.330 Treatment by staff 9.0 12.9 7.0 0.215 Cost 16.3 13.1 7.600 0.258 Note: N denotes the number of women who reporting having the first ANC checkup For PNC, women in the means-tested group were also the least likely to report having a ‘big’ problem with any of the quality indicators for postpartum care (<12% of women, Table 6.15). There was a significant difference between study sites regarding the level of satisfaction with treatment by staff, Economic Evaluation of DSF Voucher Program in Bangladesh 113 where 10% of women in the universal group said they had a ‘big’ problem with treatment by staff compared with 4% and 4% for women in control and means-tested sites, respectively. TABLE 6.15. PERCENTAGE OF WOMEN WHO REPORTED HAVING “BIG PROBLEMS” WITH CARE RECEIVED DURING HER POSTNATAL CARE ENCOUNTER (%) Have ‘big’ problem with Control Universal Means- p-value tested N=234 N=196 N=202 Times it takes to wait for the provider, complete 14.0 22.5 12.0 0.175 the service, or to receive the test result Ability to discuss 4.1 9.8 4.1 0.141 Amount of explanation 2.8 6.9 4.0 0.334 Quality of exam 3.0 10.3 5.5 0.130 Privacy 6.2 15.6 3.9 0.147 Medicine 20.5 14.4 9.9 0.349 Hours of service 1.7 5.8 2.6 0.079 Cleanliness 8.1 9.1 1.9 0.185 Treatment by staff 4.0 10.5 4.3 0.050 Cost 25.4 15.4 9.2 0.063 Note: N denotes the number of women who reported having postnatal care 6.3 QUALITATIVE RESULTS: PERCEIVED EFFECTS ON PROVIDERS Focus group discussants, key informants, and providers themselves all provided commentary on the effects of the DSF program on provider behavior, quality of care, and satisfaction. 6.3.1 BENEFICIARY PERCEPTIONS OF QUALITY OF CARE In five of the upazilas, focus group discussants indicated that they felt the voucher program had led to improved behavior on the part of doctors and nurses. Women in two upazilas felt that provider behavior had always been good and there had been no change since the inception of the program. “At present they are providing better services then before. We can conduct delivery at hospitals. The baby remains healthy.” [respondent in means-tested upazila] “The behavior of the doctors and nurses is good. They behave as before after the start of this voucher program.” [respondent in means-tested upazila] In the other three upazilas, respondents uniformly indicated that provider behavior was very negative and had not improved under the voucher program. Women reported about providers telling voucher holders to wait for extended periods, accusing them of coming just to take money, and neglecting them because they were poor: “I went to the hospital with the card. The nurse told me to come next day. I requested her for check up because I had come from very long distance. Then she took the card and threw it away and told me to go to the Government. ‘The Government will provide you proper treatment.’” [respondent in universal upazila] 114 Economic Evaluation of DSF Voucher Program in Bangladesh “When we go for treatment they tell us ‘you have come for money. Go to your house. When we get money we will inform you’. … They do not give us importance as we are poor.” [respondents in means-tested upazila] 6.3.2 KEY INFORMANT PERCEPTIONS OF CHANGES IN SERVICE QUALITY Key informants were asked to summarize whether they had noticed any change in the quality of maternal health services since the initiation of the voucher program. Most reported that service quality had improved, as demonstrated by the increased availability of medicines and equipment, better performance among providers, and women seeking care earlier in pregnancy. Many felt (anecdotally) that skilled attendance rates had increased and maternal mortality had decreased. A small number of respondents, mainly concentrated in one means-tested upazila, did not feel that quality had improved and may have actually decreased: “No improvements have been viewed in the quality of maternal health care after initiation of this program. Moreover, the quality of service has decreased. Because the time which is allocated for the program could not be given properly for the increase in work load.” – FWV 6.3.3 KEY INFORMANT PERCEPTIONS OF IMPACTS ON NON-DSF PROGRAMS We asked key informants at the upazila and central level whether they thought that voucher program responsibilities might negatively impact the routine family planning and health outreach activities of FWAs and HAs. This concern had been raised in the 2008 rapid assessment report. Almost all respondents felt that FWAs’ and HAs’ regular activities were not affected; a small number felt that FWA/HA services to beneficiaries had actually been improved by the DSF program: “As the FWAs and HAs are involved with the DSF activities, their work load has increased, but their regular activities are not hampered. The HAs and FWAs do the DSF activities … during their door-to-door visits for disseminating information on immunization and family planning.” – DSF Coordinator “Due to involvement in the DSF program, the quality of regular services (family planning) has improved.” – HA “There certainly has been improvement in acceptance rate of family planning method as we inform them about family planning during our regular services.” – FWA Less than 1 out of 8 of the key informants who responded to this question, who came from 4 of the sampled upazilas, felt that the program impacted FWA/HA services negatively: “Because of the incentives they receive under the DSF program, they are giving more importance to it. This is affecting the family planning program to some extent.” – FWV When asked whether they thought the DSF program has any negative impacts on other programs, again less than 1 out of 10 key informants expressed such concerns. The main reasons given were concerns about human resources being pulled away from FP programs to support the DSF program. Three (3) respondents worried that cash incentives were encouraging women to have children more quickly. Economic Evaluation of DSF Voucher Program in Bangladesh 115 This question was posed positively as well. Almost all key informants and providers felt that the DSF program had had positive impacts on other programs and services like FP. Nearly every respondent noted that in some way, increased contact with health sector through the DSF program has improved uptake of other services, including immunization of children, FP, and even tubal ligation at the time of C- section. Several responses mentioned that the 2-child maximum for voucher eligibility has also discouraged women from having larger families. “As they are adopting family planning between 1st and 2nd child they are getting voucher card. Some of them are doing ligation during caesarean.” – FWV “This program is greatly contributing to the immunization of the newborns.” – Civil Surgeon “During the implementation of the DSF program, it is also said that the mothers who have more than 2 children will not receive the benefits of the DSF; they are discouraged. Thus the family planning program is benefited.” – DSF Coordinator 6.3.4 PROVIDER SATISFACTION About three-quarters of the providers we interviewed said they enjoy working with the voucher program. Some of the reasons they shared were that poor patients were now receiving services; that child and maternal mortality had decreased; that family planning activities had increased; and that the reputation of the UHC had benefited. “Though we are doing hard work, we are satisfied that the poor patients are receiving services.” – Senior staff nurse “Though the amount is small, we get some money for conducting normal delivery, cesarean delivery, providing PNC services, which is extra income.” – Gynae consultant “Due to this DSF program, the number of patients at our health center has comparatively increased. Mothers are getting money and for this poverty is decreasing to some extent.” –FWV The majority of dissatisfied providers felt they were not being adequately compensated, given the amount of work. A few individuals mentioned that they did not enjoy receiving criticism and pressure from ineligible women and local politicians. A lack of accountability and a lack of logistical support were also mentioned. “The incentive is not compatible to the labor given. There is a need to increase the incentive. – Senior staff nurse “Sometimes we are subject to criticism by those who are not given voucher as they are not eligible. They think we are not giving them intentionally.” – FWA When asked specifically whether they were satisfied with the incentives they received, only one-third of providers said yes. 116 Economic Evaluation of DSF Voucher Program in Bangladesh 6.4 SUMMARY OF KEY FINDINGS Maternal health patient volumes and outcomes Overall, UHCs in voucher areas are seeing a dramatically and significantly higher number of patients for voucher-covered services than UHCs in control areas. This includes ANC visits, deliveries, and C-sections. The incidence of complicated deliveries, stillbirths, and newborn deaths was lower in voucher UHCs than in control UHCs, likely a consequence of more maternal care in voucher UHCs. The incidence of maternal deaths was higher in voucher UHCs but the difference was not statistically significant. Physicians working in voucher facilities are conducting many more deliveries within a six month period than their control facility counterparts. The most senior (in obstetrics) voucher facility nurses assist many more deliveries within a six-month period than their counterparts in control facilities. Adequacy of staffing Voucher facilities had more ob/gyns and anesthesiologists than control facilities, but fewer nurses, physicians, lab technicians, medical officers, and FWVs. Almost all (88%) voucher facilities had personnel qualified to conduct Caesarean sections and transfusions whereas only 57% could in control facilities (p=0.10). In general, voucher facilities are more adequately staffed to handle abortion complications, ANC, eclampsia, hemorrhage, normal labor and delivery, obstructed labor, postpartum care, sepsis and STDs than control facilities, although none of the differences are statistically significant. More voucher facility physicians had Emergency Obstetric Care (EOC) and Essential Newborn Care (ENC) training than control facility physicians. More voucher facility nurses had EOC (88% vs. 50%) training than control facility nurses. There was little or no difference in nurse ENC training between voucher and control areas. Supplies, equipment, and medicines In general, voucher facilities possess a greater proportion of MBP-recommended supplies for normal and complicated labor and delivery. Voucher facilities have slightly more MBP-recommended equipment than control facilities to manage hemorrhage and obstructed labor, which account for approximately one-third of maternal deaths and may account for as many perinatal deaths. More voucher than control facilities had more complete MBP recommended medications for eclampsia, hemorrhage and obstructed labor which jointly account for approximately half of maternal mortality. Referrals While the time to reach the referral facility is similar, voucher facilities take an average of 33 minutes to prepare and transfer the patient compared with 53 minutes in control facilities. Economic Evaluation of DSF Voucher Program in Bangladesh 117 Effects on family planning activities Most key informants felt that the DSF program had not negatively impacted FWAs’ and HAs’ performance of family planning activities, and some felt that family planning efforts were improved by the voucher program due to increased contacts with the health system. Provider satisfaction About three-quarters of providers indicated that they enjoy working with the voucher program. The main complaints relate to inadequate compensation given the workload. 118 Economic Evaluation of DSF Voucher Program in Bangladesh 7. DISCUSSION AND RECOMMENDATIONS This report has summarized the results of an extensive evaluation of Bangladesh’s Demand-Side Financing (DSF) voucher program. The evaluation used multiple data collection and analytic methods to obtain a comprehensive, in-depth understanding of the effects of this program on the use of maternal health services, as well as how they are provided. Qualitative and quantitative data shed light on what has worked well, and where challenges remain, in the operations, implementation, and impact of the DSF program. In this section we discuss the key findings that have emerged from this evaluation, and present recommendations based upon those findings. 7.1 EFFECTS OF THE DSF PROGRAM ON MATERNAL HEALTH SERVICE UTILIZATION AND EXPENDITURES The overall conclusion we draw from these analyses is that the DSF program has had an unprecedented positive effect on the utilization of maternal health services in the short time since its initiation. Data from all sources consistently corroborate this conclusion: women in DSF areas are much more likely to deliver with a qualified provider than women in control areas, and this result is likely attributable to the DSF program. Upazila Health Complexes in voucher areas see significantly more maternal health patients (for ANC, normal delivery and PNC) than UHCs in matched control areas. Moreover, out-of- pocket expenditures on maternal health services – antenatal, delivery and postnatal care – are lower for women in DSF areas than for women in control areas. The program has increased the likelihood that women seek antenatal, delivery, and postnatal care. The likelihood of delivering with a qualified provider is more than twice as high in intervention areas (64%) as in the control upazilas (27%). Home deliveries with a qualified provider are more common in DSF areas, particularly in means-tested areas. The DSF program is also strongly and significantly associated with higher rates of institutional deliveries. Women in DSF areas are twice as likely to deliver in a health facility (38%) as women in control areas (19%). The difference in rates of institutional deliveries is also associated with substantially lower rates of stillbirths and newborn deaths in voucher facilities – approximately half that of control facilities. While a much greater proportion of complicated deliveries are attended by qualified providers in intervention than in control areas, the effect of the DSF program on surgical deliveries is less straightforward. There is no significant difference in rates of C-section between intervention (10% of deliveries) and control upazilas (9%). These figures are close to the national average (8%). The lack of a dramatic increase in C-section rates may be attributable to close government oversight of this indicator, given fears that the substantial incentive payments to providers (Tk. 3,000 per C-section) might induce providers to encourage unnecessary surgical deliveries. Notably, according to our facility assessment in 16 upazilas, almost all facilities in DSF areas had personnel qualified to conduct C-sections and blood transfusions, whereas only 57% of control facilities had the necessary personnel. Economic Evaluation of DSF Voucher Program in Bangladesh 119 The DSF program seems to be reducing poverty-related inequalities in maternal health service use and expenditures, as well as improving these indicators in absolute terms. The gap between the poorest (quintile 1) and those in quintiles 2-5 in the likelihood of delivery with a qualified provider was smaller in DSF areas than in control areas, as was the relative wealth gap in out-of-pocket expenditures on the last pregnancy and birth. The poorest women are “catching up” to non-poor women in terms of their likelihood to seek qualified delivery assistance, while the expenditures of non-poor women have decreased more than the expenditures of the poor. Recommendations: The effects of the DSF program on maternal health service utilization are unprecedented in magnitude, given the short duration of the program. Given how rapidly and extensively the provision of safe motherhood services improved in this pilot, and given the average voucher program cost of $41 per voucher distributed, we recommend expanding this program (with some modifications as discussed below) to other areas in Bangladesh. Scaling up the voucher program will greatly increase Bangladesh’s chances of meeting Millennium Development Goal 5 (improve maternal health) and its first target (reduce maternal mortality by two-thirds by 2015). 7.2 EFFECTS ON SERVICE PROVISION Overall, UHCs in DSF program areas are seeing a dramatically and significantly higher number of patients for voucher-covered services than UHCs in control areas. The percent of deliveries with complications is lower in voucher facilities, implying that women may be presenting to facilities earlier now that financial barriers to safe delivery care have been mitigated. There is no statistically significant difference in maternal death rates between voucher facilities and control area facilities, At the same time, the facility-based incidence of stillbirths is significantly lower and the incidence of newborn deaths is substantially lower in DSF program UHCs than in control UHCs. These indicators and others imply that the quality of care in UHCs in voucher upazilas may be slightly better than the quality in control upazilas, although we cannot attribute these differences solely to the voucher program given that more voucher upazila UHCs have received EOC upgrades than control upazila UHCs. The greatest contributor to improved quality in voucher areas may simply be the fact that providers are getting more practice providing obstetric care and handling complications, which is essential to skills retention – physicians and nurses working in voucher facilities are conducting many more deliveries at UHCs than their control facility counterparts. In our sample, more voucher facility physicians and nurses had Emergency Obstetric Care (EOC) training than control facility physicians and nurses, but knowledge measures were similar. In general, the voucher facilities in our sample possessed a greater proportion of recommended supplies for normal and complicated labor and delivery. Voucher facilities had slightly more recommended equipment than control facilities to manage hemorrhage and obstructed labor, which account for approximately one-third of maternal deaths and may account for as many perinatal deaths. More voucher than control facilities had more complete medications for eclampsia, hemorrhage and obstructed labor, which jointly account for approximately half of maternal mortality. Although the sample included 8 of the 21 voucher upazila health facilities and 8 matched controls, due to the limited number of facilities most differences were not statistically significant. Recommendations: While the quality of care may be slightly better in DSF facilities, quality improvements are still very 120 Economic Evaluation of DSF Voucher Program in Bangladesh much needed throughout the government health care system. Our analysis indicated that the presence of an EOC-upgraded facility in an upazila independently increased the likelihood of women having a facility-based birth, even without the DSF program. Continued investment in EOC upgrades in other facilities is thus recommended. Currently, the DSF program does not have a standard quality checklist for participating facilities. Such a checklist should be developed for different types of public, private and NGO facilities, and regular, standardized quality monitoring should be implemented. Since the utilization of “seed fund” money is inconsistent across upazilas, we recommend greater efforts to encourage facilities to use the seed fund for making quality improvements as well as to procure drugs, supplies, labor beds, and equipment. Facilities may need better guidance on how to make these improvements; technical assistance on effective use of the seed fund should be provided. 7.3 VOUCHER DISTRIBUTION AND TARGETING Our analysis indicated that general awareness of the voucher program is high (more than 85% of women in our household survey sample in the DSF areas had heard of the program) and voucher distribution rates are also high (more than 70% of women in intervention areas had received a voucher). It appears that the means-tested upazilas are doing a somewhat better job of informing women about the program and distributing vouchers; the means-tested upazilas have been more likely to meet their voucher distribution targets thus far, and women in those areas seem to have more complete information about the program and its benefits. Some topics need to be better publicized – for instance, less than half of the women surveyed in the means-tested areas, and less than one in five in the universal areas, reportedly were told that the voucher could pay for home delivery with a qualified provider. Voucher booklets are being given to women quickly after they have been identified as eligible, but not typically during the first trimester; in means-tested areas, vouchers are getting to most women when they are 4 or 5 months pregnant, and in universal areas, when they are 5 to 7 months pregnant. A number of key informants, focus group discussants, and providers expressed concerns about corruption in the voucher distribution process, noting that women with political connections were more likely to receive them. Voucher program eligibility criteria – including parity and contraceptive use in both intervention areas, and poverty-related targeting in means-tested areas – are not being strictly applied. The main targeting challenge appears to be leakage (non-eligible women receiving vouchers) rather than undercoverage (eligible women not receiving vouchers). For instance, we found that 13% of voucher recipients in universal areas and 9% in means-tested areas were parity 3 or greater. Vouchers were distributed to approximately equal proportions of parity-2 contraceptive users and non-users. In means-tested upazilas, women from poorer quintiles were significantly more likely to receive vouchers than women from wealthier quintiles – but there was substantial leakage to women who did not meet land ownership criteria (18% of voucher recipients), asset ownership criteria (75% of voucher recipients) and income criteria (91% of voucher recipients). Part of the reason for this leakage may stem from the fact that some poverty eligibility criteria are very narrow. We found that most women in means-tested upazilas would meet the land ownership criterion (79% are functionally landless), but less than one-quarter own no productive assets, and only 7% reported household expenditure less than Tk. 2,500 per month, suggesting that the current household income criteria may be set particularly low. Many key informants, both at the local and national level, felt that income and land ownership criteria were too stringent in means-tested areas and should be relaxed. Economic Evaluation of DSF Voucher Program in Bangladesh 121 Given that assignment of upazilas to be “means-tested” or “universal” DSF areas was not determined randomly, this evaluation does not permit us to draw conclusions about whether one approach is preferable. The voucher program seems to be affecting delivery patterns differently in universal and means-tested areas. However, since poverty-related targeting was not stringently enforced in means- tested areas (and since the characteristics of voucher recipients were similar in both means-tested and universal areas), we can conclude that targeting does not account for the differences in DSF program impacts. Recommendations: Voucher program “marketing” should be improved, especially in universal areas. While there is good overall awareness of the program, information and behavior change communication campaigns should be launched to improve understanding of specific DSF benefits, as well as general safe motherhood messages. Areas where awareness could be strengthened include the fact that the program covers home delivery with a qualified provider and that women are entitled to transport stipends for all voucher-covered services. Policymakers should consider whether to revise eligibility criteria or emphasize stricter adherence to existing criteria. Parity targeting is likely easiest to enforce should the government wish to do so, though we are concerned that higher parity women are typically those least likely to deliver with a skilled birth attendant and equally (if not more) likely to need emergency obstetric care than lower parity women. Targeting according to contraceptive use between pregnancies is nearly impossible to enforce, as the condition is difficult to verify, and it is unclear that promoting this criterion is likely to affect women’s family planning practices. For this, a targeted family planning strategy is recommended – see section 7.8 below. The current poverty eligibility criteria, if fully applied, would limit DSF program eligibility in means- tested areas to a very small number of women. Policymakers may wish to consider relaxing the asset ownership and income criteria in particular. We recommend further evaluation (if possible, involving randomization of new DSF areas) of the advantages and disadvantages of means-tested versus universal voucher eligibility. Since little poverty-related targeting is occurring and therefore targeting does not account for differences in rates of delivery with qualified providers or institutional delivery, we are unable to recommend one model over the other. We note that poverty-related targeting is difficult and costly, prone to leakage, and subject to potential corruption. However, expanding universal voucher provision will be costly as well. Further evidence is required. Whether or not eligibility criteria are revised, we recommend that greater emphasis be placed on standardized, transparent application of those criteria. This will likely require better guidance to health workers and better monitoring, perhaps including random checks of a small percentage of voucher recipients to verify eligibility. Incentives could be allocated to health workers who achieve high marks on targeting (both for including all eligible women and for preventing leakage to ineligible women). Without such monitoring, corruption in the allocation of vouchers and incentives will always be of concern. 122 Economic Evaluation of DSF Voucher Program in Bangladesh 7.4 FUNDS DISBURSEMENT AND FINANCIAL MANAGEMENT Delays in the disbursement of DSF program funds, both from the national level to the upazilas and from the upazilas to beneficiaries or providers, are a serious operational challenge for the DSF program. Delays reduce provider willingness to proactively identify voucher recipients and provide them with maternal health services. They also may reduce the credibility of the program in the eyes of beneficiaries over time. Transport stipends, post-delivery cash incentives and gift boxes have not been distributed to women at the time of service in most areas – primarily because of the aforementioned problems with funds disbursement and financial management at UHCs. This has meant that women must make a special trip back to the health facility to collect their payment on a particular date, diminishing the intended health and poverty impacts of these payments. Moreover, the requirement that unused monies be returned to the Treasury at the end of the fiscal year presents an additional, serious disruption to program operations and impact. In general, financial management of the program at the upazila level has been burdensome for UHFPOs and RMOs, who do not receive any supplemental incentive payments for their administrative responsibilities; although this is currently in the process of changing. Recommendations: Continuous availability of funds at the upazila level is essential for ensuring this program’s success. We recommend that if at all possible, remaining money at the end of the fiscal year be allowed to stay at the upazila level and not be returned to the Treasury. This will require a major policy change, but is essential for the smooth running of the program. In addition, advance funds should be allocated to cover 2-3 months at the beginning of each fiscal year, when there is a delay in receiving funds. A standard protocol should be developed regarding timing and place of beneficiaries’ payment, and voucher recipients should be informed. Before the program is scaled up, the logistics of making transportation stipends and cash incentives available to women at the time they receive a service should be resolved. More broadly, we recommend that the currently small DSF program office be expanded to serve as a full-fledged Voucher Management Agency. Several full-time technical staff will be needed to ensure smooth operations, track finances, monitor adherence to policy and program results, and report to the MOHFW. A Deputy Program Manager from DGHS can chair the DSF voucher management unit; however a full-time manager should be responsible for overseeing all activities. The role of the voucher management unit may be outsourced as it is in done in some other countries, but this should be carefully considered in the Bangladeshi context. 7.5 INPUT SHORTAGES According to this evaluation, the other main operational challenge faced by the voucher program at the upazila level has been input shortages – primarily human resources, but also material (drugs, supplies, and equipment). National and upazila-level key informants, providers, and individual women expressed concerns about staff shortages, which are leading to long lines at health facilities and overloaded providers. Shortages of supplies and medicines are also an obstacle to smooth DSF program implementation. Many voucher recipients are still paying out-of-pocket for medicines outside of government health facilities, due to stock-outs. The seed funds may be helping address these shortages, but they have not eliminated the problem; cash incentives may help cover these out-of-pocket costs, but Economic Evaluation of DSF Voucher Program in Bangladesh 123 they are not provided at the time of service use and therefore do not eliminate the financial barrier. Overall, supply-side limitations still may constrain the long-term sustainability of the program. Recommendations: Since Bangladesh faces a critical shortage of human resources in health, particularly in rural and remote areas, particular emphasis should be placed on ensuring staff availability at DSF facilities. This may require implementation of a human resources incentives program, such as was detailed by Luoma et al. (2010). The program will lose its momentum in the absence of qualified providers at the facility. We recommend further advocacy to include private and NGO facilities (that meet quality standards) as this may increase the available supply of services for voucher beneficiaries, as well as potentially improve quality through competition. 7.6 MONITORING AND EVALUATION The DSF program’s monitoring and evaluation process should be strengthened. Aside from external monitoring by DSF coordinators who are sponsored by the WHO and DFID, the current M&E system is weak and not integrated with routine management information systems in health facilities. Key informants recommended that the MOHFW become more involved in routine supervision and monitoring of the program, and that specific individuals (at the central, upazila and union levels) be tasked with and paid for these responsibilities. Recommendations: Monitoring protocols should be designed focusing on the following areas: ensuring that beneficiary eligibility criteria are met tracking all services received by a given beneficiary, perhaps using a scannable bar code system with a unique ID for each voucher booklet, which will enrich program data and help inform program design tracking financial disbursements conducting regular as well as random audits of facility service provision and quality. All data should be maintained electronically. Efforts should be made immediately to integrate DSF program monitoring with the existing management information systems of DGHS and DGFP. 7.7 PROGRAM BENEFITS AND INCENTIVES Several program design-related suggestions were made by respondents. Some felt that cash payments to women should be increased; others felt that the gift box should be eliminated, or at least revised to be more useful to the new mothers. Some upazila-level key informants and providers suggested providing incentive payments to UHFPOs and RMOs, to support their administrative and management responsibilities under the DSF program (a change which is already underway). Field health workers requested that incentives for registering eligible voucher recipients be increased. 124 Economic Evaluation of DSF Voucher Program in Bangladesh Recommendations: While the gift box is popular among beneficiaries and politicians, procuring and distributing the gift box has been difficult; mothers often receive the gift box many months after delivery, if at all. As suggested by many key informants, our recommendation is to discontinue the gift box incentive. The 500 Taka allocated for the gift box could be used in several ways: To provide an additional incentive for the mother to seek PNC, including family planning counseling and a free supply of contraceptives (for some initial months); To provide an additional incentive for health providers to include neonatal care and family planning counseling as part of PNC; and To provide additional funding for the voucher management unit, including monitoring activities. The services covered by the DSF program should be expanded to explicitly include family planning counseling and essential neonatal care. UHFPOs and RMOs are faced with additional voucher management work for which they are not reimbursed; incentives for these staff are currently awaiting final government order. Our recommendation above for establishing a Voucher Management Agency and hiring additional full- time staff for voucher management will negate the need to increase incentives to providers for voucher administration and related activities. On the other hand, incentives for service provision should remain. For registration of eligible women, incentives to health workers should be increased from the currently very low 10 Taka. 7.8 IMPACT ON OTHER PROGRAMS The 2008 rapid assessment of the DSF program raised concerns about its potential negative impact on family planning outreach activities and family sizes. Our evaluation does not lend credence to these concerns. Most key informants and providers interviewed felt, by contrast, that the program was having a beneficial impact on family planning efforts, simply because more women were having more contacts with FWAs and the government health system in general. Moreover, the stated exclusion from benefits of women of parity 3 or higher was cited as an additional disincentive to having additional children. Recommendations: As alluded to above, we recommend adding a family planning page in the DSF voucher booklet to ensure that a provider signs off on providing family counseling after delivery. Such counseling should already be part of PNC services, but a separate voucher page will encourage delivery of this service. The existing incentive payment for PNC could be split between the family planning voucher and postnatal care for the mother and neonate. 7.9 NEXT STEPS FOR EVALUATION AND SCALE-UP Plans are already underway to expand the DSF program into additional upazilas. It is important to emphasize that results observed in these 21 DSF upazilas may not generalizable elsewhere, for a variety Economic Evaluation of DSF Voucher Program in Bangladesh 125 of reasons (for instance, intervention upazilas were selected non-randomly, and may have received special program attention that will not be replicated during scale-up). We suggest that a revised evaluation design be adopted in the new upazilas. If at all possible, a randomized trial design should be used, where more than one design feature can be “tested” (for example, with varying scales of incentive payments). Some intervention upazilas can be selected from the control upazilas used in this household survey since that will ensure baseline data (from this survey) is available for future evaluation. Where new upazilas are selected, we strongly recommend that a matched control upazila is identified simultaneously and baseline data collected before program initiation, so that a future evaluation with pre/post data is possible. If such a rigorous randomized evaluation of the next phase of the DSF program yields strong results, we would recommend that the program be seriously considered for scaling up nationally using a phased approach. However, as mentioned above, systematic parallel investment in the supply-side will be needed to meet increased demand and ensure improved maternal and neo-natal health outcomes. In this evaluation effort, we collected, analyzed, cross-checked, and interpreted a truly vast quantity of information about multiple dimensions of a very complex program in a very short time. The information we collected has been remarkably consistent and the results generally clear and straightforward – the DSF program seems to be making an enormous difference, but with some operational challenges. Nonetheless, given the constrained timeframe in which we operated, and the lack of an available baseline survey, it would be appropriate to implement additional ongoing survey data collection to continue to monitor and evaluate the DSF program over time. Including voucher-related questions in the upcoming maternal mortality survey, with oversampling of voucher areas, is one possibility. 126 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX A: VILLAGES VISITED FOR PHASE 1 HOUSEHOLD SURVEY Intervention Area Control Area Division District Upazila Union Village Upazila Union Village Saidkati Uttar Rahmatpur Saidkati Rahmatpur Auar Khudrakati Bahirchar Uttarkul Barisal Banaripara Babuganj Ghoshkati Baisari Dehergati Idilkati Kachua Uttar Dehergati Kajalhar Kalikapur Banaripara Chandpasha Alta Bakshichar Itbaria Kismat Haridevpur Tiakhali Golkhali Barisal Badurtali Haridevpur Char Niamatpur Gramarodon Salimpur Ratandi Nilganj Islampur Taltali Nij Haola Nabiganj Pathuakhali Kalapara Galachipa Chounga Pasha Uttar Charkhali Purba Maddhya Charkhali Chakamaya Chakamaya Chakamaiya Galachipa Dakshin Charkhali Nishanbaria Gamurbunia Chittagong Comilla Daudkandi Chandina Chittagong Mirsharai Sitakunda Palla Kesharpar Mohammadpur Kesharpar Naya para Birkot Batakandi Badalkut Noakhali Chatkhil Badalkut Senbagh Arjuntala Nazirnagar Megha Maniknpur Karihat Kadra Nayakhola Purba Kadra Magua Sreenagar Uttar Cox's Bazar Ramu Fatekharkul Chakaria Chiringa Fatekharkul Kahariaguna Banikpara Palakata Economic Evaluation of DSF Voucher Program in Bangladesh 127 Chalaipara Gunarpara Maizpara Juaria Nala Naderpara Dulahazara Shikderpara Bairagir Khil Bora Kahatiapara Chhairakhali Rashid Nagar Fasiakhali Kademerpara Bendibazar (Mohammadia Khaniarguna para) Dhaka Narsingdi Raipura Belabo Jamalpur Sarishabari Melandha Deobari Parki Gazaria Gazaria Birbasunda Pratima Banki Kasturipara Bara Chowna Char Durgapur Tangail Sakhipur Kalia Kalihati Durgapur Aripara Char Jokar Mahnandapur Balla Kakrajan Balla Bhuaid Rampur Bhanga Ward no. Kapuria Sadardi Shibpur Boalmari 2 Kaidubisadardi Dholna Panchkul Poail Chandra Maligram Chatul Faridpur Bhanga Boalmari Biswaspur Satair Chumordi Chumordi Satair Kamarhati Purba Sadardi Premtara Rauti Madan Rauti Gundhar Shibpur Khairat Dakshin Sutar Para Sekandarnagar Dhala Sutarpara Kishoreganj Tarail Uttar Karimganj Khagsiri Sekandarnagar Banduldia Jahirabad Talganga Taljanga Bara Gharia Hasanpur Char Taljanga Manikganj Harirampur Kanta Para Shibalaya Nihanda Bahalatali Balara Uthali Danestapur Isail Kuchtara Balla Machain Shibalaya Basta Shibalaya Kala Para Jagatber Boali para 128 Economic Evaluation of DSF Voucher Program in Bangladesh Naodubi Thana Jatrapur Mahadebpur Boyra Mahadebpur Andhar Manik Barangail Patibila Panzia Niamatpur Manohar Nagar Patibila Ichhapur Panjia Panchnamia Kamlapur (Panchanmala) Jessore Chaugachhia Keshabpur Phulsara Sagardari Phulsara Chanda Afra Sagardari Kumarpur Balida Para Sekhpura Tilakpur Baliadanga Swarupdaha Keshabpur Chotarhuda Maguradanga Shorol Haridasbati Godaipur Barasat Hitampur Barasat Khulna Sonatankati Pantita Terokhada Khulna Paikgachha Haridhali Mamudkati Terokhada Joy Sena Kharbaria Ajugara Nasirpur Ananda Nagar Kapilmuni Howli Kalyanpur Hazrahati Hogalbaria Poradaha Shashidharpur Tegharia Kaipal Lakshimidhardia Mathurapur Chithulia Kushtia Dalutpur Mathurapur Mirpur Dhubail Puratan Nawda Sehala Boalia Sardarpur Azampur Dil-boalia Kakuliadaha Rajshahi Joypurhat Khetlal Panchbibi Bherbheri Bhavki Bherbheri Bhognagar Hossainpur Bijoypur Dangapara Balarampur Dinajpur Khanshama Khamarpara Birganj Nijpara Bhandardaha Damaik Shetra Subarnakhuli Bhogdoma Angarpara Paltapur Angarpara Ghoraband Potajia Mirer Deulmura Potajia Pangashi Rautara Pangashi Porjana Chandaikona Sirajganj Shahjadpur Porjana Royganj Chandaikona Ranikola Baoikola Shayestabad Jhaul Kayempur Dhubil Chowdhury Saratail Ghughat Kalupur Nawabganj Shibganj Durlavhpur Dakkhin Gomastapur Rohanpur Stationpara para Economic Evaluation of DSF Voucher Program in Bangladesh 129 Gangarampur Nungola Bankul Hozrapur Manakosa Gomastapur Dadanchak Khoyrabad Dhobra Brajanathpur Shahbajpur Bhangabaria Pardilalpur Daripata Bishubari Hamindapur Darbast Jamalpur Boglagari Gayespur Chandipur Isabpur Gaibandha Gobindganj Taluk Kanupur Sadullapur Faridpur Samoshpara Dari Jamalpur Puntair Jamudanga Mahimaganj Damodarpur Sreepatipur Maruadaha Jotukarnapara Bamai Purbapara Dakshin Bamai Bamoi Paschim PurbaBaniyachong Soydatula para Uttar Paschim Senpara Lakhai Sylhet Habiganj Baniachong Lakhai Lakhai Baniyachong Minhat Swajangaon Sharifkhani Uttar Muryauk Dakshin Paschim Muriauk Dakhin Paschim Baniyachong Prothomrekh Muryauk 130 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX B: VILLAGES VISITED FOR PHASE 2 HOUSEHOLD SURVEY Intervention Area Control Area Division District Upazila Union Village Upazila Union Village Saidkathi Auar Rahmatpur Uttar Rahmatpur Barisal Banaripara Banaripara Alta Babuganj Dehergati Idilkati Tiakhali Itbaria Barisal Patuakhali Kalapara Chakamaiya Chounga pasha Fatehkharkul Uttar Fatekharkul Cox's Bazar Ramu Rashidnagar Bora Kahatiapara Mohammadpur Palla Arjuntala Nazirnagar Chittagong Noakhali Chatkhil Badalkut Magha Senbagh Kadra Magua Bhanga ward number 2 Kaidubisadardi Faridpur Bhanga Chandra Panchkul Dakshin Dhala Sekandarnagar Gunodhar Madan Kishoreganj Tarail Taljanga Taljanga Karimganj Baragharia Jahirabad Barala Kanta para Manikganj Harirampur Balla Machain Dhaka Tangail Sakhipur Kakrazan Mahnandapur Kalihati Durgapur Char durgapur Patibila Patibila Keshabpur Sagardari Sekhpura Jessore Chaugachhia Swarupdaha Chotarhuda Dalutpur Boalia Sehala Poradaha Tegharia Kushtia Mirpur Chithulia Dhubail Haridhali Mamudkati Khulna Khulna Paikgachha Kopilmuni Nasirpur Khamarpara Bhandardaha Dinajpur Khanshama Angarpara Subarnakhuli Taluk Kanupur Chandipur Sadullapur Faridpur Dari Jamalpur Gaibandha Gobindganj Darbast Bishubari Nawabganj Shibganj Manakosa Dadanchak Porjana Porjana Raiganj Dhubil Jhaul Rajshahi Sirajganj Shahjadpur Kayempur Saratail Dakshin Purba Sylhet Habiganj Baniachong Baniachong Jotukarnapara Economic Evaluation of DSF Voucher Program in Bangladesh 131 ANNEX C: ADDITIONAL TABLES ANNEX TABLE 1. SOURCE OF INFORMATION ABOUT VOUCHER PROGRAM, BY INTERVENTION TYPE (%) Source of information Universal Means-tested p-value N= 498 N =507 FWA 27.2 10.2 0.015 Health Assistant (HA) 38.3 25.2 0.198 Relatives 24.0 20.0 0.433 CSBA 19.0 44.4 0.026 Others 17.2 22.7 0.234 Notes: Denominator is women who reporting having heard about the voucher program. Percentages do not add up to 100% because multiple answers are allowed. “Others” include Dai, TBA, FWV, unqualified doctor, community nutrition worker, NGO worker, teacher, qualified doctor, nurse, and midwife. The differences between universal and means-tested are statistically significant at p<0.05 for FWA and CSBA. ANNEX TABLE 2. PERCENTAGE OF WOMEN WHO RECEIVED VOUCHERS, BY INTERVENTION TYPE (%) Beneficiary Universal Means-tested p-value identification N =552 N =552 Received voucher 70.5 72.4 0.803 Received voucher by parity 1-2 80.4 85.0 3+ 39.1 29.8 1 Received voucher by use of contraceptive between 1st and 2nd births Yes 83.1 84.4 No 84.3 81.3 Note: Samples for contraceptive use indicators include only women whose birth in question was the second birth. The samples are 117 women who reported not using contraceptive and 281 women who reported using contraceptives between the first and second births. Economic Evaluation of DSF Voucher Program in Bangladesh 133 ANNEX TABLE 3. PERCENTAGE OF WOMEN WHO RECEIVED VOUCHERS BY WEALTH QUINTILE AND INTERVENTION TYPE (%) Intervention Control p-value Personnel on site can take care of all obstetric 75% 75% 1.00 emergencies, through management or referral Facility has someone qualified to conduct C-sections 88% 50% 0.10 Facility has someone qualified to provide blood 88% 50% 0.10 transfusions Maternity services available all the time (incl 88% 75% 0.52 nights/weekends) Staff member available at night 100% 75% 0.13 Staff member available on site at night 0% 13% NA Note: the difference across quintiles in the universal area is not statistically significant while that in the means-tested area is significant at p=0.009. ANNEX TABLE 4. NUMBER OF DAYS AFTER LISTING WHEN BENEFICIARIES RECEIVED VOUCHER (%) Days received voucher after listing Universal Means-tested N =420 N= 427 <10 41.6 45.7 10-20 18.7 11.3 20-30 14.9 21.6 30-60 10.2 10.6 60-90 9.5 6.2 90+ 5.2 4.7 Note: The denominator N equals the number of women who received vouchers ANNEX TABLE 5. MAIN REPORTED VOUCHER DISTRIBUTORS (%) Sources of voucher distribution Universal Means-tested p-value N= 420 N=427 0.067 FWA 22.0 9.8 Health Assistant (HA) 33.1 34.1 CBSA 29.3 47.8 Others 15.6 8.3 Note: The denominator N equals the number of women who received vouchers. “Others” includes Dai, TBA, FWV, unqualified doctor, relative, community nutrition worker, NGO worker, teacher, qualified doctor, nurse, and midwife. 134 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 6. ANC UTILIZATION AMONG ALL WOMEN BY INTERVENTION TYPE Utilization of antenatal care Control Universal Means-tested p-value N =1104 N= 552 N= 552 Had any ANC visit 75.6 91.8 91.2 <0.001 Had at least 3 ANC check-ups 33.6 54.0 55.6 0.003 First ANC was within first 5 months of pregnancy 43.2 38.4 57.8 0.017 ANNEX TABLE 6B. ANC UTILIZATION AMONG ALL WOMEN, CONTROL VS. INTERVENTION (%) Utilization of antenatal care Control Intervention p-value N=1104 N=1104 Had any ANC visit 75.6 91.6 <0.001 Had at least 3 ANC check-ups 33.6 54.7 <0.001 First ANC was within first 5 months of pregnancy 43.2 47.4 0.402 ANNEX TABLE 7. PERCENT OF WOMEN WHO RECEIVED ANTENATAL CARE FROM A QUALIFIED PROVIDER BY INTERVENTION TYPE(%) Provider qualification for ANC Control Universal Means-tested p-value Provider was qualified at first ANC visit 64.9 (783) 75.7 (517) 90.7 (515) 0.001 Provider was qualified at first ANC visit 64.9 (783) 75.7 (517) 90.7 (515) 0.001 Provider was qualified at second ANC visit 65.2 (556) 72.3 (380) 92.7 (419) 0.001 Provider was qualified at third ANC visit 70.9 (353) 75.6 (336) 95.4 (321) 0.011 Note: Values in parentheses are the total number of eligible women who reported using each ANC service ANNEX TABLE 7B. PERCENT OF WOMEN WHO RECEIVED ANTENATAL CARE FROM A QUALIFIED PROVIDER, CONTROL VS. INTERVENTION (%) Provider qualification for ANC Control Intervention p-value Provider was qualified at first ANC visit 64.9 (783) 82.6 (1032) 0.006 Provider was qualified at second ANC visit 65.2 (556) 81.8 (911) 0.025 Provider was qualified at third ANC visit 70.9 (353) 85.0 (657) 0.097 Note: Values in parentheses are the total number of eligible women who reported using each ANC service Economic Evaluation of DSF Voucher Program in Bangladesh 135 ANNEX TABLE 8. DELIVERY CARE FOR BIRTHS TAKING PLACE IN THE 6 MONTHS PRECEDING THE SURVEY Delivery care for last birth Control Universal Means-tested p-value Provider at last birth was 27.1 (1104) 58.1 (552) 70.2 (552) <0.001 qualified Provider at last complicated 47.8 (218) 87.0 (111) 82.4 (113) <0.001 birth was qualified Birth took place in a facility 18.7 (1099) 43.9 (551) 30.1 (551) <0.001 Had C-section for last birth 9.1 (1104) 12.9 (552) 7.6 (552) 0.039 ANNEX TABLE 8B. DELIVERY CARE FOR BIRTHS TAKING PLACE IN THE 6 MONTHS PRECEDING THE SURVEY, CONTROL VS. INTERVENTION (%) Delivery care for last birth Control Intervention p-value Provider at last birth was qualified 27.1 (1104) 63.7 (1104) <0.001 Provider at last complicated birth was qualified 18.7 (218) 37.5 (224) <0.001 Last birth took place in a facility 25.2 (1099) 35.7 (1102) 0.417 Had C-section for last birth 9.1 (1104) 10.4 (1104) 0.443 ANNEX TABLE 9. SKILL LEVEL OF BIRTH ATTENDANT FOR HOME DELIVERIES, BY INTERVENTION TYPE (%) Skill level of birth Control Universal Means-tested p-value attendant for home delivery N = 879 N = 271 N = 392 <0.001 CSBA 3.1 25.8 54.3 Unqualified 90.0 72.8 42.3 Other qualified 7.0 1.4 3.4 Note: ‘Other qualified’ includes qualified Medically Trained Providers (MTPs) other than Community Skilled Birth Attendants (CSBAs), i.e. qualified doctor, qualified nurse or midwife, paramedic, and family welfare visitor. ANNEX TABLE 9B. SKILL LEVEL OF BIRTH ATTENDANT FOR HOME DELIVERIES, CONTROL VS. INTERVENTION (%) Skill level of birth attendant Control Intervention p-value for home delivery N=879 N=663 <0.001 CSBA 3.1 40.6 Unqualified 90.0 57.0 Other qualified 7.0 2.4 Note: ‘Other qualified’ includes qualified Medically Trained Providers (MTPs) other than Community Skilled Birth Attendants (CSBAs), i.e. qualified doctor, qualified nurse or midwife, paramedic, and family welfare visitor. 136 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 10. SUMMARY OF KEY MATERNAL HEALTH SERVICE UTILIZATION INDICATORS (%) Maternal health service utilization Control Intervention p-value ANC care Had any ANC visit 75.6 91.6 <0.001 Had at least 3 ANC check-ups 33.6 54.8 <0.001 First ANC was within first 5 months of 43.2 47.4 0.402 pregnancy Provider was qualified at first ANC visit 64.9 82.6 0.006 Provider was qualified at second ANC 65.2 81.8 0.025 visit Provider was qualified at third ANC 70.9 85.0 0.097 visit Delivery care Provider at last birth was qualified 27.1 63.7 <0.001 Provider at last complicated birth was 47.8 85.0 <0.001 qualified Birth took place in a facility 18.7 37.5 <0.001 Location of last birth Home 80.9 62.5 <0.001 Public sector 8.9 29.9 NGO 0.3 3.7 Qualified private 9.2 3.5 Other private 0.1 0.3 Other 0.6 0.1 Had C-section for last birth 9.1 10.4 0.443 Postnatal care Had any PNC visit 20.7 35.6 <0.001 Provider was qualified for postnatal care 72.9 87.1 0.002 within 6 weeks of birth Location of postnatal care Home 16.8 34.4 <0.001 Public sector 27.2 45.9 NGO 2.1 9.0 Qualified private 38.8 5.6 Unqualified private 15.2 5.1 ANNEX TABLE 11. PLACE OF DELIVERY AMONG VOUCHER RECIPIENTS BY INTERVENTION TYPE (%) Place of delivery Universal Means-tested p-value N = 420 N = 427 0.009 Institutional 52.5 32.9 At home with qualified attendant 20.5 50.5 At home with unqualified attendant 27.0 16.6 Note: The denominator N represents the total number of women who received a voucher. Economic Evaluation of DSF Voucher Program in Bangladesh 137 ANNEX TABLE 12. PLACE OF DELIVERY AMONG VOUCHER NON-RECIPIENTS BY INTERVENTION TYPE (%) Place of delivery Type of Study Site Universal Means-tested p-value N = 78 N =80 0.19 Institutional 23.4 23.0 At home with qualified attendant 2.6 13.4 At home with unqualified attendant 74.0 63.4 Notes: The denominator N represents the total number of women who did not receive a voucher. ANNEX TABLE 13. OOP EXPENDITURE ON FIRST ANC VISIT BY POVERTY STATUS (TAKA) OOP Control Universal Means-tested expenditure Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 on first ANC N=127 N=656 N=84 N=433 N=95 N=420 First ANC 134 488 134 188 113 220 P-value 0.003 <0.001 <0.001 Note: Values presented in parentheses are the total number of women who reported using each service ANNEX TABLE 13B. OOP EXPENDITURE ON FIRST ANC VISIT BY POVERTY STATUS (TAKA) OOP expenditure Control Intervention on first ANC Quintile 1 Quintiles 2-5 Quintile 1 Quintiles 2-5 N=127 N=656 N=179 N=853 First ANC 298 405 88 211 p-value 0.107 0.001 Note: Values presented in parentheses are the total number of women who reported using each service ANNEX TABLE 14. TOTAL OOP EXPENDITURE ASSOCIATED WITH PREGNANCY AND DELIVERY CARE BY POVERTY STATUS (IN TAKA) Total OOP Control Universal Means-tested expenditure Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 Quintile 1 Quintile 2-5 N=221 N=883 N=95 N=457 N=107 N=445 Total spending 726 2582 853 1538 654 1660 P-value <0.001 0.019 <0.001 138 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 14B. TOTAL OOP EXPENDITURE ASSOCIATED WITH PREGNANCY AND DELIVERY CARE BY POVERTY STATUS (TAKA) Total OOP Control Intervention expenditure Quintile 1 Quintiles 2-5 Quintile 1 Quintiles 2-5 N=221 N=883 N=202 N=902 Total spending 886 2603 931 1718 p-value <0.001 0.001 ANNEX TABLE 15. KEY MATERNAL AND NEONATAL HEALTH INDICATORS (ANALYSIS OF EOC FACILITIES ONLY) Intervention Control p-value Complicated deliveries as percent of total deliveries 27.7% 31.4% 0.0125 C-sections as a percent of total deliveries 13.0% 8.3% <0.001 Stillbirths as percent of total deliveries 1.37% 2.57% 0.003 Newborn deaths as percent of live births 0.01% 0.10% 0.033 Maternal deaths as percent of live births 0.05% 0.0% 0.473 ANNEX TABLE 16. AVERAGE NUMBER OF FP SERVICES PROVIDED IN 2009 Intervention Control p-value Condom 3,580 23,063 0.23 Oral contraceptive pill 18,049 56,769 0.30 Contraceptive injection 8,465 10,741 0.80 IUD 1,946 450 0.42 Norplant 333 344 0.97 Sterilization 2,365 1,437 0.73 ANNEX TABLE 17. AVERAGE NUMBER OF SUPPORT AND OTHER STAFF PER FACILITY Intervention Control p-value Guard 1.3 1.4 0.62 Sweeper 3.3 2.8 0.58 Ward Boy 2.4 2.1 0.58 Receptionist 0.0 0.1 0.35 Statistician 0.9 0.9 1.00 Store keeper 1.5 1.6 0.74 Driver 1.5 1.8 0.51 Cook 1.1 1.4 0.54 Aya 1.5 5.3 0.33 Other Spec 0.3 1.5 0.22 Economic Evaluation of DSF Voucher Program in Bangladesh 139 ANNEX TABLE 18. CAPACITY TO PROVIDE EMERGENCY OBSTETRIC CARE (EOC) Intervention Control p-value Personnel on site can take care of all obstetric 75% 75% 1.00 emergencies, through management or referral Facility has someone qualified to conduct C-sections 88% 50% 0.10 Facility has someone qualified to provide blood 88% 50% 0.10 transfusions Maternity services available all the time (incl 88% 75% 0.52 nights/weekends) Staff member available at night 100% 75% 0.13 Staff member available on site at night 0% 13% NA ANNEX TABLE 19. PERCENTAGE OF FACILITIES WITH REGISTERSAND GUIDELINES AVAILABLE Intervention Control p-value Health section ANC register 88% 75% 0.55 Delivery register 100% 88% 0.35 Clinical management protocols guidelines (Maternal 100% 88% 0.35 health, Child health and Family planning) FP register 63% 63% 1.00 FP section ANC register 88% 100% 0.35 Delivery register 63% 100% 0.08 Clinical management protocols guidelines (Maternal 100% 75% 0.17 health, Child health and Family planning) FP register 75% 75% 1.00 ANNEX TABLE 20. PERCENTAGE OF FACILITIES WITH PICTORIAL EDUCATIONAL MATERIALS AVAILABLE Intervention Control p-value Health section Pictures: warning signs of complications in pregnancy 88% 100% 0.35 Pictures: postpartum care/newborn care/breast-feeding 88% 88% 1.00 Pictures: family planning 75% 50% 0.33 Pictures: sexually transmitted diseases and/or HIV/AIDS 75% 63% 0.62 FP section Pictures: warning signs of complications in pregnancy 88% 100% 0.35 Pictures: postpartum care/newborn care/breast-feeding 88% 75% 0.55 Pictures: family planning 100% 88% 0.35 Pictures: sexually transmitted diseases and/or HIV/AIDS 50% 50% 1.00 140 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 21. PERCENTAGE OF FACILITIES WITH CHARTS AVAILABLE Intervention Control p-value Health section Charts: warning signs of complications in pregnancy 88% 75% 0.55 Charts: postpartum care/newborn care/breast-feeding 88% 63% 0.28 Charts: family planning 75% 50% 0.33 Charts: sexually transmitted diseases and/or HIV/AIDS 100% 63% 0.34 FP section Charts: warning signs of complications in pregnancy 88% 75% 0.55 Charts: postpartum care/newborn care/breast-feeding 88% 75% 0.55 Charts: family planning 88% 63% 0.28 Charts: sexually transmitted diseases and/or HIV/AIDS 75% 50% 0.56 ANNEX TABLE 22. PERCENTAGE OF FACILITIES WITH LEAFLETS OR HANDOUTS AVAILABLE Intervention Control p-value Health section Leaflets: warning signs of complications in pregnancy 75% 63% 0.62 Leaflets: postpartum care/newborn care/breast-feeding 75% 38% 0.15 Leaflets: family planning 75% 25% 0.05 Leaflets: sexually transmitted diseases and/or HIV/AIDS 88% 25% 0.14 FP section Leaflets: warning signs of complications in pregnancy 75% 63% 0.62 Leaflets: postpartum care/newborn care/breast-feeding 75% 38% 0.15 Leaflets: family planning 100% 50% 0.03 Leaflets: sexually transmitted diseases and/or HIV/AIDS 75% 13% 0.14 ANNEX TABLE 23. EDUCATIONAL SESSIONS FOR MOTHERS Intervention Control p-value Health section Percent with educational sessions for mothers 75% 63% 0.62 Frequency of sessions (days per month) 9.13 7.38 0.74 FP section Percent with educational sessions for mothers 75% 88% 0.55 Frequency of sessions (days per month) 3.88 4.13 0.92 ANNEX TABLE 24. AVAILABILITY OF EMOC SERVICES AT REFERRAL FACILITY Intervention Control p-value Obstetrician/ gynecologist always on duty 100% 100% 1.00 Anesthesiologist always on duty 100% 88% 0.35 Blood bank 100% 100% 1.00 Economic Evaluation of DSF Voucher Program in Bangladesh 141 ANNEX TABLE 25. WHO MAKES TRAVEL ARRANGEMENTS IN CASE OF OBSTETRIC EMERGENCY Intervention Control p-value Nurse 50% 63% 0.64 Family 63% 75% 0.62 Medical assistant 25% 25% 1.00 Other 13% 0% 0.35 ANNEX TABLE 26. REFERRAL PROCEDURES AND PATTERNS Intervention Control p-value Facility has referral slip or form 100% 100% 1.00 Refers patients to DH, MCWC, or MCH 100% 100% 1.00 Average number referred in past month 18.9 22.0 0.73 Receives referrals from UHFWC 100% 100% 1.00 Average number of referrals received in past month 9.3 6.3 0.64 ANNEX TABLE 27. INDICATORS OF ADEQUATE FACILITY INFRASTRUCTURE: GENERAL Intervention Control p-value Signboard on citizen's charter on rights 88% 75% 0.55 Chart indicating list of available medicines 75% 88% 0.55 An examination room or area providing client privacy 88% 88% 1.00 Storage area or cupboard for drugs and other supplies 88% 100% 0.35 Toilet facilities or latrine 100% 100% 1.00 Refrigerator or cold storage system (for TT immunization) 100% 100% 1.00 Meters to nearest water supply 10.6 13.8 0.64 Running water in facility 100% 100% 1.00 Number of antenatal exam rooms 1.6 1.3 0.49 Working window in ANC exam room 100% 75% 0.17 Adequate lighting in ANC exam room 100% 88% 0.35 142 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 28. INDICATORS OF ADEQUATE FACILITY INFRASTRUCTURE: LABOR AND DELIVERY ROOMS Intervention Control p-value Number of labor rooms 1.38 1.50 0.74 Labor rooms have windows 100% 75% 0.17 Labor rooms have adequate air flow 100% 100% 1.00 Labor room lighting: Electric lights 100% 100% 1.00 Candles 38% 25% 0.62 Kerosene lamp 38% 38% 1.00 Flashlight (torch) with working batteries 75% 75% 1.00 Generator 50% 38% 0.64 Labor room floor cracked 0% 0% 1.00 Dirt comes through floor 0% 0% 1.00 Labor room walls cracked 0% 0% 1.00 Labor room walls dirty 13% 25% 0.55 Labor room roof protects from sun and rain 75% 75% 1.00 Facility has operating room for conducting C-sections 100% 88% 0.35 Operating room has adequate lighting 100% 63% 0.08 Operating room has universal frame-type operating table with headpiece 63% 50% 0.64 ANNEX TABLE 29. AVERAGE AVAILABILITY OF REQUIRED SUPPLIES TO PROVIDE ESSENTIAL MATERNAL HEALTH SERVICES, ACCORDING TO MOTHER-BABY PACKAGE CRITERIA (%) Intervention Control p-value Facility has: ANC cards 50% 63% 0.64 Delivery register 63% 38% 0.35 Facility has adequate supplies to manage: Abortion complications 68% 54% 0.15 Severe anemia 29% 46% 0.17 ANC services 54% 59% 0.62 Sexually transmitted infections 0% 0% -- Normal deliveries 100% 73% 0.08 Eclampsia 76% 62% 0.17 Hemorrhage 68% 58% 0.29 Obstructed labor 80% 59% 0.12 Sepsis 0% 0% -- Newborn care 48% 42% 0.64 Economic Evaluation of DSF Voucher Program in Bangladesh 143 ANNEX TABLE 30. AVERAGE AVAILABILITY OF EQUIPMENT RECOMMENDED BY THE MOTHER-BABY PACKAGE (%) Intervention Control p-value Equipment to manage: Abortion complications 60% 58% 0.82 Anemia 33% 33% 0.98 ANC services 38% 54% 0.18 Normal deliveries 0% 0% -- Eclampsia 54% 50% 0.76 Hemorrhage 63% 53% 0.40 Obstructed labor 63% 53% 0.40 Sepsis 33% 58% 0.03 Newborn care 60% 68% 0.43 ANNEX TABLE 31. PRESENCE OF ADEQUATE LABORATORY CAPACITY FOR ESSENTIAL MATERNAL HEALTH SERVICES Intervention Control p-value Laboratory capacity to manage: Abortion complications 48% 38% 0.45 Anemia 48% 38% 0.45 ANC services 41% 27% 0.34 Eclampsia 56% 38% 0.29 Hemorrhage 63% 63% 1.00 Obstructed labor 63% 63% 1.00 Sepsis 75% 50% 0.33 ANNEX TABLE 32. PROPORTION OF MOST SENIOR HEALTH FACILITY STAFF WITH EMERGENCY OBSTETRIC AND EMERGENCY NEONATAL CARE TRAINING Intervention Control p-value Physicians: EOC training 88% 50% 0.12 Physicians: ENC training 50% 25% 0.33 Senior nurses: EOC training 88% 50% 0.12 Senior nurses: ENC training 29% 25% 0.89 FWVs: EOC training 13% 0% 0.35 FWVs: ENC trainings 0% 0% -- Other staff: EOC training 0% 0% -- Other staff: ENC training 0% 0% -- 144 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 33. INDICATORS OF PHYSICIAN KNOWLEDGE Average scores: (1=Good, 4=Poor) Intervention Control p-value Identification: antepartum hemorrhage 3.67 3.43 0.58 Correct action: antepartum hemorrhage 3.67 3.71 0.78 Identification: infection 3.17 2.71 0.97 Correct action: infection 4.00 3.86 0.21 Correct follow-up: infection 1.50 1.43 0.84 Correct action: pregnancy-induced hypertension 2.00 1.86 0.62 Correct action: eclampsia 4.00 3.71 1.00 Identification: dysfunctional labor 3.50 2.14 0.15 Identification: postpartum hemorrhage 4.33 3.71 0.23 Correct action: postpartum hemorrhage 3.60 4.00 0.47 Correct urgent action: postpartum hemorrhage 1.33 1.00 0.36 Knowledge to: perform prenatal risk screening 0.83 0.86 0.92 use partographs to manage labor 0.67 0.43 0.43 manually remove placentas 1.00 0.71 0.17 start IV infusions 0.83 1.00 0.36 check hemoglobin 0.83 0.86 0.92 bimanually compress the uterus (internal) 0.67 0.71 0.87 bimanually compress the uterus (external) 0.83 0.86 0.92 suture (repair) episiotomies 0.83 0.57 0.34 suture (repair) cervical lacerations 0.67 0.57 0.75 suture (repair) vaginal lacerations 0.67 0.43 0.43 suture (repair) 3rd/4th degree lacerations 0.67 0.29 0.20 perform external versions 0.33 0.29 0.87 perform internal versions 0.33 0.29 0.87 perform vacuum extractions 0.50 0.43 0.82 perform menstrual extractions 0.50 0.43 0.82 first assist at C/Section 0.83 0.57 0.34 perform reflex testing 0.50 0.71 0.48 perform speculum examinations 0.83 0.71 0.64 perform bimanual examinations 0.67 0.71 0.87 Economic Evaluation of DSF Voucher Program in Bangladesh 145 ANNEX TABLE 34. PERCENT OF PHYSICIANS THAT CURRENTLY CONDUCT THE FOLLOWING PROCEDURES Intervention Control p-value bimanually compress the uterus (external) 33% 43% 0.75 suture (repair) episiotomies 33% 57% 0.43 suture (repair) cervical lacerations 33% 57% 0.43 suture (repair) vaginal lacerations 33% 43% 0.75 suture (repair) 3rd/4th degree lacerations 50% 29% 0.48 perform external versions 50% 29% 0.48 perform internal versions 33% 0% 0.17 perform vacuum extractions 17% 0% 0.36 perform menstrual extractions 33% 14% 0.47 first assist at C/Section 33% 0% 0.17 perform reflex testing 50% 14% 0.21 perform speculum examinations 33% 57% 0.43 perform bimanual examinations 67% 57% 0.75 146 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX TABLE 35. INDICATORS OF NURSE KNOWLEDGE Average scores: (1=Good, 4=Poor) Intervention Control p-value Identification: antepartum hemorrhage 2.75 3.25 0.35 Correct action: antepartum hemorrhage 4.50 4.25 0.59 Identification: infection 3.13 3.00 0.89 Correct action: infection 3.25 4.38 0.08 Correct follow-up: infection 1.38 1.38 1.00 Correct action: pregnancy-induced hypertension 2.38 2.00 0.47 Correct action: eclampsia 3.75 4.38 0.29 Identification: dysfunctional labor 2.25 2.75 0.43 Identification: postpartum hemorrhage 3.71 4.50 0.16 Correct action: postpartum hemorrhage 3.25 4.00 0.38 Correct urgent action: postpartum hemorrhage 1.17 1.00 0.36 Knowledge to: perform prenatal risk screening 0.88 1.00 0.35 use partographs to manage labor 0.88 0.50 0.12 manually remove placentas 0.88 0.75 0.55 start IV infusions 0.75 0.88 0.55 check hemoglobin 0.75 0.88 0.55 bimanually compress the uterus (internal) 0.88 0.50 0.12 bimanually compress the uterus (external) 0.88 0.88 1.00 suture (repair) episiotomies 0.88 0.63 0.28 suture (repair) cervical lacerations 0.88 0.50 0.12 suture (repair) vaginal lacerations 0.75 0.50 0.33 suture (repair) 3rd/4th degree lacerations 0.75 0.50 0.33 perform external versions 0.38 0.50 0.64 perform internal versions 0.38 0.50 0.64 perform vacuum extractions 0.63 0.63 1.00 perform menstrual extractions 0.50 0.38 0.64 first assist at C/Section 0.88 0.88 1.00 perform reflex testing 0.38 0.63 0.35 perform speculum examinations 0.63 0.50 0.64 perform bimanual examinations 0.38 0.50 0.64 ANNEX TABLE 36. PERCENT OF NURSES THAT CURRENTLY CONDUCT THE FOLLOWING PROCEDURES suture (repair) episiotomies 75% 83% 0.73 suture (repair) cervical lacerations 88% 83% 0.85 suture (repair) vaginal lacerations 63% 50% 0.67 suture (repair) 3rd/4th degree lacerations 75% 50% 0.39 bimanually compress the uterus (external) 50% 33% 0.57 perform external versions 38% 33% 0.88 perform internal versions 13% 0% 0.35 perform vacuum extractions 13% 0% 0.35 perform menstrual extractions 38% 0% 0.08 first assist at C/Section 38% 0% 0.08 perform reflex testing 75% 50% 0.39 Economic Evaluation of DSF Voucher Program in Bangladesh 147 perform speculum examinations 25% 50% 0.39 perform bimanual examinations 38% 50% 0.67 ANNEX TABLE 37. ROUTINE CHECKS PERFORMED DURING THE FIRST ANC VISIT (%) Quality of antenatal Type of study site care Control Universal Means- p-value tested At first ANC visit: N= 780 N= 517 N= 515 Height was measured 26.4 47.5 59.8 <0.001 Blood pressure checked 86.7 82.1 93.0 0.026 Blood test taken 24.6 36.6 35.0 0.026 Ultrasound taken 15.1 7.8 7.8 0.006 Note: Results are applied to those incurring service only. Figures are shown only for those services where the difference among intervention areas was significant at p<0.05. Indicators that are not different at p<0.05 include: weight measure, urine test, abdomen exam, vaginal exam. Results for 2nd and 3rd ANC closely follow the patterns of 1st ANC. 148 Economic Evaluation of DSF Voucher Program in Bangladesh ANNEX D: BIBLIOGRAPHY Attanasio, O., Gomez, LC., Heredia, P., & Vera-Hernandez, M. 2005. The short-term impact of a conditional cash subsidy on child health and nutrition in Colombia. IFS Report Summary Familias 03. London, UK: Institute for Fiscal Studies, Center for the Evaluation of Development Policies. Retrieved from: http://www.ifs.org.uk/edepo/rs_fam03.pdf. Bangladesh Bureau of Statistics (BBS). 2001. Population Census 2001, National Report (Provisional) July 2003. Dhaka: Bangladesh Bureau of Statistics. Planning Division, Ministry of Planning, Government of Bangladesh. BBS. 2001. Population Census- 2001, National Series, Volume- 2, Union Statistics. Dhaka: Bangladesh Bureau of Statistics. Planning DivisiBBS). on, Ministry of Planning, Government of Bangladesh. BBS. 2008. Statistical Pocket Book of Bangladesh. Dhaka: Bangladesh Bureau of Statistics. Planning Division, Ministry of Planning, Government of Bangladesh. Barber SL and PJ Gertler. 2009. Empowering women to obtain high quality care: evidence from an evaluation of Mexico’s conditional cash transfer programme. Health Pol Plann. 24(1):18-25. Bhatia MR and AC Gorter. 2007. Improving access to reproductive and child health services in developing countries: are competitive voucher schemes and option? J Int Dev. In press (published online). Behrman, J. and Hoddinott, J. 2005. Program evaluation with unobserved heterogeneity and selective implementation: the Mexican Progresa impact on child nutrition. Oxford Bulletin of Economics and Statistics, 67, 547–69. Borghi J, Somanathan A, Lissner C & A Mills. 2006. Mobilising financial resources for maternal health. Lancet. 368: 1457-1465. Chankova, Slavea, Sushil R. Howlader, Syed A. Hamid, Subrata Routh, Tasnuva Sultana, and Hong Wang. 2010. Costing of Maternal Health Services in Bangladesh. Bethesda, MD: Review, Analysis and Assessment of Issues Related to Health Care Financing and Health Economics in Bangladesh, Abt Associates Inc. Devadasan N, Elias MA, John D, Grahacharya S & L Ralte. 2008. A conditional cash assistance programme for promoting institutional deliveries among the poor in India: process evaluation results. Studies in HSO & P. 24:257-273. Ensor T & J Ronoh. 2005. Effective financing of maternal health services: a review of the literature. Health Policy. 75: 49-58. Fernald, Lia CH., Paul J Gertler, Lynnette M Neufeld. 2008. Role of cash in conditional cash transfer programmes for child health, growth, and development: an analysis of Mexico's Oportunidades. Lancet, 371 (9615): 828-837. 149 Gertler, P. (2004), ‘Do conditional cash transfers improve child health? Evidence from PROGRESA’s controlled randomized experiment’. American Economic Review, 94, 336–41. Gorter AC & B Bellows. 2008. Do competitive voucher schemes improve the provision of health care to underserved and/or vulnerable population groups? Experiences from Nicaragua, India and Africa. Seminar given at Department of Social Medicine, University of Bristol, on April 24, 2008. Gottret, Pablo, George J. Schieber, and Hugh R. Waters (Eds.). 2008. Good practices in health financing: Lessons from reforms in low and middle-income countries. Washington, D.C.: The World Bank. Government of Bangladesh (GoB). 2005. Demand Side Financing (DSF) Pilot Maternal Health Voucher. Final report, Health and Socioeconomic Baseline Survey in 21 Demand Side Financing (DSF) and 21 Control Upazilas in 21 Old Districts of Bangladesh. Volume 1: Main Text. Dhaka: Ministry of Health and Family Welfare, Government of Bangladesh. Government of Bangladesh (GoB). 2005. Demand Side Financing (DSF) Pilot Maternal Health Voucher. Final report, Conducting Health and Socioeconomic Baseline Survey in 21 Demand Side Financing (DSF) and 21 Control Upazilas in 21 Old Districts of Bangladesh. Volume 2: Database Table. Dhaka: Ministry of Health and Family Welfare, Government of Bangladesh. Government of Bangladesh (GoB). 2008. Demand Side Financing Pilot Maternal Health Voucher Scheme - DSF Protocol. Draft prepared in collaboration with World Health Organization, Bangladesh. Dhaka: Ministry of Health and Family Welfare, Government of Bangladesh. Government of Bangladesh (GoB). 2008. Demand Side Financing Maternal Voucher Health Scheme, Orientation Manual. Revised, January 2008. Dhaka: Ministry of Health and Family Welfare, Government of Bangladesh. Government of Bangladesh (GoB). 2008. Demand Side Financing Pilot Maternal Health Voucher Scheme - DSF Protocol. Draft prepared in collaboration with World Health Organization, Bangladesh. June 2004. Dhaka: Ministry of Health and Family Welfare, Government of Bangladesh. Graham, Wendy J., John Cairns, Sohinee Bhattacharya, Colin H. W. Bullough, Zahidul Quayyum, and Khama Rogo, "Maternal and Perinatal Conditions." 2006. Disease Control Priorities in Developing Countries (2nd Edition),ed. , 499-530. New York: Oxford University Press. DOI: 10.1596/978-0-821- 36179-5/Chpt-26. Health Economics Unit, Ministry of Health and Family Welfare. 2008. Report of Rapid Assessment of Demand Side Financing (DSF) Pilot. Research Paper No. 35. Dhaka, Bangladesh. Iqbal, Mohammed, Sabrina Rashid, Syed Manzoor, Ahmed Hanifi and Abbas Bhuiya. 2009. Reaching the poor with performance based payment for safe deliivery services in rural Bangladesh: Potential of Performance Based Payment. Bulletin 112. Medicus Mundi Schweiz. Ir, P., Horeman D., Narin S., & Van Damme, W. 2008. Improving access to safe delivery for poor pregnant women: a case study of vouchers plus health equity funds in three health districts in Cambodia. Studies in HSO & P. 24:225-256. 150 Economic Evaluation of DSF Voucher Program in Bangladesh Islam, Mursaleena. 2006. “Vouchers for Health: A Focus on Reproductive Health and Family Planning Services.” Primer for Policymakers. Bethesda, MD: Private Sector Partnerships-One project, Abt Associates Inc. Kundu, Francis. 2007. A new initiative towards universal access to reproductive health in Kenya. National Coordinating Agency for Population & Development. Retrieved from: http://www.ncapd- ke.org. Lagarde M, A Haines & N Palmer. 2007. Health interventions in low- and middle-income countries: a systematic review. JAMA. 298(16): 1900-1910. Lenel A & D Griffith. 2007. Voucher schemes in the health sector: the experience of German Financial Cooperation. Working Paper. KfW Bankengruppe, Corporate Communication: Frankfurt am Main. Leroy, Jef L., Armando García-Guerra, Raquel García, Clara Dominguez, Juan Rivera and Lynnette M. Neufeld. 2008. The Oportunidades Program Increases the Linear Growth of Children Enrolled at Young Ages in Urban Mexico. Journal of Nutrition. 138:793-798. Luoma, Marc, Jobayda Fathema, Jamil H. Chowdhury, and Hong Wang. December 2009. Incentives to improve retention and performance of public sector doctors and nurses in Bangladesh. Bethesda, MD: Review, Analysis and Assessment of Issues Related to Health Care Financing and Health Economics in Bangladesh, Abt Associates Inc. Marie Stopes Clinic Society. 2010. Increasing access to safe delivery for the poor in hard-to-reach areas of Bangladesh: Marie Stopes Experience on Demand Side Financing. Dhaka: Marie Stopes Clinic Society. Meuwissen LE, Gorter AC, Kester ADM & JA Knotterus. 2006. Does a competitive voucher program for adolescents improve the quality of reproductive health care? A simulated patient study in Nicaragua. BMC Public Health, 6: 204. Ministry of Health and Family Welfare (Bangladesh). 2008. DSF Protocol (Draft). Prepared in collaboration with the World Health Organization. Morris SS, Rafael Flores, Pedro Olinto, and Juan Manuel Medina. 2004a. Monetary incentives in primary health care and effects on use and coverage of preventive health care interventions in rural Honduras: cluster randomised trial. Lancet, 364(9450):2030-7. Morris, S., Olinto, P., Flores, R., Nilson, E. and Figueiró, A. 2004b. Conditional cash transfers are associated with a small reduction in the rate of weight gain of preschool children in northeast Brazil. Journal of Nutrition, 134, 2336–41. National Institute of Population Research and Training (NIPORT), Mitra and Associates, and Macro International. 2009. Bangladesh Demographic and Health Survey 2007. Dhaka, Bangladesh and Calverton, Maryland, USA: National Institute of Population Research and Training, Mitra and Associates, and Macro International. National Institute of Population Research and Training (NIPORT), ORC Macro, Johns Hopkins University and ICDDR,B. Bangladesh maternal health services and maternal mortality survey 2001. 151 Dhaka, Bangladesh and Calverton, (MD), USA: NIPORT, ORC Macro, Johns Hopkins University and ICDDR,B; 2003. Patsika, Rebecca, Gael O’Sullivan, and Victoria Child. July 2009. Promoting Family Planning Use in Kenya through Output-Based Aid: Strategic Marketing Recommendations to NCAPD and KfW. Bethesda, MD: Private Sector Partnerships-One project, Abt Associates Inc. Rahman, Md. Moshiur, Ubaidur Rob, Tasnima Kibria. 2009. “Implementation of Maternal Health Financial Scheme in Rural Bangladesh: Demand-based Reproductive Health Commodity Project.” Dhaka: Population Council, Bangladesh. Rawlings LB and GL Rubio. 2005. Evaluating the impact of conditional cash transfer programs. World Bank Research Observer. 20(1):29-55. Rawlings, Laura B. 2004. A New Approach to Social Assistance: Latin America’s Experience with Conditional Cash Transfer Programs. Social Protection Discussion Paper 0416, The World Bank: Washington, DC. RTM International. 2009. Demand-Based Reproductive Health Commodity Project (DBRHCP): Activity Completion Report. Dhaka: RTM International. Schmidt, JO, Ensor, T., Hossain, A. and Khan, S. 2010 (forthcoming). Vouchers as demand side financing instruments for health care: A review of the Bangladesh maternal voucher scheme. Health Policy. Shiffman, Jeremy. 2000. Can poor countries surmount high maternal mortality? Studies in Family Planning 31(4): 274-289. Sloan, Nancy, Charlotte Quimby, Beverly Winikoff, Nina Schwalbe, Suellen Miller, and Bernard Pobiak. (2000). Guidelines and instruments for situation analysis of obstetric services. New York: Population Council. Accessed online at http://www.popcouncil.org/software/handhelds/ OBGuide.XLT. Stanton, ME and M. Koblinsky. Essential Obstetric Care and Subsets—Basic and Emergency Obstetric Care: What’s the Difference? Policy Brief 1. MotherCare Project, John Snow Inc. United Nations Population Fund [UNFPA]. 2006. Rapid assessment of Chiranjeevi Yojana in Gujarat. UNFP: New Delhi. World Bank. 2005. A Guide to Competitive Vouchers. Washington, DC: World Bank. World Bank. 2006. Disease Control Priorities in Developing Countries. Second Edition. Washington, DC: World Bank. World Health Organization. 1994. Mother-Baby Package. WHO/FHE/MSM/94.11. 152 Economic Evaluation of DSF Voucher Program in Bangladesh