Large Country-Lot Quality Assurance Sampling
Total Page:16
File Type:pdf, Size:1020Kb
HNP DISCUSSION PAPER Public Disclosure Authorized Large Country-Lot Quality Assurance Sampling: A New Method for Rapid Monitoring and Evaluation of Health, Nutrition and Population Programs at Sub-National Public Disclosure Authorized Levels About this series... This series is produced by the Health, Nutrition, and Population Family (HNP) of the World Bank’s Human Development Network. The papers in this series aim to provide a vehicle for publishing preliminary and unpolished results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper Bethany L. Hedt, Casey Olives, Marcello Pagano, Joseph J. Valadez are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual authors whose name appears on the paper. Enquiries about the series and submissions should be made directly to Public Disclosure Authorized the Editor Homira Nassery ([email protected]) or HNP Advisory Service ([email protected], tel 202 473-2256, fax 202 522-3234). For more information, see also www.worldbank.org/ hnppublications. THE WORLD BANK 1818 H Street, NW Public Disclosure Authorized Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: [email protected] May 2008 Large Country-Lot Quality Assurance Sampling: A New Method for Rapid Monitoring and Evaluation of Health, Nutrition and Population Programs at Sub-National Levels Bethany L. Hedt; PhD, MS Casey Olives; MS Marcello Pagano; PhD Joseph J. Valadez; PhD, SD, MPH May 2008 Health, Nutrition and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population Family (HNP) of the World Bank’s Human Development Network. The papers in this series aim to provide a vehicle for publishing preliminary and unpolished results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author(s) whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor, Homira Nassery ([email protected]). Submissions should have been previously reviewed and cleared by the sponsoring department, which will bear the cost of publication. No additional reviews will be undertaken after submission. The sponsoring department and author(s) bear full responsibility for the quality of the technical contents and presentation of material in the series. Since the material will be published as presented, authors should submit an electronic copy in a predefined format (available at www.worldbank.org/hnppublications on the Guide for Authors page). Drafts that do not meet minimum presentational standards may be returned to authors for more work before being accepted. For information regarding this and other World Bank publications, please contact the HNP Advisory Services at [email protected] (email), 202-473-2256 (telephone), or 202- 522-3234 (fax). c 2008 The International Bank for Reconstruction and Development/The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii Health, Nutrition and Population (HNP) Discussion Paper Large Country-Lot Quality Assurance Sampling: A New Method for Rapid Monitoring and Evaluation of Health, Nutrition and Population Programs at Sub-National Levels Bethany L. Hedt; PhD, MSa Casey Olives; MSa Marcello Pagano; PhDa Joseph J. Valadez; PhD, SD, MPHb a Department of Biostatistics, Harvard School of Public Health, Boston, MA, USA b Malaria Implementation Resource Team, AFTHD, The World Bank, Washington, DC, USA Paper prepared by the Malaria Implementation Resource Team, AFTHD, The World Bank, Washington DC, USA, May 2008. This work was funded in part by a grant to The World Bank from The ExxonMobil Foundation, and to Harvard School of Public Health by the National Institutes of Health (T32 AI007358 and R01 EB006195) Abstract: Sampling theory facilitates development of economical, effective and rapid mea- surement of a population. While national policy makers value survey results measuring indicators representative of a large area (a country, state or province), measurement in smaller areas produces information useful for managers at the local level. It is often not possible to disaggregate a national survey to obtain local information if that was not the intent of the original survey design. Cluster sampling is typically used for national or large area surveys because sampling in clusters lowers the cost of a survey. Lot Quality Assurance Sampling (LQAS) is used to measure results at a local level, since it requires small random samples and produces results useful to local managers. However, current LQAS method- ology requires all local areas (strata) be included in the survey in order to be aggregated to produce point estimates for the nation or state. In large countries it is not feasible to sample all strata for logistical and financial reasons. This paper resolves this problem by presenting Large Country-Lot Quality Assurance Sampling (LC-LQAS), a method with two concurrent objectives: (1) provide local managers with accurate local information to enable data driven decisions, and (2) provide central policy makers with the aggregate information they require. These are achieved by integrating cluster sampling with LQAS methodologies. Two examples of the implementation of LC-LQAS are provided, in an HIV/AIDS program in Kenya and a Malaria Booster Project in Nigeria. Classifications of local health units into performance categories and aggregate estimates of coverage, with associated confidence intervals, are provided for select indicators in order to demonstrate its use, analysis, and costs. This paper is written as a manual to support the use of LC-LQAS by others. iii Keywords: Monitoring and evaluation, malaria, HIV/AIDS, LQAS, community Disclaimer: The findings, interpretations and conclusions expressed in the paper are en- tirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Correspondence Details: Professor Marcello Pagano, Department of Biostatistics, Har- vard School of Public Health, 677 Huntington Avenue, Boston, MA, Telephone: 1-617-432- 4911, [email protected] iv Table of Contents ACKNOWLEDGEMENTS . ix ACRONYMS . xi LARGE COUNTRY-LOT QUALITY ASSURANCE SAMPLING:A NEW METHOD FOR RAPID MONITORING AND EVALUATION OF HEALTH, NUTRITION AND POPULATION PROGRAMS AT SUB- NATIONAL LEVELS ............................. 1 Introduction . 1 General Principles of the LQAS Method . 3 Background for Developing a Large Country-LQAS . 5 Integrating Cluster Sample Theory with LQAS . 6 Cluster Sampling to Obtain Provincial Estimates of Indicators . 6 Lot Quality Assurance Sampling in Large Countries . 6 Combining LQAS with Cluster Sample Theory . 7 Development and Application of LC-LQAS as a Program M&E Tool . 8 Constraints . 8 LC-LQAS Sample Size Formulae . 9 Estimating the Intraclass Correlation Coefficient . 10 Case Example 1.1: Determining a Sampling Frame for Nyanza Province . 11 Selecting the Final Sample of Supervision Areas and Individuals . 14 v Analysis of LC-LQAS Data . 15 Case Example 1.2: Analysis for Male Respondents on the Indicator “Know ways to prevent sexual transmission of HIV infection” in Nyanza Province, Kenya ................................... 16 Catchment Area Coverage Proportions . 17 Estimation of Confidence Interval . 18 Estimated Intraclass Correlation . 20 Improving the Estimate of ICCs . 20 Replicating LC-LQAS in Other Countries . 23 Case Example 2.1: An Application of LC-LQAS Methodology in the National Malaria Control Project in Nigeria . 23 Additional Considerations for LC-LQAS . 28 Using LC-LQAS to Obtain National Estimates . 28 Using Multiple ICC Estimates for Sample Size Calculations . 28 Conclusion . 29 APPENDIX A: DERIVATION OF LC-LQAS ESTIMATORS . 30 Notation . 30 Estimation in the Subregions . 31 Estimation in Regions . 32 Derivation of the Weights . 32 Estimator of Proportion . 33 Proof that Tb is an Unbiased Estimator of T . 35 Derivation of the Variance for Pb ............................ 36 APPENDIX B: DERIVATION OF LC-LQAS SAMPLE SIZE FORMULA 38 Comments on the sample size formula . 40 vi APPENDIX C: SAMPLE SIZE CALCULATION FORM . 42 APPENDIX D: FORM FOR ESTIMATING CATCHMENT AREA COV- ERAGE PROPORTION WITH 95% CONFIDENCE INTERVALS . 43 Catchment Area Coverage Proportions . 43 95% Confidence Interval . 45 Intraclass Correlation . 46 APPENDIX E: STATA COMMANDS FOR COVERAGE PROPORTIONS WITH 95% CONFIDENCE INTERVALS . 47 APPENDIX F: SAMPLING FRAME AND ANALYSIS FOR LC-LQAS IN KANO STATE, NIGERIA . 50 Sample Size Calculation . 50 Coverage Proportions . 52 Confidence Interval . 53 Intraclass Correlation . 54 APPENDIX G: SUMMARY OF PARAMETERS AND RESULTS FOR 7 NIGERIAN STATES . 55