Handbook for Family Planning Operations Research Design SecondEdition Andrew A. Fisher JohnE.taing JohnE.Stoe_el JohnW.Townsend Population Council One Dag Hammarskjold Plaza New York, New York 10017 www.popcouncil.org Library of Congress Andrew A. Fisher, Sc.D. is Senior Associate Cataloging In Publication Data and Director of the Africa Operations Be- search and Technical Assistance Project (Afri- Handbook for family plannin 8 operations research ca oRfrA Project), The Population Council, design / Andrew A. Fisher... let al.] -- 2nd ed. Nairobi, Kenya. p. cm. l_ev. ed. of: Handbook for family planning operations research desisn / Andrew Fisher,' John E. Laing, Ph.D. is Consultant, Austin, John Lalng, John Stoeckel. Texas. He was formerly a Senior Associate Includes bibliographical references, with The Population Council in South and ISBN 0-87834-059-9 East Asia. 1. Birth control--Research--Handbooks, manuals, etc. John E. 8toeckel, Ph.D. is Senior Associate, 2. Operations research--Handbooks, manuals, The Population Council, Bangkok, Thailand. etc. I. Fisher, Andrew, 1941- II. Fisher, Andrew, 1941-Handbook for family plannir_ JOhn W. Townsendp Ph.D. is Senior Associ- operations research design, ate and Senior Representative for Latin Ameri- HQ763.5.H36 1991 ca and the Caribbean, The Population Coun- 363.9'6'072073--clc20 91-6284 CIP cil, Mexico City, Mexico. Any part of this Handbook may be copied or adapted to meet local needs without permission from the authors or the Populatio n Council, provided that the parts copied are distributed free or at cost (not for profit). Any commercial reproduction requires prior permission from the Population Council. The authors would appreciate receiving a copy of any materials in which the text from the Handbook has been used. Printed in the United States of America First edition 1983 Secondedition 1991 Second printing 1998 Contents. Preface to the Second Edition vii 3 A Special Population Group is Acknowledgments ix Affected 4 It Relates to Ongoing Program Activities 5 There are Broad Social, ImplicationsEconomic, Health, or Political 6 It is Considered Important by 1 Introduction 1 Many Different Groups A What Is Operations Research? 1 B The Focus and Objectives of Operations Research 1 ¢_ C Categories of Operations 0¢ Research Studies 1 1 Exploratory/Diagnostic Studies 3 Strategy Selection to Solve the 2 Field Intervention Studies Problem 14 3 Evaluative Studies A Using a Quality-of-Care D The Methods and Study Framework 14 Designs of Operations Research 2 B Examples of Strategies to Test 14 C Guidelines for Selecting an Appropriate Strategy to Test 15 2 AI 2 Identifying, Defining_ and /_ Justifying the Research Problem 4 A Problem Identification 4 4 Objectives and Hypotheses 18 A Ultimate Objectives 18 B Problem Definition 7 1 Incidence and Prevalence B Immediate Objectives--Who 2 Geographic Areas Affected will do, how much of what, to 3 Characteristics of Population whom, when, where, and for Groups what purpose 19 4 Probable Reasons for the C Hypotheses 19 Problem 1 Relationships among Variables 5 Possible Solutions a Independent Variables 6 Unanswered Questions b Intervening Variables C Problem Justification 11 c Dependent Variables 1 It is Current and Timely 2 Direction of Relationship 2 A Large Population is Affected 3 Conditions of Relationship JH Ill E Guidelines for a "Good" Research Design 38 F The Principle of the Three Multiples 38 5 Intervention Description 23 1 Multiple Data Sources A Who Will Be Responsible? 23 2 Multiple Measurements Over B Where Will Activities Take Time Place? 23 3 Multiple Replications C What Activities Will Be Initiated? 23 8 8 Sampling 4o A Selecting Area and Population 40 6 Operational Definitions 26 B Selecting Cases 40 A Operational Definitions of 1 Probability Sample Variables 26 a Simple Random Sampling 1 Indicators b Systematic Sampling 2 Variable Categories c Stratified Sampling a Mutually Exclusive d Cluster Sampling b Totally Inclusive e Multistage Sampling B Operational Definitions of f Probability Proportional to Terms 27 Size (PPS) 2 Non-probability Sample C Sample Size Determination 43 7 Study Design 30 q A Reliability and Validity 30 B Validity Threats 30 9 Data Collection 47 1 History 2 Selection A Quantitative Data 47 3 Testing 1 Structured Interviews 4 Instrumentation 2 Service Statistics 5 Maturation 3 Self-Administered Questionnaires 6 Mortality 4 Secondary Data Sources C Types of Study Designs 32 B Qualitative Data 50 1 Unstructured Interviews 1 Nonexperimental Designs 2 Focus Group Discussions a Posttest-Only Design b Pretest-Posttest Design 3 Direct Observation of Operations 4 Content Analysis of Written c Static-Group Comparison Materials 2 Experimental Designs a, Pretest-Posttest Control Group C Data Quality Checks 51 Design D Confidentiality of Information 52 b Posttest-Only Control Group 3 Quasi-Experimental Designs a TimeDesignSeries Design 10 b Nonequivalent Control Group Design 10 Tabulation and Analysis of Data 53 c Separate Sample Pretest- A Preparing Tabulations 53 Posttest Design 1 Coding Data D Selecting a Study Design 37 2 Editing Coded Data 1 Ethical Issues B Plan for Data Analysis 54 2 Practical and Administrative 1 Attributes of the Data Issues a Central Tendency of the Data 3 Technical Issues b Variance in the Data iv c Differences within the Data 2 Typesd RelatioofnAnalyticalships withinProceduresthe Data 13 a Variable Transformations b Univariate Analysis 13 Limitations of the Study 68 Nominal Measurement A Design and Analysis Ordinal Measurement Limitations 68 Interval Measurement B Special Situational Factors 68 c Time Series Analysis d Comparisons e Bivariate Relationships Between Nominal Variables Between IOrdinalnterval Variables 1 4 f Multivariate Analysis All Interval Independent 14 Resources and Facilities 69 Variables A Available Resources and Both Interval and Categoric Facilities 69 Independent Variables B Study Budget 69 All Categoric Independent Variables g Cost-Effectiveness Analysis i Fertility Analysis h Use-Effectiveness Analysis 15 C Dummy Tables 63 11 15 Appendixes 71 11 DisseminationFindings of Research 64 16 A Dissemination Strategy 64 B Writing Reports 64 16 Title Page and Abstract 72 12 12 Utilization of Study Findings 66 Bibliography 75 Prefaceto the SecondEdition The Handbook was first published in English by the Population Council in 1983 and was based on the authors' experience with field research studies in Asia. Subsequently, it has been translated into French, Spanish, Thai, Bahasa Indonesia, Arabic, and Chinese. It has been widely used not only in Asia but also in Latin America and Africa. In this second edition of the Handbook, we have maintained the same basic format, but we have revised and expanded some of the sections. Where appropriate, examples from Latin America and Afiica have been added. The introductol?¢ section contains a more complete and current statement on the process of health and family planning operations research. New chapters have been included on selecting an appropriate intervention to test in an OR study, and on describing the main elements of the study intervention. The chapter on information dissem- ination has been expanded and a new chapter on the utilization of research findings has been added. The Handbook assumes some familiarity with the terms and concepts of research design and statistics, as well as some experience with field research studies. It has several objectives. First and foremost, it is specifically designed to help health and family planning researchers .develop and write a detailed operations research proposal. It has often been used in workshops as a guide for preparing a well-written and detailed research study proposal. It is also intended to help program administrators and managers understand the process of operations l_esearch and the uses of research findings for seivice delivel 3, improvement. Finally, although the Handbook is not a text on research methods, it does provide a review of key concepts and important methods essential to conducting field researeh studies. vii Acknowledgments The Handbook was originally written as training matelial incidental to the US Agency for Interna- tional Development (USAID), office of Population Contract No. DPE-0632-00-1029-00, Family Planning Operations Research in Asia. This revised edition was written as part of the USAID Office of Population Contract DPE-3030-Z-00-8065-00, Strategies fur Improving Family Planning Service Delivery, The Africa Operations Research and Technical Assistance Project {Afi'ica OR/TA Project) and Contract No. DPE-3030-Z-00-9010-00, Operations Research Project to Improve Family Plan- ning in Latin America and the Caribbean, Operations Research and Technical Assistance in Latin America and the Caribbean (INOPAL). The authors are particularly grateful for the constructive comments received fi_om many workshop participants and other users of the Handbook in Asia, Afi'ica, and Latin America. We are also thankful for the comments, support, and encouragement of colleagues at USAID, Cooperating Agencies, Columbia University, the University of Michigan, Johns Hopkins University, Tulane University, and the University of Hawaii. As fax" as possible, and without radically altering the original format, we have tried to incorporate these comments into this t_evised edition. Finally, We greatly appreciate the critical reviews and many helpful suggestions provided by out" l]'iends within the Population Council: lan Askew, Beverly Ben Salem, John Bongaarts, George Brown,