Meta-Analysis of Drug Abuse Prevention Programs
Total Page:16
File Type:pdf, Size:1020Kb
Meta-Analysis of Drug Abuse Prevention Programs Editor: William J. Bukoski, Ph.D. NIDA Research Monograph 170 1997 U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health National Institute on Drug Abuse Division of Epidemiology and Prevention Research 5600 Fishers Lane Rockville, MD 20857 i ACKNOWLEDGMENT This monograph is based on the papers from a technical review on "Meta-Analysis of Drug Abuse Prevention Programs" held on July 26-27, 1993. The review meeting was sponsored by the National Institute on Drug Abuse. COPYRIGHT STATUS The National Institute on Drug Abuse has obtained permission from the copyright holders to reproduce certain previously published material as noted in the text. Further reproduction of this copyrighted material is permitted only as part of a reprinting of the entire publication or chapter. For any other use, the copyright holder's permission is required. All other material in this volume except quoted passages from copyrighted sources is in the public domain and may be used or reproduced without permission from the Institute or the authors. Citation of the source is appreciated. Opinions expressed in this volume are those of the authors and do not necessarily reflect the opinions or official policy of the National Institute on Drug Abuse or any other part of the U.S. Department of Health and Human Services. The U.S. Government does not endorse or favor any specific commercial product or company. Trade, proprietary, or company names appearing in this publication are used only because they are considered essential in the context of the studies reported herein. National Institute on Drug Abuse NIH Publication No. 97-4147 Printed 1997 NIDA Research Monographs are indexed in the Index Medicus. They are selectively included in the coverage of American Statistics Index, BioSciences Information Service, Chemical Abstracts, Current Contents, Psychological Abstracts, and Psychopharmacology Abstracts. ii Click on title or page number to go to page Table of Contents Meta-Analysis of Drug Abuse Prevention Research............................1 William J. Bukoski Meta-Analysis of Adolescent Drug Prevention Programs: Results of the 1933 Meta-Analysis.....................................................5 Nancy S. Tobler Validity of Integrity Tests for Predicting Drug and Alcohol Abuse: A Meta-Analysis.....................................................69 Frank L. Schmidt, Vish Viswesvaran, and Deniz S. Ones Meta-Analysis and Models of Substance Abuse Prevention.............................................................................96 Betsy Jane Becker Realities of the Effect Size Calculation Process: Considerations for Beginning Meta-Analysts.................................120 Patricia D. Perry Issues and Challenges in Coding Interventions for Meta-Analysis of Prevention Research...........................................130 Elizabeth C. Devine Experiments Versus Quasi-Experiments: Do They Yield the Same Answer?..................................................147 William R. Shadish and Donna T. Heinsman Drawing Generalized Causal Inferences Based on Meta-Analysis.................................................................165 Georg E. Matt Issues in Classification in Meta-Analysis in Substance Abuse Prevention Research............................................183 William B. Hansen and Lynn A. Rose iii Improving Meta-Analysis for Policy Purposes...............................202 Larry V. Hedges Using Linked Meta-Analysis To Build Policy Models....................216 Mark W. Lipsey Some Limiting Factors in Meta-Analysis .......................................234 Robert L. Bangert-Drowns iv Meta-Analysis of Drug Abuse Prevention Research William J. Bukoski INTRODUCTION After nearly 15 years of declining rates in adolescent drug abuse, current epidemiologic research indicates significant increases in the use of a variety of illicit drugs of abuse such as inhalants, marijuana, cocaine, lysergic acid diethylamide (LSD), and phencyclidine (PCP) (Department of Health and Human Services 1994) by children and youth in the 8th, 10th, and 12th grades. Faced with these alarming increases in drug abuse, concerned parents, educators, and community leaders are turning to prevention research to better understand the nature of these recent trends and to guide prevention policy and program development. Critical to effective preventive action at all levels of Government is an assessment of the numerous scientific findings that have been published over the past decade that may indicate which prevention practices are efficacious and which drug abuse prevention strategies need to be considered for implementation in school and community programs in order to bring a halt to increased drug abuse by the Nation's youth. To assist in this deliberative process, the National Institute on Drug Abuse (NIDA) has consulted with a number of this country's best scientists to analyze prevention research findings from a variety of published studies and to integrate those results into a meaningful and objective meta-analysis in order to identify promising drug abuse prevention strategies and policies. Given the complexities of the published prevention research, it was decided that the meta-analysis of research findings should follow the systematic procedures employed in this methodology and utilize a common standard or metric that would permit the comparison and integration of outcomes across a variety of individual studies (Cook et al. 1992). Central to this process is the calculation of a metric that is called the effect size. The effect size provides, in standard deviation units, an objective and uniform measure of quantitative differences in drug prevention outcomes such as self-reported drug use, knowledge of negative consequences of drug abuse, and antidrug-abuse attitudes that could be attributed to the exposure of the treatment group that had been 1 randomly assigned to an experimental prevention intervention in comparison to a control group that did not receive the program. To conduct a meta-analysis, researchers identify salient prevention research studies. Using a standardized procedure, they calculate the effect sizes for drug-related outcome measures reported in each study. Given that effect sizes are calculated in units of standard deviation, the measurements are comparable across studies and hence subject to further analysis such as assessing the efficacy of different prevention program strategies. Rather than relying on findings from one study, meta-analysis provides a technically sound method of combining results from a variety of studies in order to identify the extent to which specific types of prevention programs are effective in reducing and preventing adolescent drug abuse. The technique of meta-analysis provides a systematic and objective assessment of prevention research findings reported by many scientific studies and results in a convergence of higher order information that can only be provided by analysis of an entire body of research findings. Meta-analysis provides a standardized approach to the identification, selection, assessment, and interpretation of the results of a variety of medical, psychiatric, and behavioral research literatures and is particularly valuable in synthesizing research findings from an emerging science, such as drug abuse prevention research. The practical outcome of NIDA's meta-analysis of prevention research is twofold: programmatic and methodological. Each chapter in this monograph addresses one of these two objectives. In the first section of the monograph, Tobler presents a meta-analysis of adolescent drug abuse prevention research findings; Schmidt and colleagues provide a meta-analysis of integrity tests for predicting drug and alcohol abuse; and Becker provides an approach for meta- analysis of drug-related risk and protective factors research. In the second section of the monograph, several chapters explore the appropriateness and special methodological considerations that must be addressed when conducting a meta-analysis of the drug abuse prevention research literature. Perry's chapter focuses upon methods to calculate effect sizes; Devine's chapter discusses issues in coding prevention intervention studies; Shadish and Heinsman assess the differences in outcomes produced by experimental versus quasi- experimental studies; Matt explores issues concerning generalized causal inferences related to program effects; Hansen reviews 2 approaches to classifying independent variables and types of correlational relationships between dependent and independent variables; in separate chapters, Lipsey and Hedges discuss potential applications of meta-analysis for policy development; and Bangert- Drowns presents general advantages and potential limitations of conducting and utilizing meta-analysis in drug abuse prevention research. Collectively these chapters provide a current overview of the efficacy of drug abuse prevention programs and related measurement systems and help define the techniques employed in meta-analysis of drug abuse prevention programs. The monograph provides firsthand guidance in the application of research findings from meta-analysis and appropriate discussion of key technical procedures that should be considered in conducting future meta-analyses of drug abuse prevention research. It also helps to delineate what prevention programs and policies appear to be the most effective in combating drug abuse by adolescents and young adults who may be