DOCSLIB.ORG

What to Do When Data Are Missing in Group Randomized Controlled Trials

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
What to Do When Data Are Missing in Group Randomized Controlled Trials NCEE 2009-0049 What to Do When Data Are Missing in Group Randomized Controlled Trials What to Do When Data Are Missing in Group Randomized Controlled Trials October 2009 Michael J. Puma Robert B. Olsen Chesapeake Research Associates, LLC Stephen H. Bell Cristofer Price Abt Associates, Inc Abstract This NCEE Technical Methods report examines how to address the problem of missing data in the analysis of data in Randomized Controlled Trials (RCTs) of educational interventions, with a particular focus on the common educational situation in which groups of students such as entire classrooms or schools are randomized. Missing outcome data are a problem for two reasons: (1) the loss of sample members can reduce the power to detect statistically significant differences, and (2) the introduction of non-random differences between the treatment and control groups can lead to bias in the estimate of the intervention’s effect. The report reviews a selection of methods available for addressing missing data, and then examines their relative performance using extensive simulations that varied a typical educational RCT on three dimensions: (1) the amount of missing data; (2) the level at which data are missing─at the level of whole schools (the assumed unit of randomization) or for students within schools; and, (3) the underlying missing data mechanism. The performance of the different methods is assessed in terms of bias in both the estimated impact and the associated standard error. NCEE 2009-0049 U.S. DEPARTMENT OF EDUCATION This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences under Contract ED-04-CO-0112/0006. Disclaimer The Institute of Education Sciences (IES) at the U.S. Department of Education contracted with Mathematica Policy Research to develop methods for assessing mediational analyses in education evaluations. The views expressed in this report are those of the author and they do not necessarily represent the opinions and positions of the Institute of Education Sciences or the U.S. Department of Education. U.S. Department of Education Arne Duncan Secretary Institute of Education Sciences John Q. Easton Director National Center for Education Evaluation and Regional Assistance John Q. Easton Acting Commissioner October 2009 This report is in the public domain. While permission to reprint this publication is not necessary, the citation should be: Puma, Michael J., Robert B. Olsen, Stephen H. Bell, and Cristofer Price (2009). What to Do When Data Are Missing in Group Randomized Controlled Trials (NCEE 2009-0049). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. This report is available on the IES website at http://ncee.ed.gov. Alternate Formats Upon request, this report is available in alternate formats such as Braille, large print, audiotape, or computer diskette. For more information, please contact the Department’s Alternate Format Center at 202-260-9895 or 202-205-8113. Disclosure of Potential Conflicts of Interest There are four authors for this report with whom IES contracted to develop the methods that are presented in this report. Michael J. Puma is an employee of Chesapeake Research Associates (CRA), LLC, and Robert B. Olsen, Stephen H. Bell, and Cristofer Price are employees of Abt Associates, Inc. The authors and other staff of CRA and Abt Associates do not have financial interests that could be affected by the content in this report. Disclosure of Potential Conflicts of Interest iii Foreword The National Center for Education Evaluation and Regional Assistance (NCEE) conducts unbiased large-scale evaluations of education programs and practices supported by federal funds; provides research-based technical assistance to educators and policymakers; and supports the synthesis and the widespread dissemination of the results of research and evaluation throughout the United States. In support of this mission, NCEE promotes methodological advancement in the field of education evaluation through investigations involving analyses using existing data sets and explorations of applications of new technical methods, including cost-effectiveness of alternative evaluation strategies. The results of these methodological investigations are published as commissioned, peer reviewed papers, under the series title, Technical Methods Reports, posted on the NCEE website at http://ies.ed.gov/ncee/pubs/. These reports are specifically designed for use by researchers, methodologists, and evaluation specialists. The reports address current methodological questions and offer guidance to resolving or advancing the application of high- quality evaluation methods in varying educational contexts. This NCEE Technical Methods report examines how to address the problem of missing data in the analysis of data in Randomized Controlled Trials (RCTs) of educational interventions, with a particular focus on the common educational situation in which groups of students such as entire classrooms or schools are randomized. Missing outcome data are a problem for two reasons: (1) the loss of sample members can reduce the power to detect statistically significant differences, and (2) the introduction of non-random differences between the treatment and control groups can lead to bias in the estimate of the intervention’s effect. The report reviews a selection of methods available for addressing missing data, and then examines their relative performance using extensive simulations that varied a typical educational RCT on three dimensions: (1) the amount of missing data; (2) the level at which data are missing─at the level of whole schools (the assumed unit of randomization) or for students within schools; and, (3) the underlying missing data mechanism. The performance of the different methods is assessed in terms of bias in both the estimated impact and the associated standard error. Foreword v Table of Contents 1. OVERVIEW AND GUIDANCE .......................................................................................................................... 1 A. INTRODUCTION .................................................................................................................................................. 1 B. MISSING DATA AND RANDOMIZED TRIALS....................................................................................................... 2 C. MISSING DATA METHODS ................................................................................................................................. 4 D. GUIDANCE TO RESEARCHERS............................................................................................................................ 5 2. RANDOMIZED CONTROLLED TRIALS (RCTS) IN EDUCATION AND THE PROBLEM OF MISSING DATA .................................................................................................................................................. 11 A. WHY CONDUCT RCTS?................................................................................................................................... 11 B. RCTS IN EDUCATION ...................................................................................................................................... 12 C. DEFINING THE ANALYSIS SAMPLE .................................................................................................................. 13 D. HOW DATA CAN BECOME MISSING ................................................................................................................ 14 E. THE MISSING DATA PROBLEM ........................................................................................................................ 16 3. SELECTED TECHNIQUES FOR ADDRESSING MISSING DATA IN RCT IMPACT ANALYSIS ..... 18 A. STANDARD MISSING DATA METHODS............................................................................................................... 19 B. METHODS TO ADDRESS MISSING DATA THAT ARE NMAR ............................................................................... 42 4. TESTING THE PERFORMANCE OF SELECTED MISSING DATA METHODS .................................. 50 A. SIMULATION METHODS ................................................................................................................................... 50 B. SIMULATION RESULTS .................................................................................................................................... 57 REFERENCES............................................................................................................................................................ 71 APPENDIX A: MISSING DATA BIAS AS A FORM OF OMITTED VARIABLE BIAS ................................. 77 APPENDIX B: RESOURCES FOR USING MULTIPLE IMPUTATION .......................................................... 80 APPENDIX C: SPECIFICATIONS FOR MISSING DATA SIMULATIONS .................................................... 83 APPENDIX D: FULL SET OF SIMULATION RESULTS ................................................................................. 107 APPENDIX E: STANDARDS FOR JUDGING THE MAGNITUDE OF THE BIAS FOR DIFFERENT MISSING DATA METHODS ................................................................................................................................. 121 vii 1. Overview and Guidance A. Introduction Most statistics textbooks provide lengthy discussions of the theory of probability, descriptive statistics, hypothesis
Load more

Recommended publications
  • Spatial Duration Data for the Spatial Re- Gression Context
    Missing Data In Spatial Regression 1 Frederick J. Boehmke Emily U. Schilling University of Iowa Washington University Jude C. Hays University of Pittsburgh July 15, 2015 Paper prepared for presentation at the Society for Political Methodology Summer Conference, University of Rochester, July 23-25, 2015. 1Corresponding author: [email protected] or [email protected]. We are grateful for funding provided by the University of Iowa Department of Political Science. Com- ments from Michael Kellermann, Walter Mebane, and Vera Troeger greatly appreciated. Abstract We discuss problems with missing data in the context of spatial regression. Even when data are MCAR or MAR based solely on exogenous factors, estimates from a spatial re- gression with listwise deletion can be biased, unlike for standard estimators with inde- pendent observations. Further, common solutions for missing data such as multiple im- putation appear to exacerbate the problem rather than fix it. We propose a solution for addressing this problem by extending previous work including the EM approach devel- oped in Hays, Schilling and Boehmke (2015). Our EM approach iterates between imputing the unobserved values and estimating the spatial regression model using these imputed values. We explore the performance of these alternatives in the face of varying amounts of missing data via Monte Carlo simulation and compare it to the performance of spatial regression with listwise deletion and a benchmark model with no missing data. 1 Introduction Political science data are messy. Subsets of the data may be missing; survey respondents may not answer all of the questions, economic data may only be available for a subset of countries, and voting records are not always complete.
    [Show full text]
  • Design and Implementation of N-Of-1 Trials: a User's Guide
    Design and Implementation of N-of-1 Trials: A User’s Guide N of 1 The Agency for Healthcare Research and Quality’s (AHRQ) Effective Health Care Program conducts and supports research focused on the outcomes, effectiveness, comparative clinical effectiveness, and appropriateness of pharmaceuticals, devices, and health care services. More information on the Effective Health Care Program and electronic copies of this report can be found at www.effectivehealthcare.ahrq. gov. This report was produced under contract to AHRQ by the Brigham and Women’s Hospital DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) Methods Center under Contract No. 290-2005-0016-I. The AHRQ Task Order Officer for this project was Parivash Nourjah, Ph.D. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views ofAHRQ or the U.S. Department of Health and Human Services. Therefore, no statement in this report should be construed as an official position of AHRQ or the U.S. Department of Health and Human Services. Persons using assistive technology may not be able to fully access information in this report. For assistance contact [email protected]. The following investigators acknowledge financial support: Dr. Naihua Duan was supported in part by NIH grants 1 R01 NR013938 and 7 P30 MH090322, Dr. Heather Kaplan is part of the investigator team that designed and developed the MyIBD platform, and this work was supported in part by Cincinnati Children’s Hospital Medical Center. Dr. Richard Kravitz was supported in part by National Institute of Nursing Research Grant R01 NR01393801.
    [Show full text]
  • Missing Data Module 1: Introduction, Overview
    Missing Data Module 1: Introduction, overview Roderick Little and Trivellore Raghunathan Course Outline 9:00-10:00 Module 1 Introduction and overview 10:00-10:30 Module 2 Complete-case analysis, including weighting methods 10:30-10:45 Break 10:45-12:00 Module 3 Imputation, multiple imputation 1:30-2:00 Module 4 A little likelihood theory 2:00-3:00 Module 5 Computational methods/software 3:00-3:30 Module 6: Nonignorable models Module 1:Introduction and Overview Module 1: Introduction and Overview • Missing data defined, patterns and mechanisms • Analysis strategies – Properties of a good method – complete-case analysis – imputation, and multiple imputation – analysis of the incomplete data matrix – historical overview • Examples of missing data problems Module 1:Introduction and Overview Module 1: Introduction and Overview • Missing data defined, patterns and mechanisms • Analysis strategies – Properties of a good method – complete-case analysis – imputation, and multiple imputation – analysis of the incomplete data matrix – historical overview • Examples of missing data problems Module 1:Introduction and Overview Missing data defined variables If filled in, meaningful for cases analysis? • Always assume missingness hides a meaningful value for analysis • Examples: – Missing data from missed clinical visit( √) – No-show for a preventive intervention (?) – In a longitudinal study of blood pressure medications: • losses to follow-up ( √) • deaths ( x) Module 1:Introduction and Overview Patterns of Missing Data • Some methods work for a general
    [Show full text]
  • Imputets: Time Series Missing Value Imputation in R by Steffen Moritz and Thomas Bartz-Beielstein
    CONTRIBUTED RESEARCH ARTICLE 1 imputeTS: Time Series Missing Value Imputation in R by Steffen Moritz and Thomas Bartz-Beielstein Abstract The imputeTS package specializes on univariate time series imputation. It offers multiple state-of-the-art imputation algorithm implementations along with plotting functions for time series missing data statistics. While imputation in general is a well-known problem and widely covered by R packages, finding packages able to fill missing values in univariate time series is more complicated. The reason for this lies in the fact, that most imputation algorithms rely on inter-attribute correlations, while univariate time series imputation instead needs to employ time dependencies. This paper provides an introduction to the imputeTS package and its provided algorithms and tools. Furthermore, it gives a short overview about univariate time series imputation in R. Introduction In almost every domain from industry (Billinton et al., 1996) to biology (Bar-Joseph et al., 2003), finance (Taylor, 2007) up to social science (Gottman, 1981) different time series data are measured. While the recorded datasets itself may be different, one common problem are missing values. Many analysis methods require missing values to be replaced with reasonable values up-front. In statistics this process of replacing missing values is called imputation. Time series imputation thereby is a special sub-field in the imputation research area. Most popular techniques like Multiple Imputation (Rubin, 1987), Expectation-Maximization (Dempster et al., 1977), Nearest Neighbor (Vacek and Ashikaga, 1980) and Hot Deck (Ford, 1983) rely on inter-attribute correlations to estimate values for the missing data. Since univariate time series do not possess more than one attribute, these algorithms cannot be applied directly.
    [Show full text]
  • Coping with Missing Data in Randomized Controlled Trials
    EVALUATION TECHNICAL ASSISTANCE BRIEF for OAH & ACYF Teenage Pregnancy Prevention Grantees May 2013 • Brief 3 Coping with Missing Data in Randomized Controlled Trials issing data in randomized controlled trials (RCTs) can respondents3 if there are no systematic differences between Mlead to biased impact estimates and a loss of statistical respondents in the treatment group and respondents in the power. In this brief we describe strategies for coping with control group. However, if respondents in the treatment group missing data in RCTs.1 Specifically, we discuss strategies to: are systematically different from respondents in the control group, then the impact we calculate for respondents lacks ● Describe the extent and nature of missing data. We suggest causal validity. We call the difference between the calculated ways to provide a clear and accurate accounting of the extent impact for respondents and the true impact for respondents and nature of missing outcome data. This will increase the “causal validity bias.” This type of bias is the focus of the transparency and credibility of the study, help with interpret- evidence review of teen pregnancy prevention programs ing findings, and also help to select an approach for addressing (Mathematica Policy Research and Child Trends 2012). missing data in impact analyses. We suggest conducting descriptive analyses designed to assess ● Address missing data in impact analyses. Missing outcome the potential for both types of bias in an RCT. or baseline data pose mechanical and substantive problems when conducting impact analyses. The mechanical problem With regard to generalizability, we suggest conducting two is that mathematical operations cannot be performed on analyses.
    [Show full text]
  • Monte Carlo Likelihood Inference for Missing Data Models
    MONTE CARLO LIKELIHOOD INFERENCE FOR MISSING DATA MODELS By Yun Ju Sung and Charles J. Geyer University of Washington and University of Minnesota Abbreviated title: Monte Carlo Likelihood Asymptotics We describe a Monte Carlo method to approximate the maximum likeli- hood estimate (MLE), when there are missing data and the observed data likelihood is not available in closed form. This method uses simulated missing data that are independent and identically distributed and independent of the observed data. Our Monte Carlo approximation to the MLE is a consistent and asymptotically normal estimate of the minimizer ¤ of the Kullback- Leibler information, as both a Monte Carlo sample size and an observed data sample size go to in¯nity simultaneously. Plug-in estimates of the asymptotic variance are provided for constructing con¯dence regions for ¤. We give Logit-Normal generalized linear mixed model examples, calculated using an R package that we wrote. AMS 2000 subject classi¯cations. Primary 62F12; secondary 65C05. Key words and phrases. Asymptotic theory, Monte Carlo, maximum likelihood, generalized linear mixed model, empirical process, model misspeci¯cation 1 1. Introduction. Missing data (Little and Rubin, 2002) either arise naturally|data that might have been observed are missing|or are intentionally chosen|a model includes random variables that are not observable (called latent variables or random e®ects). A mixture of normals or a generalized linear mixed model (GLMM) is an example of the latter. In either case, a model is speci¯ed for the complete data (x; y), where x is missing and y is observed, by their joint den- sity f(x; y), also called the complete data likelihood (when considered as a function of ).
    [Show full text]
  • Missing Data
    4 Missing Data Paul D. Allison INTRODUCTION they have spent a great deal time, money and effort in collecting. And so, many methods for Missing data are ubiquitous in psychological ‘salvaging’ the cases with missing data have research. By missing data, I mean data that become popular. are missing for some (but not all) variables For a very long time, however, missing and for some (but not all) cases. If data are data could be described as the ‘dirty little missing on a variable for all cases, then that secret’ of statistics. Although nearly everyone variable is said to be latent or unobserved. had missing data and needed to deal with On the other hand, if data are missing on the problem in some way, there was almost all variables for some cases, we have what nothing in the textbook literature to pro- is known as unit non-response, as opposed vide either theoretical or practical guidance. to item non-response which is another name Although the reasons for this reticence are for the subject of this chapter. I will not deal unclear, I suspect it was because none of the with methods for latent variables or unit non- popular methods had any solid mathematical response here, although some of the methods foundation. As it turns out, all of the most we will consider can be adapted to those commonly used methods for handling missing situations. data have serious deficiencies. Why are missing data a problem? Because Fortunately, the situation has changed conventional statistical methods and software dramatically in recent years.
    [Show full text]
  • Generalized Linear Models for Aggregated Data
    Generalized Linear Models for Aggregated Data Avradeep Bhowmik Joydeep Ghosh Oluwasanmi Koyejo University of Texas at Austin University of Texas at Austin Stanford University [email protected] [email protected] [email protected] Abstract 1 Introduction Modern life is highly data driven. Datasets with Databases in domains such as healthcare are records at the individual level are generated every routinely released to the public in aggregated day in large volumes. This creates an opportunity form. Unfortunately, na¨ıve modeling with for researchers and policy-makers to analyze the data aggregated data may significantly diminish and examine individual level inferences. However, in the accuracy of inferences at the individ- many domains, individual records are difficult to ob- ual level. This paper addresses the scenario tain. This particularly true in the healthcare industry where features are provided at the individual where protecting the privacy of patients restricts pub- level, but the target variables are only avail- lic access to much of the sensitive data. Therefore, able as histogram aggregates or order statis- in many cases, multiple Statistical Disclosure Limita- tics. We consider a limiting case of gener- tion (SDL) techniques are applied [9]. Of these, data alized linear modeling when the target vari- aggregation is the most widely used technique [5]. ables are only known up to permutation, and explore how this relates to permutation test- It is common for agencies to report both individual ing; a standard technique for assessing statis- level information for non-sensitive attributes together tical dependency. Based on this relationship, with the aggregated information in the form of sample we propose a simple algorithm to estimate statistics.
    [Show full text]
  • Parameter Estimation in Stochastic Volatility Models with Missing Data Using Particle Methods and the Em Algorithm
    PARAMETER ESTIMATION IN STOCHASTIC VOLATILITY MODELS WITH MISSING DATA USING PARTICLE METHODS AND THE EM ALGORITHM by Jeongeun Kim BS, Seoul National University, 1998 MS, Seoul National University, 2000 Submitted to the Graduate Faculty of the Department of Statistics in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2005 UNIVERSITY OF PITTSBURGH DEPARTMENT OF STATISTICS This dissertation was presented by Jeongeun Kim It was defended on April 22, 2005 and approved by David S. Stoffer, Professor Ori Rosen, Professor Wesley Thompson, Professor Anthony Brockwell, Professor Dissertation Director: David S. Stoffer, Professor ii Copyright c by Jeongeun Kim 2005 iii PARAMETER ESTIMATION IN STOCHASTIC VOLATILITY MODELS WITH MISSING DATA USING PARTICLE METHODS AND THE EM ALGORITHM Jeongeun Kim, PhD University of Pittsburgh, 2005 The main concern of financial time series analysis is how to forecast future values of financial variables, based on all available information. One of the special features of financial variables, such as stock prices and exchange rates, is that they show changes in volatility, or variance, over time. Several statistical models have been suggested to explain volatility in data, and among them Stochastic Volatility models or SV models have been commonly and successfully used. Another feature of financial variables I want to consider is the existence of several missing data. For example, there is no stock price data available for regular holidays, such as Christmas, Thanksgiving, and so on. Furthermore, even though the chance is small, stretches of data may not available for many reasons. I believe that if this feature is brought into the model, it will produce more precise results.
    [Show full text]
  • Meta-Analysis Using Individual Participant Data: One-Stage and Two-Stage Approaches, and Why They May Differ Danielle L
    CORE Metadata, citation and similar papers at core.ac.uk Provided by Keele Research Repository Tutorial in Biostatistics Received: 11 March 2016, Accepted: 13 September 2016 Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sim.7141 Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ Danielle L. Burke*† Joie Ensor and Richard D. Riley Meta-analysis using individual participant data (IPD) obtains and synthesises the raw, participant-level data from a set of relevant studies. The IPD approach is becoming an increasingly popular tool as an alternative to traditional aggregate data meta-analysis, especially as it avoids reliance on published results and provides an opportunity to investigate individual-level interactions, such as treatment-effect modifiers. There are two statistical approaches for conducting an IPD meta-analysis: one-stage and two-stage. The one-stage approach analyses the IPD from all studies simultaneously, for example, in a hierarchical regression model with random effects. The two-stage approach derives aggregate data (such as effect estimates) in each study separately and then combines these in a traditional meta-analysis model. There have been numerous comparisons of the one-stage and two-stage approaches via theoretical consideration, simulation and empirical examples, yet there remains confusion regarding when each approach should be adopted, and indeed why they may differ. In this tutorial paper, we outline the key statistical methods for one-stage and two-stage IPD meta-analyses, and provide 10 key reasons why they may produce different summary results. We explain that most differences arise because of different modelling assumptions, rather than the choice of one-stage or two-stage itself.
    [Show full text]
  • Learning to Personalize Medicine from Aggregate Data
    medRxiv preprint doi: https://doi.org/10.1101/2020.07.07.20148205; this version posted July 8, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-ND 4.0 International license . Learning to Personalize Medicine from Aggregate Data Rich Colbaugh Kristin Glass Volv Global Lausanne, Switzerland Abstract There is great interest in personalized medicine, in which treatment is tailored to the individual characteristics of pa- tients. Achieving the objectives of precision healthcare will require clinically-grounded, evidence-based approaches, which in turn demands rigorous, scalable predictive analytics. Standard strategies for deriving prediction models for medicine involve acquiring ‘training’ data for large numbers of patients, labeling each patient according to the out- come of interest, and then using the labeled examples to learn to predict the outcome for new patients. Unfortunate- ly, labeling individuals is time-consuming and expertise-intensive in medical applications and thus represents a ma- jor impediment to practical personalized medicine. We overcome this obstacle with a novel machine learning algo- rithm that enables individual-level prediction models to be induced from aggregate-level labeled data, which is read- ily-available in many health domains. The utility of the proposed learning methodology is demonstrated by: i.) lev- eraging US county-level mental health statistics to create a screening tool which detects individuals suffering from depression based upon their Twitter activity; ii.) designing a decision-support system that exploits aggregate clinical trials data on multiple sclerosis (MS) treatment to predict which therapy would work best for the presenting patient; iii.) employing group-level clinical trials data to induce a model able to find those MS patients likely to be helped by an experimental therapy.
    [Show full text]
  • Good Statistical Practices for Contemporary Meta-Analysis: Examples Based on a Systematic Review on COVID-19 in Pregnancy
    Review Good Statistical Practices for Contemporary Meta-Analysis: Examples Based on a Systematic Review on COVID-19 in Pregnancy Yuxi Zhao and Lifeng Lin * Department of Statistics, Florida State University, Tallahassee, FL 32306, USA; [email protected] * Correspondence: [email protected] Abstract: Systematic reviews and meta-analyses have been increasingly used to pool research find- ings from multiple studies in medical sciences. The reliability of the synthesized evidence depends highly on the methodological quality of a systematic review and meta-analysis. In recent years, several tools have been developed to guide the reporting and evidence appraisal of systematic reviews and meta-analyses, and much statistical effort has been paid to improve their methodological quality. Nevertheless, many contemporary meta-analyses continue to employ conventional statis- tical methods, which may be suboptimal compared with several alternative methods available in the evidence synthesis literature. Based on a recent systematic review on COVID-19 in pregnancy, this article provides an overview of select good practices for performing meta-analyses from sta- tistical perspectives. Specifically, we suggest meta-analysts (1) providing sufficient information of included studies, (2) providing information for reproducibility of meta-analyses, (3) using appro- priate terminologies, (4) double-checking presented results, (5) considering alternative estimators of between-study variance, (6) considering alternative confidence intervals, (7) reporting predic- Citation: Zhao, Y.; Lin, L. Good tion intervals, (8) assessing small-study effects whenever possible, and (9) considering one-stage Statistical Practices for Contemporary methods. We use worked examples to illustrate these good practices. Relevant statistical code Meta-Analysis: Examples Based on a is also provided.
    [Show full text]